review: gate strategic-log, portfolio, chat, and digest on reviewer
Extends the reviewer agent — previously only protecting indicator
summaries — to every AI-generated surface that reaches a user. The
reviewer's prompt already rejects scratchpad, truncation,
meta-commentary, and (since a6e476b) financial advice; wiring it in
turns those rules from prompt-level "asks" into structural gates.
Four call sites updated:
- ai_log_job.run() : after each tone/analysis variant is generated,
pass through review_read. On reject, log the reason and skip the
StrategicLog insert; the API's existing "latest StrategicLog" lookup
falls back to the previous clean log.
- services/portfolio_analysis.analyse() : on reject, raise a clean
RuntimeError that the /api/analyze router already maps to HTTP 502
with a retry-able message. Portfolio analysis isn't cached server-
side, so the user retries; the reviewer's verdict reason goes into
the AICall ledger as the leaked-status row's error column.
- routers/chat.chat() : on reject, instead of returning the raw
assistant content we return a short refusal explaining the limit
and inviting a rephrase. Adds ~1-2 s of latency per turn (one extra
LLM call to Haiku) — the only user-facing latency tax.
- jobs/email_digest_job._generate_variants() : on reject, the variant
is dropped for the cycle. Recipients on the rejected tone get no
digest email this run, which is better than delivering inbox copy
that drifts into advice (emails are unrecallable once sent).
In every case the AICall ledger row records the reviewer cost so
month_spend stays accurate across all paths.
The reviewer system prompt is slightly generalised to cover both the
indicator-summary case and the longer-form log/digest/chat case:
- removes "short interpretive read" framing
- softens the "any question" rule so genuine rhetorical structure in
a long-form log doesn't trigger a reject
tests/conftest.py grows an autouse fixture that stubs review_read to
clean=True in every consumer module. Tests that mock the generator
shouldn't have to also mock the safety gate behind it; tests that
specifically want the reject branch can override with their own
monkeypatch. test_output_review.py is unaffected — it imports
review_read directly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
parent
a6e476b851
commit
f9534f7ad6
6 changed files with 161 additions and 19 deletions
|
|
@ -22,6 +22,38 @@ os.environ.setdefault("CASSANDRA_MOCK", "1")
|
|||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def stub_reviewer(monkeypatch):
|
||||
"""Replace review_read with a clean-passing stub in every consumer
|
||||
module. Tests that mock the generator's call_llm shouldn't also
|
||||
have to mock the reviewer that runs after it — the reviewer is a
|
||||
safety gate, not behaviour under test.
|
||||
|
||||
Tests in test_output_review.py exercise review_read through its
|
||||
own module and are unaffected. Tests that want to assert the
|
||||
reviewer-rejected branch can override with their own
|
||||
monkeypatch.setattr — later wins.
|
||||
"""
|
||||
from app.services.output_review import Verdict
|
||||
|
||||
async def _clean(_client, _candidate):
|
||||
return Verdict(clean=True, reason="stubbed-by-conftest", cost_usd=0.0)
|
||||
|
||||
for mod_path in (
|
||||
"app.services.portfolio_analysis",
|
||||
"app.routers.chat",
|
||||
"app.jobs.ai_log_job",
|
||||
"app.jobs.email_digest_job",
|
||||
"app.jobs.indicator_summary_job",
|
||||
):
|
||||
try:
|
||||
mod = __import__(mod_path, fromlist=["review_read"])
|
||||
except ImportError:
|
||||
continue
|
||||
if hasattr(mod, "review_read"):
|
||||
monkeypatch.setattr(mod, "review_read", _clean)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def db_factory(tmp_path):
|
||||
"""Per-test sqlite engine + async session factory.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue