giorgio/read.markets
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

ai: route reviewer through OpenRouter + Claude Haiku 4.5

Browse source 
The DeepSeek-V4-flash reviewer was unreliable in production: it pads
its JSON verdicts with internal chain-of-thought even when the prompt
forbids it, so the verdict gets truncated at any reasonable max_tokens
cap and the parser drops it as malformed (a false-negative verdict
that would purge clean rows). A live run on 50 rows reproduced the
failure on 8 of 12 rejections, even at 800 tokens.

Fix: pin the reviewer call to OpenRouter with anthropic/claude-haiku-4.5.
Haiku answers structured-output classification tersely (no scratchpad
preamble), which means a 300-token cap is comfortably above the
~30-token JSON verdict. Cost is roughly the same (~$0.0001-$0.0003 per
review) and the latency tax is smaller.

To enable the pinned-provider call without disrupting other callers,
call_llm grows an optional `provider` parameter: when set, only that
provider is used (no fallback chain). All existing call sites
default to provider=None and keep the chain behaviour.

REVIEWER_MODEL is read from settings via getattr-with-fallback so an
env override can swap models without code changes — useful if we want
to A/B test against e.g. gemini-2.5-flash later.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-05-29 13:21:26 +02:00
parent 8b9d3c9c3e
commit 788563a81f
3 changed files with 45 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

15
tests/test_output_review.py
View file

@ -62,13 +62,20 @@ def _mock_post(handler):
def _configure(monkeypatch):
"""Minimal env so call_llm believes a provider is configured."""
monkeypatch.setattr(ot, "get_settings", lambda: type("S", (), {
"""Minimal env so call_llm believes a provider is configured.
Both review_read (which pins to OpenRouter for a non-thinking model)
and the openrouter module itself read get_settings, so we patch
both module-level references."""
import app.services.output_review as orr
settings = type("S", (), {
"LLM_PROVIDER": "deepseek", "LLM_FALLBACK": "",
"DEEPSEEK_API_KEY": "sk-d", "OPENROUTER_API_KEY": "",
"DEEPSEEK_API_KEY": "sk-d", "OPENROUTER_API_KEY": "sk-or",
"DEEPSEEK_URL": "https://x/deepseek", "DEEPSEEK_MODEL": "deepseek-v4-flash",
"OPENROUTER_URL": "https://x/or", "OPENROUTER_MODEL": "deepseek/deepseek-v4-flash",
})())
"REVIEWER_MODEL": "anthropic/claude-haiku-4.5",
})()
monkeypatch.setattr(ot, "get_settings", lambda: settings)
monkeypatch.setattr(orr, "get_settings", lambda: settings)
@pytest.mark.asyncio