giorgio/read.markets
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

ai: route reviewer through OpenRouter + Claude Haiku 4.5

Browse source 
The DeepSeek-V4-flash reviewer was unreliable in production: it pads
its JSON verdicts with internal chain-of-thought even when the prompt
forbids it, so the verdict gets truncated at any reasonable max_tokens
cap and the parser drops it as malformed (a false-negative verdict
that would purge clean rows). A live run on 50 rows reproduced the
failure on 8 of 12 rejections, even at 800 tokens.

Fix: pin the reviewer call to OpenRouter with anthropic/claude-haiku-4.5.
Haiku answers structured-output classification tersely (no scratchpad
preamble), which means a 300-token cap is comfortably above the
~30-token JSON verdict. Cost is roughly the same (~$0.0001-$0.0003 per
review) and the latency tax is smaller.

To enable the pinned-provider call without disrupting other callers,
call_llm grows an optional `provider` parameter: when set, only that
provider is used (no fallback chain). All existing call sites
default to provider=None and keep the chain behaviour.

REVIEWER_MODEL is read from settings via getattr-with-fallback so an
env override can swap models without code changes — useful if we want
to A/B test against e.g. gemini-2.5-flash later.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-05-29 13:21:26 +02:00
parent 8b9d3c9c3e
commit 788563a81f
3 changed files with 45 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

13
app/services/openrouter.py
View file

@ -199,6 +199,7 @@ async def call_llm(
model: str | None = None,
max_tokens: int = 4000,
response_format: dict | None = None,
provider: str | None = None,
) -> LogResult:
"""Provider-aware chat completion with fallback. Tries primary
(LLM_PROVIDER) first; if it raises after retries, falls through to
@ -211,8 +212,16 @@ async def call_llm(
Pass response_format={"type": "json_object"} to force JSON-mode
output (the model still needs to be instructed in the system prompt
to emit valid JSON — this flag enforces, not asks)."""
chain = _provider_chain()
to emit valid JSON — this flag enforces, not asks).
Pass `provider` (e.g. "openrouter") to skip the configured chain
and pin the call to a specific provider. Used by the reviewer agent
to force routing through OpenRouter so it can address a non-DeepSeek
model that doesn't pre-think before emitting JSON."""
if provider is not None:
chain = [provider]
else:
chain = _provider_chain()
if not chain:
raise RuntimeError("No LLM provider configured (no API key set)")