giorgio/read.markets
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

review: strip markdown code-fences from JSON verdicts

Browse source 
Haiku 4.5 occasionally wraps its JSON response in a markdown code
fence even with response_format={"type":"json_object"} enforced:

    ```json
    {"clean": true, "reason": "polished read"}
    ```

Live testing the new reviewer caught this — every verdict was being
dropped as "reviewer returned non-JSON". Strip a single leading
trailing fence before json.loads. Defensive for any model that does
the same (Claude variants commonly fence JSON even when told not to).

Adds a unit test covering fenced output.
This commit is contained in:
Giorgio Gilestro 2026-05-29 13:27:37 +02:00
parent 788563a81f
commit 385c5fdc60
2 changed files with 34 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

16
app/services/output_review.py
View file

@ -114,8 +114,22 @@ async def review_read(client: httpx.AsyncClient, candidate: str) -> Verdict:
return Verdict(clean=False, reason=f"reviewer error: {str(e)[:80]}",
cost_usd=None)
# Haiku (and several other models) occasionally wrap their JSON
# output in a markdown code fence even with response_format set —
# ```json\n{...}\n``` — so strip a single leading/trailing fence
# before parsing. We do this defensively for any model; it's a
# no-op for callers that already emit bare JSON.
raw = result.content.strip()
if raw.startswith("```"):
first_nl = raw.find("\n")
if first_nl != -1:
raw = raw[first_nl + 1:]
if raw.rstrip().endswith("```"):
raw = raw.rstrip()[:-3].rstrip()
raw = raw.strip()
try:
parsed = json.loads(result.content)
parsed = json.loads(raw)
except json.JSONDecodeError:
log.warning("review.parse_failed", preview=result.content[:200])
return Verdict(clean=False, reason="reviewer returned non-JSON",