ai: structured-output + reviewer agent for indicator summaries

Replaces the regex-based clean_summary / looks_like_leakage pipeline
that produced the 2026-05-29 valuation-read leak. Two layers of defence
in depth:

1. JSON-mode generation. The per-group and aggregate summary system
   prompts now require the model to emit a single object
   {"read": "..."}; response_format={"type":"json_object"} is passed
   through to the provider so the API enforces well-formed JSON. Prose
   outside the field is physically impossible. The "read" field is the
   only schema slot, so the model has nowhere to spill scratchpad
   into the envelope.

2. Reviewer agent. services/output_review.review_read() makes a second
   small LLM call that judges whether the candidate "read" string is
   publishable. It catches the residual failure mode — scratchpad
   INSIDE the field ("Let's see…", multi-question parentheticals,
   meta-commentary) — and returns a JSON verdict {"clean": bool,
   "reason": str}. Any failure (provider error, parse error, missing
   field) returns clean=false (fail-safe). Cost ~$0.0001/check; latency
   ~1-2 s in the hourly job, no user-facing latency.

The old regex scaffolding (_LEAK_PATTERNS, clean_summary,
looks_like_leakage, _TRAILING_QUOTE) is deleted entirely. It produced
false positives (chopped legitimate "The indicators are…" leaders) and
false negatives (never matched the chain-of-thought patterns the model
actually emits). The reviewer agent is strictly better on both.

On reviewer/parse rejection: don't persist a new IndicatorSummary; the
API's existing fallback to the previous good row continues to serve
the panel. Failures are logged as ind_summary.json_invalid /
ind_summary.reviewer_rejected so we can measure the rejection rate.

Reviewer cost is added to the row's recorded cost_usd so the monthly
budget cap covers the full pipeline.

Adds tests/test_output_review.py: 11 cases covering _extract_read
(JSON envelope handling — invalid JSON, missing field, wrong types,
empty values) and review_read (clean / unclean verdicts plus three
fail-safe paths for malformed reviewer responses).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

app/services/llm_prompts.py

@@ -296,12 +296,25 @@ question via the chat sidebar.
 def build_summary_system_prompt(tone: str, analysis: str) -> str:
     """A lean, focused system prompt for the per-indicator-group hourly
     summary. INTERPRETATION not description — the reader has the table
-    next to this paragraph; they don't need numbers recited at them."""
+    next to this paragraph; they don't need numbers recited at them.
+
+    Output is JSON-mode: the model must emit a single object
+    {"read": "..."}. The wrapper makes scratchpad outside the field
+    physically impossible — the API enforces well-formed JSON, and the
+    only schema slot is the publishable read. Scratchpad inside the
+    field is caught by the reviewer agent (services/output_review)."""
     tone_block = _TONE[_resolve_tone(tone)]
     analysis_block = _ANALYSIS.get(analysis.upper(), _ANALYSIS["SPECULATIVE"])
     return f"""You write a TINY interpretation (≤60 words, 2-3 sentences) \
 of ONE indicator group for a strategic markets dashboard.
 
+# Output format (strict)
+Return ONLY a single JSON object with exactly one field:
+{{"read": "<your 2-3 sentence interpretation>"}}
+Nothing outside that JSON object. No preamble. No markdown fences. \
+No additional fields. The "read" string is what the user sees verbatim, \
+so it must already be the finished, publishable text — never your thinking.
+
 # What this is for
 The reader is looking at the table of numbers right next to your text. \
 They can see the values. They CANNOT see the meaning. Your job is to \
@@ -316,19 +329,20 @@ Even at 2-3 sentences, contrast what the underlying factors justify \
 they don't diverge, say so in one clause. Never just describe the move \
 without placing it on this axis.
 
-# Hard constraints
+# Hard constraints on the "read" string
 - Plain prose, ONE paragraph. No markdown, no headers, no lists, no labels.
 - Open IMMEDIATELY with substance. NEVER start with: "I need to", "I'll", \
 "We need to", "We are asked", "Here's", "Let me", "Let's", "Sure", "Looking \
 at", "Based on", "Summary:", "The data shows", "First", "To address". No \
 meta-commentary at all.
+- No rhetorical questions, no "X? Actually Y?" self-corrections, no \
+parenthetical asides that question your own numbers. The text is the \
+finished read, not the thinking.
 - Cite at most 2-3 specific numbers and ONLY when they anchor an \
 interpretation. Don't list moves; explain them.
 - Multi-week / multi-month horizon. 1-day moves under 2% are noise — skip.
 - No buy/sell language. No predictions. No watch list. No TL;DR. No date \
 header. No "system temperature" line — that belongs to the full daily log.
-- Output the read directly. Do NOT include phrases like "Example", "Good \
-example", "Bad example", "Reference", or any meta-framing of your output.
 
 {tone_block}
 
@@ -350,13 +364,22 @@ def build_summary_user_prompt(group_name: str, quotes: list[dict]) -> str:
 
 def build_aggregate_summary_system_prompt(tone: str, analysis: str) -> str:
     """System prompt for the cross-group aggregate read shown on the dashboard.
-    Wider lens than a per-group summary — synthesise across all groups."""
+    Wider lens than a per-group summary — synthesise across all groups.
+
+    Same JSON-mode contract as build_summary_system_prompt: output is
+    {"read": "..."} only; the field is the publishable text verbatim."""
     tone_block = _TONE[_resolve_tone(tone)]
     analysis_block = _ANALYSIS.get(analysis.upper(), _ANALYSIS["SPECULATIVE"])
     return f"""You write a single SHORT cross-asset INTERPRETATION (≤80 \
 words, 2-4 sentences) for the dashboard header. The reader is glancing — \
 give them the meaning of the whole tape, not a recap.
 
+# Output format (strict)
+Return ONLY a single JSON object with exactly one field:
+{{"read": "<your 2-4 sentence cross-asset interpretation>"}}
+Nothing outside that JSON object. No preamble. No markdown fences. \
+No additional fields. The "read" string is what the user sees verbatim.
+
 # What this is for
 The reader can see every indicator on the dashboard below this paragraph. \
 Your job is NOT to summarise the moves. It is to explain what the moves, \
@@ -371,19 +394,19 @@ crowd is actually doing (irrational: positioning, narrative momentum, \
 flows). At least one of the 2-4 sentences must name this gap or, if the \
 two cohere, explicitly say so.
 
-# Hard constraints
+# Hard constraints on the "read" string
 - Plain prose, ONE paragraph. No markdown, headers, lists, or labels.
 - Open IMMEDIATELY with substance. NEVER start with: "I need to", "I'll", \
 "We need to", "Here's", "Let me", "Looking at", "Based on", "Sure", "Summary:", \
 "The data shows", "Across the board". No meta-commentary.
+- No rhetorical questions, no "X? Actually Y?" self-corrections, no \
+parenthetical asides that question your own numbers.
 - Identify the single most important **cross-asset implication**: e.g. \
 "rates and credit disagree", "equities outrun fundamentals", "geopolitical \
 risk premium is in commodities but not vol". Cite no more than 3 specific \
 numbers, and only as anchors for the interpretation.
 - Multi-week / multi-month horizon. 1-day moves under 2% are noise.
 - No buy/sell language. No predictions of specific levels.
-- Output the read directly. Do NOT include phrases like "Example", "Good \
-example", "Bad example", "Reference", or any meta-framing of your output.
 
 {tone_block}