Two changes to the LLM call path that together close the
chain-of-thought leakage surface:
1. _call_provider accepts an optional `response_format` (forwarded to
the OpenAI-shaped API — DeepSeek and OpenRouter both honour
{"type": "json_object"}). Threaded through call_llm so callers can
force structured output without monkey-patching the body. The
indicator-summary job will use this next: it'll require the model
to emit {"read": "..."} and parse the field, making prose outside
the JSON object physically impossible to publish.
2. Empty `content` no longer falls back to the `reasoning` field.
`reasoning` is the model's internal scratchpad — "Let's see...",
half-formed math, planning notes. We had a fallback that surfaced
it when content was null, but the field is intended for debugging
the model, not for publication. After the 2026-05-29 valuation
read leaked into production, the fallback is gone: an empty
content row now raises so the caller retries or skips, and the
previous good row remains visible. Test updated to assert this
safer behaviour.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
DeepSeek's native API returns prompt_tokens/completion_tokens but not
`usage.cost`. OpenRouter returns both. Result: with DeepSeek-direct as
primary (current default), every LogResult.cost_usd was None — and
every downstream cost ledger row (AICall, StrategicLog,
IndicatorSummary, translation tables) stored None instead of the real
spend.
Added a per-model rate table and fallback computation in _call_provider:
when the upstream omits cost, multiply tokens by the table rates. If the
upstream DOES return cost, keep it (authoritative). Falls back to None
if both the upstream and the table miss.
deepseek-v4-flash rates: \$0.07/M input, \$0.28/M output (per DeepSeek).
openrouter.py was 790 lines mixing two orthogonal concerns:
- Prompt engineering (build_system_prompt, build_summary_*,
build_chat_*, build_daily_digest_*, etc.) — ~400 lines, changes
weekly as PROMPT_VERSION bumps
- LLM transport (call_llm, _provider_chain, _call_provider, retry
+ fallback machinery) — ~250 lines, rarely changes
Extracted the prompt-engineering surface to app/services/llm_prompts.py.
Transport stays in openrouter.py (consistent with the filename — the
OpenRouter URL is the transport's anchor).
All import sites (jobs, routers, services, tests) split their
multi-import lines into two: prompt-things from llm_prompts, transport
from openrouter. PROMPT_VERSION constant, _TONE_ALIASES, _resolve_tone,
and SYSTEM_PROMPT moved with the prompt functions.
No behaviour change — pure relocation. Function signatures, body, and
naming all preserved.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- indicator_summary_job.py imported its own copies of _month_spend and
_latest_quotes_by_group; _market_context.py already exposes these.
Switched to the canonical imports. Also fixed _market_context's
latest_quotes_by_group to actually filter null prices (it claimed to
in its docstring but lacked the WHERE clause).
- api.py duplicated REFERENCE_LINE as CHAT_REFERENCE_LINE — same string,
two sources of truth. Now imports REFERENCE_LINE.
- Chat endpoint used the deprecated `call_openrouter` alias and passed
an explicit `model=` that bypassed the provider chain. Switched to
`call_llm` with default model selection, then removed the alias.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the unauthenticated surface that's needed to invite outsiders:
- Landing (/) — dual-purpose root: dashboard for logged-in users,
landing for everyone else. New maybe_current_user soft-auth helper
in app/auth.py supports it without disturbing the per-route
require_token deps on /news, /log, /upload, /settings.
- About, Pricing, Disclaimer, Terms, Privacy — own router
(app/routers/public.py), no auth dep, shared public_base layout
(brand link, thin nav, footer with legal links + ICO ref + date).
- Editorial positioning: news aggregator with a macro brain; tagline
"Understand markets. Don't gamble on them."; anti-trading-as-gambling
stance carried through About and Landing.
Legal pass following an independent lawyer-style review:
- Privacy: explicit UK-GDPR Art. 6 lawful-basis section; Art. 22
automated-decision line; explicit consent for sessionStorage sync
key (PECR); 30-day IP-log retention; Art. 21 objection right;
Children clause; Art. 33/34 breach-notification clause;
international-transfer mechanism (IDTA + UK Addendum). ICO
registration ZC098928 surfaced at the top.
- Pricing: paid-card AI-portfolio-analysis bullet rewritten to remove
advice-shaped wording ("what would invalidate the posture" gone);
added italic carve-out citing FSMA / FCA COBS.
- Disclaimer: separate EU/EEA carve-out + MAR 596/2014 Art. 3(1)(34)
commentator safe-harbour; "qualifies the Terms" line; hallucination
wording fixed.
- Terms: cl.4 explicit AI-training prohibition + harassment line;
cl.5 CCR 2013 14-day cancellation; cl.7 softened AI copyright
claim under CDPA s.9(3) ambiguity; cl.8 proportionate suspension +
pro-rata refund for paid users; cl.10 CRA 2015 Pt 1 statutory-rights
carve-out from the liability cap; cl.11 right to close account on
material change; cl.12 non-exclusive jurisdiction + UK consumer
local courts.
Code-side enforcement of the Privacy claim:
- openrouter.py: outbound OpenRouter calls now carry
X-OR-Allow-Training: false. DeepSeek doesn't expose a per-request
flag; the Privacy page discloses this caveat verbatim.
Apex domain prep:
- branding.APP_URL flipped to https://read.markets (was app.). DNS for
the apex already resolves; pending operator NPM step is a cert that
covers the bare apex + a 301 from app.read.markets. No hard-coded
subdomain references remain in code (verified with grep).
Nav + chrome:
- app dropdown gains Pricing / Terms / Privacy / Disclaimer links.
- login.html gains a small legal-links footer for the
highest-leverage moment to surface them.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- dashboard grid: explicit "header" area as the first row so the
aggregated read panel renders at the top instead of being
auto-placed after the named areas.
- indicators: hide rows flagged stale (older than the group's
freshness threshold). Server still computes stale_symbols;
rendering can be re-enabled by removing the
`{% if not is_stale %}` wrapper in indicators.html.
- /log: add tone-changed to #log-content's hx-trigger and include
it in cassandraSetTone's selector list — toggling Novice /
Intermediate on the Log page was previously a no-op.
- prompts: bump PROMPT_VERSION 7→8. Strengthen the rational-vs-
irrational framing in the strategic-log system prompt from
aspirational to mandatory ("a paragraph without both lenses must
be rewritten"). Require the same lens in the per-group summary,
cross-asset aggregate, and portfolio commentary overrides.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The product is now "Read the Markets" served at https://read.markets,
with the app at https://app.read.markets. "Cassandra" survives only as
the in-product AI persona (system prompt + "Ask Cassandra" chat label).
Centralised the brand in app/branding.py: BRAND_NAME, BRAND_SHORT,
DOMAIN, SITE_URL, APP_URL, EMAIL_FROM_DEFAULT. Jinja templates pull
{{ BRAND_NAME }} via globals registered in templates_env.py; Python
code reads branding.BRAND_NAME directly. The future-rename surface
is now a one-liner.
Updated: FastAPI app title, every page title (dashboard, news, log,
settings, upload, login, verify), header brand div, auth-card brands,
OTP email subject + HTML + plain-text bodies (incl. uppercase header
tag), OpenRouter X-Title + HTTP-Referer attribution headers, README.
Email tests now assert against branding.BRAND_NAME rather than the
literal string.
Internal identifiers deliberately kept on the legacy "cassandra" name
to avoid invalidating live sessions / advisory locks / configs:
cookies (cassandra_session, cassandra_pending) + itsdangerous salts,
MariaDB GET_LOCK keys, CASSANDRA_TOKEN env var, cassandra.css filename,
pyproject package name, localStorage prefs, outbound User-Agent strings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Move news_job from hourly to 3x/hour (cron 10,30,50), with a CadencePolicy
gate that throttles to active hours (07-21 UTC weekdays at 20 min), off-hours
(3 h), weekends (6 h). Keeps the daytime feed fresh without spamming RSS
sources overnight.
- Tag each headline on ingestion via DeepSeek (BATCH_SIZE=25, max_tokens=4000,
json.JSONDecoder().raw_decode + per-row regex recovery for resilient parsing).
Vocabulary: 16 tags including new EU / USA / AI / Conflict. NULL tags are
picked up automatically on the next news_job run, so back-tagging is implicit
rather than a separate migration step.
- Tag UI: pill bar above the feed with off → include → exclude cycle on click;
shift-click jumps straight to exclude. State persists in localStorage and is
injected into /api/news requests via htmx:configRequest. Per-row chips sit to
the right of the headline (new 5-column grid: age | source | title | tags |
UTC) so vertical density stays high.
- Strategic log header bug: model was hallucinating "(Updated 21:30 UTC)" in
future tense. Bumped PROMPT_VERSION 6→7, added explicit ban on time-of-day
clauses, and supply the actual current UTC time in the user prompt so the
model has no need to invent one.
Migration 0012 adds headlines.tags (JSON, nullable). Tests cover vocabulary
integrity, validation/normalisation, and the JSON-recovery parser (17 tests).
Server no longer holds portfolios. Holdings live in the browser
(localStorage); the server publishes an anonymous ticker_universe and a
gzipped /api/universe payload identical for every authenticated user, so
access patterns can't betray which tickers a user holds. AI commentary
is generated ephemerally from the browser-supplied pie and the cost
ledger row records no positions. Migrations 0009-0011 added the
universe table and dropped positions / portfolio_snapshots /
portfolios.
Authentication is now e-mail OTP only. Migration 0010 dropped
password_hash and email_verified (every active session is by
construction proof of email control). The /signup endpoint is gone;
signup and login share a single email-entry page. Email rendering is
HTML+plain-text multipart with a shared brand palette (app/branding.py)
asserted in sync with the CSS by a drift-detection test.
LLM provider defaults to DeepSeek-direct (cheaper, api.deepseek.com)
with OpenRouter as automatic fallback if DeepSeek fails. ai_log_job and
indicator_summary_job now iterate the two tones (NOVICE, INTERMEDIATE)
per cycle so the dashboard's tone toggle is instant; PROMPT_VERSION
bumped to 6 with an educational anti-TA / anti-gambling stance baked
into _CORE. NOVICE mode renders a curated glossary inline (CBOE VIX,
yield curve, HY OAS, etc.) with JS-positioned tooltips that survive
viewport edges and sticky bars. Model name and tokens hidden from the
user UI; still recorded in StrategicLog.model and AICall for admin.
Layout adds a sticky top nav, a sticky bottom markets bar (one chip per
exchange with status LED + headline index + 1d change), and
Phase H feedback reporting is queued in tasks/todo.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two changes that together cut OpenRouter spend ~50% and give the daily
log temporal awareness.
1. CadencePolicy (app/services/cadence.py): expensive AI jobs only
fire hourly during the EU/US active window (Mon-Fri 07-21 UTC).
Off-hours weekdays throttle to every 4h; weekends to every 12h.
ai_log_job and indicator_summary_job both consult the policy before
doing real work; market/news/portfolio ingest jobs stay hourly
(cheap, no API cost). Skipped runs land in job_runs with status
'skipped' and the throttle reason in error.
2. Update mode for ai_log_job: when an earlier log exists for the
current UTC day, it's passed to the model as 'Earlier log from
today (generated HH:MM UTC)'. The system prompt grows an Update
mode section instructing the model to revise — not restart — and
anchor on what has CHANGED since the earlier draft. The TL;DR
leads with intra-day change when meaningful, the watch list evolves
rather than restarts. PROMPT_VERSION bumped to 5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DeepSeek occasionally regurgitates the system prompt verbatim
("Constraints: ≤60 words...", "Example good: ..."). Three-pronged fix:
1. Removed the inline good/bad example blocks from the per-group and
aggregate system prompts — DeepSeek was treating them as templates
to copy. The hard constraints alone are clear enough.
2. Expanded the LEAK_PATTERNS list to catch the prompt-label echoes
that still occasionally slip through ("Key observations:", "The
indicators are:", "Must cite ...", "Should give ...", bare "Key:").
Cleanup now runs up to 6 passes for compound leakage.
3. Added looks_like_leakage() — if the cleaned output still contains
tell-tale phrases ("≤60 words", "instructions:", etc.), the summary
is skipped rather than persisted. Logs a 'leakage_detected' warning
and an ai_calls row with status=leaked so we can see the failure
rate over time. The previous good summary stays visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three new data sources hooked into the existing SOURCES registry. All
open APIs, no keys:
- EUROSTAT: prefix EUROSTAT:dataset?dim=val&... — current EU bond
yields (Bund/OAT/BTP/EZ) and Eurozone economic indicators that
FRED's OECD-mirror series stopped updating in 2022-2023.
- ONS: prefix ONS:topic/cdid/dataset — current UK CPI, unemployment,
GDP, industrial production. Replaces the 5+ month-stale FRED
LRHUTTTTGBM156S mirror.
New indicator groups in default.toml feed the strategic/fundamental
lens we converged on: valuation (CAPE/Buffett anchors), bubble_watch
(SKEW/VVIX/RSP vs SPY/HYG vs TLT/IPO/crypto), economy (multi-region,
ALL current-or-stale-flagged), bonds (UK/EU/US/JPN sovereign yields).
Indicator panel now opens with an AI "read" interpretation per group
(generated hourly at :07 UTC alongside an aggregate cross-group read
shown in the dashboard header). The aggregate is grounded by a markets
strip — NYSE/LSE/Frankfurt/Tokyo/HK/Shanghai with open/closed LEDs and
next-open countdown, computed locally from each exchange's tz.
Other UX bits: indicator-row tooltips populated from TOML notes;
rows whose last observation is >90 days old get a 'stale' chip;
ghost symbols (in DB but no longer in TOML) filtered out of the
panel; Eurostat/ONS symbols display as short codes rather than the
full API path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Containerised macro-strategy dashboard: 4-panel web UI (indicators,
portfolio, flash news, AI strategic log), MariaDB store, hourly
ingestion jobs, OpenRouter-backed AI analysis.
Ports the four prototype scripts in the parent dir (market_pulse,
flash_news, trading212, strategic_log) into async services backed by a
persistent DB and served via FastAPI + Jinja2 + HTMX. APScheduler runs
as a separate compose service for crash-safety and easier restarts.
Portfolio composition + position names come live from Trading 212;
news per-ticker headlines reuse those names. Tone (NOVICE/INTERMEDIATE/
PRO) and analysis style (DRY/SPECULATIVE) are env-configurable and
stored on each log row so historical entries show what produced them.
Default model is deepseek/deepseek-v4-flash (overridable via env).
Light/dark theme toggle, sans-serif for prose surfaces, monospace for
data. Bearer-token auth, OpenRouter monthly cost cap, RSS feeds auto-
disabled on consecutive failures.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
