Extends the reviewer agent — previously only protecting indicator
summaries — to every AI-generated surface that reaches a user. The
reviewer's prompt already rejects scratchpad, truncation,
meta-commentary, and (since a6e476b) financial advice; wiring it in
turns those rules from prompt-level "asks" into structural gates.
Four call sites updated:
- ai_log_job.run() : after each tone/analysis variant is generated,
pass through review_read. On reject, log the reason and skip the
StrategicLog insert; the API's existing "latest StrategicLog" lookup
falls back to the previous clean log.
- services/portfolio_analysis.analyse() : on reject, raise a clean
RuntimeError that the /api/analyze router already maps to HTTP 502
with a retry-able message. Portfolio analysis isn't cached server-
side, so the user retries; the reviewer's verdict reason goes into
the AICall ledger as the leaked-status row's error column.
- routers/chat.chat() : on reject, instead of returning the raw
assistant content we return a short refusal explaining the limit
and inviting a rephrase. Adds ~1-2 s of latency per turn (one extra
LLM call to Haiku) — the only user-facing latency tax.
- jobs/email_digest_job._generate_variants() : on reject, the variant
is dropped for the cycle. Recipients on the rejected tone get no
digest email this run, which is better than delivering inbox copy
that drifts into advice (emails are unrecallable once sent).
In every case the AICall ledger row records the reviewer cost so
month_spend stays accurate across all paths.
The reviewer system prompt is slightly generalised to cover both the
indicator-summary case and the longer-form log/digest/chat case:
- removes "short interpretive read" framing
- softens the "any question" rule so genuine rhetorical structure in
a long-form log doesn't trigger a reject
tests/conftest.py grows an autouse fixture that stubs review_read to
clean=True in every consumer module. Tests that mock the generator
shouldn't have to also mock the safety gate behind it; tests that
specifically want the reject branch can override with their own
monkeypatch. test_output_review.py is unaffected — it imports
review_read directly.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a new UNCLEAN criterion to the reviewer agent's system prompt:
direct recommendation language (buy/sell/hold/accumulate/trim/rotate),
allocation guidance (overweight/underweight, "X% in bonds"), price
targets, and personalised framing ("you should", "investors should")
all trigger a reject.
The operator is not licensed to give investment advice; this is
editorial commentary on public data. The generator's system prompt
already forbids buy/sell language, but a prompt-only constraint is
not an enforcement layer. The reviewer agent — already in the
pipeline for chain-of-thought / truncation / meta-commentary — is the
right place to enforce the regulatory boundary structurally: rows
that drift into advice get dropped, and the API falls back to the
previous compliant row.
Descriptive / interpretive language about market state remains
explicitly allowed ("valuations are stretched", "real yields are
restrictive"). The criterion is state vs action: states publish,
actions don't.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Haiku 4.5 occasionally wraps its JSON response in a markdown code
fence even with response_format={"type":"json_object"} enforced:
```json
{"clean": true, "reason": "polished read"}
```
Live testing the new reviewer caught this — every verdict was being
dropped as "reviewer returned non-JSON". Strip a single leading
trailing fence before json.loads. Defensive for any model that does
the same (Claude variants commonly fence JSON even when told not to).
Adds a unit test covering fenced output.
The DeepSeek-V4-flash reviewer was unreliable in production: it pads
its JSON verdicts with internal chain-of-thought even when the prompt
forbids it, so the verdict gets truncated at any reasonable max_tokens
cap and the parser drops it as malformed (a false-negative verdict
that would purge clean rows). A live run on 50 rows reproduced the
failure on 8 of 12 rejections, even at 800 tokens.
Fix: pin the reviewer call to OpenRouter with anthropic/claude-haiku-4.5.
Haiku answers structured-output classification tersely (no scratchpad
preamble), which means a 300-token cap is comfortably above the
~30-token JSON verdict. Cost is roughly the same (~$0.0001-$0.0003 per
review) and the latency tax is smaller.
To enable the pinned-provider call without disrupting other callers,
call_llm grows an optional `provider` parameter: when set, only that
provider is used (no fallback chain). All existing call sites
default to provider=None and keep the chain behaviour.
REVIEWER_MODEL is read from settings via getattr-with-fallback so an
env override can swap models without code changes — useful if we want
to A/B test against e.g. gemini-2.5-flash later.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Live re-check on 50 recent IndicatorSummary rows after the previous
120 → 300 bump still produced 4 'reviewer returned non-JSON' verdicts
out of 12 rejections. DeepSeek-V4-flash sometimes prefixes its JSON
output with a short stretch of thinking even though response_format
is enforced, which truncates the JSON at the back end of the 300-token
cap.
800 tokens is comfortably above any realistic verdict + preamble at
~$0.00022/call (DeepSeek output rates). Negligible cost given the
hourly call volume.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A live sanity-check on 50 recent IndicatorSummary rows found 6 of 10
reviewer rejections were the reviewer hitting its own max_tokens cap
mid-verdict ('{"clean": false, "reason": "Truncated sent…'). The
parser then dropped the candidate as malformed JSON, producing a
false-negative verdict that would have purged legitimately clean
rows.
300 tokens is well above the ~30-token verdict the prompt asks for;
the extra headroom removes the artefact at ~$0.00015 per call.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Replaces the regex-based clean_summary / looks_like_leakage pipeline
that produced the 2026-05-29 valuation-read leak. Two layers of defence
in depth:
1. JSON-mode generation. The per-group and aggregate summary system
prompts now require the model to emit a single object
{"read": "..."}; response_format={"type":"json_object"} is passed
through to the provider so the API enforces well-formed JSON. Prose
outside the field is physically impossible. The "read" field is the
only schema slot, so the model has nowhere to spill scratchpad
into the envelope.
2. Reviewer agent. services/output_review.review_read() makes a second
small LLM call that judges whether the candidate "read" string is
publishable. It catches the residual failure mode — scratchpad
INSIDE the field ("Let's see…", multi-question parentheticals,
meta-commentary) — and returns a JSON verdict {"clean": bool,
"reason": str}. Any failure (provider error, parse error, missing
field) returns clean=false (fail-safe). Cost ~$0.0001/check; latency
~1-2 s in the hourly job, no user-facing latency.
The old regex scaffolding (_LEAK_PATTERNS, clean_summary,
looks_like_leakage, _TRAILING_QUOTE) is deleted entirely. It produced
false positives (chopped legitimate "The indicators are…" leaders) and
false negatives (never matched the chain-of-thought patterns the model
actually emits). The reviewer agent is strictly better on both.
On reviewer/parse rejection: don't persist a new IndicatorSummary; the
API's existing fallback to the previous good row continues to serve
the panel. Failures are logged as ind_summary.json_invalid /
ind_summary.reviewer_rejected so we can measure the rejection rate.
Reviewer cost is added to the row's recorded cost_usd so the monthly
budget cap covers the full pipeline.
Adds tests/test_output_review.py: 11 cases covering _extract_read
(JSON envelope handling — invalid JSON, missing field, wrong types,
empty values) and review_read (clean / unclean verdicts plus three
fail-safe paths for malformed reviewer responses).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two changes to the LLM call path that together close the
chain-of-thought leakage surface:
1. _call_provider accepts an optional `response_format` (forwarded to
the OpenAI-shaped API — DeepSeek and OpenRouter both honour
{"type": "json_object"}). Threaded through call_llm so callers can
force structured output without monkey-patching the body. The
indicator-summary job will use this next: it'll require the model
to emit {"read": "..."} and parse the field, making prose outside
the JSON object physically impossible to publish.
2. Empty `content` no longer falls back to the `reasoning` field.
`reasoning` is the model's internal scratchpad — "Let's see...",
half-formed math, planning notes. We had a fallback that surfaced
it when content was null, but the field is intended for debugging
the model, not for publication. After the 2026-05-29 valuation
read leaked into production, the fallback is gone: an empty
content row now raises so the caller retries or skips, and the
previous good row remains visible. Test updated to assert this
safer behaviour.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three connected fixes after the user spotted the 2026-05-28 IT log
cutting off mid-sentence:
1. translation: bump max_tokens 4000 → 8000.
call_llm()'s default cap was 4000, which is what the English log
generator itself uses as its ceiling. Italian expands roughly 15-25 %
over English in tokens, so any near-cap English source produced an
IT translation that hit finish_reason=length and returned a
truncated body — silently, because _call_provider() only raises when
content is fully empty. The strategic_log_translations table has
dozens of rows where completion_tokens landed at exactly 4000 with
content well under half the source length. 8000 gives ample
headroom for any of the five LANGUAGES we ship (en/it/es/fr/de).
2. log.html: localise the chat sidebar strings.
user_lang was already passed into the template by pages.py, so an
inline {% if user_lang == 'it' %} keeps it simple. Covers the
"Ask Cassandra" title, the "grounded on…" hint, the helper lede,
the textarea placeholder, and the Send button label.
3. chat endpoint: append respond_in_clause(user.lang) to the system
prompt. The chat conversation can now happen in IT — the model's
first reply lands in the right language even when the user's first
turn is short.
scripts/backfill_truncated_translations.py: one-off cleanup utility.
Scans strategic_log_translations for rows whose translated content is
< 70 % of the English source (the truncation signal — IT *expands*
beyond English, so a shorter translation is always suspect), deletes
them, and re-translates via the now-uncapped service. Supports --date,
--since, --all and --dry-run. The 2026-05-28 fan-out has already been
re-translated (13/13 rows). Other historical dates still hold older
truncations; the user can decide whether to backfill those (the script
is idempotent).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
DeepSeek's native API returns prompt_tokens/completion_tokens but not
`usage.cost`. OpenRouter returns both. Result: with DeepSeek-direct as
primary (current default), every LogResult.cost_usd was None — and
every downstream cost ledger row (AICall, StrategicLog,
IndicatorSummary, translation tables) stored None instead of the real
spend.
Added a per-model rate table and fallback computation in _call_provider:
when the upstream omits cost, multiply tokens by the table rates. If the
upstream DOES return cost, keep it (authoritative). Falls back to None
if both the upstream and the table miss.
deepseek-v4-flash rates: \$0.07/M input, \$0.28/M output (per DeepSeek).
Splits the 2571-line cassandra.css into ten focused stylesheets:
tokens (palette + fonts), layout (chrome), panels, dashboard,
portfolio, log-chat, auth, settings, news, public. base.html and
public_base.html load only what they need; auth pages (login,
verify, unsubscribe confirm) load tokens + layout + auth.
Brand drift-detection test repointed at tokens.css (where the
palette now lives). 291 tests still pass.
email_service.py was 428 lines covering three different concerns:
SMTP transport, OTP/welcome rendering (tightly coupled — same brand
template + theme), and digest rendering (a totally different shape
of email, different layout, different copy cadence). The two halves
changed at different cadences and made the file noisy to navigate.
Extracted render_digest_email + _DIGEST_HTML_TEMPLATE +
_strip_html_to_text to app/services/digest_email.py. SMTP transport
and the OTP/welcome surface stay in email_service.py.
Import sites updated: email_digest_job and test_email_render now
import render_digest_email from digest_email. The OTP/welcome
import sites (auth router, branding tests, test_email_service) are
untouched.
No behaviour change — pure relocation. Templates byte-identical.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
openrouter.py was 790 lines mixing two orthogonal concerns:
- Prompt engineering (build_system_prompt, build_summary_*,
build_chat_*, build_daily_digest_*, etc.) — ~400 lines, changes
weekly as PROMPT_VERSION bumps
- LLM transport (call_llm, _provider_chain, _call_provider, retry
+ fallback machinery) — ~250 lines, rarely changes
Extracted the prompt-engineering surface to app/services/llm_prompts.py.
Transport stays in openrouter.py (consistent with the filename — the
OpenRouter URL is the transport's anchor).
All import sites (jobs, routers, services, tests) split their
multi-import lines into two: prompt-things from llm_prompts, transport
from openrouter. PROMPT_VERSION constant, _TONE_ALIASES, _resolve_tone,
and SYSTEM_PROMPT moved with the prompt functions.
No behaviour change — pure relocation. Function signatures, body, and
naming all preserved.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three recently-added tables (strategic_log_translations,
indicator_summary_translations, csv_format_templates) drifted from
the codebase's existing naming convention:
- llm_model -> model
- llm_cost_usd -> cost_usd
- content_md -> content (on the two translation tables; csv_format
doesn't have a content field)
Also added prompt_tokens and completion_tokens to the three tables;
they were silently dropped at write time despite LogResult exposing
them.
All writer call sites (ai_log_job, indicator_summary_job,
llm_csv_parser) and reader call sites (api.py localized helpers)
updated to match. Tests realigned.
Migration 0025 uses batch_alter_table for SQLite compatibility.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- indicator_summary_job.py imported its own copies of _month_spend and
_latest_quotes_by_group; _market_context.py already exposes these.
Switched to the canonical imports. Also fixed _market_context's
latest_quotes_by_group to actually filter null prices (it claimed to
in its docstring but lacked the WHERE clause).
- api.py duplicated REFERENCE_LINE as CHAT_REFERENCE_LINE — same string,
two sources of truth. Now imports REFERENCE_LINE.
- Chat endpoint used the deprecated `call_openrouter` alias and passed
an explicit `model=` that bypassed the provider chain. Switched to
`call_llm` with default model selection, then removed the alias.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stale comments referencing completed migrations:
- universe.py "remain live until step 10 of Phase G" — endpoints gone
- api.py "Portfolio endpoints moved to universe.py" — empty block
- csv_import.py "persist_pie removed in Phase G" — historical context
Dead Settings fields (all confirmed unreferenced by app code):
- CASSANDRA_PORT — port is hardcoded in docker-compose / uvicorn cmd
- POLAR_API_KEY — Polar was replaced by Stripe
- CASSANDRA_MOCK — env var still set by tests as a sentinel; the
Settings field itself was never read
- CASSANDRA_BASE_CURRENCY — "GBP" hardcoded inline elsewhere
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The route's resolve-slice loop is T212-specific — it looks tickers up
against the InstrumentMap, which only has T212's universe. For the LLM
path the ticker is already Yahoo-ready (e.g. VOD.L, ASML.AS), so
sending it through resolve_slice produced spurious "could not be
resolved" warnings and dropped the positions.
Fix: ParsedPie gains a ``tickers_resolved`` flag (default False for
T212 backward-compat); _apply_mapping in the LLM path sets it True
and also extracts currency from the LLM-mapped currency_col into a
new ``ParsedPosition.currency`` field. The route branches on the flag:
LLM-path positions are kept verbatim with a best-effort InstrumentMap
lookup for nicer name/currency overrides, never dropped.
Integration test tightened to assert all 5 IBKR fixture positions
round-trip with the right currencies (USD / GBP / EUR).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Heuristic refined from the plan draft: candidate header rows must be
followed by a row containing at least one numeric token. Without this,
IBKR-style multi-line preambles (all-text rows before the real header)
would be mistaken for the header at preamble=0.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The "Phase D.1/D.2/D.3" comment scaffolding and the "Paddle webhook
will fill this in" references became actively misleading after D.3
landed — anyone reading the code would think referral conversion was
still pending. Also corrects a stale "Paddle" reference to "Stripe"
(we never shipped Paddle; ended up on Stripe after the Paddle → Polar
→ Stripe MoR onboarding pivot).
Pure docstring sweep, no behaviour change.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The referral feature was half-built: codes captured, banner shown,
counts displayed — but no money flowed when a referred user paid.
The Settings page hard-coded "— (D.3)" for Active credits and the
marketing copy promised "50% off for 3 months" with nothing behind it.
Closing the loop:
- New `convert_referral(session, user)` in referral_service.py looks
up the user's Referral row, stamps `converted_at` + `credited_at`,
and extends `credit_until` by 45 days on BOTH the buyer and the
referrer. Idempotent — replayed webhooks and renewals are no-ops.
Stacks correctly when the user already has a credit window running
(anchors at max(now, current_credit_until) like cli.grant_credit).
- Stripe webhook wires this into `_grant_paid`. A captured
`first_paid_transition = user.tier != "paid"` gate avoids the DB
lookup on every renewal event; convert_referral's own idempotency
is the second line of defence.
- `_grant_paid` now takes `session` as its first positional arg so
the conversion runs inside the same transaction as the tier flip
and audit-row write. A mid-flight failure rolls everything back
together — no partial state.
- Settings page replaces the "— (D.3)" placeholder with the live
count of conversions still inside their 45-day credit window, plus
a "+N days on your account" hint when the user has any credit of
their own (referrer bonus, admin grant, or future refund-as-credit).
- Marketing copy on pricing.html + settings.html switches from "50%
off for 3 months" to "45 days of paid access" — same economic value,
honest about the actual mechanism (full free access rather than
discounted billing).
Credit-amount rationale: 50% × 3 months ≈ 1.5 months of free
service ≈ 45 days. Pure-credit delivery is processor-agnostic, needs
no Stripe coupon plumbing, and stacks cleanly across referrals.
7 new tests in test_referral_conversion.py cover the happy path,
idempotency, no-referral no-op, credit stacking, deleted-referrer
survival, end-to-end webhook → credit landing, and the renewal-event
no-double-credit guarantee.
Also bundled: the Restore-button class fix from earlier
(portfolio.js — the cloud-restore "Restore" submit was unstyled and
picked up browser defaults; now uses .settings-btn like the rest of
the action-button family).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Settings page tidy-up driven by user feedback that it had grown too busy:
- Each section (Import, Invite, Email digests, Cloud sync) is now a
native <details>/<summary> accordion. Import stays open by default
because /settings#import is the deep-link target from the dashboard
CTA; the others collapse so the page lands quiet.
- Manage subscription is a right-aligned gear-icon button instead of
a rectangular text button — the descriptive copy moves into the
tooltip. Frees up the Tier row of visual weight.
Auth + modal inputs were too small (verify code box, portfolio restore
PIN): the auth-card selector now covers text inputs as well, and a new
.modal-input class standardises 16px / 12px-padding fields used in the
cloud-sync enable modal and the portfolio restore prompt.
The verify page no longer carries the "Email me the digest" checkbox —
it was misleading on repeat logins (server-side it only applied on
first sign-up but rendered every time). Default-opt-in lives in the
User row at creation; per-user changes happen on /settings. First
successful verify now triggers a one-shot welcome email explaining the
digest cadence and pointing at /settings for opt-out; SMTP failure is
logged but does not block the login.
Tests rewritten to cover the new welcome-email path:
- first login sends exactly one welcome email
- returning user gets none
- SMTP failure does not break the redirect
- regression guard: returning user who opted out stays opted out
Also lands the paddle merchant-summary doc that was written earlier
during the Paddle → Polar → Stripe onboarding pivot.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Marketing + behaviour pass to get the site ready for Paddle approval.
Pricing page
- £7/month, £70/year headline (was "Coming soon").
- Bigger tier names (was 11px uppercase mono — looked like chips).
- Real CTAs (button base styles were only scoped to .hero__ctas).
- "Best value" badge + drop-shadow on the Paid card; full-width
block CTAs that align across both cards.
- "Free vs Paid at a glance" comparison table beneath the cards.
- Compact "Invite a friend — both get 50% off for 3 months"
callout with the detail explanation behind a <dialog> popup.
Tier copy + behaviour now consistent
- Free strategic-log refresh is every 6 hours, not hourly. New
read-side filter on /api/log/{latest,by-date} restricts free
users to logs at boundary hours (00/06/12/18 UTC); paid users
still see the most recent.
- Follow-up chat is paid-only. /api/chat returns 402 for free;
the chat sidebar on /log is replaced with a locked aside and
chat.js no longer loads at all for free users.
- Dashboard meta lines + landing copy softened so they no longer
promise hourly to everyone.
Future-proofing copy on public pages
- Dropped "free forever" wording (we may close the free tier).
- "Trading 212 CSV" became "broker CSV (Trading 212 today; more
planned)" on pricing + landing; the actual import UIs stay
T212-specific.
Terms
- Renamed Terms of Service -> Terms and Conditions (Paddle
expectation), bumped last-updated to 2026-05-26.
- New §6 Refunds covering the 14-day cooling off, post-window
cancellation, termination-by-us refunds, statutory rights, and
how to request a refund.
- Renumbered §7-§14 and fixed the disclaimer link labels.
Tests
- 6 new tests in tests/test_chat_and_log_gates.py cover the
chat 402 + the boundary-hour filter on both log endpoints.
- Full suite: 205 passed, 5 skipped, 0 failed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds render_digest_email(kind, date_str, content_html, unsubscribe_url,
settings_url) -> tuple[str, str, str] to email_service.py, following the
same contract as render_otp_email. Includes _DIGEST_HTML_TEMPLATE with
light/dark palette from branding and _strip_html_to_text for the plain-text
fallback. Unit tests in tests/test_email_render.py cover daily, weekly, and
invalid-kind cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds an 8-byte HKDF fingerprint of the current pepper to portfolio_sync
rows. On fetch, a mismatch surfaces as 410 Gone (distinct from genuine
GCM corruption → 500), and the UI silently cleans up the dead row and
shows a soft "please re-import" notice instead of a confusing PIN
re-prompt. Legacy rows (pepper_fp NULL) are probed optimistically and
backfilled on success.
Also fixes a latent bug in unwrap(): AESGCM.decrypt args were swapped
(ct, nonce instead of nonce, ct), so restore-from-cloud always failed
even when the pepper was correct.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the unauthenticated surface that's needed to invite outsiders:
- Landing (/) — dual-purpose root: dashboard for logged-in users,
landing for everyone else. New maybe_current_user soft-auth helper
in app/auth.py supports it without disturbing the per-route
require_token deps on /news, /log, /upload, /settings.
- About, Pricing, Disclaimer, Terms, Privacy — own router
(app/routers/public.py), no auth dep, shared public_base layout
(brand link, thin nav, footer with legal links + ICO ref + date).
- Editorial positioning: news aggregator with a macro brain; tagline
"Understand markets. Don't gamble on them."; anti-trading-as-gambling
stance carried through About and Landing.
Legal pass following an independent lawyer-style review:
- Privacy: explicit UK-GDPR Art. 6 lawful-basis section; Art. 22
automated-decision line; explicit consent for sessionStorage sync
key (PECR); 30-day IP-log retention; Art. 21 objection right;
Children clause; Art. 33/34 breach-notification clause;
international-transfer mechanism (IDTA + UK Addendum). ICO
registration ZC098928 surfaced at the top.
- Pricing: paid-card AI-portfolio-analysis bullet rewritten to remove
advice-shaped wording ("what would invalidate the posture" gone);
added italic carve-out citing FSMA / FCA COBS.
- Disclaimer: separate EU/EEA carve-out + MAR 596/2014 Art. 3(1)(34)
commentator safe-harbour; "qualifies the Terms" line; hallucination
wording fixed.
- Terms: cl.4 explicit AI-training prohibition + harassment line;
cl.5 CCR 2013 14-day cancellation; cl.7 softened AI copyright
claim under CDPA s.9(3) ambiguity; cl.8 proportionate suspension +
pro-rata refund for paid users; cl.10 CRA 2015 Pt 1 statutory-rights
carve-out from the liability cap; cl.11 right to close account on
material change; cl.12 non-exclusive jurisdiction + UK consumer
local courts.
Code-side enforcement of the Privacy claim:
- openrouter.py: outbound OpenRouter calls now carry
X-OR-Allow-Training: false. DeepSeek doesn't expose a per-request
flag; the Privacy page discloses this caveat verbatim.
Apex domain prep:
- branding.APP_URL flipped to https://read.markets (was app.). DNS for
the apex already resolves; pending operator NPM step is a cert that
covers the bare apex + a 301 from app.read.markets. No hard-coded
subdomain references remain in code (verified with grep).
Nav + chrome:
- app dropdown gains Pricing / Terms / Privacy / Disclaimer links.
- login.html gains a small legal-links footer for the
highest-leverage moment to surface them.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
A 6h gap meant weekend visitors could see the feed sit on the same
1pm batch through to dinner. Tightening to 2h gives roughly 12
ingests/weekend-day at a fraction of the active-window load (which
stays at 20-min cadence).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- dashboard grid: explicit "header" area as the first row so the
aggregated read panel renders at the top instead of being
auto-placed after the named areas.
- indicators: hide rows flagged stale (older than the group's
freshness threshold). Server still computes stale_symbols;
rendering can be re-enabled by removing the
`{% if not is_stale %}` wrapper in indicators.html.
- /log: add tone-changed to #log-content's hx-trigger and include
it in cassandraSetTone's selector list — toggling Novice /
Intermediate on the Log page was previously a no-op.
- prompts: bump PROMPT_VERSION 7→8. Strengthen the rational-vs-
irrational framing in the strategic-log system prompt from
aspirational to mandatory ("a paragraph without both lenses must
be rewritten"). Require the same lens in the per-group summary,
cross-asset aggregate, and portfolio commentary overrides.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The product is now "Read the Markets" served at https://read.markets,
with the app at https://app.read.markets. "Cassandra" survives only as
the in-product AI persona (system prompt + "Ask Cassandra" chat label).
Centralised the brand in app/branding.py: BRAND_NAME, BRAND_SHORT,
DOMAIN, SITE_URL, APP_URL, EMAIL_FROM_DEFAULT. Jinja templates pull
{{ BRAND_NAME }} via globals registered in templates_env.py; Python
code reads branding.BRAND_NAME directly. The future-rename surface
is now a one-liner.
Updated: FastAPI app title, every page title (dashboard, news, log,
settings, upload, login, verify), header brand div, auth-card brands,
OTP email subject + HTML + plain-text bodies (incl. uppercase header
tag), OpenRouter X-Title + HTTP-Referer attribution headers, README.
Email tests now assert against branding.BRAND_NAME rather than the
literal string.
Internal identifiers deliberately kept on the legacy "cassandra" name
to avoid invalidating live sessions / advisory locks / configs:
cookies (cassandra_session, cassandra_pending) + itsdangerous salts,
MariaDB GET_LOCK keys, CASSANDRA_TOKEN env var, cassandra.css filename,
pyproject package name, localStorage prefs, outbound User-Agent strings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lays the billing-prep spine before Paddle lands in D.3.
D.1 — referrals
- users.referral_code: unique 8-char URL-safe code (alphabet excludes the
ambiguous 0/O/1/I/L). Generated lazily on first /settings hit so existing
accounts pick one up without a backfill migration.
- users.referred_by_user_id + new referrals audit table (referrer,
referred, created_at, converted_at, credited_at). converted_at /
credited_at stay null until D.3 fills them via the Paddle webhook.
- POST /login accepts ?ref=<code>; the code rides on the signed
pending-verify cookie so it survives the GET → POST → /verify hop.
- /settings page: email, tier badge, referral code chip + invite link
with one-click copy, pending/converted/active-credits stats grid.
Settings nav link added to the top bar.
Reward shape: when the referred user makes their first paid Paddle
subscription, both they and the referrer get 50% off for 3 months.
(D.3 wires the actual credit application via the Paddle webhook.)
D.2 — paid-access gate
- users.credit_until: timestamp until which a free-tier account has
paid-tier access. Null = no credit. Populated by admin CLI now and the
D.3 webhook later.
- app.services.access exposes paid_status(user) → PaidStatus dataclass
(active / source / expires_at / days_remaining), is_paid_active() with
admin-bearer-token bypass, and a require_paid FastAPI dependency that
raises 402 Payment Required for free-tier callers.
- POST /api/analyze (portfolio AI commentary) gated behind require_paid.
- Settings page surfaces credit window when active ("free · credit · N
day(s) remaining (expires YYYY-MM-DD)") and the upgrade hint when not.
- Admin CLI: python -m app.cli {grant-credit,revoke-credit,show-status}.
grant-credit is idempotent — extends from max(now, current expiry) so
re-running the command never erodes an existing grant.
Migrations 0013 (referrals) and 0014 (credit_until). Tests cover the
paid-status truth table, code generation + normalisation, CLI argument
parsing, and the pending-cookie ref roundtrip (29 new tests).
- Move news_job from hourly to 3x/hour (cron 10,30,50), with a CadencePolicy
gate that throttles to active hours (07-21 UTC weekdays at 20 min), off-hours
(3 h), weekends (6 h). Keeps the daytime feed fresh without spamming RSS
sources overnight.
- Tag each headline on ingestion via DeepSeek (BATCH_SIZE=25, max_tokens=4000,
json.JSONDecoder().raw_decode + per-row regex recovery for resilient parsing).
Vocabulary: 16 tags including new EU / USA / AI / Conflict. NULL tags are
picked up automatically on the next news_job run, so back-tagging is implicit
rather than a separate migration step.
- Tag UI: pill bar above the feed with off → include → exclude cycle on click;
shift-click jumps straight to exclude. State persists in localStorage and is
injected into /api/news requests via htmx:configRequest. Per-row chips sit to
the right of the headline (new 5-column grid: age | source | title | tags |
UTC) so vertical density stays high.
- Strategic log header bug: model was hallucinating "(Updated 21:30 UTC)" in
future tense. Bumped PROMPT_VERSION 6→7, added explicit ban on time-of-day
clauses, and supply the actual current UTC time in the user prompt so the
model has no need to invent one.
Migration 0012 adds headlines.tags (JSON, nullable). Tests cover vocabulary
integrity, validation/normalisation, and the JSON-recovery parser (17 tests).
Server no longer holds portfolios. Holdings live in the browser
(localStorage); the server publishes an anonymous ticker_universe and a
gzipped /api/universe payload identical for every authenticated user, so
access patterns can't betray which tickers a user holds. AI commentary
is generated ephemerally from the browser-supplied pie and the cost
ledger row records no positions. Migrations 0009-0011 added the
universe table and dropped positions / portfolio_snapshots /
portfolios.
Authentication is now e-mail OTP only. Migration 0010 dropped
password_hash and email_verified (every active session is by
construction proof of email control). The /signup endpoint is gone;
signup and login share a single email-entry page. Email rendering is
HTML+plain-text multipart with a shared brand palette (app/branding.py)
asserted in sync with the CSS by a drift-detection test.
LLM provider defaults to DeepSeek-direct (cheaper, api.deepseek.com)
with OpenRouter as automatic fallback if DeepSeek fails. ai_log_job and
indicator_summary_job now iterate the two tones (NOVICE, INTERMEDIATE)
per cycle so the dashboard's tone toggle is instant; PROMPT_VERSION
bumped to 6 with an educational anti-TA / anti-gambling stance baked
into _CORE. NOVICE mode renders a curated glossary inline (CBOE VIX,
yield curve, HY OAS, etc.) with JS-positioned tooltips that survive
viewport edges and sticky bars. Model name and tokens hidden from the
user UI; still recorded in StrategicLog.model and AICall for admin.
Layout adds a sticky top nav, a sticky bottom markets bar (one chip per
exchange with status LED + headline index + 1d change), and
Phase H feedback reporting is queued in tasks/todo.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the static bearer-token gate with a real auth boundary. The
existing CASSANDRA_TOKEN path is retained as an admin / scripting escape
hatch — kept compatible by aliasing require_token to require_auth.
- New users table (migration 0007): email, argon2 password_hash, tier,
email_verified (declared but not enforced until phase E), settings_json
for the tone/analysis/anchor knobs we'll wire in phase D.
- app/services/auth_service.py: argon2-cffi password hashing with timing-
attack-resistant authenticate() (always runs a hash verify even on
unknown-email to deny a username-enumeration oracle).
- app/auth.py rewritten: require_auth returns a CurrentUser with either
is_admin=True (bearer path) or a User object (session path). Failing
requests get 303 → /login for HTML, 401 for API. Sessions signed with
itsdangerous against CASSANDRA_SESSION_SECRET; 14-day TTL.
- app/routers/auth.py: /login, /signup, /logout. Login form preserves the
?next=… param for redirect-after-login. Signup respects a new
CASSANDRA_SIGNUP_ENABLED flag.
- Standalone /login + /signup templates (no app chrome). base.html grows
a user chip + logout link in the header (reads request.state.current_user).
Phase A's main known limitations are documented in the plan: email
verification is declared but not enforced; session revocation is
best-effort (cookie-only, not DB-backed). Both land in phase E.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Completes Phase B. The full alternative-onboarding flow is now end-to-end:
drop a T212 pie CSV → parser → InstrumentMap resolver → PortfolioSnapshot
+ Position rows, all without ever asking the user for broker credentials.
- persist_pie() in app/services/csv_import.py: takes a ParsedPie, resolves
each Slice via InstrumentMap, writes Portfolio + Snapshot + Position
rows. Unmapped slices are still persisted using their CSV values and
surfaced in the response for the UI to warn about.
- POST /api/portfolios/upload: multipart endpoint accepting CSV file +
optional portfolio_name + currency. 2 MiB cap. Returns import summary.
- /upload page with drag-drop dropzone, file input fallback, and inline
result panel showing invested/value/result + unmapped-slice warnings.
- New "Import" link in the header nav.
Verified end-to-end against the real T212 export: all 13 positions land
with correct T212 tickers (incl. FPp_EQ for the Paris TotalEnergies
listing the heuristic resolver picks), zero unmapped slices, totals
reconcile to the penny.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First two slices of the multi-user roadmap (Phase B). Validates the
core onboarding mechanic against the user's real T212 export before
paying any auth/tenancy tax.
CSV parser (app/services/csv_import.py):
- Header-name matched (survives T212 reordering columns between
exports), tolerant of UTF-8 BOM, dash/N/A/empty markers, thousand-
separator commas, blank rows, zero-quantity stubs, missing Total row.
- Returns ParsedPie(name, positions, invested, value, result) with
derived avg_price + current_price per share in account currency.
- 14 tests covering happy path on the real CSV + 13 edge cases.
InstrumentMap (migration 0006 + app/services/instrument_map.py):
- Catalogue table mapping T212 ticker → Yahoo ticker, populated by
sync_from_t212() against the dev's read-only API key. Manual rows
(manual=True) are protected from auto-overwrite.
- Pure t212_ticker_to_yahoo() handles both suffix forms: single
trailing exchange letter (l/a/p/d/m/s/...) and country code (US,
DE, FR, IT, CA, ...). All 13 of the user's holdings + 15 case-
coverage tests pass.
- Live sync against T212 ingests 17,050 instruments (~2.2% unmappable
on exotic exchanges; can extend the suffix map later).
- resolve_slice() picks the right listing per shortName using a
UK-friendly currency preference (GBX > GBP > EUR > USD). Resolved
correctly for all 13 of the user's positions, including TTE on
Paris vs the NYSE dual-listing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Refactored CadencePolicy.active_start_hour/active_end_hour into a tuple
of (start, end) hour pairs so additional regional windows can be added
without code changes. Default keeps EU/US-only behaviour identical.
The Asia window (00:00-08:00 UTC — Tokyo + HK + Shanghai) is included
as a commented-out tuple in the dataclass default. Uncomment one line
to enable hourly AI cadence during the Asia session as well.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two changes that together cut OpenRouter spend ~50% and give the daily
log temporal awareness.
1. CadencePolicy (app/services/cadence.py): expensive AI jobs only
fire hourly during the EU/US active window (Mon-Fri 07-21 UTC).
Off-hours weekdays throttle to every 4h; weekends to every 12h.
ai_log_job and indicator_summary_job both consult the policy before
doing real work; market/news/portfolio ingest jobs stay hourly
(cheap, no API cost). Skipped runs land in job_runs with status
'skipped' and the throttle reason in error.
2. Update mode for ai_log_job: when an earlier log exists for the
current UTC day, it's passed to the model as 'Earlier log from
today (generated HH:MM UTC)'. The system prompt grows an Update
mode section instructing the model to revise — not restart — and
anchor on what has CHANGED since the earlier draft. The TL;DR
leads with intra-day change when meaningful, the watch list evolves
rather than restarts. PROMPT_VERSION bumped to 5.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DeepSeek occasionally regurgitates the system prompt verbatim
("Constraints: ≤60 words...", "Example good: ..."). Three-pronged fix:
1. Removed the inline good/bad example blocks from the per-group and
aggregate system prompts — DeepSeek was treating them as templates
to copy. The hard constraints alone are clear enough.
2. Expanded the LEAK_PATTERNS list to catch the prompt-label echoes
that still occasionally slip through ("Key observations:", "The
indicators are:", "Must cite ...", "Should give ...", bare "Key:").
Cleanup now runs up to 6 passes for compound leakage.
3. Added looks_like_leakage() — if the cleaned output still contains
tell-tale phrases ("≤60 words", "instructions:", etc.), the summary
is skipped rather than persisted. Logs a 'leakage_detected' warning
and an ai_calls row with status=leaked so we can see the failure
rate over time. The previous good summary stays visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ECB Statistical Data Warehouse joins as a 5th data source — open API,
no key, daily euro-area yield curve data. Symbol format
'ECB:dataset/series_key', e.g. 'ECB:YC/B.U2.EUR.4F.G_N_A.SV_C_YM.SR_10Y'
for daily 10y AAA spot rate.
Bonds tab adds ECB EZ 10y AAA + 2y AAA so there's at least some
currently-fresh European sovereign data alongside the US Treasuries.
Country-specific yields (Bund/OAT/BTP/Gilt/JGB) remain on Eurostat/FRED
monthly mirrors — no free daily source exists for those.
Stale threshold is now per-group instead of a flat 90 days. Daily-tape
groups (bonds, rates, equity, etc.) flag stale after a week or three;
monthly groups (economy, macro, valuation) stay at 60-90 days. The
bonds tab will now correctly show 30-60 day-old country yields as
stale next to the daily US/ECB ones.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
