openrouter.py was 790 lines mixing two orthogonal concerns:
- Prompt engineering (build_system_prompt, build_summary_*,
build_chat_*, build_daily_digest_*, etc.) — ~400 lines, changes
weekly as PROMPT_VERSION bumps
- LLM transport (call_llm, _provider_chain, _call_provider, retry
+ fallback machinery) — ~250 lines, rarely changes
Extracted the prompt-engineering surface to app/services/llm_prompts.py.
Transport stays in openrouter.py (consistent with the filename — the
OpenRouter URL is the transport's anchor).
All import sites (jobs, routers, services, tests) split their
multi-import lines into two: prompt-things from llm_prompts, transport
from openrouter. PROMPT_VERSION constant, _TONE_ALIASES, _resolve_tone,
and SYSTEM_PROMPT moved with the prompt functions.
No behaviour change — pure relocation. Function signatures, body, and
naming all preserved.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Three recently-added tables (strategic_log_translations,
indicator_summary_translations, csv_format_templates) drifted from
the codebase's existing naming convention:
- llm_model -> model
- llm_cost_usd -> cost_usd
- content_md -> content (on the two translation tables; csv_format
doesn't have a content field)
Also added prompt_tokens and completion_tokens to the three tables;
they were silently dropped at write time despite LogResult exposing
them.
All writer call sites (ai_log_job, indicator_summary_job,
llm_csv_parser) and reader call sites (api.py localized helpers)
updated to match. Tests realigned.
Migration 0025 uses batch_alter_table for SQLite compatibility.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The same per-test sqlite-engine setup was duplicated across 14 test
files (~30 lines each). Consolidated into a single async fixture
`db_factory` in tests/conftest.py; tests now take db_factory as a
parameter and use `async with db_factory() as session` directly.
No behaviour change — same function-scope, same in-memory schema
created via Base.metadata.create_all, same app.db._engine /
_session_factory rebinding so module-level helpers see the test
engine. Just ~420 lines of boilerplate removed.
The HTMX log endpoints in api.py do their own localization via
_localized_content; the pages.py helper was added during the
initial localization wiring but was bypassed once HTMX rendering
landed. No call sites remain.
- pyproject already sets asyncio_mode=auto, so async def tests are
collected as async automatically. Removed the redundant decorator
from four files (test_i18n, test_llm_csv_parser, test_ticker_validate,
test_localization_integration); the bare async def is enough.
- StrategicLogTranslation.log_id used the _PK autoincrement type for
a non-PK FK column. Replaced with a portable BigInteger that emits
Integer on SQLite and BigInteger elsewhere — matches the migration's
sa.BigInteger() declaration.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds module-level _resolve_log_content(session, log_id, lang) helper
to app/routers/pages.py: looks up StrategicLogTranslation by (log_id,
lang) when lang != 'en'; falls back silently to the English original
when no translation row exists yet (the expected case for the first
hour after a new language activates, or when translation fails for a
specific log).
log_page / log_page_day pull cu.user.lang and thread it through
_log_page_context so the template renders the right variant.
Two tests cover both branches.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The route's resolve-slice loop is T212-specific — it looks tickers up
against the InstrumentMap, which only has T212's universe. For the LLM
path the ticker is already Yahoo-ready (e.g. VOD.L, ASML.AS), so
sending it through resolve_slice produced spurious "could not be
resolved" warnings and dropped the positions.
Fix: ParsedPie gains a ``tickers_resolved`` flag (default False for
T212 backward-compat); _apply_mapping in the LLM path sets it True
and also extracts currency from the LLM-mapped currency_col into a
new ``ParsedPosition.currency`` field. The route branches on the flag:
LLM-path positions are kept verbatim with a best-effort InstrumentMap
lookup for nicer name/currency overrides, never dropped.
Integration test tightened to assert all 5 IBKR fixture positions
round-trip with the right currencies (USD / GBP / EUR).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Heuristic refined from the plan draft: candidate header rows must be
followed by a row containing at least one numeric token. Without this,
IBKR-style multi-line preambles (all-text rows before the real header)
would be mistaken for the header at preamble=0.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The referral feature was half-built: codes captured, banner shown,
counts displayed — but no money flowed when a referred user paid.
The Settings page hard-coded "— (D.3)" for Active credits and the
marketing copy promised "50% off for 3 months" with nothing behind it.
Closing the loop:
- New `convert_referral(session, user)` in referral_service.py looks
up the user's Referral row, stamps `converted_at` + `credited_at`,
and extends `credit_until` by 45 days on BOTH the buyer and the
referrer. Idempotent — replayed webhooks and renewals are no-ops.
Stacks correctly when the user already has a credit window running
(anchors at max(now, current_credit_until) like cli.grant_credit).
- Stripe webhook wires this into `_grant_paid`. A captured
`first_paid_transition = user.tier != "paid"` gate avoids the DB
lookup on every renewal event; convert_referral's own idempotency
is the second line of defence.
- `_grant_paid` now takes `session` as its first positional arg so
the conversion runs inside the same transaction as the tier flip
and audit-row write. A mid-flight failure rolls everything back
together — no partial state.
- Settings page replaces the "— (D.3)" placeholder with the live
count of conversions still inside their 45-day credit window, plus
a "+N days on your account" hint when the user has any credit of
their own (referrer bonus, admin grant, or future refund-as-credit).
- Marketing copy on pricing.html + settings.html switches from "50%
off for 3 months" to "45 days of paid access" — same economic value,
honest about the actual mechanism (full free access rather than
discounted billing).
Credit-amount rationale: 50% × 3 months ≈ 1.5 months of free
service ≈ 45 days. Pure-credit delivery is processor-agnostic, needs
no Stripe coupon plumbing, and stacks cleanly across referrals.
7 new tests in test_referral_conversion.py cover the happy path,
idempotency, no-referral no-op, credit stacking, deleted-referrer
survival, end-to-end webhook → credit landing, and the renewal-event
no-double-credit guarantee.
Also bundled: the Restore-button class fix from earlier
(portfolio.js — the cloud-restore "Restore" submit was unstyled and
picked up browser defaults; now uses .settings-btn like the rest of
the action-button family).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Settings page tidy-up driven by user feedback that it had grown too busy:
- Each section (Import, Invite, Email digests, Cloud sync) is now a
native <details>/<summary> accordion. Import stays open by default
because /settings#import is the deep-link target from the dashboard
CTA; the others collapse so the page lands quiet.
- Manage subscription is a right-aligned gear-icon button instead of
a rectangular text button — the descriptive copy moves into the
tooltip. Frees up the Tier row of visual weight.
Auth + modal inputs were too small (verify code box, portfolio restore
PIN): the auth-card selector now covers text inputs as well, and a new
.modal-input class standardises 16px / 12px-padding fields used in the
cloud-sync enable modal and the portfolio restore prompt.
The verify page no longer carries the "Email me the digest" checkbox —
it was misleading on repeat logins (server-side it only applied on
first sign-up but rendered every time). Default-opt-in lives in the
User row at creation; per-user changes happen on /settings. First
successful verify now triggers a one-shot welcome email explaining the
digest cadence and pointing at /settings for opt-out; SMTP failure is
logged but does not block the login.
Tests rewritten to cover the new welcome-email path:
- first login sends exactly one welcome email
- returning user gets none
- SMTP failure does not break the redirect
- regression guard: returning user who opted out stays opted out
Also lands the paddle merchant-summary doc that was written earlier
during the Paddle → Polar → Stripe onboarding pivot.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bundles three related pieces that came out of the operator's first
end-to-end test of the paid flow:
1. Manage subscription button on /settings (paid users with a real
Stripe sub — i.e. not credit-granted access). POSTs to the existing
/api/stripe/portal endpoint; Stripe-hosted customer portal handles
card updates, cancellation, monthly↔annual switch, invoice history.
Replaces the stale "Paid features unlock with Paddle (D.3) or
invite credits" hint for free users with a live link to /pricing.
2. Per-cadence cooling-off treatment:
- **Annual £70**: 14-day free trial via
subscription_data.trial_period_days=14. No money moves during
the trial, so the CCR 2013 14-day refund question doesn't arise
(nothing paid = nothing to refund). Card is still required at
checkout so Stripe can charge on day 15.
- **Monthly £7**: bills immediately. A 14-day trial there would
give away ~50% of cycle one. Instead, /pricing now carries a
required tick-box above the Subscribe buttons (subscribe stays
disabled until checked) — by ticking, the user expressly
consents to begin performance immediately and acknowledges that
this extinguishes their statutory 14-day right under Reg 36
CCR 2013. Consent collected on our own page (not via Stripe's
account-wide consent_collection.terms_of_service) so each
product can keep its own Terms URL as we add more.
3. T&C §6 clause 1 split into 1a (annual / trial substitute) +
1b (monthly / Reg 36 waiver via on-page tick-box). Clause 2
(post-cooling-off cancellation) unchanged.
Settings page shows "Free trial — N days remaining" while the
sub is in `trialing` status, falling back to "Paid subscription
active." once it transitions to active. Countdown is computed
server-side from User.stripe_trial_end_at (new column, migration
0020) populated by the subscription.created/updated webhook from
the Stripe trial_end timestamp; cleared on the trialing→active
transition and on revoke.
Drive-by: fixed a structlog kwarg-name collision on
`log.warning(..., event=event_type, ...)` in both polar_webhook.py
and stripe_billing.py — `event` is structlog's positional event
name and "got multiple values" crashed the user-not-found log
path. Renamed to `event_type=` everywhere it appeared. Caught by
the new trialing-stores-trial-end test.
Tests
- 4 new in test_stripe_billing.py covering monthly (no trial, no
consent_collection), annual (trial, no consent), trialing stores
trial_end, trialing→active clears trial_end.
- 1 existing test renamed + reworked for the consent split.
- Full suite: 224 passed, 5 skipped.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Stripe is the merchant-on-record for read.markets after Polar/Paddle
both declined the financial-media category. This commit lands the
full subscription flow: an "Upgrade" button on /pricing now opens a
real Stripe-hosted Checkout, completes the subscription, and the
webhook flips user.tier to "paid" idempotently.
Endpoints
- POST /api/stripe/checkout (require_auth) — creates a hosted
Checkout Session in subscription mode, passes user.id as
client_reference_id + email as customer_email, returns the URL
for the page-side JS to redirect to. Reuses an existing
stripe_customer_id to avoid duplicate Stripe customers on repeat
checkouts. allow_promotion_codes=True so the referral-credit
redemption can attach a coupon at checkout once that flow ships.
- POST /api/stripe/portal (require_auth) — mints a Stripe Customer
Portal session. Used by /settings; returns 404 until the user has
a stripe_customer_id (i.e. completed at least one checkout).
- POST /api/stripe/webhook — signature-verified via
stripe.Webhook.construct_event. Idempotent via UNIQUE on
stripe_events.event_id. Event dispatch:
checkout.session.completed → grant paid, store IDs
customer.subscription.created → grant paid (active/trialing)
customer.subscription.updated → grant paid (active/trialing)
customer.subscription.deleted → drop to free, clear sub id
invoice.paid / failed → audit only
charge.refunded → audit only
Stripe-SDK objects don't expose dict.get(); we use the SDK for
signature verification then re-parse the JSON body for handler
dispatch — cleaner than reaching into StripeObject internals.
Schema (migration 0019)
- users.stripe_customer_id, users.stripe_subscription_id (nullable
String(64), UNIQUE on customer_id).
- stripe_events table mirroring polar_events: event_id (unique),
event_type, received_at, processed_at, error, raw payload
(truncated to 16 KiB).
Settings (.env)
- STRIPE_API_KEY (rk_test_… for dev, rk_live_… for GA)
- STRIPE_WEBHOOK_SECRET (whsec_… from the dashboard endpoint)
- STRIPE_PRICE_MONTHLY (price_xxx for £7/month)
- STRIPE_PRICE_ANNUAL (price_xxx for £70/year)
Pricing page
- Free tier CTA unchanged.
- Paid CTA branches three ways: paid → "Manage subscription" to
/settings; logged-in free → two buttons (£7/mo, £70/yr) that POST
to /api/stripe/checkout and redirect; anonymous → /login?next=/pricing.
- Inline JS intercepts the button click, calls the checkout
endpoint, redirects on success, surfaces errors via alert(). No
Stripe.js dep — we use the hosted-checkout URL directly.
Polar handler stays in place for berengar.io / flyroom.net which
still ship through Polar. polar_* and stripe_* columns coexist
independently on the User row.
Tests
- 9 in tests/test_stripe_billing.py covering: bad signature → 401,
missing signature → 400, checkout.session.completed flips tier +
stores IDs, subscription.updated active grants paid,
subscription.deleted drops to free with customer id preserved,
replayed event id is no-op (one row in stripe_events),
unknown event acked 200, checkout endpoint mocks the SDK and
returns the hosted URL, checkout requires login.
- Full suite: 221 passed, 5 skipped.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Standalone router for inbound Polar (merchant-of-record) deliveries.
No bearer-token dep — authenticity comes from the Standard Webhooks
HMAC instead. Wired up so it's safe to deploy dark: empty
POLAR_WEBHOOK_SECRET makes the endpoint return 503 (loud) rather than
accept unsigned events.
Behaviour
- Standard Webhooks signature verification: HMAC-SHA256 over
`{webhook-id}.{webhook-timestamp}.{body}`, base64 secret prefixed
whsec_, ±5min replay window, constant-time compare against any of
the space-separated v1 tokens.
- Idempotency via UNIQUE on polar_events.event_id — a replayed
webhook-id short-circuits to 200 "duplicate" without re-running.
- Event dispatch table covers the 10 events we subscribed to:
subscription.{created,active,updated,uncanceled} -> tier=paid +
persist polar_customer_id / polar_subscription_id.
subscription.revoked -> tier=free (customer id kept so a resub
matches the same User row).
canceled / past_due / order.* / refund.created -> audit only.
- Unknown event types are acked 200 + recorded; we don't want to 4xx
on something Polar adds in the future and trigger their retry loop.
Schema (migration 0018)
- users.polar_customer_id, users.polar_subscription_id (both nullable
String(64)); UNIQUE on polar_customer_id so two users can't claim
the same Polar identity.
- polar_events table: event_id (unique), event_type, received_at,
processed_at, error, raw payload (truncated to 16 KiB).
Tests
- 7 in tests/test_polar_webhook.py: bad signature -> 401, stale
timestamp -> 401, missing headers -> 400, subscription.active flips
tier to paid + stores IDs, subscription.revoked drops to free while
keeping customer link, replayed webhook-id is no-op, unknown event
is acked.
- Full suite: 212 passed, 5 skipped.
Operator next steps before saving the webhook in Polar
1. Pull this branch to prod and apply migration 0018.
2. Save the webhook in Polar pointing at
https://read.markets/api/polar/webhook — Polar will accept the
save even though our endpoint still 503s (no secret yet).
3. Copy the secret Polar reveals into the prod .env as
POLAR_WEBHOOK_SECRET=whsec_... and restart the app.
4. Trigger a test event from Polar's dashboard to confirm 200 OK.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Marketing + behaviour pass to get the site ready for Paddle approval.
Pricing page
- £7/month, £70/year headline (was "Coming soon").
- Bigger tier names (was 11px uppercase mono — looked like chips).
- Real CTAs (button base styles were only scoped to .hero__ctas).
- "Best value" badge + drop-shadow on the Paid card; full-width
block CTAs that align across both cards.
- "Free vs Paid at a glance" comparison table beneath the cards.
- Compact "Invite a friend — both get 50% off for 3 months"
callout with the detail explanation behind a <dialog> popup.
Tier copy + behaviour now consistent
- Free strategic-log refresh is every 6 hours, not hourly. New
read-side filter on /api/log/{latest,by-date} restricts free
users to logs at boundary hours (00/06/12/18 UTC); paid users
still see the most recent.
- Follow-up chat is paid-only. /api/chat returns 402 for free;
the chat sidebar on /log is replaced with a locked aside and
chat.js no longer loads at all for free users.
- Dashboard meta lines + landing copy softened so they no longer
promise hourly to everyone.
Future-proofing copy on public pages
- Dropped "free forever" wording (we may close the free tier).
- "Trading 212 CSV" became "broker CSV (Trading 212 today; more
planned)" on pricing + landing; the actual import UIs stay
T212-specific.
Terms
- Renamed Terms of Service -> Terms and Conditions (Paddle
expectation), bumped last-updated to 2026-05-26.
- New §6 Refunds covering the 14-day cooling off, post-window
cancellation, termination-by-us refunds, statutory rights, and
how to request a refund.
- Renumbered §7-§14 and fixed the disclaimer link labels.
Tests
- 6 new tests in tests/test_chat_and_log_gates.py cover the
chat 402 + the boundary-hour filter on both log endpoints.
- Full suite: 205 passed, 5 skipped, 0 failed.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The "pie" indicator group was removed in v0.2 when portfolio composition
moved to live Trading 212 sourcing (see the v0.2 note in
config/portfolio.toml). The test wasn't updated at the time. Suite now
runs cleanly: 199 passed, 5 skipped, 0 failed.
Five fixes uncovered by actually running the suite in docker-compose.test.yml:
1. (real prod bug) PATCH /api/settings/digest mutated principal.user which
require_token had loaded in a now-closed session — the commit on the
handler's session persisted nothing. Re-fetch the user via the active
session before writing.
2. Portable PK type. SQLite only auto-fills `INTEGER PRIMARY KEY`; plain
BIGINT requires explicit values. Define a `_PK` alias of
`BigInteger().with_variant(Integer(), "sqlite")` and use it for all 10
autoincrement primary keys in app/models.py. No prod-schema change
(MariaDB still gets BIGINT).
3. job_lifecycle's MariaDB GET_LOCK / RELEASE_LOCK is now gated behind
`dialect.name == "mysql"`, so the test SQLite engine doesn't trip on
the missing function. Single-process test runs can't race themselves.
4. tests/test_news_window.py seeded Headline rows without `fingerprint`,
which is NOT NULL — added an `fp-{title}` value per row.
5. tests/test_email_digest_job.py now also patches `llm_configured` to
True so the job doesn't short-circuit on the missing API key.
6. (test container hygiene) Drop `COPY tests ./tests` from the test stage
in the Dockerfile — .dockerignore excludes `tests/` (correct: prod
image must not bake tests), and docker-compose.test.yml bind-mounts
./tests at run time anyway.
Suite now: 198 passed, 5 skipped, 1 pre-existing failure
(test_default_groups_present — Phase G dropped the "pie" group from
config/default.toml but the assertion wasn't updated; unrelated to this
branch).
- verify_submit now applies the subscribe checkbox only at first sign-up.
Returning users keep whatever they set via Settings or the one-click
unsubscribe link — previously, every login silently re-enrolled them.
- JOB_NAMES gains email_digest_job so the ops footer reflects its health.
Adds tests/test_verify_subscribe.py::test_returning_user_login_preserves_unsubscribe.
Adds render_digest_email(kind, date_str, content_html, unsubscribe_url,
settings_url) -> tuple[str, str, str] to email_service.py, following the
same contract as render_otp_email. Includes _DIGEST_HTML_TEMPLATE with
light/dark palette from branding and _strip_html_to_text for the plain-text
fallback. Unit tests in tests/test_email_render.py cover daily, weekly, and
invalid-kind cases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds an 8-byte HKDF fingerprint of the current pepper to portfolio_sync
rows. On fetch, a mismatch surfaces as 410 Gone (distinct from genuine
GCM corruption → 500), and the UI silently cleans up the dead row and
shows a soft "please re-import" notice instead of a confusing PIN
re-prompt. Legacy rows (pepper_fp NULL) are probed optimistically and
backfilled on success.
Also fixes a latent bug in unwrap(): AESGCM.decrypt args were swapped
(ct, nonce instead of nonce, ct), so restore-from-cloud always failed
even when the pepper was correct.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Swaps the role of `:root` (now light) and the data-theme attribute
(now `[data-theme="dark"]`) in cassandra.css, flips the localStorage
fallback from 'dark' to 'light' in base/login/verify templates, and
updates the theme-toggle label and the branding-consistency test
selectors to match.
Existing users with cassandra.theme=dark in localStorage still see
dark — their explicit preference wins. Only first-time visitors and
users with no stored preference shift to light.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The product is now "Read the Markets" served at https://read.markets,
with the app at https://app.read.markets. "Cassandra" survives only as
the in-product AI persona (system prompt + "Ask Cassandra" chat label).
Centralised the brand in app/branding.py: BRAND_NAME, BRAND_SHORT,
DOMAIN, SITE_URL, APP_URL, EMAIL_FROM_DEFAULT. Jinja templates pull
{{ BRAND_NAME }} via globals registered in templates_env.py; Python
code reads branding.BRAND_NAME directly. The future-rename surface
is now a one-liner.
Updated: FastAPI app title, every page title (dashboard, news, log,
settings, upload, login, verify), header brand div, auth-card brands,
OTP email subject + HTML + plain-text bodies (incl. uppercase header
tag), OpenRouter X-Title + HTTP-Referer attribution headers, README.
Email tests now assert against branding.BRAND_NAME rather than the
literal string.
Internal identifiers deliberately kept on the legacy "cassandra" name
to avoid invalidating live sessions / advisory locks / configs:
cookies (cassandra_session, cassandra_pending) + itsdangerous salts,
MariaDB GET_LOCK keys, CASSANDRA_TOKEN env var, cassandra.css filename,
pyproject package name, localStorage prefs, outbound User-Agent strings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Lays the billing-prep spine before Paddle lands in D.3.
D.1 — referrals
- users.referral_code: unique 8-char URL-safe code (alphabet excludes the
ambiguous 0/O/1/I/L). Generated lazily on first /settings hit so existing
accounts pick one up without a backfill migration.
- users.referred_by_user_id + new referrals audit table (referrer,
referred, created_at, converted_at, credited_at). converted_at /
credited_at stay null until D.3 fills them via the Paddle webhook.
- POST /login accepts ?ref=<code>; the code rides on the signed
pending-verify cookie so it survives the GET → POST → /verify hop.
- /settings page: email, tier badge, referral code chip + invite link
with one-click copy, pending/converted/active-credits stats grid.
Settings nav link added to the top bar.
Reward shape: when the referred user makes their first paid Paddle
subscription, both they and the referrer get 50% off for 3 months.
(D.3 wires the actual credit application via the Paddle webhook.)
D.2 — paid-access gate
- users.credit_until: timestamp until which a free-tier account has
paid-tier access. Null = no credit. Populated by admin CLI now and the
D.3 webhook later.
- app.services.access exposes paid_status(user) → PaidStatus dataclass
(active / source / expires_at / days_remaining), is_paid_active() with
admin-bearer-token bypass, and a require_paid FastAPI dependency that
raises 402 Payment Required for free-tier callers.
- POST /api/analyze (portfolio AI commentary) gated behind require_paid.
- Settings page surfaces credit window when active ("free · credit · N
day(s) remaining (expires YYYY-MM-DD)") and the upgrade hint when not.
- Admin CLI: python -m app.cli {grant-credit,revoke-credit,show-status}.
grant-credit is idempotent — extends from max(now, current expiry) so
re-running the command never erodes an existing grant.
Migrations 0013 (referrals) and 0014 (credit_until). Tests cover the
paid-status truth table, code generation + normalisation, CLI argument
parsing, and the pending-cookie ref roundtrip (29 new tests).
