phase G: data minimisation + passwordless auth + DeepSeek-first LLM

Server no longer holds portfolios. Holdings live in the browser
(localStorage); the server publishes an anonymous ticker_universe and a
gzipped /api/universe payload identical for every authenticated user, so
access patterns can't betray which tickers a user holds. AI commentary
is generated ephemerally from the browser-supplied pie and the cost
ledger row records no positions. Migrations 0009-0011 added the
universe table and dropped positions / portfolio_snapshots /
portfolios.

Authentication is now e-mail OTP only. Migration 0010 dropped
password_hash and email_verified (every active session is by
construction proof of email control). The /signup endpoint is gone;
signup and login share a single email-entry page. Email rendering is
HTML+plain-text multipart with a shared brand palette (app/branding.py)
asserted in sync with the CSS by a drift-detection test.

LLM provider defaults to DeepSeek-direct (cheaper, api.deepseek.com)
with OpenRouter as automatic fallback if DeepSeek fails. ai_log_job and
indicator_summary_job now iterate the two tones (NOVICE, INTERMEDIATE)
per cycle so the dashboard's tone toggle is instant; PROMPT_VERSION
bumped to 6 with an educational anti-TA / anti-gambling stance baked
into _CORE. NOVICE mode renders a curated glossary inline (CBOE VIX,
yield curve, HY OAS, etc.) with JS-positioned tooltips that survive
viewport edges and sticky bars. Model name and tokens hidden from the
user UI; still recorded in StrategicLog.model and AICall for admin.

Layout adds a sticky top nav, a sticky bottom markets bar (one chip per
exchange with status LED + headline index + 1d change), and
Phase H feedback reporting is queued in tasks/todo.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-05-18 14:16:57 +01:00
parent 480fd311c5
commit 6e7f57c6b2
54 changed files with 5005 additions and 916 deletions

View file

@ -1,5 +1,6 @@
"""Hourly market ingestion: fetch every (symbol, group) defined in TOML and
insert one Quote row per fetch."""
"""Hourly market ingestion: fetch every (symbol, group) defined in TOML
*plus* every ticker in the Phase G shared ticker_universe, inserting one
Quote row per fetch."""
from __future__ import annotations
import asyncio
@ -11,6 +12,7 @@ from app.db import utcnow
from app.jobs._helpers import job_lifecycle, log
from app.models import Quote
from app.services.market import fetch
from app.services.ticker_universe import get_all_tickers
async def run() -> None:
@ -21,11 +23,27 @@ async def run() -> None:
groups = load_groups(s.BASELINE_TOML, s.PORTFOLIO_TOML)
anchor = s.CASSANDRA_ANCHOR_DATE or None
# Build the (group, symbol, label, note) work list from config TOML.
items_flat: list[tuple[str, str, str, str]] = [
(group, sym, lab, note)
for group, items in groups.items()
for sym, lab, note in items
]
configured_syms = {sym for _, sym, _, _ in items_flat}
# Phase G: extend with anything in ticker_universe that isn't
# already covered by config. These land under group_name="universe"
# — the /api/universe endpoint reads the latest quote per symbol
# regardless of group.
universe_tickers = await get_all_tickers(session)
for t in universe_tickers:
if t not in configured_syms:
items_flat.append(("universe", t, t, ""))
async with httpx.AsyncClient(follow_redirects=True) as client:
tasks = [
fetch(client, sym, lab, note, anchor)
for group, items in groups.items()
for sym, lab, note in items
for _, sym, lab, note in items_flat
]
# Run in parallel but bounded — Yahoo can throttle if we hammer.
sem = asyncio.Semaphore(16)
@ -34,14 +52,8 @@ async def run() -> None:
return await t
quotes = await asyncio.gather(*(bounded(t) for t in tasks))
# Re-index quotes back to their group for persistence.
items_flat = [
(group, sym)
for group, items in groups.items()
for sym, _, _ in items
]
now = utcnow()
for (group, _sym), q in zip(items_flat, quotes):
for (group, _sym, _lab, _note), q in zip(items_flat, quotes):
session.add(Quote(
symbol=q.symbol,
source=q.source,
@ -58,7 +70,12 @@ async def run() -> None:
))
await session.commit()
run.items_written = len(quotes)
log.info("market_job.done", count=len(quotes))
log.info(
"market_job.done",
count=len(quotes),
configured=len(configured_syms),
universe=len(universe_tickers),
)
if __name__ == "__main__":