add ECB Data Portal source; group-aware stale thresholds

ECB Statistical Data Warehouse joins as a 5th data source — open API,
no key, daily euro-area yield curve data. Symbol format
'ECB:dataset/series_key', e.g. 'ECB:YC/B.U2.EUR.4F.G_N_A.SV_C_YM.SR_10Y'
for daily 10y AAA spot rate.

Bonds tab adds ECB EZ 10y AAA + 2y AAA so there's at least some
currently-fresh European sovereign data alongside the US Treasuries.
Country-specific yields (Bund/OAT/BTP/Gilt/JGB) remain on Eurostat/FRED
monthly mirrors — no free daily source exists for those.

Stale threshold is now per-group instead of a flat 90 days. Daily-tape
groups (bonds, rates, equity, etc.) flag stale after a week or three;
monthly groups (economy, macro, valuation) stay at 60-90 days. The
bonds tab will now correctly show 30-60 day-old country yields as
stale next to the daily US/ECB ones.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-05-15 23:13:58 +01:00
parent 1edf9cad41
commit 4e7e4981e3
3 changed files with 118 additions and 6 deletions

View file

@ -55,6 +55,25 @@ router = APIRouter(dependencies=[Depends(require_token)])
JOB_NAMES = ("market_job", "news_job", "portfolio_job", "ai_log_job", "rollup_job")
JOB_STALE_HOURS = 2.0 # job is "warn" if its last success was >2h ago
# Per-group expected freshness — bonds and intraday tape want daily data,
# macro/economy/valuation are monthly/quarterly by nature. Older than this
# many days from today → row gets a "stale" badge.
_STALE_DAYS_BY_GROUP = {
"bonds": 21,
"rates": 7,
"equity": 7,
"mag7": 7,
"commodities": 7,
"fx": 7,
"tech_ai": 7,
"financials": 7,
"bubble_watch":21,
"valuation": 60,
"macro": 60,
"economy": 90,
}
_STALE_DAYS_DEFAULT = 90
# --- Small helpers -----------------------------------------------------------
@ -158,17 +177,18 @@ async def indicators(
.limit(1)
)).scalar_one_or_none()
# Mark rows whose `as_of` is older than 90 days as stale so the UI
# can dim them — some FRED international series are months/years
# behind their primary source.
# Mark rows whose `as_of` is older than the group-specific threshold.
# Daily-tape groups (bonds, rates, equity, ...) flag stale earlier
# than monthly groups (economy, macro, valuation).
today = utcnow().date()
threshold = _STALE_DAYS_BY_GROUP.get(group, _STALE_DAYS_DEFAULT)
stale_symbols: set[str] = set()
for r in rows:
try:
as_of_d = datetime.strptime(r.as_of, "%Y-%m-%d").date() if r.as_of else None
except ValueError:
as_of_d = None
if as_of_d and (today - as_of_d).days > 90:
if as_of_d and (today - as_of_d).days > threshold:
stale_symbols.add(r.symbol)
return templates.TemplateResponse(