History

Giorgio Gilestro 3b9f4f7893 docs: accessible figure legends (README.md) for all figures One self-contained README.md per results/ figure folder (Layer 1 E1-E6 and Layer 1.5 bridge/collapse/grounding/architectures/recombination): plain-language claim, setup, a compact symbol glossary, a panel-by-panel walkthrough, and the takeaway + falsifier. Auto-renders when browsing the folder; carries the honest caveats (grounding's ruler reframing, the excluded VAE). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>		2026-07-05 08:43:04 +01:00
..
collapse.pdf	neural: grounding refinement + all five Layer-1.5 figures	2026-07-05 08:14:19 +01:00
collapse.png	neural: grounding refinement + all five Layer-1.5 figures	2026-07-05 08:14:19 +01:00
manifest.json	Rename neural experiments to descriptive paths (drop N* codes)	2026-07-04 21:21:56 +01:00
README.md	docs: accessible figure legends (README.md) for all figures	2026-07-05 08:43:04 +01:00
resolved_config.yaml	Rename neural experiments to descriptive paths (drop N* codes)	2026-07-04 21:21:56 +01:00

README.md

collapse — model collapse in REAL neural weights, arrested by grounding

Claim tested: does the collapse we proved in math actually happen in a trained neural network — and does a little grounding stop it?

Setup (Layer 1.5). The model is now a small autoregressive GRU (a recurrent net, ~128 hidden units). Each generation a fresh GRU is trained from scratch, by ordinary next-token prediction, on the previous GRU's own generated sequences (plus any real samples). K = 256 modes, n = 200, 25 generations, grounding g ∈ {0, 0.02, 0.05, 0.1}, 5 repeats. A generation-0 fidelity check confirms the GRU reproduces the truth almost perfectly (KL ≈ 0.008) before any collapse is measured.

Symbols

GRU — a small recurrent neural net that emits sequences token by token; retrained each generation on the prior generation's output.
forward-KL D(p*‖p_t) — distance from truth; the metric that actually sees neural collapse.
H diversity; tail items alive — how many rare modes still appear.

The four panels

Collapse in weights. Forward-KL over generations, one line per g. The dry lineage (g = 0) climbs (drifts from truth) toward ~2.3; grounded lineages are held lower. Collapse is real in trained weights.
H barely moves. Diversity over generations sits near H* for all arms — the GRU's smoothing bias keeps spurious spread alive, so diversity H hides the collapse. (This is why forward-KL, not H, is the operative neural metric — see grounding.)
Stationary divergence vs g. End-state forward-KL falls as grounding rises: more real data → closer to truth.
Tail survival vs g. Fraction of rare modes alive rises with grounding.

Takeaway

The core phenomenon transfers from math to real neural nets: a GRU trained on its own output drifts from truth, and grounding arrests it — the sign Layer-1 predicts (blueprint §3.5). Note the honest caveat surfaced here and developed in grounding: diversity H is the wrong ruler for a neural net (smoothing keeps it high even during collapse); distance-from-truth is the right one.