Giorgio Gilestro
6f8cef1ac5
Removed from main (all preserved on the dev branch): the arXiv build and
its sources, design documents (blueprint, results summary, review responses,
essay drafts), tasks/ and CLAUDE.md, the cover letter and reference tooling,
two unused manuscript figures, and every experiment that feeds no figure or
number in the paper: the collapse null, the sexual-vs-asexual lineage, the
NK speciation variant, the 0.5B single-seed LLM prototypes, the compose and
society experiments with their calibration and pilot runs, and their
configs, runners, tests, figure scripts and PBS jobs. Their result bundles
are moved to results/_archive/ (ignored) so the parquets stay on disk.
Also: plot_llm_speciation reads the s{seed}/ layout; the mating-breadth
plot writes under its bundle name; Makefile targets reduced to the kept
experiments; REPRODUCING.md and README point to dev for the rest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
|
||
|---|---|---|
| .. | ||
| collapse.pdf | ||
| collapse.png | ||
| manifest.json | ||
| README.md | ||
| resolved_config.yaml | ||
collapse — model collapse in REAL neural weights, arrested by grounding
Claim tested: does the collapse we proved in math actually happen in a trained neural network — and does a little grounding stop it?
Setup (Layer 1.5). The model is now a small autoregressive GRU (a recurrent net, ~128 hidden
units). Each generation a fresh GRU is trained from scratch, by ordinary next-token prediction, on
the previous GRU's own generated sequences (plus any real samples). K = 256 modes, n = 200,
25 generations, grounding g ∈ {0, 0.02, 0.05, 0.1}, 5 repeats. A generation-0 fidelity check
confirms the GRU reproduces the truth almost perfectly (KL ≈ 0.008) before any collapse is measured.
Symbols
- GRU — a small recurrent neural net that emits sequences token by token; retrained each generation on the prior generation's output.
- forward-KL
D(p*‖p_t)— distance from truth; the metric that actually sees neural collapse. Hdiversity; tail items alive — how many rare modes still appear.
The four panels
- Collapse in weights. Forward-KL over generations, one line per
g. The dry lineage (g = 0) climbs (drifts from truth) toward ~2.3; grounded lineages are held lower. Collapse is real in trained weights. Hbarely moves. Diversity over generations sits nearH*for all arms — the GRU's smoothing bias keeps spurious spread alive, so diversityHhides the collapse. (This is why forward-KL, notH, is the operative neural metric — seegrounding.)- Stationary divergence vs
g. End-state forward-KL falls as grounding rises: more real data → closer to truth. - Tail survival vs
g. Fraction of rare modes alive rises with grounding.
Takeaway
The core phenomenon transfers from math to real neural nets: a GRU trained on its own output drifts
from truth, and grounding arrests it — the sign Layer-1 predicts (blueprint §3.5). Note the
honest caveat surfaced here and developed in grounding: diversity H is the wrong ruler for a
neural net (smoothing keeps it high even during collapse); distance-from-truth is the right one.