docs: accessible figure legends (README.md) for all figures
One self-contained README.md per results/ figure folder (Layer 1 E1-E6 and Layer 1.5 bridge/collapse/grounding/architectures/recombination): plain-language claim, setup, a compact symbol glossary, a panel-by-panel walkthrough, and the takeaway + falsifier. Auto-renders when browsing the folder; carries the honest caveats (grounding's ruler reframing, the excluded VAE). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
b8da418034
commit
3b9f4f7893
11 changed files with 369 additions and 0 deletions
32
results/collapse/README.md
Normal file
32
results/collapse/README.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# collapse — model collapse in REAL neural weights, arrested by grounding
|
||||
|
||||
**Claim tested:** does the collapse we proved in math actually happen in a *trained neural network* —
|
||||
and does a little grounding stop it?
|
||||
|
||||
**Setup (Layer 1.5).** The model is now a small **autoregressive GRU** (a recurrent net, ~128 hidden
|
||||
units). Each generation a **fresh** GRU is trained from scratch, by ordinary next-token prediction, on
|
||||
the previous GRU's own generated sequences (plus any real samples). `K = 256` modes, `n = 200`,
|
||||
25 generations, grounding `g ∈ {0, 0.02, 0.05, 0.1}`, 5 repeats. A generation-0 fidelity check
|
||||
confirms the GRU reproduces the truth almost perfectly (KL ≈ 0.008) before any collapse is measured.
|
||||
|
||||
### Symbols
|
||||
- **GRU** — a small recurrent neural net that emits sequences token by token; retrained each generation on the prior generation's output.
|
||||
- **forward-KL** `D(p*‖p_t)` — distance from truth; the metric that actually sees neural collapse.
|
||||
- **`H`** diversity; **tail items alive** — how many rare modes still appear.
|
||||
|
||||
### The four panels
|
||||
1. **Collapse in weights.** Forward-KL over generations, one line per `g`. The dry lineage (`g = 0`)
|
||||
**climbs** (drifts from truth) toward ~2.3; grounded lineages are held lower. Collapse is real in
|
||||
trained weights.
|
||||
2. **`H` barely moves.** Diversity over generations sits near `H*` for all arms — the GRU's smoothing
|
||||
bias keeps spurious spread alive, so **diversity `H` hides the collapse**. (This is why forward-KL,
|
||||
not `H`, is the operative neural metric — see `grounding`.)
|
||||
3. **Stationary divergence vs `g`.** End-state forward-KL falls as grounding rises: more real data →
|
||||
closer to truth.
|
||||
4. **Tail survival vs `g`.** Fraction of rare modes alive rises with grounding.
|
||||
|
||||
### Takeaway
|
||||
The core phenomenon transfers from math to real neural nets: **a GRU trained on its own output drifts
|
||||
from truth, and grounding arrests it** — the *sign* Layer-1 predicts (blueprint §3.5). Note the
|
||||
honest caveat surfaced here and developed in `grounding`: **diversity `H` is the wrong ruler for a
|
||||
neural net** (smoothing keeps it high even during collapse); distance-from-truth is the right one.
|
||||
Loading…
Add table
Add a link
Reference in a new issue