docs: accessible figure legends (README.md) for all figures
One self-contained README.md per results/ figure folder (Layer 1 E1-E6 and Layer 1.5 bridge/collapse/grounding/architectures/recombination): plain-language claim, setup, a compact symbol glossary, a panel-by-panel walkthrough, and the takeaway + falsifier. Auto-renders when browsing the folder; carries the honest caveats (grounding's ruler reframing, the excluded VAE). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
b8da418034
commit
3b9f4f7893
11 changed files with 369 additions and 0 deletions
32
results/E1/README.md
Normal file
32
results/E1/README.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# E1 — Distillation without grounding collapses, tail-first
|
||||
|
||||
**Claim tested:** if a model is trained only on the previous model's output, generation after
|
||||
generation, does it lose knowledge — and does the *rare* knowledge go first?
|
||||
|
||||
**Setup (Layer 1, pure math).** A "population" of `K = 500` items with a fixed true frequency
|
||||
`p*` shaped like a Zipf curve (a few common items, a long tail of rare ones). Each generation we
|
||||
draw `n = 100` samples from the current model and refit — **no real data is ever added** (`g = 0`).
|
||||
Run for 600 generations, averaged over 100 independent repeats.
|
||||
|
||||
### Symbols
|
||||
- **`p*`** — the true frequencies (fixed reality). **`p_t`** — the model's frequencies at generation *t* (drifts).
|
||||
- **`H`** heterozygosity = diversity (1 = everything equally likely, 0 = one item left). **`H*`** = diversity of the truth.
|
||||
- **forward-KL** `D(p*‖p_t)` — how far the model has drifted from truth (0 = perfect, grows without bound as the tail is forgotten).
|
||||
- **support** = how many items still have any probability. **head/tail** = common/rare items.
|
||||
|
||||
### The three panels
|
||||
1. **Geometric decay.** Blue = the simulated diversity `H`; black dashed = the exact textbook law
|
||||
`H₀·(1 − 1/n)^t`. They sit on top of each other — the loss of diversity is *exactly* the
|
||||
population-genetics drift law, not an approximation. (This is the validation gate: if these two
|
||||
curves disagreed, the simulator would be wrong.)
|
||||
2. **Tail dies first** (log axis). Red = fraction of *rare* (tail) items still alive; green =
|
||||
fraction of *common* (head) items still alive. The red curve plunges far faster — rare knowledge
|
||||
is lost roughly an order of magnitude sooner than common knowledge.
|
||||
3. **Collapse.** Purple (left axis, log) = number of distinct items surviving, falling from 500
|
||||
toward ~1 (everything collapses onto a single dominant item). Orange (right axis) = forward-KL to
|
||||
truth, diverging as the tail vanishes.
|
||||
|
||||
### Takeaway
|
||||
Unchecked model-on-model training is a ratchet: diversity decays on a precise mathematical schedule,
|
||||
and the rare tail is destroyed first. **Falsifier (not triggered):** if `H` had stayed flat, the
|
||||
whole thesis would fail. It didn't.
|
||||
35
results/E2/README.md
Normal file
35
results/E2/README.md
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
# E2 — A tiny dose of real data rescues diversity (the headline)
|
||||
|
||||
**Claim tested:** how much *real* data must you mix back in each generation to stop collapse — a
|
||||
lot, or a little?
|
||||
|
||||
**Setup (Layer 1, pure math).** `K = 1000` items, Zipf truth `p*`, `n = 200` inherited samples per
|
||||
generation, 500 generations, 100 repeats. Each generation we also mix in `m` fresh **real** samples
|
||||
drawn from `p*`. The knob swept is the **grounding fraction** `g = m/(n+m)` — the share of the
|
||||
training pool that is real — across `g ∈ {0, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.4}`.
|
||||
|
||||
### Symbols
|
||||
- **`g`** grounding fraction (share of real data); **`g*`** the *critical* value that restores diversity.
|
||||
- **`p*`** truth, **`p_t`** model, **`H`** diversity, **`H*`** truth's diversity.
|
||||
- **grounding = "immigration"** in the genetics analogy: real samples are migrants that re-introduce alleles drift keeps killing.
|
||||
|
||||
### The four panels
|
||||
1. **Trajectories.** Diversity `H` over generations, one line per `g` (dark = dry, bright = more
|
||||
grounding). `g = 0` slides toward 0; any `g > 0` levels off on a plateau — the collapse is
|
||||
arrested.
|
||||
2. **The phase boundary (the headline).** Dots = stationary diversity vs `g`; the black dashed curve
|
||||
is the *exact* closed-form equilibrium `H_eq`; the red line marks the critical
|
||||
**`g* ≈ 0.048` (95% CI [0.047, 0.050])** where `H` reaches 95% of the truth's diversity. Only
|
||||
**~5% real data** buys back essentially all the diversity. The `g = 0` point is drawn hollow (it
|
||||
is still sliding — its true equilibrium is 0).
|
||||
3. **Tail coverage.** Fraction of the rare tail retained, by item-count (red) and truth-mass-weighted
|
||||
(purple). Both rise with `g` but stay modest at feasible grounding: a little grounding restores
|
||||
*diversity* long before it restores the *deep* tail — which motivates E4 (recombination) and E6.
|
||||
4. **Per-rarity band.** The tail split into rarity bands (bright = shallowest, dark = deepest). Deep
|
||||
bands lag: an item survives only once `m·p*ᵢ ≳ 1` (enough real samples per generation to land it
|
||||
at least once). The sharp threshold lives here, in discrete item survival — not in the smooth `H`.
|
||||
|
||||
### Takeaway
|
||||
There is a **critical grounding fraction `g* ≪ 1`**: a small, constant trickle of reality
|
||||
indefinitely holds off collapse. **Falsifier (not triggered):** if diversity had only recovered as
|
||||
`g → 1`, the practical thesis would die. It recovers at `g ≈ 0.05`.
|
||||
33
results/E3/README.md
Normal file
33
results/E3/README.md
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
# E3 — Grounding must overlap the content it protects
|
||||
|
||||
**Claim tested:** if your budget of real data is fixed, does it matter *where* you spend it? Is it
|
||||
enough to sprinkle real data uniformly, or must it target the knowledge you care about?
|
||||
|
||||
**Setup (Layer 1, pure math).** `K = 1000` items divided into `R = 10` disjoint **regions** (think
|
||||
subject areas). `n = 200`, 400 generations, 100 repeats. The **same total** real-data budget is
|
||||
spent two ways: **uniform** (spread evenly over all 10 regions) vs **matched** (concentrated on the
|
||||
"exercised" region we want to protect). One region is designated exercised (region 0 here) and has a
|
||||
rare tail we track.
|
||||
|
||||
### Symbols
|
||||
- **region** — a block of related items; a stand-in for a task family / subject area.
|
||||
- **matched vs uniform** — real data aimed at the exercised region vs spread evenly, at equal total budget.
|
||||
- **tail items alive** — how many rare items in a region still have any probability.
|
||||
|
||||
### The two panels
|
||||
1. **Target region over time.** Rare-item survival in the exercised region, generation by generation:
|
||||
blue = matched, red = uniform (bands = 95% CI). Matched **holds** the region's tail alive; uniform
|
||||
spreads too thin and lets it **collapse**.
|
||||
2. **Every region at steady state.** Bar chart of stationary tail survival per region, matched (blue)
|
||||
vs uniform (red); the dotted line marks the exercised region. Matched wins big *there* — at the
|
||||
cost of the regions it deliberately ignores. Uniform is mediocre everywhere.
|
||||
|
||||
*(Note: per-region diversity `H` is confounded by how much probability mass sits in a region, so this
|
||||
figure uses the honest, mass-independent metric — tail-item survival.)*
|
||||
|
||||
### Takeaway
|
||||
Reality checks only protect what they actually cover. To keep a capability alive you must ground
|
||||
**on that capability**, not on data in general — grounding is local, not a global tonic. Key numbers:
|
||||
exercised-region tail survival ≈ **0.49 (matched)** vs **0.07 (uniform)** at equal budget.
|
||||
**Falsifier (not triggered):** if uniform had protected the region as well as matched, the
|
||||
targeting claim would die.
|
||||
33
results/E4/README.md
Normal file
33
results/E4/README.md
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
# E4 — Recombination supplies the rare tail; only a union-preserving *merge* realises it
|
||||
|
||||
**Claim tested:** if several specialist models each remember a different slice of the rare tail, can
|
||||
combining them reconstruct the whole tail? And does *how* you combine them matter?
|
||||
|
||||
**Setup (Layer 1, pure math).** `K = 500` items. We build `K_T` teacher models that each retain the
|
||||
rare tail with probability `q`, and we control how **correlated** their retained tails are with a
|
||||
single knob `ρ` (rho): `ρ = 0` = fully complementary teachers, `ρ = 1` = identical teachers. Swept:
|
||||
`K_T ∈ {1,2,3,5}`, `ρ ∈ {0, 0.25, 0.5, 0.75, 1}`, grounding `g ∈ {0, 0.02, 0.05}`, 200 repeats.
|
||||
|
||||
### Symbols
|
||||
- **`K_T`** number of teacher models; **`ρ`** how correlated their retained tails are (0 = diverse, 1 = clones).
|
||||
- **union coverage** — fraction of the tail covered by *at least one* teacher (the raw supply).
|
||||
- **surviving coverage** — fraction that actually survives in the pupil after it retrains on the combination.
|
||||
- **mean-distill** — pupil trained on the pooled/averaged teacher outputs. **max-merge** — keep, per item, the strongest teacher (a union-preserving merge, à la M2N2).
|
||||
|
||||
### The three panels
|
||||
1. **Supply.** Union tail coverage vs `ρ`, one curve per `K_T`; solid lines are the exact closed form
|
||||
`U(K_T, ρ, q)`. More teachers and more diversity (lower `ρ`) supply more of the tail — and the
|
||||
simulation matches the formula exactly.
|
||||
2. **Realisation.** Surviving coverage vs `ρ`. **Solid = max-merge rises** with more/diverse teachers;
|
||||
**dashed = mean-distill stays flat.** Averaging dilutes each teacher's rare items back below the
|
||||
survival threshold — the gain is supplied but not realised.
|
||||
3. **The benefit needs the right operator (`ρ = 0`).** Surviving coverage vs `K_T` under both
|
||||
operators. Max-merge climbs with teacher count; mean-distill is flat — a **conservation law**:
|
||||
averaging's `1/K_T` dilution exactly cancels the union gain.
|
||||
|
||||
### Takeaway — "merge, don't average"
|
||||
Complementary specialists *contain* enough to rebuild the tail, but **naive multi-teacher distillation
|
||||
(averaging) throws it away**; you must combine them with a union-preserving merge. This is load-bearing
|
||||
for the paper's recombination claim and is re-tested in real neural weights in `results/recombination/`.
|
||||
**Falsifier (not triggered):** if mean-distill had also risen with `K_T`, or max never beat mean, the
|
||||
recombination story would collapse into "just average your models."
|
||||
32
results/E5/README.md
Normal file
32
results/E5/README.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# E5 — Quality-diversity selection preserves diversity where greedy selection destroys it
|
||||
|
||||
**Claim tested:** if each generation you *select* which outputs to keep, does chasing the "best"
|
||||
outputs (greedy) accelerate collapse — and does rewarding novelty instead prevent it?
|
||||
|
||||
**Setup (Layer 1, pure math).** `K = 500` items, Zipf truth, `n = 200`, 400 generations, 100 repeats,
|
||||
all arms given the same grounding. Three selection modes: **none** (grounding only, no selection),
|
||||
**greedy** (keep the fittest — highest-`p*` — items), and **quality-diversity (QD)** (a novelty
|
||||
bonus `w_i ∝ f_i · p_i^{-α}` that up-weights rare items). The novelty exponent `α` is swept over
|
||||
`{0.5, 1, 2}`.
|
||||
|
||||
### Symbols
|
||||
- **greedy** — select toward the fittest/most-probable items (directional pressure).
|
||||
- **QD (quality-diversity)** — select for fitness *and* novelty; `α` = strength of the novelty bonus.
|
||||
- **`H`** diversity; **support** = number of distinct items surviving.
|
||||
|
||||
### The three panels
|
||||
1. **Diversity trajectories.** `H` over generations: red = greedy (crashes toward ~0, i.e. fixation
|
||||
on a few items); orange/blue = QD at `α = 1, 2` (holds a high plateau); green = none (reference).
|
||||
Greedy selection is a *second* collapse engine on top of drift.
|
||||
2. **Novelty dose–response.** Stationary `H` vs the novelty exponent `α` for QD (orange dots), with
|
||||
greedy (red dashed) and none (green dashed) as reference lines. QD sits above greedy for **every**
|
||||
`α`, and rises as the novelty bonus strengthens.
|
||||
3. **Surviving items per arm.** Stationary support (number of distinct items alive) as bars: greedy is
|
||||
lowest; QD arms keep progressively more items alive as `α` grows; none is the reference.
|
||||
|
||||
### Takeaway
|
||||
Optimising only for "what looks best" (greedy) collapses the population onto a handful of winners; a
|
||||
novelty-rewarding, quality-diversity objective actively **re-introduces and holds the tail**. Key
|
||||
numbers: greedy `H ≈ 0.01` (near-total fixation) vs QD `H ≈ 0.48–0.88` rising with `α`.
|
||||
**Falsifier (not triggered):** if QD's stationary `H` had been ≤ greedy's, quality-diversity would be
|
||||
doing no work.
|
||||
34
results/E6/README.md
Normal file
34
results/E6/README.md
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
# E6 — Re-minting is irreversible; gate it on diversity
|
||||
|
||||
**Claim tested:** what happens if you "re-baseline" — declare the current model's output to be the new
|
||||
ground truth and throw away the original? If you do this while the model is already collapsed, is the
|
||||
damage permanent? And can a simple safeguard prevent it?
|
||||
|
||||
**Setup (Layer 1, pure math).** `K = 500`, `n = 200`, 400 generations, 100 repeats. **Re-minting**
|
||||
periodically freezes the current distribution as the new grounding reference and *discards the
|
||||
original truth* (it survives only as a yardstick for measuring drift). Four arms:
|
||||
- **healthy re-mint** — generous grounding (`m = 60`), re-mint while still diverse;
|
||||
- **collapsed re-mint (ungated)** — starved grounding (`m = 1`), re-mint anyway;
|
||||
- **collapsed + diversity gate** — same starvation, but only re-mint if diversity `H ≥ 0.75`;
|
||||
- **collapsed, no re-mint** — the baseline.
|
||||
|
||||
### Symbols
|
||||
- **re-mint** — adopt the current model's output as the new "reality" and discard the original truth (a founder event).
|
||||
- **diversity gate** — refuse to re-mint while `H` is below a threshold (here 0.75).
|
||||
- **forward-KL to ORIGINAL truth** — how far the lineage has drifted from the *real* original, even after it changed its own reference.
|
||||
|
||||
### The two panels
|
||||
1. **Lock-in.** Forward-KL to the *original* truth over generations; dotted verticals mark re-mint
|
||||
events. Red (collapsed, ungated) **jumps up at each re-mint and never comes back** — once the
|
||||
original tails are gone, re-baselining onto the impoverished distribution makes the loss permanent
|
||||
(they can no longer be grounded back). Green (healthy) and blue (gated) stay low; grey (baseline)
|
||||
is the reference.
|
||||
2. **What the gate reads.** Diversity `H` over generations, same colour key, with the gate threshold
|
||||
(`H = 0.75`, dashed). The gated arm simply **refuses to re-mint while below the line**, so it never
|
||||
locks in a collapsed state; the ungated collapsed arm re-mints into the floor.
|
||||
|
||||
### Takeaway
|
||||
Re-minting a collapsed model **crystallises** the collapse — it is a one-way door. A trivial
|
||||
safeguard (only re-baseline when diversity is still high) preserves recoverability; re-minting a
|
||||
healthy model is harmless. **Falsifier (not triggered):** if the collapsed lineage had recovered its
|
||||
original tails after re-minting, the irreversibility claim would be overstated.
|
||||
32
results/architectures/README.md
Normal file
32
results/architectures/README.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# architectures — collapse and rescue are architecture-general
|
||||
|
||||
**Claim tested:** is model collapse (and its cure, grounding) a quirk of one model type, or does the
|
||||
same signature appear across genuinely different neural architectures?
|
||||
|
||||
**Setup (Layer 1.5).** The identical generational loop is run with three different generative models —
|
||||
an **exact histogram** (no neural net), an **autoregressive GRU** (recurrent), and a **causal-masked
|
||||
MLP** (feed-forward) — each a distinct "inductive bias." `K = 256` modes, `n = 200`, 22 generations,
|
||||
5 repeats, compared at **dry (`g = 0`)** vs **grounded (`g = 0.05`)**.
|
||||
|
||||
### Symbols
|
||||
- **inductive bias** — the built-in assumptions of a model type (a histogram has none; a GRU and an MLP smooth differently).
|
||||
- **forward-KL** distance from truth; **tail items alive** — how many rare modes still appear.
|
||||
- **dry** = no grounding; **grounded** = 5% real data mixed in.
|
||||
|
||||
### The three panels
|
||||
1. **Trajectories.** Forward-KL over generations, coloured by architecture; **solid = dry** (climbs,
|
||||
collapse) vs **dashed = grounded** (held down). The dry-up / grounded-down gap appears in **every**
|
||||
architecture.
|
||||
2. **Stationary forward-KL (grouped bars).** For each architecture, dry (red) vs grounded (green).
|
||||
Divergence **falls with grounding across all three** — histogram, GRU, MLP.
|
||||
3. **Tail-item survival (grouped bars).** Same grouping. Survival **rises with grounding across all
|
||||
three.** (Note the histogram's bars are tiny: with no smoothing it drops rare modes outright,
|
||||
whereas the GRU/MLP keep some alive — an inductive-bias difference, not a contradiction.)
|
||||
|
||||
### Takeaway
|
||||
The Wright–Fisher collapse operator and the grounding rescue are **not artefacts of one model** — they
|
||||
show up in an exact counter, a recurrent net, and a feed-forward net alike. This is the
|
||||
architecture-generality claim of Layer 1.5. **Falsifier (not triggered):** if the signs had appeared
|
||||
only for the histogram, collapse would be a property of the idealised operator, not of trained models.
|
||||
*(A VAE was also implemented but fails the generation-0 fidelity check on this task, so it is excluded
|
||||
to avoid confusing underfitting with collapse — documented as a known limitation.)*
|
||||
32
results/bridge/README.md
Normal file
32
results/bridge/README.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# bridge — the histogram model reproduces Layer-1 E2 exactly (the HARD GATE)
|
||||
|
||||
**Claim tested (a plumbing check, not science):** does the neural experiment harness, when run with a
|
||||
*trivial* model, reproduce the Layer-1 math exactly? If not, no later neural result could be trusted.
|
||||
|
||||
**Setup (Layer 1.5).** Same generational loop as every neural experiment — each generation, draw the
|
||||
parent's samples, optionally mix in real data, retrain, measure — but the "model" is a **histogram**:
|
||||
it just counts which modes appeared and resamples them (no neural net, no smoothing). This reduces the
|
||||
neural setup *exactly* back to Wright–Fisher drift. `K = 200` modes, `n = 200`, 200 generations,
|
||||
60 repeats, grounding swept `g ∈ {0, …, 0.4}`.
|
||||
|
||||
### Symbols
|
||||
- **mode** = one of the `K` items (Layer-1.5 word for "item"); read off each generated sequence by a zero-error oracle.
|
||||
- **`g`** grounding fraction, **`g*`** its critical value, **`H`** diversity, **`H_eq`** the exact closed-form equilibrium diversity.
|
||||
|
||||
### The four panels
|
||||
1. **Trajectories.** Diversity `H` per `g`. `g = 0` collapses; `g > 0` plateaus — the E2 picture,
|
||||
now produced by the *neural runner*.
|
||||
2. **Bridge = Layer 1 (the gate).** Dots = the neural histogram runner's stationary `H` vs `g`; the
|
||||
black dashed curve = the exact `H_eq` closed form from Layer 1. The dots sit **on** the curve, and
|
||||
the recovered critical grounding is **`g* = 0.047` (CI [0.045, 0.052])** — matching Layer-1's
|
||||
0.048. This equality is what licenses every trained-model result to be read against the analytic
|
||||
core.
|
||||
3. **Tail survival.** Fraction of the rare tail retained vs `g` (item-count red, truth-mass purple) —
|
||||
rises with grounding, deep tail lags, exactly as in E2.
|
||||
4. **Per-rarity band.** Survival by rarity band vs `g`; deep bands need more grounding
|
||||
(`m·p*ᵢ ≳ 1`).
|
||||
|
||||
### Takeaway
|
||||
The harness is faithful: with a memoryless model it reproduces Layer 1 to the decimal. **This is a
|
||||
gate, not a finding** — passing it is the precondition for the RNN/MLP experiments
|
||||
(`collapse`, `grounding`, `architectures`, `recombination`), where the *model* is what changes.
|
||||
32
results/collapse/README.md
Normal file
32
results/collapse/README.md
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# collapse — model collapse in REAL neural weights, arrested by grounding
|
||||
|
||||
**Claim tested:** does the collapse we proved in math actually happen in a *trained neural network* —
|
||||
and does a little grounding stop it?
|
||||
|
||||
**Setup (Layer 1.5).** The model is now a small **autoregressive GRU** (a recurrent net, ~128 hidden
|
||||
units). Each generation a **fresh** GRU is trained from scratch, by ordinary next-token prediction, on
|
||||
the previous GRU's own generated sequences (plus any real samples). `K = 256` modes, `n = 200`,
|
||||
25 generations, grounding `g ∈ {0, 0.02, 0.05, 0.1}`, 5 repeats. A generation-0 fidelity check
|
||||
confirms the GRU reproduces the truth almost perfectly (KL ≈ 0.008) before any collapse is measured.
|
||||
|
||||
### Symbols
|
||||
- **GRU** — a small recurrent neural net that emits sequences token by token; retrained each generation on the prior generation's output.
|
||||
- **forward-KL** `D(p*‖p_t)` — distance from truth; the metric that actually sees neural collapse.
|
||||
- **`H`** diversity; **tail items alive** — how many rare modes still appear.
|
||||
|
||||
### The four panels
|
||||
1. **Collapse in weights.** Forward-KL over generations, one line per `g`. The dry lineage (`g = 0`)
|
||||
**climbs** (drifts from truth) toward ~2.3; grounded lineages are held lower. Collapse is real in
|
||||
trained weights.
|
||||
2. **`H` barely moves.** Diversity over generations sits near `H*` for all arms — the GRU's smoothing
|
||||
bias keeps spurious spread alive, so **diversity `H` hides the collapse**. (This is why forward-KL,
|
||||
not `H`, is the operative neural metric — see `grounding`.)
|
||||
3. **Stationary divergence vs `g`.** End-state forward-KL falls as grounding rises: more real data →
|
||||
closer to truth.
|
||||
4. **Tail survival vs `g`.** Fraction of rare modes alive rises with grounding.
|
||||
|
||||
### Takeaway
|
||||
The core phenomenon transfers from math to real neural nets: **a GRU trained on its own output drifts
|
||||
from truth, and grounding arrests it** — the *sign* Layer-1 predicts (blueprint §3.5). Note the
|
||||
honest caveat surfaced here and developed in `grounding`: **diversity `H` is the wrong ruler for a
|
||||
neural net** (smoothing keeps it high even during collapse); distance-from-truth is the right one.
|
||||
38
results/grounding/README.md
Normal file
38
results/grounding/README.md
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
# grounding — the grounding response in real weights (and why the ruler matters)
|
||||
|
||||
**Claim tested:** does the E2 result — a small dose of real data (`g* ≈ 0.05`) rescues diversity —
|
||||
reproduce in a trained GRU? The honest answer reframes the question.
|
||||
|
||||
**Setup (Layer 1.5).** Autoregressive GRU, `K = 256` modes, `n = 200`, 30 generations, grounding
|
||||
swept over 9 values `g ∈ {0, 0.005, …, 0.2}`, **18 repeats** (many repeats are needed because each
|
||||
lineage's fate is genuinely noisy under `n = 200` drift). The falsifier was pinned in the config
|
||||
*before* running.
|
||||
|
||||
### Symbols
|
||||
- **`g`** grounding fraction (share of real data), **`g*`** its critical value.
|
||||
- **forward-KL** distance from truth (the operative neural collapse metric here).
|
||||
- **tail survival** `tail_truth_mass_alive` — truth-weighted fraction of the rare tail retained. **`H`** diversity.
|
||||
- **recovery fraction** — how much of the achievable forward-KL reduction a given `g` has bought (0 = dry, 1 = best observed).
|
||||
|
||||
### The four panels
|
||||
1. **Trajectories.** Forward-KL over generations per `g`: grounding suppresses the climb.
|
||||
2. **Phase boundary.** Stationary forward-KL vs `g` **falls monotonically** (dry ≈ 2.08 → `g = 0.2`
|
||||
≈ 0.75); the effect is statistically significant (paired *t* up to 3.3; 89% of lineages improve at
|
||||
`g = 0.2`). **The SIGN is confirmed.**
|
||||
3. **Recovery curve.** Fraction of the divergence gap closed vs `g`. **Half the gap closes by a
|
||||
median-recovery grounding of `g ≈ 0.04`** (CI [0.004, 0.116]) — a striking echo of Layer-1's 0.048
|
||||
(black dashed) — **but full recovery needs `g ≈ 0.19`**, far more than the exact histogram: the
|
||||
GRU's smoothing both caps the collapse and slows the rescue.
|
||||
4. **Why forward-KL (the key methodological panel).** Normalised responses of three rulers vs `g`:
|
||||
`H/H*` (flat ~0.8) and **tail survival (flat / non-monotone — dry is as high as grounded!)** both
|
||||
fail to register the effect, while **forward-KL recovery** responds cleanly. A smoothing model keeps
|
||||
*spurious* tail support alive, so counting surviving modes is misleading; only distance-from-truth
|
||||
is honest.
|
||||
|
||||
### Takeaway (an honest reframing)
|
||||
Two results: **(1)** the operative neural collapse metric is **forward-KL**, not `H` or tail-survival —
|
||||
smoothing decouples "modes alive" from "close to truth." **(2)** The *sharp* threshold `g* ≪ 1` is a
|
||||
property of the exact operator, carried quantitatively by the histogram **bridge** (`g* = 0.047`); the
|
||||
trained GRU confirms grounding's **direction** and **softens** its sharpness. The pre-registered
|
||||
95%-of-`H*`/tail falsifier is *not* met — but because those are the wrong rulers for a smoothing
|
||||
model, not because grounding fails; the blueprint §3.5 directional claim holds robustly.
|
||||
36
results/recombination/README.md
Normal file
36
results/recombination/README.md
Normal file
|
|
@ -0,0 +1,36 @@
|
|||
# recombination — "merge, don't average" holds in real neural weights (E4 in the flesh)
|
||||
|
||||
**Claim tested:** the E4 finding — complementary specialists can rebuild the rare tail, but only if you
|
||||
*merge* them rather than average them — was proven in math. Does it survive in trained neural nets?
|
||||
|
||||
**Setup (Layer 1.5).** `K_T` specialist **GRUs** are each trained on a different slice of the rare
|
||||
tail (slices set by the exact shared-switch construction, so teacher count `K_T`, correlation `ρ` and
|
||||
retention `q` stay clean knobs and the construction-level union matches the closed form). The pupil then
|
||||
recombines the teachers' *measured* distributions two ways: **mean** (naive pooling) vs
|
||||
**oracle-guided max-merge** (per mode, keep the strongest teacher — a union-preserving merge). `K = 256`,
|
||||
`n = 200`, `q = 0.5`, swept `K_T ∈ {1,2,3,5}` × `ρ ∈ {0, 1}`, 8 repeats.
|
||||
|
||||
### Symbols
|
||||
- **`K_T`** number of specialist teachers; **`ρ`** how correlated their retained tails are (0 = complementary, 1 = identical clones).
|
||||
- **union coverage** — tail covered by ≥1 teacher (the raw supply). **surviving coverage** — what remains after the pupil resamples.
|
||||
- **mean-distill** — average/pool the teachers. **max-merge** — keep each mode's strongest teacher (union-preserving).
|
||||
- **target vs trained** — coverage computed from the *assigned* distributions vs from the *trained GRU* outputs.
|
||||
|
||||
### The four panels
|
||||
1. **Supply.** Union coverage vs `K_T`: at `ρ = 0` it climbs 0.49 → 0.96 and matches the closed form
|
||||
`U(K_T, ρ, q)`; at `ρ = 1` (clones) it is flat. Diverse teachers supply more tail.
|
||||
2. **Analytic teachers (`ρ = 0`).** Surviving coverage vs `K_T`: **max-merge (green) rises**
|
||||
0.043 → 0.087 while **mean-distill (red) stays flat ~0.045** — the conservation law from E4,
|
||||
reproduced on the assigned distributions.
|
||||
3. **Trained GRU teachers (`ρ = 0`).** The same comparison on *real trained weights*: same signs —
|
||||
max-merge above mean-distill — but **compressed and noisier** (the GRU's smoothing inflates the
|
||||
baseline and the deep tail barely clears `n = 200` resampling). The honest inductive-bias caveat.
|
||||
4. **Control (`ρ = 1`).** Identical teachers: union, max and mean are all flat — **more clones buy
|
||||
nothing.** Decorrelation is what the benefit needs.
|
||||
|
||||
### Takeaway
|
||||
"Merge, don't average" is not a mathematical artefact — in trained neural nets, a union-preserving
|
||||
**max-merge realises the multi-teacher tail benefit while naive averaging conserves the collapse.**
|
||||
This is load-bearing for the paper's recombination claim (and points at merging over distillation for
|
||||
the deferred LLM layer). **Falsifier (not triggered):** if mean-distill had also risen with `K_T`, or
|
||||
max never beat mean, the recombination lesson would have died in real weights.
|
||||
Loading…
Add table
Add a link
Reference in a new issue