docs: accessible figure legends (README.md) for all figures

One self-contained README.md per results/ figure folder (Layer 1 E1-E6
and Layer 1.5 bridge/collapse/grounding/architectures/recombination):
plain-language claim, setup, a compact symbol glossary, a panel-by-panel
walkthrough, and the takeaway + falsifier. Auto-renders when browsing the
folder; carries the honest caveats (grounding's ruler reframing, the
excluded VAE).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-07-05 08:43:04 +01:00
parent b8da418034
commit 3b9f4f7893
11 changed files with 369 additions and 0 deletions

32
results/E1/README.md Normal file
View file

@ -0,0 +1,32 @@
# E1 — Distillation without grounding collapses, tail-first
**Claim tested:** if a model is trained only on the previous model's output, generation after
generation, does it lose knowledge — and does the *rare* knowledge go first?
**Setup (Layer 1, pure math).** A "population" of `K = 500` items with a fixed true frequency
`p*` shaped like a Zipf curve (a few common items, a long tail of rare ones). Each generation we
draw `n = 100` samples from the current model and refit — **no real data is ever added** (`g = 0`).
Run for 600 generations, averaged over 100 independent repeats.
### Symbols
- **`p*`** — the true frequencies (fixed reality). **`p_t`** — the model's frequencies at generation *t* (drifts).
- **`H`** heterozygosity = diversity (1 = everything equally likely, 0 = one item left). **`H*`** = diversity of the truth.
- **forward-KL** `D(p*‖p_t)` — how far the model has drifted from truth (0 = perfect, grows without bound as the tail is forgotten).
- **support** = how many items still have any probability. **head/tail** = common/rare items.
### The three panels
1. **Geometric decay.** Blue = the simulated diversity `H`; black dashed = the exact textbook law
`H₀·(1 1/n)^t`. They sit on top of each other — the loss of diversity is *exactly* the
population-genetics drift law, not an approximation. (This is the validation gate: if these two
curves disagreed, the simulator would be wrong.)
2. **Tail dies first** (log axis). Red = fraction of *rare* (tail) items still alive; green =
fraction of *common* (head) items still alive. The red curve plunges far faster — rare knowledge
is lost roughly an order of magnitude sooner than common knowledge.
3. **Collapse.** Purple (left axis, log) = number of distinct items surviving, falling from 500
toward ~1 (everything collapses onto a single dominant item). Orange (right axis) = forward-KL to
truth, diverging as the tail vanishes.
### Takeaway
Unchecked model-on-model training is a ratchet: diversity decays on a precise mathematical schedule,
and the rare tail is destroyed first. **Falsifier (not triggered):** if `H` had stayed flat, the
whole thesis would fail. It didn't.

35
results/E2/README.md Normal file
View file

@ -0,0 +1,35 @@
# E2 — A tiny dose of real data rescues diversity (the headline)
**Claim tested:** how much *real* data must you mix back in each generation to stop collapse — a
lot, or a little?
**Setup (Layer 1, pure math).** `K = 1000` items, Zipf truth `p*`, `n = 200` inherited samples per
generation, 500 generations, 100 repeats. Each generation we also mix in `m` fresh **real** samples
drawn from `p*`. The knob swept is the **grounding fraction** `g = m/(n+m)` — the share of the
training pool that is real — across `g ∈ {0, 0.005, 0.01, 0.02, 0.05, 0.1, 0.2, 0.4}`.
### Symbols
- **`g`** grounding fraction (share of real data); **`g*`** the *critical* value that restores diversity.
- **`p*`** truth, **`p_t`** model, **`H`** diversity, **`H*`** truth's diversity.
- **grounding = "immigration"** in the genetics analogy: real samples are migrants that re-introduce alleles drift keeps killing.
### The four panels
1. **Trajectories.** Diversity `H` over generations, one line per `g` (dark = dry, bright = more
grounding). `g = 0` slides toward 0; any `g > 0` levels off on a plateau — the collapse is
arrested.
2. **The phase boundary (the headline).** Dots = stationary diversity vs `g`; the black dashed curve
is the *exact* closed-form equilibrium `H_eq`; the red line marks the critical
**`g* ≈ 0.048` (95% CI [0.047, 0.050])** where `H` reaches 95% of the truth's diversity. Only
**~5% real data** buys back essentially all the diversity. The `g = 0` point is drawn hollow (it
is still sliding — its true equilibrium is 0).
3. **Tail coverage.** Fraction of the rare tail retained, by item-count (red) and truth-mass-weighted
(purple). Both rise with `g` but stay modest at feasible grounding: a little grounding restores
*diversity* long before it restores the *deep* tail — which motivates E4 (recombination) and E6.
4. **Per-rarity band.** The tail split into rarity bands (bright = shallowest, dark = deepest). Deep
bands lag: an item survives only once `m·p*ᵢ ≳ 1` (enough real samples per generation to land it
at least once). The sharp threshold lives here, in discrete item survival — not in the smooth `H`.
### Takeaway
There is a **critical grounding fraction `g* ≪ 1`**: a small, constant trickle of reality
indefinitely holds off collapse. **Falsifier (not triggered):** if diversity had only recovered as
`g → 1`, the practical thesis would die. It recovers at `g ≈ 0.05`.

33
results/E3/README.md Normal file
View file

@ -0,0 +1,33 @@
# E3 — Grounding must overlap the content it protects
**Claim tested:** if your budget of real data is fixed, does it matter *where* you spend it? Is it
enough to sprinkle real data uniformly, or must it target the knowledge you care about?
**Setup (Layer 1, pure math).** `K = 1000` items divided into `R = 10` disjoint **regions** (think
subject areas). `n = 200`, 400 generations, 100 repeats. The **same total** real-data budget is
spent two ways: **uniform** (spread evenly over all 10 regions) vs **matched** (concentrated on the
"exercised" region we want to protect). One region is designated exercised (region 0 here) and has a
rare tail we track.
### Symbols
- **region** — a block of related items; a stand-in for a task family / subject area.
- **matched vs uniform** — real data aimed at the exercised region vs spread evenly, at equal total budget.
- **tail items alive** — how many rare items in a region still have any probability.
### The two panels
1. **Target region over time.** Rare-item survival in the exercised region, generation by generation:
blue = matched, red = uniform (bands = 95% CI). Matched **holds** the region's tail alive; uniform
spreads too thin and lets it **collapse**.
2. **Every region at steady state.** Bar chart of stationary tail survival per region, matched (blue)
vs uniform (red); the dotted line marks the exercised region. Matched wins big *there* — at the
cost of the regions it deliberately ignores. Uniform is mediocre everywhere.
*(Note: per-region diversity `H` is confounded by how much probability mass sits in a region, so this
figure uses the honest, mass-independent metric — tail-item survival.)*
### Takeaway
Reality checks only protect what they actually cover. To keep a capability alive you must ground
**on that capability**, not on data in general — grounding is local, not a global tonic. Key numbers:
exercised-region tail survival ≈ **0.49 (matched)** vs **0.07 (uniform)** at equal budget.
**Falsifier (not triggered):** if uniform had protected the region as well as matched, the
targeting claim would die.

33
results/E4/README.md Normal file
View file

@ -0,0 +1,33 @@
# E4 — Recombination supplies the rare tail; only a union-preserving *merge* realises it
**Claim tested:** if several specialist models each remember a different slice of the rare tail, can
combining them reconstruct the whole tail? And does *how* you combine them matter?
**Setup (Layer 1, pure math).** `K = 500` items. We build `K_T` teacher models that each retain the
rare tail with probability `q`, and we control how **correlated** their retained tails are with a
single knob `ρ` (rho): `ρ = 0` = fully complementary teachers, `ρ = 1` = identical teachers. Swept:
`K_T ∈ {1,2,3,5}`, `ρ ∈ {0, 0.25, 0.5, 0.75, 1}`, grounding `g ∈ {0, 0.02, 0.05}`, 200 repeats.
### Symbols
- **`K_T`** number of teacher models; **`ρ`** how correlated their retained tails are (0 = diverse, 1 = clones).
- **union coverage** — fraction of the tail covered by *at least one* teacher (the raw supply).
- **surviving coverage** — fraction that actually survives in the pupil after it retrains on the combination.
- **mean-distill** — pupil trained on the pooled/averaged teacher outputs. **max-merge** — keep, per item, the strongest teacher (a union-preserving merge, à la M2N2).
### The three panels
1. **Supply.** Union tail coverage vs `ρ`, one curve per `K_T`; solid lines are the exact closed form
`U(K_T, ρ, q)`. More teachers and more diversity (lower `ρ`) supply more of the tail — and the
simulation matches the formula exactly.
2. **Realisation.** Surviving coverage vs `ρ`. **Solid = max-merge rises** with more/diverse teachers;
**dashed = mean-distill stays flat.** Averaging dilutes each teacher's rare items back below the
survival threshold — the gain is supplied but not realised.
3. **The benefit needs the right operator (`ρ = 0`).** Surviving coverage vs `K_T` under both
operators. Max-merge climbs with teacher count; mean-distill is flat — a **conservation law**:
averaging's `1/K_T` dilution exactly cancels the union gain.
### Takeaway — "merge, don't average"
Complementary specialists *contain* enough to rebuild the tail, but **naive multi-teacher distillation
(averaging) throws it away**; you must combine them with a union-preserving merge. This is load-bearing
for the paper's recombination claim and is re-tested in real neural weights in `results/recombination/`.
**Falsifier (not triggered):** if mean-distill had also risen with `K_T`, or max never beat mean, the
recombination story would collapse into "just average your models."

32
results/E5/README.md Normal file
View file

@ -0,0 +1,32 @@
# E5 — Quality-diversity selection preserves diversity where greedy selection destroys it
**Claim tested:** if each generation you *select* which outputs to keep, does chasing the "best"
outputs (greedy) accelerate collapse — and does rewarding novelty instead prevent it?
**Setup (Layer 1, pure math).** `K = 500` items, Zipf truth, `n = 200`, 400 generations, 100 repeats,
all arms given the same grounding. Three selection modes: **none** (grounding only, no selection),
**greedy** (keep the fittest — highest-`p*` — items), and **quality-diversity (QD)** (a novelty
bonus `w_i ∝ f_i · p_i^{-α}` that up-weights rare items). The novelty exponent `α` is swept over
`{0.5, 1, 2}`.
### Symbols
- **greedy** — select toward the fittest/most-probable items (directional pressure).
- **QD (quality-diversity)** — select for fitness *and* novelty; `α` = strength of the novelty bonus.
- **`H`** diversity; **support** = number of distinct items surviving.
### The three panels
1. **Diversity trajectories.** `H` over generations: red = greedy (crashes toward ~0, i.e. fixation
on a few items); orange/blue = QD at `α = 1, 2` (holds a high plateau); green = none (reference).
Greedy selection is a *second* collapse engine on top of drift.
2. **Novelty doseresponse.** Stationary `H` vs the novelty exponent `α` for QD (orange dots), with
greedy (red dashed) and none (green dashed) as reference lines. QD sits above greedy for **every**
`α`, and rises as the novelty bonus strengthens.
3. **Surviving items per arm.** Stationary support (number of distinct items alive) as bars: greedy is
lowest; QD arms keep progressively more items alive as `α` grows; none is the reference.
### Takeaway
Optimising only for "what looks best" (greedy) collapses the population onto a handful of winners; a
novelty-rewarding, quality-diversity objective actively **re-introduces and holds the tail**. Key
numbers: greedy `H ≈ 0.01` (near-total fixation) vs QD `H ≈ 0.480.88` rising with `α`.
**Falsifier (not triggered):** if QD's stationary `H` had been ≤ greedy's, quality-diversity would be
doing no work.

34
results/E6/README.md Normal file
View file

@ -0,0 +1,34 @@
# E6 — Re-minting is irreversible; gate it on diversity
**Claim tested:** what happens if you "re-baseline" — declare the current model's output to be the new
ground truth and throw away the original? If you do this while the model is already collapsed, is the
damage permanent? And can a simple safeguard prevent it?
**Setup (Layer 1, pure math).** `K = 500`, `n = 200`, 400 generations, 100 repeats. **Re-minting**
periodically freezes the current distribution as the new grounding reference and *discards the
original truth* (it survives only as a yardstick for measuring drift). Four arms:
- **healthy re-mint** — generous grounding (`m = 60`), re-mint while still diverse;
- **collapsed re-mint (ungated)** — starved grounding (`m = 1`), re-mint anyway;
- **collapsed + diversity gate** — same starvation, but only re-mint if diversity `H ≥ 0.75`;
- **collapsed, no re-mint** — the baseline.
### Symbols
- **re-mint** — adopt the current model's output as the new "reality" and discard the original truth (a founder event).
- **diversity gate** — refuse to re-mint while `H` is below a threshold (here 0.75).
- **forward-KL to ORIGINAL truth** — how far the lineage has drifted from the *real* original, even after it changed its own reference.
### The two panels
1. **Lock-in.** Forward-KL to the *original* truth over generations; dotted verticals mark re-mint
events. Red (collapsed, ungated) **jumps up at each re-mint and never comes back** — once the
original tails are gone, re-baselining onto the impoverished distribution makes the loss permanent
(they can no longer be grounded back). Green (healthy) and blue (gated) stay low; grey (baseline)
is the reference.
2. **What the gate reads.** Diversity `H` over generations, same colour key, with the gate threshold
(`H = 0.75`, dashed). The gated arm simply **refuses to re-mint while below the line**, so it never
locks in a collapsed state; the ungated collapsed arm re-mints into the floor.
### Takeaway
Re-minting a collapsed model **crystallises** the collapse — it is a one-way door. A trivial
safeguard (only re-baseline when diversity is still high) preserves recoverability; re-minting a
healthy model is harmless. **Falsifier (not triggered):** if the collapsed lineage had recovered its
original tails after re-minting, the irreversibility claim would be overstated.

View file

@ -0,0 +1,32 @@
# architectures — collapse and rescue are architecture-general
**Claim tested:** is model collapse (and its cure, grounding) a quirk of one model type, or does the
same signature appear across genuinely different neural architectures?
**Setup (Layer 1.5).** The identical generational loop is run with three different generative models —
an **exact histogram** (no neural net), an **autoregressive GRU** (recurrent), and a **causal-masked
MLP** (feed-forward) — each a distinct "inductive bias." `K = 256` modes, `n = 200`, 22 generations,
5 repeats, compared at **dry (`g = 0`)** vs **grounded (`g = 0.05`)**.
### Symbols
- **inductive bias** — the built-in assumptions of a model type (a histogram has none; a GRU and an MLP smooth differently).
- **forward-KL** distance from truth; **tail items alive** — how many rare modes still appear.
- **dry** = no grounding; **grounded** = 5% real data mixed in.
### The three panels
1. **Trajectories.** Forward-KL over generations, coloured by architecture; **solid = dry** (climbs,
collapse) vs **dashed = grounded** (held down). The dry-up / grounded-down gap appears in **every**
architecture.
2. **Stationary forward-KL (grouped bars).** For each architecture, dry (red) vs grounded (green).
Divergence **falls with grounding across all three** — histogram, GRU, MLP.
3. **Tail-item survival (grouped bars).** Same grouping. Survival **rises with grounding across all
three.** (Note the histogram's bars are tiny: with no smoothing it drops rare modes outright,
whereas the GRU/MLP keep some alive — an inductive-bias difference, not a contradiction.)
### Takeaway
The WrightFisher collapse operator and the grounding rescue are **not artefacts of one model** — they
show up in an exact counter, a recurrent net, and a feed-forward net alike. This is the
architecture-generality claim of Layer 1.5. **Falsifier (not triggered):** if the signs had appeared
only for the histogram, collapse would be a property of the idealised operator, not of trained models.
*(A VAE was also implemented but fails the generation-0 fidelity check on this task, so it is excluded
to avoid confusing underfitting with collapse — documented as a known limitation.)*

32
results/bridge/README.md Normal file
View file

@ -0,0 +1,32 @@
# bridge — the histogram model reproduces Layer-1 E2 exactly (the HARD GATE)
**Claim tested (a plumbing check, not science):** does the neural experiment harness, when run with a
*trivial* model, reproduce the Layer-1 math exactly? If not, no later neural result could be trusted.
**Setup (Layer 1.5).** Same generational loop as every neural experiment — each generation, draw the
parent's samples, optionally mix in real data, retrain, measure — but the "model" is a **histogram**:
it just counts which modes appeared and resamples them (no neural net, no smoothing). This reduces the
neural setup *exactly* back to WrightFisher drift. `K = 200` modes, `n = 200`, 200 generations,
60 repeats, grounding swept `g ∈ {0, …, 0.4}`.
### Symbols
- **mode** = one of the `K` items (Layer-1.5 word for "item"); read off each generated sequence by a zero-error oracle.
- **`g`** grounding fraction, **`g*`** its critical value, **`H`** diversity, **`H_eq`** the exact closed-form equilibrium diversity.
### The four panels
1. **Trajectories.** Diversity `H` per `g`. `g = 0` collapses; `g > 0` plateaus — the E2 picture,
now produced by the *neural runner*.
2. **Bridge = Layer 1 (the gate).** Dots = the neural histogram runner's stationary `H` vs `g`; the
black dashed curve = the exact `H_eq` closed form from Layer 1. The dots sit **on** the curve, and
the recovered critical grounding is **`g* = 0.047` (CI [0.045, 0.052])** — matching Layer-1's
0.048. This equality is what licenses every trained-model result to be read against the analytic
core.
3. **Tail survival.** Fraction of the rare tail retained vs `g` (item-count red, truth-mass purple) —
rises with grounding, deep tail lags, exactly as in E2.
4. **Per-rarity band.** Survival by rarity band vs `g`; deep bands need more grounding
(`m·p*ᵢ ≳ 1`).
### Takeaway
The harness is faithful: with a memoryless model it reproduces Layer 1 to the decimal. **This is a
gate, not a finding** — passing it is the precondition for the RNN/MLP experiments
(`collapse`, `grounding`, `architectures`, `recombination`), where the *model* is what changes.

View file

@ -0,0 +1,32 @@
# collapse — model collapse in REAL neural weights, arrested by grounding
**Claim tested:** does the collapse we proved in math actually happen in a *trained neural network*
and does a little grounding stop it?
**Setup (Layer 1.5).** The model is now a small **autoregressive GRU** (a recurrent net, ~128 hidden
units). Each generation a **fresh** GRU is trained from scratch, by ordinary next-token prediction, on
the previous GRU's own generated sequences (plus any real samples). `K = 256` modes, `n = 200`,
25 generations, grounding `g ∈ {0, 0.02, 0.05, 0.1}`, 5 repeats. A generation-0 fidelity check
confirms the GRU reproduces the truth almost perfectly (KL ≈ 0.008) before any collapse is measured.
### Symbols
- **GRU** — a small recurrent neural net that emits sequences token by token; retrained each generation on the prior generation's output.
- **forward-KL** `D(p*‖p_t)` — distance from truth; the metric that actually sees neural collapse.
- **`H`** diversity; **tail items alive** — how many rare modes still appear.
### The four panels
1. **Collapse in weights.** Forward-KL over generations, one line per `g`. The dry lineage (`g = 0`)
**climbs** (drifts from truth) toward ~2.3; grounded lineages are held lower. Collapse is real in
trained weights.
2. **`H` barely moves.** Diversity over generations sits near `H*` for all arms — the GRU's smoothing
bias keeps spurious spread alive, so **diversity `H` hides the collapse**. (This is why forward-KL,
not `H`, is the operative neural metric — see `grounding`.)
3. **Stationary divergence vs `g`.** End-state forward-KL falls as grounding rises: more real data →
closer to truth.
4. **Tail survival vs `g`.** Fraction of rare modes alive rises with grounding.
### Takeaway
The core phenomenon transfers from math to real neural nets: **a GRU trained on its own output drifts
from truth, and grounding arrests it** — the *sign* Layer-1 predicts (blueprint §3.5). Note the
honest caveat surfaced here and developed in `grounding`: **diversity `H` is the wrong ruler for a
neural net** (smoothing keeps it high even during collapse); distance-from-truth is the right one.

View file

@ -0,0 +1,38 @@
# grounding — the grounding response in real weights (and why the ruler matters)
**Claim tested:** does the E2 result — a small dose of real data (`g* ≈ 0.05`) rescues diversity —
reproduce in a trained GRU? The honest answer reframes the question.
**Setup (Layer 1.5).** Autoregressive GRU, `K = 256` modes, `n = 200`, 30 generations, grounding
swept over 9 values `g ∈ {0, 0.005, …, 0.2}`, **18 repeats** (many repeats are needed because each
lineage's fate is genuinely noisy under `n = 200` drift). The falsifier was pinned in the config
*before* running.
### Symbols
- **`g`** grounding fraction (share of real data), **`g*`** its critical value.
- **forward-KL** distance from truth (the operative neural collapse metric here).
- **tail survival** `tail_truth_mass_alive` — truth-weighted fraction of the rare tail retained. **`H`** diversity.
- **recovery fraction** — how much of the achievable forward-KL reduction a given `g` has bought (0 = dry, 1 = best observed).
### The four panels
1. **Trajectories.** Forward-KL over generations per `g`: grounding suppresses the climb.
2. **Phase boundary.** Stationary forward-KL vs `g` **falls monotonically** (dry ≈ 2.08 → `g = 0.2`
≈ 0.75); the effect is statistically significant (paired *t* up to 3.3; 89% of lineages improve at
`g = 0.2`). **The SIGN is confirmed.**
3. **Recovery curve.** Fraction of the divergence gap closed vs `g`. **Half the gap closes by a
median-recovery grounding of `g ≈ 0.04`** (CI [0.004, 0.116]) — a striking echo of Layer-1's 0.048
(black dashed) — **but full recovery needs `g ≈ 0.19`**, far more than the exact histogram: the
GRU's smoothing both caps the collapse and slows the rescue.
4. **Why forward-KL (the key methodological panel).** Normalised responses of three rulers vs `g`:
`H/H*` (flat ~0.8) and **tail survival (flat / non-monotone — dry is as high as grounded!)** both
fail to register the effect, while **forward-KL recovery** responds cleanly. A smoothing model keeps
*spurious* tail support alive, so counting surviving modes is misleading; only distance-from-truth
is honest.
### Takeaway (an honest reframing)
Two results: **(1)** the operative neural collapse metric is **forward-KL**, not `H` or tail-survival —
smoothing decouples "modes alive" from "close to truth." **(2)** The *sharp* threshold `g* ≪ 1` is a
property of the exact operator, carried quantitatively by the histogram **bridge** (`g* = 0.047`); the
trained GRU confirms grounding's **direction** and **softens** its sharpness. The pre-registered
95%-of-`H*`/tail falsifier is *not* met — but because those are the wrong rulers for a smoothing
model, not because grounding fails; the blueprint §3.5 directional claim holds robustly.

View file

@ -0,0 +1,36 @@
# recombination — "merge, don't average" holds in real neural weights (E4 in the flesh)
**Claim tested:** the E4 finding — complementary specialists can rebuild the rare tail, but only if you
*merge* them rather than average them — was proven in math. Does it survive in trained neural nets?
**Setup (Layer 1.5).** `K_T` specialist **GRUs** are each trained on a different slice of the rare
tail (slices set by the exact shared-switch construction, so teacher count `K_T`, correlation `ρ` and
retention `q` stay clean knobs and the construction-level union matches the closed form). The pupil then
recombines the teachers' *measured* distributions two ways: **mean** (naive pooling) vs
**oracle-guided max-merge** (per mode, keep the strongest teacher — a union-preserving merge). `K = 256`,
`n = 200`, `q = 0.5`, swept `K_T ∈ {1,2,3,5}` × `ρ ∈ {0, 1}`, 8 repeats.
### Symbols
- **`K_T`** number of specialist teachers; **`ρ`** how correlated their retained tails are (0 = complementary, 1 = identical clones).
- **union coverage** — tail covered by ≥1 teacher (the raw supply). **surviving coverage** — what remains after the pupil resamples.
- **mean-distill** — average/pool the teachers. **max-merge** — keep each mode's strongest teacher (union-preserving).
- **target vs trained** — coverage computed from the *assigned* distributions vs from the *trained GRU* outputs.
### The four panels
1. **Supply.** Union coverage vs `K_T`: at `ρ = 0` it climbs 0.49 → 0.96 and matches the closed form
`U(K_T, ρ, q)`; at `ρ = 1` (clones) it is flat. Diverse teachers supply more tail.
2. **Analytic teachers (`ρ = 0`).** Surviving coverage vs `K_T`: **max-merge (green) rises**
0.043 → 0.087 while **mean-distill (red) stays flat ~0.045** — the conservation law from E4,
reproduced on the assigned distributions.
3. **Trained GRU teachers (`ρ = 0`).** The same comparison on *real trained weights*: same signs —
max-merge above mean-distill — but **compressed and noisier** (the GRU's smoothing inflates the
baseline and the deep tail barely clears `n = 200` resampling). The honest inductive-bias caveat.
4. **Control (`ρ = 1`).** Identical teachers: union, max and mean are all flat — **more clones buy
nothing.** Decorrelation is what the benefit needs.
### Takeaway
"Merge, don't average" is not a mathematical artefact — in trained neural nets, a union-preserving
**max-merge realises the multi-teacher tail benefit while naive averaging conserves the collapse.**
This is load-bearing for the paper's recombination claim (and points at merging over distillation for
the deferred LLM layer). **Falsifier (not triggered):** if mean-distill had also risen with `K_T`, or
max never beat mean, the recombination lesson would have died in real weights.