# recombination — "merge, don't average" holds in real neural weights (E4 in the flesh)

**Claim tested:** the E4 finding — complementary specialists can rebuild the rare tail, but only if you
*merge* them rather than average them — was proven in math. Does it survive in trained neural nets?

**Setup (Layer 1.5).** `K_T` specialist **GRUs** are each trained on a different slice of the rare
tail (slices set by the exact shared-switch construction, so teacher count `K_T`, correlation `ρ` and
retention `q` stay clean knobs and the construction-level union matches the closed form). The pupil then
recombines the teachers' *measured* distributions two ways: **mean** (naive pooling) vs
**oracle-guided max-merge** (per mode, keep the strongest teacher — a union-preserving merge). `K = 256`,
`n = 200`, `q = 0.5`, swept `K_T ∈ {1,2,3,5}` × `ρ ∈ {0, 1}`, 8 repeats.

### Symbols
- **`K_T`** number of specialist teachers; **`ρ`** how correlated their retained tails are (0 = complementary, 1 = identical clones).
- **union coverage** — tail covered by ≥1 teacher (the raw supply). **surviving coverage** — what remains after the pupil resamples.
- **mean-distill** — average/pool the teachers. **max-merge** — keep each mode's strongest teacher (union-preserving).
- **target vs trained** — coverage computed from the *assigned* distributions vs from the *trained GRU* outputs.

### The four panels
1. **Supply.** Union coverage vs `K_T`: at `ρ = 0` it climbs 0.49 → 0.96 and matches the closed form
   `U(K_T, ρ, q)`; at `ρ = 1` (clones) it is flat. Diverse teachers supply more tail.
2. **Analytic teachers (`ρ = 0`).** Surviving coverage vs `K_T`: **max-merge (green) rises**
   0.043 → 0.087 while **mean-distill (red) stays flat ~0.045** — the conservation law from E4,
   reproduced on the assigned distributions.
3. **Trained GRU teachers (`ρ = 0`).** The same comparison on *real trained weights*: same signs —
   max-merge above mean-distill — but **compressed and noisier** (the GRU's smoothing inflates the
   baseline and the deep tail barely clears `n = 200` resampling). The honest inductive-bias caveat.
4. **Control (`ρ = 1`).** Identical teachers: union, max and mean are all flat — **more clones buy
   nothing.** Decorrelation is what the benefit needs.

### Takeaway
"Merge, don't average" is not a mathematical artefact — in trained neural nets, a union-preserving
**max-merge realises the multi-teacher tail benefit while naive averaging conserves the collapse.**
This is load-bearing for the paper's recombination claim (and points at merging over distillation for
the deferred LLM layer). **Falsifier (not triggered):** if mean-distill had also risen with `K_T`, or
max never beat mean, the recombination lesson would have died in real weights.