MachineSex/results/recombination/README.md

recombination — "merge, don't average" holds in real neural weights (E4 in the flesh)

Claim tested: the E4 finding — complementary specialists can rebuild the rare tail, but only if you merge them rather than average them — was proven in math. Does it survive in trained neural nets?

Setup (Layer 1.5). K_T specialist GRUs are each trained on a different slice of the rare tail (slices set by the exact shared-switch construction, so teacher count K_T, correlation ρ and retention q stay clean knobs and the construction-level union matches the closed form). The pupil then recombines the teachers' measured distributions two ways: mean (naive pooling) vs oracle-guided max-merge (per mode, keep the strongest teacher — a union-preserving merge). K = 256, n = 200, q = 0.5, swept K_T ∈ {1,2,3,5} × ρ ∈ {0, 1}, 8 repeats.

Symbols

• K_T number of specialist teachers; ρ how correlated their retained tails are (0 = complementary, 1 = identical clones).
• union coverage — tail covered by ≥1 teacher (the raw supply). surviving coverage — what remains after the pupil resamples.
• mean-distill — average/pool the teachers. max-merge — keep each mode's strongest teacher (union-preserving).
• target vs trained — coverage computed from the assigned distributions vs from the trained GRU outputs.

The four panels

1. Supply. Union coverage vs K_T: at ρ = 0 it climbs 0.49 → 0.96 and matches the closed form U(K_T, ρ, q); at ρ = 1 (clones) it is flat. Diverse teachers supply more tail.
2. Analytic teachers (ρ = 0). Surviving coverage vs K_T: max-merge (green) rises 0.043 → 0.087 while mean-distill (red) stays flat ~0.045 — the conservation law from E4, reproduced on the assigned distributions.
3. Trained GRU teachers (ρ = 0). The same comparison on real trained weights: same signs — max-merge above mean-distill — but compressed and noisier (the GRU's smoothing inflates the baseline and the deep tail barely clears n = 200 resampling). The honest inductive-bias caveat.
4. Control (ρ = 1). Identical teachers: union, max and mean are all flat — more clones buy nothing. Decorrelation is what the benefit needs.

Takeaway

"Merge, don't average" is not a mathematical artefact — in trained neural nets, a union-preserving max-merge realises the multi-teacher tail benefit while naive averaging conserves the collapse. This is load-bearing for the paper's recombination claim (and points at merging over distillation for the deferred LLM layer). Falsifier (not triggered): if mean-distill had also risen with K_T, or max never beat mean, the recombination lesson would have died in real weights.