Giorgio Gilestro
6f8cef1ac5
Removed from main (all preserved on the dev branch): the arXiv build and
its sources, design documents (blueprint, results summary, review responses,
essay drafts), tasks/ and CLAUDE.md, the cover letter and reference tooling,
two unused manuscript figures, and every experiment that feeds no figure or
number in the paper: the collapse null, the sexual-vs-asexual lineage, the
NK speciation variant, the 0.5B single-seed LLM prototypes, the compose and
society experiments with their calibration and pilot runs, and their
configs, runners, tests, figure scripts and PBS jobs. Their result bundles
are moved to results/_archive/ (ignored) so the parquets stay on disk.
Also: plot_llm_speciation reads the s{seed}/ layout; the mating-breadth
plot writes under its bundle name; Makefile targets reduced to the kept
experiments; REPRODUCING.md and README point to dev for the rest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
|
||
|---|---|---|
| .. | ||
| figS8_multiparent_union.pdf | ||
| figS8_multiparent_union.png | ||
| manifest.json | ||
| README.md | ||
| resolved_config.yaml | ||
E4 — Recombination supplies the rare tail; only a union-preserving merge realises it
Claim tested: if several specialist models each remember a different slice of the rare tail, can combining them reconstruct the whole tail? And does how you combine them matter?
Setup (Layer 1, pure math). K = 500 items. We build K_T teacher models that each retain the
rare tail with probability q, and we control how correlated their retained tails are with a
single knob ρ (rho): ρ = 0 = fully complementary teachers, ρ = 1 = identical teachers. Swept:
K_T ∈ {1,2,3,5}, ρ ∈ {0, 0.25, 0.5, 0.75, 1}, grounding g ∈ {0, 0.02, 0.05}, 200 repeats.
Symbols
K_Tnumber of teacher models;ρhow correlated their retained tails are (0 = diverse, 1 = clones).- union coverage — fraction of the tail covered by at least one teacher (the raw supply).
- surviving coverage — fraction that actually survives in the pupil after it retrains on the combination.
- mean-distill — pupil trained on the pooled/averaged teacher outputs. max-merge — keep, per item, the strongest teacher (a union-preserving merge, à la M2N2).
The three panels
- Supply. Union tail coverage vs
ρ, one curve perK_T; solid lines are the exact closed formU(K_T, ρ, q). More teachers and more diversity (lowerρ) supply more of the tail — and the simulation matches the formula exactly. - Realisation. Surviving coverage vs
ρ. Solid = max-merge rises with more/diverse teachers; dashed = mean-distill stays flat. Averaging dilutes each teacher's rare items back below the survival threshold — the gain is supplied but not realised. - The benefit needs the right operator (
ρ = 0). Surviving coverage vsK_Tunder both operators. Max-merge climbs with teacher count; mean-distill is flat — a conservation law: averaging's1/K_Tdilution exactly cancels the union gain.
Takeaway — "merge, don't average"
Complementary specialists contain enough to rebuild the tail, but naive multi-teacher distillation
(averaging) throws it away; you must combine them with a union-preserving merge. This is load-bearing
for the paper's recombination claim and is re-tested in real neural weights in results/recombination/.
Falsifier (not triggered): if mean-distill had also risen with K_T, or max never beat mean, the
recombination story would collapse into "just average your models."