paper (Phase 2): fold hardened E13 into both versions, citation refresh, arXiv package

Speciation section rewritten around the hardened results: alignment
modulo the full function-preserving symmetry group (answers 2606.23607
preemptively), the hybrid-fitness cliff (0.97 -> 0.03), the mu(S)/2
floor, and the pre-registered emergent converse (no isolation without
functional conflict; the merge rescues forgetting specialists) — in the
abstract, §5, §13 ledger, and the accessible version.

Citation refresh (author names verified via arXiv API): concede
First-Extinction Law (Benati 2509.20101) and quantitative-trait collapse
(Yoon 2407.17493) alongside Riis; add verifier-injection (Yi 2510.16657),
Livnat & Papadimitriou (CACM 2016) as the sex-as-computation precursor,
and the adjacent 2024-26 merge/LMC/multi-agent literature (Ainsworth,
Pari, Zhou, Cao, Sharma, Hu, Kozodoi, Li & Shen, Harris, Chen, Tanaka).

arXiv package (paper/arxiv/): md2tex.py — a small block-based
Markdown->LaTeX converter keeping the Markdown as source of truth —
main.tex, generated body.tex, 3 vector figures; builds clean under
tectonic (20 pp; pdflatex hint guarded for arXiv); ARXIV-SUBMISSION.md
carries categories, license note, and a <=1,920-char abstract. 149 tests
green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
Giorgio Gilestro 2026-09-06 12:49:52 +01:00
parent ea051a5f92
commit d6a5c5cacd
11 changed files with 671 additions and 44 deletions

View file

@ -83,9 +83,13 @@ prediction — sex has a **limit**: as two models diverge they undergo **speciat
merge-compatibility cliff (compatible → outbreeding depression → hybrid inviability) whose onset is set
by divergence *and* epistasis via **BatesonDobzhanskyMuller incompatibilities**, and whose damage
grows *super-linearly* (the OrrTurelli snowball). We introduce and model this "model speciation"
directly, and confirm it in real trained weights: after permutation alignment (Git Re-Basin), a
residual, epistasis-driven merge barrier survives that alignment provably cannot remove — reproductive
isolation, not a coordinate artefact. AI also has an advantage biology lacks: **directed sex** — unbounded parents, chosen mates,
directly, and confirm it in real trained weights: a merge barrier that survives alignment under the
*full* function-preserving symmetry group of the network (not just Git Re-Basin permutations), rising
with functional conflict while hybrid fitness falls to inviability — with an honest converse we
pre-registered and found: absent conflicting training signals, divergently-specialised lineages of
shared ancestry developed *no* isolation at any divergence tested, the merge instead *rescuing* the
forgetting specialists. Isolation must be provoked by conflict; specialisation alone did not speciate.
AI also has an advantage biology lacks: **directed sex** — unbounded parents, chosen mates,
and offspring screened before they are kept — which converts recombination from a gamble into a
reliable engine and has no biological analogue.
@ -129,14 +133,22 @@ with an overtly evolutionary vocabulary: crossover-mutation-selection over LLM p
2025), niching and "mate choice" (Sakana's M2N2 — 2025), and evolutionary search over merge recipes
(Akiba et al., *Nature Mach. Intell.* 2024/25).
We are candid about the consequence. Two things we do **not** claim. First, that collapse is
WrightFisher drift: formalised independently (Riis, 2026; Shumailov et al., 2024) and conceded here.
We are candid about the consequence. Three things we do **not** claim. First, that collapse is
WrightFisher drift: formalised independently (Riis, 2026; Shumailov et al., 2024), sharpened to a
closed-form first-extinction law whose onset coincides with collapse (Benati et al., 2025) and to a
quantitative-trait-genetics account for diffusion models (Yoon et al., ICLR 2025), and conceded here.
Second, the bare empirical facts that a merged model can beat its parents, that decorrelated parents
merge better, and that naive averaging is inferior to sign- or routing-based merges (TIES, DARE,
mixture-of-experts routing): all established. What is genuinely unoccupied — and what a geneticist is
placed to supply — is a **theory** rather than a search heuristic. Every one of the works above uses
evolution as *metaphor over an optimiser*; none imports the predictive apparatus of the evolution of
sex. Nobody has stated the **"merge, don't average" conservation law**, derived **offspring-exceed-parents
mixture-of-experts routing): all established. Third, that merge success can be *predicted at all*:
machine-learning-native predictors exist, from interpretable pairwise metrics (gradient alignment —
Zhou et al., 2026) to capacity/rate-distortion accounts of "merging collapse" (2026); what they lack,
and we supply, is the *mechanism* — when and why the failure is a coordinate artefact versus genuine
functional incompatibility, and what moves the cliff. What is genuinely unoccupied — and what a
geneticist is placed to supply — is a **theory** rather than a search heuristic. The nearest precursor
is a theory-of-computation tradition reading sex as an algorithm for *mixability* (Livnat &
Papadimitriou, 2016), pre-dating model merging and never applied to it. Every one of the works above
uses evolution as *metaphor over an optimiser*; none imports the predictive apparatus of the evolution
of sex. Nobody has stated the **"merge, don't average" conservation law**, derived **offspring-exceed-parents
as FisherMuller**, predicted **outbreeding depression on rugged task landscapes**, framed **grounding
as migrationdrift balance** with a critical fraction, or connected **reproductive isolation** to when
two models can be merged at all. An evolutionary algorithm that *finds* a super-parent is evidence for
@ -357,22 +369,49 @@ event. (Figure: `results/E12/E12.png`.)
**The real-weight confirmation.** The obvious objection to the analytic model is that its
"incompatibility" is a re-labelled loss barrier, and loss barriers between independently trained
networks are famously a *coordinate* artefact — two nets that learned the same function in a permuted
basis look incompatible until their neurons are aligned (Git Re-Basin). We therefore ran the experiment
that the objection demands, in real trained weights. Two small MLPs are forked from a shared MNIST base,
basis look incompatible until their neurons are aligned (Git Re-Basin), and recent work shows that
symmetry groups *richer* than permutations remove still more of the barrier (functionality-preserving
rescalings and rotations — Scaling LMC, 2026; neuron-identifiability approaches). We therefore ran the
experiment the objection demands, in real trained weights, aligning modulo the **full**
function-preserving unit symmetry group of the architecture (per-unit positive rescaling composed with
permutation — for a plain ReLU network, all of it). Two small MLPs are forked from a shared MNIST base,
trained, weight-averaged, and their linear-mode-connectivity error barrier is measured *before and
after* in-house Git Re-Basin permutation alignment; the after-alignment **residual** is the part of the
incompatibility that alignment provably cannot explain away. The decomposition is clean (Figure:
after* alignment; the after-alignment **residual** is the part of the incompatibility that no
re-coordination can explain away. The decomposition is clean (Figure:
`results/speciation_real/speciation_real.png`): two nets trained *from different random initialisations
on the same task* have a real naive barrier that alignment **removes ~98 % of** (residual ≈ 0.001) —
same species, different basis, exactly the canonical Re-Basin result, which also proves our aligner
works. Two nets that learned *conflicting* label maps have a large barrier that alignment **removes none
of** (residual ≈ 0.50) — genuine reproductive isolation, not a coordinate artefact, and it cannot be
dismissed as a failure to align because the very same aligner erased the same-task barrier. Sweeping the
fraction of conflicting classes traces the **isolation cliff in real weights**: the residual (after
alignment) barrier climbs monotonically from 0 to ~0.49 with task conflict — the real-weight image of
E12's analytic cliff, and the direct answer to "isn't this just a permutation artefact?" It is not: the
part that survives alignment is real speciation, and it rises with the functional conflict between the
lineages exactly as the DobzhanskyMuller frame predicts.
on the same task* have a real naive barrier that alignment removes almost entirely (residual ≈ 0.001,
and the aligned merge performs at parent level) — same species, different basis, the canonical Re-Basin
result, which also proves the aligner works. Two nets that learned *conflicting* label maps have a
large barrier of which the full symmetry group removes **essentially nothing** (0.502 → 0.497) —
genuine reproductive isolation, not a missed symmetry, and it cannot be dismissed as a failure to align
because the very same aligner erased the same-task barrier. It also carries a floor no future alignment
method can breach: models loyal to label maps that conflict on a fraction *μ* of inputs cannot both be
served by *any* single merged model, which must err at rate ≥ *μ*/2 against at least one parent
(SI proposition). Sweeping the fraction of conflicting classes traces the **isolation cliff in real
weights**, now readable directly as *hybrid fitness*: the residual barrier climbs monotonically while
the merged model's accuracy falls from 0.97 to 0.03 — E12's compatible → depression → inviability
trajectory, measured.
**And its honest converse: speciation must be provoked; it did not emerge.** A true
DobzhanskyMuller incompatibility is *emergent* — each lineage's changes harmless alone, incompatible
only in combination — whereas the conflict condition above *imposes* contradiction. So we pre-registered
the emergent test: fork two children from a shared base and let them diverge with **no conflicting
training signal anywhere** — one pair as complementary class specialists (one child trains only on
digits 04, the other only on 59), one pair with divergent input conventions (views shifted in
opposite directions) — out to divergences 6.4× the base training. The result is the second
pre-registered reading, and it sharpens the theory's scope rather than confirming its most dramatic
form: the residual barrier is **0.000 at every divergence in both conditions**, and far from failing,
the merge *rescues* the two specialists — each parent decays toward ~0.50 on the full task
(catastrophically forgetting the classes it no longer sees) while the merged model holds ~0.95
throughout, a sustained FisherMuller rescue at zero barrier. In real weights, at least in this regime
of shared ancestry and compatible tasks, **reproductive isolation requires functional conflict; it does
not arise spontaneously from divergent specialisation.** The design rule sharpens accordingly: *merge
freely across divergently-specialised lineages of shared ancestry — what speciates model populations is
conflicting conventions, not specialisation per se.* Whether long-horizon over-specialisation erodes
mergeability at language-model scale — as the empirical merging literature hints (experts trained
longer merge worse under averaging) — is exactly the next tier's question, and the theory now makes the
prediction crisp: it should depend on whether extended training induces *conflicting conventions on
shared circuitry*, not on divergence time itself.
One question remains, and the rest of the paper is largely about it: recombination combines what the
parents kept — but *who decides what each parent keeps, and which offspring are worth keeping?*
@ -641,14 +680,17 @@ shape.)
**What is borrowed, and what is ours.** We are deliberate about the ledger, because the surrounding
literature is crowded and a reader deserves to know exactly where the line falls. **Conceded as prior
art:** (a) *model collapse is genetic drift* — derived independently and cleanly (Riis, 2026; and the
WrightFisher collapse literature following Shumailov et al., 2024); (b) the empirical facts that a
merged model can *beat its parents*, that *decorrelated* parents merge better, and that *naive averaging
is inferior* to sign-reconciled or routed merges (model soups, TIES, DARE, mixture-of-experts routing);
(c) that a *population* of merging or self-improving models can climb (GENOME, M2N2, Multiagent
Finetuning, the DarwinGödel Machine); and (d) that even the *magnitude* of multi-task merge degradation
has a machine-learning-native predictive account (recent stability/scaling analyses). We claim none of
these.
art:** (a) *model collapse is genetic drift* — derived independently and cleanly (Riis, 2026; the
WrightFisher collapse literature following Shumailov et al., 2024; the closed-form first-extinction
law of Benati et al., 2025; the quantitative-trait account of Yoon et al., 2025); (b) the empirical
facts that a merged model can *beat its parents*, that *decorrelated* parents merge better, and that
*naive averaging is inferior* to sign-reconciled or routed merges (model soups, TIES, DARE,
mixture-of-experts routing); (c) that a *population* of merging or self-improving models can climb
(GENOME, M2N2, Multiagent Finetuning, the DarwinGödel Machine); (d) that merge success has
machine-learning-native *predictors* — interpretable pairwise metrics (Zhou et al., 2026),
capacity/rate-distortion accounts of merging collapse (Cao et al., 2026), and stability/scaling
analyses of multi-task degradation; and (e) that verifier-screened synthetic data can avert collapse
(Yi et al., 2025) — the statistical cousin of our grounding operator. We claim none of these.
**Ours** is the theory those results have outrun: a **population-genetics of sex** applied to model
societies, which is *generative* where the incumbents are empirical. Concretely — the **"merge, don't
@ -668,8 +710,12 @@ models are too diverged to be merged at all*. We model it explicitly (§5), pred
compatible → outbreeding-depression → inviability curve, its super-linear (snowball) onset, and its
control by epistasis rather than divergence alone — the one place the merge literature has phenomena
(Pari et al., 2024; Zhou et al., 2026) but no theory — and we confirm it in real trained weights, where
a merge barrier survives permutation alignment (Git Re-Basin) as a residual, epistasis-driven
reproductive isolation that the coordinate-artefact account cannot explain away. In one sentence: the field agrees on the disease
a merge barrier survives alignment under the *full* function-preserving symmetry group (not only
Re-Basin permutations) as a residual, functional reproductive isolation with an information-theoretic
floor — together with the pre-registered emergent converse: absent conflicting training signals,
divergently-specialised lineages of shared ancestry showed *no* isolation at any divergence tested, the
merge instead rescuing the forgetting specialists (isolation must be provoked; specialisation alone did
not speciate). In one sentence: the field agrees on the disease
and tinkers at the cure with evolutionary metaphors; we bring the evolutionary *theory*, and it makes
falsifiable predictions — a merge-compatibility cliff among them — that the metaphors do not.
@ -724,6 +770,7 @@ The operators, checked; the living society, next.
- Otto, S. P., & Lenormand, T. (2002). Resolving the paradox of sex and recombination. *Nature Reviews Genetics.*
- Kondrashov, A. S. (1993). Classification of hypotheses on the advantage of amphimixis. *Journal of Heredity.*
- Dobzhansky, T. (1936); Muller, H. J. (1942). BatesonDobzhanskyMuller incompatibilities (reproductive isolation).
- Livnat, A., & Papadimitriou, C. (2016). Sex as an algorithm: the theory of evolution under the lens of computation. *Communications of the ACM 59(11).* (The theory-of-computation precursor: recombination selects for mixability.)
*The 20252026 landscape this paper positions against:*
@ -735,5 +782,19 @@ The operators, checked; the living society, next.
- Yu, L., Yu, B., Yu, H., Huang, F., & Li, Y. (2023). Language models are super Mario: absorbing abilities from homologous models (DARE). *arXiv:2311.03099.*
- Gerstgrasser, M., et al. (2024). Is model collapse inevitable? Breaking the curse of recursion by accumulating real and synthetic data. *arXiv:2404.01413.*
- Guo, D., Wu, J., & Yiu, S. M. (2026). Model collapse as cultural evolution. *arXiv:2605.23054.*
- Benati, M., Londei, A., Lanzieri, D., & Loreto, V. (2025). First-extinction law for resampling processes. *arXiv:2509.20101.* (Collapse onset = the WrightFisher first-extinction time.)
- Yoon, Y., Hu, D., Weissburg, I., Qin, Y., & Jeong, H. (2025). Model collapse in the self-consuming chain of diffusion finetuning: a novel perspective from quantitative trait modeling. *ICLR 2025 / arXiv:2407.17493.*
- Yi, B., Liu, Q., Cheng, Y., & Xu, H. (2025). Escaping model collapse via synthetic data verification. *arXiv:2510.16657.*
- Ainsworth, S., Hayase, J., & Srinivasa, S. (2022). Git Re-Basin: merging models modulo permutation symmetries. *arXiv:2209.04836.*
- Li, T., & Shen, Z. (2026). Scaling linear mode connectivity and merging to billion-parameter pretrained transformers. *arXiv:2606.23607.* (Symmetry groups richer than permutations remove more of the barrier.)
- Sharma, E., Roy, D. M., & Dziugaite, G. K. (2024). The non-local model merging problem: permutation symmetries and variance collapse. *arXiv:2410.12766.*
- Pari, J., Jelassi, S., & Agrawal, P. (2024). Collective model intelligence requires compatible specialization. *arXiv:2411.02207.*
- Zhou, L., Zhao, B., Yu, R., & Rodolà, E. (2026). Demystifying mergeability: interpretable properties to predict model merging success. *arXiv:2601.22285.*
- Cao, Y., Ran, D., Guo, Y., Wu, M., Chen, S., et al. (2026). An empirical study and theoretical explanation on task-level model-merging collapse. *arXiv:2603.09463.*
- Hu, Y., Yao, Y., Zhang, N., Chen, H., & Deng, S. (2024). Exploring model kinship for merging large language models. *arXiv:2410.12613.*
- Kozodoi, N., Afolabi, Z., & Butler, J. (2026). Are we merging the right models? Impact of expert training duration on model merging for LLMs. *arXiv:2607.11997.*
- Harris, K. D. (2026). A mathematical theory of evolution for self-designing AIs. *arXiv:2604.05142.*
- Chen, N., Tong, Y., Yang, Y., He, Y., Zhang, X., et al. (2026). Diversity collapse in multi-agent LLM systems: structural coupling and collective failure in open-ended idea generation. *arXiv:2604.18005.*
- Tanaka, H. (2026). When is collective intelligence a lottery? Multi-agent scaling laws for memetic drift in LLMs. *arXiv:2603.24676.*
*Still to engage in a full version: the machine-learning-native theory of merge degradation with task count; tacit knowledge (Polanyi) and human capital (Becker).*
*Still to engage in a full version: tacit knowledge (Polanyi) and human capital (Becker).*