paper (v6): fold E13 real-weight speciation into the limit-of-sex section
Update the "flagged not claimed" paragraph to a reported result: after Git Re-Basin permutation alignment, independent-init barriers drop ~98% (coordinate artefact) while conflicting-task barriers lose 0% (residual 0.49, genuine reproductive isolation), and the residual traces the isolation cliff with task conflict. Also threads the real-weight confirmation through the abstract (claim v + the reproducible-models sentence) and the §13 ledger, and drops the now-done Git Re-Basin item from the "still to engage" note. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
01d87e504f
commit
b950ea37e4
1 changed files with 30 additions and 8 deletions
|
|
@ -83,12 +83,15 @@ prediction — sex has a **limit**: as two models diverge they undergo **speciat
|
|||
merge-compatibility cliff (compatible → outbreeding depression → hybrid inviability) whose onset is set
|
||||
by divergence *and* epistasis via **Bateson–Dobzhansky–Muller incompatibilities**, and whose damage
|
||||
grows *super-linearly* (the Orr–Turelli snowball). We introduce and model this "model speciation"
|
||||
directly. AI also has an advantage biology lacks: **directed sex** — unbounded parents, chosen mates,
|
||||
directly, and confirm it in real trained weights: after permutation alignment (Git Re-Basin), a
|
||||
residual, epistasis-driven merge barrier survives that alignment provably cannot remove — reproductive
|
||||
isolation, not a coordinate artefact. AI also has an advantage biology lacks: **directed sex** — unbounded parents, chosen mates,
|
||||
and offspring screened before they are kept — which converts recombination from a gamble into a
|
||||
reliable engine and has no biological analogue.
|
||||
|
||||
We support the argument with **minimal, reproducible models** — a closed-form-exact account of drift
|
||||
and grounding, the same effects in small trained networks and an MNIST image generator, and
|
||||
and grounding, the same effects in small trained networks and an MNIST image generator, a real-weight
|
||||
demonstration of the speciation cliff (a Git Re-Basin residual that survives neuron alignment), and
|
||||
evolutionary simulations of the whole society — and a first **language-model prototype**: merging
|
||||
LoRA-specialised Qwen models (to 7B on a GPU cluster) yields a generalist that beats every specialist
|
||||
parent, with the sharp headroom condition under which "merge, don't average" bites. The scope is
|
||||
|
|
@ -333,10 +336,27 @@ increasing specialisation eventually breaks merging and that one should then rou
|
|||
trained models is a coordinate artefact removable by aligning neurons (Git Re-Basin — Ainsworth et al.,
|
||||
2022). What the frame adds is the *theory* of the phenomenon they observe: its functional form, its
|
||||
super-linear (snowball) onset, and its dependence on epistasis — merge failure as a Dobzhansky–Muller
|
||||
event. The honest next step, flagged not claimed, is the real-weight confirmation: merge models at
|
||||
increasing divergence *after* permutation alignment, and show the residual, epistasis-driven
|
||||
incompatibility that alignment cannot remove — the true speciation signal, as opposed to a re-labelled
|
||||
loss barrier. (Figure: `results/E12/E12.png`.)
|
||||
event. (Figure: `results/E12/E12.png`.)
|
||||
|
||||
**The real-weight confirmation.** The obvious objection to the analytic model is that its
|
||||
"incompatibility" is a re-labelled loss barrier, and loss barriers between independently trained
|
||||
networks are famously a *coordinate* artefact — two nets that learned the same function in a permuted
|
||||
basis look incompatible until their neurons are aligned (Git Re-Basin). We therefore ran the experiment
|
||||
that the objection demands, in real trained weights. Two small MLPs are forked from a shared MNIST base,
|
||||
trained, weight-averaged, and their linear-mode-connectivity error barrier is measured *before and
|
||||
after* in-house Git Re-Basin permutation alignment; the after-alignment **residual** is the part of the
|
||||
incompatibility that alignment provably cannot explain away. The decomposition is clean (Figure:
|
||||
`results/speciation_real/speciation_real.png`): two nets trained *from different random initialisations
|
||||
on the same task* have a real naive barrier that alignment **removes ~98 % of** (residual ≈ 0.001) —
|
||||
same species, different basis, exactly the canonical Re-Basin result, which also proves our aligner
|
||||
works. Two nets that learned *conflicting* label maps have a large barrier that alignment **removes none
|
||||
of** (residual ≈ 0.50) — genuine reproductive isolation, not a coordinate artefact, and it cannot be
|
||||
dismissed as a failure to align because the very same aligner erased the same-task barrier. Sweeping the
|
||||
fraction of conflicting classes traces the **isolation cliff in real weights**: the residual (after
|
||||
alignment) barrier climbs monotonically from 0 to ~0.49 with task conflict — the real-weight image of
|
||||
E12's analytic cliff, and the direct answer to "isn't this just a permutation artefact?" It is not: the
|
||||
part that survives alignment is real speciation, and it rises with the functional conflict between the
|
||||
lineages exactly as the Dobzhansky–Muller frame predicts.
|
||||
|
||||
One question remains, and the rest of the paper is largely about it: recombination combines what the
|
||||
parents kept — but *who decides what each parent keeps, and which offspring are worth keeping?*
|
||||
|
|
@ -624,7 +644,9 @@ begins to occupy — a question nobody has framed: **model speciation**, the pop
|
|||
models are too diverged to be merged at all*. We model it explicitly (§5), predicting the
|
||||
compatible → outbreeding-depression → inviability curve, its super-linear (snowball) onset, and its
|
||||
control by epistasis rather than divergence alone — the one place the merge literature has phenomena
|
||||
(Pari et al., 2024; Zhou et al., 2026) but no theory. In one sentence: the field agrees on the disease
|
||||
(Pari et al., 2024; Zhou et al., 2026) but no theory — and we confirm it in real trained weights, where
|
||||
a merge barrier survives permutation alignment (Git Re-Basin) as a residual, epistasis-driven
|
||||
reproductive isolation that the coordinate-artefact account cannot explain away. In one sentence: the field agrees on the disease
|
||||
and tinkers at the cure with evolutionary metaphors; we bring the evolutionary *theory*, and it makes
|
||||
falsifiable predictions — a merge-compatibility cliff among them — that the metaphors do not.
|
||||
|
||||
|
|
@ -691,4 +713,4 @@ The operators, checked; the living society, next.
|
|||
- Gerstgrasser, M., et al. (2024). Is model collapse inevitable? Breaking the curse of recursion by accumulating real and synthetic data. *arXiv:2404.01413.*
|
||||
- Guo, D., Wu, J., & Yiu, S. M. (2026). Model collapse as cultural evolution. *arXiv:2605.23054.*
|
||||
|
||||
*Still to engage in a full version: reproductive-isolation/speciation for merge compatibility (Git Re-Basin and linear mode connectivity as the mechanism); the machine-learning-native theory of merge degradation with task count; tacit knowledge (Polanyi) and human capital (Becker).*
|
||||
*Still to engage in a full version: the machine-learning-native theory of merge degradation with task count; tacit knowledge (Polanyi) and human capital (Becker).*
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue