paper (v6): fold E13 real-weight speciation into the limit-of-sex section
Update the "flagged not claimed" paragraph to a reported result: after Git Re-Basin permutation alignment, independent-init barriers drop ~98% (coordinate artefact) while conflicting-task barriers lose 0% (residual 0.49, genuine reproductive isolation), and the residual traces the isolation cliff with task conflict. Also threads the real-weight confirmation through the abstract (claim v + the reproducible-models sentence) and the §13 ledger, and drops the now-done Git Re-Basin item from the "still to engage" note. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
01d87e504f
commit
b950ea37e4
1 changed files with 30 additions and 8 deletions
|
|
@ -83,12 +83,15 @@ prediction — sex has a **limit**: as two models diverge they undergo **speciat
|
||||||
merge-compatibility cliff (compatible → outbreeding depression → hybrid inviability) whose onset is set
|
merge-compatibility cliff (compatible → outbreeding depression → hybrid inviability) whose onset is set
|
||||||
by divergence *and* epistasis via **Bateson–Dobzhansky–Muller incompatibilities**, and whose damage
|
by divergence *and* epistasis via **Bateson–Dobzhansky–Muller incompatibilities**, and whose damage
|
||||||
grows *super-linearly* (the Orr–Turelli snowball). We introduce and model this "model speciation"
|
grows *super-linearly* (the Orr–Turelli snowball). We introduce and model this "model speciation"
|
||||||
directly. AI also has an advantage biology lacks: **directed sex** — unbounded parents, chosen mates,
|
directly, and confirm it in real trained weights: after permutation alignment (Git Re-Basin), a
|
||||||
|
residual, epistasis-driven merge barrier survives that alignment provably cannot remove — reproductive
|
||||||
|
isolation, not a coordinate artefact. AI also has an advantage biology lacks: **directed sex** — unbounded parents, chosen mates,
|
||||||
and offspring screened before they are kept — which converts recombination from a gamble into a
|
and offspring screened before they are kept — which converts recombination from a gamble into a
|
||||||
reliable engine and has no biological analogue.
|
reliable engine and has no biological analogue.
|
||||||
|
|
||||||
We support the argument with **minimal, reproducible models** — a closed-form-exact account of drift
|
We support the argument with **minimal, reproducible models** — a closed-form-exact account of drift
|
||||||
and grounding, the same effects in small trained networks and an MNIST image generator, and
|
and grounding, the same effects in small trained networks and an MNIST image generator, a real-weight
|
||||||
|
demonstration of the speciation cliff (a Git Re-Basin residual that survives neuron alignment), and
|
||||||
evolutionary simulations of the whole society — and a first **language-model prototype**: merging
|
evolutionary simulations of the whole society — and a first **language-model prototype**: merging
|
||||||
LoRA-specialised Qwen models (to 7B on a GPU cluster) yields a generalist that beats every specialist
|
LoRA-specialised Qwen models (to 7B on a GPU cluster) yields a generalist that beats every specialist
|
||||||
parent, with the sharp headroom condition under which "merge, don't average" bites. The scope is
|
parent, with the sharp headroom condition under which "merge, don't average" bites. The scope is
|
||||||
|
|
@ -333,10 +336,27 @@ increasing specialisation eventually breaks merging and that one should then rou
|
||||||
trained models is a coordinate artefact removable by aligning neurons (Git Re-Basin — Ainsworth et al.,
|
trained models is a coordinate artefact removable by aligning neurons (Git Re-Basin — Ainsworth et al.,
|
||||||
2022). What the frame adds is the *theory* of the phenomenon they observe: its functional form, its
|
2022). What the frame adds is the *theory* of the phenomenon they observe: its functional form, its
|
||||||
super-linear (snowball) onset, and its dependence on epistasis — merge failure as a Dobzhansky–Muller
|
super-linear (snowball) onset, and its dependence on epistasis — merge failure as a Dobzhansky–Muller
|
||||||
event. The honest next step, flagged not claimed, is the real-weight confirmation: merge models at
|
event. (Figure: `results/E12/E12.png`.)
|
||||||
increasing divergence *after* permutation alignment, and show the residual, epistasis-driven
|
|
||||||
incompatibility that alignment cannot remove — the true speciation signal, as opposed to a re-labelled
|
**The real-weight confirmation.** The obvious objection to the analytic model is that its
|
||||||
loss barrier. (Figure: `results/E12/E12.png`.)
|
"incompatibility" is a re-labelled loss barrier, and loss barriers between independently trained
|
||||||
|
networks are famously a *coordinate* artefact — two nets that learned the same function in a permuted
|
||||||
|
basis look incompatible until their neurons are aligned (Git Re-Basin). We therefore ran the experiment
|
||||||
|
that the objection demands, in real trained weights. Two small MLPs are forked from a shared MNIST base,
|
||||||
|
trained, weight-averaged, and their linear-mode-connectivity error barrier is measured *before and
|
||||||
|
after* in-house Git Re-Basin permutation alignment; the after-alignment **residual** is the part of the
|
||||||
|
incompatibility that alignment provably cannot explain away. The decomposition is clean (Figure:
|
||||||
|
`results/speciation_real/speciation_real.png`): two nets trained *from different random initialisations
|
||||||
|
on the same task* have a real naive barrier that alignment **removes ~98 % of** (residual ≈ 0.001) —
|
||||||
|
same species, different basis, exactly the canonical Re-Basin result, which also proves our aligner
|
||||||
|
works. Two nets that learned *conflicting* label maps have a large barrier that alignment **removes none
|
||||||
|
of** (residual ≈ 0.50) — genuine reproductive isolation, not a coordinate artefact, and it cannot be
|
||||||
|
dismissed as a failure to align because the very same aligner erased the same-task barrier. Sweeping the
|
||||||
|
fraction of conflicting classes traces the **isolation cliff in real weights**: the residual (after
|
||||||
|
alignment) barrier climbs monotonically from 0 to ~0.49 with task conflict — the real-weight image of
|
||||||
|
E12's analytic cliff, and the direct answer to "isn't this just a permutation artefact?" It is not: the
|
||||||
|
part that survives alignment is real speciation, and it rises with the functional conflict between the
|
||||||
|
lineages exactly as the Dobzhansky–Muller frame predicts.
|
||||||
|
|
||||||
One question remains, and the rest of the paper is largely about it: recombination combines what the
|
One question remains, and the rest of the paper is largely about it: recombination combines what the
|
||||||
parents kept — but *who decides what each parent keeps, and which offspring are worth keeping?*
|
parents kept — but *who decides what each parent keeps, and which offspring are worth keeping?*
|
||||||
|
|
@ -624,7 +644,9 @@ begins to occupy — a question nobody has framed: **model speciation**, the pop
|
||||||
models are too diverged to be merged at all*. We model it explicitly (§5), predicting the
|
models are too diverged to be merged at all*. We model it explicitly (§5), predicting the
|
||||||
compatible → outbreeding-depression → inviability curve, its super-linear (snowball) onset, and its
|
compatible → outbreeding-depression → inviability curve, its super-linear (snowball) onset, and its
|
||||||
control by epistasis rather than divergence alone — the one place the merge literature has phenomena
|
control by epistasis rather than divergence alone — the one place the merge literature has phenomena
|
||||||
(Pari et al., 2024; Zhou et al., 2026) but no theory. In one sentence: the field agrees on the disease
|
(Pari et al., 2024; Zhou et al., 2026) but no theory — and we confirm it in real trained weights, where
|
||||||
|
a merge barrier survives permutation alignment (Git Re-Basin) as a residual, epistasis-driven
|
||||||
|
reproductive isolation that the coordinate-artefact account cannot explain away. In one sentence: the field agrees on the disease
|
||||||
and tinkers at the cure with evolutionary metaphors; we bring the evolutionary *theory*, and it makes
|
and tinkers at the cure with evolutionary metaphors; we bring the evolutionary *theory*, and it makes
|
||||||
falsifiable predictions — a merge-compatibility cliff among them — that the metaphors do not.
|
falsifiable predictions — a merge-compatibility cliff among them — that the metaphors do not.
|
||||||
|
|
||||||
|
|
@ -691,4 +713,4 @@ The operators, checked; the living society, next.
|
||||||
- Gerstgrasser, M., et al. (2024). Is model collapse inevitable? Breaking the curse of recursion by accumulating real and synthetic data. *arXiv:2404.01413.*
|
- Gerstgrasser, M., et al. (2024). Is model collapse inevitable? Breaking the curse of recursion by accumulating real and synthetic data. *arXiv:2404.01413.*
|
||||||
- Guo, D., Wu, J., & Yiu, S. M. (2026). Model collapse as cultural evolution. *arXiv:2605.23054.*
|
- Guo, D., Wu, J., & Yiu, S. M. (2026). Model collapse as cultural evolution. *arXiv:2605.23054.*
|
||||||
|
|
||||||
*Still to engage in a full version: reproductive-isolation/speciation for merge compatibility (Git Re-Basin and linear mode connectivity as the mechanism); the machine-learning-native theory of merge degradation with task count; tacit knowledge (Polanyi) and human capital (Becker).*
|
*Still to engage in a full version: the machine-learning-native theory of merge degradation with task count; tacit knowledge (Polanyi) and human capital (Becker).*
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue