third review round: mathematical corrections + operator separation + headline calibration
The five priority fixes, in the PNAS draft and propagated to the
long-form document and results documentation:
1. The averaging proposition now proves what it claims: a FIRST-ORDER
cancellation of the multi-parent retention gain under output-mean
inheritance in the rare-item regime (n·p/K << 1), with the convexity
boundary stated (averaging's variance reduction can reduce extinction
outside that regime — the reviewer's argument) and the union
operator's renormalisation + oracle requirement explicit. "Adding
parents cannot help" deleted everywhere.
2. Grounding: g*~=0.05 restated as an operational threshold (equilibrium
smooth in g — no phase transition); m·p floor restated as
1−exp(−m·p) per-batch observation probability with
retention/occupancy/reintroduction distinguished; the deep-tail rule
de-categoricalised (stratified sampling; recombination recovers only
what parents retain).
3. Grounded INHERITANCE (data channel) separated from grounded
EVALUATION (fitness channel) in the society section; retitled to
"complementary contributions"; general joint necessity disclaimed.
Table 1 + v6 ledger updated.
4. Alignment contradiction removed everywhere ("cannot be an alignment
failure" -> the reviewer's formulation); abstract says "remaining
after permutation-and-rescaling alignment"; group = search space,
control recovery != global optimality; "specialisation is merge-safe"
-> "do not treat divergence/specialisation alone as evidence of
incompatibility".
5. Significance headline matched to the bounded evidence; seed-
dependence sensitivity added (per-seed rho stable +0.37..+0.53 for
functional measures, ~0 for geometry, gradient alignment
seed-UNSTABLE −0.11..−0.55 — reported as its own caveat; LOSO ranges
in stats script).
Presentation: review-process meta-language stripped; "exact" reserved
for closed forms ("analytic model" labels); headroom rule qualitative;
directed-sex phrasing per review; ratchet = consequence-level
correspondence; compact results table (Table 2) added. Response letter:
paper/response-to-review-3.md. Both PDFs rebuilt; 151 tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
parent
bf4b1c077c
commit
6b5591c92f
9 changed files with 322 additions and 138 deletions
|
|
@ -23,7 +23,7 @@ AI is turning from single frozen models to \textbf{populations of agents} that p
|
|||
|
||||
We take one diagnosis as settled and cite it as such: training each generation on the last is \textbf{genetic drift}, and the resulting \textbf{model collapse} is the loss of rare variants a finite population always suffers (the Wright--Fisher process; formalised for language models by Shumailov et al., 2024, and Riis, 2026). We claim none of that. Our contribution is on the remedy side. Single- teacher copying is \textbf{asexual} reproduction, and the irreversible arm of its decay corresponds to \textbf{Muller's ratchet} (a correspondence we state with its scope, not as identity); the remedy biology found for the ratchet is \textbf{sex}. A society of models should reproduce sexually --- each new model \textbf{recombined from several complementary parents} (which the field already does, as \emph{model merging}), selection \textbf{anchored to a reality that can say no} (not to the consensus of other models), and diversity actively \textbf{preserved}. In our models --- from closed-form to trained networks to a language-model prototype --- those three ingredients together let a lineage not merely avoid collapse but \textbf{climb}, producing models fitter than any ancestor (the \textbf{Fisher--Muller effect}) while each specialty is re-earned and exceeded; whether the full recipe holds at frontier scale is the open question the framework is built to test.
|
||||
|
||||
From the geneticist's apparatus we extract falsifiable, load-bearing claims (each stated with its operator and scope in the text): (i) \textbf{``merge, don't average''} --- a conservation result: refitting a child to the \emph{mean of its parents' output distributions} conserves rare-capability mass at the single-parent level, so adding parents cannot help, while union-preserving operators realise the gain --- exact in the minimal model, with its weight-space image verified as the headroom rule below; (ii) \textbf{offspring can exceed every parent} (Fisher--Muller), the real argument for sex in model societies; (iii) on \textbf{rugged, epistatic} task landscapes, blind recombination causes \textbf{outbreeding depression}, yielding a design rule --- \emph{merge freely when skills are additive, sparingly and with selection when entangled, and route rather than blend under overlap}; (iv) \textbf{grounding is immigration} from a non-drifting reality, giving a critical real-data fraction far below one; and (v) --- the sharpest new prediction --- sex has a \textbf{limit}: as two models diverge they undergo \textbf{speciation}, a merge-compatibility cliff (compatible \(\rightarrow\) outbreeding depression \(\rightarrow\) hybrid inviability) whose onset is set by divergence \emph{and} epistasis via \textbf{Bateson--Dobzhansky--Muller incompatibilities}, and whose damage grows \emph{super-linearly} (the Orr--Turelli snowball). We introduce and model this ``model speciation'' directly, and confirm it in real trained weights: a merge barrier that survives alignment under the \emph{full} function-preserving symmetry group of the network (not just Git Re-Basin permutations), rising with functional conflict while hybrid fitness falls to inviability --- with an honest converse we pre-registered and found: absent conflicting training signals, divergently-specialised lineages of shared ancestry developed \emph{no} isolation at any divergence tested, the merge instead \emph{rescuing} the forgetting specialists. Isolation must be provoked by conflict; specialisation alone did not speciate. AI also has an advantage biology lacks: \textbf{directed sex} --- unbounded parents, chosen mates, and offspring screened before they are kept --- which converts recombination from a gamble into a reliable engine and has no biological analogue.
|
||||
From the geneticist's apparatus we extract falsifiable, load-bearing claims (each stated with its operator and scope in the text): (i) \textbf{``merge, don't average''} --- a conservation result: refitting a child to the \emph{mean of its parents' output distributions} conserves expected rare-capability mass at the single-parent level, cancelling the multi-parent gain \emph{to first order in the rare-item regime} (outside it, variance reduction from averaging can help --- the result is a first-order cancellation, not a universal impossibility), while union-preserving operators realise the gain in all regimes --- derived in the minimal model, with its weight-space image the headroom rule below; (ii) \textbf{offspring can exceed every parent} (Fisher--Muller), the real argument for sex in model societies; (iii) on \textbf{rugged, epistatic} task landscapes, blind recombination causes \textbf{outbreeding depression}, yielding a design rule --- \emph{merge freely when skills are additive, sparingly and with selection when entangled, and route rather than blend under overlap}; (iv) \textbf{grounding is immigration} from a non-drifting reality, giving a critical real-data fraction far below one; and (v) --- the sharpest new prediction --- sex has a \textbf{limit}: as two models diverge they undergo \textbf{speciation}, a merge-compatibility cliff (compatible \(\rightarrow\) outbreeding depression \(\rightarrow\) hybrid inviability) whose onset is set by divergence \emph{and} epistasis via \textbf{Bateson--Dobzhansky--Muller incompatibilities}, and whose damage grows \emph{super-linearly} (the Orr--Turelli snowball). We introduce and model this ``model speciation'' directly, and confirm it in real trained weights: a merge barrier that survives alignment under the \emph{full} function-preserving symmetry group of the network (not just Git Re-Basin permutations), rising with functional conflict while hybrid fitness falls to inviability --- with an honest converse we pre-registered and found: absent conflicting training signals, divergently-specialised lineages of shared ancestry developed \emph{no} isolation at any divergence tested, the merge instead \emph{rescuing} the forgetting specialists. Isolation must be provoked by conflict; specialisation alone did not speciate. AI also has an advantage biology lacks: \textbf{directed sex} --- unbounded parents, chosen mates, and offspring screened before they are kept --- engineered recombination with a flexibility of parent choice and pre-deployment screening that natural mating systems do not approach.
|
||||
|
||||
We support the argument with \textbf{minimal, reproducible models} --- a closed-form-exact account of drift and grounding, the same effects in small trained networks and an MNIST image generator, a real-weight demonstration of the speciation cliff (a Git Re-Basin residual that survives neuron alignment), and evolutionary simulations of the whole society --- and a first \textbf{language-model prototype}: merging LoRA-specialised Qwen models (to 7B on a GPU cluster) yields a generalist that beats every specialist parent, with the sharp headroom condition under which ``merge, don't average'' bites. The scope is honest: these are existence proofs and design rules; the \emph{whole grounded society} on a large language model is the open step. We position the work carefully against the crowded 2025--2026 landscape of evolutionary-AI and merging methods --- conceding what they own and marking, precisely, what a genuine population-genetics of sex adds.
|
||||
|
||||
|
|
@ -39,7 +39,7 @@ The unit that matters is therefore the \textbf{generation}, and the event that m
|
|||
|
||||
This axis is suddenly crowded. By 2026 several groups build \textbf{populations of models or agents that improve across generations}: societies of independently-specialised models that self-improve for more rounds than a single agent (Multiagent Finetuning --- Subramaniam et al., 2025); open-ended archives of self-rewriting coding agents (the Darwin--Gödel Machine --- Zhang et al., 2025); groups that evolve by sharing experience across branches (Weng et al., 2026); persistent agent \emph{ecologies} with reproduction and cumulative culture (TerraLingua --- 2026). In parallel, \textbf{model merging} has become a small industry with an overtly evolutionary vocabulary: crossover-mutation-selection over LLM populations (GENOME --- 2025), niching and ``mate choice'' (Sakana's M2N2 --- 2025), and evolutionary search over merge recipes (Akiba et al., \emph{Nature Mach. Intell.} 2024/25).
|
||||
|
||||
We are candid about the consequence. Three things we do \textbf{not} claim. First, that collapse is Wright--Fisher drift: formalised independently (Riis, 2026; Shumailov et al., 2024), sharpened to a closed-form first-extinction law whose onset coincides with collapse (Benati et al., 2025) and to a quantitative-trait-genetics account for diffusion models (Yoon et al., ICLR 2025), and conceded here. Second, the bare empirical facts that a merged model can beat its parents, that decorrelated parents merge better, and that naive averaging is inferior to sign- or routing-based merges (TIES, DARE, mixture-of-experts routing): all established. Third, that merge success can be \emph{predicted at all}: machine-learning-native predictors exist, from interpretable pairwise metrics (gradient alignment --- Zhou et al., 2026) to capacity/rate-distortion accounts of ``merging collapse'' (2026); what they lack, and we supply, is the \emph{mechanism} --- when and why the failure is a coordinate artefact versus genuine functional incompatibility, and what moves the cliff. What a geneticist is placed to supply is a \textbf{framework} rather than a search heuristic. The nearest precursor is a theory-of-computation tradition reading sex as an algorithm for \emph{mixability} (Livnat \& Papadimitriou, 2016), pre-dating model merging; the works above use evolution chiefly as vocabulary over an optimiser, and --- to our knowledge --- the quantitative apparatus of the evolution of sex (Fisher--Muller, outbreeding depression, migration--drift balance, reproductive isolation) has not previously been carried over as more than metaphor. We are also candid about what \emph{kind} of contribution each of our claims is, because three different things are easily conflated: \textbf{interpretation} (an existing result is usefully understood in these terms --- e.g., merged offspring beating their parents as Fisher--Muller), \textbf{explanation} (the transferred mechanism accounts for observations existing accounts leave open --- e.g., which merge failures are coordinate artefacts and which are functional), and \textbf{prediction} (the framework forecasts an unmeasured outcome and improves a design decision --- e.g., an epistasis measure taken \emph{before} merging that beats geometry-based predictors of merge success). This paper is strongest on the first, makes concrete progress on the second, and states the third as its open, decisive test --- proposed here with pre-registered falsifiers, not claimed as done. The organising shift we argue for is prior to any single mechanism: \textbf{treat multigenerational model populations as systems whose inheritance, diversity, and compatibility must be managed --- not merely as collections of models to optimise.}
|
||||
We are candid about the consequence. Three things we do \textbf{not} claim. First, that collapse is Wright--Fisher drift: formalised independently (Riis, 2026; Shumailov et al., 2024), sharpened to a closed-form first-extinction law whose onset coincides with collapse (Benati et al., 2025) and to a quantitative-trait-genetics account for diffusion models (Yoon et al., ICLR 2025), and conceded here. Second, the bare empirical facts that a merged model can beat its parents, that decorrelated parents merge better, and that naive averaging is inferior to sign- or routing-based merges (TIES, DARE, mixture-of-experts routing): all established. Third, that merge success can be \emph{predicted at all}: machine-learning-native predictors exist, from interpretable pairwise metrics (gradient alignment --- Zhou et al., 2026) to capacity/rate-distortion accounts of ``merging collapse'' (2026); what they lack, and we supply, is the \emph{mechanism} --- when and why the failure is a coordinate artefact versus genuine functional incompatibility, and what moves the cliff. What a geneticist is placed to supply is a \textbf{framework} rather than a search heuristic. The nearest precursor is a theory-of-computation tradition reading sex as an algorithm for \emph{mixability} (Livnat \& Papadimitriou, 2016), pre-dating model merging; the works above use evolution chiefly as vocabulary over an optimiser, and --- to our knowledge --- the quantitative apparatus of the evolution of sex (Fisher--Muller, outbreeding depression, migration--drift balance, reproductive isolation) has not previously been carried over as more than metaphor. We are also candid about what \emph{kind} of contribution each of our claims is, because three different things are easily conflated: \textbf{interpretation} (an existing result is usefully understood in these terms --- e.g., merged offspring beating their parents as Fisher--Muller), \textbf{explanation} (the transferred mechanism accounts for observations existing accounts leave open --- e.g., which merge failures are coordinate artefacts and which are functional), and \textbf{prediction} (the framework forecasts an unmeasured outcome and improves a design decision). This paper is strongest on the first, makes concrete progress on the second, and reports a first, bounded step on the third: a \textbf{controlled predictive test} at small scale in which pre-merge \emph{functional-disagreement} measures --- chosen by the framework --- showed a detectable, held-out-robust association with merge damage on a constructed task grid, while the selected weight-geometry baselines did not. We are precise about that result's boundary where it is reported: it is a small-model demonstration on a constructed grid; the proposed epistasis-specific refinement did not outperform plain disagreement; predictor differences are not individually significant head-to-head; and whether the prediction improves a budget-matched operator choice remains open. The organising shift we argue for is prior to any single mechanism: \textbf{treat multigenerational model populations as systems whose inheritance, diversity, and compatibility must be managed --- not merely as collections of models to optimise.}
|
||||
|
||||
\section*{2. Why today's models cannot do this}
|
||||
|
||||
|
|
@ -87,7 +87,7 @@ Geneticists call this the \textbf{Fisher--Muller effect} (Fisher, 1930; Muller,
|
|||
|
||||
This is no longer only a simulation. In a first language-model prototype --- LoRA specialists on disjoint task families, recombined and judged by an exact verifier --- a merge of three specialist Qwen models (7B, on a GPU cluster) \textbf{beats every single specialist}, overall and on every family: the Fisher--Muller effect, in real weights. The same prototype pins down \emph{when} the finer ``inherit the union, don't average'' rule actually bites. Keeping each parent whole and \textbf{routing} each input to the right one beats the tail-thinning average --- but only when the task is hard enough to leave room to lose: on easy tasks a strong model's plain average is already at the ceiling, so the crude soup is fine, whereas on hard tasks the average dilutes a hard-won specialist so badly it falls below even the best single parent, and routing wins by a wide margin. The rule is therefore precise: \textbf{the union beats the average in exact proportion to how far the average is from the best attainable} --- a caveat that sharpens rather than weakens the claim, and that a practitioner needs before spending compute on the fancier operator.
|
||||
|
||||
\textbf{The operator boundaries (stated, because ``merge, don't average'' is not one claim but a family).} Four different operators travel under these words, and the conservation result belongs to exactly one of them. What is \emph{derived} is this: when a pupil's knowledge is refit to the \textbf{mean of the parents' output distributions}, the expected mass on any rare item is conserved at the single-parent level --- in the rare-item regime the 1/K dilution of averaging exactly cancels the union gain of having K parents --- so adding parents cannot help; whereas an operator that keeps, per item, its \textbf{strongest source} realises the union. That statement is exact in the minimal model, and it presupposes an oracle (or verifier) able to say which source is strongest. The two operators the LLM prototype tests --- \textbf{weight averaging} (a nonlinear network's weight-mean does not compute the mean of its parents' outputs) and \textbf{routing among intact specialists} (which keeps K models' storage and an input classifier, a different parameter and inference budget from one fixed-size child) --- are \emph{empirical cousins} of the two sides of that law, not instances of it. The headroom rule above is precisely the empirical bridge: it says when the weight-average behaves like the diluting mean (hard tasks, weak base) and when a capable base absorbs the dilution (easy tasks). And all of it operates within a capacity boundary: when parental capabilities genuinely cannot coexist in the child's capacity, no operator preserves the union --- that regime is the subject of the speciation section below.
|
||||
\textbf{The operator boundaries (stated, because ``merge, don't average'' is not one claim but a family).} Four different operators travel under these words, and the conservation result belongs to exactly one of them. What is \emph{derived} is this: when a pupil's knowledge is refit to the \textbf{mean of the parents' output distributions}, the expected mass on any rare item is conserved at the single-parent level --- in the rare-item regime (\texttt{n\(\cdot\)p/K ≪ 1}) the 1/K dilution of averaging cancels the union gain of having K parents to first order --- outside that regime, survival is convex in mixed mass and averaging's variance reduction can help, so this is a first-order cancellation, not a universal impossibility; whereas an operator that keeps, per item, its \textbf{strongest source} (and renormalises, which itself redistributes mass) realises the union in all regimes. That statement is exact in the minimal model, and it presupposes an oracle (or verifier) able to say which source is strongest. The two operators the LLM prototype tests --- \textbf{weight averaging} (a nonlinear network's weight-mean does not compute the mean of its parents' outputs) and \textbf{routing among intact specialists} (which keeps K models' storage and an input classifier, a different parameter and inference budget from one fixed-size child) --- are \emph{empirical cousins} of the two sides of that law, not instances of it. The headroom rule above is precisely the empirical bridge: it says when the weight-average behaves like the diluting mean (hard tasks, weak base) and when a capable base absorbs the dilution (easy tasks). And all of it operates within a capacity boundary: when parental capabilities genuinely cannot coexist in the child's capacity, no operator preserves the union --- that regime is the subject of the speciation section below.
|
||||
|
||||
Three results keep this honest, and all are results, not hand-waving.
|
||||
|
||||
|
|
@ -125,7 +125,7 @@ This is where a geneticist's lens earns its keep. The machine-learning literatur
|
|||
\end{figure*}
|
||||
|
||||
|
||||
\textbf{The real-weight confirmation.} The obvious objection to the analytic model is that its ``incompatibility'' is a re-labelled loss barrier, and loss barriers between independently trained networks are famously a \emph{coordinate} artefact --- two nets that learned the same function in a permuted basis look incompatible until their neurons are aligned (Git Re-Basin), and recent work shows that symmetry groups \emph{richer} than permutations remove still more of the barrier (functionality-preserving rescalings and rotations --- Scaling LMC, 2026; neuron-identifiability approaches). We therefore ran the experiment the objection demands, in real trained weights, aligning modulo the \textbf{full} function-preserving unit symmetry group of the architecture (per-unit positive rescaling composed with permutation --- for a plain ReLU network, all of it). Two small MLPs are forked from a shared MNIST base, trained, weight-averaged, and their linear-mode-connectivity error barrier is measured \emph{before and after} alignment; the after-alignment \textbf{residual} is the part of the incompatibility that no re-coordination can explain away. The decomposition is clean : two nets trained \emph{from different random initialisations on the same task} have a real naive barrier that alignment removes almost entirely (residual \(\approx\) 0.001, and the aligned merge performs at parent level) --- same species, different basis, the canonical Re-Basin result, which also proves the aligner works. Two nets that learned \emph{conflicting} label maps have a large barrier of which the full symmetry group removes \textbf{essentially nothing} (0.502 \(\rightarrow\) 0.497) --- genuine reproductive isolation, not a missed symmetry, and it cannot be dismissed as a failure to align because the very same aligner erased the same-task barrier. It also carries a floor no future alignment method can breach: models loyal to label maps that conflict on a fraction \emph{\(\mu\)} of inputs cannot both be served by \emph{any} single merged model, which must err at rate \(\geq\) \emph{\(\mu\)}/2 against at least one parent (SI proposition). Sweeping the fraction of conflicting classes traces the \textbf{isolation cliff in real weights}, now readable directly as \emph{hybrid fitness}: the residual barrier climbs monotonically while the merged model's accuracy falls from 0.97 to 0.03 --- E12's compatible \(\rightarrow\) depression \(\rightarrow\) inviability trajectory, measured.
|
||||
\textbf{The real-weight confirmation.} The obvious objection to the analytic model is that its ``incompatibility'' is a re-labelled loss barrier, and loss barriers between independently trained networks are famously a \emph{coordinate} artefact --- two nets that learned the same function in a permuted basis look incompatible until their neurons are aligned (Git Re-Basin), and recent work shows that symmetry groups \emph{richer} than permutations remove still more of the barrier (functionality-preserving rescalings and rotations --- Scaling LMC, 2026; neuron-identifiability approaches). We therefore ran the experiment the objection demands, in real trained weights, aligning modulo the \textbf{full} function-preserving unit symmetry group of the architecture (per-unit positive rescaling composed with permutation --- for a plain ReLU network, all of it). Two small MLPs are forked from a shared MNIST base, trained, weight-averaged, and their linear-mode-connectivity error barrier is measured \emph{before and after} alignment; the after-alignment \textbf{residual} is the part of the incompatibility that no re-coordination can explain away. The decomposition is clean : two nets trained \emph{from different random initialisations on the same task} have a real naive barrier that alignment removes almost entirely (residual \(\approx\) 0.001, and the aligned merge performs at parent level) --- same species, different basis, the canonical Re-Basin result, which also proves the aligner works. Two nets that learned \emph{conflicting} label maps have a large barrier of which the full symmetry group removes \textbf{essentially nothing} (0.502 \(\rightarrow\) 0.497) --- a conflict-associated barrier the tested alignment leaves largely unchanged --- supporting a functional-conflict interpretation without proving optimal alignment (control recovery validates a special case; the removable share is a lower bound, the residual an upper bound). It also carries a floor no future alignment method can breach: models loyal to label maps that conflict on a fraction \emph{\(\mu\)} of inputs cannot both be served by \emph{any} single merged model, which must err at rate \(\geq\) \emph{\(\mu\)}/2 against at least one parent (SI proposition). Sweeping the fraction of conflicting classes traces the \textbf{isolation cliff in real weights}, now readable directly as \emph{hybrid fitness}: the residual barrier climbs monotonically while the merged model's accuracy falls from 0.97 to 0.03 --- E12's compatible \(\rightarrow\) depression \(\rightarrow\) inviability trajectory, measured.
|
||||
|
||||
\begin{figure*}[t]\centering
|
||||
\includegraphics[width=\textwidth]{figs/speciation_real.pdf}
|
||||
|
|
@ -135,7 +135,7 @@ This is where a geneticist's lens earns its keep. The machine-learning literatur
|
|||
|
||||
\textbf{And its honest converse: speciation must be provoked; it did not emerge.} A true Dobzhansky--Muller incompatibility is \emph{emergent} --- each lineage's changes harmless alone, incompatible only in combination --- whereas the conflict condition above \emph{imposes} contradiction. So we pre-registered the emergent test: fork two children from a shared base and let them diverge with \textbf{no conflicting training signal anywhere} --- one pair as complementary class specialists (one child trains only on digits 0--4, the other only on 5--9), one pair with divergent input conventions (views shifted in opposite directions) --- out to divergences 6.4\(\times\) the base training. The result is the second pre-registered reading, and it sharpens the theory's scope rather than confirming its most dramatic form: the residual barrier is \textbf{0.000 at every divergence in both conditions}, and far from failing, the merge \emph{rescues} the two specialists --- each parent decays toward \textasciitilde{}0.50 on the full task (catastrophically forgetting the classes it no longer sees) while the merged model holds \textasciitilde{}0.95 throughout, a sustained Fisher--Muller rescue at zero barrier. In real weights, at least in this regime of shared ancestry and compatible tasks, \textbf{reproductive isolation requires functional conflict; it does not arise spontaneously from divergent specialisation.} The design rule sharpens accordingly: \emph{merge freely across divergently-specialised lineages of shared ancestry --- what speciates model populations is conflicting conventions, not specialisation per se.} Whether long-horizon over-specialisation erodes mergeability at language-model scale --- as the empirical merging literature hints (experts trained longer merge worse under averaging) --- is exactly the next tier's question, and the theory now makes the prediction crisp: it should depend on whether extended training induces \emph{conflicting conventions on shared circuitry}, not on divergence time itself.
|
||||
|
||||
\textbf{What these experiments do and do not establish.} Stated at exactly the strength of the evidence: they establish that \emph{some merge failures reflect incompatible functional requirements rather than a mismatch of coordinates} --- a residual that survives the full unit-symmetry group of the architecture tested, rises with functional conflict, and is absent under compatible specialisation. Three qualifiers. First, the impossibility at the heart of the conflict condition --- one deterministic model cannot satisfy two contradictory answer conventions --- is information-theoretic and needs no population genetics; what the genetic frame adds is \emph{structure around it}: which divergences generate such conflicts, the prediction that epistasis rather than distance sets the cliff's position, and the snowball's super-linear onset --- the latter two verified so far only in the analytic model, and therefore carried as \textbf{hypotheses at the neural tier, not results}. Second, our alignment removes the symmetries we enumerate for this architecture class; richer transformation families for other architectures could reapportion removable vs residual, though not below the conflict floor. Third, ``unmergeable'' here means by aligned linear interpolation of weights --- a barrier to that operator does not preclude every conceivable recombination method (routing, for one, sidesteps it by not blending). Emergent Dobzhansky--Muller incompatibilities in real weights remain the flagship \emph{hypothesis} of this programme: our tested regimes found none, which bounds where they can live --- longer horizons, shifted data distributions, capacity pressure --- and the decisive experiment (predicting merge success \emph{before} merging from an operational epistasis measure, against geometry- and gradient-based predictors) is posed in the closing section.
|
||||
\textbf{What these experiments do and do not establish.} Stated at exactly the strength of the evidence: they establish that \emph{some merge failures reflect incompatible functional requirements rather than a mismatch of coordinates} --- a residual that survives the full unit-symmetry group of the architecture tested, rises with functional conflict, and is absent under compatible specialisation. Three qualifiers. First, the impossibility at the heart of the conflict condition --- one deterministic model cannot satisfy two contradictory answer conventions --- is information-theoretic and needs no population genetics; what the genetic frame adds is \emph{structure around it}: which divergences generate such conflicts, the prediction that epistasis rather than distance sets the cliff's position, and the snowball's super-linear onset --- the latter two verified so far only in the analytic model, and therefore carried as \textbf{hypotheses at the neural tier, not results}. (On the snowball, one more distinction: super-linear growth in the \emph{number} of incompatibilities does not by itself entail a sharp \emph{performance} cliff --- that needs the link from incompatibility count through effect sizes to measured performance, which the analytic model supplies under its assumptions and any neural test must establish separately.) Second, our alignment removes the symmetries we enumerate for this architecture class, and exactly recovering a permuted-and-rescaled copy validates a special case rather than proving global optimality for independently trained networks --- so the removable share is a lower bound and the residual an upper bound; richer transformation families for other architectures could reapportion the split, though not below the conflict floor. Third, ``unmergeable'' here means by aligned linear interpolation of weights --- a barrier to that operator does not preclude every conceivable recombination method (routing, for one, sidesteps it by not blending). Emergent Dobzhansky--Muller incompatibilities in real weights remain the flagship \emph{hypothesis} of this programme: our tested regimes found none, which bounds where they can live --- longer horizons, shifted data distributions, capacity pressure --- and the decisive experiment (predicting merge success \emph{before} merging from an operational epistasis measure, against geometry- and gradient-based predictors) is posed in the closing section.
|
||||
|
||||
One question remains, and the rest of the paper is largely about it: recombination combines what the parents kept --- but \emph{who decides what each parent keeps, and which offspring are worth keeping?}
|
||||
|
||||
|
|
@ -206,7 +206,7 @@ Three honest riders, because re-minting is the most consequential step in the sc
|
|||
\item \textbf{Speciation.} A re-minting is a founder event. Different laboratories, re-basing on different criteria, will mint divergent bases; the lineage branches. This is not a defect but \emph{adaptive radiation}, and it is exactly what open weights make possible. The society grows not as one heavy trunk but as a branching tree of bases.
|
||||
\end{itemize}
|
||||
|
||||
So the answer to ``can it grow forever?'' is \textbf{yes --- but only because it forgets and consolidates at every level, including the base.} Nothing is retained without bound anywhere; unbounded growth of \emph{capability} is bought by \emph{bounded} storage plus periodic consolidation.
|
||||
So the honest answer to ``can it grow forever?'' is: *\emph{the architecture removes the }storage\emph{ obstacle to indefinite accumulation}* --- nothing is retained without bound anywhere, and consolidation resets the soft budget each epoch --- but that is a statement about bookkeeping, not a demonstration of unbounded capability growth, which no fixed-capacity system can promise and our finite models (deliberately scoped as ``effectively open-ended relative to the sample size, not astronomically open-ended'') do not test. What the design claims is the weaker, defensible thing: at no level does a full store force the lineage to stop learning.
|
||||
|
||||
\section*{12. One process, four timescales}
|
||||
|
||||
|
|
@ -255,14 +255,17 @@ Offspring exceed every parent (Fisher--Muller) & Interpretation + empirical & Co
|
|||
Outbreeding depression on rugged landscapes; operator design rule & Exact-model result; hypothesis at LLM scale & NK epistasis stands in for skill entanglement & E9--E10; directed selection rescues & Not yet mapped onto a real task-entanglement measure \\[3pt]
|
||||
Optimal mate-pool breadth shrinks with ruggedness & Exact-model result; hypothesis for merging populations & Ring population, local selection & E14 & Phenomenon known to island-model evolutionary computation; our contribution is the mapping and the diversity/mean decomposition \\[3pt]
|
||||
Merge failure decomposes into coordinate artefact + functional residual & Empirical (MLP tier; LLM tier in progress) & Alignment enumerates the architecture's unit symmetries & Full-symmetry residual \(\approx\) 0 (compatible) vs \(\approx\) naive (conflict); cliff in hybrid fitness & Scoped to aligned linear interpolation; conflict floor is information-theoretic, not genetic \\[3pt]
|
||||
Epistasis (not divergence) sets the cliff; snowball onset & Exact-model result; \textbf{hypothesis} at the neural tier & BDM incompatibility structure & E12 & The decisive pre-merge prediction test is proposed, not run \\[3pt]
|
||||
Epistasis (not divergence) sets the cliff; snowball onset & Exact-model result; \textbf{hypothesis} at the neural tier & BDM incompatibility structure & E12 & Snowball count ≠ performance cliff without the effect-size link; neural test outstanding \\[3pt]
|
||||
Pre-merge functional disagreement predicts merge penalty & Empirical, within a controlled grid (0.5B, 13 conditions \(\times\) 3 seeds) & Constructed conflict/overlap/duration axes; oracle-potential outcome (pre-registered; ordering sensitive to reference) & Clustered CIs exclude 0; held-out LOCO ρ\(\approx\)0.4; selected geometry baselines \(\approx\) 0 & Head-to-head predictor differences not individually significant; only selected baselines; generalisation to real task pairs open \\[3pt]
|
||||
Confidence weighting improves rank prediction over raw disagreement & \textbf{Not supported} (pre-registered internal prediction) & --- & Paired Δ\textbackslash{} & ρ\textbackslash{} \\[3pt]
|
||||
The predictor improves budget-matched operator choice & \textbf{Open} & --- & Soup-vs-route gap readout noise-dominated at 0.5B & The practical payoff; untested \\[3pt]
|
||||
Emergent speciation without conflict & \textbf{Not observed} (pre-registered) & Shared ancestry, compatible tasks, tested divergences & E13b: residual 0.000; merge rescues specialists & Bounds the hypothesis; longer horizons/distribution shift/capacity pressure untested \\[3pt]
|
||||
Grounding + sex + diversity jointly necessary & Exact-model result; hypothesis at LLM scale & Conformity stands in for self-consumption & E11 four-arm ablation, each arm failing distinctly & The full grounded LLM society is unbuilt \\[3pt]
|
||||
Grounding + sex + diversity complementary (each ablation fails distinctly) & Analytic-model result; hypothesis at LLM scale & Conformity stands in for self-consumption; general joint necessity not established & E11 four-arm ablation & The full grounded LLM society is unbuilt; alternative schemes untested \\[3pt]
|
||||
\hline\end{tabular}\end{center}\medskip
|
||||
|
||||
\textbf{What is borrowed, and what is ours.} We are deliberate about the ledger, because the surrounding literature is crowded and a reader deserves to know exactly where the line falls. \textbf{Conceded as prior art:} (a) \emph{model collapse is genetic drift} --- derived independently and cleanly (Riis, 2026; the Wright--Fisher collapse literature following Shumailov et al., 2024; the closed-form first-extinction law of Benati et al., 2025; the quantitative-trait account of Yoon et al., 2025); (b) the empirical facts that a merged model can \emph{beat its parents}, that \emph{decorrelated} parents merge better, and that \emph{naive averaging is inferior} to sign-reconciled or routed merges (model soups, TIES, DARE, mixture-of-experts routing); (c) that a \emph{population} of merging or self-improving models can climb (GENOME, M2N2, Multiagent Finetuning, the Darwin--Gödel Machine); (d) that merge success has machine-learning-native \emph{predictors} --- interpretable pairwise metrics (Zhou et al., 2026), capacity/rate-distortion accounts of merging collapse (Cao et al., 2026), and stability/scaling analyses of multi-task degradation; and (e) that verifier-screened synthetic data can avert collapse (Yi et al., 2025) --- the statistical cousin of our grounding operator. We claim none of these.
|
||||
|
||||
\textbf{Ours} is the framework those results invite: a \textbf{population-genetics of sex} applied to model societies, generative where the incumbents are empirical. Concretely --- the \textbf{``merge, don't average'' conservation law} (recombination preserves the union; blending inheritance cancels it), derived not observed; \textbf{Fisher--Muller} named and used to explain \emph{why} offspring exceed parents; \textbf{outbreeding depression on rugged/epistatic landscapes}, which turns ``when does merging help vs hurt'' from a thing you must run a search to discover into a thing the landscape's ruggedness \emph{predicts}, with the operator-choice design rule that follows (average / union-route / directed-select); \textbf{grounding as migration--drift balance}, giving a critical real-data fraction and a phase boundary a closed self-consuming loop cannot have; \textbf{directed sex} as the distinctly-AI advantage (unbounded parents, offspring preview, mate choice); and the \textbf{integrated society} whose four operators are shown \emph{jointly necessary}. The value-add over the machine-learning-native merge theory is that ours predicts \emph{which operator to use and when it will backfire}, not merely how fast quality decays. And it opens --- and begins to occupy --- a question nobody has framed: \textbf{model speciation}, the population-genetics of \emph{reproductive isolation} (Bateson--Dobzhansky--Muller incompatibilities) as the account of \emph{when two models are too diverged to be merged at all}. We model it explicitly (§5), predicting the compatible \(\rightarrow\) outbreeding-depression \(\rightarrow\) inviability curve, its super-linear (snowball) onset, and its control by epistasis rather than divergence alone --- the one place the merge literature has phenomena (Pari et al., 2024; Zhou et al., 2026) but no theory --- and we confirm it in real trained weights, where a merge barrier survives alignment under the \emph{full} function-preserving symmetry group (not only Re-Basin permutations) as a residual, functional reproductive isolation with an information-theoretic floor --- together with the pre-registered emergent converse: absent conflicting training signals, divergently-specialised lineages of shared ancestry showed \emph{no} isolation at any divergence tested, the merge instead rescuing the forgetting specialists (isolation must be provoked; specialisation alone did not speciate). In one sentence: the field agrees on the disease and tinkers at the cure with evolutionary metaphors; we bring the evolutionary \emph{theory}, and it makes falsifiable predictions --- a merge-compatibility cliff among them --- that the metaphors do not.
|
||||
\textbf{Ours} is the framework those results invite: a \textbf{population-genetics of sex} applied to model societies, generative where the incumbents are empirical. Concretely --- the \textbf{``merge, don't average'' conservation law} (recombination preserves the union; blending inheritance cancels it), derived not observed; \textbf{Fisher--Muller} named and used to explain \emph{why} offspring exceed parents; \textbf{outbreeding depression on rugged/epistatic landscapes}, which turns ``when does merging help vs hurt'' from a thing you must run a search to discover into a thing the landscape's ruggedness \emph{predicts}, with the operator-choice design rule that follows (average / union-route / directed-select); \textbf{grounding as migration--drift balance}, giving a critical real-data fraction and a phase boundary a closed self-consuming loop cannot have; \textbf{directed sex} as the distinctly-AI advantage (unbounded parents, offspring preview, mate choice); and the \textbf{integrated society} whose operators make \emph{complementary, distinctly-failing contributions} in the tested model (general joint necessity is not established). The value-add over the machine-learning-native merge theory is that ours predicts \emph{which operator to use and when it will backfire}, not merely how fast quality decays. And it opens --- and begins to occupy --- a question nobody has framed: \textbf{model speciation}, the population-genetics of \emph{reproductive isolation} (Bateson--Dobzhansky--Muller incompatibilities) as the account of \emph{when two models are too diverged to be merged at all}. We model it explicitly (§5), predicting the compatible \(\rightarrow\) outbreeding-depression \(\rightarrow\) inviability curve, its super-linear (snowball) onset, and its control by epistasis rather than divergence alone --- the one place the merge literature has phenomena (Pari et al., 2024; Zhou et al., 2026) but no theory --- and we confirm it in real trained weights, where a merge barrier survives alignment under the \emph{full} function-preserving symmetry group (not only Re-Basin permutations) as a residual, functional reproductive isolation with an information-theoretic floor --- together with the pre-registered emergent converse: absent conflicting training signals, divergently-specialised lineages of shared ancestry showed \emph{no} isolation at any divergence tested, the merge instead rescuing the forgetting specialists (isolation must be provoked; specialisation alone did not speciate). In one sentence: the field agrees on the disease and tinkers at the cure with evolutionary metaphors; we bring the evolutionary \emph{theory}, and it makes falsifiable predictions --- a merge-compatibility cliff among them --- that the metaphors do not.
|
||||
|
||||
\textbf{What is still open --- honestly.} The old hole (what to select) we fill in kind: don't design the selector, evolve it. But the hole has \emph{moved}, not closed, and the new one is harder: \textbf{the fitness function} --- what reality-anchored measure selects for \emph{truth} without also selecting for \emph{persuasion}, given that in our own species the two have been at war for the whole history of ideas. Alongside it: the \textbf{institutions} that let contemporaries correct one another before error is inherited (§8), which we do not solve; and the \textbf{calibration} of everything the results left as knobs --- how many parents, how complementary, at what ratio of inherited-to-real data, and how healthy a lineage must be before its knowledge is safe to make irreversibly innate. These are, at least, \emph{measurable} --- which is the difference between an open problem and a hole. And the largest gap of all: the \emph{recombination} claims now hold in real language models, but the \emph{society} --- the grounded, diversity-preserving, continually reproducing loop --- does not yet. The real test is to build that whole system out of actual open-weight language models, and see whether all the signs survive contact with a system too big to write down. The operators, checked; the living society, next.
|
||||
|
||||
|
|
|
|||
Binary file not shown.
Loading…
Add table
Add a link
Reference in a new issue