Paper-wide rename: the pop-gen construct is "the biological model"; "(exact)" dropped
"Model" now means an AI model everywhere; the Wright-Fisher construct is "the
biological model" throughout (19 occurrences): tier header, section title (now
"The biological model, and where trained learners depart from it"), Table 1
support column ("Exact" -> "Closed form"; "Analytic model" -> "Biological
model"), Results, Discussion, Methods ("Biological-model tier"), and all
figure captions. "Exact" survives only in technical noun phrases (exact-match
verifier, exact oracle, exact equilibrium, exact recovery); the abstract's
"exactly Wright-Fisher" is now "literally Wright-Fisher".
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
parent
f1970728cf
commit
2d8f661924
7 changed files with 67 additions and 63 deletions
|
|
@ -4,7 +4,7 @@ Artificial intelligence increasingly consists of populations of models rather th
|
|||
|
||||
\section*{Abstract}
|
||||
|
||||
AI development increasingly resembles a population process. Models are specialised, retrained on model output, and recombined by weight merging, and the practice is described in evolutionary vocabulary with little use of evolutionary theory. I treat multigenerational model populations as systems whose inheritance, diversity, and compatibility must be managed, and transfer the quantitative framework of the evolution of sex. Its starting point, that training on model output is genetic drift and model collapse its signature, is by now established from several independent directions; I develop the structure that follows from it. In a minimal inheritance model that is exactly Wright--Fisher, and measurably Wright--Fisher plus estimator bias in trained networks, I derive and test remedies. Grounding acts as immigration: a real-data fraction far below one retained most equilibrium diversity, with a per-capability observation floor that makes the rarest knowledge expensive under unstratified sampling. Refitting a child to the mean of its parents' output distributions cancels the multi-parent gain to first order in the rare-item regime; union-preserving operators realise it. Merged language-model specialists exceeded every parent in replicated experiments. Blind recombination fails on rugged task landscapes; screening candidate offspring restores the gain. The optimal mating breadth narrows as skills entangle. Finally, I introduce model speciation: a merge barrier remaining after permutation-and-rescaling alignment tracks functional conflict, isolation did not emerge from compatible specialisation, and in a controlled test pre-merge functional disagreement predicted merge damage while weight-geometry baselines showed no detectable association.
|
||||
AI development increasingly resembles a population process. Models are specialised, retrained on model output, and recombined by weight merging, and the practice is described in evolutionary vocabulary with little use of evolutionary theory. I treat multigenerational model populations as systems whose inheritance, diversity, and compatibility must be managed, and transfer the quantitative framework of the evolution of sex. Its starting point, that training on model output is genetic drift and model collapse its signature, is by now established from several independent directions; I develop the structure that follows from it. In a minimal biological model of inheritance that is literally Wright--Fisher, and measurably Wright--Fisher plus estimator bias in trained networks, I derive and test remedies. Grounding acts as immigration: a real-data fraction far below one retained most equilibrium diversity, with a per-capability observation floor that makes the rarest knowledge expensive under unstratified sampling. Refitting a child to the mean of its parents' output distributions cancels the multi-parent gain to first order in the rare-item regime; union-preserving operators realise it. Merged language-model specialists exceeded every parent in replicated experiments. Blind recombination fails on rugged task landscapes; screening candidate offspring restores the gain. The optimal mating breadth narrows as skills entangle. Finally, I introduce model speciation: a merge barrier remaining after permutation-and-rescaling alignment tracks functional conflict, isolation did not emerge from compatible specialisation, and in a controlled test pre-merge functional disagreement predicted merge damage while weight-geometry baselines showed no detectable association.
|
||||
|
||||
\medskip\hrule\medskip
|
||||
|
||||
|
|
@ -16,53 +16,53 @@ The generations are coupled through data as well as through weights. Successive
|
|||
|
||||
Training each generation of a model on the previous generation's output degrades it (\emph{model collapse}): rare capabilities vanish first, and the lineage drifts toward its own most common behaviour (21). That degradation is, mathematically, \emph{genetic drift}, the loss of rare variants that any finite population suffers when each generation is a finite sample of the last --- the same sampling accident by which rare surnames vanish from small villages and rare alleles (gene variants) drift out of island populations with no selection against them. The identification has been made repeatedly and independently: for sequential inference chains before deep learning (22), for language-model text ecosystems (23), as a closed-form first-extinction law placing collapse onset at the Wright--Fisher first-extinction time (24), and in quantitative-genetic form for self-consuming diffusion models (25). A diagnosis reached so often, from such different starting points, marks population genetics as the natural mathematics of the setting, though only as its entry point: population genetics is not, at heart, a theory of decay; it is a theory of the mechanisms that maintain and build populations despite decay (immigration, recombination, selection, population structure) and of where those mechanisms reach their limits. This paper develops that fuller structure for model populations: the arc from drift through its remedies to its limit, reproductive isolation --- the point at which diverged lineages can no longer produce working offspring, biology's boundary between species --- carried as one framework from closed forms to trained networks to language models.
|
||||
|
||||
An operator of a model population faces recurring decisions for which there is no principled guidance: how much verified real data does retraining need before a lineage decays; will combining two particular models compose their abilities or damage them; can incompatibility be detected before paying for a failed merge; and when should specialists be kept separate rather than consolidated? In practice these are settled by convention and by trial-and-error search. They are also, recognisably, machine learning's oldest problem at a new scale: \emph{continual learning}, the struggle to acquire new abilities without losing old ones (26, 27), transposed from a single network to a population whose members inherit from one another. Population genetics, I will argue, prices these decisions. Table 1 summarises the correspondences on which the argument runs. Fig. 1A maps the experimental programme built on them: the same abstractions tested at three tiers --- an exact population-genetic simulator, trained neural networks, and language models --- with the sections that follow climbing that ladder. Fig. 1B draws the shift of perspective the whole transfer rests on: the model ecosystem read not as a society in space, contemporaries exchanging messages, but as a society in time, generations coupled by inheritance, recombination, and immigration.
|
||||
An operator of a model population faces recurring decisions for which there is no principled guidance: how much verified real data does retraining need before a lineage decays; will combining two particular models compose their abilities or damage them; can incompatibility be detected before paying for a failed merge; and when should specialists be kept separate rather than consolidated? In practice these are settled by convention and by trial-and-error search. They are also, recognisably, machine learning's oldest problem at a new scale: \emph{continual learning}, the struggle to acquire new abilities without losing old ones (26, 27), transposed from a single network to a population whose members inherit from one another. Population genetics, I will argue, prices these decisions. Table 1 summarises the correspondences on which the argument runs. Fig. 1A maps the experimental programme built on them: the same abstractions tested at three tiers --- a biological model in simulation, trained neural networks, and language models --- with the sections that follow climbing that ladder. Fig. 1B draws the shift of perspective the whole transfer rests on: the model ecosystem read not as a society in space, contemporaries exchanging messages, but as a society in time, generations coupled by inheritance, recombination, and immigration.
|
||||
|
||||
\begin{figure*}[p]\centering % fig1
|
||||
\includegraphics[width=\textwidth]{figs/fig1a.pdf}\\[6pt]
|
||||
\includegraphics[width=\textwidth]{figs/fig1b.pdf}
|
||||
\caption{(A) The experimental programme. Each population-genetic abstraction (Table 1) is tested at up to three tiers, ordered left to right by increasing realism: an exact Wright--Fisher simulator over knowledge distributions (closed forms; bitwise-reproducible), trained neural networks measured against exact oracles (recurrent, feedforward, and variational-autoencoder generators on a synthetic mode universe, and a convolutional VAE on MNIST), and language models (LoRA specialists on Qwen bases at 0.5B and 7B, scored by an exact-match verifier). Colour separates the two categories: the population-genetic theory tier in blue, the two AI-model tiers in oranges. The same abstractions are carried across all three. Rows are the framework's mechanisms, each defined at the left margin; filled cells name the experiments run at each tier, and each carries, in its corner, the figure or table where that result is reported, so this figure doubles as a map of the paper. Each claim is tested at the cheapest tier that can falsify it, and a costlier tier is entered only where it adds a discriminating test rather than a replication: grounding at language-model scale is established in prior work (21, 30) and is not re-run; epistasis and the society skip the middle tier, whose distinctive value (exact oracles) does not bear on those operator-level questions; and the society at language-model scale is the integrative experiment this paper specifies but does not run --- its stated gap. (B) The conceptual basis of the transfer. A population of models is usually pictured as a society in space: contemporaries exchanging messages. The couplings this paper studies run instead between generations --- training on model output (inheritance), weight-space merging (recombination), verified real data entering each generation (immigration from reality) --- a society in time, the object population genetics was built to describe. Dots mark capabilities: the rare one (gold) is lost under single-parent inheritance, reassembled by merging complementary parents, and re-supplied by grounding.}\label{fig1}
|
||||
\caption{(A) The experimental programme. Each population-genetic abstraction (Table 1) is tested at up to three tiers, ordered left to right by increasing realism: a biological model (a Wright--Fisher simulator over knowledge distributions; closed forms, bitwise-reproducible), trained neural networks measured against exact oracles (recurrent, feedforward, and variational-autoencoder generators on a synthetic mode universe, and a convolutional VAE on MNIST), and language models (LoRA specialists on Qwen bases at 0.5B and 7B, scored by an exact-match verifier). Colour separates the two categories: the biological model in blue, the two AI-model tiers in oranges. The same abstractions are carried across all three. Rows are the framework's mechanisms, each defined at the left margin; filled cells name the experiments run at each tier, and each carries, in its corner, the figure or table where that result is reported, so this figure doubles as a map of the paper. Each claim is tested at the cheapest tier that can falsify it, and a costlier tier is entered only where it adds a discriminating test rather than a replication: grounding at language-model scale is established in prior work (21, 30) and is not re-run; epistasis and the society skip the middle tier, whose distinctive value (exact oracles) does not bear on those operator-level questions; and the society at language-model scale is the integrative experiment this paper specifies but does not run --- its stated gap. (B) The conceptual basis of the transfer. A population of models is usually pictured as a society in space: contemporaries exchanging messages. The couplings this paper studies run instead between generations --- training on model output (inheritance), weight-space merging (recombination), verified real data entering each generation (immigration from reality) --- a society in time, the object population genetics was built to describe. Dots mark capabilities: the rare one (gold) is lost under single-parent inheritance, reassembled by merging complementary parents, and re-supplied by grounding.}\label{fig1}
|
||||
\end{figure*}
|
||||
|
||||
\section*{The minimal model, and where its exactness ends}
|
||||
\section*{The biological model, and where trained learners depart from it}
|
||||
|
||||
Knowledge is modelled as a distribution \texttt{p\_t} over \texttt{K} discrete items (capabilities, facts, modes of behaviour), with a fixed true distribution \texttt{p*} whose rare tail carries the knowledge most at risk. One generation is: *draw \texttt{n} samples from the parent's distribution, optionally mix in \texttt{m} verified real samples (``grounding'', \texttt{g = m/(n+m)}), and refit the child\emph{. In this minimal inheritance model the resampling step }is* the Wright--Fisher process, population genetics' canonical model of neutral evolution, in which each new generation is a random sample of size \texttt{n} from the previous one and every statistical property of drift follows from that one step. Diversity throughout this paper is \emph{heterozygosity}, \texttt{H = 1 − Σ p\_i²}: the probability that two random draws differ (one minus a collision probability), high when many items share the mass, zero at total collapse. The identity is exploited as an engineering gate: the simulator reproduces the classical closed forms (heterozygosity decay \texttt{E[H\_t] = H\_0(1 − 1/n)\textasciicircum{}t}; the exact immigration--drift equilibrium; the closed-form multi-teacher union) to within 0.5\%, and these are standing tests in the codebase, not one-off checks.
|
||||
Knowledge is modelled as a distribution \texttt{p\_t} over \texttt{K} discrete items (capabilities, facts, modes of behaviour), with a fixed true distribution \texttt{p*} whose rare tail carries the knowledge most at risk. One generation is: *draw \texttt{n} samples from the parent's distribution, optionally mix in \texttt{m} verified real samples (``grounding'', \texttt{g = m/(n+m)}), and refit the child\emph{. This construction is the paper's }biological model\emph{, deliberately minimal, and its resampling step }is* the Wright--Fisher process, population genetics' canonical model of neutral evolution, in which each new generation is a random sample of size \texttt{n} from the previous one and every statistical property of drift follows from that one step. Diversity throughout this paper is \emph{heterozygosity}, \texttt{H = 1 − Σ p\_i²}: the probability that two random draws differ (one minus a collision probability), high when many items share the mass, zero at total collapse. The identity is exploited as an engineering gate: the simulator reproduces the classical closed forms (heterozygosity decay \texttt{E[H\_t] = H\_0(1 − 1/n)\textasciicircum{}t}; the exact immigration--drift equilibrium; the closed-form multi-teacher union) to within 0.5\%, and these are standing tests in the codebase, not one-off checks.
|
||||
|
||||
The boundary of the exactness matters, and I measured it rather than assumed it. Real training adds approximation, optimisation noise, and inductive bias, and when trained networks are fit against the exact drift null they deviate in \emph{opposite, architecture-specific} directions: a smoothing recurrent network resists collapse (keeping spurious variants alive), while a sharpening image generator accelerates it. A one-parameter \emph{learning kernel} (a smoothing knob and a sharpening knob on the refit) reproduces both. Throughout, a real learner is therefore treated as Wright--Fisher \emph{plus a signed, measurable estimator bias}, and the drift signs (rare-first loss; the grounding response) survived that bias in every architecture I tested, including a convolutional VAE retrained on its own generated digits, where the dry lineage collapses to a single blurred digit class while 10\% grounding holds all thirty modes (Fig. 2). Retraining on a single parent is \emph{asexual reproduction}, and sustained loss under it carries the defining consequence of \emph{Muller's ratchet} (28), the mechanism by which lineages that never recombine decay irreversibly --- the reason non-recombining genomes such as the Y chromosome have shed most of their ancestral genes. Once every copy of a rare capability is gone from all parents and sources, no recombination can rebuild it: each such loss is a click of the ratchet, and remedies must act while copies still survive somewhere (a consequence-level correspondence; the minimal model lacks the ratchet's recurrent-mutation driver).
|
||||
Where a real learner departs from this model matters, and I measured that boundary rather than assumed it. Real training adds approximation, optimisation noise, and inductive bias, and when trained networks are fit against the exact drift null they deviate in \emph{opposite, architecture-specific} directions: a smoothing recurrent network resists collapse (keeping spurious variants alive), while a sharpening image generator accelerates it. A one-parameter \emph{learning kernel} (a smoothing knob and a sharpening knob on the refit) reproduces both. Throughout, a real learner is therefore treated as Wright--Fisher \emph{plus a signed, measurable estimator bias}, and the drift signs (rare-first loss; the grounding response) survived that bias in every architecture I tested, including a convolutional VAE retrained on its own generated digits, where the dry lineage collapses to a single blurred digit class while 10\% grounding holds all thirty modes (Fig. 2). Retraining on a single parent is \emph{asexual reproduction}, and sustained loss under it carries the defining consequence of \emph{Muller's ratchet} (28), the mechanism by which lineages that never recombine decay irreversibly --- the reason non-recombining genomes such as the Y chromosome have shed most of their ancestral genes. Once every copy of a rare capability is gone from all parents and sources, no recombination can rebuild it: each such loss is a click of the ratchet, and remedies must act while copies still survive somewhere (a consequence-level correspondence; the biological model lacks the ratchet's recurrent-mutation driver).
|
||||
|
||||
\textbf{Table 1.} The dictionary. Each biological term is introduced in the section that develops it; each correspondence is stated with the level of support it currently has (exact = closed form in the minimal model; empirical = measured in trained systems; hypothesis = stated with a falsifier, untested or unconfirmed). The full claim-by-claim ledger with assumptions and known limits is SI Appendix, Table S1.
|
||||
\textbf{Table 1.} The dictionary. Each biological term is introduced in the section that develops it; each correspondence is stated with the level of support it currently has (closed form = derived in the biological model and verified against simulation; empirical = measured in trained systems; hypothesis = stated with a falsifier, untested or unconfirmed). The full claim-by-claim ledger with assumptions and known limits is SI Appendix, Table S1.
|
||||
|
||||
\medskip\noindent\begin{center}\footnotesize
|
||||
\begin{tabular}{p{0.307\textwidth} p{0.307\textwidth} p{0.307\textwidth}}
|
||||
\hline
|
||||
Population genetics & Model populations & Support \\ \hline
|
||||
Genetic drift in a finite population & Training on finite samples of model output & Exact (minimal model); signs in trained nets; diagnosis conceded to prior work \\[3pt]
|
||||
Immigration from a fixed source & Grounding with verified real data & Exact equilibrium; signs in RNN/MLP/VAE/MNIST \\[3pt]
|
||||
Genetic drift in a finite population & Training on finite samples of model output & Closed form (biological model); signs in trained nets; diagnosis conceded to prior work \\[3pt]
|
||||
Immigration from a fixed source & Grounding with verified real data & Closed-form equilibrium; signs in RNN/MLP/VAE/MNIST \\[3pt]
|
||||
Muller's ratchet (asexual decay) & Irreversible arm of model collapse & Correspondence, scoped: applies to unrecoverable loss \\[3pt]
|
||||
Recombination / sexual reproduction & Model merging & Empirical at 0.5B--7B \\[3pt]
|
||||
Fisher--Muller effect & Merged specialists exceed every parent & Analytic model; replicated in LLMs \\[3pt]
|
||||
Outbreeding depression under epistasis & Merging entangled skills harms offspring & Analytic model (NK landscapes); hypothesis at LLM scale \\[3pt]
|
||||
Mating systems / population structure & Who merges with whom (breadth of the parent pool) & Analytic model; hypothesis for real populations \\[3pt]
|
||||
Fisher--Muller effect & Merged specialists exceed every parent & Biological model; replicated in LLMs \\[3pt]
|
||||
Outbreeding depression under epistasis & Merging entangled skills harms offspring & Biological model (NK landscapes); hypothesis at LLM scale \\[3pt]
|
||||
Mating systems / population structure & Who merges with whom (breadth of the parent pool) & Biological model; hypothesis for real populations \\[3pt]
|
||||
Reproductive isolation (BDM incompatibilities) & Merge failure from functional conflict & Empirical (MLP + LLM tiers, conflict-associated); emergent form not observed \\[3pt]
|
||||
Selection on a fitness function & Verifier-anchored selection (``reality that can say no'') & Analytic model (complementary with recombination and diversity in the tested society) \\[3pt]
|
||||
Selection on a fitness function & Verifier-anchored selection (``reality that can say no'') & Biological model (complementary with recombination and diversity in the tested society) \\[3pt]
|
||||
\hline\end{tabular}\end{center}\medskip
|
||||
|
||||
\section*{Results}
|
||||
|
||||
\subsection*{Grounding is immigration: cheap, with a floor}
|
||||
|
||||
In the minimal model, grounding from a fixed real source is \emph{immigration} into a drifting population (29--31). Immigration is what conservation managers prescribe when a fragmented reserve loses diversity, and its striking property there is how little is needed --- the field's rule of thumb is that one migrant per generation holds an isolated population's diversity (32). The same economy appears here: the equilibrium diversity has a closed form the simulator matches exactly. That equilibrium is \emph{smooth} in the grounding fraction (there is no phase transition in aggregate diversity), so the practical number is an operational threshold, and I define it as such: under the tested population size and Zipf source distribution, \texttt{g \(\approx\) 0.05} retained most (\(\geq\)95\%) of equilibrium diversity indefinitely, with the required fraction depending on sample size, source distribution, and the chosen retention target (dependencies in SI). Verified real data remains, on any of these definitions, cheap insurance at fractions far below one. But the same analysis yields a floor the field's average-loss framing misses: under unstratified sampling from the source, a capability of rarity \texttt{p} appears in a real-data batch of size \texttt{m} with probability \texttt{1 − e\textasciicircum{}{−m\(\cdot\)p}}, so \texttt{m\(\cdot\)p \(\approx\) 1} marks roughly a 63\% chance of one example per batch: a soft observation floor, with higher confidence priced accordingly, and with distinct consequences for continuous retention, stationary occupancy, and reintroduction after loss (immigration can restore an absent item; SI separates these). Protecting the rarest knowledge under unstratified grounding is therefore priced per item at cost \texttt{\(\propto\) 1/p}; targeted or stratified sampling changes that cost, and recombination can recover rare capabilities \emph{that are still retained across complementary parents} (next section). In trained networks the \emph{sign} of the grounding response transfers everywhere I looked, with two deviations, both traced to the estimator bias above: sharp thresholds soften, and support-counting metrics decouple from truth (forward-KL is the operative collapse metric for a smoothing learner). On real images (Fig. 2B), dry self-training collapses a convolutional VAE to one mode while \textasciitilde{}10\% grounding holds all thirty (the trained model needs roughly twice the exact-operator fraction, the measured price of the estimator bias).
|
||||
In the biological model, grounding from a fixed real source is \emph{immigration} into a drifting population (29--31). Immigration is what conservation managers prescribe when a fragmented reserve loses diversity, and its striking property there is how little is needed --- the field's rule of thumb is that one migrant per generation holds an isolated population's diversity (32). The same economy appears here: the equilibrium diversity has a closed form the simulator matches exactly. That equilibrium is \emph{smooth} in the grounding fraction (there is no phase transition in aggregate diversity), so the practical number is an operational threshold, and I define it as such: under the tested population size and Zipf source distribution, \texttt{g \(\approx\) 0.05} retained most (\(\geq\)95\%) of equilibrium diversity indefinitely, with the required fraction depending on sample size, source distribution, and the chosen retention target (dependencies in SI). Verified real data remains, on any of these definitions, cheap insurance at fractions far below one. But the same analysis yields a floor the field's average-loss framing misses: under unstratified sampling from the source, a capability of rarity \texttt{p} appears in a real-data batch of size \texttt{m} with probability \texttt{1 − e\textasciicircum{}{−m\(\cdot\)p}}, so \texttt{m\(\cdot\)p \(\approx\) 1} marks roughly a 63\% chance of one example per batch: a soft observation floor, with higher confidence priced accordingly, and with distinct consequences for continuous retention, stationary occupancy, and reintroduction after loss (immigration can restore an absent item; SI separates these). Protecting the rarest knowledge under unstratified grounding is therefore priced per item at cost \texttt{\(\propto\) 1/p}; targeted or stratified sampling changes that cost, and recombination can recover rare capabilities \emph{that are still retained across complementary parents} (next section). In trained networks the \emph{sign} of the grounding response transfers everywhere I looked, with two deviations, both traced to the estimator bias above: sharp thresholds soften, and support-counting metrics decouple from truth (forward-KL is the operative collapse metric for a smoothing learner). On real images (Fig. 2B), dry self-training collapses a convolutional VAE to one mode while \textasciitilde{}10\% grounding holds all thirty (the trained model needs roughly twice the biological model's fraction, the measured price of the estimator bias).
|
||||
|
||||
\begin{figure*}[p]\centering % fig2
|
||||
\includegraphics[width=\textwidth]{figs/fig2.pdf}
|
||||
\caption{Grounding is immigration. (A) Stationary diversity against the grounding fraction in the minimal inheritance model: simulation (points, 95\% CI) matches the exact immigration--drift equilibrium (dashed). The equilibrium is smooth in $g$; $g \approx 0.05$ marks the operational threshold retaining 95\% of source diversity in this setting (red line, bootstrap CI shaded); the hollow point at $g=0$ is a finite-time value (the true equilibrium is zero). (B) The same signs on real images: samples from a convolutional VAE retrained each generation on its own output (rows: generations 0--15 of an ungrounded lineage) collapse toward a single blurred mode; 10\% grounding holds all thirty modes (quantified in SI).}\label{fig2}
|
||||
\caption{Grounding is immigration. (A) Stationary diversity against the grounding fraction in the biological model: simulation (points, 95\% CI) matches the exact immigration--drift equilibrium (dashed). The equilibrium is smooth in $g$; $g \approx 0.05$ marks the operational threshold retaining 95\% of source diversity in this setting (red line, bootstrap CI shaded); the hollow point at $g=0$ is a finite-time value (the true equilibrium is zero). (B) The same signs on real images: samples from a convolutional VAE retrained each generation on its own output (rows: generations 0--15 of an ungrounded lineage) collapse toward a single blurred mode; 10\% grounding holds all thirty modes (quantified in SI).}\label{fig2}
|
||||
\end{figure*}
|
||||
|
||||
\subsection*{Recombination: a conservation law, its operators, and offspring that exceed every parent}
|
||||
|
||||
The largest returns from the transfer concern merging. \emph{Blending inheritance} --- offspring as the average of their parents --- is the failure mode at the root of population genetics' founding controversy: the swamping argument pressed in Jenkin's 1867 review of \emph{The Origin of Species}, that under blending a rare advantageous variant is diluted toward the common type faster than selection can multiply it (33), an objection dissolved only by Mendel's particulate inheritance, in which discrete variants pass through generations undiluted. Refitting a child model to the mean of its parents' output distributions is blending inheritance, and the proposition below is Jenkin's dilution made exact. \textbf{Proposition (blending inheritance, rare-item regime).} Let K parents independently retain a rare item (mass \texttt{p} when retained), and let the child draw \texttt{n} samples either from one parent chosen at random or from the \emph{mean of the parents' output distributions}. Expected item mass is identical under the two schemes; and in the rare-item regime \texttt{n\(\cdot\)p/K \(\ll\) 1}, where per-item survival is first-order in sampled mass, expected \emph{survival} is also identical: the 1/K dilution of averaging cancels the K-parent union gain to first order, so in this regime adding parents through the output-mean does not increase expected tail retention. Two boundaries: outside that regime, survival is a convex function of mixed mass, so the variance reduction from averaging can \emph{reduce} extinction relative to a randomly chosen single parent; the cancellation is a first-order result about rare items, not a universal impossibility; and the contrasting union operator (keep each item's strongest source, then renormalise, which itself redistributes mass and presupposes a verifier or oracle to identify the strongest source) increases expected retention with K in all regimes in the minimal model. The practically important operators, \emph{weight averaging} (a nonlinear network's weight-mean does not compute its parents' output-mean) and \emph{routing among intact specialists} (34) (different storage and inference budgets from a single child), are its empirical cousins, and the measured bridge is a \emph{headroom rule}, stated qualitatively: in language models, union-preserving operators beat the weight-average where that average falls short of attainable performance, and add nothing where it does not (easy-versus-hard contrasts at two scales; a quantitative form of the relationship is untested). On easy tasks a capable base's average is already at ceiling and refinements add nothing; on hard tasks the average dilutes a fragile specialist below even the best single parent and routing wins by a wide margin (Fig. 3A--B).
|
||||
The largest returns from the transfer concern merging. \emph{Blending inheritance} --- offspring as the average of their parents --- is the failure mode at the root of population genetics' founding controversy: the swamping argument pressed in Jenkin's 1867 review of \emph{The Origin of Species}, that under blending a rare advantageous variant is diluted toward the common type faster than selection can multiply it (33), an objection dissolved only by Mendel's particulate inheritance, in which discrete variants pass through generations undiluted. Refitting a child model to the mean of its parents' output distributions is blending inheritance, and the proposition below is Jenkin's dilution made exact. \textbf{Proposition (blending inheritance, rare-item regime).} Let K parents independently retain a rare item (mass \texttt{p} when retained), and let the child draw \texttt{n} samples either from one parent chosen at random or from the \emph{mean of the parents' output distributions}. Expected item mass is identical under the two schemes; and in the rare-item regime \texttt{n\(\cdot\)p/K \(\ll\) 1}, where per-item survival is first-order in sampled mass, expected \emph{survival} is also identical: the 1/K dilution of averaging cancels the K-parent union gain to first order, so in this regime adding parents through the output-mean does not increase expected tail retention. Two boundaries: outside that regime, survival is a convex function of mixed mass, so the variance reduction from averaging can \emph{reduce} extinction relative to a randomly chosen single parent; the cancellation is a first-order result about rare items, not a universal impossibility; and the contrasting union operator (keep each item's strongest source, then renormalise, which itself redistributes mass and presupposes a verifier or oracle to identify the strongest source) increases expected retention with K in all regimes in the biological model. The practically important operators, \emph{weight averaging} (a nonlinear network's weight-mean does not compute its parents' output-mean) and \emph{routing among intact specialists} (34) (different storage and inference budgets from a single child), are its empirical cousins, and the measured bridge is a \emph{headroom rule}, stated qualitatively: in language models, union-preserving operators beat the weight-average where that average falls short of attainable performance, and add nothing where it does not (easy-versus-hard contrasts at two scales; a quantitative form of the relationship is untested). On easy tasks a capable base's average is already at ceiling and refinements add nothing; on hard tasks the average dilutes a fragile specialist below even the best single parent and routing wins by a wide margin (Fig. 3A--B).
|
||||
|
||||
The generative payoff is the \emph{Fisher--Muller effect} (35, 36), the classical account of why sex speeds adaptation: in an asexual population, beneficial variants arising in different individuals can only compete until all but one lineage is lost, whereas recombination assembles them in one offspring, producing a \emph{genotype} (an individual's combination of variants, one at each \emph{locus}, or position) fitter than any parent. In the multi-locus model, sexual merging of decorrelated specialists climbs to the global optimum, a genotype no parent held, while the best single parent and the blended average both plateau below (Fig. 4). In real language models the signature replicates under seed replication: merges of three LoRA (37) specialists beat every parent overall (decisively at 7B: 0.87 vs 0.77), and on the sharper worst-family metric the merged models are the only ones competent everywhere, in every seed (Fig. 3A).
|
||||
The generative payoff is the \emph{Fisher--Muller effect} (35, 36), the classical account of why sex speeds adaptation: in an asexual population, beneficial variants arising in different individuals can only compete until all but one lineage is lost, whereas recombination assembles them in one offspring, producing a \emph{genotype} (an individual's combination of variants, one at each \emph{locus}, or position) fitter than any parent. In the multi-locus biological model, sexual merging of decorrelated specialists climbs to the global optimum, a genotype no parent held, while the best single parent and the blended average both plateau below (Fig. 4). In real language models the signature replicates under seed replication: merges of three LoRA (37) specialists beat every parent overall (decisively at 7B: 0.87 vs 0.77), and on the sharper worst-family metric the merged models are the only ones competent everywhere, in every seed (Fig. 3A).
|
||||
|
||||
Sex has risks and, for AI, an unfair advantage, both quantified on Kauffman's NK fitness landscapes (38), the standard model of \emph{epistasis}, biology's term for interaction between genes: the fitness contribution of a variant depends on which variants occupy the other loci, much as a component's value in an ML system depends on the components around it. Each of the landscape's \texttt{N} sites interacts with \texttt{K} others (the model's eponymous parameters), and raising that interaction count tunes the landscape from smooth and additive to rugged and many-peaked (Fig. 5). When skills are entangled, blind recombination produces offspring \emph{below} their parents, worsening with ruggedness, and the optimal recombination rate shrinks as entanglement grows. Biology knows this failure as \emph{outbreeding depression}, the reason conservation practice warns against crossing locally adapted populations: in the textbook case, an ibex herd in the Tatra Mountains restocked with animals from Turkey and Sinai produced fertile hybrids that bore their young in the coldest month of winter, and the herd died out (39). But an engineered population can do what biology cannot: recombine unbounded parents, choose complementary mates, and \emph{screen many candidate offspring against a verifier before keeping one}. This directed sex converts the outbreeding catastrophe into a reliable gain in the model (tracking or exceeding the best parent at every ruggedness) and replicates as a sign in language models: bred-and-screened merges beat the a-priori blend in every seed on headroom tasks, including one seed where the blend failed catastrophically and selection was immune (SI Appendix, Table S2). Finally, population \emph{structure} is itself a knob: sweeping the mate-pool breadth from monogamous (repeated local pairings) to promiscuous (\emph{panmixia}: any model may merge with any other) against ruggedness, wide mixing maximises the population mean while monotonically destroying diversity, and the best \emph{champion} shifts from wide breadth on smooth landscapes to intermediate breadth on rugged ones (Fig. 5C), the mating-system phenomenon known to structured-population search, mapped onto merging populations.
|
||||
|
||||
|
|
@ -73,7 +73,7 @@ Sex has risks and, for AI, an unfair advantage, both quantified on Kauffman's NK
|
|||
|
||||
\begin{figure*}[p]\centering % fig4
|
||||
\includegraphics[width=\textwidth]{figs/fig4.pdf}
|
||||
\caption{Recombination in the minimal model: blending inheritance and the Fisher--Muller effect. (A) Expected rare-capability survival in a child refit from $K$ uncorrelated parents: the output-mean (blending) stays at the single-parent level --- the first-order cancellation --- while the union operator (strongest source per item, renormalised, oracle-identified) rises with parent count. (B) Multi-locus recombination of decorrelated specialists produces offspring fitter than any parent, approaching the optimum as parents are added; the best single parent and the blended average plateau below (mean $\pm$ 95\% CI).}\label{fig4}
|
||||
\caption{Recombination in the biological model: blending inheritance and the Fisher--Muller effect. (A) Expected rare-capability survival in a child refit from $K$ uncorrelated parents: the output-mean (blending) stays at the single-parent level --- the first-order cancellation --- while the union operator (strongest source per item, renormalised, oracle-identified) rises with parent count. (B) Multi-locus recombination of decorrelated specialists produces offspring fitter than any parent, approaching the optimum as parents are added; the best single parent and the blended average plateau below (mean $\pm$ 95\% CI).}\label{fig4}
|
||||
\end{figure*}
|
||||
|
||||
\begin{figure*}[p]\centering % fig5
|
||||
|
|
@ -92,7 +92,7 @@ Composing the operators (Fig. 6) requires one definitional distinction first. In
|
|||
|
||||
\subsection*{The limit of sex: model speciation}
|
||||
|
||||
Recombination presupposes compatible parents. In biology, lineages pushed far enough apart become separate species (\emph{reproductive isolation}) through Bateson--Dobzhansky--Muller incompatibilities (41, 42): changes harmless on their own genetic background but deleterious in combination --- the mechanism behind the mule's sterility and the inviability of many between-species crosses, in which two genomes that each work perfectly cannot run in the same cell. A merged model is exactly the exposed hybrid. I built the analytic model (Fig. 7A): hybrid fitness tracks the parents while compatible, then peels off and crashes below the ancestor; the isolation cliff arrives earlier the denser the incompatibilities; and the incompatibility \emph{count} snowballs quadratically with divergence (42). Note that a super-linear count does not by itself entail a sharp performance cliff without the count-to-effect-size link, which the analytic model supplies under its assumptions and any neural test must establish separately.
|
||||
Recombination presupposes compatible parents. In biology, lineages pushed far enough apart become separate species (\emph{reproductive isolation}) through Bateson--Dobzhansky--Muller incompatibilities (41, 42): changes harmless on their own genetic background but deleterious in combination --- the mechanism behind the mule's sterility and the inviability of many between-species crosses, in which two genomes that each work perfectly cannot run in the same cell. A merged model is exactly the exposed hybrid. I built the biological model of the process (Fig. 7A): hybrid fitness tracks the parents while compatible, then peels off and crashes below the ancestor; the isolation cliff arrives earlier the denser the incompatibilities; and the incompatibility \emph{count} snowballs quadratically with divergence (42). Note that a super-linear count does not by itself entail a sharp performance cliff without the count-to-effect-size link, which the biological model supplies under its assumptions and any neural test must establish separately.
|
||||
|
||||
In trained networks, the claim must survive a known alternative: merge barriers between independently trained networks are famously \emph{coordinate artefacts}, removable by re-aligning hidden units (43); richer symmetry groups remove more (44), with known failures beyond the shared-data regime (45). I therefore aligned under the composition of permutation matching and exact per-unit rescaling (the unit symmetry group of plain ReLU MLPs, as the search space) and decomposed the barrier (Fig. 7 C and D): two networks trained from different initialisations on the \emph{same} task have a barrier that this alignment removes essentially entirely (residual \(\approx\) 0.001, the aligned merge performing at parent level): coordinate, not functional; two networks trained on \emph{conflicting} label maps have a barrier the same alignment leaves largely unchanged (0.502 \(\rightarrow\) 0.497), with the merged model functionally dead. The tested alignment removes the same-task barrier but leaves the conflict-associated barrier intact, supporting a functional-conflict interpretation without proving optimal alignment: exact recovery of a permuted-and-rescaled copy validates a special case, so the removable share is a lower bound and the residual an upper bound. Sweeping conflict traces the cliff as hybrid fitness, 0.97 \(\rightarrow\) 0.03. The conflict floor itself is information-theoretic (no single model can satisfy contradictory conventions; SI Appendix, Proposition S2), with the framework's role being the \emph{structure around it}: which divergences generate conflict, and what moves the cliff.
|
||||
|
||||
|
|
@ -100,7 +100,7 @@ The pre-registered \emph{emergent test} constrains the claim most: true BDM inco
|
|||
|
||||
\begin{figure*}[p]\centering % fig7
|
||||
\includegraphics[width=\textwidth]{figs/fig7.pdf}
|
||||
\caption{Model speciation at three tiers. (A) Analytic model: hybrid fitness tracks the parents while lineages are compatible, then falls to inviability; the denser the incompatibilities, the earlier the fall. (B) The isolation cliff: probability of hybrid inviability against divergence, by incompatibility density. (C) Trained networks: the merge error barrier between two MLPs before and after permutation-and-rescaling alignment --- the same-task/different-start barrier is a coordinate artefact (removed by alignment); the conflicting-task barrier is left essentially unchanged. (D) Sweeping the fraction of conflicting classes: the residual barrier rises while merged-model accuracy falls from 0.97 to 0.03. (E) Language models (0.5B LoRA children of a shared base): on shared ambiguous prompts each parent performs under its own convention while the merged model falls below both --- function-specific hybrid breakdown. (F) Divergence without conflict: over-training disjoint specialists from 1 to 12 epochs produces no isolation; the merged model tracks or exceeds the parents throughout.}\label{fig7}
|
||||
\caption{Model speciation at three tiers. (A) Biological model: hybrid fitness tracks the parents while lineages are compatible, then falls to inviability; the denser the incompatibilities, the earlier the fall. (B) The isolation cliff: probability of hybrid inviability against divergence, by incompatibility density. (C) Trained networks: the merge error barrier between two MLPs before and after permutation-and-rescaling alignment --- the same-task/different-start barrier is a coordinate artefact (removed by alignment); the conflicting-task barrier is left essentially unchanged. (D) Sweeping the fraction of conflicting classes: the residual barrier rises while merged-model accuracy falls from 0.97 to 0.03. (E) Language models (0.5B LoRA children of a shared base): on shared ambiguous prompts each parent performs under its own convention while the merged model falls below both --- function-specific hybrid breakdown. (F) Divergence without conflict: over-training disjoint specialists from 1 to 12 epochs produces no isolation; the merged model tracks or exceeds the parents throughout.}\label{fig7}
|
||||
\end{figure*}
|
||||
|
||||
\subsection*{A controlled predictive test: functional conflict, measured pre-merge, predicts merge damage}
|
||||
|
|
@ -115,11 +115,11 @@ Headline quantitative results, with sample sizes, uncertainty, and outcome defin
|
|||
|
||||
\textbf{Design rules.} As engineering guidance, the results reduce to rules that an operator of a model population can apply, answering the four decisions posed in the Introduction. \emph{Ground every generation} in verified reality --- a few percent retained most diversity in the tested settings --- but price the rarest capabilities individually (observation probability \texttt{1 − e\textasciicircum{}{−m\(\cdot\)p}} per batch under unstratified sampling), consider targeted sampling for the deep tail, and use recombination to recover rare capabilities still retained across complementary parents. \emph{Merge, don't blend, when there is headroom}: keep specialists intact and route, or breed-and-screen candidate merges, whenever the naive average is far from ceiling; plain averaging is adequate only where a strong base has already composed the skills. \emph{Match the operator to entanglement}: merge freely when skills are additive; sparingly, with offspring selection, when they entangle; and expect the champion-optimal mating breadth to narrow as landscapes roughen. \emph{Preserve diversity as a first-class objective}, because selection can only preserve variety that exists, and in the tested society its removal produced a distinct failure mode. \emph{Before merging, measure functional conflict} --- cheap, pre-merge, and in the controlled setting predictive where the tested weight-distance baselines were not; and \emph{do not treat divergence or specialisation alone as evidence of incompatibility} --- in every regime tested here, what broke merging was conflicting conventions on shared circuitry, which is the thing to detect.
|
||||
|
||||
\textbf{Continual learning at the population scale.} Within a single network, the discipline's remedies for forgetting are this framework's operators writ small. Rehearsal and replay of stored data (26, 27) is grounded inheritance within one lineage, and the replay fractions the field settled on empirically, on the order of 1\% for instruction tuning (48) and 5\% to 25\% by distribution-shift strength in continual pretraining (49), sit where the minimal model's operational threshold lies. \emph{Pseudo-rehearsal}, the replay of a network's own generated samples, proposed as a cure in 1995 (50) and revived as generative replay (51), is precisely the ungrounded null studied here: immigration from a drifting source, benign for one hop, compounding over generations, with verifier-filtering (29, 52) converting it back into grounding. Parameter isolation (53), including frozen-base adapters, which forget far less (54), is engineered decorrelation; complementary-learning-systems consolidation (55--57) is the periodic adapter-into-base merge; the recent turn to merging as a continual-learning mechanism (58--61) applies recombination within one lineage over time, where this paper applies it across lineages; and the observation that rare examples and long-tail knowledge are forgotten first (62--64) is tail extinction seen one model at a time. The mechanisms differ (forgetting is largely deterministic interference, collapse is sampling drift) but the victims and the remedies coincide, and to my knowledge no prior work carries population-genetic formalism into continual learning. Read into that field, the results offer: (i) an equilibrium theory for the replay ratio, with the sharper prediction that the required fraction is set by the rarest capability one refuses to lose (the \texttt{1 − e\textasciicircum{}{−m\(\cdot\)p}} law) rather than by average loss, testable against published replay sweeps; (ii) a \emph{failure theory for generative replay}: self-generated rehearsal is safe for short horizons and compounds into collapse across generations unless verifier-filtered back into grounding (29, 50--52); (iii) \emph{pre-merge interference prediction with a mechanism}: where the current state of the art fits regressions over candidate metrics (47), the functional-conflict measure arrives at a convergent signal from principle and comes with an operator prescription --- when conflict is high, do not average; route or breed-and-screen; (iv) a candidate \emph{decision rule for the consolidate-versus-stay-modular question} that currently splits the field's practice (keep adapters separate vs merge them; 57--61): union-preserving operators where headroom exists, fusion where the base composes, consolidation as the slow-store step; and (v) \emph{tail monitoring as the leading indicator}: continual-learning evaluation that averages over capabilities hides exactly the losses that drift theory says come first and, past a threshold, become irreversible. On that last point I note the standing objection that apparent forgetting can be skewed task-inference over latent capability rather than erasure (65); the irreversibility results here concern oracle-measured behavioural distributions, and distinguishing latent from extinct capability at language-model scale is an open experiment whose outcome would be decisive for both readings.
|
||||
\textbf{Continual learning at the population scale.} Within a single network, the discipline's remedies for forgetting are this framework's operators writ small. Rehearsal and replay of stored data (26, 27) is grounded inheritance within one lineage, and the replay fractions the field settled on empirically, on the order of 1\% for instruction tuning (48) and 5\% to 25\% by distribution-shift strength in continual pretraining (49), sit where the biological model's operational threshold lies. \emph{Pseudo-rehearsal}, the replay of a network's own generated samples, proposed as a cure in 1995 (50) and revived as generative replay (51), is precisely the ungrounded null studied here: immigration from a drifting source, benign for one hop, compounding over generations, with verifier-filtering (29, 52) converting it back into grounding. Parameter isolation (53), including frozen-base adapters, which forget far less (54), is engineered decorrelation; complementary-learning-systems consolidation (55--57) is the periodic adapter-into-base merge; the recent turn to merging as a continual-learning mechanism (58--61) applies recombination within one lineage over time, where this paper applies it across lineages; and the observation that rare examples and long-tail knowledge are forgotten first (62--64) is tail extinction seen one model at a time. The mechanisms differ (forgetting is largely deterministic interference, collapse is sampling drift) but the victims and the remedies coincide, and to my knowledge no prior work carries population-genetic formalism into continual learning. Read into that field, the results offer: (i) an equilibrium theory for the replay ratio, with the sharper prediction that the required fraction is set by the rarest capability one refuses to lose (the \texttt{1 − e\textasciicircum{}{−m\(\cdot\)p}} law) rather than by average loss, testable against published replay sweeps; (ii) a \emph{failure theory for generative replay}: self-generated rehearsal is safe for short horizons and compounds into collapse across generations unless verifier-filtered back into grounding (29, 50--52); (iii) \emph{pre-merge interference prediction with a mechanism}: where the current state of the art fits regressions over candidate metrics (47), the functional-conflict measure arrives at a convergent signal from principle and comes with an operator prescription --- when conflict is high, do not average; route or breed-and-screen; (iv) a candidate \emph{decision rule for the consolidate-versus-stay-modular question} that currently splits the field's practice (keep adapters separate vs merge them; 57--61): union-preserving operators where headroom exists, fusion where the base composes, consolidation as the slow-store step; and (v) \emph{tail monitoring as the leading indicator}: continual-learning evaluation that averages over capabilities hides exactly the losses that drift theory says come first and, past a threshold, become irreversible. On that last point I note the standing objection that apparent forgetting can be skewed task-inference over latent capability rather than erasure (65); the irreversibility results here concern oracle-measured behavioural distributions, and distinguishing latent from extinct capability at language-model scale is an open experiment whose outcome would be decisive for both readings.
|
||||
|
||||
\textbf{What is borrowed and what is new.} The collapse-as-drift diagnosis is established prior work (21--25); so are the empirical facts that merges can beat parents, that decorrelated parents merge better, and that naive averaging loses to interference-aware or routed merges (4, 66, 67), that model populations can climb (5, 8--10), and that merge success admits ML-native predictors (47, 68), correlational where this framework supplies mechanism; the reading of sex as an algorithm for mixability in the theory of computation (69) anticipated the transfer before model merging existed. New here is the framework-level synthesis --- inheritance, diversity, and compatibility as managed quantities --- together with: the conservation law for blending inheritance and its operator boundaries; the per-item grounding floor; the society ablation with its complementary failure modes; model speciation as a named, tested question, with the coordinate-versus-functional decomposition under permutation-and-rescaling alignment and the emergent null that bounds it; and the controlled predictive test with its controls. I claim the framework generated these measurements and experiments; I do not claim that their outcomes validate a uniquely population-genetic mechanism, and one refinement it proposed was not supported.
|
||||
|
||||
\textbf{Limits and open problems.} The demonstrations are deliberately small: exact where small is a virtue, sign-level and seed-replicated at the language-model tier, on constructed task families with a trivially separable router and one model lineage (Qwen, 0.5B--7B). The composed society has not been built at language-model scale. The predictive test's next bars, in order of value: generalisation to \emph{unfamiliar} conflict structures and real task pairs; a demonstrably better \emph{budget-matched} merging decision; then scale replication. Beyond engineering, the framework's hardest open problem is the fitness function itself: selection optimises what is measured, and for knowledge systems the persuasive and the true compete --- grounding against a reality that can refuse is the only anchor I trust, and institutionalising that anchor (verification, replication, and challenge among models) is the society-level problem this paper poses but does not solve. What biology receives in return is a new model system: populations of learners where every genotype, environment, and mating decision is observable and manipulable --- where the evolution of sex can be studied with interventions (unbounded parents, offspring preview, directed mating) that no living system permits.
|
||||
\textbf{Limits and open problems.} The demonstrations are deliberately small: closed-form where small is a virtue, sign-level and seed-replicated at the language-model tier, on constructed task families with a trivially separable router and one model lineage (Qwen, 0.5B--7B). The composed society has not been built at language-model scale. The predictive test's next bars, in order of value: generalisation to \emph{unfamiliar} conflict structures and real task pairs; a demonstrably better \emph{budget-matched} merging decision; then scale replication. Beyond engineering, the framework's hardest open problem is the fitness function itself: selection optimises what is measured, and for knowledge systems the persuasive and the true compete --- grounding against a reality that can refuse is the only anchor I trust, and institutionalising that anchor (verification, replication, and challenge among models) is the society-level problem this paper poses but does not solve. What biology receives in return is a new model system: populations of learners where every genotype, environment, and mating decision is observable and manipulable --- where the evolution of sex can be studied with interventions (unbounded parents, offspring preview, directed mating) that no living system permits.
|
||||
|
||||
\textbf{Creative diversity.} Collapse is not confined to facts and skills. Homogenisation of \emph{style} is already measurable: models trained on model output lose lexical and syntactic diversity across generations (70), writing produced with model assistance is individually better but collectively less diverse than writing produced without it (71, 72), and the house styles of the large assistants are recognisable enough that their tics serve as signatures. In this framework these are the same phenomenon at a different locus. A voice is a distribution over rare stylistic variants, exactly the tail that drift erases first and that blending inheritance averages into a common register. The remedies transfer unchanged, though they are untested here: grounding on stylistically diverse human sources, decorrelated lineages maintained as distinct voices rather than merged into one, union-preserving recombination over blending, and selection that rewards being different as well as being good. Whether these preserve measured stylistic diversity at scale is an open experiment that the framework specifies.
|
||||
|
||||
|
|
@ -127,9 +127,9 @@ Headline quantitative results, with sample sizes, uncertainty, and outcome defin
|
|||
|
||||
\section*{Materials and Methods}
|
||||
|
||||
\textbf{Analytic tier.} Pure NumPy/SciPy Wright--Fisher simulator over \texttt{K}-item distributions (knowledge as \texttt{p\_t}; Zipf-tailed truth \texttt{p*}; drift--grounding--refit generations), extended with a learning kernel (smoothing/sharpening refit), multi-locus genotypes on additive and Kauffman NK landscapes, n-parent crossover, and finite-population society loops. All parameters live in per-experiment YAML configs; every run derives all randomness from one master seed (\texttt{SeedSequence.spawn}) and is bitwise reproducible; scientific-validation tests assert the closed forms (heterozygosity decay, immigration equilibrium, closed-form union) to <0.5\% and run in CI with 151 further correctness tests.
|
||||
\textbf{Biological-model tier.} Pure NumPy/SciPy Wright--Fisher simulator over \texttt{K}-item distributions (knowledge as \texttt{p\_t}; Zipf-tailed truth \texttt{p*}; drift--grounding--refit generations), extended with a learning kernel (smoothing/sharpening refit), multi-locus genotypes on additive and Kauffman NK landscapes, n-parent crossover, and finite-population society loops. All parameters live in per-experiment YAML configs; every run derives all randomness from one master seed (\texttt{SeedSequence.spawn}) and is bitwise reproducible; scientific-validation tests assert the closed forms (heterozygosity decay, immigration equilibrium, closed-form union) to <0.5\% and run in CI with 151 further correctness tests.
|
||||
|
||||
\textbf{Neural tier.} Trained-network experiments realise the same abstractions with an exact oracle: histogram/RNN/MLP/VAE generators on a synthetic mode universe (the histogram model reduces the harness exactly to the analytic tier --- the bridge gate), and a convolutional VAE on MNIST with a frozen CNN oracle (98.5\% mode accuracy; confusion matrix recorded as the measurement floor). Speciation experiments fork no-BatchNorm MLPs (784--512--512--10) from a shared base, weight-average, and measure linear-mode-connectivity error barriers before and after alignment; alignment composes deterministic Git Re-Basin permutation matching with exact per-unit scale canonicalisation (the unit symmetry group of this class, as the alignment search space; control recovery does not establish global optimality), gated by exact recovery of a permuted-and-rescaled copy.
|
||||
\textbf{Neural tier.} Trained-network experiments realise the same abstractions with an exact oracle: histogram/RNN/MLP/VAE generators on a synthetic mode universe (the histogram model reduces the harness exactly to the biological model --- the bridge gate), and a convolutional VAE on MNIST with a frozen CNN oracle (98.5\% mode accuracy; confusion matrix recorded as the measurement floor). Speciation experiments fork no-BatchNorm MLPs (784--512--512--10) from a shared base, weight-average, and measure linear-mode-connectivity error barriers before and after alignment; alignment composes deterministic Git Re-Basin permutation matching with exact per-unit scale canonicalisation (the unit symmetry group of this class, as the alignment search space; control recovery does not establish global optimality), gated by exact recovery of a permuted-and-rescaled copy.
|
||||
|
||||
\textbf{Language-model tier.} LoRA specialists (rank 16) on procedurally generated task families with an exact-match verifier, on frozen Qwen2.5-Instruct bases (0.5B on one 16 GB GPU; 7B on one L40S). Operators: weight-space merges (soup/TIES via adapter arithmetic), per-input routing, and Dirichlet-sampled offspring populations screened on held-out validation splits. Multi-seed protocols fix the test sets and vary the training seed. The predictive test computes all predictors pre-merge (generation confidence from token log-probabilities; base-model gradient cosines; exact r-space LoRA-delta geometry) and evaluates merges on held-out tests; robust statistics (condition-clustered bootstrap, paired predictor contrasts, leave-one-condition-out prediction, multi-reference outcomes) are produced by a committed script. Statistical, per-seed reproducibility is documented for GPU tiers.
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue