Fig. 1: rename first tier "Exact model" -> "Population genetics (exact)"
"Model" is reserved for AI models in this paper, so a tier named "Exact
model" read as a kind of AI model; the tier is the Wright-Fisher theory run
exactly. Caption and Introduction adjusted to match ("three tiers" rather
than "three tiers of model architecture"; "an exact population-genetic
simulator").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
parent
e42fe497ff
commit
5cb49943db
6 changed files with 5 additions and 5 deletions
|
|
@ -16,11 +16,11 @@ The generations are coupled through data as well as through weights. Successive
|
|||
|
||||
Training each generation of a model on the previous generation's output degrades it (\emph{model collapse}): rare capabilities vanish first, and the lineage drifts toward its own most common behaviour (21). That degradation is, mathematically, \emph{genetic drift}, the loss of rare variants that any finite population suffers when each generation is a finite sample of the last --- the same sampling accident by which rare surnames vanish from small villages and rare alleles (gene variants) drift out of island populations with no selection against them. The identification has been made repeatedly and independently: for sequential inference chains before deep learning (22), for language-model text ecosystems (23), as a closed-form first-extinction law placing collapse onset at the Wright--Fisher first-extinction time (24), and in quantitative-genetic form for self-consuming diffusion models (25). A diagnosis reached so often, from such different starting points, marks population genetics as the natural mathematics of the setting, though only as its entry point: population genetics is not, at heart, a theory of decay; it is a theory of the mechanisms that maintain and build populations despite decay (immigration, recombination, selection, population structure) and of where those mechanisms reach their limits. This paper develops that fuller structure for model populations: the arc from drift through its remedies to its limit, reproductive isolation --- the point at which diverged lineages can no longer produce working offspring, biology's boundary between species --- carried as one framework from closed forms to trained networks to language models.
|
||||
|
||||
An operator of a model population faces recurring decisions for which there is no principled guidance: how much verified real data does retraining need before a lineage decays; will combining two particular models compose their abilities or damage them; can incompatibility be detected before paying for a failed merge; and when should specialists be kept separate rather than consolidated? In practice these are settled by convention and by trial-and-error search. They are also, recognisably, machine learning's oldest problem at a new scale: \emph{continual learning}, the struggle to acquire new abilities without losing old ones (26, 27), transposed from a single network to a population whose members inherit from one another. Population genetics, I will argue, prices these decisions. Table 1 summarises the correspondences on which the argument runs, and Fig. 1 maps the experimental programme built on them: the same abstractions tested at three tiers of model architecture --- an exact simulator, trained neural networks, and language models --- with the sections that follow climbing that ladder.
|
||||
An operator of a model population faces recurring decisions for which there is no principled guidance: how much verified real data does retraining need before a lineage decays; will combining two particular models compose their abilities or damage them; can incompatibility be detected before paying for a failed merge; and when should specialists be kept separate rather than consolidated? In practice these are settled by convention and by trial-and-error search. They are also, recognisably, machine learning's oldest problem at a new scale: \emph{continual learning}, the struggle to acquire new abilities without losing old ones (26, 27), transposed from a single network to a population whose members inherit from one another. Population genetics, I will argue, prices these decisions. Table 1 summarises the correspondences on which the argument runs, and Fig. 1 maps the experimental programme built on them: the same abstractions tested at three tiers --- an exact population-genetic simulator, trained neural networks, and language models --- with the sections that follow climbing that ladder.
|
||||
|
||||
\begin{figure*}[p]\centering % fig1
|
||||
\includegraphics[width=\textwidth]{figs/fig1.pdf}
|
||||
\caption{The experimental programme. Each population-genetic abstraction (Table 1) is tested at up to three tiers of model architecture, ordered left to right by increasing realism: an exact Wright--Fisher simulator over knowledge distributions (closed forms; bitwise-reproducible), trained neural networks measured against exact oracles (recurrent, feedforward, and variational-autoencoder generators on a synthetic mode universe, and a convolutional VAE on MNIST), and language models (LoRA specialists on Qwen bases at 0.5B and 7B, scored by an exact-match verifier). The same population-genetic abstractions are carried across the three tiers. Rows are the framework's mechanisms, each defined at the left margin; filled cells name the experiments run at each tier; dashed cells were not tested, and the composed society at language-model scale is the paper's stated gap.}\label{fig1}
|
||||
\caption{The experimental programme. Each population-genetic abstraction (Table 1) is tested at up to three tiers, ordered left to right by increasing realism: an exact Wright--Fisher simulator over knowledge distributions (closed forms; bitwise-reproducible), trained neural networks measured against exact oracles (recurrent, feedforward, and variational-autoencoder generators on a synthetic mode universe, and a convolutional VAE on MNIST), and language models (LoRA specialists on Qwen bases at 0.5B and 7B, scored by an exact-match verifier). The same population-genetic abstractions are carried across the three tiers. Rows are the framework's mechanisms, each defined at the left margin; filled cells name the experiments run at each tier; dashed cells were not tested, and the composed society at language-model scale is the paper's stated gap.}\label{fig1}
|
||||
\end{figure*}
|
||||
|
||||
\section*{The minimal model, and where its exactness ends}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue