style: remove herald sentences (announcer claudisms)
Nine sentences of the 'The diagnosis comes first.' genus removed or folded — announcements that add nothing over the sentence that follows: 'This population also reproduces.', 'One consequence of drift deserves its genetic name.', 'It is also only the entry point.' (folded), 'The question this paper addresses is...', 'I begin with a result...', 'The engineering point survives the definition:' (rewritten plain), 'The supported conclusion, stated conditionally:', 'The strongest constraint comes from...' (inverted to content-first), 'The framework's reading of that trajectory is concrete.'. Functional enumerator colons and content- bearing topic sentences kept. 19 pp rebuild clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
parent
e2b398c897
commit
6325286a0e
3 changed files with 22 additions and 25 deletions
|
|
@ -47,8 +47,8 @@ millions of models (Hugging Face alone grew past three million by 2026), and the
|
|||
independent creations: the overwhelming majority are fine-tunes, distillations, or merges of a small
|
||||
number of foundation models, forming large family trees whose lineage structure, inherited traits,
|
||||
and mutation dynamics are already being mapped with explicitly phylogenetic methods (1–3).
|
||||
This population also reproduces. Weight-space *model merging*, the direct combination of trained
|
||||
parents into a new model, is mainstream community practice with standard tooling and thousands of
|
||||
Weight-space *model merging*, the direct combination of trained parents into a new model, is
|
||||
mainstream community practice with standard tooling and thousands of
|
||||
hybrid checkpoints, including leaderboard-topping ones (4–7), and the engineering literature
|
||||
describes it in evolutionary vocabulary: "crossover," "mutation," "mate choice," populations of
|
||||
merging models that climb benchmarks (5, 8–10).
|
||||
|
|
@ -66,7 +66,7 @@ transfer the quantitative framework of the branch of biology built for exactly t
|
|||
population genetics of the evolution of sex, and use it to treat multigenerational model populations
|
||||
as systems whose inheritance, diversity, and compatibility can be measured, predicted, and managed.
|
||||
|
||||
The diagnosis comes first. Training each generation of a model on the previous generation's output
|
||||
Training each generation of a model on the previous generation's output
|
||||
degrades it (*model collapse*): rare capabilities vanish first, and the lineage drifts toward its own
|
||||
most common behaviour (21). That degradation is, mathematically, *genetic drift*, the loss of rare
|
||||
variants that any finite population suffers when each generation is a finite sample of the last. The
|
||||
|
|
@ -74,15 +74,14 @@ identification has been made repeatedly and independently: for sequential infere
|
|||
learning (22), for language-model text ecosystems (23), as a closed-form first-extinction law placing
|
||||
collapse onset at the Wright–Fisher first-extinction time (24), and in quantitative-genetic form for
|
||||
self-consuming diffusion models (25). A diagnosis reached so often, from such different starting
|
||||
points, marks population genetics as the natural mathematics of the setting. It is also only the entry
|
||||
point. Population genetics is not, at heart, a theory of decay; it is a theory of the mechanisms that
|
||||
points, marks population genetics as the natural mathematics of the setting, though only as its entry
|
||||
point: population genetics is not, at heart, a theory of decay; it is a theory of the mechanisms that
|
||||
maintain and build populations despite decay (immigration, recombination, selection, population
|
||||
structure) and of where those mechanisms reach their limits. This paper develops that fuller structure
|
||||
for model populations: the arc from drift through its remedies to its limit, reproductive isolation,
|
||||
carried as one framework from closed forms to trained networks to language models.
|
||||
|
||||
The question this paper addresses is what to do with that diagnosis. An operator of a model
|
||||
population faces recurring decisions for which there is no principled guidance: how much verified
|
||||
An operator of a model population faces recurring decisions for which there is no principled guidance: how much verified
|
||||
real data does retraining need before a lineage decays; will combining two particular models compose
|
||||
their abilities or damage them; can incompatibility be detected before paying for a failed merge; and
|
||||
when should specialists be kept separate rather than consolidated? In practice these are settled by
|
||||
|
|
@ -113,8 +112,7 @@ refit) reproduces both. Throughout, a real learner is therefore treated as Wrigh
|
|||
estimator bias*, and the drift signs (rare-first loss; the grounding response)
|
||||
survived that bias in every architecture I tested, including a convolutional VAE retrained on its own
|
||||
generated digits, where the dry lineage collapses to a single blurred digit class while 10% grounding
|
||||
holds all thirty modes (Fig. 1). One consequence of drift deserves its genetic name. Retraining on a
|
||||
single parent is *asexual reproduction*, and sustained loss under it carries the defining consequence
|
||||
holds all thirty modes (Fig. 1). Retraining on a single parent is *asexual reproduction*, and sustained loss under it carries the defining consequence
|
||||
of *Muller's ratchet* (28): once every copy of a rare capability is gone from all parents and sources,
|
||||
no recombination can rebuild it, so remedies must act before fixation-by-loss (a consequence-level
|
||||
correspondence; the minimal model lacks the ratchet's recurrent-mutation driver).
|
||||
|
|
@ -146,8 +144,8 @@ In the minimal model, grounding from a fixed real source is *immigration* into a
|
|||
practical number is an operational threshold, and I define it as such: under the tested population
|
||||
size and Zipf source distribution, `g ≈ 0.05` retained most (≥95%) of equilibrium diversity
|
||||
indefinitely, with the required fraction depending on sample size, source distribution, and the
|
||||
chosen retention target (dependencies in SI). The engineering point survives the definition: verified
|
||||
real data is cheap insurance at fractions far below one. But the same analysis yields a floor the field's
|
||||
chosen retention target (dependencies in SI). Verified real data remains, on any of these
|
||||
definitions, cheap insurance at fractions far below one. But the same analysis yields a floor the field's
|
||||
average-loss framing misses: under unstratified sampling from the source, a capability of rarity `p`
|
||||
appears in a real-data batch of size `m` with probability `1 − e^{−m·p}`, so `m·p ≈ 1` marks roughly a
|
||||
63% chance of one example per batch: a soft observation floor, with higher confidence priced
|
||||
|
|
@ -166,8 +164,7 @@ fraction, the measured price of the estimator bias).
|
|||
|
||||
### Recombination: a conservation law, its operators, and offspring that exceed every parent
|
||||
|
||||
The largest returns from the transfer concern merging. I begin with a result about the most common
|
||||
operator, stated with its assumptions. **Proposition (blending inheritance, rare-item
|
||||
The largest returns from the transfer concern merging. **Proposition (blending inheritance, rare-item
|
||||
regime).** Let K parents independently retain a rare item (mass `p` when retained), and let the child
|
||||
draw `n` samples either from one parent chosen at random or from the *mean of the parents' output
|
||||
distributions*. Expected item mass is identical under the two schemes; and in the rare-item regime
|
||||
|
|
@ -265,7 +262,7 @@ information-theoretic (no single model can satisfy contradictory conventions; SI
|
|||
Proposition S2), with the framework's role being the *structure around it*: which divergences
|
||||
generate conflict, and what moves the cliff.
|
||||
|
||||
The strongest constraint comes from the pre-registered *emergent test*: true BDM incompatibilities are
|
||||
The pre-registered *emergent test* constrains the claim most: true BDM incompatibilities are
|
||||
emergent (each lineage's changes harmless alone), so I let children diverge with *no conflicting
|
||||
signal anywhere*, using complementary class specialists and divergent input conventions, to 6.4× the base
|
||||
training. No isolation emerged (residual 0.000 throughout); instead the merge *rescued* the two
|
||||
|
|
@ -298,7 +295,7 @@ baseline. The pre-registered outcome is the merge penalty against oracle parent
|
|||
hybrid-load analogue), also reported against best- and mean-parent references because the predictor
|
||||
ordering is sensitive to that choice.
|
||||
|
||||
The supported conclusion, stated conditionally: across this controlled grid, pre-merge functional
|
||||
Across this controlled grid, pre-merge functional
|
||||
disagreement predicted merge penalties (clustered bootstrap CIs excluding zero; held-out
|
||||
leave-one-condition-out ρ ≈ 0.35–0.40), whereas LoRA-delta cosine and L2 showed no statistically
|
||||
detectable association; gradient alignment carried intermediate signal. Head-to-head predictor
|
||||
|
|
@ -417,7 +414,7 @@ an evolving population, is also a forecast about where these results matter next
|
|||
development is consolidating around exactly the operators studied here: synthetic-data flywheels
|
||||
(inheritance), merging and routing of specialist fine-tunes (recombination and population structure),
|
||||
verifier-gated data pipelines (grounded selection), and periodic consolidation of adapters into new
|
||||
bases. The framework's reading of that trajectory is concrete. If coming model generations remain what
|
||||
bases. If coming model generations remain what
|
||||
the tested regimes found, freely recombinable in the absence of conflicting conventions, then the
|
||||
ecosystem evolves as one interbreeding population, and the levers that matter are grounding budgets
|
||||
priced per rare capability and diversity preserved deliberately. If instead long-horizon
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue