narrative: make the intellectual contribution explicit

Two insertions answering the editor's implicit questions. Introduction
gains "stated as a problem": the four recurring decisions a model-
population operator cannot currently answer from principle (replay
budget; compose-or-damage; pre-merge detection; separate-vs-consolidate)
and the framework's counterintuitive answers — averaging cancels the
multi-parent benefit exactly in the tail regime; specialisation/
divergence produced no incompatibility anywhere tested while conflicting
conventions always did; weight distance carried no predictive signal
while cheap behavioural disagreement did; and the theory's numbers land
on the replay constants practice converged to independently
(unexpectedness + problem-solved + external check, at reviewer-hardened
calibration). Discussion gains a closing "Outlook: the evolution of
language models": the ecosystem's trajectory is consolidating on exactly
these operators, and the framework's fork is concrete and measurable —
either models stay freely recombinable (one interbreeding population;
levers = per-capability grounding budgets + deliberate diversity) or
long-horizon specialisation at scale begins emergent speciation (a
routed archipelago of diverging lineages), decidable now with the
pre-merge conflict instruments tested here. Design rules tie back to the
posed decisions. 5.2k words, citation invariant intact, 20 pp.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
Giorgio Gilestro 2026-09-07 09:20:21 +01:00
parent aaa146863f
commit e2981c845d
3 changed files with 35 additions and 2 deletions

View file

@ -114,6 +114,20 @@ predictive test in which pre-merge functional-disagreement measures, chosen by t
predicted merge damage on a constructed task grid while the tested weight-geometry baselines showed
no detectable association.
Stated as a problem: an operator of a model population today has no principled answer to four
recurring decisions. How much verified real data does retraining need before a lineage decays?
Will combining two particular models compose their abilities or damage them? Can incompatibility be
detected before paying for a failed merge? And when should specialists be kept separate rather than
consolidated? Current practice answers these with folklore constants and trial-and-error searches.
The framework prices each decision, and several of its answers are not the intuitive ones. Averaging,
the default combining operator, cancels the benefit of multiple parents to first order precisely in
the regime where that benefit matters most, the rare-capability tail. Specialisation and divergence,
widely treated as the threat to mergeability, produced no incompatibility in any regime we tested;
conflicting conventions always did. Weight distance, the field's default compatibility signal, carried
no detectable predictive signal in our controlled test, while a cheap behavioural measure did. And
where the framework's numbers can be checked against settled practice, they land on it: the replay
fractions that continual learning converged on empirically sit at the minimal model's threshold.
The correspondences we develop, summarised in Table 1: single-teacher retraining is *asexual
reproduction*, and the irreversible arm of its decay shares the defining consequence of *Muller's
ratchet* (46): once every copy of a rare capability is gone from all parents and sources, no
@ -377,7 +391,7 @@ are the experiment's open front.
## Discussion
**Design rules.** As engineering guidance, the results reduce to rules that an operator of a model
population can apply. *Ground every generation* in verified reality — a few percent retained most diversity in our tested
population can apply, answering the four decisions posed in the Introduction. *Ground every generation* in verified reality — a few percent retained most diversity in our tested
settings — but price the rarest capabilities individually (observation probability `1 e^{m·p}` per
batch under unstratified sampling), consider targeted sampling for the deep tail, and use
recombination to recover rare capabilities still retained across complementary parents. *Merge, don't blend, when there is headroom*: keep specialists
@ -439,6 +453,21 @@ system: populations of learners where every genotype, environment, and mating de
and manipulable — where the evolution of sex can be studied with interventions (unbounded parents,
offspring preview, directed mating) that no living system permits.
**Outlook: the evolution of language models.** The Introduction's premise, that the model ecosystem is
an evolving population, is also a forecast about where these results matter next. Language-model
development is consolidating around exactly the operators studied here: synthetic-data flywheels
(inheritance), merging and routing of specialist fine-tunes (recombination and population structure),
verifier-gated data pipelines (grounded selection), and periodic consolidation of adapters into new
bases. The framework's reading of that trajectory is concrete. If coming model generations remain what
our tested regimes found, freely recombinable in the absence of conflicting conventions, then the
ecosystem evolves as one interbreeding population, and the levers that matter are grounding budgets
priced per rare capability and diversity preserved deliberately. If instead long-horizon
specialisation at scale begins to produce emergent incompatibility, as the expert-training-duration
observations hint (65) and our small-scale null does not rule out, then lineages will begin to
speciate, and the ecosystem's future is a set of diverging species connected by routing rather than by
merging. Which of the two it will be is measurable now, with the pre-merge conflict instruments this
paper tested.
## Materials and Methods
**Analytic tier.** Pure NumPy/SciPy WrightFisher simulator over `K`-item distributions (knowledge as