papers: consolidate the real-LLM recombination arc (merge/moe/directed + headroom law)
Folds the four LLM runs into both papers now that the arc is fully characterised.
results-summary.md: new section 5 "The claims tested in real LLM weights" — the
Fisher-Muller generalist (7B merge 0.87 > best specialist 0.77), union-vs-fusion
(routing beats averaging where there is headroom), directed sex (breed + select),
and the unifying HEADROOM law that resolves the earlier saturation confound.
Updates design rule 2 ("merge, don't average — where there is headroom"), adds a
plain-language point 8, refreshes the validation counts (131 tests, real-LLM tier).
the-lamarckian-society-v5.md: softens the three "not a language model yet" claims
to acknowledge the prototype; adds the real-LLM confirmation after the
Fisher-Muller and directed-sex claims (with the headroom caveat as a sharpening,
not a weakening); adds an LLM bullet to section 13; reframes the closing gap to
"the operators, checked; the living society, next."
Tone held to the accessibility/honesty bar: prototype scope flagged (signs not
magnitudes), caveats presented as sharpening the claims, nothing over-celebrated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
39f6c9f4df
commit
4e86d8b602
2 changed files with 111 additions and 27 deletions
|
|
@ -35,9 +35,11 @@ you skipped a definition:
|
|||
it knew when it learns something new.
|
||||
|
||||
We have tried to keep the big picture legible on every page, and to be candid about what is argument
|
||||
and what is evidence. The evidence is from **deliberately small models** — mathematics, small neural
|
||||
networks, image generators, and evolutionary simulations. None of it is a large language model yet.
|
||||
We will say so repeatedly, because the gap matters.
|
||||
and what is evidence. The evidence is mostly from **deliberately small models** — mathematics, small
|
||||
neural networks, image generators, and evolutionary simulations. A first bridge to real language
|
||||
models exists — a prototype that recombines LoRA-specialised Qwen models up to 7B on a GPU cluster,
|
||||
which confirms the recombination signs (below) — but the *full grounded society* has not yet been
|
||||
built on a large language model. We will say so repeatedly, because the gap matters.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -67,13 +69,16 @@ AI has one advantage biology lacks: its "sex" has **no two-parent limit**, its m
|
|||
for complementarity, and its offspring can be **screened before they are kept**. We call this
|
||||
*directed sex*, and it turns recombination from a gamble into a reliable engine.
|
||||
|
||||
We support the argument with a set of **minimal models** (not language models): a mathematically exact
|
||||
account of collapse and its cure; reproductions of the same effects in small trained neural networks
|
||||
and in a generator of handwritten digits; and evolutionary simulations in which a population of models
|
||||
climbs a "fitness landscape" that stands in for reality. In these, every claim above either holds or
|
||||
fails visibly, and removing any single ingredient breaks the system in a distinct way. The scope is
|
||||
honest: these are existence proofs and design rules, and the eventual test is to build the same thing
|
||||
out of real language models. We also note where our diagnosis is no longer novel — the reading of
|
||||
We support the argument with a set of **minimal models**: a mathematically exact account of collapse
|
||||
and its cure; reproductions of the same effects in small trained neural networks and in a generator of
|
||||
handwritten digits; and evolutionary simulations in which a population of models climbs a "fitness
|
||||
landscape" that stands in for reality. In these, every claim above either holds or fails visibly, and
|
||||
removing any single ingredient breaks the system in a distinct way. A first **language-model
|
||||
prototype** then confirms the recombination claims in real weights — merging LoRA-specialised Qwen
|
||||
models (up to 7B on a GPU cluster) produces a generalist that exceeds every parent, and shows the
|
||||
sharp condition under which the "merge, don't average" refinement matters (below). The scope is
|
||||
honest: these are existence proofs and design rules, and the eventual test is to build the *whole
|
||||
grounded society* out of real language models. We also note where our diagnosis is no longer novel — the reading of
|
||||
collapse as genetic drift has since been derived independently — and locate our contribution in the
|
||||
**cure** rather than the diagnosis.
|
||||
|
||||
|
|
@ -207,6 +212,19 @@ knowledge; specialisation is re-earned each generation," and it is why the refra
|
|||
teacher→pupil to *sexual reproduction* is not cosmetic: **copying can only recover a ceiling;
|
||||
recombination can exceed it.**
|
||||
|
||||
This is no longer only a simulation. In a first language-model prototype — LoRA specialists on
|
||||
disjoint task families, recombined and judged by an exact verifier — a merge of three specialist Qwen
|
||||
models (7B, on a GPU cluster) **beats every single specialist**, overall and on every family: the
|
||||
Fisher–Muller effect, in real weights. The same prototype pins down *when* the finer "inherit the
|
||||
union, don't average" rule actually bites. Keeping each parent whole and **routing** each input to the
|
||||
right one beats the tail-thinning average — but only when the task is hard enough to leave room to
|
||||
lose: on easy tasks a strong model's plain average is already at the ceiling, so the crude soup is
|
||||
fine, whereas on hard tasks the average dilutes a hard-won specialist so badly it falls below even the
|
||||
best single parent, and routing wins by a wide margin. The rule is therefore precise: **the union
|
||||
beats the average in exact proportion to how far the average is from the best attainable** — a caveat
|
||||
that sharpens rather than weakens the claim, and that a practitioner needs before spending compute on
|
||||
the fancier operator.
|
||||
|
||||
Two caveats keep this honest, and both are results, not hand-waving.
|
||||
|
||||
*Sex can backfire.* When the parents' skills are not cleanly separable but **entangled** — when the
|
||||
|
|
@ -223,8 +241,11 @@ and cannot inspect an offspring before it is born. An AI has none of those limit
|
|||
**generate many candidate offspring and keep only the fittest**, screening them against reality before
|
||||
committing. We call this **directed sex**, and in our simulations it converts the outbreeding-depression
|
||||
catastrophe into a reliable gain: where blind recombination collapses on entangled problems, directed
|
||||
recombination matches or beats the best parent every time. This is a genuine advantage of engineered
|
||||
reproduction over the biological kind, and we think it is one of the more useful ideas in the paper.
|
||||
recombination matches or beats the best parent every time. The language-model prototype shows the same
|
||||
sign where it can: breeding many recombined Qwen offspring and keeping the one the verifier scores
|
||||
highest beats the single averaged soup on hard tasks (and, unsurprisingly, does nothing extra on easy
|
||||
tasks the soup already solves). This is a genuine advantage of engineered reproduction over the
|
||||
biological kind, and we think it is one of the more useful ideas in the paper.
|
||||
|
||||
So the picture of §5 is: single-teacher copying is asexual and collapses (Muller's ratchet = model
|
||||
collapse); the cure is to *ground* every birth in reality and to reproduce *sexually*, recombining
|
||||
|
|
@ -443,9 +464,10 @@ refute, or you evolve toward what is merely convincing.**
|
|||
## 13. What we built, what we found, and what is still open
|
||||
|
||||
The previous drafts of this paper promised a "companion paper" that *would* make this concrete. That
|
||||
work now exists — as a set of **minimal, laptop-reproducible models**, not language models — and it is
|
||||
worth stating plainly what it does and does not show. (A separate results document gives the numbers;
|
||||
here is the shape.)
|
||||
work now exists — mostly as a set of **minimal, laptop-reproducible models**, with a first bridge to
|
||||
**real language models** (a LoRA-merge prototype, up to 7B on a GPU cluster) — and it is worth stating
|
||||
plainly what it does and does not show. (A separate results document gives the numbers; here is the
|
||||
shape.)
|
||||
|
||||
**What we built and found.**
|
||||
|
||||
|
|
@ -471,6 +493,15 @@ here is the shape.)
|
|||
produces below-parent offspring (outbreeding depression) — and *directed* recombination (choose mates,
|
||||
screen offspring, unbounded parents) reliably fixes it. This is the concrete evidence for the paper's
|
||||
central reframing.
|
||||
- *The recombination claims, in real language models — with a sharp condition.* Merging LoRA-specialised
|
||||
Qwen models (up to 7B on a GPU cluster) produces a generalist that beats every specialist parent
|
||||
(Fisher–Muller, for real); and keeping parents intact and *routing*, or *breeding and screening*
|
||||
offspring, beats the naive average — but *only when the task leaves headroom*. On easy tasks a strong
|
||||
model's plain average is already at the ceiling and the refinements add nothing; on hard tasks the
|
||||
average dilutes a specialist below even the best single parent, and the union-preserving operators win
|
||||
clearly. The practical rule is exact: these tricks pay off in proportion to how far the naive average
|
||||
is from the best attainable. This is a prototype (three task families, one seed), so we read it as
|
||||
signs, not magnitudes; the *whole grounded society* on a language model remains the open step.
|
||||
- *The whole society, and why every part is needed.* In a population evolving on a "reality" landscape,
|
||||
the full system — grounding + sexual recombination + preserved diversity — climbs to the top while
|
||||
keeping its specialists. Remove *grounding* and it collapses into a confident, wrong consensus (a
|
||||
|
|
@ -499,10 +530,11 @@ the **institutions** that let contemporaries correct one another before error is
|
|||
we do not solve; and the **calibration** of everything the results left as knobs — how many parents,
|
||||
how complementary, at what ratio of inherited-to-real data, and how healthy a lineage must be before
|
||||
its knowledge is safe to make irreversibly innate. These are, at least, *measurable* — which is the
|
||||
difference between an open problem and a hole. And the largest gap of all: every result here is from a
|
||||
minimal model. The real test is to build the same grounded, sexually-reproducing, diversity-preserving
|
||||
society out of actual open-weight language models, and see whether the three signs survive contact with
|
||||
a system too big to write down. Theory and toys here; the real thing, next.
|
||||
difference between an open problem and a hole. And the largest gap of all: the *recombination* claims
|
||||
now hold in real language models, but the *society* — the grounded, diversity-preserving, continually
|
||||
reproducing loop — does not yet. The real test is to build that whole system out of actual open-weight
|
||||
language models, and see whether all the signs survive contact with a system too big to write down.
|
||||
The operators, checked; the living society, next.
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue