papers: consolidate the real-LLM recombination arc (merge/moe/directed + headroom law)
Folds the four LLM runs into both papers now that the arc is fully characterised.
results-summary.md: new section 5 "The claims tested in real LLM weights" — the
Fisher-Muller generalist (7B merge 0.87 > best specialist 0.77), union-vs-fusion
(routing beats averaging where there is headroom), directed sex (breed + select),
and the unifying HEADROOM law that resolves the earlier saturation confound.
Updates design rule 2 ("merge, don't average — where there is headroom"), adds a
plain-language point 8, refreshes the validation counts (131 tests, real-LLM tier).
the-lamarckian-society-v5.md: softens the three "not a language model yet" claims
to acknowledge the prototype; adds the real-LLM confirmation after the
Fisher-Muller and directed-sex claims (with the headroom caveat as a sharpening,
not a weakening); adds an LLM bullet to section 13; reframes the closing gap to
"the operators, checked; the living society, next."
Tone held to the accessibility/honesty bar: prototype scope flagged (signs not
magnitudes), caveats presented as sharpening the claims, nothing over-celebrated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
39f6c9f4df
commit
4e86d8b602
2 changed files with 111 additions and 27 deletions
|
|
@ -13,8 +13,9 @@ sexual society**: reality-checking (grounding) plus **recombination across many
|
|||
parents** (sexual reproduction, not teacher→pupil copying) plus **quality-diversity** selection.
|
||||
The payoff is not merely arrested collapse but a population whose **offspring exceed their
|
||||
parents** and whose general capability **climbs** while specialties are re-earned — and removing
|
||||
any one operator breaks it. **122 tests pass**; three of the core predictions are exact closed
|
||||
forms, so the headline curves sit on analytic targets rather than eyeballing.
|
||||
any one operator breaks it, and a first **real-LLM prototype** (up to 7B on HPC) confirms the
|
||||
recombination claims in trained weights. **131 tests pass**; three of the core predictions are exact
|
||||
closed forms, so the headline curves sit on analytic targets rather than eyeballing.
|
||||
|
||||
---
|
||||
|
||||
|
|
@ -145,7 +146,47 @@ Fisher–Muller effect):
|
|||
a rugged landscape you need diversity to explore basins, sex to recombine them, and grounding to
|
||||
select on reality.
|
||||
|
||||
### 5. Positioning — what is prior art, what is ours
|
||||
### 5. The claims tested in real LLM weights (Layer 2 prototype)
|
||||
|
||||
The analytic operators above make three claims that are testable in *real language-model weights*:
|
||||
"merge, don't average" (E4), the Fisher–Muller generalist-from-specialists (E8), and directed sex
|
||||
(E10). We tested them with a small, reproducible pipeline — LoRA **specialists** fine-tuned on three
|
||||
*disjoint*, procedurally-generated task families (`lists`, `strings`, `arith`) with an **exact-match
|
||||
verifier** as reality's "no" — recombined by various operators and scored on held-out tasks. Runs are
|
||||
at two scales: **Qwen2.5-0.5B** locally and **Qwen2.5-7B** on Imperial College's CX3 HPC (one L40S).
|
||||
This is a prototype (three families, one seed), so read it as **signs, not magnitudes**; the analytic
|
||||
layer carries the quantities.
|
||||
|
||||
- **Recombining specialists yields a generalist that exceeds every parent (E8).** At 7B, a weight-space
|
||||
**merge of the three specialists reaches 0.87 overall vs 0.77 for the best single specialist**, and
|
||||
beats every specialist on every family — the Fisher–Muller signature, in real weights. At 0.5B the
|
||||
same sign is present but marginal, because a weak base *dilutes* (see below).
|
||||
- **Union beats averaging — when there is headroom (E4).** Two ways to recombine: **fusion** (average
|
||||
the LoRA deltas — a "model soup") versus **union** (keep each specialist intact and *route* each
|
||||
input to the right one — a Mixture-of-Experts). On tasks with headroom, **routing beats fusion**
|
||||
because averaging dilutes a specialist's contribution: at 0.5B routing 0.74 > soup 0.64; on *hard*
|
||||
7B tasks routing 0.50 > soup 0.40, with fusion diluting the fragile string-cipher specialist so badly
|
||||
(0.67→0.30) that the soup even loses to the best single specialist. This is E8's `max > mean` in real
|
||||
weights.
|
||||
- **Directed sex helps — when there is headroom (E10).** Breeding a population of recombinant offspring
|
||||
(specialists merged at many weightings), scoring each on a held-out split with the verifier, and
|
||||
keeping the fittest **beats the single a-priori soup** where the soup is suboptimal: 0.69 > 0.64 at
|
||||
0.5B, and 0.49 > 0.39 on hard 7B tasks — recovering most of routing's benefit from one deployable
|
||||
model. This is offspring selection (which biology cannot preview) realised in weight space.
|
||||
- **The one law that unifies them: headroom, not scale.** Our first 7B runs on *easy* tasks showed the
|
||||
opposite — fusion beating union, and directed selection tied with the soup. That was a **saturation
|
||||
artefact**: the easy families saturated 7B at 1.00, leaving no room to lose to dilution, so the naive
|
||||
soup was already optimal. On *hard* (unsaturated) 7B tasks the original ordering returns. **The
|
||||
operative variable is headroom**: union-preserving merging and offspring selection pay off whenever
|
||||
there is room to lose to dilution — a weak base *or* hard tasks — and only the degenerate corner of
|
||||
easy tasks on a strong base makes naive averaging look sufficient. This both confirms E4/E8/E10 in
|
||||
real weights and delimits exactly when the refinements matter.
|
||||
|
||||
Honest scope: three lexically-distinct families (so the router is trivially accurate — an ambiguous-
|
||||
skill benchmark is the interesting next stress test), one seed, small LoRA. The full *grounded*
|
||||
society on LLMs (collapse under dry self-training, the dynamic society) is the HPC-scale next step.
|
||||
|
||||
### 6. Positioning — what is prior art, what is ours
|
||||
|
||||
An independent 2026 paper (Riis, *Drift and selection in LLM text ecosystems*) rigorously formalises
|
||||
**collapse = Wright–Fisher drift** (martingale of minority mass, rare-first extinction, de Bruijn
|
||||
|
|
@ -166,8 +207,11 @@ engineered cure and its integration.
|
|||
|
||||
1. **Never inherit dry, and ground where it matters** — a little reality (`g*≈5%`) protects most
|
||||
diversity, but it protects only what it overlaps, and it can't hold the deep tail.
|
||||
2. **Merge, don't average** — union-preserving model-merging realises the multi-teacher benefit;
|
||||
averaging cancels it.
|
||||
2. **Merge, don't average — where there is headroom** — union-preserving model-merging (routing /
|
||||
max-merge) realises the multi-teacher benefit; averaging dilutes it. Confirmed in real LLM weights,
|
||||
with a sharp caveat: dilution only bites when the task leaves room for it — on easy tasks a strong
|
||||
base's naive soup already composes to the ceiling, so averaging looks fine. The refinement matters
|
||||
exactly in proportion to how far the naive soup is from optimal.
|
||||
3. **Sex, with no parent limit** — recombining decorrelated specialists yields offspring that exceed
|
||||
any parent; more, complementary parents climb higher.
|
||||
4. **Match recombination to entanglement** — merge freely for complementary skills; sparingly for
|
||||
|
|
@ -230,6 +274,13 @@ that lets a society climb instead of collapse.
|
|||
consensus (the exact analogue of training on the internet's AI-generated crowd); remove **sex** and
|
||||
it gets stuck; remove **diversity** and it converges too fast to a worse answer. Each failure is
|
||||
distinct; only the full society wins.
|
||||
8. **It shows up in real language models — with one clean caveat.** We merged LoRA-specialised
|
||||
Qwen models (0.5B locally, 7B on a university GPU cluster): the recombined model beats every
|
||||
specialist (Fisher–Muller, for real), and *routing* / *offspring-selection* beat naive averaging —
|
||||
but only when the tasks are hard enough to leave room. On easy tasks a strong model's plain average
|
||||
is already at ceiling, so the fancier operators don't help. The lesson is precise: **these
|
||||
recombination tricks matter exactly in proportion to how far the naive average is from the best you
|
||||
could do** — which is a genuinely useful thing to know before you spend compute on them.
|
||||
|
||||
### Why it is novel and why it matters
|
||||
|
||||
|
|
@ -240,9 +291,10 @@ that lets a society climb instead of collapse.
|
|||
- **The sexual-reproduction frame is, we believe, genuinely new for AI** — model merging reframed as
|
||||
meiotic recombination, with a rigorous account of when it helps (complementary skills), when it
|
||||
hurts (entangled skills), and how to make it reliably win (directed sex, unbounded parents).
|
||||
- **It is validated, not vibes.** Three core predictions are exact closed forms; the neural and
|
||||
image results confirm the *signs* in real trained weights; 122 automated tests pass; the whole
|
||||
study is laptop-reproducible from a seed.
|
||||
- **It is validated, not vibes.** Three core predictions are exact closed forms; the neural, image,
|
||||
and **real-LLM** results confirm the *signs* in real trained weights (up to 7B on HPC); 131
|
||||
automated tests pass; the whole study is laptop-reproducible from a seed (the LLM tier statistically
|
||||
reproducible on one GPU).
|
||||
- **It gives concrete design rules** for anyone building self-improving or model-merging systems:
|
||||
ground where it matters, merge-don't-average, match recombination to skill-entanglement, select
|
||||
offspring, keep diversity, and gate irreversible consolidation on health.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue