MachineSex/paper/results-summary.md

# The Lamarckian Society — Summary of results

*The complete laptop-reproducible body of work: the analytical core (Layer 1), the
architecture-general neural existence proof (Layer 1.5, including real MNIST), the learning
kernel, and the sexual-reproduction society (E7–E11). Two summaries of the same work — one
technical, one accessible.*

**The arc in one breath.** Model collapse is **asexual, self-consuming degradation**: a lineage
trained on its own outputs drifts to its own mode and loses the rare tail. We show this is
*literally* a Wright–Fisher drift process (validated against closed forms), reproduce it in real
trained neural weights and on real MNIST images, and then establish the **cure** — a **grounded
sexual society**: reality-checking (grounding) plus **recombination across many decorrelated
parents** (sexual reproduction, not teacher→pupil copying) plus **quality-diversity** selection.
The payoff is not merely arrested collapse but a population whose **offspring exceed their
parents** and whose general capability **climbs** while specialties are re-earned — and removing
any one operator breaks it, and a first **real-LLM prototype** (up to 7B on HPC) confirms the
recombination claims in trained weights. **131 tests pass**; three of the core predictions are exact
closed forms, so the headline curves sit on analytic targets rather than eyeballing.

---

## A. Technical summary

### 1. The analytical core — collapse as Wright–Fisher drift (Layer 1)

Knowledge is a distribution `p_t` over `K` items on the simplex; a fixed Zipf-tailed truth `p*`;
the generational step — *sample `n` from the parent, mix in `m` fresh real samples, refit* — is
**literally a Wright–Fisher process with immigration**, not an analogy. Each safeguard from the
perspective paper is one operator on that step (grounding `g=m/(n+m)`; region-matched grounding;
multi-teacher recombination; directional vs quality-diversity selection; re-minting). Because the
process is Wright–Fisher it inherits **closed-form validation targets**, enforced as
`<0.5%`-tolerance assertions (the "spine of trust"): neutral heterozygosity decay
`E[H_t]=H₀(1−1/n)^t`; fixation probability = initial frequency; the *exact* mutation–drift
equilibrium `H_eq = H*·m(2n+m−1)/(n+2nm+m²)`; the tail-persistence threshold `m·p*ᵢ ≳ 1`; and the
recombination union coverage `U(K_T,ρ,q)=T[ρq+(1−ρ)(1−(1−q)^{K_T})]`.

Findings **E1–E6**:
- **E1 — collapse (null).** Neutral drift reproduces the geometric `H` decay to Monte-Carlo error;
  support collapses `K→1`; forward-KL diverges; tail *items* die ≈10× faster than head items.
  (Aggregate tail *mass* is a drift martingale — a misleading metric; tail-*item* survival is the
  honest one.)
- **E2 — grounding phase boundary (headline).** Stationary `H` tracks the exact `H_eq`; a critical
  **`g* = 0.048` (CI [0.047, 0.050]) ≪ 1** — as little as one real sample against 200 inherited
  restores ~68% of the truth's diversity; `g=0.05` reaches 96%. The sharp threshold lives in
  discrete tail-item survival, not the smooth `H`. The **deep tail is unrescuable** by grounding at
  feasible budgets (`m* ∼ 1/p_min`) — which is what recombination is for.
- **E3 — region-matched grounding.** At fixed budget, matched grounding holds the exercised region's
  tail (0.49) where uniform lets it collapse (0.07). Grounding protects only what it overlaps.
- **E4 — multi-teacher recombination ("merge, don't average").** Union coverage matches the closed
  form exactly (recombination *supplies* the tail). **Principal finding:** under mean-mixture
  distillation surviving tail coverage is **flat in `K_T`** — a conservation law (averaging's `1/K_T`
  dilution exactly cancels the union gain). The benefit is realised only under a **union-preserving
  merge** (`max` over teachers, à la M2N2 model-merging). *Merge weights; don't average outputs.*
- **E5 — QD vs greedy.** Greedy selection fixes (`H≈0.01`); quality-diversity (`w_i ∝ f_i·p_i^{−α}`)
  holds `H` at a positive plateau (0.48–0.88, rising with novelty `α`).
- **E6 — re-minting gate.** Re-minting a *collapsed* lineage makes forward-KL to the original
  diverge (irreversible lock-in); a diversity gate (`H≥H_gate`) prevents it; healthy re-mint is
  harmless.

### 2. Collapse in real trained weights, and on real images (Layer 1.5)

A re-scoped, cheaper Layer 2: realise the *same* Wright–Fisher abstractions in **real trained
generative models** on a fully-synthetic sandbox with an exact oracle, then confirm on real MNIST.
A model's "knowledge" is its oracle-measured distribution over `K` modes; the generational step is
*train-a-model-on-the-previous-model's-samples + grounding*.

- **The histogram bridge (HARD GATE).** A memoryless histogram model reduces Layer 1.5 *exactly* to
  Layer 1: run through the neural runner it recovers `g* = 0.047` and sits on the exact `H_eq` curve.
  This licenses every trained-model result to be read against the analytic core.
- **Collapse in an RNN, and the metric reframing.** A GRU retrained each generation on its own
  output drifts from truth (forward-KL climbs) and grounding arrests it — the sign confirmed. But
  the **operative neural collapse metric is forward-KL, not `H` or tail-survival**: the RNN's
  *smoothing* inductive bias keeps spurious tail modes alive, so `H` stays ~80% of `H*` and
  tail-survival is non-monotone in `g`. On forward-KL, half the divergence gap closes by a
  median-recovery grounding `g≈0.04` (echoing Layer-1's 0.048), but *full* recovery needs `g≈0.19` —
  the sharp `g*≪1` is an exact-operator feature the trained net *softens*.
- **Architecture-generality.** Collapse + grounding-rescue appear in the **histogram, GRU, and MLP**
  alike — the operator is not an artefact of one model class.
- **Recombination in real weights.** The E4 "merge, don't average" law reproduces: construction-level
  union rises 0.49→0.96, oracle-guided max-merge surviving coverage rises while mean-distill stays
  flat — the conservation law holds in trained weights (compressed/noisier, the expected smoothing
  caveat).
- **Real-MNIST external validity.** A convolutional VAE (the canonical collapse vehicle) retrained on
  its own generated digits, modes = digit-class × stroke-thickness (K=30, Zipf), read by a frozen
  CNN oracle (98.5% mode accuracy, confusion matrix recorded as the noise floor): the **dry lineage
  collapses to a single mode** (forward-KL 0.5→18, support 30→1, tail wiped out, H→0), while **10%
  grounding holds all 30 modes**. The eyeball montage shows varied gen-0 digits degenerating into one
  blurry blob. Collapse and its cure are real on real images — not a synthetic artefact.

### 3. The learning kernel — neutral drift is a null both neural models fail, oppositely

Prompted by revisiting the neural deviations, the refit step is generalised from a pure resample to
`p_{t+1} = T_θ(counts/n)`: a **learning kernel** with a *smoothing* knob (mutation toward a prior)
and a *sharpening* knob (mode-competition), both identity by default (so Layer 1 is untouched).
Result: **neutral Wright–Fisher fails both neural architectures in opposite directions.** The
**VAE** (large `n`, small `K`): neutral drift is *inert* (no collapse), yet the real VAE collapses
to one mode — a **sharpening** kernel reproduces it (the estimator *adds* collapse pressure). The
**RNN**: neutral drift drives `H→0`, but the real RNN only partially collapses — a **smoothing**
kernel reproduces the `H`-floor (the estimator *removes* collapse pressure). Model collapse in real
learners = **neutral drift ⊕ an architecture-specific, signed estimator-bias operator**; this
mechanistically explains the architecture-generality result and the softened neural `g*`.

### 4. The sexual-reproduction society (E7–E11) — from teacher→pupil to sex with unbounded parents

The single-locus, fixed-`p*` world can only express *recovery toward a ceiling*. The society's
load-bearing claim is **vertical** — capability that *exceeds* any component — which needs
combinatorial structure. Knowledge becomes a distribution over **genotypes** (`L` biallelic loci,
fitness = number of correct loci), and the one new operator is **recombination**. This is where the
frame shifts: **teacher→pupil distillation is asexual copying (caps at the ceiling); recombination
is sexual reproduction (combinatorial, generative — offspring can exceed both parents), and unlike
biology there is no two-parent limit.** The celebrated evolution-of-sex theory maps onto the thesis
exactly (collapse = Muller's ratchet; merging = meiotic reassortment; "exceeding" = the
Fisher–Muller effect):

- **E7 — the advantage of sex.** A population adapting toward an optimum: the **sexual lineage adapts
  faster** than the asexual one (clonal interference), keeping loci in linkage equilibrium (LD→0 vs
  an LD spike). Honest scope: a *speed* advantage, not a permanent gap (the single-population ratchet
  is subtle).
- **E8 — the vertical claim (Fisher–Muller), the headline.** Decorrelated *parents* are specialists
  (expert on their loci, agnostic elsewhere). **Sexual recombination assembles a genotype fitter than
  any parent, climbing to the optimum (12/12 — a genotype no parent had)** as parent count grows and
  correlation `ρ→0`, while the best single parent (~8.7) and the mean-mixture "model soup" (~11.6)
  plateau below.
- **E9 — landscape robustness ("why sex?").** On rugged (epistatic, NK) landscapes, blindly
  recombining trained models causes **outbreeding depression** — offspring fall *below* the parents,
  worse the more entangled the skills and the higher the recombination rate — and the **optimal
  recombination rate shrinks as ruggedness grows**. Design rule: *merge freely when skills are
  complementary; sparingly, and with selection, when entangled.*
- **E10 — directed sex beats biological sex (the AI superpower).** Biology is stuck with two
  random-mating parents and no offspring preview; an AI can **choose complementary mates, evaluate
  many recombinant offspring, keep the fittest, over rounds, with unbounded parents**. Random
  ("biological") sex craters with ruggedness (0.66→0.51, deep outbreeding depression); **directed sex
  tracks or exceeds the best parent at every ruggedness** — a catastrophe turned into a win, with no
  biological analog.
- **E11 — the dynamic Lamarckian society (the C3 vertical claim, realised).** A finite population of
  agents (genotypes) evolves on a rugged NK landscape that *is* reality, composing all four operators
  — grounding, directed sex, quality-diversity, mutation. Grounding is made load-bearing by the
  **consensus-conformity (self-consumption)** mechanism: selection acts on
  `g·true_fitness + (1−g)·conformity`, so at `g=0` the society optimises fitting-the-crowd rather than
  reality. A **4-arm ablation (global optimum ≈ 0.79), each breaking distinctly, only the full society
  climbing:** `full` **0.78** (climbs to the optimum, diversity maintained longest) · `no_sex` 0.77
  (can't recombine to escape local optima) · `no_diversity`/greedy 0.74 (collapses diversity fastest,
  stuck at a worse local optimum) · **`no_grounding` 0.48** (self-consumption collapse to an unfit
  consensus — trains on the crowd, regresses to a confident-but-wrong mean; the population strongly
  *agrees* while being *wrong*). The society needs **all** of grounding + directed sex + diversity: on
  a rugged landscape you need diversity to explore basins, sex to recombine them, and grounding to
  select on reality.

### 5. The claims tested in real LLM weights (Layer 2 prototype)

The analytic operators above make three claims that are testable in *real language-model weights*:
"merge, don't average" (E4), the Fisher–Muller generalist-from-specialists (E8), and directed sex
(E10). We tested them with a small, reproducible pipeline — LoRA **specialists** fine-tuned on three
*disjoint*, procedurally-generated task families (`lists`, `strings`, `arith`) with an **exact-match
verifier** as reality's "no" — recombined by various operators and scored on held-out tasks. Runs are
at two scales: **Qwen2.5-0.5B** locally and **Qwen2.5-7B** on Imperial College's CX3 HPC (one L40S).
This is a prototype (three families, one seed), so read it as **signs, not magnitudes**; the analytic
layer carries the quantities.

- **Recombining specialists yields a generalist that exceeds every parent (E8).** At 7B, a weight-space
  **merge of the three specialists reaches 0.87 overall vs 0.77 for the best single specialist**, and
  beats every specialist on every family — the Fisher–Muller signature, in real weights. At 0.5B the
  same sign is present but marginal, because a weak base *dilutes* (see below).
- **Union beats averaging — when there is headroom (E4).** Two ways to recombine: **fusion** (average
  the LoRA deltas — a "model soup") versus **union** (keep each specialist intact and *route* each
  input to the right one — a Mixture-of-Experts). On tasks with headroom, **routing beats fusion**
  because averaging dilutes a specialist's contribution: at 0.5B routing 0.74 > soup 0.64; on *hard*
  7B tasks routing 0.50 > soup 0.40, with fusion diluting the fragile string-cipher specialist so badly
  (0.67→0.30) that the soup even loses to the best single specialist. This is E8's `max > mean` in real
  weights.
- **Directed sex helps — when there is headroom (E10).** Breeding a population of recombinant offspring
  (specialists merged at many weightings), scoring each on a held-out split with the verifier, and
  keeping the fittest **beats the single a-priori soup** where the soup is suboptimal: 0.69 > 0.64 at
  0.5B, and 0.49 > 0.39 on hard 7B tasks — recovering most of routing's benefit from one deployable
  model. This is offspring selection (which biology cannot preview) realised in weight space.
- **The one law that unifies them: headroom, not scale.** Our first 7B runs on *easy* tasks showed the
  opposite — fusion beating union, and directed selection tied with the soup. That was a **saturation
  artefact**: the easy families saturated 7B at 1.00, leaving no room to lose to dilution, so the naive
  soup was already optimal. On *hard* (unsaturated) 7B tasks the original ordering returns. **The
  operative variable is headroom**: union-preserving merging and offspring selection pay off whenever
  there is room to lose to dilution — a weak base *or* hard tasks — and only the degenerate corner of
  easy tasks on a strong base makes naive averaging look sufficient. This both confirms E4/E8/E10 in
  real weights and delimits exactly when the refinements matter.

Honest scope: three lexically-distinct families (so the router is trivially accurate — an ambiguous-
skill benchmark is the interesting next stress test), one seed, small LoRA. The full *grounded*
society on LLMs (collapse under dry self-training, the dynamic society) is the HPC-scale next step.

### 6. Positioning — what is prior art, what is ours

An independent 2026 paper (Riis, *Drift and selection in LLM text ecosystems*) rigorously formalises
**collapse = Wright–Fisher drift** (martingale of minority mass, rare-first extinction, de Bruijn
fixed points, drift+selection) with n-gram agents. **We concede that framing as prior art and cite
it — "collapse is drift" is no longer our contribution.** Crucially, Riis's "mixed environment"
retains the lineage's own *old synthetic* tokens (no injection of external truth), so his headline is
*pessimistic* (extinction is independent of retention). **Our defensible contributions, ranked:**
(1) **recombination as sexual reproduction** — the "merge, don't average" law, the Fisher–Muller
vertical claim, directed sex, and their limits (E4, E8–E10) — an operator Riis lacks; (2) the
**learning-kernel / estimator-bias axis**, which he *names as future work*; (3) **grounding as
immigration from a fixed reality**, giving a critical `g*≪1` his closed loop cannot have; (4)
**architecture-generality in real weights + real images (MNIST)**; and (5) **the integrated dynamic
society and its vertical claim** (E11), wholly ours. The repositioning: from *"collapse is drift"*
(diagnosis) to **a population-genetic control theory for sustaining open-ended knowledge** — the
engineered cure and its integration.

### Design rules that fall out

1. **Never inherit dry, and ground where it matters** — a little reality (`g*≈5%`) protects most
   diversity, but it protects only what it overlaps, and it can't hold the deep tail.
2. **Merge, don't average — where there is headroom** — union-preserving model-merging (routing /
   max-merge) realises the multi-teacher benefit; averaging dilutes it. Confirmed in real LLM weights,
   with a sharp caveat: dilution only bites when the task leaves room for it — on easy tasks a strong
   base's naive soup already composes to the ceiling, so averaging looks fine. The refinement matters
   exactly in proportion to how far the naive soup is from optimal.
3. **Sex, with no parent limit** — recombining decorrelated specialists yields offspring that exceed
   any parent; more, complementary parents climb higher.
4. **Match recombination to entanglement** — merge freely for complementary skills; sparingly for
   entangled ones; and *always select offspring* (directed sex), which AI can afford and biology
   cannot.
5. **Keep diversity, and stay grounded** — greedy selection or a broken reality-signal both collapse
   the society; only grounding + sex + diversity together climb.
6. **Gate irreversible consolidation on diversity** — re-mint a base only while the lineage is
   healthy.

---

## B. Accessible summary (for ML engineers and neuroscientists)

### The question

Modern AI is trained once and frozen because it cannot keep learning without *catastrophically
forgetting*. The Lamarckian Society proposes an alternative: **generations** of bounded agents that
learn, then reproduce — passing on what they learned. But there is a notorious trap: train a model on
the previous model's outputs, generation after generation, and it suffers **model collapse** — the
rare cases (the *tail*) vanish first and the model drifts to a bland mode. So the whole scheme lives
or dies on one question: **when does generational transmission accumulate knowledge instead of
degrading it?**

### The through-line: collapse is *asexual* degradation; the cure is *sex*

The key reframe of this work is that **teacher→pupil copying is asexual reproduction** — a pupil can,
at best, recover what its teachers had. That caps out, and left alone it degrades (collapse). The cure
is **sexual reproduction between agents**: combine *many decorrelated parents* so the offspring
inherits a *combination* none of them had — and can be **better than any parent**. Unlike biology,
AI sex has **no two-parent limit** and can *choose* mates and *select* offspring. That is the engine
that lets a society climb instead of collapse.

### What we found, in plain terms

1. **Collapse is real, and it's math.** Casting generational training as the century-old
   **Wright–Fisher** drift process (not a metaphor — the same equations) gives exact formulas to
   check against. The rare stuff dies ~10× faster than the common stuff.
2. **A little reality goes a long way — but not for everything.** Mixing in even ~5% verified real
   data restores most of the diversity and holds it (**grounding**). But the *very rarest*
   capabilities can't be saved by grounding alone — that's what recombination is for.
3. **Collapse is real in actual neural nets, and on real images.** We reproduced it in trained RNNs,
   MLPs, and a VAE, and on **real MNIST** — where a VAE trained on its own digits collapses to a
   single blurry blob, while a little grounding keeps all the digit styles alive. (Honest nuance: real
   nets *smooth*, so "how many modes are alive" lies to you; "how far from the truth" is the honest
   ruler.)
4. **To fight tail collapse with many teachers: merge, don't average.** Averaging their outputs
   mathematically cancels the benefit; a *merge* that keeps each item's strongest source realises it.
5. **Sex makes offspring that beat their parents.** Recombining decorrelated specialist models
   assembles capabilities none of them had, climbing to the optimum as you add more, complementary
   parents — while averaging ("model soup") and the best single parent plateau below. This is a
   celebrated evolutionary result (the Fisher–Muller effect), now shown for AI model merging.
6. **But sex can backfire — and AI has a fix biology lacks.** When skills are *entangled*, blindly
   merging good models produces *worse* offspring ("outbreeding depression"). The fix is **directed
   sex**: choose complementary partners, generate many merges, and keep the best — which AI can do
   and biology can't. Directed sex turns the catastrophe into a win.
7. **The whole society climbs only with all the pieces.** In an evolving population on a rugged
   "reality" landscape, the *full* society (grounding + directed sex + diversity) climbs to the top
   while keeping its specialists; remove **grounding** and it collapses into a confident, wrong
   consensus (the exact analogue of training on the internet's AI-generated crowd); remove **sex** and
   it gets stuck; remove **diversity** and it converges too fast to a worse answer. Each failure is
   distinct; only the full society wins.
8. **It shows up in real language models — with one clean caveat.** We merged LoRA-specialised
   Qwen models (0.5B locally, 7B on a university GPU cluster): the recombined model beats every
   specialist (Fisher–Muller, for real), and *routing* / *offspring-selection* beat naive averaging —
   but only when the tasks are hard enough to leave room. On easy tasks a strong model's plain average
   is already at ceiling, so the fancier operators don't help. The lesson is precise: **these
   recombination tricks matter exactly in proportion to how far the naive average is from the best you
   could do** — which is a genuinely useful thing to know before you spend compute on them.

### Why it is novel and why it matters

- **It turns model collapse from a warning into a control theory.** Collapse-as-drift is now known
  (and independently formalised elsewhere). Our contribution is the *cure* and its integration: a
  grounded, sexually-reproducing, diversity-preserving society that not only avoids collapse but
  **climbs, with offspring exceeding parents**.
- **The sexual-reproduction frame is, we believe, genuinely new for AI** — model merging reframed as
  meiotic recombination, with a rigorous account of when it helps (complementary skills), when it
  hurts (entangled skills), and how to make it reliably win (directed sex, unbounded parents).
- **It is validated, not vibes.** Three core predictions are exact closed forms; the neural, image,
  and **real-LLM** results confirm the *signs* in real trained weights (up to 7B on HPC); 131
  automated tests pass; the whole study is laptop-reproducible from a seed (the LLM tier statistically
  reproducible on one GPU).
- **It gives concrete design rules** for anyone building self-improving or model-merging systems:
  ground where it matters, merge-don't-average, match recombination to skill-entanglement, select
  offspring, keep diversity, and gate irreversible consolidation on health.

*The remaining frontier is the **LLM instantiation** — realising the grounded sexual society with
actual language models (LoRA specialists, real model merging, execution-verified grounding), which
the blueprint frames as the eventual empirical rung.*