MachineSex/paper/results-summary.md

The Lamarckian Society — Summary of results

The complete laptop-reproducible body of work: the analytical core (Layer 1), the architecture-general neural existence proof (Layer 1.5, including real MNIST), the learning kernel, and the sexual-reproduction society (E7–E11). Two summaries of the same work — one technical, one accessible.

The arc in one breath. Model collapse is asexual, self-consuming degradation: a lineage trained on its own outputs drifts to its own mode and loses the rare tail. We show this is literally a Wright–Fisher drift process (validated against closed forms), reproduce it in real trained neural weights and on real MNIST images, and then establish the cure — a grounded sexual society: reality-checking (grounding) plus recombination across many decorrelated parents (sexual reproduction, not teacher→pupil copying) plus quality-diversity selection. The payoff is not merely arrested collapse but a population whose offspring exceed their parents and whose general capability climbs while specialties are re-earned — and removing any one operator breaks it, and a first real-LLM prototype (up to 7B on HPC) confirms the recombination claims in trained weights. 131 tests pass; three of the core predictions are exact closed forms, so the headline curves sit on analytic targets rather than eyeballing.

---

A. Technical summary

1. The analytical core — collapse as Wright–Fisher drift (Layer 1)

Knowledge is a distribution p_t over K items on the simplex; a fixed Zipf-tailed truth p*; the generational step — sample n from the parent, mix in m fresh real samples, refit — is literally a Wright–Fisher process with immigration, not an analogy. Each safeguard from the perspective paper is one operator on that step (grounding g=m/(n+m); region-matched grounding; multi-teacher recombination; directional vs quality-diversity selection; re-minting). Because the process is Wright–Fisher it inherits closed-form validation targets, enforced as <0.5%-tolerance assertions (the "spine of trust"): neutral heterozygosity decay E[H_t]=H₀(1−1/n)^t; fixation probability = initial frequency; the exact mutation–drift equilibrium H_eq = H*·m(2n+m−1)/(n+2nm+m²); the tail-persistence threshold m·p*ᵢ ≳ 1; and the recombination union coverage U(K_T,ρ,q)=T[ρq+(1−ρ)(1−(1−q)^{K_T})].

Findings E1–E6:

• E1 — collapse (null). Neutral drift reproduces the geometric H decay to Monte-Carlo error; support collapses K→1; forward-KL diverges; tail items die ≈10× faster than head items. (Aggregate tail mass is a drift martingale — a misleading metric; tail-item survival is the honest one.)
• E2 — grounding phase boundary (headline). Stationary H tracks the exact H_eq; a critical g* = 0.048 (CI [0.047, 0.050]) ≪ 1 — as little as one real sample against 200 inherited restores ~68% of the truth's diversity; g=0.05 reaches 96%. The sharp threshold lives in discrete tail-item survival, not the smooth H. The deep tail is unrescuable by grounding at feasible budgets (m* ∼ 1/p_min) — which is what recombination is for.
• E3 — region-matched grounding. At fixed budget, matched grounding holds the exercised region's tail (0.49) where uniform lets it collapse (0.07). Grounding protects only what it overlaps.
• E4 — multi-teacher recombination ("merge, don't average"). Union coverage matches the closed form exactly (recombination supplies the tail). Principal finding: under mean-mixture distillation surviving tail coverage is flat in K_T — a conservation law (averaging's 1/K_T dilution exactly cancels the union gain). The benefit is realised only under a union-preserving merge (max over teachers, à la M2N2 model-merging). Merge weights; don't average outputs.
• E5 — QD vs greedy. Greedy selection fixes (H≈0.01); quality-diversity (w_i ∝ f_i·p_i^{−α}) holds H at a positive plateau (0.48–0.88, rising with novelty α).
• E6 — re-minting gate. Re-minting a collapsed lineage makes forward-KL to the original diverge (irreversible lock-in); a diversity gate (H≥H_gate) prevents it; healthy re-mint is harmless.

2. Collapse in real trained weights, and on real images (Layer 1.5)

A re-scoped, cheaper Layer 2: realise the same Wright–Fisher abstractions in real trained generative models on a fully-synthetic sandbox with an exact oracle, then confirm on real MNIST. A model's "knowledge" is its oracle-measured distribution over K modes; the generational step is train-a-model-on-the-previous-model's-samples + grounding.

• The histogram bridge (HARD GATE). A memoryless histogram model reduces Layer 1.5 exactly to Layer 1: run through the neural runner it recovers g* = 0.047 and sits on the exact H_eq curve. This licenses every trained-model result to be read against the analytic core.
• Collapse in an RNN, and the metric reframing. A GRU retrained each generation on its own output drifts from truth (forward-KL climbs) and grounding arrests it — the sign confirmed. But the operative neural collapse metric is forward-KL, not H or tail-survival: the RNN's smoothing inductive bias keeps spurious tail modes alive, so H stays ~80% of H* and tail-survival is non-monotone in g. On forward-KL, half the divergence gap closes by a median-recovery grounding g≈0.04 (echoing Layer-1's 0.048), but full recovery needs g≈0.19 — the sharp g*≪1 is an exact-operator feature the trained net softens.
• Architecture-generality. Collapse + grounding-rescue appear in the histogram, GRU, and MLP alike — the operator is not an artefact of one model class.
• Recombination in real weights. The E4 "merge, don't average" law reproduces: construction-level union rises 0.49→0.96, oracle-guided max-merge surviving coverage rises while mean-distill stays flat — the conservation law holds in trained weights (compressed/noisier, the expected smoothing caveat).
• Real-MNIST external validity. A convolutional VAE (the canonical collapse vehicle) retrained on its own generated digits, modes = digit-class × stroke-thickness (K=30, Zipf), read by a frozen CNN oracle (98.5% mode accuracy, confusion matrix recorded as the noise floor): the dry lineage collapses to a single mode (forward-KL 0.5→18, support 30→1, tail wiped out, H→0), while 10% grounding holds all 30 modes. The eyeball montage shows varied gen-0 digits degenerating into one blurry blob. Collapse and its cure are real on real images — not a synthetic artefact.

3. The learning kernel — neutral drift is a null both neural models fail, oppositely

Prompted by revisiting the neural deviations, the refit step is generalised from a pure resample to p_{t+1} = T_θ(counts/n): a learning kernel with a smoothing knob (mutation toward a prior) and a sharpening knob (mode-competition), both identity by default (so Layer 1 is untouched). Result: neutral Wright–Fisher fails both neural architectures in opposite directions. The VAE (large n, small K): neutral drift is inert (no collapse), yet the real VAE collapses to one mode — a sharpening kernel reproduces it (the estimator adds collapse pressure). The RNN: neutral drift drives H→0, but the real RNN only partially collapses — a smoothing kernel reproduces the H-floor (the estimator removes collapse pressure). Model collapse in real learners = neutral drift ⊕ an architecture-specific, signed estimator-bias operator; this mechanistically explains the architecture-generality result and the softened neural g*.

4. The sexual-reproduction society (E7–E11) — from teacher→pupil to sex with unbounded parents

The single-locus, fixed-p* world can only express recovery toward a ceiling. The society's load-bearing claim is vertical — capability that exceeds any component — which needs combinatorial structure. Knowledge becomes a distribution over genotypes (L biallelic loci, fitness = number of correct loci), and the one new operator is recombination. This is where the frame shifts: teacher→pupil distillation is asexual copying (caps at the ceiling); recombination is sexual reproduction (combinatorial, generative — offspring can exceed both parents), and unlike biology there is no two-parent limit. The celebrated evolution-of-sex theory maps onto the thesis exactly (collapse = Muller's ratchet; merging = meiotic reassortment; "exceeding" = the Fisher–Muller effect):

• E7 — the advantage of sex. A population adapting toward an optimum: the sexual lineage adapts faster than the asexual one (clonal interference), keeping loci in linkage equilibrium (LD→0 vs an LD spike). Honest scope: a speed advantage, not a permanent gap (the single-population ratchet is subtle).
• E8 — the vertical claim (Fisher–Muller), the headline. Decorrelated parents are specialists (expert on their loci, agnostic elsewhere). Sexual recombination assembles a genotype fitter than any parent, climbing to the optimum (12/12 — a genotype no parent had) as parent count grows and correlation ρ→0, while the best single parent (~8.7) and the mean-mixture "model soup" (~11.6) plateau below.
• E9 — landscape robustness ("why sex?"). On rugged (epistatic, NK) landscapes, blindly recombining trained models causes outbreeding depression — offspring fall below the parents, worse the more entangled the skills and the higher the recombination rate — and the optimal recombination rate shrinks as ruggedness grows. Design rule: merge freely when skills are complementary; sparingly, and with selection, when entangled.
• E10 — directed sex beats biological sex (the AI superpower). Biology is stuck with two random-mating parents and no offspring preview; an AI can choose complementary mates, evaluate many recombinant offspring, keep the fittest, over rounds, with unbounded parents. Random ("biological") sex craters with ruggedness (0.66→0.51, deep outbreeding depression); directed sex tracks or exceeds the best parent at every ruggedness — a catastrophe turned into a win, with no biological analog.
• E11 — the dynamic Lamarckian society (the C3 vertical claim, realised). A finite population of agents (genotypes) evolves on a rugged NK landscape that is reality, composing all four operators — grounding, directed sex, quality-diversity, mutation. Grounding is made load-bearing by the consensus-conformity (self-consumption) mechanism: selection acts on g·true_fitness + (1−g)·conformity, so at g=0 the society optimises fitting-the-crowd rather than reality. A 4-arm ablation (global optimum ≈ 0.79), each breaking distinctly, only the full society climbing: full 0.78 (climbs to the optimum, diversity maintained longest) · no_sex 0.77 (can't recombine to escape local optima) · no_diversity/greedy 0.74 (collapses diversity fastest, stuck at a worse local optimum) · no_grounding 0.48 (self-consumption collapse to an unfit consensus — trains on the crowd, regresses to a confident-but-wrong mean; the population strongly agrees while being wrong). The society needs all of grounding + directed sex + diversity: on a rugged landscape you need diversity to explore basins, sex to recombine them, and grounding to select on reality.

5. The claims tested in real LLM weights (Layer 2 prototype)

The analytic operators above make three claims that are testable in real language-model weights: "merge, don't average" (E4), the Fisher–Muller generalist-from-specialists (E8), and directed sex (E10). We tested them with a small, reproducible pipeline — LoRA specialists fine-tuned on three disjoint, procedurally-generated task families (lists, strings, arith) with an exact-match verifier as reality's "no" — recombined by various operators and scored on held-out tasks. Runs are at two scales: Qwen2.5-0.5B locally and Qwen2.5-7B on Imperial College's CX3 HPC (one L40S). This is a prototype (three families, one seed), so read it as signs, not magnitudes; the analytic layer carries the quantities.

• Recombining specialists yields a generalist that exceeds every parent (E8). At 7B, a weight-space merge of the three specialists reaches 0.87 overall vs 0.77 for the best single specialist, and beats every specialist on every family — the Fisher–Muller signature, in real weights. At 0.5B the same sign is present but marginal, because a weak base dilutes (see below).
• Union beats averaging — when there is headroom (E4). Two ways to recombine: fusion (average the LoRA deltas — a "model soup") versus union (keep each specialist intact and route each input to the right one — a Mixture-of-Experts). On tasks with headroom, routing beats fusion because averaging dilutes a specialist's contribution: at 0.5B routing 0.74 > soup 0.64; on hard 7B tasks routing 0.50 > soup 0.40, with fusion diluting the fragile string-cipher specialist so badly (0.67→0.30) that the soup even loses to the best single specialist. This is E8's max > mean in real weights.
• Directed sex helps — when there is headroom (E10). Breeding a population of recombinant offspring (specialists merged at many weightings), scoring each on a held-out split with the verifier, and keeping the fittest beats the single a-priori soup where the soup is suboptimal: 0.69 > 0.64 at 0.5B, and 0.49 > 0.39 on hard 7B tasks — recovering most of routing's benefit from one deployable model. This is offspring selection (which biology cannot preview) realised in weight space.
• The one law that unifies them: headroom, not scale. Our first 7B runs on easy tasks showed the opposite — fusion beating union, and directed selection tied with the soup. That was a saturation artefact: the easy families saturated 7B at 1.00, leaving no room to lose to dilution, so the naive soup was already optimal. On hard (unsaturated) 7B tasks the original ordering returns. The operative variable is headroom: union-preserving merging and offspring selection pay off whenever there is room to lose to dilution — a weak base or hard tasks — and only the degenerate corner of easy tasks on a strong base makes naive averaging look sufficient. This both confirms E4/E8/E10 in real weights and delimits exactly when the refinements matter.

Honest scope: three lexically-distinct families (so the router is trivially accurate — an ambiguous- skill benchmark is the interesting next stress test), one seed, small LoRA. The full grounded society on LLMs (collapse under dry self-training, the dynamic society) is the HPC-scale next step.

6. Positioning — what is prior art, what is ours

An independent 2026 paper (Riis, Drift and selection in LLM text ecosystems) rigorously formalises collapse = Wright–Fisher drift (martingale of minority mass, rare-first extinction, de Bruijn fixed points, drift+selection) with n-gram agents. We concede that framing as prior art and cite it — "collapse is drift" is no longer our contribution. Crucially, Riis's "mixed environment" retains the lineage's own old synthetic tokens (no injection of external truth), so his headline is pessimistic (extinction is independent of retention). Our defensible contributions, ranked: (1) recombination as sexual reproduction — the "merge, don't average" law, the Fisher–Muller vertical claim, directed sex, and their limits (E4, E8–E10) — an operator Riis lacks; (2) the learning-kernel / estimator-bias axis, which he names as future work; (3) grounding as immigration from a fixed reality, giving a critical g*≪1 his closed loop cannot have; (4) architecture-generality in real weights + real images (MNIST); and (5) the integrated dynamic society and its vertical claim (E11), wholly ours. The repositioning: from "collapse is drift" (diagnosis) to a population-genetic control theory for sustaining open-ended knowledge — the engineered cure and its integration.

Design rules that fall out

1. Never inherit dry, and ground where it matters — a little reality (g*≈5%) protects most diversity, but it protects only what it overlaps, and it can't hold the deep tail.
2. Merge, don't average — where there is headroom — union-preserving model-merging (routing / max-merge) realises the multi-teacher benefit; averaging dilutes it. Confirmed in real LLM weights, with a sharp caveat: dilution only bites when the task leaves room for it — on easy tasks a strong base's naive soup already composes to the ceiling, so averaging looks fine. The refinement matters exactly in proportion to how far the naive soup is from optimal.
3. Sex, with no parent limit — recombining decorrelated specialists yields offspring that exceed any parent; more, complementary parents climb higher.
4. Match recombination to entanglement — merge freely for complementary skills; sparingly for entangled ones; and always select offspring (directed sex), which AI can afford and biology cannot.
5. Keep diversity, and stay grounded — greedy selection or a broken reality-signal both collapse the society; only grounding + sex + diversity together climb.
6. Gate irreversible consolidation on diversity — re-mint a base only while the lineage is healthy.

---

B. Accessible summary (for ML engineers and neuroscientists)

The question

Modern AI is trained once and frozen because it cannot keep learning without catastrophically forgetting. The Lamarckian Society proposes an alternative: generations of bounded agents that learn, then reproduce — passing on what they learned. But there is a notorious trap: train a model on the previous model's outputs, generation after generation, and it suffers model collapse — the rare cases (the tail) vanish first and the model drifts to a bland mode. So the whole scheme lives or dies on one question: when does generational transmission accumulate knowledge instead of degrading it?

The through-line: collapse is asexual degradation; the cure is sex

The key reframe of this work is that teacher→pupil copying is asexual reproduction — a pupil can, at best, recover what its teachers had. That caps out, and left alone it degrades (collapse). The cure is sexual reproduction between agents: combine many decorrelated parents so the offspring inherits a combination none of them had — and can be better than any parent. Unlike biology, AI sex has no two-parent limit and can choose mates and select offspring. That is the engine that lets a society climb instead of collapse.

What we found, in plain terms

1. Collapse is real, and it's math. Casting generational training as the century-old Wright–Fisher drift process (not a metaphor — the same equations) gives exact formulas to check against. The rare stuff dies ~10× faster than the common stuff.
2. A little reality goes a long way — but not for everything. Mixing in even ~5% verified real data restores most of the diversity and holds it (grounding). But the very rarest capabilities can't be saved by grounding alone — that's what recombination is for.
3. Collapse is real in actual neural nets, and on real images. We reproduced it in trained RNNs, MLPs, and a VAE, and on real MNIST — where a VAE trained on its own digits collapses to a single blurry blob, while a little grounding keeps all the digit styles alive. (Honest nuance: real nets smooth, so "how many modes are alive" lies to you; "how far from the truth" is the honest ruler.)
4. To fight tail collapse with many teachers: merge, don't average. Averaging their outputs mathematically cancels the benefit; a merge that keeps each item's strongest source realises it.
5. Sex makes offspring that beat their parents. Recombining decorrelated specialist models assembles capabilities none of them had, climbing to the optimum as you add more, complementary parents — while averaging ("model soup") and the best single parent plateau below. This is a celebrated evolutionary result (the Fisher–Muller effect), now shown for AI model merging.
6. But sex can backfire — and AI has a fix biology lacks. When skills are entangled, blindly merging good models produces worse offspring ("outbreeding depression"). The fix is directed sex: choose complementary partners, generate many merges, and keep the best — which AI can do and biology can't. Directed sex turns the catastrophe into a win.
7. The whole society climbs only with all the pieces. In an evolving population on a rugged "reality" landscape, the full society (grounding + directed sex + diversity) climbs to the top while keeping its specialists; remove grounding and it collapses into a confident, wrong consensus (the exact analogue of training on the internet's AI-generated crowd); remove sex and it gets stuck; remove diversity and it converges too fast to a worse answer. Each failure is distinct; only the full society wins.
8. It shows up in real language models — with one clean caveat. We merged LoRA-specialised Qwen models (0.5B locally, 7B on a university GPU cluster): the recombined model beats every specialist (Fisher–Muller, for real), and routing / offspring-selection beat naive averaging — but only when the tasks are hard enough to leave room. On easy tasks a strong model's plain average is already at ceiling, so the fancier operators don't help. The lesson is precise: these recombination tricks matter exactly in proportion to how far the naive average is from the best you could do — which is a genuinely useful thing to know before you spend compute on them.

Why it is novel and why it matters

• It turns model collapse from a warning into a control theory. Collapse-as-drift is now known (and independently formalised elsewhere). Our contribution is the cure and its integration: a grounded, sexually-reproducing, diversity-preserving society that not only avoids collapse but climbs, with offspring exceeding parents.
• The sexual-reproduction frame is, we believe, genuinely new for AI — model merging reframed as meiotic recombination, with a rigorous account of when it helps (complementary skills), when it hurts (entangled skills), and how to make it reliably win (directed sex, unbounded parents).
• It is validated, not vibes. Three core predictions are exact closed forms; the neural, image, and real-LLM results confirm the signs in real trained weights (up to 7B on HPC); 131 automated tests pass; the whole study is laptop-reproducible from a seed (the LLM tier statistically reproducible on one GPU).
• It gives concrete design rules for anyone building self-improving or model-merging systems: ground where it matters, merge-don't-average, match recombination to skill-entanglement, select offspring, keep diversity, and gate irreversible consolidation on health.

The remaining frontier is the LLM instantiation — realising the grounded sexual society with actual language models (LoRA specialists, real model merging, execution-verified grounding), which the blueprint frames as the eventual empirical rung.