diff --git a/paper/layer1-summary.md b/paper/layer1-summary.md deleted file mode 100644 index a27c7f8..0000000 --- a/paper/layer1-summary.md +++ /dev/null @@ -1,208 +0,0 @@ -# Layer 1 — Summary of results - -*The Lamarckian Society, analytical core. Two summaries of the same work: one technical, -one accessible to ML engineers and neuroscientists with no population-genetics background.* - ---- - -## A. Technical summary - -### What was built - -Layer 1 is a parametric model of generational knowledge transmission, built on the -observation that the generational step — *sample from the parent distribution, optionally -mix in fresh real samples, refit* — is **literally a Wright–Fisher process with -immigration**, not merely analogous to one. Knowledge is a distribution `p_t` over `K` -discrete items on the simplex; a fixed true distribution `p*` carries a deliberate heavy -(Zipf) tail; "model collapse" is the loss of rare alleles under drift. Each safeguard -from the perspective paper is one operator on the step: - -- **grounding** `g = m/(n+m)` — immigration of `m` real samples per `n` inherited (mutation supply); -- **region-matched grounding** — immigration structured by locus; -- **multi-teacher distillation** — recombination across lineages; -- **selection** — directional (`greedy`) vs. balancing/novelty (`qd`); -- **re-minting** — a founder event that freezes `p_t` as the new reference and discards `p*`. - -Because the process is Wright–Fisher, it inherits **closed-form validation targets**, which -are enforced as `test_scientific_validation.py` assertions (the "spine of trust"): - -1. neutral heterozygosity decay `E[H_t] = H₀(1−1/n)^t`; -2. fixation probability = initial frequency; -3. **exact** mutation–drift equilibrium `H_eq = H*·m(2n+m−1)/(n+2nm+m²)` (not the textbook `θ/(1+θ)` approximation); -4. tail-persistence threshold `m·p*_i ≳ 1`; -5. recombination union coverage `U(K_T,ρ,q) = T[ρq + (1−ρ)(1−(1−q)^{K_T})]`, with teachers built by a shared-switch exchangeable-Bernoulli construction giving *exact* marginal retention `q` and pairwise correlation `ρ`. - -The simulator matches (1), (3), (5) to `<0.5%` and (2), (4) statistically. 71 tests pass. - -### Findings (E1–E6) - -- **E1 — collapse (null).** Neutral drift reproduces the geometric `H` decay to within - Monte-Carlo error; support collapses `K→1`; forward KL to truth diverges. Tail *items* - go extinct ≈10× faster than head items. **Subtlety:** aggregate tail *mass* is a drift - martingale (mean-conserved), so it is a misleading collapse metric; tail-*item* survival - is the honest one. - -- **E2 — grounding phase boundary (headline).** Stationary `H` tracks the exact `H_eq` - across the sweep. An operational critical grounding `g* = 0.048` (95% bootstrap CI - [0.047, 0.050]) marks where `H` reaches 95% of `H*`; **g* ≪ 1** — as little as `m=1` - real sample against `n=200` inherited (`g=0.005`) restores 68% of the truth's diversity; - `g=0.05` reaches 96%. The phase boundary in `H` is *smooth* (H is continuous in `m`); the - sharp threshold lives in discrete tail-item survival. Per-rarity-band analysis makes the - `m·p*_i ≳ 1` law visible: at feasible grounding the **deep tail is unrescuable** — diversity - is cheap to protect, but the rarest items require grounding budgets that scale as `1/p_min`. - -- **E3 — region-matched grounding.** At fixed total budget, `matched` grounding preserves - the exercised region's tail (survival 0.49) where `uniform` spreads thin and lets it - collapse (0.07). Grounding protects only what it overlaps. (Per-region `H` is confounded - by region mass under matched grounding; tail-item survival is the clean metric.) - -- **E4 — multi-teacher recombination.** Union coverage matches `U(K_T,ρ,q)` exactly - (recombination *supplies* the tail). **Principal finding:** under the blueprint's - mean-mixture distillation, surviving tail coverage is **flat in `K_T`** — a conservation - law, since averaging preserves expected pupil tail mass at `q·(tail mass of p*)` - regardless of `K_T`, and in the rare-tail (linear-survival) regime the `1/K_T` dilution - *exactly cancels* the union gain. The recombination benefit is realised only under a - **union-preserving merge** (`max` over teachers, à la M2N2 model-merging), where surviving - coverage rises with `K_T` and with decorrelation `(1−ρ)`. E4 reports both operators. - -- **E5 — QD vs. greedy.** At matched grounding, greedy (directional) selection drives - fixation (`H≈0.01`); quality-diversity selection (`w_i ∝ f_i·p_i^{−α}`) holds `H` at a - positive plateau (0.48–0.88, rising with the novelty exponent α). qd ≫ greedy. - -- **E6 — re-minting gate.** Re-minting a *collapsed* lineage discards the original truth and - makes forward KL to the original **diverge** (irreversible lock-in), and even accelerates - the `H` collapse (grounding now reinforces the surviving few). A diversity gate - (`H ≥ H_gate`) refuses to re-mint while collapsed and keeps KL bounded; re-minting a - healthy lineage is harmless. - -### Implications - -1. **The economic bet holds for diversity, not the deep tail.** The architecture's central - claim — "a little grounding protects a lot of inheritance" — is confirmed *for overall - diversity* (`g* ≪ 1`). But the deepest tail cannot be held by grounding at any feasible - budget (`m* ∼ 1/p_min`). Preserving the deep tail is therefore *not* grounding's job — it - is recombination's, which sets up E4 and the paper's multi-teacher argument. - -2. **Naive multi-teacher distillation does not prevent tail collapse; merging does.** This is - the sharpest new result. The paper's recombination benefit is real at the *supply* (union) - level but is annihilated by mean-mixture averaging at matched budget. The benefit survives - into the pupil only under a union-preserving merge operator. The paper's recombination - claim should therefore rest on **model-merging (M2N2)**, not on averaging distillation — - a concrete, falsifiable design constraint carried into Layer 2 (contrast C4). - -3. **Re-minting is a one-way door and must be gated.** Assimilating soft inheritance into a - new base while the lineage has narrowed locks in the collapse irreversibly. A cheap - diversity gate suffices to prevent it. - -4. **Everything is anchored to closed forms.** Three of the five predictions are exact, so - the simulator is *validated*, not merely plausible — the headline curves sit on analytic - targets. The study is bitwise-reproducible from a seed (uv-locked environment). - ---- - -## B. Accessible summary (for ML engineers and neuroscientists) - -### The question - -Modern AI is trained once and frozen; it cannot keep learning without *catastrophically -forgetting*. The Lamarckian Society proposes an alternative: **generations** of bounded -agents that learn through a working life, then *teach* a fresh pupil, who inherits the -compressed knowledge and starts ahead — a cultural ratchet. The danger is well known to ML -engineers under a different name: train a model on the previous model's outputs, generation -after generation, and it suffers **model collapse** — the rare, improbable cases (the *tail*) -vanish first and the model drifts to its own mode. The teaching step in this architecture *is* -that collapse operation. So the whole scheme lives or dies on one question: **under what -conditions does generational teaching accumulate knowledge instead of degrading it?** Layer 1 -answers that quantitatively, before any GPUs are involved. - -### The one idea that makes it rigorous - -Represent a model's knowledge as a probability distribution over discrete "items" -(capabilities, facts, behaviours). One generation = *draw a finite sample of size `n` from the -teacher, and refit the pupil to it.* That finite-sampling step is **mathematically identical** -to genetic drift in a finite population — the century-old **Wright–Fisher** process. That is -not a metaphor; it is the same equations. The payoff: population genetics already has **exact -formulas** for how diversity decays, what survives, and how "immigration" of fresh individuals -holds a population together. We inherit those formulas as **ground truth to check the simulator -against** — so the results below are *provably correct*, not just plausible-looking curves. - -A small dictionary: - -| in this model | ML reading | neuroscience reading | -|---|---|---| -| knowledge item | a capability / mode of the model | a memory / stored pattern | -| sample size `n` | how much data the student distils from | consolidation bandwidth | -| the tail | rare capabilities / long-tail inputs | rare episodic detail | -| grounding `g` | fraction of fresh **verified** real data in the training mix | new lived experience replenishing memory | -| heterozygosity `H` | diversity of the model's knowledge | richness / non-degeneracy of memory | -| collapse | mode-seeking / catastrophic forgetting | memory degradation, loss of the improbable | - -### What we found, in plain terms - -1. **Without fresh data, teaching collapses — and the rare stuff goes first, fast.** Pure - generation-on-generation distillation loses diversity exponentially, at a rate set by how - much data the student sees. Rare items go extinct roughly 10× faster than common ones. - (This reproduces, exactly, the known math of drift.) - -2. **A little fresh grounded data rescues almost all the diversity — this is the headline.** - Mixing in even ~5% verified real data (in the extreme, *one* real sample against 200 - inherited) restores ~70–96% of the model's diversity and holds it there indefinitely. - Grounding is cheap and it works. **But** there is a hard limit: the *very rarest* - capabilities still cannot be saved by grounding alone — protecting an item of rarity `p` - needs a real-data budget that grows like `1/p`. So grounding rescues *diversity* cheaply, - but not the deepest tail. (That is a feature, not a bug — it tells us what the other - mechanisms are for.) - -3. **Grounding only protects what it overlaps.** Spreading a fixed amount of fresh data thinly - across everything fails to protect any specific area; you must ground the *specific* region - you want to keep. "Don't inherit dry, region by region" is literally true. - -4. **Learning from several diverse teachers can preserve rare knowledge one teacher would - lose — but only if you combine them correctly. This is the surprising, important one.** - Multiple decorrelated teachers *collectively* retain far more of the tail than any one of - them (we verified this against an exact formula). But whether the *pupil* keeps that - depends entirely on **how you merge the teachers**. The standard approach — averaging their - outputs (ordinary multi-teacher distillation) — **mathematically cancels the benefit**: the - averaging dilutes each teacher's rare knowledge by exactly the factor by which more teachers - would have helped. A **"keep-the-strongest-teacher-per-item" merge** (the style of model - *merging*, e.g. Sakana's M2N2) *does* realise the benefit — rare-capability retention rises - with the number and diversity of teachers. **Design lesson: to fight tail collapse with - multiple teachers, merge their weights; don't average their outputs.** - -5. **Optimising for "quality" alone collapses diversity; rewarding novelty too keeps it - alive.** Selecting for fitness drives everything to the single best item (fixation); - rewarding rareness alongside fitness (quality-diversity selection) maintains a rich, - diverse population. (Familiar to anyone who has watched a population-based or RLHF pipeline - mode-collapse.) - -6. **"Baking in" accumulated knowledge into a new base model is a one-way door.** Periodically - consolidating soft inheritance into fresh base weights lets the system grow without bound — - but if you do it *after* the model has already narrowed, you lock in the damage - **permanently** (the original, uncollapsed reference is gone). A cheap check — only - consolidate while diversity is still high — prevents the irreversible mistake. - -### Why it is novel and why it matters - -- **It turns a hand-wavy debate into exact, falsifiable science.** "Does generational - distillation ratchet up or collapse?" was an argument by analogy. Casting it as - Wright–Fisher makes it a set of equations with closed-form answers, and the simulator is - validated against them — so the headline curves *sit on analytic targets*, not on - eyeballing. - -- **It quantifies the feasibility of the whole architecture.** The result that a *tiny* - grounding fraction protects most of the diversity (`g* ≪ 1`) is what makes a - continually-teaching society economically plausible rather than a data-hungry fantasy. - -- **It corrects how the field should build multi-teacher systems.** The finding that ordinary - averaging distillation gives *no* protection against tail collapse — while weight-merging - does — is a concrete, testable design constraint that most current multi-agent/distillation - setups get wrong by default. - -- **It gives an operational safety rule for self-improving systems.** "Consolidate only while - diversity is high" is a simple, measurable gate against a failure mode (irreversible - collapse-in-place) that self-distilling systems are otherwise prone to. - -All of this is at the level of *distributions and dynamics*, deliberately upstream of neural -networks — Layer 2 then checks that the same three signs (grounded inheritance holds where dry -inheritance degrades; complementary teachers preserve what one sheds; general capability climbs -while each specialty is re-earned) appear in real LoRA-adapted language models. diff --git a/paper/results-summary.md b/paper/results-summary.md new file mode 100644 index 0000000..07e9782 --- /dev/null +++ b/paper/results-summary.md @@ -0,0 +1,252 @@ +# The Lamarckian Society — Summary of results + +*The complete laptop-reproducible body of work: the analytical core (Layer 1), the +architecture-general neural existence proof (Layer 1.5, including real MNIST), the learning +kernel, and the sexual-reproduction society (E7–E11). Two summaries of the same work — one +technical, one accessible.* + +**The arc in one breath.** Model collapse is **asexual, self-consuming degradation**: a lineage +trained on its own outputs drifts to its own mode and loses the rare tail. We show this is +*literally* a Wright–Fisher drift process (validated against closed forms), reproduce it in real +trained neural weights and on real MNIST images, and then establish the **cure** — a **grounded +sexual society**: reality-checking (grounding) plus **recombination across many decorrelated +parents** (sexual reproduction, not teacher→pupil copying) plus **quality-diversity** selection. +The payoff is not merely arrested collapse but a population whose **offspring exceed their +parents** and whose general capability **climbs** while specialties are re-earned — and removing +any one operator breaks it. **122 tests pass**; three of the core predictions are exact closed +forms, so the headline curves sit on analytic targets rather than eyeballing. + +--- + +## A. Technical summary + +### 1. The analytical core — collapse as Wright–Fisher drift (Layer 1) + +Knowledge is a distribution `p_t` over `K` items on the simplex; a fixed Zipf-tailed truth `p*`; +the generational step — *sample `n` from the parent, mix in `m` fresh real samples, refit* — is +**literally a Wright–Fisher process with immigration**, not an analogy. Each safeguard from the +perspective paper is one operator on that step (grounding `g=m/(n+m)`; region-matched grounding; +multi-teacher recombination; directional vs quality-diversity selection; re-minting). Because the +process is Wright–Fisher it inherits **closed-form validation targets**, enforced as +`<0.5%`-tolerance assertions (the "spine of trust"): neutral heterozygosity decay +`E[H_t]=H₀(1−1/n)^t`; fixation probability = initial frequency; the *exact* mutation–drift +equilibrium `H_eq = H*·m(2n+m−1)/(n+2nm+m²)`; the tail-persistence threshold `m·p*ᵢ ≳ 1`; and the +recombination union coverage `U(K_T,ρ,q)=T[ρq+(1−ρ)(1−(1−q)^{K_T})]`. + +Findings **E1–E6**: +- **E1 — collapse (null).** Neutral drift reproduces the geometric `H` decay to Monte-Carlo error; + support collapses `K→1`; forward-KL diverges; tail *items* die ≈10× faster than head items. + (Aggregate tail *mass* is a drift martingale — a misleading metric; tail-*item* survival is the + honest one.) +- **E2 — grounding phase boundary (headline).** Stationary `H` tracks the exact `H_eq`; a critical + **`g* = 0.048` (CI [0.047, 0.050]) ≪ 1** — as little as one real sample against 200 inherited + restores ~68% of the truth's diversity; `g=0.05` reaches 96%. The sharp threshold lives in + discrete tail-item survival, not the smooth `H`. The **deep tail is unrescuable** by grounding at + feasible budgets (`m* ∼ 1/p_min`) — which is what recombination is for. +- **E3 — region-matched grounding.** At fixed budget, matched grounding holds the exercised region's + tail (0.49) where uniform lets it collapse (0.07). Grounding protects only what it overlaps. +- **E4 — multi-teacher recombination ("merge, don't average").** Union coverage matches the closed + form exactly (recombination *supplies* the tail). **Principal finding:** under mean-mixture + distillation surviving tail coverage is **flat in `K_T`** — a conservation law (averaging's `1/K_T` + dilution exactly cancels the union gain). The benefit is realised only under a **union-preserving + merge** (`max` over teachers, à la M2N2 model-merging). *Merge weights; don't average outputs.* +- **E5 — QD vs greedy.** Greedy selection fixes (`H≈0.01`); quality-diversity (`w_i ∝ f_i·p_i^{−α}`) + holds `H` at a positive plateau (0.48–0.88, rising with novelty `α`). +- **E6 — re-minting gate.** Re-minting a *collapsed* lineage makes forward-KL to the original + diverge (irreversible lock-in); a diversity gate (`H≥H_gate`) prevents it; healthy re-mint is + harmless. + +### 2. Collapse in real trained weights, and on real images (Layer 1.5) + +A re-scoped, cheaper Layer 2: realise the *same* Wright–Fisher abstractions in **real trained +generative models** on a fully-synthetic sandbox with an exact oracle, then confirm on real MNIST. +A model's "knowledge" is its oracle-measured distribution over `K` modes; the generational step is +*train-a-model-on-the-previous-model's-samples + grounding*. + +- **The histogram bridge (HARD GATE).** A memoryless histogram model reduces Layer 1.5 *exactly* to + Layer 1: run through the neural runner it recovers `g* = 0.047` and sits on the exact `H_eq` curve. + This licenses every trained-model result to be read against the analytic core. +- **Collapse in an RNN, and the metric reframing.** A GRU retrained each generation on its own + output drifts from truth (forward-KL climbs) and grounding arrests it — the sign confirmed. But + the **operative neural collapse metric is forward-KL, not `H` or tail-survival**: the RNN's + *smoothing* inductive bias keeps spurious tail modes alive, so `H` stays ~80% of `H*` and + tail-survival is non-monotone in `g`. On forward-KL, half the divergence gap closes by a + median-recovery grounding `g≈0.04` (echoing Layer-1's 0.048), but *full* recovery needs `g≈0.19` — + the sharp `g*≪1` is an exact-operator feature the trained net *softens*. +- **Architecture-generality.** Collapse + grounding-rescue appear in the **histogram, GRU, and MLP** + alike — the operator is not an artefact of one model class. +- **Recombination in real weights.** The E4 "merge, don't average" law reproduces: construction-level + union rises 0.49→0.96, oracle-guided max-merge surviving coverage rises while mean-distill stays + flat — the conservation law holds in trained weights (compressed/noisier, the expected smoothing + caveat). +- **Real-MNIST external validity.** A convolutional VAE (the canonical collapse vehicle) retrained on + its own generated digits, modes = digit-class × stroke-thickness (K=30, Zipf), read by a frozen + CNN oracle (98.5% mode accuracy, confusion matrix recorded as the noise floor): the **dry lineage + collapses to a single mode** (forward-KL 0.5→18, support 30→1, tail wiped out, H→0), while **10% + grounding holds all 30 modes**. The eyeball montage shows varied gen-0 digits degenerating into one + blurry blob. Collapse and its cure are real on real images — not a synthetic artefact. + +### 3. The learning kernel — neutral drift is a null both neural models fail, oppositely + +Prompted by revisiting the neural deviations, the refit step is generalised from a pure resample to +`p_{t+1} = T_θ(counts/n)`: a **learning kernel** with a *smoothing* knob (mutation toward a prior) +and a *sharpening* knob (mode-competition), both identity by default (so Layer 1 is untouched). +Result: **neutral Wright–Fisher fails both neural architectures in opposite directions.** The +**VAE** (large `n`, small `K`): neutral drift is *inert* (no collapse), yet the real VAE collapses +to one mode — a **sharpening** kernel reproduces it (the estimator *adds* collapse pressure). The +**RNN**: neutral drift drives `H→0`, but the real RNN only partially collapses — a **smoothing** +kernel reproduces the `H`-floor (the estimator *removes* collapse pressure). Model collapse in real +learners = **neutral drift ⊕ an architecture-specific, signed estimator-bias operator**; this +mechanistically explains the architecture-generality result and the softened neural `g*`. + +### 4. The sexual-reproduction society (E7–E11) — from teacher→pupil to sex with unbounded parents + +The single-locus, fixed-`p*` world can only express *recovery toward a ceiling*. The society's +load-bearing claim is **vertical** — capability that *exceeds* any component — which needs +combinatorial structure. Knowledge becomes a distribution over **genotypes** (`L` biallelic loci, +fitness = number of correct loci), and the one new operator is **recombination**. This is where the +frame shifts: **teacher→pupil distillation is asexual copying (caps at the ceiling); recombination +is sexual reproduction (combinatorial, generative — offspring can exceed both parents), and unlike +biology there is no two-parent limit.** The celebrated evolution-of-sex theory maps onto the thesis +exactly (collapse = Muller's ratchet; merging = meiotic reassortment; "exceeding" = the +Fisher–Muller effect): + +- **E7 — the advantage of sex.** A population adapting toward an optimum: the **sexual lineage adapts + faster** than the asexual one (clonal interference), keeping loci in linkage equilibrium (LD→0 vs + an LD spike). Honest scope: a *speed* advantage, not a permanent gap (the single-population ratchet + is subtle). +- **E8 — the vertical claim (Fisher–Muller), the headline.** Decorrelated *parents* are specialists + (expert on their loci, agnostic elsewhere). **Sexual recombination assembles a genotype fitter than + any parent, climbing to the optimum (12/12 — a genotype no parent had)** as parent count grows and + correlation `ρ→0`, while the best single parent (~8.7) and the mean-mixture "model soup" (~11.6) + plateau below. +- **E9 — landscape robustness ("why sex?").** On rugged (epistatic, NK) landscapes, blindly + recombining trained models causes **outbreeding depression** — offspring fall *below* the parents, + worse the more entangled the skills and the higher the recombination rate — and the **optimal + recombination rate shrinks as ruggedness grows**. Design rule: *merge freely when skills are + complementary; sparingly, and with selection, when entangled.* +- **E10 — directed sex beats biological sex (the AI superpower).** Biology is stuck with two + random-mating parents and no offspring preview; an AI can **choose complementary mates, evaluate + many recombinant offspring, keep the fittest, over rounds, with unbounded parents**. Random + ("biological") sex craters with ruggedness (0.66→0.51, deep outbreeding depression); **directed sex + tracks or exceeds the best parent at every ruggedness** — a catastrophe turned into a win, with no + biological analog. +- **E11 — the dynamic Lamarckian society (the C3 vertical claim, realised).** A finite population of + agents (genotypes) evolves on a rugged NK landscape that *is* reality, composing all four operators + — grounding, directed sex, quality-diversity, mutation. Grounding is made load-bearing by the + **consensus-conformity (self-consumption)** mechanism: selection acts on + `g·true_fitness + (1−g)·conformity`, so at `g=0` the society optimises fitting-the-crowd rather than + reality. A **4-arm ablation (global optimum ≈ 0.79), each breaking distinctly, only the full society + climbing:** `full` **0.78** (climbs to the optimum, diversity maintained longest) · `no_sex` 0.77 + (can't recombine to escape local optima) · `no_diversity`/greedy 0.74 (collapses diversity fastest, + stuck at a worse local optimum) · **`no_grounding` 0.48** (self-consumption collapse to an unfit + consensus — trains on the crowd, regresses to a confident-but-wrong mean; the population strongly + *agrees* while being *wrong*). The society needs **all** of grounding + directed sex + diversity: on + a rugged landscape you need diversity to explore basins, sex to recombine them, and grounding to + select on reality. + +### 5. Positioning — what is prior art, what is ours + +An independent 2026 paper (Riis, *Drift and selection in LLM text ecosystems*) rigorously formalises +**collapse = Wright–Fisher drift** (martingale of minority mass, rare-first extinction, de Bruijn +fixed points, drift+selection) with n-gram agents. **We concede that framing as prior art and cite +it — "collapse is drift" is no longer our contribution.** Crucially, Riis's "mixed environment" +retains the lineage's own *old synthetic* tokens (no injection of external truth), so his headline is +*pessimistic* (extinction is independent of retention). **Our defensible contributions, ranked:** +(1) **recombination as sexual reproduction** — the "merge, don't average" law, the Fisher–Muller +vertical claim, directed sex, and their limits (E4, E8–E10) — an operator Riis lacks; (2) the +**learning-kernel / estimator-bias axis**, which he *names as future work*; (3) **grounding as +immigration from a fixed reality**, giving a critical `g*≪1` his closed loop cannot have; (4) +**architecture-generality in real weights + real images (MNIST)**; and (5) **the integrated dynamic +society and its vertical claim** (E11), wholly ours. The repositioning: from *"collapse is drift"* +(diagnosis) to **a population-genetic control theory for sustaining open-ended knowledge** — the +engineered cure and its integration. + +### Design rules that fall out + +1. **Never inherit dry, and ground where it matters** — a little reality (`g*≈5%`) protects most + diversity, but it protects only what it overlaps, and it can't hold the deep tail. +2. **Merge, don't average** — union-preserving model-merging realises the multi-teacher benefit; + averaging cancels it. +3. **Sex, with no parent limit** — recombining decorrelated specialists yields offspring that exceed + any parent; more, complementary parents climb higher. +4. **Match recombination to entanglement** — merge freely for complementary skills; sparingly for + entangled ones; and *always select offspring* (directed sex), which AI can afford and biology + cannot. +5. **Keep diversity, and stay grounded** — greedy selection or a broken reality-signal both collapse + the society; only grounding + sex + diversity together climb. +6. **Gate irreversible consolidation on diversity** — re-mint a base only while the lineage is + healthy. + +--- + +## B. Accessible summary (for ML engineers and neuroscientists) + +### The question + +Modern AI is trained once and frozen because it cannot keep learning without *catastrophically +forgetting*. The Lamarckian Society proposes an alternative: **generations** of bounded agents that +learn, then reproduce — passing on what they learned. But there is a notorious trap: train a model on +the previous model's outputs, generation after generation, and it suffers **model collapse** — the +rare cases (the *tail*) vanish first and the model drifts to a bland mode. So the whole scheme lives +or dies on one question: **when does generational transmission accumulate knowledge instead of +degrading it?** + +### The through-line: collapse is *asexual* degradation; the cure is *sex* + +The key reframe of this work is that **teacher→pupil copying is asexual reproduction** — a pupil can, +at best, recover what its teachers had. That caps out, and left alone it degrades (collapse). The cure +is **sexual reproduction between agents**: combine *many decorrelated parents* so the offspring +inherits a *combination* none of them had — and can be **better than any parent**. Unlike biology, +AI sex has **no two-parent limit** and can *choose* mates and *select* offspring. That is the engine +that lets a society climb instead of collapse. + +### What we found, in plain terms + +1. **Collapse is real, and it's math.** Casting generational training as the century-old + **Wright–Fisher** drift process (not a metaphor — the same equations) gives exact formulas to + check against. The rare stuff dies ~10× faster than the common stuff. +2. **A little reality goes a long way — but not for everything.** Mixing in even ~5% verified real + data restores most of the diversity and holds it (**grounding**). But the *very rarest* + capabilities can't be saved by grounding alone — that's what recombination is for. +3. **Collapse is real in actual neural nets, and on real images.** We reproduced it in trained RNNs, + MLPs, and a VAE, and on **real MNIST** — where a VAE trained on its own digits collapses to a + single blurry blob, while a little grounding keeps all the digit styles alive. (Honest nuance: real + nets *smooth*, so "how many modes are alive" lies to you; "how far from the truth" is the honest + ruler.) +4. **To fight tail collapse with many teachers: merge, don't average.** Averaging their outputs + mathematically cancels the benefit; a *merge* that keeps each item's strongest source realises it. +5. **Sex makes offspring that beat their parents.** Recombining decorrelated specialist models + assembles capabilities none of them had, climbing to the optimum as you add more, complementary + parents — while averaging ("model soup") and the best single parent plateau below. This is a + celebrated evolutionary result (the Fisher–Muller effect), now shown for AI model merging. +6. **But sex can backfire — and AI has a fix biology lacks.** When skills are *entangled*, blindly + merging good models produces *worse* offspring ("outbreeding depression"). The fix is **directed + sex**: choose complementary partners, generate many merges, and keep the best — which AI can do + and biology can't. Directed sex turns the catastrophe into a win. +7. **The whole society climbs only with all the pieces.** In an evolving population on a rugged + "reality" landscape, the *full* society (grounding + directed sex + diversity) climbs to the top + while keeping its specialists; remove **grounding** and it collapses into a confident, wrong + consensus (the exact analogue of training on the internet's AI-generated crowd); remove **sex** and + it gets stuck; remove **diversity** and it converges too fast to a worse answer. Each failure is + distinct; only the full society wins. + +### Why it is novel and why it matters + +- **It turns model collapse from a warning into a control theory.** Collapse-as-drift is now known + (and independently formalised elsewhere). Our contribution is the *cure* and its integration: a + grounded, sexually-reproducing, diversity-preserving society that not only avoids collapse but + **climbs, with offspring exceeding parents**. +- **The sexual-reproduction frame is, we believe, genuinely new for AI** — model merging reframed as + meiotic recombination, with a rigorous account of when it helps (complementary skills), when it + hurts (entangled skills), and how to make it reliably win (directed sex, unbounded parents). +- **It is validated, not vibes.** Three core predictions are exact closed forms; the neural and + image results confirm the *signs* in real trained weights; 122 automated tests pass; the whole + study is laptop-reproducible from a seed. +- **It gives concrete design rules** for anyone building self-improving or model-merging systems: + ground where it matters, merge-don't-average, match recombination to skill-entanglement, select + offspring, keep diversity, and gate irreversible consolidation on health. + +*The remaining frontier is the **LLM instantiation** — realising the grounded sexual society with +actual language models (LoRA specialists, real model merging, execution-verified grounding), which +the blueprint frames as the eventual empirical rung.*