# Layer 1 — Summary of results *The Lamarckian Society, analytical core. Two summaries of the same work: one technical, one accessible to ML engineers and neuroscientists with no population-genetics background.* --- ## A. Technical summary ### What was built Layer 1 is a parametric model of generational knowledge transmission, built on the observation that the generational step — *sample from the parent distribution, optionally mix in fresh real samples, refit* — is **literally a Wright–Fisher process with immigration**, not merely analogous to one. Knowledge is a distribution `p_t` over `K` discrete items on the simplex; a fixed true distribution `p*` carries a deliberate heavy (Zipf) tail; "model collapse" is the loss of rare alleles under drift. Each safeguard from the perspective paper is one operator on the step: - **grounding** `g = m/(n+m)` — immigration of `m` real samples per `n` inherited (mutation supply); - **region-matched grounding** — immigration structured by locus; - **multi-teacher distillation** — recombination across lineages; - **selection** — directional (`greedy`) vs. balancing/novelty (`qd`); - **re-minting** — a founder event that freezes `p_t` as the new reference and discards `p*`. Because the process is Wright–Fisher, it inherits **closed-form validation targets**, which are enforced as `test_scientific_validation.py` assertions (the "spine of trust"): 1. neutral heterozygosity decay `E[H_t] = H₀(1−1/n)^t`; 2. fixation probability = initial frequency; 3. **exact** mutation–drift equilibrium `H_eq = H*·m(2n+m−1)/(n+2nm+m²)` (not the textbook `θ/(1+θ)` approximation); 4. tail-persistence threshold `m·p*_i ≳ 1`; 5. recombination union coverage `U(K_T,ρ,q) = T[ρq + (1−ρ)(1−(1−q)^{K_T})]`, with teachers built by a shared-switch exchangeable-Bernoulli construction giving *exact* marginal retention `q` and pairwise correlation `ρ`. The simulator matches (1), (3), (5) to `<0.5%` and (2), (4) statistically. 71 tests pass. ### Findings (E1–E6) - **E1 — collapse (null).** Neutral drift reproduces the geometric `H` decay to within Monte-Carlo error; support collapses `K→1`; forward KL to truth diverges. Tail *items* go extinct ≈10× faster than head items. **Subtlety:** aggregate tail *mass* is a drift martingale (mean-conserved), so it is a misleading collapse metric; tail-*item* survival is the honest one. - **E2 — grounding phase boundary (headline).** Stationary `H` tracks the exact `H_eq` across the sweep. An operational critical grounding `g* = 0.048` (95% bootstrap CI [0.047, 0.050]) marks where `H` reaches 95% of `H*`; **g* ≪ 1** — as little as `m=1` real sample against `n=200` inherited (`g=0.005`) restores 68% of the truth's diversity; `g=0.05` reaches 96%. The phase boundary in `H` is *smooth* (H is continuous in `m`); the sharp threshold lives in discrete tail-item survival. Per-rarity-band analysis makes the `m·p*_i ≳ 1` law visible: at feasible grounding the **deep tail is unrescuable** — diversity is cheap to protect, but the rarest items require grounding budgets that scale as `1/p_min`. - **E3 — region-matched grounding.** At fixed total budget, `matched` grounding preserves the exercised region's tail (survival 0.49) where `uniform` spreads thin and lets it collapse (0.07). Grounding protects only what it overlaps. (Per-region `H` is confounded by region mass under matched grounding; tail-item survival is the clean metric.) - **E4 — multi-teacher recombination.** Union coverage matches `U(K_T,ρ,q)` exactly (recombination *supplies* the tail). **Principal finding:** under the blueprint's mean-mixture distillation, surviving tail coverage is **flat in `K_T`** — a conservation law, since averaging preserves expected pupil tail mass at `q·(tail mass of p*)` regardless of `K_T`, and in the rare-tail (linear-survival) regime the `1/K_T` dilution *exactly cancels* the union gain. The recombination benefit is realised only under a **union-preserving merge** (`max` over teachers, à la M2N2 model-merging), where surviving coverage rises with `K_T` and with decorrelation `(1−ρ)`. E4 reports both operators. - **E5 — QD vs. greedy.** At matched grounding, greedy (directional) selection drives fixation (`H≈0.01`); quality-diversity selection (`w_i ∝ f_i·p_i^{−α}`) holds `H` at a positive plateau (0.48–0.88, rising with the novelty exponent α). qd ≫ greedy. - **E6 — re-minting gate.** Re-minting a *collapsed* lineage discards the original truth and makes forward KL to the original **diverge** (irreversible lock-in), and even accelerates the `H` collapse (grounding now reinforces the surviving few). A diversity gate (`H ≥ H_gate`) refuses to re-mint while collapsed and keeps KL bounded; re-minting a healthy lineage is harmless. ### Implications 1. **The economic bet holds for diversity, not the deep tail.** The architecture's central claim — "a little grounding protects a lot of inheritance" — is confirmed *for overall diversity* (`g* ≪ 1`). But the deepest tail cannot be held by grounding at any feasible budget (`m* ∼ 1/p_min`). Preserving the deep tail is therefore *not* grounding's job — it is recombination's, which sets up E4 and the paper's multi-teacher argument. 2. **Naive multi-teacher distillation does not prevent tail collapse; merging does.** This is the sharpest new result. The paper's recombination benefit is real at the *supply* (union) level but is annihilated by mean-mixture averaging at matched budget. The benefit survives into the pupil only under a union-preserving merge operator. The paper's recombination claim should therefore rest on **model-merging (M2N2)**, not on averaging distillation — a concrete, falsifiable design constraint carried into Layer 2 (contrast C4). 3. **Re-minting is a one-way door and must be gated.** Assimilating soft inheritance into a new base while the lineage has narrowed locks in the collapse irreversibly. A cheap diversity gate suffices to prevent it. 4. **Everything is anchored to closed forms.** Three of the five predictions are exact, so the simulator is *validated*, not merely plausible — the headline curves sit on analytic targets. The study is bitwise-reproducible from a seed (uv-locked environment). --- ## B. Accessible summary (for ML engineers and neuroscientists) ### The question Modern AI is trained once and frozen; it cannot keep learning without *catastrophically forgetting*. The Lamarckian Society proposes an alternative: **generations** of bounded agents that learn through a working life, then *teach* a fresh pupil, who inherits the compressed knowledge and starts ahead — a cultural ratchet. The danger is well known to ML engineers under a different name: train a model on the previous model's outputs, generation after generation, and it suffers **model collapse** — the rare, improbable cases (the *tail*) vanish first and the model drifts to its own mode. The teaching step in this architecture *is* that collapse operation. So the whole scheme lives or dies on one question: **under what conditions does generational teaching accumulate knowledge instead of degrading it?** Layer 1 answers that quantitatively, before any GPUs are involved. ### The one idea that makes it rigorous Represent a model's knowledge as a probability distribution over discrete "items" (capabilities, facts, behaviours). One generation = *draw a finite sample of size `n` from the teacher, and refit the pupil to it.* That finite-sampling step is **mathematically identical** to genetic drift in a finite population — the century-old **Wright–Fisher** process. That is not a metaphor; it is the same equations. The payoff: population genetics already has **exact formulas** for how diversity decays, what survives, and how "immigration" of fresh individuals holds a population together. We inherit those formulas as **ground truth to check the simulator against** — so the results below are *provably correct*, not just plausible-looking curves. A small dictionary: | in this model | ML reading | neuroscience reading | |---|---|---| | knowledge item | a capability / mode of the model | a memory / stored pattern | | sample size `n` | how much data the student distils from | consolidation bandwidth | | the tail | rare capabilities / long-tail inputs | rare episodic detail | | grounding `g` | fraction of fresh **verified** real data in the training mix | new lived experience replenishing memory | | heterozygosity `H` | diversity of the model's knowledge | richness / non-degeneracy of memory | | collapse | mode-seeking / catastrophic forgetting | memory degradation, loss of the improbable | ### What we found, in plain terms 1. **Without fresh data, teaching collapses — and the rare stuff goes first, fast.** Pure generation-on-generation distillation loses diversity exponentially, at a rate set by how much data the student sees. Rare items go extinct roughly 10× faster than common ones. (This reproduces, exactly, the known math of drift.) 2. **A little fresh grounded data rescues almost all the diversity — this is the headline.** Mixing in even ~5% verified real data (in the extreme, *one* real sample against 200 inherited) restores ~70–96% of the model's diversity and holds it there indefinitely. Grounding is cheap and it works. **But** there is a hard limit: the *very rarest* capabilities still cannot be saved by grounding alone — protecting an item of rarity `p` needs a real-data budget that grows like `1/p`. So grounding rescues *diversity* cheaply, but not the deepest tail. (That is a feature, not a bug — it tells us what the other mechanisms are for.) 3. **Grounding only protects what it overlaps.** Spreading a fixed amount of fresh data thinly across everything fails to protect any specific area; you must ground the *specific* region you want to keep. "Don't inherit dry, region by region" is literally true. 4. **Learning from several diverse teachers can preserve rare knowledge one teacher would lose — but only if you combine them correctly. This is the surprising, important one.** Multiple decorrelated teachers *collectively* retain far more of the tail than any one of them (we verified this against an exact formula). But whether the *pupil* keeps that depends entirely on **how you merge the teachers**. The standard approach — averaging their outputs (ordinary multi-teacher distillation) — **mathematically cancels the benefit**: the averaging dilutes each teacher's rare knowledge by exactly the factor by which more teachers would have helped. A **"keep-the-strongest-teacher-per-item" merge** (the style of model *merging*, e.g. Sakana's M2N2) *does* realise the benefit — rare-capability retention rises with the number and diversity of teachers. **Design lesson: to fight tail collapse with multiple teachers, merge their weights; don't average their outputs.** 5. **Optimising for "quality" alone collapses diversity; rewarding novelty too keeps it alive.** Selecting for fitness drives everything to the single best item (fixation); rewarding rareness alongside fitness (quality-diversity selection) maintains a rich, diverse population. (Familiar to anyone who has watched a population-based or RLHF pipeline mode-collapse.) 6. **"Baking in" accumulated knowledge into a new base model is a one-way door.** Periodically consolidating soft inheritance into fresh base weights lets the system grow without bound — but if you do it *after* the model has already narrowed, you lock in the damage **permanently** (the original, uncollapsed reference is gone). A cheap check — only consolidate while diversity is still high — prevents the irreversible mistake. ### Why it is novel and why it matters - **It turns a hand-wavy debate into exact, falsifiable science.** "Does generational distillation ratchet up or collapse?" was an argument by analogy. Casting it as Wright–Fisher makes it a set of equations with closed-form answers, and the simulator is validated against them — so the headline curves *sit on analytic targets*, not on eyeballing. - **It quantifies the feasibility of the whole architecture.** The result that a *tiny* grounding fraction protects most of the diversity (`g* ≪ 1`) is what makes a continually-teaching society economically plausible rather than a data-hungry fantasy. - **It corrects how the field should build multi-teacher systems.** The finding that ordinary averaging distillation gives *no* protection against tail collapse — while weight-merging does — is a concrete, testable design constraint that most current multi-agent/distillation setups get wrong by default. - **It gives an operational safety rule for self-improving systems.** "Consolidate only while diversity is high" is a simple, measurable gate against a failure mode (irreversible collapse-in-place) that self-distilling systems are otherwise prone to. All of this is at the level of *distributions and dynamics*, deliberately upstream of neural networks — Layer 2 then checks that the same three signs (grounded inheritance holds where dry inheritance degrades; complementary teachers preserve what one sheds; general capability climbs while each specialty is re-earned) appear in real LoRA-adapted language models.