MachineSex/paper/layer1-summary.md

# Layer 1 — Summary of results

*The Lamarckian Society, analytical core. Two summaries of the same work: one technical,
one accessible to ML engineers and neuroscientists with no population-genetics background.*

---

## A. Technical summary

### What was built

Layer 1 is a parametric model of generational knowledge transmission, built on the
observation that the generational step — *sample from the parent distribution, optionally
mix in fresh real samples, refit* — is **literally a Wright–Fisher process with
immigration**, not merely analogous to one. Knowledge is a distribution `p_t` over `K`
discrete items on the simplex; a fixed true distribution `p*` carries a deliberate heavy
(Zipf) tail; "model collapse" is the loss of rare alleles under drift. Each safeguard
from the perspective paper is one operator on the step:

- **grounding** `g = m/(n+m)` — immigration of `m` real samples per `n` inherited (mutation supply);
- **region-matched grounding** — immigration structured by locus;
- **multi-teacher distillation** — recombination across lineages;
- **selection** — directional (`greedy`) vs. balancing/novelty (`qd`);
- **re-minting** — a founder event that freezes `p_t` as the new reference and discards `p*`.

Because the process is Wright–Fisher, it inherits **closed-form validation targets**, which
are enforced as `test_scientific_validation.py` assertions (the "spine of trust"):

1. neutral heterozygosity decay `E[H_t] = H₀(1−1/n)^t`;
2. fixation probability = initial frequency;
3. **exact** mutation–drift equilibrium `H_eq = H*·m(2n+m−1)/(n+2nm+m²)` (not the textbook `θ/(1+θ)` approximation);
4. tail-persistence threshold `m·p*_i ≳ 1`;
5. recombination union coverage `U(K_T,ρ,q) = T[ρq + (1−ρ)(1−(1−q)^{K_T})]`, with teachers built by a shared-switch exchangeable-Bernoulli construction giving *exact* marginal retention `q` and pairwise correlation `ρ`.

The simulator matches (1), (3), (5) to `<0.5%` and (2), (4) statistically. 71 tests pass.

### Findings (E1–E6)

- **E1 — collapse (null).** Neutral drift reproduces the geometric `H` decay to within
  Monte-Carlo error; support collapses `K→1`; forward KL to truth diverges. Tail *items*
  go extinct ≈10× faster than head items. **Subtlety:** aggregate tail *mass* is a drift
  martingale (mean-conserved), so it is a misleading collapse metric; tail-*item* survival
  is the honest one.

- **E2 — grounding phase boundary (headline).** Stationary `H` tracks the exact `H_eq`
  across the sweep. An operational critical grounding `g* = 0.048` (95% bootstrap CI
  [0.047, 0.050]) marks where `H` reaches 95% of `H*`; **g* ≪ 1** — as little as `m=1`
  real sample against `n=200` inherited (`g=0.005`) restores 68% of the truth's diversity;
  `g=0.05` reaches 96%. The phase boundary in `H` is *smooth* (H is continuous in `m`); the
  sharp threshold lives in discrete tail-item survival. Per-rarity-band analysis makes the
  `m·p*_i ≳ 1` law visible: at feasible grounding the **deep tail is unrescuable** — diversity
  is cheap to protect, but the rarest items require grounding budgets that scale as `1/p_min`.

- **E3 — region-matched grounding.** At fixed total budget, `matched` grounding preserves
  the exercised region's tail (survival 0.49) where `uniform` spreads thin and lets it
  collapse (0.07). Grounding protects only what it overlaps. (Per-region `H` is confounded
  by region mass under matched grounding; tail-item survival is the clean metric.)

- **E4 — multi-teacher recombination.** Union coverage matches `U(K_T,ρ,q)` exactly
  (recombination *supplies* the tail). **Principal finding:** under the blueprint's
  mean-mixture distillation, surviving tail coverage is **flat in `K_T`** — a conservation
  law, since averaging preserves expected pupil tail mass at `q·(tail mass of p*)`
  regardless of `K_T`, and in the rare-tail (linear-survival) regime the `1/K_T` dilution
  *exactly cancels* the union gain. The recombination benefit is realised only under a
  **union-preserving merge** (`max` over teachers, à la M2N2 model-merging), where surviving
  coverage rises with `K_T` and with decorrelation `(1−ρ)`. E4 reports both operators.

- **E5 — QD vs. greedy.** At matched grounding, greedy (directional) selection drives
  fixation (`H≈0.01`); quality-diversity selection (`w_i ∝ f_i·p_i^{−α}`) holds `H` at a
  positive plateau (0.48–0.88, rising with the novelty exponent α). qd ≫ greedy.

- **E6 — re-minting gate.** Re-minting a *collapsed* lineage discards the original truth and
  makes forward KL to the original **diverge** (irreversible lock-in), and even accelerates
  the `H` collapse (grounding now reinforces the surviving few). A diversity gate
  (`H ≥ H_gate`) refuses to re-mint while collapsed and keeps KL bounded; re-minting a
  healthy lineage is harmless.

### Implications

1. **The economic bet holds for diversity, not the deep tail.** The architecture's central
   claim — "a little grounding protects a lot of inheritance" — is confirmed *for overall
   diversity* (`g* ≪ 1`). But the deepest tail cannot be held by grounding at any feasible
   budget (`m* ∼ 1/p_min`). Preserving the deep tail is therefore *not* grounding's job — it
   is recombination's, which sets up E4 and the paper's multi-teacher argument.

2. **Naive multi-teacher distillation does not prevent tail collapse; merging does.** This is
   the sharpest new result. The paper's recombination benefit is real at the *supply* (union)
   level but is annihilated by mean-mixture averaging at matched budget. The benefit survives
   into the pupil only under a union-preserving merge operator. The paper's recombination
   claim should therefore rest on **model-merging (M2N2)**, not on averaging distillation —
   a concrete, falsifiable design constraint carried into Layer 2 (contrast C4).

3. **Re-minting is a one-way door and must be gated.** Assimilating soft inheritance into a
   new base while the lineage has narrowed locks in the collapse irreversibly. A cheap
   diversity gate suffices to prevent it.

4. **Everything is anchored to closed forms.** Three of the five predictions are exact, so
   the simulator is *validated*, not merely plausible — the headline curves sit on analytic
   targets. The study is bitwise-reproducible from a seed (uv-locked environment).

---

## B. Accessible summary (for ML engineers and neuroscientists)

### The question

Modern AI is trained once and frozen; it cannot keep learning without *catastrophically
forgetting*. The Lamarckian Society proposes an alternative: **generations** of bounded
agents that learn through a working life, then *teach* a fresh pupil, who inherits the
compressed knowledge and starts ahead — a cultural ratchet. The danger is well known to ML
engineers under a different name: train a model on the previous model's outputs, generation
after generation, and it suffers **model collapse** — the rare, improbable cases (the *tail*)
vanish first and the model drifts to its own mode. The teaching step in this architecture *is*
that collapse operation. So the whole scheme lives or dies on one question: **under what
conditions does generational teaching accumulate knowledge instead of degrading it?** Layer 1
answers that quantitatively, before any GPUs are involved.

### The one idea that makes it rigorous

Represent a model's knowledge as a probability distribution over discrete "items"
(capabilities, facts, behaviours). One generation = *draw a finite sample of size `n` from the
teacher, and refit the pupil to it.* That finite-sampling step is **mathematically identical**
to genetic drift in a finite population — the century-old **Wright–Fisher** process. That is
not a metaphor; it is the same equations. The payoff: population genetics already has **exact
formulas** for how diversity decays, what survives, and how "immigration" of fresh individuals
holds a population together. We inherit those formulas as **ground truth to check the simulator
against** — so the results below are *provably correct*, not just plausible-looking curves.

A small dictionary:

| in this model | ML reading | neuroscience reading |
|---|---|---|
| knowledge item | a capability / mode of the model | a memory / stored pattern |
| sample size `n` | how much data the student distils from | consolidation bandwidth |
| the tail | rare capabilities / long-tail inputs | rare episodic detail |
| grounding `g` | fraction of fresh **verified** real data in the training mix | new lived experience replenishing memory |
| heterozygosity `H` | diversity of the model's knowledge | richness / non-degeneracy of memory |
| collapse | mode-seeking / catastrophic forgetting | memory degradation, loss of the improbable |

### What we found, in plain terms

1. **Without fresh data, teaching collapses — and the rare stuff goes first, fast.** Pure
   generation-on-generation distillation loses diversity exponentially, at a rate set by how
   much data the student sees. Rare items go extinct roughly 10× faster than common ones.
   (This reproduces, exactly, the known math of drift.)

2. **A little fresh grounded data rescues almost all the diversity — this is the headline.**
   Mixing in even ~5% verified real data (in the extreme, *one* real sample against 200
   inherited) restores ~70–96% of the model's diversity and holds it there indefinitely.
   Grounding is cheap and it works. **But** there is a hard limit: the *very rarest*
   capabilities still cannot be saved by grounding alone — protecting an item of rarity `p`
   needs a real-data budget that grows like `1/p`. So grounding rescues *diversity* cheaply,
   but not the deepest tail. (That is a feature, not a bug — it tells us what the other
   mechanisms are for.)

3. **Grounding only protects what it overlaps.** Spreading a fixed amount of fresh data thinly
   across everything fails to protect any specific area; you must ground the *specific* region
   you want to keep. "Don't inherit dry, region by region" is literally true.

4. **Learning from several diverse teachers can preserve rare knowledge one teacher would
   lose — but only if you combine them correctly. This is the surprising, important one.**
   Multiple decorrelated teachers *collectively* retain far more of the tail than any one of
   them (we verified this against an exact formula). But whether the *pupil* keeps that
   depends entirely on **how you merge the teachers**. The standard approach — averaging their
   outputs (ordinary multi-teacher distillation) — **mathematically cancels the benefit**: the
   averaging dilutes each teacher's rare knowledge by exactly the factor by which more teachers
   would have helped. A **"keep-the-strongest-teacher-per-item" merge** (the style of model
   *merging*, e.g. Sakana's M2N2) *does* realise the benefit — rare-capability retention rises
   with the number and diversity of teachers. **Design lesson: to fight tail collapse with
   multiple teachers, merge their weights; don't average their outputs.**

5. **Optimising for "quality" alone collapses diversity; rewarding novelty too keeps it
   alive.** Selecting for fitness drives everything to the single best item (fixation);
   rewarding rareness alongside fitness (quality-diversity selection) maintains a rich,
   diverse population. (Familiar to anyone who has watched a population-based or RLHF pipeline
   mode-collapse.)

6. **"Baking in" accumulated knowledge into a new base model is a one-way door.** Periodically
   consolidating soft inheritance into fresh base weights lets the system grow without bound —
   but if you do it *after* the model has already narrowed, you lock in the damage
   **permanently** (the original, uncollapsed reference is gone). A cheap check — only
   consolidate while diversity is still high — prevents the irreversible mistake.

### Why it is novel and why it matters

- **It turns a hand-wavy debate into exact, falsifiable science.** "Does generational
  distillation ratchet up or collapse?" was an argument by analogy. Casting it as
  Wright–Fisher makes it a set of equations with closed-form answers, and the simulator is
  validated against them — so the headline curves *sit on analytic targets*, not on
  eyeballing.

- **It quantifies the feasibility of the whole architecture.** The result that a *tiny*
  grounding fraction protects most of the diversity (`g* ≪ 1`) is what makes a
  continually-teaching society economically plausible rather than a data-hungry fantasy.

- **It corrects how the field should build multi-teacher systems.** The finding that ordinary
  averaging distillation gives *no* protection against tail collapse — while weight-merging
  does — is a concrete, testable design constraint that most current multi-agent/distillation
  setups get wrong by default.

- **It gives an operational safety rule for self-improving systems.** "Consolidate only while
  diversity is high" is a simple, measurable gate against a failure mode (irreversible
  collapse-in-place) that self-distilling systems are otherwise prone to.

All of this is at the level of *distributions and dynamics*, deliberately upstream of neural
networks — Layer 2 then checks that the same three signs (grounded inheritance holds where dry
inheritance degrades; complementary teachers preserve what one sheds; general capability climbs
while each specialty is re-earned) appear in real LoRA-adapted language models.