MachineSex/paper/layer1-summary.md

Layer 1 — Summary of results

The Lamarckian Society, analytical core. Two summaries of the same work: one technical, one accessible to ML engineers and neuroscientists with no population-genetics background.

---

A. Technical summary

What was built

Layer 1 is a parametric model of generational knowledge transmission, built on the observation that the generational step — sample from the parent distribution, optionally mix in fresh real samples, refit — is literally a Wright–Fisher process with immigration, not merely analogous to one. Knowledge is a distribution p_t over K discrete items on the simplex; a fixed true distribution p* carries a deliberate heavy (Zipf) tail; "model collapse" is the loss of rare alleles under drift. Each safeguard from the perspective paper is one operator on the step:

• grounding g = m/(n+m) — immigration of m real samples per n inherited (mutation supply);
• region-matched grounding — immigration structured by locus;
• multi-teacher distillation — recombination across lineages;
• selection — directional (greedy) vs. balancing/novelty (qd);
• re-minting — a founder event that freezes p_t as the new reference and discards p*.

Because the process is Wright–Fisher, it inherits closed-form validation targets, which are enforced as test_scientific_validation.py assertions (the "spine of trust"):

1. neutral heterozygosity decay E[H_t] = H₀(1−1/n)^t;
2. fixation probability = initial frequency;
3. exact mutation–drift equilibrium H_eq = H*·m(2n+m−1)/(n+2nm+m²) (not the textbook θ/(1+θ) approximation);
4. tail-persistence threshold m·p*_i ≳ 1;
5. recombination union coverage U(K_T,ρ,q) = T[ρq + (1−ρ)(1−(1−q)^{K_T})], with teachers built by a shared-switch exchangeable-Bernoulli construction giving exact marginal retention q and pairwise correlation ρ.

The simulator matches (1), (3), (5) to <0.5% and (2), (4) statistically. 71 tests pass.

Findings (E1–E6)

• E1 — collapse (null). Neutral drift reproduces the geometric H decay to within Monte-Carlo error; support collapses K→1; forward KL to truth diverges. Tail items go extinct ≈10× faster than head items. Subtlety: aggregate tail mass is a drift martingale (mean-conserved), so it is a misleading collapse metric; tail-item survival is the honest one.

• E2 — grounding phase boundary (headline). Stationary H tracks the exact H_eq across the sweep. An operational critical grounding g* = 0.048 (95% bootstrap CI [0.047, 0.050]) marks where H reaches 95% of H*; g ≪ 1* — as little as m=1 real sample against n=200 inherited (g=0.005) restores 68% of the truth's diversity; g=0.05 reaches 96%. The phase boundary in H is smooth (H is continuous in m); the sharp threshold lives in discrete tail-item survival. Per-rarity-band analysis makes the m·p*_i ≳ 1 law visible: at feasible grounding the deep tail is unrescuable — diversity is cheap to protect, but the rarest items require grounding budgets that scale as 1/p_min.

• E3 — region-matched grounding. At fixed total budget, matched grounding preserves the exercised region's tail (survival 0.49) where uniform spreads thin and lets it collapse (0.07). Grounding protects only what it overlaps. (Per-region H is confounded by region mass under matched grounding; tail-item survival is the clean metric.)

• E4 — multi-teacher recombination. Union coverage matches U(K_T,ρ,q) exactly (recombination supplies the tail). Principal finding: under the blueprint's mean-mixture distillation, surviving tail coverage is flat in K_T — a conservation law, since averaging preserves expected pupil tail mass at q·(tail mass of p*) regardless of K_T, and in the rare-tail (linear-survival) regime the 1/K_T dilution exactly cancels the union gain. The recombination benefit is realised only under a union-preserving merge (max over teachers, à la M2N2 model-merging), where surviving coverage rises with K_T and with decorrelation (1−ρ). E4 reports both operators.

• E5 — QD vs. greedy. At matched grounding, greedy (directional) selection drives fixation (H≈0.01); quality-diversity selection (w_i ∝ f_i·p_i^{−α}) holds H at a positive plateau (0.48–0.88, rising with the novelty exponent α). qd ≫ greedy.

• E6 — re-minting gate. Re-minting a collapsed lineage discards the original truth and makes forward KL to the original diverge (irreversible lock-in), and even accelerates the H collapse (grounding now reinforces the surviving few). A diversity gate (H ≥ H_gate) refuses to re-mint while collapsed and keeps KL bounded; re-minting a healthy lineage is harmless.

Implications

1. The economic bet holds for diversity, not the deep tail. The architecture's central claim — "a little grounding protects a lot of inheritance" — is confirmed for overall diversity (g* ≪ 1). But the deepest tail cannot be held by grounding at any feasible budget (m* ∼ 1/p_min). Preserving the deep tail is therefore not grounding's job — it is recombination's, which sets up E4 and the paper's multi-teacher argument.

2. Naive multi-teacher distillation does not prevent tail collapse; merging does. This is the sharpest new result. The paper's recombination benefit is real at the supply (union) level but is annihilated by mean-mixture averaging at matched budget. The benefit survives into the pupil only under a union-preserving merge operator. The paper's recombination claim should therefore rest on model-merging (M2N2), not on averaging distillation — a concrete, falsifiable design constraint carried into Layer 2 (contrast C4).

3. Re-minting is a one-way door and must be gated. Assimilating soft inheritance into a new base while the lineage has narrowed locks in the collapse irreversibly. A cheap diversity gate suffices to prevent it.

4. Everything is anchored to closed forms. Three of the five predictions are exact, so the simulator is validated, not merely plausible — the headline curves sit on analytic targets. The study is bitwise-reproducible from a seed (uv-locked environment).

---

B. Accessible summary (for ML engineers and neuroscientists)

The question

Modern AI is trained once and frozen; it cannot keep learning without catastrophically forgetting. The Lamarckian Society proposes an alternative: generations of bounded agents that learn through a working life, then teach a fresh pupil, who inherits the compressed knowledge and starts ahead — a cultural ratchet. The danger is well known to ML engineers under a different name: train a model on the previous model's outputs, generation after generation, and it suffers model collapse — the rare, improbable cases (the tail) vanish first and the model drifts to its own mode. The teaching step in this architecture is that collapse operation. So the whole scheme lives or dies on one question: under what conditions does generational teaching accumulate knowledge instead of degrading it? Layer 1 answers that quantitatively, before any GPUs are involved.

The one idea that makes it rigorous

Represent a model's knowledge as a probability distribution over discrete "items" (capabilities, facts, behaviours). One generation = draw a finite sample of size n from the teacher, and refit the pupil to it. That finite-sampling step is mathematically identical to genetic drift in a finite population — the century-old Wright–Fisher process. That is not a metaphor; it is the same equations. The payoff: population genetics already has exact formulas for how diversity decays, what survives, and how "immigration" of fresh individuals holds a population together. We inherit those formulas as ground truth to check the simulator against — so the results below are provably correct, not just plausible-looking curves.

A small dictionary:

in this model	ML reading	neuroscience reading
knowledge item	a capability / mode of the model	a memory / stored pattern
sample size n	how much data the student distils from	consolidation bandwidth
the tail	rare capabilities / long-tail inputs	rare episodic detail
grounding g	fraction of fresh verified real data in the training mix	new lived experience replenishing memory
heterozygosity H	diversity of the model's knowledge	richness / non-degeneracy of memory
collapse	mode-seeking / catastrophic forgetting	memory degradation, loss of the improbable

What we found, in plain terms

1. Without fresh data, teaching collapses — and the rare stuff goes first, fast. Pure generation-on-generation distillation loses diversity exponentially, at a rate set by how much data the student sees. Rare items go extinct roughly 10× faster than common ones. (This reproduces, exactly, the known math of drift.)

2. A little fresh grounded data rescues almost all the diversity — this is the headline. Mixing in even ~5% verified real data (in the extreme, one real sample against 200 inherited) restores ~70–96% of the model's diversity and holds it there indefinitely. Grounding is cheap and it works. But there is a hard limit: the very rarest capabilities still cannot be saved by grounding alone — protecting an item of rarity p needs a real-data budget that grows like 1/p. So grounding rescues diversity cheaply, but not the deepest tail. (That is a feature, not a bug — it tells us what the other mechanisms are for.)

3. Grounding only protects what it overlaps. Spreading a fixed amount of fresh data thinly across everything fails to protect any specific area; you must ground the specific region you want to keep. "Don't inherit dry, region by region" is literally true.

4. Learning from several diverse teachers can preserve rare knowledge one teacher would lose — but only if you combine them correctly. This is the surprising, important one. Multiple decorrelated teachers collectively retain far more of the tail than any one of them (we verified this against an exact formula). But whether the pupil keeps that depends entirely on how you merge the teachers. The standard approach — averaging their outputs (ordinary multi-teacher distillation) — mathematically cancels the benefit: the averaging dilutes each teacher's rare knowledge by exactly the factor by which more teachers would have helped. A "keep-the-strongest-teacher-per-item" merge (the style of model merging, e.g. Sakana's M2N2) does realise the benefit — rare-capability retention rises with the number and diversity of teachers. Design lesson: to fight tail collapse with multiple teachers, merge their weights; don't average their outputs.

5. Optimising for "quality" alone collapses diversity; rewarding novelty too keeps it alive. Selecting for fitness drives everything to the single best item (fixation); rewarding rareness alongside fitness (quality-diversity selection) maintains a rich, diverse population. (Familiar to anyone who has watched a population-based or RLHF pipeline mode-collapse.)

6. "Baking in" accumulated knowledge into a new base model is a one-way door. Periodically consolidating soft inheritance into fresh base weights lets the system grow without bound — but if you do it after the model has already narrowed, you lock in the damage permanently (the original, uncollapsed reference is gone). A cheap check — only consolidate while diversity is still high — prevents the irreversible mistake.

Why it is novel and why it matters

• It turns a hand-wavy debate into exact, falsifiable science. "Does generational distillation ratchet up or collapse?" was an argument by analogy. Casting it as Wright–Fisher makes it a set of equations with closed-form answers, and the simulator is validated against them — so the headline curves sit on analytic targets, not on eyeballing.

• It quantifies the feasibility of the whole architecture. The result that a tiny grounding fraction protects most of the diversity (g* ≪ 1) is what makes a continually-teaching society economically plausible rather than a data-hungry fantasy.

• It corrects how the field should build multi-teacher systems. The finding that ordinary averaging distillation gives no protection against tail collapse — while weight-merging does — is a concrete, testable design constraint that most current multi-agent/distillation setups get wrong by default.

• It gives an operational safety rule for self-improving systems. "Consolidate only while diversity is high" is a simple, measurable gate against a failure mode (irreversible collapse-in-place) that self-distilling systems are otherwise prone to.

All of this is at the level of distributions and dynamics, deliberately upstream of neural networks — Layer 2 then checks that the same three signs (grounded inheritance holds where dry inheritance degrades; complementary teachers preserve what one sheds; general capability climbs while each specialty is re-earned) appear in real LoRA-adapted language models.