MachineSex/paper/blueprint.md

A Technical Blueprint for Modelling the Lamarckian Society

An implementation specification: two layers, one population-genetics engine

Companion technical paper to "The Lamarckian Society." Blueprint v1 — written to be handed, whole, to an autonomous coding agent (Claude Code) for implementation. Every quantitative claim in the perspective paper is reduced here to a state variable, an update rule, an analytic prediction, and a falsifier.

---

0. How to use this document

This is a build specification, not an essay. It is written so that a coding agent can implement the entire study from it with minimal further decisions, and so that a human reader can verify every modelling choice against the theory it is meant to test.

Scope. Two layers, deliberately staged by cost:

• Layer 1 — the analytical core. A parametric population-genetics model of knowledge transmission across generations. Pure NumPy/SciPy. Runs on a laptop in minutes. This layer carries the paper's quantitative claims: the grounding phase boundary, the decorrelation curve, region-matched grounding, quality-diversity vs. greedy selection, and the re-minting gate. Several of its predictions are analytically solvable, which turns validation into an exact test rather than a vibe.
• Layer 2 — the neural existence proof. A minimal demonstration that the same effects appear in real weights: small open-weight models, LoRA specialisation, distillation/merging across two–three generations, with an execution-based verifier standing in for "reality's no." One consumer GPU. This layer answers the single most predictable reviewer objection to Layer 1 ("you assumed the collapse operator") by showing the sign of the key effects without assuming them.

The relationship between layers. Layer 1 defines the abstractions (region, rarity, grounding fraction, teacher decorrelation, diversity metric). Layer 2 realises the same abstractions in a neural pipeline: a "region" is a task family, "rarity" is task-type frequency, "grounding" is verifier-passed samples, "decorrelation" is teachers specialised on disjoint task families. Keeping the abstractions identical across layers is a hard requirement — it is what lets a Layer-2 result be read as confirming a Layer-1 prediction.

Non-goals for v1. No human-in-the-loop interaction (the verifier is the refuter; humans are future work). No foundation-scale training. No claim about the horizontal prediction (generation size vs. domain decomposability) beyond an optional abstract treatment in §2.7 — it is the most compute-hungry claim and is explicitly deferred. The load-bearing target is the vertical claim (general knowledge climbs while each specialty is re-earned and exceeded).

Reader's map. §1 gives the formal dictionary between population genetics and knowledge transmission — read this first; everything else is an instantiation of it. §2 is Layer 1 in full (theory, experiments, code interfaces). §3 is Layer 2. §4 is the shared reproducibility standard. §5 is the repository layout. §6 is the claims→experiments→figures traceability matrix. §7 is the suggested build order for the coding agent.

---

1. The formal mapping: knowledge transmission is a Wright–Fisher process

The perspective paper argues by analogy that generational distillation resembles genetic drift, that multi-teacher distillation resembles recombination, and that heterodox input plus quality-diversity selection resembles mutation supply plus balancing selection. This blueprint drops the word "resembles." At the level of a distribution over discrete items evolving by finite resampling, these are not analogies; they are the same stochastic process, and the population-genetics literature has already solved large parts of it. We therefore adopt Wright–Fisher dynamics as the engine and inherit its exact results as our validation targets.

The core object is a distribution over discrete knowledge items — facts, capabilities, behaviours, or "modes." Call the items 1, \dots, K. A model at generation t holds a distribution p_t = (p_t^1, \dots, p_t^K) on the simplex. There is a fixed true distribution p^\* over the same items, some of which are rare (the tail — the improbable events whose loss defines model collapse).

The generational step is: sample from the parent, optionally mix in fresh real samples, refit. That single step is Wright–Fisher reproduction with immigration. Here is the dictionary, and it is meant to be used literally throughout implementation:

Knowledge-transmission concept (perspective paper)	Population-genetics object (this blueprint)	Governs
Knowledge item / capability / mode	Allele / type	State space
Model's knowledge distribution p_t	Allele-frequency vector	State
True distribution p^\* (with rare tail)	Ancestral/immigrant frequencies	Grounding target
Distillation sample size n	Population size N	Drift strength (\propto 1/n)
Lossy compression / "shed the idiosyncratic"	Genetic drift	Tail-first loss
Model collapse (tail lost first)	Loss of rare alleles under drift	The central failure
Grounding: m fresh real samples per passage	Immigration / mutation supply	Tail replenishment
"No dry inheritance," region-matched	Immigration structured by locus	Which tails are protected
Multi-teacher distillation	Recombination across lineages	Reconstitutes lost tails
Teacher decorrelation \rho	Linkage / shared ancestry	Recombination benefit
Single-teacher irreversible error	Muller's ratchet (clonal lineage)	Why one teacher is unsafe
Diversity metric (§7 of paper)	Expected heterozygosity H = 1 - \sum_i p_i^2	Health of the lineage
Greedy fitness-max selection	Directional selection → fixation	Accelerated collapse
Quality-diversity / novelty selection	Balancing / negative frequency-dependent selection	Maintained polymorphism
Re-minting the base (§11 of paper)	Founder event / new reference genome	Irreversibility
Heterodox contributor supply	Standing variation of founding stock	Raw material

Every experiment below is a manipulation of this one process. The value of the mapping is that it hands us closed-form predictions — heterozygosity decay, mutation–drift equilibrium, fixation probabilities — against which the simulator must agree before any headline result is trusted. Validation and theory are the same equations.

---

2. Layer 1 — the analytical core

2.1 State and the base dynamics (neutral drift = Shumailov collapse)

State. A single lineage is a point p_t on the $K$-simplex. Items are partitioned into R regions (disjoint blocks of the K items); regions are how we express "different areas of knowledge," and they are what grounding and specialisation are targeted at. The true distribution p^\* is fixed and chosen to have a deliberate tail: most probability mass on common items, a long thin tail of rare items (e.g. a Zipfian or a two-component mixture of "head" and "tail" items, tunable).

The generational step, base case (no grounding, single teacher). Given parent p_t and drift strength n:

1. Draw counts c \sim \mathrm{Multinomial}(n, p_t).
2. Set p_{t+1}^i = c^i / n.

This is exactly neutral Wright–Fisher with haploid population size n. It is also exactly Shumailov's recursive resampling with a perfect refit. Rare items are lost first (once c^i = 0, item i is gone and — with no grounding — cannot return); the lineage drifts to fixation on a single item.

This is the null model and the first validation. It must reproduce, within Monte-Carlo error, the classical drift results in §2.4. If it does not, nothing downstream is trustworthy.

2.2 The five mechanisms, each as an operator on the step

Each perspective-paper safeguard is one modification of the generational step. They compose; the full step applies them in the order below.

(A) Grounding — immigration / mutation supply. Fresh real data enters every passage. Replace the single draw with a pooled draw:

• Draw c_{\text{syn}} \sim \mathrm{Multinomial}(n, p_t) (inherited / teacher output).
• Draw c_{\text{real}} \sim \mathrm{Multinomial}(m, p^\*) (grounding).
• p_{t+1}^i = (c_{\text{syn}}^i + c_{\text{real}}^i)/(n+m).

Define the grounding fraction g = m/(n+m). This is the single most important control variable in the paper: it is the ratio of freshly-grounded to inherited information per passage. The claim "no dry inheritance" is g > 0; the claim that a critical g exists is the phase-boundary experiment E2.

(B) Region-matched grounding — structured immigration. Grounding is a vector, not a scalar: m = (m_1, \dots, m_R) real samples allocated across regions. "No dry inheritance, region by region" means grounding must be delivered to the regions whose tails are at risk, not spread uniformly. Two allocation policies are compared in E3: uniform (spread m evenly over regions) vs. matched (allocate m to the regions actually being inherited/exercised this passage). The prediction is that uniform grounding fails to protect a specific inherited region's tail even at the same total m.

(C) Multi-teacher distillation — recombination. Instead of one parent, the pupil is drawn from K_T teachers \{p_t^{(1)}, \dots, p_t^{(K_T)}\}, each of which has its own history and has ground different regions (hence retains different tails). The pupil's inherited draw is taken from the mixture \bar p_t = \frac{1}{K_T}\sum_k p_t^{(k)} (equivalently, n/K_T samples from each teacher). Teacher decorrelation is the controlled quantity: generate teacher sets with a tunable pairwise correlation \rho in which tail items they have retained (see §2.7 for the generative model of correlated teachers). Prediction (E4): tail coverage of the pupil rises as teachers decorrelate, and the mixture's tails are the union of the teachers' tails at \rho = 0 and no better than a single teacher at \rho = 1.

(D) Selection — directional vs. balancing. Between drawing and refitting, a selection operator reweights items by a fitness before the pupil is formed. Two regimes:

• greedy: fitness-proportional (or top-k) selection toward the highest-fitness items — directional selection. Fitness is predictive accuracy against p^\* (a reality-anchored score; see falsifier note). Drives fixation.
• qd (quality-diversity): fitness plus a novelty bonus that is a decreasing function of an item's current frequency — negative frequency-dependent / balancing selection. Formally, effective weight w^i \propto f^i \cdot (p_t^i)^{-\alpha} with novelty exponent \alpha \ge 0; \alpha = 0 recovers greedy. Maintains polymorphism.

Prediction (E5): at matched input diversity, greedy drives heterozygosity to zero (fixation); qd holds it at a positive stationary value.

(E) Re-minting — founder event. Every \tau generations, optionally replace the grounding reference: set p^\*_{\text{eff}} \leftarrow p_t (the lineage's current distribution becomes the new "truth" it is grounded against, modelling assimilation of the soft delta into a new immutable base). Crucially, once re-minted, the original p^\* is discarded — grounding can now only replenish tails that still exist in p_t at re-mint time. A gate conditions re-minting on the diversity metric: only re-mint if H(p_t) \ge H_{\text{gate}}. Prediction (E6): re-minting while collapsed (H low) locks in the collapse irreversibly (KL to the original truth stays high forever); gated re-minting does not.

Full composed step (reference pseudocode).

def generation_step(teachers, p_star_eff, cfg, rng):
    # teachers: list of frequency vectors (length 1 for single-teacher)
    # (C) recombination: mixture over teachers
    p_parent = mean(teachers)                       # or weighted mixture
    # inherited draw (drift, strength n)
    c_syn = rng.multinomial(cfg.n, p_parent)
    # (A,B) grounding: structured immigration from the true distribution
    c_real = structured_multinomial(cfg.m_vector, p_star_eff, regions, cfg.grounding_policy, rng)
    counts = c_syn + c_real
    p_next = counts / counts.sum()
    # (D) selection operator (identity if 'none')
    p_next = apply_selection(p_next, p_star_eff, cfg.selection, cfg.novelty_alpha)
    return normalize(p_next)

Selection is applied after refitting for simplicity; an alternative (select-then-sample) is a documented config switch, and the two should be checked to give qualitatively identical phase behaviour (robustness, not a headline).

2.3 Metrics (computed every generation, logged to disk)

• Forward KL to truth, D_{\mathrm{KL}}(p^\* \,\|\, p_t) = \sum_i p^{\*i}\log(p^{\*i}/p_t^i). This is the correct primary metric: it diverges when p_t drops mass that p^\* has — i.e. it explicitly punishes forgetting the improbable. (Reverse KL would reward mode-seeking; do not use it as the primary.) Use a small floor \epsilon on p_t to keep it finite and log the floor.
• Expected heterozygosity / diversity, H_t = 1 - \sum_i (p_t^i)^2. The lineage-health metric; the quantity the re-mint gate reads.
• Tail mass retained, T_t = \sum_{i \in \text{tail}} p_t^i where the tail set is \{i : p^{\*i} < \theta_{\text{tail}}\}. The direct measure of collapse.
• Support size, |\{i : p_t^i > \epsilon\}|. Number of surviving items.
• Per-region diversity and tail mass, the above restricted to each region (needed for E3).

All metrics are recorded per generation, per replicate (independent seed), so every reported curve carries a confidence band over replicates. Number of replicates is a config value; default 100 for Layer 1 (cheap).

2.4 Analytic predictions — the validation targets

These are the closed forms the simulator must match. They are implemented as assertions in the test suite (§4), so scientific validation and code correctness are one thing.

1. Neutral heterozygosity decay (base case, m=0): \mathbb{E}[H_{t+1}] = (1 - 1/n)\,\mathbb{E}[H_t], hence \mathbb{E}[H_t] = H_0 (1 - 1/n)^t. The simulator's mean H_t over replicates must match this geometric decay within Monte-Carlo error. (This is the quantitative form of "collapse is tail-first and its rate is set by the distillation sample size n.")
2. Fixation probability (base case): the probability that item i is the one eventually fixed equals its initial frequency p_0^i. Checkable by long runs.
3. Mutation–drift equilibrium (with grounding). For the immigration model actually implemented — p_{t+1} = (\mathrm{Multinomial}(n,p_t) + \mathrm{Multinomial}(m,p^\*))/(n+m) — the stationary heterozygosity is exact (from the coupled mean recursions for \sum p_t^2 and the overlap \sum p_t p^\*, whose fixed point is \sum (p^\*)^2):

H_{\text{eq}} = H^\* \cdot \frac{m\,(2n + m - 1)}{\,n + 2nm + m^2\,}, \qquad H^\* = 1 - \textstyle\sum_i (p^{\*i})^2.

Limits: m\to0 \Rightarrow H_{\text{eq}}\to0 (collapse to fixation); m\to\infty \Rightarrow H_{\text{eq}}\to H^\* (the truth's own heterozygosity is recovered); and in the rare-immigrant / many-types limit (H^\*\approx1, m\ll n) it reduces to the textbook infinite-alleles form H_{\text{eq}}\approx \theta/(1+\theta) with \theta = 2m. The simulator's stationary H under grounding must track the exact form as m varies (verified against simulation to <0.1% rel. error; it is a test_scientific_validation.py assertion). (This is the phase boundary in closed form; note H itself is smooth in m — the sharp threshold lives in discrete tail-item survival, prediction 4.) 4. Tail-persistence threshold: an item of true frequency p^{\*i} is maintained against drift only if its expected reintroduction per generation m \cdot p^{\*i} \gtrsim 1. Hence the critical grounding for protecting a tail of rarity p_{\min} is m^\* \approx 1/p_{\min}. (This is why grounding must be region-matched: uniform m spread over R regions gives each region m/R, so a rare region's tail is protected only if m/R \gtrsim 1/p_{\min}^{\text{region}}.) 5. Recombination benefit (Muller's ratchet): a single clonal lineage accumulates irreversible loss at the drift rate; the expected tail coverage of a pupil drawn from K_T teachers with pairwise retention-correlation \rho interpolates between the single-teacher value (\rho = 1) and the union of K_T independent lineages (\rho = 0). With the construction of §2.7.1 (each teacher retains a fraction q of the T tail items, at exact pairwise correlation \rho) the interpolation is closed-form: the expected number of tail items retained by at least one teacher is

U(K_T,\rho,q) = T\left[\rho q + (1-\rho)\left(1-(1-q)^{K_T}\right)\right].

Check the limits: K_T=1 \Rightarrow Tq (single teacher, independent of \rho); \rho=1 \Rightarrow Tq (identical teachers, union = one); \rho=0 \Rightarrow T[1-(1-q)^{K_T}] (independent teachers, maximal union). The pupil's realised tail coverage tracks U up to the drift-survival threshold of prediction 4 — a tail item present in the mixture only survives the pupil's resampling of size n if its mixture mass clears \sim 1/n, which ties E4 back to E2/E3. (Numerically verified against the construction to three decimals; it is a test_scientific_validation.py assertion.)

2.5 Experiments E1–E6

Each experiment is one config file (§4), one runner invocation, one results artifact, and one figure script. Each states its prediction and its falsifier — the outcome that would refute the corresponding claim in the perspective paper.

E1 — Reproduce collapse (null model). m=0, single teacher, no selection. Expect: H_t = H_0(1-1/n)^t; support shrinks to 1; forward KL diverges; tail mass → 0, tail first. Validates against: predictions 1–2. Falsifier of the harness (not the theory): if drift does not reproduce the analytic decay, the simulator is wrong — fix before proceeding.

E2 — Grounding phase boundary (headline). Sweep g = m/(n+m) from 0 to a high value; single teacher; uniform grounding; no selection. Expect: a critical g^\* below which tail mass and H decay to (near) zero and above which they stabilise at a positive stationary value tracking H_{\text{eq}} = \theta/(1+\theta). This is the paper's central quantitative prediction: the ratio of inherited-to-grounded information has a threshold. Falsifier: if the stationary tail mass is flat in g, or if it only stabilises as g \to 1 (you always need essentially all-real data), then inheritance buys nothing and the multigenerational thesis is refuted. Report g^\* with a CI.

E3 — Region-matched grounding. Fixed total m; compare uniform vs. matched allocation across R regions, with one designated "inherited-but-not-freshly-grounded-under-uniform" region carrying a rare tail. Expect: under uniform, the target region's tail collapses even though global grounding is nonzero; under matched, it persists. Validates: prediction 4 and the "grounding must overlap the content it protects" claim. Falsifier: if uniform grounding protects the region as well as matched, the region-matching requirement is unnecessary and that paragraph of the paper should be cut.

E4 — Multi-teacher decorrelation. Teachers built by the §2.7.1 construction; sweep number of teachers K_T \in \{1,2,3,5\} and retention-correlation \rho \in [0,1] (at fixed marginal retention q); matched total data budget (so more teachers ≠ more data — each contributes n/K_T). Report two coverages: the construction-level union U(K_T,\rho,q) (must match the §2.4-5 closed form exactly) and the post-distillation surviving coverage after the pupil's resampling. Expect: both rise with K_T and with (1-\rho); collapse suppression \propto decorrelation; at \rho=1, multiple teachers give no benefit over one; and the gap between union and surviving coverage shrinks as grounding g rises (recombination supplies the tail, grounding holds it). Validates: prediction 5 (now closed-form). Falsifier: if K_T decorrelated teachers give no surviving-coverage benefit over one at matched budget, the recombination claim dies and single-teacher distillation is fine.

E5 — Quality-diversity vs. greedy. Same starting diversity; greedy vs. qd selection; sweep novelty exponent \alpha. Expect: greedy → fixation (H \to 0); qd holds H at a positive plateau and re-introduces lost tail items. Validates: the two-level anti-convergence argument (§7 of paper). Falsifier: if qd does not maintain higher stationary H than greedy, quality-diversity is not doing the work the paper assigns it.

E6 — Re-minting gate (irreversibility). Run a lineage to a chosen diversity level, then re-mint (freeze current p_t as new reference, discard original p^\*); compare re-minting at high H vs. low H. Expect: re-mint while collapsed → KL to original truth is locked high forever (tails unrecoverable); gated re-mint at high H → no lock-in. Validates: §11's "re-minting is irreversible; gate it on diversity." Falsifier: if a collapsed lineage recovers its original-truth tails after re-minting, the irreversibility warning is overstated.

2.6 Layer-1 falsifiers, collected

The model is built to be able to kill the thesis. If E2 shows no threshold, or E4 shows no decorrelation benefit, or E6 shows no lock-in, the corresponding claims are refuted and the paper must say so. A blueprint that cannot fail is not a test. The single load-bearing positive result is E2's phase boundary at g^\* \ll 1: it says a little grounding protects a lot of inheritance, which is the whole economic and conceptual bet of the architecture.

2.7 Implementation spec for Layer 1

Language / libraries. Python ≥ 3.11; NumPy, SciPy (stats), pandas (results), matplotlib (figures). No GPU. No other heavy deps. Everything seedable from a single integer.

Core module interfaces (the coding agent should implement to these signatures; names are normative so downstream scripts are stable):

# knowledge/truth.py
def make_true_distribution(K: int, R: int, tail: str, tail_frac: float,
                           zipf_s: float, seed: int) -> TrueDist:
    """Return p_star (length K), region assignment (length K), and the tail mask."""

# knowledge/teachers.py
def make_retention_matrix(T: int, K_T: int, rho: float, q: float, rng) -> np.ndarray:
    """Return an (K_T, T) binary retention matrix R with exact marginal retention
    E[R]=q and exact pairwise column-correlation rho, via the shared-switch
    construction of §2.7.1. rho=0 -> independent tails; rho=1 -> identical."""

def make_correlated_teachers(p_star, tail_mask, K_T: int, rho: float, q: float,
                             region_assignment=None, region_specialisation=False,
                             tail_floor: float = 1e-9, seed: int = 0) -> list[np.ndarray]:
    """Build K_T teacher distributions from a retention matrix (§2.7.1): every teacher
    keeps all head items; teacher k keeps tail item j at ~p_star mass iff R[k,j]=1,
    else at tail_floor; renormalise. If region_specialisation, force R[k,j]=1 for tail
    items in teacher k's home region and apply the rho construction only off-home.
    The exact-construction path is preferred for E4; a drift-based path (running
    independent grounded lineages) is provided as a realism cross-check only."""

# knowledge/step.py
def generation_step(teachers, p_star_eff, cfg, rng) -> np.ndarray: ...
def apply_selection(p, p_star, mode: str, alpha: float) -> np.ndarray: ...
def structured_multinomial(m_vector, p_star, regions, policy: str, rng) -> np.ndarray: ...

# knowledge/lineage.py
def run_lineage(cfg, seed) -> pd.DataFrame:
    """Run T generations for one seed; return a tidy frame with one row per
    (generation) and columns for every metric in §2.3 (global and per-region)."""

# knowledge/metrics.py
def forward_kl(p_star, p, eps): ...
def heterozygosity(p): ...
def tail_mass(p, tail_mask): ...
def support_size(p, eps): ...

# knowledge/experiment.py
def run_experiment(cfg) -> pd.DataFrame:
    """Sweep the declared parameter grid x n_replicates seeds; return long-form
    results with confidence intervals; write parquet + the exact resolved config."""

Config schema (one YAML per experiment; all parameters explicit, no magic numbers in code). Illustrative default:

experiment: E2_grounding_phase_boundary
seed: 20260704
n_replicates: 100
generations: 300
truth:
  K: 1000            # number of knowledge items
  R: 10              # regions
  tail: zipf         # {zipf, twocomponent}
  zipf_s: 1.1
  tail_frac: 0.5     # fraction of items designated 'tail'
  tail_threshold: 1.0e-3
dynamics:
  n: 200             # distillation sample size (drift strength)
  teachers:
    K_T: 1
    rho: 0.0
  grounding:
    sweep: {param: g, values: [0.0, 0.01, 0.02, 0.05, 0.1, 0.2, 0.4]}
    policy: uniform  # {uniform, matched}
  selection:
    mode: none       # {none, greedy, qd}
    novelty_alpha: 0.0
  remint:
    enabled: false
    period: null
    H_gate: null
metrics:
  kl_floor: 1.0e-9
output:
  dir: results/E2/
  save_per_generation: true

Determinism. One master seed → per-replicate seeds via a documented np.random.SeedSequence spawn. No global RNG state; pass rng explicitly everywhere. Results are a pure function of the resolved config. The resolved config (after sweep expansion) is written next to the results artifact.

Outputs. Each experiment writes: results.parquet (long form), resolved_config.yaml, and a manifest.json recording library versions, seed, git commit, and a content hash of the results. Figures are generated by a separate figures/plot_EX.py reading only results.parquet, so any figure is regenerable from committed data without rerunning the simulation.

2.7.1 The correlated-teacher construction (E4's one non-obvious piece)

E4 needs teacher sets whose pairwise retention-correlation \rho is a clean, swept knob. Tuning the drift parameters until an emergent \rho appears is the wrong design: that \rho would be a tangled function of n, m, tail size and generation count, un-sweepable and confounded with the very drift whose effect E4 is trying to hold fixed. So we construct the retention structure directly, with \rho an independent control. For an experiment whose entire purpose is to isolate the effect of decorrelation, \rho must be manipulated, not inferred.

The mechanism — shared-switch exchangeable Bernoulli. Let the tail have T items. Each teacher k retains a fraction q of them; we want the retention indicators to have marginal q and exact pairwise correlation \rho across teachers. For each tail item j:

1. draw a shared switch z_j \sim \mathrm{Bernoulli}(\rho) (one draw, common to all teachers for item j);
2. draw a shared retention s_j \sim \mathrm{Bernoulli}(q) (common to all teachers);
3. draw independent retentions u^{(k)}_j \sim \mathrm{Bernoulli}(q), one per teacher;
4. set r^{(k)}_j = s_j if z_j = 1, else u^{(k)}_j.

That is the whole construction. It yields a retention matrix R \in \{0,1\}^{K_T \times T}.

Why it is exact. Marginally \mathbb{E}[r^{(k)}_j] = \rho q + (1-\rho)q = q regardless of \rho. For any two teachers, conditioning on the switch gives \mathbb{E}[r^{(k)}_j r^{(k')}_j] = \rho\,\mathbb{E}[s_j^2] + (1-\rho)q^2 = \rho q + (1-\rho)q^2 (using s_j^2 = s_j), so \mathrm{Cov} = \rho q(1-q) and, since \mathrm{Var} = q(1-q), the pairwise correlation is exactly \rho. The set is exchangeable — every teacher pair has the same \rho — which is what makes \rho a single scalar knob. (Verified numerically: marginal and pairwise correlation hit target across a (\rho, q) grid, and the union closed form below matches to three decimals.)

From retention to teacher distributions. Given R, build teacher $k$'s distribution p^{(k)}: assign every head item its p^\* mass (all teachers keep the common core); assign tail item j its p^\*_j mass if r^{(k)}_j = 1 and a floor \varepsilon_{\text{tail}} otherwise; renormalise. (Renormalising lets the mass of dropped tails flow to what the teacher kept — the realistic signature of a partially-collapsed model concentrating on its survivors. Reallocating instead to the head is a documented config switch.)

Region specialisation (structured decorrelation, optional). With region_specialisation=True, give each teacher a home region and force r^{(k)}_j = 1 for every tail item j in teacher $k$'s home region (each teacher fully retains the tails of the region it ground against reality), applying the \rho construction only to off-home tail items. This models "each teacher ground a different region," ties E4 to E3's region-matched grounding, and is the discrete image of the perspective paper's "distil from teachers who each earned a different region."

The analytic target (E4's exact check). The probability that tail item j is retained by at least one of K_T teachers is \rho q + (1-\rho)\big(1-(1-q)^{K_T}\big), so the expected union tail-coverage is

U(K_T,\rho,q) = T\left[\rho q + (1-\rho)\left(1-(1-q)^{K_T}\right)\right],

the closed form of §2.4-5. make_retention_matrix must reproduce it (and the target \rho, q) within Monte-Carlo tolerance in test_scientific_validation.py. The pupil's realised coverage after distillation equals U only for tail items whose mixture mass clears the drift-survival threshold \sim 1/n (§2.4-4) — so E4 should report both the union coverage (construction-level) and the post-distillation surviving coverage (dynamics-level), and their gap is itself informative: it is exactly the tail that recombination supplied but drift re-erased because grounding was too thin to hold it.

Continuous-mass extension (optional realism, not a headline). Binary retention is the default because it matches prediction 5 exactly. For a realism cross-check, replace binary retention with correlated masses: draw per-teacher log-masses on tail items from a multivariate normal with equicorrelation \rho (a Gaussian copula), exponentiate, and normalise. This generalises to an arbitrary correlation matrix between teachers (unequal pairwise \rho), which the shared-switch construction — being exchangeable — cannot express; use it only if the paper later needs non-exchangeable teacher sets.

Optional abstract treatment of the horizontal claim (§2.7-H, low priority). If time permits, model a domain as a modular graph with tunable modularity Q; define the optimal number of concurrent specialists as the point where marginal coverage gain from an added specialist falls below a cost; show optimal specialist count rises with Q. This is the horizontal prediction in abstract form and does not require the neural layer. Flag clearly as exploratory; it is not a headline and can be dropped without weakening the paper.

---

3. Layer 2 — the neural existence proof

3.1 Purpose and the single objection it answers

Layer 1 assumes the tail-deletion operator (drift). A reviewer will say: you built your conclusion into the operator. Layer 2 answers exactly that objection and no more. It does not need to be a society. It needs to show, in real LoRA-adapted weights, that the sign of three effects is as Layer 1 predicts: dry inheritance degrades, grounded inheritance holds; multiple decorrelated teachers preserve capability that one teacher sheds; and across generations general capability holds or climbs while each specialty is re-earned and exceeded. If those three signs appear at 1B scale on one GPU, the abstraction in Layer 1 is grounded in mechanism.

Minimality is a virtue here, not a compromise. Every additional degree of freedom (bigger model, more generations, more domains) multiplies cost and reviewer surface without strengthening the core claim. Build the smallest thing that can show the signs.

3.2 Design choices (with open-science defaults)

Base model. Default to a fully open small model to honour reproducibility end-to-end: OLMo-2-1B or SmolLM2-1.7B (open weights, open or well-documented data, permissive licence). Capable fallback if the open models are too weak on the task: Qwen2.5-1.5B-Instruct. Pin the exact Hugging Face revision hash in config; never track main. The pipeline must be model-agnostic behind a thin adapter so swapping bases is a config change.

Specialisation. LoRA (via PEFT), small rank (e.g. 8–16), on a single task family = one "region." Hours on one consumer GPU, consistent with the perspective paper's cost claim. Each teacher is one base + one LoRA adapter.

The verifier = "reality's no." The domain must have a cheap, deterministic, uncontrollable oracle. Program synthesis with unit tests is ideal: a generated solution either passes its tests or it does not, and neither the model nor the experimenter controls the verdict. This is the minimal honest instance of the perspective paper's "predictive success under intervention." Execution happens in a sandbox (subprocess with a hard timeout and no network, run inside the container of §4; document the sandbox precisely).

Task domain — synthetic-primary, benchmark-secondary. To keep the Layer-1 abstractions (region, rarity, tail) exact, the primary domain is a synthetic program-synthesis generator: a family of small, verifiable tasks parameterised by operation type (the region: e.g. string ops, list ops, arithmetic, dict manipulation, recursion) and difficulty, with I/O unit tests generated automatically. Task-type frequency is tunable, so "rare tail task types" is a dial, exactly matching Layer 1's rarity. For external validity, add a secondary evaluation on a held-out slice of a public benchmark (e.g. MBPP-sanitised / HumanEval) — pinned by version — to show the effect is not an artefact of the synthetic generator. The synthetic generator's spec (grammar, per-region templates, test-generation rule, rarity distribution) is itself a committed, seeded artifact.

Grounding, concretely. A generational passage produces pupil training data as a mixture of (i) inherited teacher-generated solutions and (ii) grounded solutions that have been filtered to pass the verifier and/or freshly drawn from verified references in the target region. The grounding fraction $g$ is the proportion of verifier-passed/real items in the pupil's training mixture — the same knob as Layer 1's g. "Dry" = g low / unfiltered teacher output; "grounded" = g raised with region-matched verified data.

Recombination, concretely. Multiple teachers = multiple LoRA specialists on disjoint task families (decorrelated by construction). The pupil is trained on the pooled outputs of all teachers (distillation) or, as a cheaper alternative, the specialists are merged (M2N2 / model-merge style, citing Sakana's demonstration that this search runs with no retraining) and the merged model is the pupil. Distillation and merging are two config-selectable recombination operators; report at least distillation, and merging if compute allows.

3.3 Contrasts C1–C4 (the whole experimental content)

C1 — Dry vs. grounded, single teacher, across generations. One teacher, one region. Arm A: pupil trained on unfiltered teacher outputs (dry, g\approx0). Arm B: pupil trained on verifier-passed teacher outputs plus fresh verified region data (g>0). Run 2–3 generations. Expect: Arm A pass@k degrades generation over generation and its solution diversity narrows; Arm B holds. Maps to: Layer-1 E2. Falsifier: if dry inheritance does not degrade at this scale, collapse is not reachable here and the neural claim is unsupported (report honestly; possibly scale down grounding or up generations).

C2 — One teacher vs. N complementary teachers, matched data budget. Fix the pupil's total training-token budget. Arm A: all budget from one teacher (one region). Arm B: same budget split across N teachers on disjoint regions. Evaluate the pupil on all regions, including rare-type tasks. Expect: Arm B retains capability across the union of regions (the "tail" of rare task types survives); Arm A loses regions it did not inherit. Maps to: Layer-1 E4. Falsifier: no union benefit at matched budget → recombination claim unsupported neurally.

C3 — The vertical claim (load-bearing). Over 2–3 generations, track (a) a general held-out benchmark spanning all regions and (b) per-specialty performance, where each generation re-specialises (re-earns) its region against the verifier. Expect: general benchmark is monotonically non-decreasing across generations, while each specialty is re-earned and its peak exceeds the parent's peak in that specialty. This is the test of the actual thesis, not the borrowed Shumailov scaffolding. Maps to: the paper's vertical prediction. Falsifier: if general capability falls across generations, or specialties are not re-earned-and-exceeded, the multigenerational ratchet is not demonstrated.

C4 — Recombination operator: distillation vs. merging (optional). If compute allows, show C2's benefit is robust to whether recombination is done by distillation or by weight-merging. Strengthens generality and directly connects to the M2N2 citation. Drop first if time-constrained.

3.4 Metrics (Layer 2)

• pass@1 and pass@k on held-out tasks, per region and overall (the neural analogue of KL-to-truth / capability).
• Region coverage = fraction of regions (incl. rare task types) with pass@1 above a floor — the neural analogue of support size / tail survival.
• Solution diversity = distinct-n or behavioural diversity of generated solutions per task (the neural analogue of heterozygosity H; narrowing diversity is the neural signature of collapse).
• Generational degradation curve = each metric as a function of generation index, per arm.

All with multiple seeds (fewer than Layer 1 — GPU cost — but at least 3; report per-seed points, not just means, given small n). Pin decoding parameters (temperature, top-p, max tokens, sampling seed) in config; they materially affect pass@k and must not float.

3.5 Layer-2 honesty riders

GPU non-determinism means Layer 2 is statistically reproducible (same distribution of outcomes under re-run), not bitwise reproducible. Document this explicitly; pin everything pinnable (model revision, dataset version, decoding params, library versions, seeds); report seeds individually. The claim Layer 2 supports is directional ("the sign of the effect is as predicted"), and the writing must not overclaim precision the setup cannot deliver.

3.6 Implementation spec for Layer 2

Libraries. PyTorch; Hugging Face transformers + peft (LoRA); datasets; optional vllm for fast generation; the synthetic task generator (project-local). Execution sandbox: subprocess with resource limits and timeout, inside the container.

Module interfaces (normative names):

# neural/tasks.py
def generate_task_bank(regions, rarity, n_tasks, seed) -> TaskBank:
    """Synthetic verifiable tasks; each task carries prompt, region, difficulty,
    and an executable unit-test suite."""
def verify(solution_code: str, task) -> VerifyResult:
    """Run tests in a sandbox; return pass/fail + diagnostics. Deterministic."""

# neural/specialise.py
def train_lora_specialist(base_id, revision, region, task_bank, lora_cfg, seed) -> AdapterPath: ...

# neural/distill.py
def generate_teacher_data(teachers, task_bank, grounding_fraction, policy, decode_cfg, seed) -> Corpus:
    """Produce the pupil's training corpus: mixture of inherited (teacher) and
    grounded (verifier-passed / fresh-verified) items, region-matched."""
def train_pupil(base_id, revision, corpus, lora_cfg, seed) -> AdapterPath: ...

# neural/merge.py
def merge_specialists(base_id, adapters, method, seed) -> ModelPath:  # optional (C4)

# neural/evaluate.py
def evaluate(model, task_bank_heldout, decode_cfg, seed) -> pd.DataFrame:
    """pass@k, per-region coverage, solution diversity; tidy per-(region) frame."""

# neural/generation_loop.py
def run_generations(cfg, seed) -> pd.DataFrame:
    """Orchestrate T generations for one arm; log every metric per generation."""

Config mirrors Layer 1's structure (one YAML per contrast; pinned base_id + revision; explicit grounding_fraction, n_teachers, regions, generations, decode_cfg, lora_cfg, seed, n_seeds). Same output contract: results.parquet + resolved_config.yaml + manifest.json (with model revision hashes and dataset versions). Figures regenerable from results.parquet alone.

---

4. Reproducibility and engineering standard (both layers)

Open science is a hard requirement of this project, not a preference. The standard below is normative.

Environment. Pin everything. Provide (a) a pyproject.toml + lockfile via uv (fast, reproducible resolver) and (b) an Apptainer/Singularity definition file (HPC-friendly, rootless, open) that builds the exact environment; optionally a Dockerfile. The container is the source of truth for "it runs." Record Python, CUDA, and key library versions in every run's manifest.json.

Seeding. One master seed per experiment in config; derive all sub-seeds via SeedSequence.spawn; never touch global RNG state. Layer 1 is bitwise-reproducible from seed. Layer 2 is statistically reproducible; document the residual GPU non-determinism and set the available determinism flags (torch.use_deterministic_algorithms(True) where feasible, cudnn deterministic, documented exceptions).

Configuration. No magic numbers in code — every parameter lives in a YAML resolved at run time; the resolved config is written beside results. Use a single config system (Hydra or a thin equivalent). Sweeps are declared in config, not hard-coded in scripts.

Data & model provenance. Layer 1 data is synthetic-from-seed (fully reproducible; no external data). Layer 2 pins model revision hashes and dataset versions; the synthetic task generator is committed and seeded. Cache external downloads with recorded hashes.

Experiment tracking. Prefer open tooling: MLflow (open source) or plain versioned CSV/Parquet + committed configs; avoid closed SaaS trackers to keep the pipeline fully open. Whatever is chosen, the invariant is: every figure is a pure function of a committed results artifact.

Testing. pytest. Two kinds of test, and both are required:

• Correctness tests: the module does what it says (shapes, normalisation, sandbox isolation).
• Scientific validation tests: the simulator reproduces the §2.4 analytic results within tolerance (heterozygosity decay, fixation probability, mutation–drift equilibrium). These tests failing means the science is wrong, not just the code — they are the spine of trust in Layer 1.

Automation. A Makefile (or justfile) with targets: env, test, layer1 (runs E1–E6), layer2 (runs C1–C3, C4 optional), figures, paper (assembles the figure manifest), all, clean. One command reproduces the study from a clean checkout inside the container.

Repro entry point. A top-level reproduce.sh that: builds/enters the container, runs tests, runs all experiments at the committed seeds, regenerates all figures, and writes a REPRODUCED.md diff against committed result hashes. If hashes match (Layer 1) / distributions match within CI (Layer 2), the run is verified.

Licensing & citation. Author to choose; suggested: code under a permissive OSI licence (MIT/Apache-2.0) or copyleft (GPL-3.0) per the author's open-source preference; text/figures under CC-BY. Include LICENSE, CITATION.cff, and a DATA_AND_MODELS.md recording every external artifact and its pinned version/hash.

---

5. Repository layout

lamarckian-society/
├── README.md                  # what this is, how to reproduce
├── reproduce.sh               # one-command full reproduction (in-container)
├── Makefile                   # env, test, layer1, layer2, figures, all
├── pyproject.toml             # deps
├── uv.lock                    # pinned resolution
├── apptainer.def              # container definition (source of truth)
├── Dockerfile                 # optional
├── LICENSE  CITATION.cff  DATA_AND_MODELS.md
├── configs/
│   ├── layer1/E1..E6.yaml
│   └── layer2/C1..C4.yaml
├── src/
│   ├── knowledge/             # Layer 1: truth, teachers, step, lineage, metrics, experiment
│   └── neural/                # Layer 2: tasks, specialise, distill, merge, evaluate, generation_loop
├── figures/                   # plot_EX.py / plot_CX.py — read results.parquet only
├── results/                   # written artifacts (gitignored, hashes tracked)
├── tests/
│   ├── test_correctness.py
│   └── test_scientific_validation.py   # §2.4 analytic checks
└── paper/
    ├── blueprint.md           # this document
    └── figure_manifest.md     # claim -> experiment -> figure

---

6. Traceability: claim → experiment → figure

Every claim the technical paper will make is bound to exactly one experiment and one figure. This table is the paper's spine and the coding agent's definition of done.

Perspective-paper claim	Layer	Experiment	Primary figure	Analytic check	Falsifier
Distillation without grounding collapses, tail first, at rate set by n	1	E1	H_t decay + tail mass vs. gen	Pred. 1–2	harness invalid if decay ≠ analytic
A critical grounding ratio g^\* separates ratchet from collapse	1	E2	tail mass / H stationary vs. g	Pred. 3	flat in g, or only stable at g\to1
Grounding must be region-matched, not merely present	1	E3	target-region tail: uniform vs. matched	Pred. 4	uniform protects as well as matched
Multi-teacher distillation suppresses collapse ∝ decorrelation	1	E4	tail coverage surface over (K_T,\rho)	Pred. 5 (closed form U(K_T,\rho,q))	no benefit at matched budget
QD selection maintains diversity where greedy fixes it	1	E5	H plateau: qd vs. greedy	(mutation-selection-drift)	qd ≤ greedy in stationary H
Re-minting is irreversible; gate on diversity	1	E6	KL-to-original vs. $H$-at-remint	—	collapsed lineage recovers after remint
Dry inheritance degrades in real weights; grounded holds	2	C1	pass@k vs. gen, dry vs. grounded	—	dry does not degrade
Complementary teachers preserve capability one teacher sheds	2	C2	region coverage, 1 vs. N, matched budget	—	no union benefit
General knowledge climbs while each specialty is re-earned and exceeded	2	C3	general benchmark ↑ + per-specialty re-earn	—	general falls / no re-earn
Recombination benefit robust to distillation vs. merging	2	C4 (opt)	C2 metric under both operators	—	benefit only under one operator

---

7. Suggested build order for the coding agent

Staged so that each step is independently testable and the cheapest, highest-value results land first. Do not start Layer 2 until Layer 1's scientific-validation tests pass.

1. Scaffold + environment. Repo layout (§5), container (§4), pytest skeleton, config system, seeding utilities. Target make test green on trivial tests.
2. Layer 1 core + validation. knowledge/ modules to the §2.7 interfaces. Implement the null model first. Write and pass test_scientific_validation.py against §2.4 predictions 1–2. Gate: do not proceed until drift matches analytic decay.
3. Layer 1 mechanisms + E1–E2. Add grounding; validate mutation–drift equilibrium (pred. 3); run E1 and the E2 phase-boundary sweep; produce the headline figure. This is the paper's core result and it should exist before anything neural.
4. Layer 1 E3–E6. Region-matching, multi-teacher/decorrelation, QD-vs-greedy, re-minting gate, with figures. Layer 1 is now a complete, laptop-reproducible paper on its own.
5. Layer 2 scaffold + verifier. Synthetic task generator, sandboxed verify, evaluation harness. Test the verifier's determinism and isolation before any training.
6. Layer 2 C1 + C3. Single-teacher dry-vs-grounded (C1) and the vertical claim (C3) — the two that most directly test the thesis. C3 is load-bearing; prioritise it.
7. Layer 2 C2 (+ C4 if compute allows). Multi-teacher recombination; optional merging operator.
8. Reproduction pass. reproduce.sh end-to-end; commit result hashes; write REPRODUCED.md; assemble the figure manifest.

Definition of done: every row of §6 has a committed figure produced by make figures from committed results, every §2.4 analytic check passes in CI, and reproduce.sh verifies from a clean checkout inside the container.

---

End of blueprint v1. The perspective paper states the idea; this document states the test. If Layer 1's E2 finds no threshold, or E4 no decorrelation benefit, or Layer 2's C3 no vertical climb, the thesis is wrong in exactly the places these experiments probe — which is the point of writing them down this precisely.