# A Technical Blueprint for Modelling the Lamarckian Society

### An implementation specification: two layers, one population-genetics engine

*Companion technical paper to "The Lamarckian Society." Blueprint v1 — written to be handed, whole, to an autonomous coding agent (Claude Code) for implementation. Every quantitative claim in the perspective paper is reduced here to a state variable, an update rule, an analytic prediction, and a falsifier.*

---

## 0. How to use this document

This is a build specification, not an essay. It is written so that a coding agent can implement the entire study from it with minimal further decisions, and so that a human reader can verify every modelling choice against the theory it is meant to test.

**Scope.** Two layers, deliberately staged by cost:

- **Layer 1 — the analytical core.** A parametric population-genetics model of knowledge transmission across generations. Pure NumPy/SciPy. Runs on a laptop in minutes. This layer carries the paper's quantitative claims: the grounding phase boundary, the decorrelation curve, region-matched grounding, quality-diversity vs. greedy selection, and the re-minting gate. Several of its predictions are analytically solvable, which turns validation into an exact test rather than a vibe.
- **Layer 2 — the neural existence proof.** A minimal demonstration that the same effects appear in *real weights*: small open-weight models, LoRA specialisation, distillation/merging across two–three generations, with an execution-based verifier standing in for "reality's no." One consumer GPU. This layer answers the single most predictable reviewer objection to Layer 1 ("you assumed the collapse operator") by showing the sign of the key effects without assuming them.

**The relationship between layers.** Layer 1 defines the abstractions (region, rarity, grounding fraction, teacher decorrelation, diversity metric). Layer 2 realises the *same abstractions* in a neural pipeline: a "region" is a task family, "rarity" is task-type frequency, "grounding" is verifier-passed samples, "decorrelation" is teachers specialised on disjoint task families. Keeping the abstractions identical across layers is a hard requirement — it is what lets a Layer-2 result be read as confirming a Layer-1 prediction.

**Non-goals for v1.** No human-in-the-loop interaction (the verifier is the refuter; humans are future work). No foundation-scale training. No claim about the *horizontal* prediction (generation size vs. domain decomposability) beyond an optional abstract treatment in §2.7 — it is the most compute-hungry claim and is explicitly deferred. The load-bearing target is the *vertical* claim (general knowledge climbs while each specialty is re-earned and exceeded).

**Reader's map.** §1 gives the formal dictionary between population genetics and knowledge transmission — read this first; everything else is an instantiation of it. §2 is Layer 1 in full (theory, experiments, code interfaces). §3 is Layer 2. §4 is the shared reproducibility standard. §5 is the repository layout. §6 is the claims→experiments→figures traceability matrix. §7 is the suggested build order for the coding agent.

---

## 1. The formal mapping: knowledge transmission *is* a Wright–Fisher process

The perspective paper argues by analogy that generational distillation resembles genetic drift, that multi-teacher distillation resembles recombination, and that heterodox input plus quality-diversity selection resembles mutation supply plus balancing selection. This blueprint drops the word "resembles." At the level of a distribution over discrete items evolving by finite resampling, these are not analogies; they are the *same stochastic process*, and the population-genetics literature has already solved large parts of it. We therefore adopt Wright–Fisher dynamics as the engine and inherit its exact results as our validation targets.

The core object is a **distribution over discrete knowledge items** — facts, capabilities, behaviours, or "modes." Call the items $1, \dots, K$. A model at generation $t$ holds a distribution $p_t = (p_t^1, \dots, p_t^K)$ on the simplex. There is a fixed **true distribution** $p^\* $ over the same items, some of which are rare (the *tail* — the improbable events whose loss defines model collapse).

The generational step is: sample from the parent, optionally mix in fresh real samples, refit. That single step is Wright–Fisher reproduction with immigration. Here is the dictionary, and it is meant to be used literally throughout implementation:

| Knowledge-transmission concept (perspective paper) | Population-genetics object (this blueprint) | Governs |
|---|---|---|
| Knowledge item / capability / mode | Allele / type | State space |
| Model's knowledge distribution $p_t$ | Allele-frequency vector | State |
| True distribution $p^\*$ (with rare tail) | Ancestral/immigrant frequencies | Grounding target |
| Distillation sample size $n$ | Population size $N$ | Drift strength ($\propto 1/n$) |
| Lossy compression / "shed the idiosyncratic" | Genetic drift | Tail-first loss |
| Model collapse (tail lost first) | Loss of rare alleles under drift | The central failure |
| Grounding: $m$ fresh real samples per passage | Immigration / mutation supply | Tail replenishment |
| "No dry inheritance," region-matched | Immigration structured by locus | Which tails are protected |
| Multi-teacher distillation | Recombination across lineages | Reconstitutes lost tails |
| Teacher decorrelation $\rho$ | Linkage / shared ancestry | Recombination benefit |
| Single-teacher irreversible error | Muller's ratchet (clonal lineage) | Why one teacher is unsafe |
| Diversity metric (§7 of paper) | Expected heterozygosity $H = 1 - \sum_i p_i^2$ | Health of the lineage |
| Greedy fitness-max selection | Directional selection → fixation | Accelerated collapse |
| Quality-diversity / novelty selection | Balancing / negative frequency-dependent selection | Maintained polymorphism |
| Re-minting the base (§11 of paper) | Founder event / new reference genome | Irreversibility |
| Heterodox contributor supply | Standing variation of founding stock | Raw material |

Every experiment below is a manipulation of this one process. The value of the mapping is that it hands us closed-form predictions — heterozygosity decay, mutation–drift equilibrium, fixation probabilities — against which the simulator must agree before any headline result is trusted. Validation and theory are the same equations.

---

## 2. Layer 1 — the analytical core

### 2.1 State and the base dynamics (neutral drift = Shumailov collapse)

**State.** A single lineage is a point $p_t$ on the $K$-simplex. Items are partitioned into $R$ **regions** (disjoint blocks of the $K$ items); regions are how we express "different areas of knowledge," and they are what grounding and specialisation are *targeted at*. The true distribution $p^\*$ is fixed and chosen to have a deliberate **tail**: most probability mass on common items, a long thin tail of rare items (e.g. a Zipfian or a two-component mixture of "head" and "tail" items, tunable).

**The generational step, base case (no grounding, single teacher).** Given parent $p_t$ and drift strength $n$:

1. Draw counts $c \sim \mathrm{Multinomial}(n, p_t)$.
2. Set $p_{t+1}^i = c^i / n$.

This is exactly neutral Wright–Fisher with haploid population size $n$. It is also exactly Shumailov's recursive resampling with a perfect refit. Rare items are lost first (once $c^i = 0$, item $i$ is gone and — with no grounding — cannot return); the lineage drifts to fixation on a single item.

**This is the null model and the first validation.** It must reproduce, within Monte-Carlo error, the classical drift results in §2.4. If it does not, nothing downstream is trustworthy.

### 2.2 The five mechanisms, each as an operator on the step

Each perspective-paper safeguard is one modification of the generational step. They compose; the full step applies them in the order below.

**(A) Grounding — immigration / mutation supply.** Fresh real data enters every passage. Replace the single draw with a *pooled* draw:

- Draw $c_{\text{syn}} \sim \mathrm{Multinomial}(n, p_t)$ (inherited / teacher output).
- Draw $c_{\text{real}} \sim \mathrm{Multinomial}(m, p^\*)$ (grounding).
- $p_{t+1}^i = (c_{\text{syn}}^i + c_{\text{real}}^i)/(n+m)$.

Define the **grounding fraction** $g = m/(n+m)$. This is the single most important control variable in the paper: it is the ratio of freshly-grounded to inherited information per passage. The claim "no dry inheritance" is $g > 0$; the claim that a *critical* $g$ exists is the phase-boundary experiment E2.

**(B) Region-matched grounding — structured immigration.** Grounding is a vector, not a scalar: $m = (m_1, \dots, m_R)$ real samples allocated across regions. "No dry inheritance, region by region" means grounding must be delivered *to the regions whose tails are at risk*, not spread uniformly. Two allocation policies are compared in E3: `uniform` (spread $m$ evenly over regions) vs. `matched` (allocate $m$ to the regions actually being inherited/exercised this passage). The prediction is that uniform grounding fails to protect a specific inherited region's tail even at the same total $m$.

**(C) Multi-teacher distillation — recombination.** Instead of one parent, the pupil is drawn from $K_T$ teachers $\{p_t^{(1)}, \dots, p_t^{(K_T)}\}$, each of which has its own history and has ground different regions (hence retains different tails). The pupil's inherited draw is taken from the mixture $\bar p_t = \frac{1}{K_T}\sum_k p_t^{(k)}$ (equivalently, $n/K_T$ samples from each teacher). Teacher **decorrelation** is the controlled quantity: generate teacher sets with a tunable pairwise correlation $\rho$ in *which tail items they have retained* (see §2.7 for the generative model of correlated teachers). Prediction (E4): tail coverage of the pupil rises as teachers decorrelate, and the mixture's tails are the *union* of the teachers' tails at $\rho = 0$ and no better than a single teacher at $\rho = 1$.

**(D) Selection — directional vs. balancing.** Between drawing and refitting, a selection operator reweights items by a fitness before the pupil is formed. Two regimes:

- `greedy`: fitness-proportional (or top-$k$) selection toward the highest-fitness items — directional selection. Fitness is predictive accuracy against $p^\*$ (a reality-anchored score; see falsifier note). Drives fixation.
- `qd` (quality-diversity): fitness *plus* a novelty bonus that is a decreasing function of an item's current frequency — negative frequency-dependent / balancing selection. Formally, effective weight $w^i \propto f^i \cdot (p_t^i)^{-\alpha}$ with novelty exponent $\alpha \ge 0$; $\alpha = 0$ recovers greedy. Maintains polymorphism.

Prediction (E5): at matched input diversity, `greedy` drives heterozygosity to zero (fixation); `qd` holds it at a positive stationary value.

**(E) Re-minting — founder event.** Every $\tau$ generations, optionally replace the grounding reference: set $p^\*_{\text{eff}} \leftarrow p_t$ (the lineage's *current* distribution becomes the new "truth" it is grounded against, modelling assimilation of the soft delta into a new immutable base). Crucially, once re-minted, the *original* $p^\*$ is discarded — grounding can now only replenish tails that still exist in $p_t$ at re-mint time. A **gate** conditions re-minting on the diversity metric: only re-mint if $H(p_t) \ge H_{\text{gate}}$. Prediction (E6): re-minting while collapsed ($H$ low) locks in the collapse irreversibly (KL to the *original* truth stays high forever); gated re-minting does not.

**Full composed step (reference pseudocode).**

```
def generation_step(teachers, p_star_eff, cfg, rng):
    # teachers: list of frequency vectors (length 1 for single-teacher)
    # (C) recombination: mixture over teachers
    p_parent = mean(teachers)                       # or weighted mixture
    # inherited draw (drift, strength n)
    c_syn = rng.multinomial(cfg.n, p_parent)
    # (A,B) grounding: structured immigration from the true distribution
    c_real = structured_multinomial(cfg.m_vector, p_star_eff, regions, cfg.grounding_policy, rng)
    counts = c_syn + c_real
    p_next = counts / counts.sum()
    # (D) selection operator (identity if 'none')
    p_next = apply_selection(p_next, p_star_eff, cfg.selection, cfg.novelty_alpha)
    return normalize(p_next)
```

Selection is applied after refitting for simplicity; an alternative (select-then-sample) is a documented config switch, and the two should be checked to give qualitatively identical phase behaviour (robustness, not a headline).

### 2.3 Metrics (computed every generation, logged to disk)

- **Forward KL to truth**, $D_{\mathrm{KL}}(p^\* \,\|\, p_t) = \sum_i p^{\*i}\log(p^{\*i}/p_t^i)$. This is the correct primary metric: it *diverges* when $p_t$ drops mass that $p^\*$ has — i.e. it explicitly punishes forgetting the improbable. (Reverse KL would reward mode-seeking; do not use it as the primary.) Use a small floor $\epsilon$ on $p_t$ to keep it finite and log the floor.
- **Expected heterozygosity / diversity**, $H_t = 1 - \sum_i (p_t^i)^2$. The lineage-health metric; the quantity the re-mint gate reads.
- **Tail mass retained**, $T_t = \sum_{i \in \text{tail}} p_t^i$ where the tail set is $\{i : p^{\*i} < \theta_{\text{tail}}\}$. The direct measure of collapse.
- **Support size**, $|\{i : p_t^i > \epsilon\}|$. Number of surviving items.
- **Per-region diversity and tail mass**, the above restricted to each region (needed for E3).

All metrics are recorded per generation, per replicate (independent seed), so every reported curve carries a confidence band over replicates. Number of replicates is a config value; default 100 for Layer 1 (cheap).

### 2.4 Analytic predictions — the validation targets

These are the closed forms the simulator must match. They are implemented as assertions in the test suite (§4), so scientific validation and code correctness are one thing.

1. **Neutral heterozygosity decay** (base case, $m=0$): $\mathbb{E}[H_{t+1}] = (1 - 1/n)\,\mathbb{E}[H_t]$, hence $\mathbb{E}[H_t] = H_0 (1 - 1/n)^t$. The simulator's mean $H_t$ over replicates must match this geometric decay within Monte-Carlo error. *(This is the quantitative form of "collapse is tail-first and its rate is set by the distillation sample size $n$.")*
2. **Fixation probability** (base case): the probability that item $i$ is the one eventually fixed equals its initial frequency $p_0^i$. Checkable by long runs.
3. **Mutation–drift equilibrium** (with grounding). For the immigration model actually implemented — $p_{t+1} = (\mathrm{Multinomial}(n,p_t) + \mathrm{Multinomial}(m,p^\*))/(n+m)$ — the stationary heterozygosity is **exact** (from the coupled mean recursions for $\sum p_t^2$ and the overlap $\sum p_t p^\*$, whose fixed point is $\sum (p^\*)^2$):
$$H_{\text{eq}} = H^\* \cdot \frac{m\,(2n + m - 1)}{\,n + 2nm + m^2\,}, \qquad H^\* = 1 - \textstyle\sum_i (p^{\*i})^2.$$
Limits: $m\to0 \Rightarrow H_{\text{eq}}\to0$ (collapse to fixation); $m\to\infty \Rightarrow H_{\text{eq}}\to H^\*$ (the truth's own heterozygosity is recovered); and in the rare-immigrant / many-types limit ($H^\*\approx1$, $m\ll n$) it reduces to the textbook infinite-alleles form $H_{\text{eq}}\approx \theta/(1+\theta)$ with $\theta = 2m$. The simulator's stationary $H$ under grounding must track the **exact** form as $m$ varies (verified against simulation to <0.1% rel. error; it is a `test_scientific_validation.py` assertion). *(This is the phase boundary in closed form; note $H$ itself is smooth in $m$ — the sharp threshold lives in discrete tail-item survival, prediction 4.)*
4. **Tail-persistence threshold**: an item of true frequency $p^{\*i}$ is maintained against drift only if its expected reintroduction per generation $m \cdot p^{\*i} \gtrsim 1$. Hence the critical grounding for protecting a tail of rarity $p_{\min}$ is $m^\* \approx 1/p_{\min}$. *(This is why grounding must be region-matched: uniform $m$ spread over $R$ regions gives each region $m/R$, so a rare region's tail is protected only if $m/R \gtrsim 1/p_{\min}^{\text{region}}$.)*
5. **Recombination benefit** (Muller's ratchet): a single clonal lineage accumulates irreversible loss at the drift rate; the expected tail coverage of a pupil drawn from $K_T$ teachers with pairwise retention-correlation $\rho$ interpolates between the single-teacher value ($\rho = 1$) and the union of $K_T$ independent lineages ($\rho = 0$). With the construction of §2.7.1 (each teacher retains a fraction $q$ of the $T$ tail items, at exact pairwise correlation $\rho$) the interpolation is **closed-form**: the expected number of tail items retained by at least one teacher is
$$U(K_T,\rho,q) = T\left[\rho q + (1-\rho)\left(1-(1-q)^{K_T}\right)\right].$$
Check the limits: $K_T=1 \Rightarrow Tq$ (single teacher, independent of $\rho$); $\rho=1 \Rightarrow Tq$ (identical teachers, union = one); $\rho=0 \Rightarrow T[1-(1-q)^{K_T}]$ (independent teachers, maximal union). The pupil's realised tail coverage tracks $U$ *up to* the drift-survival threshold of prediction 4 — a tail item present in the mixture only survives the pupil's resampling of size $n$ if its mixture mass clears $\sim 1/n$, which ties E4 back to E2/E3. (Numerically verified against the construction to three decimals; it is a `test_scientific_validation.py` assertion.)

### 2.5 Experiments E1–E6

Each experiment is one config file (§4), one runner invocation, one results artifact, and one figure script. Each states its prediction and its **falsifier** — the outcome that would refute the corresponding claim in the perspective paper.

**E1 — Reproduce collapse (null model).** $m=0$, single teacher, no selection. *Expect:* $H_t = H_0(1-1/n)^t$; support shrinks to 1; forward KL diverges; tail mass → 0, tail first. *Validates against:* predictions 1–2. *Falsifier of the harness (not the theory):* if drift does not reproduce the analytic decay, the simulator is wrong — fix before proceeding.

**E2 — Grounding phase boundary (headline).** Sweep $g = m/(n+m)$ from 0 to a high value; single teacher; uniform grounding; no selection. *Expect:* a critical $g^\*$ below which tail mass and $H$ decay to (near) zero and above which they stabilise at a positive stationary value tracking $H_{\text{eq}} = \theta/(1+\theta)$. *This is the paper's central quantitative prediction: the ratio of inherited-to-grounded information has a threshold.* *Falsifier:* if the stationary tail mass is flat in $g$, or if it only stabilises as $g \to 1$ (you always need essentially all-real data), then inheritance buys nothing and the multigenerational thesis is refuted. Report $g^\*$ with a CI.

**E3 — Region-matched grounding.** Fixed total $m$; compare `uniform` vs. `matched` allocation across $R$ regions, with one designated "inherited-but-not-freshly-grounded-under-uniform" region carrying a rare tail. *Expect:* under `uniform`, the target region's tail collapses even though global grounding is nonzero; under `matched`, it persists. *Validates:* prediction 4 and the "grounding must overlap the content it protects" claim. *Falsifier:* if uniform grounding protects the region as well as matched, the region-matching requirement is unnecessary and that paragraph of the paper should be cut.

**E4 — Multi-teacher decorrelation.** Teachers built by the §2.7.1 construction; sweep number of teachers $K_T \in \{1,2,3,5\}$ and retention-correlation $\rho \in [0,1]$ (at fixed marginal retention $q$); matched total data budget (so more teachers ≠ more data — each contributes $n/K_T$). Report **two** coverages: the construction-level union $U(K_T,\rho,q)$ (must match the §2.4-5 closed form exactly) and the post-distillation *surviving* coverage after the pupil's resampling. *Expect:* both rise with $K_T$ and with $(1-\rho)$; collapse suppression $\propto$ decorrelation; at $\rho=1$, multiple teachers give no benefit over one; and the gap between union and surviving coverage shrinks as grounding $g$ rises (recombination supplies the tail, grounding holds it). *Validates:* prediction 5 (now closed-form). *Falsifier:* if $K_T$ decorrelated teachers give no surviving-coverage benefit over one at matched budget, the recombination claim dies and single-teacher distillation is fine.

**E5 — Quality-diversity vs. greedy.** Same starting diversity; `greedy` vs. `qd` selection; sweep novelty exponent $\alpha$. *Expect:* `greedy` → fixation ($H \to 0$); `qd` holds $H$ at a positive plateau and re-introduces lost tail items. *Validates:* the two-level anti-convergence argument (§7 of paper). *Falsifier:* if `qd` does not maintain higher stationary $H$ than `greedy`, quality-diversity is not doing the work the paper assigns it.

**E6 — Re-minting gate (irreversibility).** Run a lineage to a chosen diversity level, then re-mint (freeze current $p_t$ as new reference, discard original $p^\*$); compare re-minting at high $H$ vs. low $H$. *Expect:* re-mint while collapsed → KL to *original* truth is locked high forever (tails unrecoverable); gated re-mint at high $H$ → no lock-in. *Validates:* §11's "re-minting is irreversible; gate it on diversity." *Falsifier:* if a collapsed lineage recovers its original-truth tails after re-minting, the irreversibility warning is overstated.

### 2.6 Layer-1 falsifiers, collected

The model is built to be *able to kill the thesis*. If E2 shows no threshold, or E4 shows no decorrelation benefit, or E6 shows no lock-in, the corresponding claims are refuted and the paper must say so. A blueprint that cannot fail is not a test. The single load-bearing positive result is E2's phase boundary at $g^\* \ll 1$: it says a little grounding protects a lot of inheritance, which is the whole economic and conceptual bet of the architecture.

### 2.7 Implementation spec for Layer 1

**Language / libraries.** Python ≥ 3.11; NumPy, SciPy (stats), pandas (results), matplotlib (figures). No GPU. No other heavy deps. Everything seedable from a single integer.

**Core module interfaces** (the coding agent should implement to these signatures; names are normative so downstream scripts are stable):

```python
# knowledge/truth.py
def make_true_distribution(K: int, R: int, tail: str, tail_frac: float,
                           zipf_s: float, seed: int) -> TrueDist:
    """Return p_star (length K), region assignment (length K), and the tail mask."""

# knowledge/teachers.py
def make_retention_matrix(T: int, K_T: int, rho: float, q: float, rng) -> np.ndarray:
    """Return an (K_T, T) binary retention matrix R with exact marginal retention
    E[R]=q and exact pairwise column-correlation rho, via the shared-switch
    construction of §2.7.1. rho=0 -> independent tails; rho=1 -> identical."""

def make_correlated_teachers(p_star, tail_mask, K_T: int, rho: float, q: float,
                             region_assignment=None, region_specialisation=False,
                             tail_floor: float = 1e-9, seed: int = 0) -> list[np.ndarray]:
    """Build K_T teacher distributions from a retention matrix (§2.7.1): every teacher
    keeps all head items; teacher k keeps tail item j at ~p_star mass iff R[k,j]=1,
    else at tail_floor; renormalise. If region_specialisation, force R[k,j]=1 for tail
    items in teacher k's home region and apply the rho construction only off-home.
    The exact-construction path is preferred for E4; a drift-based path (running
    independent grounded lineages) is provided as a realism cross-check only."""

# knowledge/step.py
def generation_step(teachers, p_star_eff, cfg, rng) -> np.ndarray: ...
def apply_selection(p, p_star, mode: str, alpha: float) -> np.ndarray: ...
def structured_multinomial(m_vector, p_star, regions, policy: str, rng) -> np.ndarray: ...

# knowledge/lineage.py
def run_lineage(cfg, seed) -> pd.DataFrame:
    """Run T generations for one seed; return a tidy frame with one row per
    (generation) and columns for every metric in §2.3 (global and per-region)."""

# knowledge/metrics.py
def forward_kl(p_star, p, eps): ...
def heterozygosity(p): ...
def tail_mass(p, tail_mask): ...
def support_size(p, eps): ...

# knowledge/experiment.py
def run_experiment(cfg) -> pd.DataFrame:
    """Sweep the declared parameter grid x n_replicates seeds; return long-form
    results with confidence intervals; write parquet + the exact resolved config."""
```

**Config schema** (one YAML per experiment; all parameters explicit, no magic numbers in code). Illustrative default:

```yaml
experiment: E2_grounding_phase_boundary
seed: 20260704
n_replicates: 100
generations: 300
truth:
  K: 1000            # number of knowledge items
  R: 10              # regions
  tail: zipf         # {zipf, twocomponent}
  zipf_s: 1.1
  tail_frac: 0.5     # fraction of items designated 'tail'
  tail_threshold: 1.0e-3
dynamics:
  n: 200             # distillation sample size (drift strength)
  teachers:
    K_T: 1
    rho: 0.0
  grounding:
    sweep: {param: g, values: [0.0, 0.01, 0.02, 0.05, 0.1, 0.2, 0.4]}
    policy: uniform  # {uniform, matched}
  selection:
    mode: none       # {none, greedy, qd}
    novelty_alpha: 0.0
  remint:
    enabled: false
    period: null
    H_gate: null
metrics:
  kl_floor: 1.0e-9
output:
  dir: results/fig2_grounding_sweep/
  save_per_generation: true
```

**Determinism.** One master seed → per-replicate seeds via a documented `np.random.SeedSequence` spawn. No global RNG state; pass `rng` explicitly everywhere. Results are a pure function of the resolved config. The resolved config (after sweep expansion) is written next to the results artifact.

**Outputs.** Each experiment writes: `results.parquet` (long form), `resolved_config.yaml`, and a `manifest.json` recording library versions, seed, git commit, and a content hash of the results. Figures are generated by a separate `figures/plot_EX.py` reading only `results.parquet`, so any figure is regenerable from committed data without rerunning the simulation.

#### 2.7.1 The correlated-teacher construction (E4's one non-obvious piece)

E4 needs teacher sets whose pairwise **retention-correlation** $\rho$ is a clean, swept knob. Tuning the drift parameters until an emergent $\rho$ appears is the wrong design: that $\rho$ would be a tangled function of $n$, $m$, tail size and generation count, un-sweepable and confounded with the very drift whose effect E4 is trying to hold fixed. So we **construct the retention structure directly**, with $\rho$ an independent control. For an experiment whose entire purpose is to isolate the effect of decorrelation, $\rho$ must be manipulated, not inferred.

**The mechanism — shared-switch exchangeable Bernoulli.** Let the tail have $T$ items. Each teacher $k$ retains a fraction $q$ of them; we want the retention indicators to have marginal $q$ and *exact* pairwise correlation $\rho$ across teachers. For each tail item $j$:

1. draw a **shared switch** $z_j \sim \mathrm{Bernoulli}(\rho)$ (one draw, common to all teachers for item $j$);
2. draw a **shared retention** $s_j \sim \mathrm{Bernoulli}(q)$ (common to all teachers);
3. draw **independent retentions** $u^{(k)}_j \sim \mathrm{Bernoulli}(q)$, one per teacher;
4. set $r^{(k)}_j = s_j$ if $z_j = 1$, else $u^{(k)}_j$.

That is the whole construction. It yields a retention matrix $R \in \{0,1\}^{K_T \times T}$.

**Why it is exact.** Marginally $\mathbb{E}[r^{(k)}_j] = \rho q + (1-\rho)q = q$ regardless of $\rho$. For any two teachers, conditioning on the switch gives $\mathbb{E}[r^{(k)}_j r^{(k')}_j] = \rho\,\mathbb{E}[s_j^2] + (1-\rho)q^2 = \rho q + (1-\rho)q^2$ (using $s_j^2 = s_j$), so $\mathrm{Cov} = \rho q(1-q)$ and, since $\mathrm{Var} = q(1-q)$, the pairwise correlation is exactly $\rho$. The set is *exchangeable* — every teacher pair has the same $\rho$ — which is what makes $\rho$ a single scalar knob. (Verified numerically: marginal and pairwise correlation hit target across a $(\rho, q)$ grid, and the union closed form below matches to three decimals.)

**From retention to teacher distributions.** Given $R$, build teacher $k$'s distribution $p^{(k)}$: assign every **head** item its $p^\*$ mass (all teachers keep the common core); assign **tail** item $j$ its $p^\*_j$ mass if $r^{(k)}_j = 1$ and a floor $\varepsilon_{\text{tail}}$ otherwise; renormalise. (Renormalising lets the mass of dropped tails flow to what the teacher kept — the realistic signature of a partially-collapsed model concentrating on its survivors. Reallocating instead to the head is a documented config switch.)

**Region specialisation (structured decorrelation, optional).** With `region_specialisation=True`, give each teacher a **home region** and force $r^{(k)}_j = 1$ for every tail item $j$ in teacher $k$'s home region (each teacher fully retains the tails of the region it ground against reality), applying the $\rho$ construction only to off-home tail items. This models "each teacher ground a different region," ties E4 to E3's region-matched grounding, and is the discrete image of the perspective paper's "distil from teachers who each earned a different region."

**The analytic target (E4's exact check).** The probability that tail item $j$ is retained by at least one of $K_T$ teachers is $\rho q + (1-\rho)\big(1-(1-q)^{K_T}\big)$, so the expected **union tail-coverage** is
$$U(K_T,\rho,q) = T\left[\rho q + (1-\rho)\left(1-(1-q)^{K_T}\right)\right],$$
the closed form of §2.4-5. `make_retention_matrix` must reproduce it (and the target $\rho$, $q$) within Monte-Carlo tolerance in `test_scientific_validation.py`. The pupil's *realised* coverage after distillation equals $U$ only for tail items whose mixture mass clears the drift-survival threshold $\sim 1/n$ (§2.4-4) — so E4 should report both the union coverage (construction-level) and the post-distillation surviving coverage (dynamics-level), and their gap is itself informative: it is exactly the tail that recombination *supplied* but drift *re-erased* because grounding was too thin to hold it.

**Continuous-mass extension (optional realism, not a headline).** Binary retention is the default because it matches prediction 5 exactly. For a realism cross-check, replace binary retention with correlated *masses*: draw per-teacher log-masses on tail items from a multivariate normal with equicorrelation $\rho$ (a Gaussian copula), exponentiate, and normalise. This generalises to an arbitrary correlation *matrix* between teachers (unequal pairwise $\rho$), which the shared-switch construction — being exchangeable — cannot express; use it only if the paper later needs non-exchangeable teacher sets.

**Optional abstract treatment of the horizontal claim (§2.7-H, low priority).** If time permits, model a domain as a modular graph with tunable modularity $Q$; define the optimal number of concurrent specialists as the point where marginal coverage gain from an added specialist falls below a cost; show optimal specialist count rises with $Q$. This is the *horizontal* prediction in abstract form and does not require the neural layer. Flag clearly as exploratory; it is not a headline and can be dropped without weakening the paper.

---

## 3. Layer 2 — the neural existence proof

### 3.1 Purpose and the single objection it answers

Layer 1 assumes the tail-deletion operator (drift). A reviewer will say: *you built your conclusion into the operator.* Layer 2 answers exactly that objection and no more. It does not need to be a society. It needs to show, in **real LoRA-adapted weights**, that the *sign* of three effects is as Layer 1 predicts: dry inheritance degrades, grounded inheritance holds; multiple decorrelated teachers preserve capability that one teacher sheds; and across generations general capability holds or climbs while each specialty is re-earned and exceeded. If those three signs appear at 1B scale on one GPU, the abstraction in Layer 1 is grounded in mechanism.

Minimality is a virtue here, not a compromise. Every additional degree of freedom (bigger model, more generations, more domains) multiplies cost and reviewer surface without strengthening the core claim. Build the smallest thing that can show the signs.

### 3.2 Design choices (with open-science defaults)

**Base model.** Default to a *fully open* small model to honour reproducibility end-to-end: **OLMo-2-1B** or **SmolLM2-1.7B** (open weights, open or well-documented data, permissive licence). Capable fallback if the open models are too weak on the task: **Qwen2.5-1.5B-Instruct**. Pin the exact Hugging Face revision hash in config; never track `main`. The pipeline must be model-agnostic behind a thin adapter so swapping bases is a config change.

**Specialisation.** LoRA (via PEFT), small rank (e.g. 8–16), on a single task family = one "region." Hours on one consumer GPU, consistent with the perspective paper's cost claim. Each teacher is one base + one LoRA adapter.

**The verifier = "reality's no."** The domain must have a cheap, deterministic, uncontrollable oracle. **Program synthesis with unit tests** is ideal: a generated solution either passes its tests or it does not, and neither the model nor the experimenter controls the verdict. This is the minimal honest instance of the perspective paper's "predictive success under intervention." Execution happens in a sandbox (subprocess with a hard timeout and no network, run inside the container of §4; document the sandbox precisely).

**Task domain — synthetic-primary, benchmark-secondary.** To keep the Layer-1 abstractions (region, rarity, tail) exact, the *primary* domain is a **synthetic program-synthesis generator**: a family of small, verifiable tasks parameterised by *operation type* (the region: e.g. string ops, list ops, arithmetic, dict manipulation, recursion) and *difficulty*, with I/O unit tests generated automatically. Task-type frequency is tunable, so "rare tail task types" is a dial, exactly matching Layer 1's rarity. For *external validity*, add a secondary evaluation on a held-out slice of a public benchmark (e.g. MBPP-sanitised / HumanEval) — pinned by version — to show the effect is not an artefact of the synthetic generator. The synthetic generator's spec (grammar, per-region templates, test-generation rule, rarity distribution) is itself a committed, seeded artifact.

**Grounding, concretely.** A generational passage produces pupil training data as a mixture of (i) *inherited* teacher-generated solutions and (ii) *grounded* solutions that have been **filtered to pass the verifier** and/or freshly drawn from verified references in the target region. The **grounding fraction $g$** is the proportion of verifier-passed/real items in the pupil's training mixture — the *same knob* as Layer 1's $g$. "Dry" = $g$ low / unfiltered teacher output; "grounded" = $g$ raised with region-matched verified data.

**Recombination, concretely.** Multiple teachers = multiple LoRA specialists on *disjoint* task families (decorrelated by construction). The pupil is trained on the pooled outputs of all teachers (distillation) or, as a cheaper alternative, the specialists are **merged** (M2N2 / model-merge style, citing Sakana's demonstration that this search runs with no retraining) and the merged model is the pupil. Distillation and merging are two config-selectable recombination operators; report at least distillation, and merging if compute allows.

### 3.3 Contrasts C1–C4 (the whole experimental content)

**C1 — Dry vs. grounded, single teacher, across generations.** One teacher, one region. Arm A: pupil trained on *unfiltered* teacher outputs (dry, $g\approx0$). Arm B: pupil trained on *verifier-passed* teacher outputs plus fresh verified region data ($g>0$). Run 2–3 generations. *Expect:* Arm A pass@k degrades generation over generation and its solution diversity narrows; Arm B holds. *Maps to:* Layer-1 E2. *Falsifier:* if dry inheritance does not degrade at this scale, collapse is not reachable here and the neural claim is unsupported (report honestly; possibly scale down grounding or up generations).

**C2 — One teacher vs. N complementary teachers, matched data budget.** Fix the pupil's total training-token budget. Arm A: all budget from one teacher (one region). Arm B: same budget split across $N$ teachers on disjoint regions. Evaluate the pupil on *all* regions, including rare-type tasks. *Expect:* Arm B retains capability across the union of regions (the "tail" of rare task types survives); Arm A loses regions it did not inherit. *Maps to:* Layer-1 E4. *Falsifier:* no union benefit at matched budget → recombination claim unsupported neurally.

**C3 — The vertical claim (load-bearing).** Over 2–3 generations, track (a) a *general* held-out benchmark spanning all regions and (b) *per-specialty* performance, where each generation re-specialises (re-earns) its region against the verifier. *Expect:* general benchmark is **monotonically non-decreasing** across generations, while each specialty is re-earned and its peak **exceeds the parent's** peak in that specialty. *This is the test of the actual thesis, not the borrowed Shumailov scaffolding.* *Maps to:* the paper's vertical prediction. *Falsifier:* if general capability falls across generations, or specialties are not re-earned-and-exceeded, the multigenerational ratchet is not demonstrated.

**C4 — Recombination operator: distillation vs. merging (optional).** If compute allows, show C2's benefit is robust to whether recombination is done by distillation or by weight-merging. Strengthens generality and directly connects to the M2N2 citation. Drop first if time-constrained.

### 3.4 Metrics (Layer 2)

- **pass@1 and pass@k** on held-out tasks, per region and overall (the neural analogue of KL-to-truth / capability).
- **Region coverage** = fraction of regions (incl. rare task types) with pass@1 above a floor — the neural analogue of *support size* / tail survival.
- **Solution diversity** = distinct-$n$ or behavioural diversity of generated solutions per task (the neural analogue of heterozygosity $H$; narrowing diversity is the neural signature of collapse).
- **Generational degradation curve** = each metric as a function of generation index, per arm.

All with multiple seeds (fewer than Layer 1 — GPU cost — but at least 3; report per-seed points, not just means, given small $n$). Pin decoding parameters (temperature, top-p, max tokens, sampling seed) in config; they materially affect pass@k and must not float.

### 3.5 Layer-2 honesty riders

GPU non-determinism means Layer 2 is *statistically* reproducible (same distribution of outcomes under re-run), not bitwise reproducible. Document this explicitly; pin everything pinnable (model revision, dataset version, decoding params, library versions, seeds); report seeds individually. The claim Layer 2 supports is directional ("the sign of the effect is as predicted"), and the writing must not overclaim precision the setup cannot deliver.

### 3.6 Implementation spec for Layer 2

**Libraries.** PyTorch; Hugging Face `transformers` + `peft` (LoRA); `datasets`; optional `vllm` for fast generation; the synthetic task generator (project-local). Execution sandbox: `subprocess` with `resource` limits and timeout, inside the container.

**Module interfaces (normative names):**

```python
# neural/tasks.py
def generate_task_bank(regions, rarity, n_tasks, seed) -> TaskBank:
    """Synthetic verifiable tasks; each task carries prompt, region, difficulty,
    and an executable unit-test suite."""
def verify(solution_code: str, task) -> VerifyResult:
    """Run tests in a sandbox; return pass/fail + diagnostics. Deterministic."""

# neural/specialise.py
def train_lora_specialist(base_id, revision, region, task_bank, lora_cfg, seed) -> AdapterPath: ...

# neural/distill.py
def generate_teacher_data(teachers, task_bank, grounding_fraction, policy, decode_cfg, seed) -> Corpus:
    """Produce the pupil's training corpus: mixture of inherited (teacher) and
    grounded (verifier-passed / fresh-verified) items, region-matched."""
def train_pupil(base_id, revision, corpus, lora_cfg, seed) -> AdapterPath: ...

# neural/merge.py
def merge_specialists(base_id, adapters, method, seed) -> ModelPath:  # optional (C4)

# neural/evaluate.py
def evaluate(model, task_bank_heldout, decode_cfg, seed) -> pd.DataFrame:
    """pass@k, per-region coverage, solution diversity; tidy per-(region) frame."""

# neural/generation_loop.py
def run_generations(cfg, seed) -> pd.DataFrame:
    """Orchestrate T generations for one arm; log every metric per generation."""
```

**Config** mirrors Layer 1's structure (one YAML per contrast; pinned `base_id` + `revision`; explicit `grounding_fraction`, `n_teachers`, `regions`, `generations`, `decode_cfg`, `lora_cfg`, `seed`, `n_seeds`). Same output contract: `results.parquet` + `resolved_config.yaml` + `manifest.json` (with model revision hashes and dataset versions). Figures regenerable from `results.parquet` alone.

---

## 4. Reproducibility and engineering standard (both layers)

Open science is a hard requirement of this project, not a preference. The standard below is normative.

**Environment.** Pin everything. Provide (a) a `pyproject.toml` + lockfile via **uv** (fast, reproducible resolver) and (b) an **Apptainer/Singularity** definition file (HPC-friendly, rootless, open) that builds the exact environment; optionally a Dockerfile. The container is the source of truth for "it runs." Record Python, CUDA, and key library versions in every run's `manifest.json`.

**Seeding.** One master seed per experiment in config; derive all sub-seeds via `SeedSequence.spawn`; never touch global RNG state. Layer 1 is bitwise-reproducible from seed. Layer 2 is statistically reproducible; document the residual GPU non-determinism and set the available determinism flags (`torch.use_deterministic_algorithms(True)` where feasible, cudnn deterministic, documented exceptions).

**Configuration.** No magic numbers in code — every parameter lives in a YAML resolved at run time; the *resolved* config is written beside results. Use a single config system (Hydra or a thin equivalent). Sweeps are declared in config, not hard-coded in scripts.

**Data & model provenance.** Layer 1 data is synthetic-from-seed (fully reproducible; no external data). Layer 2 pins model revision hashes and dataset versions; the synthetic task generator is committed and seeded. Cache external downloads with recorded hashes.

**Experiment tracking.** Prefer open tooling: **MLflow** (open source) or plain versioned CSV/Parquet + committed configs; avoid closed SaaS trackers to keep the pipeline fully open. Whatever is chosen, the invariant is: every figure is a pure function of a committed results artifact.

**Testing.** `pytest`. Two kinds of test, and both are required:
- *Correctness tests*: the module does what it says (shapes, normalisation, sandbox isolation).
- *Scientific validation tests*: the simulator reproduces the §2.4 analytic results within tolerance (heterozygosity decay, fixation probability, mutation–drift equilibrium). These tests failing means the science is wrong, not just the code — they are the spine of trust in Layer 1.

**Automation.** A `Makefile` (or `justfile`) with targets: `env`, `test`, `layer1` (runs E1–E6), `layer2` (runs C1–C3, C4 optional), `figures`, `paper` (assembles the figure manifest), `all`, `clean`. One command reproduces the study from a clean checkout inside the container.

**Repro entry point.** A top-level `reproduce.sh` that: builds/enters the container, runs tests, runs all experiments at the committed seeds, regenerates all figures, and writes a `REPRODUCED.md` diff against committed result hashes. If hashes match (Layer 1) / distributions match within CI (Layer 2), the run is verified.

**Licensing & citation.** Author to choose; suggested: code under a permissive OSI licence (MIT/Apache-2.0) or copyleft (GPL-3.0) per the author's open-source preference; text/figures under CC-BY. Include `LICENSE`, `CITATION.cff`, and a `DATA_AND_MODELS.md` recording every external artifact and its pinned version/hash.

---

## 5. Repository layout

```
lamarckian-society/
├── README.md                  # what this is, how to reproduce
├── reproduce.sh               # one-command full reproduction (in-container)
├── Makefile                   # env, test, layer1, layer2, figures, all
├── pyproject.toml             # deps
├── uv.lock                    # pinned resolution
├── apptainer.def              # container definition (source of truth)
├── Dockerfile                 # optional
├── LICENSE  CITATION.cff  DATA_AND_MODELS.md
├── configs/
│   ├── layer1/E1..E6.yaml
│   └── layer2/C1..C4.yaml
├── src/
│   ├── knowledge/             # Layer 1: truth, teachers, step, lineage, metrics, experiment
│   └── neural/                # Layer 2: tasks, specialise, distill, merge, evaluate, generation_loop
├── figures/                   # plot_EX.py / plot_CX.py — read results.parquet only
├── results/                   # written artifacts (gitignored, hashes tracked)
├── tests/
│   ├── test_correctness.py
│   └── test_scientific_validation.py   # §2.4 analytic checks
└── paper/
    ├── blueprint.md           # this document
    └── figure_manifest.md     # claim -> experiment -> figure
```

---

## 6. Traceability: claim → experiment → figure

Every claim the technical paper will make is bound to exactly one experiment and one figure. This table is the paper's spine and the coding agent's definition of done.

| Perspective-paper claim | Layer | Experiment | Primary figure | Analytic check | Falsifier |
|---|---|---|---|---|---|
| Distillation without grounding collapses, tail first, at rate set by $n$ | 1 | E1 | $H_t$ decay + tail mass vs. gen | Pred. 1–2 | harness invalid if decay ≠ analytic |
| A critical grounding ratio $g^\*$ separates ratchet from collapse | 1 | E2 | tail mass / $H$ stationary vs. $g$ | Pred. 3 | flat in $g$, or only stable at $g\to1$ |
| Grounding must be region-matched, not merely present | 1 | E3 | target-region tail: uniform vs. matched | Pred. 4 | uniform protects as well as matched |
| Multi-teacher distillation suppresses collapse ∝ decorrelation | 1 | E4 | tail coverage surface over $(K_T,\rho)$ | Pred. 5 (closed form $U(K_T,\rho,q)$) | no benefit at matched budget |
| QD selection maintains diversity where greedy fixes it | 1 | E5 | $H$ plateau: qd vs. greedy | (mutation-selection-drift) | qd ≤ greedy in stationary $H$ |
| Re-minting is irreversible; gate on diversity | 1 | E6 | KL-to-original vs. $H$-at-remint | — | collapsed lineage recovers after remint |
| Dry inheritance degrades in real weights; grounded holds | 2 | C1 | pass@k vs. gen, dry vs. grounded | — | dry does not degrade |
| Complementary teachers preserve capability one teacher sheds | 2 | C2 | region coverage, 1 vs. N, matched budget | — | no union benefit |
| **General knowledge climbs while each specialty is re-earned and exceeded** | 2 | C3 | general benchmark ↑ + per-specialty re-earn | — | general falls / no re-earn |
| Recombination benefit robust to distillation vs. merging | 2 | C4 (opt) | C2 metric under both operators | — | benefit only under one operator |

---

## 7. Suggested build order for the coding agent

Staged so that each step is independently testable and the cheapest, highest-value results land first. Do not start Layer 2 until Layer 1's scientific-validation tests pass.

1. **Scaffold + environment.** Repo layout (§5), container (§4), `pytest` skeleton, config system, seeding utilities. Target `make test` green on trivial tests.
2. **Layer 1 core + validation.** `knowledge/` modules to the §2.7 interfaces. Implement the null model first. Write and pass `test_scientific_validation.py` against §2.4 predictions 1–2. **Gate: do not proceed until drift matches analytic decay.**
3. **Layer 1 mechanisms + E1–E2.** Add grounding; validate mutation–drift equilibrium (pred. 3); run E1 and the E2 phase-boundary sweep; produce the headline figure. This is the paper's core result and it should exist before anything neural.
4. **Layer 1 E3–E6.** Region-matching, multi-teacher/decorrelation, QD-vs-greedy, re-minting gate, with figures. Layer 1 is now a complete, laptop-reproducible paper on its own.
5. **Layer 2 scaffold + verifier.** Synthetic task generator, sandboxed `verify`, evaluation harness. Test the verifier's determinism and isolation before any training.
6. **Layer 2 C1 + C3.** Single-teacher dry-vs-grounded (C1) and the vertical claim (C3) — the two that most directly test the thesis. C3 is load-bearing; prioritise it.
7. **Layer 2 C2 (+ C4 if compute allows).** Multi-teacher recombination; optional merging operator.
8. **Reproduction pass.** `reproduce.sh` end-to-end; commit result hashes; write `REPRODUCED.md`; assemble the figure manifest.

**Definition of done:** every row of §6 has a committed figure produced by `make figures` from committed results, every §2.4 analytic check passes in CI, and `reproduce.sh` verifies from a clean checkout inside the container.

---

*End of blueprint v1. The perspective paper states the idea; this document states the test. If Layer 1's E2 finds no threshold, or E4 no decorrelation benefit, or Layer 2's C3 no vertical climb, the thesis is wrong in exactly the places these experiments probe — which is the point of writing them down this precisely.*