MachineSex/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Current state: Layer 1 complete; Layer 1.5 (neural) in progress

• Layer 1 (src/knowledge/) — complete and validated. All six experiments E1–E6, the closed-form scientific-validation tests, figures, and reproducibility harness exist. Headline: critical grounding g* = 0.048 ≪ 1; the E4 finding that mean-mixture distillation conserves collapse while only a union-preserving max-merge realises the recombination benefit.
• Layer 1.5 (src/neural/) — in progress. An architecture-general neural existence proof (re-scoped Layer 2): the same Wright–Fisher abstractions realised in real trained generative models (histogram bridge + RNN + MLP; VAE implemented but not fidelity-passing) on a fully-synthetic sandbox with an exact oracle, plus real MNIST as a later secondary tier. See tasks/todo.md for status and ~/.claude/plans/we-are-going-to-cheerful-fog.md for the plan. Done: scaffold, the histogram bridge gate (reproduces Layer 1 exactly), bridge (neural g*=0.047 ≈ Layer 1), collapse (in RNN weights), grounding (refined; sign confirmed, threshold softened by neural smoothing — see finding below), architectures (architecture-generality), recombination (the E4 "merge, don't average" finding reproduced in real weights), all six neural figures, and the real-MNIST external-validity tier (mnist_collapse: a conv-VAE collapses to a single mode under dry self-training, ~10% grounding holds all 30 modes; frozen-CNN oracle, confusion matrix recorded). Remaining: region_matched, remint, the synthetic-VAE fidelity fix — all optional. The LLM/LoRA rung and C3 vertical claim stay deferred. The LLM/LoRA rung and the C3 vertical claim are deferred. Experiments are named descriptively (configs/neural/<name>.yaml), not by code.

The two design documents are the source of truth for intent:

• paper/the-lamarckian-society-v5.md — the perspective paper (the "why"; reframed around sexual reproduction). paper/results-summary.md — the plain-language + technical summary of all results.
• paper/blueprint.md — the technical blueprint (the "what"/"how"). It is normative for Layer 1 and the LLM Layer 2; Layer 1.5 is a cost-staged intermediate the blueprint does not cover, designed to preserve the same §1 abstractions.

Everything below summarizes the blueprint so you can orient fast, but the blueprint is the source of truth. When they conflict, the blueprint wins; when the blueprint is silent, minimize decisions and match its established patterns.

The one idea you must hold in your head

Knowledge transmission across agent generations is modelled literally as a Wright–Fisher population-genetics process — not by analogy. A model's knowledge is a distribution p_t over K discrete items on a simplex; a fixed true distribution p* has a rare tail; each generational step is "sample from parent (drift) + mix in fresh real samples (immigration/grounding) + refit." Model collapse = loss of rare alleles under drift. Every experiment is a manipulation of this single process.

The population-genetics dictionary in blueprint §1 is the spine. Keep its abstractions identical across both layers — this is a hard requirement, because it is the only thing that lets a Layer-2 neural result count as confirming a Layer-1 analytic prediction:

Abstraction	Layer 1 (analytic)	Layer 2 (neural)
region	disjoint block of the K items	task family (e.g. string ops, recursion)
rarity / tail	low p* items	low-frequency task types
grounding fraction g	m/(n+m) real-vs-inherited samples	proportion of verifier-passed items in pupil's training mix
decorrelation ρ	shared retained-tail correlation between teachers	LoRA specialists on disjoint task families
diversity H	heterozygosity 1 − Σ pᵢ²	solution diversity of generated code
reality's "no"	grounding against p*	execution-based unit-test verifier

Two layers, staged by cost

• Layer 1 — analytical core (src/knowledge/). Pure NumPy/SciPy Wright–Fisher simulator. Laptop, minutes, no GPU. Carries the paper's quantitative claims. Three of the five §2.4 predictions are closed-form, so validation is an exact test, not a vibe check — these become <0.1%-tolerance assertions in test_scientific_validation.py:
  • Pred. 1 — neutral heterozygosity decay: E[Hₜ] = H₀(1 − 1/n)ᵗ.
  • Pred. 3 — exact mutation–drift equilibrium for the implemented immigration model: H_eq = H* · m(2n+m−1)/(n+2nm+m²), with H* = 1 − Σ(p*ᵢ)². The textbook θ/(1+θ) (θ=2m) is only the rare-immigrant limit. Critical nuance: H is smooth in m — the sharp phase threshold lives in discrete tail-item survival (Pred. 4: an item survives iff m·p*ᵢ ≳ 1), not in H. Do not describe E2 as a discontinuity in H.
  • Pred. 5 — closed-form recombination benefit: U(K_T, ρ, q) = T[ρq + (1−ρ)(1−(1−q)^K_T)] (expected tail items retained by ≥1 of K_T teachers).
• Layer 2 — neural existence proof (src/neural/). Small open-weight models (default OLMo-2-1B / SmolLM2-1.7B, fallback Qwen2.5-1.5B-Instruct; pin the HF revision hash, never track main), LoRA specialisation, distillation/merging across 2–3 generations, program-synthesis-with-unit-tests as the verifier. One consumer GPU. Only needs to show the sign of three effects, not precise magnitudes.

Experiments and their falsifiers

Each experiment is one config file → one runner invocation → one results.parquet → one figure. Every experiment has a falsifier — an outcome that would refute the corresponding claim. The design is built to be able to kill the thesis; preserve that.

• Layer 1: E1 reproduce collapse (null), E2 grounding phase boundary (headline: is there a critical g* ≪ 1?), E3 region-matched grounding, E4 multi-teacher decorrelation, E5 quality-diversity vs. greedy selection, E6 re-minting gate / irreversibility.
• Layer 2: C1 dry vs. grounded, C2 one vs. N complementary teachers at matched budget, C3 the vertical claim (general knowledge climbs while each specialty is re-earned and exceeded — this is load-bearing, prioritize it), C4 distillation vs. merging (optional).

Blueprint §6 is the claim→experiment→figure→falsifier traceability matrix and is the definition of done.

The one non-obvious implementation piece: the correlated-teacher construction (§2.7.1)

E4's whole purpose is to isolate the effect of teacher decorrelation ρ, so ρ must be a directly constructed, independently-swept knob — never an emergent quantity you get by tuning drift (that ρ would be confounded with n, m, tail size, and generation count, i.e. with the very drift E4 holds fixed). The construction is a shared-switch exchangeable Bernoulli: for each of the T tail items, draw a shared switch z~Bern(ρ), a shared retention s~Bern(q), and per-teacher independent u⁽ᵏ⁾~Bern(q); set teacher k's retention r⁽ᵏ⁾ = s if z else u⁽ᵏ⁾. This yields exact marginal retention q and exact pairwise correlation ρ (provable: Cov = ρq(1−q), Var = q(1−q)), and is exchangeable so ρ is a single scalar. make_retention_matrix(T, K_T, rho, q, rng) returns the (K_T, T) binary matrix; make_correlated_teachers maps it to distributions (head items always kept at p*; tail item kept at p*ᵢ if retained, else tail_floor; renormalise so dropped-tail mass flows to survivors). The exact-construction path is preferred for E4; the drift-based path exists only as a realism cross-check. region_specialisation=True forces full retention of a teacher's home-region tails and applies the ρ construction only off-home.

E4 reports two coverages, and their gap is a result, not noise: the construction-level union U(K_T,ρ,q) (must match the closed form exactly) and the post-distillation surviving coverage after the pupil's size-n resampling. A tail item present in the mixture only survives if its mixture mass clears ~1/n (Pred. 4) — so the gap is precisely "the tail recombination supplied but drift re-erased because grounding was too thin," which ties E4 back to E2/E3.

Finding (2026-07-04, E4) — the recombination operator matters, and mean-mixture distillation does not realise the benefit. Under the blueprint's mean-mixture pupil (p̄ = mean(teachers)), surviving tail coverage is flat in K_T — a conservation law: averaging preserves expected pupil tail mass at q·(tail mass of p*) regardless of K_T, and in the rare-tail (linear-survival) regime the 1/K_T dilution exactly cancels the union gain. The recombination benefit is realised only under a union-preserving merge (max over teachers, à la M2N2), where surviving rises with K_T and decorrelation. So E4 reports surviving under both operators (surviving_mean, surviving_max): union = supply (validated vs closed form), max-merge = realised benefit, mean-distill = the null that motivates why merging/grounding is needed. GG decision: report both. This sharpens rather than refutes the thesis, but the paper's recombination claim rests on the merge operator, not naive mean distillation — worth carrying into Layer 2 (C4) and the write-up.

Finding (2026-07-05, neural grounding) — grounding's SIGN transfers to trained RNN weights, but the sharp g* does not; and tail-survival is the wrong neural collapse metric. Re-ran the phase-boundary sweep at 18 replicates. Two results: (1) forward-KL is the operative neural collapse metric, not H or tail-survival. The RNN's smoothing inductive bias keeps spurious tail modes alive (it generalises to unseen codewords), so tail_truth_mass_alive is flat/non-monotone in g (dry 0.54 > most grounded points) and H stays 0.77–0.85 of H* throughout — neither shows a threshold. Stationary forward-KL falls monotonically (dry 2.08 → g=0.2: 0.75), significant at g≥0.05 (paired t up to 3.3; 89% of lineages improve at g=0.2). This refines the earlier "forward-KL AND tail survival" note: for a smoothing model, support-counting decouples from closeness-to-truth. (2) The sharp g*≪1 is an exact-operator feature, softened by neural inductive bias. Half the achievable KL reduction closes by a median-recovery grounding g≈0.04 (bootstrap CI [0.004, 0.116]) — a striking echo of Layer-1's 0.048 — but full (95%) recovery needs g≈0.19, far more than the histogram bridge, because smoothing both caps dry collapse (KL2, not ∞) and slows full recovery. So the quantitative g*≪1 claim rests on the histogram bridge (g*=0.047, exact reduction to Layer 1), which the trained RNN confirms in sign and softens in sharpness. Honest note: the pre-registered 95%-of-H*/tail-survival falsifier is not met, but that is because those are the wrong metrics for a smoothing model, not because grounding fails — the blueprint §3.5 directional claim (grounding arrests collapse) holds robustly. Robustness fix landed alongside: a fully-degenerate RNN can emit only invalid codewords, so measure_distribution returns a terminal-collapse sentinel (fixation on the dominant mode) instead of crashing a long sweep.

Finding (2026-07-05, real-MNIST mnist_collapse) — collapse and grounding-rescue reproduce on real images. External-validity tier: a small convolutional VAE (the canonical generative-collapse model) is retrained each generation on its own generated digits. Modes = digit class × stroke-thickness bin (K=30, Zipf, ~18 tail modes); the oracle is a frozen CNN + deterministic thickness at 98.5% mode accuracy (its 30×30 confusion matrix is recorded in the manifest as the measurement floor). Result (4 reps): the dry (g=0) lineage collapses to a single mode — forward-KL 0.5→18, support 30→1, tail truth-mass 1.0→0.06, H→0 — while 10% grounding holds all 30 modes (KL≈0.6, full tail, H≈0.9). The VAE needs ~10% grounding here vs the synthetic histogram's ~5%, consistent with the grounding finding that trained neural models need more grounding than the exact operator. Confirmation-only (signs, not magnitudes; blueprint §3.5) — the exact synthetic oracle stays the quantitative anchor. figures/mnist_montage.py is an eyeball diagnostic (re-runs a short dry lineage; NOT a parquet figure). Build gates passed: CNN mode accuracy 98.5%; VAE gen-0 recovers full 30/30 support (over-smooths frequencies, KL≈0.5, no prior hole — unlike the synthetic-codeword VAE, which is why the MNIST VAE works where that one didn't). The MNIST tier is heavy (torchvision --extra mnist, downloads MNIST, ~5 min): make mnist, kept out of the make neural loop.

Finding (2026-07-05, learning kernel) — neutral drift is a null both real models fail, oppositely; the estimator bias is a signed operator. Layer-1 extension (knowledge/kernel.py, LearningKernelCfg): the refit becomes p_{t+1} = T_θ(counts/n) with two pop-gen knobs — reset u (mutation toward a prior = smoothing) and temperature τ (sharpening = mode-competition) — both identity at their defaults, so the histogram bridge and every scientific-validation test are unchanged (68 core tests still green). Result: neutral Wright–Fisher fails both neural architectures, in opposite directions. VAE regime (n=6000, K=30): neutral drift is inert (no collapse), yet the real VAE collapsed to one mode — sharpening τ=0.8 reproduces it (the estimator ADDS collapse). RNN regime (n=200, K=256): neutral drives H→0, but the real RNN only partially collapses — mutation u=0.006 reproduces the H-floor (the estimator REMOVES collapse). Honest caveat: uniform-mutation matches the RNN H-floor but overshoots its forward-KL (~5 vs ~2), evidence the RNN's smoothing prior is truth-like, not uniform (future refinement). Configs configs/layer1/kernel_{sharpen,smooth}.yaml, figure plot_kernel.py. This mechanistically explains the architecture-generality result and the softened neural g*.

Strategic positioning vs Riis 2026 (arXiv:2604.08554, "Drift and selection in LLM text ecosystems"). Riis independently formalizes collapse = Wright–Fisher drift (his Thm 1) with n-gram agents: minority-mass martingale, rare-first extinction, single-token dropout ≈ αe^{−α}, de Bruijn-polytope fixed points, plus descriptive-vs-normative selection (Thm 2). Concede as prior art: "collapse is literally Wright–Fisher", the martingale, rare-first loss, the WF/effective-population formalism — cite him; do not frame these as our contribution. Crucial distinction that protects us: his "mixed environment" retains the lineage's own old synthetic tokens — there is no injection of fresh real data from a fixed p*, so his headline is pessimistic (Thm 1c: extinction is independent of α — retention only changes speed). Our grounding is immigration from a non-drifting external truth, giving a stationary H_eq>0 and a critical g*≪1 that prevents collapse — the mechanism his closed loop lacks. Our defensible novelty, ranked: (1) recombination + "merge, don't average" conservation law (E4) — he has no model-merging operator; flagship; (2) the learning-kernel / estimator-bias axis — he explicitly names it as future work; we now build+measure it; (3) grounding threshold (solid anchor, but immigration–drift balance is classic — not a flagship); (4) architecture-generality in real weights + MNIST; (5) the Lamarckian society + the vertical/cumulative C3 claim — wholly ours, not yet run. Reposition the paper from "collapse is drift" (now contested) to a population-genetic control theory for sustaining open-ended knowledge: drift is the diagnosed disease (cite Riis), our contribution is the engineered remedies and their integration.

Finding (2026-07-05, E7/E8 — the multi-locus society frame; raises the ceiling). To express the vertical claim (capability that exceeds any component), knowledge is generalized from a single-locus fixed-p* distribution to a distribution over genotypes (L biallelic loci, K=2^L; fitness = # correct loci; reuses all the K-mode machinery). The one new operator is recombination (knowledge/genotype.py): free recombination sends p → ⊗ per-locus marginals (linkage equilibrium). Two experiments, both analytic. E8 (the star, kind: society) — the vertical claim / Fisher–Muller: decorrelated parents (specialists, expert on their loci, agnostic elsewhere) are recombined; sexual merge assembles a genotype fitter than any parent, climbing to the optimum (12/12) as parent count grows and ρ→0, while the best single parent (~8.7) and the mean-mixture "model soup" (~11.6) plateau below. Clean, dramatic, 40 reps; reuses make_retention_matrix (locus mastery replaces tail-item retention). E7 (kind: genotype_lineage) — the advantage of sex: a single population adapting toward the optimum; the sexual lineage adapts faster (clonal interference slows the asexual one) by keeping loci in linkage equilibrium (LD→0 vs LD spike). Honest scope: a speed advantage, not a permanent Muller's-ratchet gap (the single-population ratchet is subtle to force; E8 carries the headline). Metaphor shift (GG, 2026-07-05): the society is framed as sexual reproduction with unbounded parents, not teacher→pupil — teacher→pupil caps at the ceiling (recovery), n-parent recombination is combinatorial and generative (exceeds any parent), and unlike biology there is no two-parent limit. Collapse = asexual degradation; the cure = sex. This unifies E4 (merge≠average) + E6 (irreversibility) under evolution-of-sex theory and stakes ground Riis's single-locus n-grams cannot reach. Scope is bounded: fixed combinatorial space (L≤12, "effectively open-ended relative to n"), additive fitness (NK/epistasis is an optional extension).

Finding (2026-07-05, E9/E10 — the sexual-transmission model made rigorous: when sex helps, and directed sex). Deepening the sexual metaphor (GG excited; wanted it robust before the full society). Added a Kauffman NK landscape (genotype.nk_fitness, tunable ruggedness K), finite crossover (genotype.crossover, n-parent, per-gap recombination rate), and hill-climb (parents = local optima = "trained models"). E9 (kind: recomb_landscape) — landscape robustness / "why sex?": E8's dramatic transgression used an additive landscape; on rugged (epistatic) landscapes, blindly recombining local optima causes outbreeding depression — mean offspring fall below the parents, worse with ruggedness AND recombination rate (K=8, free recomb: ≈ −0.23), and the optimal recombination rate shrinks as ruggedness grows. Design rule: merge freely when skills are complementary/additive; sparingly + with selection when entangled. E10 (kind: directed_sex) — directed sex beats biological sex (the AI superpower): biology is stuck with 2 random-mating parents and no offspring preview; an AI can choose complementary mates + evaluate many recombinant offspring + keep the fittest + use unbounded parents (iterated recombine-then-select). Result: random ("biological") sex craters with ruggedness (0.66→0.51), while directed sex tracks/exceeds the best parent at every ruggedness — converting the outbreeding-depression catastrophe into a win. This is the practical, distinctly-AI payoff and has no biological analog. configs/layer1/{E9,E10}.yaml, plot_{E9,E10}.py, READMEs, +5 tests (117 green). Complete sexual-transmission picture: dramatic super-parent offspring when skills are complementary (E8); outbreeding-depression risk when entangled (E9); directed sex resolves the risk (E10).

Finding (2026-07-05, E11 — the dynamic Lamarckian society: the vertical claim / C3, realized). The culmination: a finite population of N agents (genotypes, L loci) evolves on a rugged NK landscape that is reality (knowledge/dynamic_society.py), composing the four operators the whole study built toward — grounding, directed recombination (sex), quality-diversity selection, mutation. Grounding is made load-bearing via the consensus-conformity (self-consumption) mechanism (GG decision): selection acts on g·true_fitness + (1−g)·conformity (conformity = agreement with the population's own consensus), so g=0 optimises fitting-the-crowd rather than reality. 4-arm ablation (12 reps), each breaking distinctly, only the full society climbing (global_opt≈0.79): full 0.78 (climbs to the optimum, diversity maintained longest) · no_sex 0.77 (can't recombine to escape local optima) · no_diversity/greedy 0.74 (collapses diversity fastest, stuck at a worse local optimum) · no_grounding 0.48 (self-consumption collapse to an unfit consensus — trains on the crowd, regresses to a confident-but-wrong mean; conformity−true gap ≈0.5). This integrates E1–E6 + the kernel + E7–E10 into one system and shows the society needs all of grounding + directed sex + diversity: on a rugged landscape you need diversity to explore basins, sex to recombine them, grounding to select on reality — remove any and you fail differently. configs/layer1/E11.yaml, plot_E11.py, README, +5 tests (122 green). This closes the C3 vertical claim analytically (the LLM rung remains the eventual empirical instantiation).

Finding (2026-07-05, LLM prototype llm_merge — the first real-LLM step; honest/partial). First move from toy models toward real LLMs (blueprint C2/C4, the real-LLM image of E8), on one 16 GB GPU. New src/llm/ package: procedural task families + exact-match verifier (tasks.py), batched eval (evaluate.py), LoRA specialisation (specialise.py, manual answer-only SFT), weight-space merge via peft add_weighted_adapter (merge.py: soup=averaged deltas, ties=sign-reconciled union), runner (experiment.py, kind llm_merge). Base = Qwen2.5-0.5B-Instruct (Apache-2.0). Three disjoint, deliberately-hard families (lists/strings/arith); one LoRA specialist each (~90 s total). Result (seed 1): each specialist spikes on its own family; the merges are the only models competent across ALL families — worst-family ≈0.25 vs <0.16 for every single specialist (the Fisher-Muller "generalist from specialists" signature, robust). But the stronger "exceeds every parent overall" claim is only marginal at this scale (soup 0.64 vs best specialist 0.63; ties 0.61 below it), and averaging visibly dilutes peaks (lists: specialist 0.43 → merge 0.26 — the E4 "merge, don't average" caveat in real weights). Honest scope: pipeline works end-to-end; the balance/retention half reproduces; the strict overall-exceeds and the soup-vs-ties distinction need scale (bigger base, more/cleaner families, seeds, dilution-resistant/offspring-selected merge). Env notes: Python 3.14 + transformers 5.13 works (cp314 wheels exist); transformers 5.x changed apply_chat_template (returns a dict; render to text then tokenize; pass **inputs to generate). make env-llm / make llm; adapters cached under gitignored models/llm/, base in the HF cache (outside the repo). 125 tests green (+3 pure task/verifier). The full grounded sexual society on LLMs (C1 collapse, directed sex, the dynamic society) is the HPC-scale next step.

Finding (2026-07-05, llm_merge_hpc — the 7B firm-up on Imperial CX3; the marginal sign becomes decisive). Re-ran llm_merge at a capable base — Qwen2.5-7B-Instruct, 200 test tasks/family, one L40S (46 GB) GPU, 8 min walltime — via the /imperial-hpc runbook (see memory/hpc-setup.md). Both merges reach 0.87 overall, decisively above the best single specialist (0.77) and above every specialist on every family; worst-family 0.62 vs ≤0.57 for any specialist. The two 0.5 B caveats are resolved: (1) the strict Fisher–Muller "exceeds every parent overall" claim is now clean (+10 points, not marginal); (2) the dilution vanishes — at 7 B the merge beats the lists-specialist on lists (0.62 > 0.57), where at 0.5 B averaging diluted it (0.43 → 0.26). Interpretation: dilution is a small-model artefact; a capable base has enough headroom that weight-space averaging composes rather than dilutes — the "merge, don't average" concern (E4) softens once parents are strong (soup ≈ ties at K=3). Results synced to results/llm_merge_hpc/ (README legend + data-driven figure title). The natural refinement is module-level union-preserving recombination (MoE-expert / adapter-union merge, the real-weight image of E8's max-merge) rather than delta-averaging.

Finding (2026-07-05, llm_moe — the union operator in real weights; E8's max vs mean, 0.5B). Added the union-preserving recombination operator that llm_merge lacked (src/llm/moe.py, kind: llm_moe): never average the parents — keep each specialist LoRA intact and select the right one per prompt (MoE router: oracle, or learned = training-free nearest-centroid over the base model's own prompt embeddings) or per module (max_merge = winner-take-all by delta-norm). Reuses the cached llm_merge specialists (no retraining). Result (0.5B, seed 1): routing wins decisively over fusion — overall 0.74 / worst-family 0.43 vs soup 0.64/0.26 — and recovers each specialist's own-family peak exactly (no dilution: fusion diluted the lists-specialist 0.43→0.26, routing keeps 0.43). This is E8's max(union) > mean(average) in real LLM weights. Two honest riders: (1) the learned router is trivially perfect (1.00) because the three families are lexically distinct — routing's win here rests partly on the routing problem being easy (ambiguous/overlapping skills would make the router the bottleneck — the interesting next failure mode); (2) router-free max_merge is a poor union (0.46) — static per-module winner-take-all isn't input-adaptive, so it collapses toward the strongest-norm modules; the union benefit needs routing, not weight surgery. configs/llm/moe.yaml, plot_llm_moe.py, results/llm_moe/README.md, +2 router tests (127 green). The regime question — does routing still beat fusion once a capable base lets fusion compose rather than dilute (7B soup already beats its specialists)? — is the llm_moe_hpc 7B run below.

Finding (2026-07-05, llm_moe_hpc — the regime flips at 7B; "merge, don't average" is a weak-base law). Re-ran llm_moe at Qwen2.5-7B-Instruct (L40S, 9 min, reusing the cached 7B specialists). The union-vs-fusion ordering inverts: at 0.5B union won (routing 0.74 > soup 0.64); at 7B fusion wins — soup 0.87 > routing 0.84 > max_merge 0.78. Mechanism, and it's the deep point: routing selects one intact specialist so it is capped at the best parent per family (lists 0.57 = spec_lists, strings 0.97 = spec_strings), whereas fusion blends deltas and, at a capable base, composes beyond any parent (soup lists 0.62 > spec 0.57, strings 1.00 > spec 0.97). Selection can't synthesise something better than its best component; averaging-that-composes can. So the E4/E8 "merge, don't average" law is regime-dependent — a weak-parent / small-model law, not universal: union wins exactly when averaging dilutes (0.5B), fusion wins once the base has headroom to compose (7B). This refines rather than contradicts E8 (whose additive-landscape max>mean assumed no compositional headroom). The operator to actually want is fusion-that-composes + selection over recombinant offspring = the "directed sex" ideal (E10), the natural next experiment. results/llm_moe_hpc/ (README + regime-aware figure title). Riders unchanged: learned router trivially perfect (lexical families), max_merge the weakest union (not input-adaptive).

Finding (2026-07-05, llm_directed — directed sex in weights; refinements pay off only when the default blend is suboptimal). E10 in real LLM weights (src/llm/directed.py, kind: llm_directed): breed a population of recombinant offspring (specialists merged at Dirichlet-sampled weights), score each against the verifier on a held-out validation split, keep the fittest — reported on a fresh test split (no selection-on-test leakage). Two breeding objectives (best-overall, best-worst-family). The value scales with how far the uniform soup is from optimal, giving a clean regime split: 0.5B — soup dilutes, so directed selection beats it (directed_overall 0.69 > soup 0.64; directed_balanced worst-family 0.37 > soup 0.26), though single-objective selection trades off the other axis (breeding for overall tanks the rare lists to 0.17) and a global blend still trails per-input routing (0.74). 7B — soup already composes to the ceiling on these near-saturated families (strings & arith at 1.00), so directed selection finds nothing better: directed 0.868 ≈ soup 0.873 (marginally below, a val/test overfit gap). Honest limitation: the 7B families are near-saturated (2/3 at 1.00), which structurally caps the headroom — this run can't separate "directed sex doesn't help at scale" from "these tasks are too easy at 7B"; a harder, unsaturated benchmark is the fair next test. Through-line across all four LLM runs: "merge, don't average" and its refinements (routing, directed selection) are weak-base / suboptimal-default phenomena — they pay off at 0.5B (soup far from optimal) and are inert at 7B (soup near-optimal on saturated tasks). configs/llm/{directed,directed_hpc}.yaml, plot_llm_directed.py, results/llm_directed{,_hpc}/, hpc/llm_directed.pbs, +3 tests (130 green).

Finding (2026-07-05, HARD benchmark llm_moe_hard_hpc + llm_directed_hard_hpc — the 7B "fusion wins / no headroom" results were SATURATION artefacts; the law is HEADROOM, not base-size). The easy families saturated 7B (strings & arith at 1.00), so the 7B nulls (moe: fusion 0.87 > union 0.84; directed ≈ soup) couldn't separate "refinements don't help at scale" from "tasks too easy." Built a hard task variant (hard: true in tasks.py: multi-step lists, Caesar ciphers / letter transforms, multi-step & larger arithmetic — same family labels & answer formats, threaded through make_tasks/train_specialist/runners; hard specialists cache separately as spec_*_hard) and re-ran both at 7B on Imperial CX3 (one L40S, 24 min, unsaturated: arith ≈0.48, strings 0.67, lists 0.34). Both nulls flip back to the 0.5B ordering: (1) union beats fusion again — routing 0.500 > fusion 0.40 (soup 0.392/ties 0.400), the same 10-pt margin as 0.5B; fusion dilutes the fragile strings-specialist so hard (0.665 → soup 0.300) that soup even trails the best single specialist (0.425), while routing keeps it (0.670). (2) directed selection beats soup again — 0.492 > 0.392 (+10 pts), recovering most of routing's benefit from one deployable merged model (lifts strings back to 0.630). Correction to the earlier interpretation: the llm_moe_hpc "regime flip" (fusion wins at 7B) and llm_directed_hpc "no headroom" were both driven by task saturation, not base capability. The operative variable is headroom: "merge, don't average" (union > fusion) and "directed sex" (selection > single blend) hold whenever there's room to lose to dilution — weak base (0.5B) or hard tasks at a strong base (7B-hard); fusion only wins in the degenerate corner where easy tasks let a strong base compose to the 1.00 ceiling. This vindicates E8's max > mean in real 7B weights once saturation is controlled. configs/llm/{moe_hard,moe_hard_hpc,directed_hard_hpc}.yaml, hpc/llm_hard.pbs, results/llm_{moe,directed}_hard_hpc/, +1 hard-task test (131 green).

Finding (2026-09-11, three controls from the manuscript review; results/llm_curriculum_v5_{stop3,decor}/, results/llm_*_hpc/s{1,2,3}/). (1) Forced stop (merge_until: 3): obligate merging through generation 2 then none finishes 0.793 vs the declinable merge's 0.792 (per seed −0.008/−0.006/+0.011) — the veto's outcome is explained by when it stopped. (2) Decorrelated curriculum (orders: key; complementarity 0.00→0.70→0.00 instead of the Latin square's monotone fall): declines still rise with generation (0.44→0.89); pooled partial ρ(declined, complementarity | generation) = −0.07, CI (−0.21, +0.09), partial ρ with generation +0.31. The recombination-modifier / reduction-principle reading of Fig. 4B is withdrawn; the Latin-square ρ = −0.57 was carried by generation (adapter age, skill count and destroyer arrival are confounded). What stands: one bit of selection per merge, or a fixed early stop, avoids the obligate-merge collapse at no cost against never merging. (3) 7B seeds 2–3 (33 min/seed, L40S): merge − best specialist +0.066±0.036, routing − soup +0.094±0.015, directed − soup +0.073±0.031, all 3/3 seeds; not replicated: "soup below the best specialist on hard tasks" (1/3, mean +0.001) — softened in text. Stats: figures/stats_llm_curriculum.py (also the single source of curriculum arm labels, used by make_figs), figures/stats_llm_7b_seeds.py; _figlib.load_seed_bundles reads s{seed}/ layouts.

Finding (2026-09-12, four experiments from the dropped "Limits"; results/llm_speciation/s{1,2,3}, results/llm_curriculum_v5_{early,late,early_obl,late_obl,cull}/s{1,2,3}, results/llm_{merge_seeds,moe_hard_seeds}_smol). GG's rule: a limitation that names a runnable experiment is run, not stated. (1) Speciation seeds 2–3: conflict cliff (merge 0.02/0.12/0.16 vs parents 0.23–0.25) and duration null (0.76→0.95) hold in 3/3 seeds; seed 1's cliff was the deepest. Fig. 5C–D now has CI bands. (2) Conflict-arrival curricula (orders:; boolq/winogrande in generations 1–2 or 5–6, age and skill count rising identically): declines and the obligate collapse follow generation, not conflict arrival (partial ρ with conflict-present, generation controlled: −0.09, CI (−0.45, 0.15); with generation: +0.45); conflict-early dips at arrival, recovers, collapses from generation 5; conflict-late collapses from generation 4 with its pair still to come. What stays confounded is adapter age with skill count. (3) Second base lineage (SmolLM2-1.7B-Instruct, adapters_dir: models/llm_smol because the specialist cache is keyed by family+seed only): Fisher–Muller replicates 5/5 (soup +0.049±0.022, TIES +0.097±0.020 over best specialist), headroom 3/3 with a larger margin (routing − soup +0.162±0.036; soup below best specialist in 3/3). (4) Differential reproduction (cull: true, cull_step/inherit_slot): parity again — with selection, declinable 0.793 vs never-merge 0.804 (−0.011±0.003, 3/3 below); selection − none +0.007±0.030; recombination's early lead present with and without selection, gone by generation 5. The Discussion's prediction (selection turns speed into level) is withdrawn: under a curriculum that delivers every skill to every lineage the ceiling is what one adapter carries. Speciation adapters now live in speciation_s{seed}/ (an array over seeds used to race on a shared dir). SI Figs. S14–S16; stats in figures/stats_llm_{curriculum,speciation_seeds,smol}.py.

Build order (blueprint §7) — respect the gate

1. Scaffold: repo layout (§5), container, pytest skeleton, config system, seeding utils. make test green.
2. Layer 1 core + null model + test_scientific_validation.py against §2.4 predictions 1–2. HARD GATE: do not proceed until simulated drift matches the analytic heterozygosity decay E[Hₜ] = H₀(1 − 1/n)ᵗ.
3. Layer 1 grounding + E1–E2 (the headline result).
4. Layer 1 E3–E6. Layer 1 is now a complete laptop-reproducible paper on its own.
5. Layer 2 scaffold + verifier (test determinism & sandbox isolation before any training).
6. Layer 2 C1 + C3.
7. Layer 2 C2 (+ C4 if compute allows).
8. Reproduction pass.

Do not start Layer 2 until Layer 1's scientific-validation tests pass.

Prescribed structure and commands (do not yet exist — create per blueprint §4–5)

Target module interfaces are given with normative names in blueprint §2.7 (Layer 1) and §3.6 (Layer 2); downstream scripts depend on these signatures, so implement to them exactly. Target repo layout is §5. Planned automation:

make env        # uv sync -> .venv from committed uv.lock
make test       # correctness tests + scientific-validation tests
make layer1     # run E1–E6
make layer2     # run C1–C3 (C4 optional)
make figures    # regenerate every figure from committed results.parquet
make all
./reproduce.sh  # uv sync → test → run all at committed seeds → regen figures → REPRODUCED.md

Single-experiment run pattern: one YAML config per experiment under configs/layer1/EX.yaml or configs/layer2/CX.yaml, fed to the experiment runner. Figures are regenerated separately by figures/plot_EX.py reading only results.parquet (no re-simulation).

Non-negotiable engineering standard (blueprint §4)

• Reproducibility is a hard requirement, not a preference (this is a paper). The environment is a uv venv built from a committed, hash-pinned uv.lock — that lockfile is the source of truth for "it runs" (Apptainer is dropped; a Dockerfile may later wrap the same lockfile for Layer 2's GPU work). Layer 1 is bitwise-reproducible from a single master seed; Layer 2 is statistically reproducible (document residual GPU non-determinism, set determinism flags, report per-seed points).
• Seeding: one master seed in config → derive all sub-seeds via np.random.SeedSequence.spawn. Never touch global RNG state; pass rng explicitly everywhere. Results are a pure function of the resolved config.
• No magic numbers in code. Every parameter lives in a YAML resolved at run time; the resolved config (after sweep expansion) is written next to results. Sweeps are declared in config, not hard-coded.
• Output contract for every run: results.parquet (long form) + resolved_config.yaml + manifest.json (library/CUDA versions, seed, git commit, model revision hashes, content hash of results). Every figure must be a pure function of a committed results artifact.
• Scientific-validation tests are the spine of trust. They assert the simulator reproduces the §2.4 closed forms within tolerance; if they fail, the science is wrong, not just the code. Keep them.
• Open science end-to-end: open-weight models only, permissive/open tooling (uv, MLflow or plain versioned Parquet — avoid closed SaaS trackers), results/ gitignored but hashes tracked.

Stack

Python ≥ 3.11. Layer 1: NumPy, SciPy, pandas, matplotlib — no GPU, no heavy deps. Layer 2: PyTorch, HF transformers + peft (LoRA), datasets, optional vllm; sandboxed subprocess verifier. Config via a thin pydantic + PyYAML loader (not Hydra — its global state/chdir fights the pure-function-of-resolved-config contract). Env via a uv venv from a committed uv.lock — the lockfile is the reproducibility source of truth; Layer 1 needs no container.