MachineSex/tasks/prereg-llm-compose-v3.md

Pre-registration — llm_compose v3: does a composed capability survive inheritance?

Status: draft for GG review, 2026-09-07. Supersedes prereg-llm-society-v2.md (calibrated, no-go at 0.5B) and workorder-llm-society.md (v1, run, negative). Nothing runs until §4's gates pass and GG signs off §12.

The question, in one sentence. Every model-merging paper merges once; this asks what happens to a composed capability when the models that carry it keep reproducing, and whether the population-genetic closed forms predict the trajectory.

---

0. Why the design changed, and what carried over

GG's objection to the v2→7B plan (2026-09-07): "there is no structural reason 7B would succeed if 0.5B failed. Most likely we are simply using the wrong LoRA specialisations… look in the literature and see what kind of test people use as paradigmatic for LoRA." Correct on both points. Three things came out of the literature check:

1. The paradigmatic test is binary skill composition on a held-out, out-of-domain target. LoRA Soups (COLING 2025): Llama-2-7B, rank 8, math (MetaMathQA) × code (Code Alpaca) → GSM8k-Hard with program-aided evaluation; also manual × instruction-following → closed-book QA. LoraHub (COLM 2024): many Flan modules → held-out BBH. MergeBench: math, code, multilingual, safety, IF. Nobody uses procedurally generated puzzle families.

2. Our v1/v2 families were disjoint but non-composable. Sorting a list and counting letters combine into nothing, so fitness had to be the average of nine separate objectives — a capacity test (can one r = 16 adapter hold six skills?), which v2's calibration answered: no, ≈ 0.85× per skill for two, worse for more. E8's genotype model has loci contributing to one fitness function. Composable skills restore that and need only two parents per child, so the capacity ceiling never binds. This, not scale, was the fault.

3. The operator was wrong in a way with a clean algebraic diagnosis. peft combination_type="linear" computes ΔW = (α₁B₁+α₂B₂)(α₁A₁+α₂A₂)ᵀ, which carries cross terms B₁A₂ᵀ and B₂A₁ᵀ — one parent's output projection driven by the other's input projection. combination_type="cat" gives α₁B₁A₁ᵀ + α₂B₂A₂ᵀ: each parent's rank-r subspace intact, rank 2r. That is E4's union operator in the natural algebra of the medium, and the cross terms are the mechanism of blending dilution. Their GSM-Hard numbers: CAT 21.11 > TIES 15.77 > DARE 14.78 > MoE-routing 13.5 > LoRAHub 4.1 (below the 5.91 base). Routing below CAT is our own llm_moe_hpc reading — selection is capped at the best parent, composition is not.

Carried over from v2's calibration (not wasted — it calibrated the channel v3 uses). The self-consumption inheritance channel: a child distilled from its source's own answers loses 20–40% per generation to interference from confident off-expertise answers; gating on the source's own confidence (τ = 0.85) restores single-skill retention to 0.87–0.93, and v3's lineages carry one skill each, which is exactly the regime the gate was measured in. k = 300 prompts/skill, 3 epochs, r = 16, confidence gate τ = 0.85.

Prior art to cite rather than re-demonstrate. Offspring capability neither parent had is established: Akiba et al. (Nature Mach. Intell., Japanese × math) and LoRA Soups' super-linear improvement (base 5.91 → +code 8.04 → +math 14.18 → CAT 21.11; ≥ 5% of solved problems solved by neither parent). Population-based LLM evolution with crossover/mutation/selection exists (Zhang et al. 2025, 40 models), as does iterated merging (M2N2, EvoGM). None of them iterates the reproduction loop: they optimise a merge, they do not ask what a merged capability does over generations. That gap is the experiment.

Positioning (GG, 2026-09-07): the manuscript does not need repositioning. The paper's claim is that population-genetic rules describe and predict the phenomenon — closed forms, thresholds, conservation laws — not that recombination or collapse were discovered here. §5's H3 is that claim made falsifiable at the language-model tier. One genuine overlap to co-cite: Model Collapse as Cultural Evolution runs ten generations of self-training and finds rare-variants-lost-first plus quality-filtering-as-remedy — our E1 and E2 in spirit — but under iterated learning (Bayesian convergence to the prior), with no equilibrium closed form, no threshold, no recombination, and no population structure.

---

1. The design

Two lineages, one measurement. Two single-skill LoRA lineages on a shared frozen base — a math lineage and a code lineage. Each generation, each lineage reproduces by self-consumption (a fresh LoRA distilled from its own confidence-gated answers on fresh prompts). Each generation, the current two parents are merged and the composed model is evaluated on the held-out composed task.

The composed model is a measurement, not a lineage — re-formed each generation from whatever the parents currently are. This isolates the question ("does composition survive parental drift?") from a confound ("does the composed model itself drift?"). An optional third arm makes the composed model a lineage too (§1.4).

1.1 Task. Math (MetaMathQA subset) × code (Code Alpaca subset) → GSM8k-Hard, program-aided: the model emits Python, the code is executed in a sandboxed subprocess, and the return value is compared to the reference answer. Execution is the verifier — reality's "no" — and returns Layer 2 to the blueprint's original §3.6 specification.

1.2 Operators. cat (union; the campaign operator) and linear (blending; the H6 control), both at merge weights fixed a priori to (0.5, 0.5) — not tuned, because a tuned blend would confound the operator contrast with search. LoRA Soups' learned-CAT is a stronger operator than ours; we do not need it, and using the untuned version makes the comparison to linear clean.

1.3 Grounding. The arm knob, in the training mix this time (E2's immigration, not E11's selection channel): the dry arm's children see only the parent's own answers; the grounded arm mixes a fraction g = 0.10 of fresh verified real examples (correct answers from the held-out pool of the lineage's own dataset) into each child's training data. This is the first LLM-tier test of immigration in this project; Fig. 1A's note that training-mix grounding at LLM scale is established elsewhere stands, but here it is the manipulated variable, not a claim of novelty.

1.4 Arms.

arm	parent reproduction	grounding	operator	tests
dry	self-consumption	g = 0	cat	H2, H3, H5
grounded	self-consumption	g = 0.10	cat	H4
dry_linear	self-consumption	g = 0	linear	H6
dry_composed (optional)	the merged child becomes the next parent of both lineages	g = 0	cat	does composition survive in a self-consuming composed lineage

1.5 Generations and seeds. G = 6 generations, seeds 1–3, fixed now.

1.6 Measured each generation. Per lineage: own-skill accuracy (math on MATH-500 subset; code on HumanEval-subset) → q_t, the retained skill. Between lineages: ρ_t, the correlation of their behaviour, measured as in llm_epistasis (agreement rate on a shared probe pool, and LoRA-delta cosine as a geometric companion). Composed: GSM-Hard accuracy, plus the surplus (composed − best parent on the composed task) and the union-exceedance (fraction of composed-solved problems that neither parent solves — LoRA Soups' super-linear signature).

---

2. What the framework predicts, quantitatively

E4's closed form for two parents, U(K=2, ρ, q) = ρq + (1−ρ)(1−(1−q)²), gives the expected coverage of a capability held by either parent. Composition on a two-skill task is the conjunction rather than the union, so the corresponding prediction for a task needing both skills is the product form

Ĉ_t = c₀ · q_t^math · q_t^code · (1 − ρ_t)/(1 − ρ₀)

with c₀ fixed by generation 0 (one free scale parameter, fit once, never refit). Two consequences the merging literature has no reason to expect:

• Composition decays faster than either parent. Ĉ depends on the product of both retentions and on decorrelation. If each parent retains 0.9 per generation, the composed capability retains 0.81 before any ρ effect. Super-linear gain becomes super-linear loss.
• ρ rises under dry self-training, because both lineages drift toward the same attractor — the base model's prior. Rising ρ removes the complementarity composition depends on, so the surplus collapses even where q is still respectable. This is the mechanism, and it is measurable.

---

3. Hypotheses, thresholds, falsifiers

Primary outcome: composition surplus S_t = (composed GSM-Hard accuracy) − (best single parent on GSM-Hard), per arm per seed per generation. Secondary: union-exceedance, q_t per lineage, ρ_t, and the predicted Ĉ_t.

	Prediction (source)	Threshold	Falsified if
H1 (gate, not a claim)	Generation 0 reproduces the literature: CAT composes	S₀ ≥ +0.05 and union-exceedance ≥ 0.03 and CAT > linear by ≥ 0.03, in ≥ 2 of 3 seeds	any of these fails → the setup does not reproduce a published effect; stop and fix before iterating
H2	Composition decays faster than its parents (§2)	S_t declines monotonically (Spearman ρ ≤ −0.7 vs t) and the composed capability's fractional loss by G exceeds each parent's own fractional loss, in ≥ 2 of 3 seeds	S_t flat or rising, or composed decays no faster than parents
H3	The closed form predicts the trajectory (E4/§2) — the paper's central claim, made falsifiable	Ĉ_t (one parameter, fit at t=0) predicts observed composed accuracy with mean absolute error ≤ 0.05 across t = 1…G, and beats a two-parameter exponential-decay baseline on AIC	MAE > 0.10, or the atheoretical baseline wins → the closed form describes nothing the data did not already say
H4	Grounding arrests it (E2 immigration)	S_G(grounded) − S_G(dry) ≥ +0.08, paired per seed, 3/3 seeds positive	grounded ≤ dry, or difference < 0.03
H5	Rising ρ is the mechanism	ρ_t rises monotonically in dry (Spearman ≥ +0.7) and is flat-or-lower in grounded; partial correlation of S_t with ρ_t controlling for q_t is negative	ρ flat in dry, or S_t–ρ_t partial correlation ≈ 0 → decay is pure retention loss, not lost complementarity (report either way; it is a mechanism result, not a claim of failure)
H6	Blending conserves (E4)	dry_linear shows S₀ ≤ +0.02 and union-exceedance ≤ 0.01 at every generation — the conservation law, in the operator the literature already shows is worse	linear matches cat at generation 0 → the cross-term account of dilution is wrong

Analysis: per-seed paired contrasts, mean ± 95% CI over 3 seeds, sign counts reported. H3 is pre-registered as a prediction with a fixed functional form and one free parameter; the fit is at t = 0 only and is never refit.

---

4. Calibration gates (before any campaign)

Gate	What	Pass criterion	Cost
C1 base	Smallest Qwen2.5-Instruct (0.5B → 1.5B → 3B → 7B) whose generation-0 CAT composition on GSM-Hard lands in [0.25, 0.70]	first size in band wins; if 7B exceeds 0.70 the task is saturated and GSM-Hard is swapped for its large-number variant	≤ 2 h, escalating
C2 verifier	Execution sandbox: determinism (same code → same verdict ×3), isolation (no filesystem/network), timeout, and agreement with reference answers on 100 gold solutions	100% determinism, ≥ 0.98 agreement, no escape	1 h, no GPU
C3 specialists	Math and code LoRAs each beat base on their own skill by ≥ 0.15 and are ≤ 0.4 on the other skill (genuine specialists, decorrelated)	both	1 h
C4 replication	H1 at generation 0 (above)	as H1	1 h
C5 transmission	Single-skill retention through one gated self-consumption step, per lineage, as v2's C2b	≥ 0.85 per lineage	1 h

C1's escalation is the honest form of the scale question: the base is chosen by task discriminability, not by hope. If 0.5B or 1.5B lands in band, the campaign is cheap.

---

4a. Calibration record

C2 execution verifier — PASS (2026-09-07). 100/100 agreement with GSM-Hard's own reference solution() functions at 32 ms/item; deterministic across three runs; every hazard contained (infinite loop → timeout, allocation → memory, write outside the jail → PermissionError, socket → PermissionError, subprocess → PermissionError, syntax error, recursion). Two bugs the gate caught: a file write initially escaped (fixed with a sys.addaudithook guard) and CPU-rlimit kills were misreported as errors rather than timeouts. A third surfaced only on CX3 — temp paths are symlinked there, so the jail check needed realpath, not abspath.

C1 base — the escalation axis is instruction-tuning, not size. Zero-shot GSM-Hard, program-aided:

base	GSM-Hard	executable	GSM8K
Qwen2.5-1.5B-Instruct	0.500	0.950	—
Qwen2.5-3B-Instruct	0.417	0.617	—
Qwen2.5-3B (base)	0.633	0.950	0.750
Qwen2.5-1.5B (base)	0.067	0.117	0.117

A base that already has the skills makes the specialists vacuous — the v2 disease in new clothes, and it would have been worse at 7B, which is the quantitative form of GG's objection to the 7B plan. Qwen2.5-1.5B base is within noise of LoRA Soups' Llama-2-7B starting point (0.059), so the escalation runs along instruction-tuning rather than parameter count. Amendment: C1's band applies to the base (≤ 0.15) as well as to the composed model ([0.25, 0.70]).

C3 specialists + C4 replication (results/llm_compose_gate) — H1 FAILS, with a clean diagnosis.

	composed (GSM-Hard)	executable	own skill
math parent (MetaMathQA)	0.073	0.153	GSM8K 0.620
code parent (CodeAlpaca)	0.427	0.953	MBPP 0.075
cat merge, 0.5/0.5	0.407	0.827	—

Surplus −0.020 (needs ≥ +0.05) → the pre-registered gate fails and no campaign is submitted on this configuration. But union-exceedance is 0.073: the merge solves 7.3% of items that neither parent solves, so composition is occurring and is being cancelled by a format cost (executability 0.953 → 0.827 when the non-code parent is blended in).

The skill pair is unbalanced for this base, and the measurement says so precisely. In the published setup math-only (0.142) beats code-only (0.080); here the ordering is inverted — code-only 0.427, math-only 0.073 — because Qwen2.5's pretraining already carries the maths, so code/format is the scarce skill and maths is not. E8's premise is that each parent supplies something the child could not otherwise get; that holds for the code parent and fails for the math parent.

Diagnostic before any redesign (running): a merge-weight sweep (0.5/0.5 → 0.1/0.9) under both operators, reusing the cached founders. It separates two possibilities that the single 0.5/0.5 point cannot: (i) the surplus is positive somewhere in weight space and 0.5/0.5 was a strawman — in which case the correct experiment is E10's directed version (breed offspring across weights, select on the verifier), which is this project's own operator and was fixed to 0.5/0.5 only to keep the operator contrast clean; or (ii) no weighting yields a positive surplus, in which case the pair is simply wrong for this base and the fix is a target whose maths the base cannot do (competition-level MATH program-aided), not a different merge.

Merge-weight sweep (cat, GSM8k-Hard, founders reused) — 0.5/0.5 was a strawman, and there is an interior optimum.

math/code	composed	surplus	union-exceedance	executable
0.5/0.5	0.407	−0.020	0.073	0.827
0.3/0.7	0.453	+0.027	0.093	0.967
0.2/0.8	0.433	+0.007	0.080	0.967
0.1/0.9	0.433	+0.007	0.040	0.960

The surplus is positive over a range and peaks at an interior weight — E9's "optimal recombination rate is intermediate", in real weights — and the executability cost of blending disappears once the non-code parent is down-weighted (0.827 → 0.967, above even the code parent's 0.953). It is also LoRA Soups' own result that learned CAT beats static CAT, arrived at independently. Amendment (adopted): merge weights are chosen per generation on a disjoint validation split of the composed target and reported on the test split — E10's directed recombination, which was fixed at 0.5/0.5 in §1.2 only to keep the operator contrast clean. The linear control arm keeps the same treatment, so the operator contrast survives. _split_pool makes val/test disjointness structural rather than a property of seeds, which matters at MATH-500's pool size.

Still short of the gate: peak surplus +0.027 against a +0.05 threshold, and n = 150 gives SE ≈ 0.04. The bar is high because the best parent is at 0.427, where LoRA Soups' was 0.142. So the second gate configuration (configs/llm/compose_gate_math500.yaml) moves the target to MATH-500 rather than moving the threshold.

Full gen-0 sweep, both operators (GSM8k-Hard, 150 items, best parent 0.427):

operator	0.5/0.5	0.3/0.7	0.2/0.8	0.1/0.9	range
cat	0.407	0.453	0.433	0.433	0.046
linear	0.333	0.467	0.507	0.413	0.174

H6 as pre-registered is FALSIFIED, and what replaces it is more interesting. The prediction was that blending "conserves" — no composition to lose. In fact linear blending produces the largest union-exceedance (0.133 vs cat's 0.093) and the highest composed accuracy of any configuration (0.507, surplus +0.080), provided the weight is chosen. What distinguishes the operators is variance, not mean: concatenation is nearly flat in the blend ratio (range 0.046) and never catastrophic, while blending swings by 0.174 — worst of all at equal weights (0.333, executability 0.680, the cross-terms wrecking the code parent's format), best of all at 0.2/0.8. That is E9's structure in real weights: blind recombination → outbreeding depression; directed recombination → gain; the union operator is the conservative strategy. Revised H6 (recorded before the campaign): the operator ordering is weight-dependent at generation 0; does it stay so across generations, or does one operator degrade faster? — measured by the dry_cat arm.

Second target, MATH-500 (results/llm_compose_gate_math500, cat + selected weights): composed 0.200, best parent 0.158, surplus +0.042, union-exceedance 0.100. The balanced-pair prediction holds — a much weaker best parent (0.158 vs 0.427) leaves more headroom, and cat's surplus rises from +0.027 to +0.042. The composition effect therefore reproduces on two independent targets, at the cost of one extra evaluation pass since the founders are shared.

Gate verdict: PASS on the amended configuration (directed weight selection; GSM8k-Hard primary, MATH-500 as the generality check). Campaign launched 2026-09-07: seed 1 local, seeds 2-3 as CX3 array 4000472 (6 elements, 3 arms x 2 seeds).

5. Cost

Per generation-arm: 2 lineages × (2700 gated inheritance answers + train ~1000 × 3 epochs) + composed eval on 300 GSM-Hard items with execution. At 1.5B: ≈ 25 min. At 7B: ≈ 75 min.

base (from C1)	arms	seeds	G	total
1.5B	3	3	6	≈ 12 GPU-h (local, overnight)
3B	3	3	6	≈ 25 GPU-h (local or 4 CX3 jobs)
7B	3	3	6	≈ 40 L40S-h (9 array elements, 6 h each)

The optional dry_composed arm adds a third. Every figure in §3 is drawn from one parquet; checkpoint/resume carries over from society_v2.py.

---

6. Anticipated failure modes

• Gen-0 does not compose (C4 fails). Most likely cause is the base being too weak for program-aided math at all. C1's escalation should prevent it; if it survives C1, stop — the experiment has no signal to measure the decay of.
• Parents don't drift. If gated self-consumption is too good, q stays ≈ 1 and there is nothing to observe. Mitigation: the gate is a knob (v2 measured τ = 0.5 → 0.87 and ungated → 0.81); if q_6 > 0.9 in the dry arm at τ = 0.85, drop to ungated, which is the more faithful self-consumption channel anyway. Pre-declared, not a post-hoc rescue.
• ρ unmeasurable. Math and code lineages answer disjoint prompt types, so behavioural agreement may be uninformative. Fallback: ρ from LoRA-delta cosine (already implemented in epistasis.py) and from agreement on the shared GSM-Hard prompts.
• Execution verifier flakiness — timeouts counted as failures, reported as a rate.
• GSM-Hard contamination in an Instruct base: report the base's zero-shot GSM-Hard number; if it is implausibly high, switch to the perturbed-number variant.

---

7. Engineering checklist

• src/llm/execute.py — sandboxed subprocess execution verifier (timeout, no network, no filesystem writes, deterministic), + tests.
• src/llm/datasets.py — MetaMathQA / Code Alpaca / GSM8k-Hard loaders, fixed subsets, cached.
• src/llm/compose.py — kind: llm_compose; two lineages, per-generation merge-and-measure, cat/linear, grounding fraction, confidence gate, checkpoint/resume (port from society_v2.py).
• q_t / ρ_t instrumentation (reuse epistasis.generate_with_confidence, delta_geometry).
• configs/llm/compose_calib_{c1..c5}.yaml, compose_s{1,2,3}.yaml, hpc/llm_compose.pbs.
• figures/plot_llm_compose.py (4 panels: S_t per arm; q_t per lineage; ρ_t; observed vs Ĉ_t) and figures/stats_llm_compose.py (H2–H6) — written before unblinding, as in v2.
• Smoke: 1.5B, G = 2, all arms, 50 eval items.

8. Outcome → manuscript

Outcome	What changes
H1–H4 pass, H3 within tolerance	Fig. 1A's "open — the stated gap" cell is filled by a different and better experiment than the one specified: the closed form predicting a real LLM capability trajectory over generations. New figure; the recombination and grounding sections each gain their language-model rung.
H2 + H4 pass, H3 fails	The signs transfer, the quantitative law does not — report as such; the paper's predictive claim stays anchored on the biological model, and the LLM tier is confirmatory (which is what Fig. 1A already says of the other rows).
H1 fails at every base size	No campaign. The SI records the 0.5B transmission ceiling (v2) plus the failure to reproduce a published composition effect in our harness — an infrastructure result, honestly labelled.

9. Decisions for GG

1. Base escalation cap — stop at 3B (cheap, local, likely enough) or allow 7B if C1 demands it?
2. Optional dry_composed arm (+33% cost): does composition survive when the merged model is itself the reproducing lineage? It is the closest thing to the original society question.
3. Skill pair — math × code (best-anchored to the literature) or a second pair alongside (manual × instruction-following) for generality at double the cost?
4. Whether the v2 no-go and this redesign are worth a short SI subsection on negative results, or stay in the repository record only.