giorgio/MachineSex
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Manuscript revision and pending experiment work, snapshot before restructuring

Browse source 
Clarity pass over the main text (36-item audit), Discussion rewrite and cut,
acknowledgements, Souly et al. as ref 62, lettered SI panels, model section
moved under Results; plus the untracked curriculum/society/compose/smol
configs, runners, figures, stats and tests that the SI already cites.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
This commit is contained in:
Giorgio Gilestro 2026-09-13 16:54:09 +01:00
parent e4804adabc
commit 84124de143
450 changed files with 52813 additions and 1202 deletions
Download patch file Download diff file Expand all files Collapse all files

345
tasks/prereg-llm-compose-v3.md Normal file
View file

@ -0,0 +1,345 @@
# Pre-registration — `llm_compose` v3: does a composed capability survive inheritance?
**Status:** draft for GG review, 2026-09-07. Supersedes `prereg-llm-society-v2.md` (calibrated,
no-go at 0.5B) and `workorder-llm-society.md` (v1, run, negative). Nothing runs until §4's gates pass
and GG signs off §12.
**The question, in one sentence.** Every model-merging paper merges *once*; this asks what happens to a
composed capability when the models that carry it keep reproducing, and whether the population-genetic
closed forms predict the trajectory.
---
## 0. Why the design changed, and what carried over
GG's objection to the v2→7B plan (2026-09-07): *"there is no structural reason 7B would succeed if
0.5B failed. Most likely we are simply using the wrong LoRA specialisations… look in the literature
and see what kind of test people use as paradigmatic for LoRA."* Correct on both points. Three things
came out of the literature check:
1. **The paradigmatic test is binary skill composition on a held-out, out-of-domain target.**
[LoRA Soups](https://aclanthology.org/2025.coling-industry.55.pdf) (COLING 2025): Llama-2-7B,
rank 8, math (MetaMathQA) × code (Code Alpaca) → GSM8k-Hard with program-aided evaluation;
also manual × instruction-following → closed-book QA. [LoraHub](https://arxiv.org/abs/2307.13269)
(COLM 2024): many Flan modules → held-out BBH. [MergeBench](https://arxiv.org/pdf/2505.10833):
math, code, multilingual, safety, IF. Nobody uses procedurally generated puzzle families.
2. **Our v1/v2 families were disjoint but *non-composable*.** Sorting a list and counting letters
combine into nothing, so fitness had to be the *average of nine separate objectives* — a
**capacity** test (can one r = 16 adapter hold six skills?), which v2's calibration answered: no,
≈ 0.85× per skill for two, worse for more. E8's genotype model has loci contributing to **one**
fitness function. Composable skills restore that and need only **two** parents per child, so the
capacity ceiling never binds. This, not scale, was the fault.
3. **The operator was wrong in a way with a clean algebraic diagnosis.** peft
`combination_type="linear"` computes ΔW = (α₁B₁+α₂B₂)(α₁A₁+α₂A₂)ᵀ, which carries **cross terms**
B₁A₂ᵀ and B₂A₁ᵀ — one parent's output projection driven by the other's input projection.
`combination_type="cat"` gives α₁B₁A₁ᵀ + α₂B₂A₂ᵀ: each parent's rank-r subspace intact, rank 2r.
**That is E4's union operator in the natural algebra of the medium**, and the cross terms are the
mechanism of blending dilution. Their GSM-Hard numbers: CAT 21.11 > TIES 15.77 > DARE 14.78 >
MoE-routing 13.5 > LoRAHub 4.1 (below the 5.91 base). Routing below CAT is our own `llm_moe_hpc`
reading — selection is capped at the best parent, composition is not.
**Carried over from v2's calibration (not wasted — it calibrated the channel v3 uses).** The
self-consumption inheritance channel: a child distilled from its source's own answers loses 20–40%
per generation to interference from confident off-expertise answers; gating on the source's own
confidence (τ = 0.85) restores single-skill retention to **0.87–0.93**, and v3's lineages carry
**one skill each**, which is exactly the regime the gate was measured in. `k = 300` prompts/skill,
3 epochs, r = 16, confidence gate τ = 0.85.
**Prior art to cite rather than re-demonstrate.** Offspring capability neither parent had is
established: [Akiba et al.](https://www.nature.com/articles/s42256-024-00975-8) (Nature Mach. Intell.,
Japanese × math) and LoRA Soups' *super-linear improvement* (base 5.91 → +code 8.04 → +math 14.18 →
CAT 21.11; ≥ 5% of solved problems solved by neither parent). Population-based LLM evolution with
crossover/mutation/selection exists ([Zhang et al. 2025](https://arxiv.org/abs/2503.01155), 40 models),
as does iterated merging ([M2N2](https://arxiv.org/html/2508.16204v1), EvoGM). **None of them
iterates the *reproduction* loop**: they optimise a merge, they do not ask what a merged capability
does over generations. That gap is the experiment.
**Positioning (GG, 2026-09-07): the manuscript does not need repositioning.** The paper's claim is
that population-genetic *rules describe and predict* the phenomenon — closed forms, thresholds,
conservation laws — not that recombination or collapse were discovered here. §5's H3 is that claim
made falsifiable at the language-model tier. One genuine overlap to co-cite:
[Model Collapse as Cultural Evolution](https://arxiv.org/html/2605.23054) runs ten generations of
self-training and finds rare-variants-lost-first plus quality-filtering-as-remedy — our E1 and E2 in
spirit — but under *iterated learning* (Bayesian convergence to the prior), with no equilibrium
closed form, no threshold, no recombination, and no population structure.
---
## 1. The design
**Two lineages, one measurement.** Two single-skill LoRA lineages on a shared frozen base — a **math**
lineage and a **code** lineage. Each generation, each lineage reproduces by self-consumption (a fresh
LoRA distilled from its own confidence-gated answers on fresh prompts). Each generation, the *current*
two parents are merged and the composed model is evaluated on the held-out composed task.
The composed model is a **measurement, not a lineage** — re-formed each generation from whatever the
parents currently are. This isolates the question ("does composition survive parental drift?") from a
confound ("does the composed model itself drift?"). An optional third arm makes the composed model a
lineage too (§1.4).
**1.1 Task.** Math (MetaMathQA subset) × code (Code Alpaca subset) → **GSM8k-Hard**, program-aided:
the model emits Python, the code is **executed in a sandboxed subprocess**, and the return value is
compared to the reference answer. Execution is the verifier — reality's "no" — and returns Layer 2 to
the blueprint's original §3.6 specification.
**1.2 Operators.** `cat` (union; the campaign operator) and `linear` (blending; the H6 control), both
at merge weights fixed a priori to (0.5, 0.5) — *not* tuned, because a tuned blend would confound the
operator contrast with search. LoRA Soups' learned-CAT is a stronger operator than ours; we do not
need it, and using the untuned version makes the comparison to `linear` clean.
**1.3 Grounding.** The arm knob, in the *training mix* this time (E2's immigration, not E11's
selection channel): the dry arm's children see only the parent's own answers; the grounded arm mixes a
fraction **g = 0.10** of fresh verified real examples (correct answers from the held-out pool of the
lineage's own dataset) into each child's training data. This is the first LLM-tier test of *immigration*
in this project; Fig. 1A's note that training-mix grounding at LLM scale is established elsewhere
stands, but here it is the manipulated variable, not a claim of novelty.
**1.4 Arms.**
| arm | parent reproduction | grounding | operator | tests |
|---|---|---|---|---|
| `dry` | self-consumption | g = 0 | cat | H2, H3, H5 |
| `grounded` | self-consumption | g = 0.10 | cat | H4 |
| `dry_linear` | self-consumption | g = 0 | linear | H6 |
| `dry_composed` *(optional)* | the merged child becomes the next parent of both lineages | g = 0 | cat | does composition survive in a self-consuming *composed* lineage |
**1.5 Generations and seeds.** G = 6 generations, seeds 1–3, fixed now.
**1.6 Measured each generation.** Per lineage: own-skill accuracy (math on MATH-500 subset; code on
HumanEval-subset) → **q_t**, the retained skill. Between lineages: **ρ_t**, the correlation of their
behaviour, measured as in `llm_epistasis` (agreement rate on a shared probe pool, and LoRA-delta
cosine as a geometric companion). Composed: GSM-Hard accuracy, plus the **surplus** (composed − best
parent on the composed task) and the **union-exceedance** (fraction of composed-solved problems that
neither parent solves — LoRA Soups' super-linear signature).
---
## 2. What the framework predicts, quantitatively
E4's closed form for two parents, U(K=2, ρ, q) = ρq + (1−ρ)(1−(1−q)²), gives the expected coverage of
a capability held by either parent. Composition on a two-skill task is the conjunction rather than the
union, so the corresponding prediction for a task needing *both* skills is the product form
**Ĉ_t = c₀ · q_t^math · q_t^code · (1 − ρ_t)/(1 − ρ₀)**
with c₀ fixed by generation 0 (one free scale parameter, fit once, never refit). Two consequences the
merging literature has no reason to expect:
- **Composition decays faster than either parent.** Ĉ depends on the *product* of both retentions and
on decorrelation. If each parent retains 0.9 per generation, the composed capability retains 0.81
before any ρ effect. Super-linear gain becomes super-linear loss.
- **ρ rises under dry self-training**, because both lineages drift toward the same attractor — the
base model's prior. Rising ρ removes the complementarity composition depends on, so the surplus
collapses even where q is still respectable. This is the mechanism, and it is measurable.
---
## 3. Hypotheses, thresholds, falsifiers
Primary outcome: **composition surplus** S_t = (composed GSM-Hard accuracy) − (best single parent on
GSM-Hard), per arm per seed per generation. Secondary: union-exceedance, q_t per lineage, ρ_t, and the
predicted Ĉ_t.
| | Prediction (source) | Threshold | Falsified if |
|---|---|---|---|
| **H1** *(gate, not a claim)* | Generation 0 reproduces the literature: CAT composes | S₀ ≥ +0.05 and union-exceedance ≥ 0.03 and CAT > linear by ≥ 0.03, in ≥ 2 of 3 seeds | any of these fails → the setup does not reproduce a published effect; **stop and fix before iterating** |
| **H2** | Composition decays faster than its parents (§2) | S_t declines monotonically (Spearman ρ ≤ −0.7 vs t) and the composed capability's fractional loss by G exceeds each parent's own fractional loss, in ≥ 2 of 3 seeds | S_t flat or rising, or composed decays no faster than parents |
| **H3** | **The closed form predicts the trajectory** (E4/§2) — the paper's central claim, made falsifiable | Ĉ_t (one parameter, fit at t=0) predicts observed composed accuracy with mean absolute error ≤ 0.05 across t = 1…G, and beats a two-parameter exponential-decay baseline on AIC | MAE > 0.10, or the atheoretical baseline wins → the closed form describes nothing the data did not already say |
| **H4** | Grounding arrests it (E2 immigration) | S_G(grounded) − S_G(dry) ≥ +0.08, paired per seed, 3/3 seeds positive | grounded ≤ dry, or difference < 0.03 |
| **H5** | Rising ρ is the mechanism | ρ_t rises monotonically in `dry` (Spearman ≥ +0.7) and is flat-or-lower in `grounded`; partial correlation of S_t with ρ_t controlling for q_t is negative | ρ flat in dry, or S_t–ρ_t partial correlation ≈ 0 → decay is pure retention loss, not lost complementarity (report either way; it is a mechanism result, not a claim of failure) |
| **H6** | Blending conserves (E4) | `dry_linear` shows S₀ ≤ +0.02 and union-exceedance ≤ 0.01 at every generation — the conservation law, in the operator the literature already shows is worse | linear matches cat at generation 0 → the cross-term account of dilution is wrong |
Analysis: per-seed paired contrasts, mean ± 95% CI over 3 seeds, sign counts reported. H3 is
pre-registered as a *prediction with a fixed functional form and one free parameter*; the fit is at
t = 0 only and is never refit.
---
## 4. Calibration gates (before any campaign)
| Gate | What | Pass criterion | Cost |
|---|---|---|---|
| **C1 base** | Smallest Qwen2.5-Instruct (0.5B → 1.5B → 3B → 7B) whose *generation-0* CAT composition on GSM-Hard lands in **[0.25, 0.70]** | first size in band wins; if 7B exceeds 0.70 the task is saturated and GSM-Hard is swapped for its large-number variant | ≤ 2 h, escalating |
| **C2 verifier** | Execution sandbox: determinism (same code → same verdict ×3), isolation (no filesystem/network), timeout, and agreement with reference answers on 100 gold solutions | 100% determinism, ≥ 0.98 agreement, no escape | 1 h, no GPU |
| **C3 specialists** | Math and code LoRAs each beat base on their *own* skill by ≥ 0.15 and are ≤ 0.4 on the *other* skill (genuine specialists, decorrelated) | both | 1 h |
| **C4 replication** | H1 at generation 0 (above) | as H1 | 1 h |
| **C5 transmission** | Single-skill retention through one gated self-consumption step, per lineage, as v2's C2b | ≥ 0.85 per lineage | 1 h |
C1's escalation is the honest form of the scale question: the base is chosen by *task discriminability*,
not by hope. If 0.5B or 1.5B lands in band, the campaign is cheap.
---
## 4a. Calibration record
**C2 execution verifier — PASS (2026-09-07).** 100/100 agreement with GSM-Hard's own reference
`solution()` functions at 32 ms/item; deterministic across three runs; every hazard contained
(infinite loop → timeout, allocation → memory, write outside the jail → PermissionError, socket →
PermissionError, subprocess → PermissionError, syntax error, recursion). Two bugs the gate caught:
a file write initially escaped (fixed with a `sys.addaudithook` guard) and CPU-rlimit kills were
misreported as errors rather than timeouts. A third surfaced only on CX3 — temp paths are symlinked
there, so the jail check needed `realpath`, not `abspath`.
**C1 base — the escalation axis is instruction-tuning, not size.** Zero-shot GSM-Hard, program-aided:
| base | GSM-Hard | executable | GSM8K |
|---|---|---|---|
| Qwen2.5-1.5B-**Instruct** | 0.500 | 0.950 | — |
| Qwen2.5-3B-**Instruct** | 0.417 | 0.617 | — |
| Qwen2.5-3B (base) | 0.633 | 0.950 | 0.750 |
| **Qwen2.5-1.5B (base)** | **0.067** | 0.117 | 0.117 |
A base that already has the skills makes the specialists vacuous — the v2 disease in new clothes, and
it would have been *worse* at 7B, which is the quantitative form of GG's objection to the 7B plan.
Qwen2.5-1.5B base is within noise of LoRA Soups' Llama-2-7B starting point (0.059), so the escalation
runs along instruction-tuning rather than parameter count. **Amendment:** C1's band applies to the
*base* (≤ 0.15) as well as to the composed model ([0.25, 0.70]).
**C3 specialists + C4 replication (`results/llm_compose_gate`) — H1 FAILS, with a clean diagnosis.**
| | composed (GSM-Hard) | executable | own skill |
|---|---|---|---|
| math parent (MetaMathQA) | 0.073 | 0.153 | GSM8K 0.620 |
| code parent (CodeAlpaca) | **0.427** | 0.953 | MBPP 0.075 |
| cat merge, 0.5/0.5 | 0.407 | 0.827 | — |
Surplus **−0.020** (needs ≥ +0.05) → the pre-registered gate fails and no campaign is submitted on
this configuration. But **union-exceedance is 0.073**: the merge solves 7.3% of items that *neither*
parent solves, so composition is occurring and is being cancelled by a format cost (executability
0.953 → 0.827 when the non-code parent is blended in).
**The skill pair is unbalanced for this base, and the measurement says so precisely.** In the
published setup math-only (0.142) beats code-only (0.080); here the ordering is *inverted* — code-only
0.427, math-only 0.073 — because Qwen2.5's pretraining already carries the maths, so **code/format is
the scarce skill and maths is not**. E8's premise is that each parent supplies something the child
could not otherwise get; that holds for the code parent and fails for the math parent.
**Diagnostic before any redesign (running):** a merge-weight sweep (0.5/0.5 → 0.1/0.9) under both
operators, reusing the cached founders. It separates two possibilities that the single 0.5/0.5 point
cannot: *(i)* the surplus is positive somewhere in weight space and 0.5/0.5 was a strawman — in which
case the correct experiment is E10's directed version (breed offspring across weights, select on the
verifier), which is this project's own operator and was fixed to 0.5/0.5 only to keep the operator
contrast clean; or *(ii)* no weighting yields a positive surplus, in which case the pair is simply
wrong for this base and the fix is a target whose *maths* the base cannot do (competition-level MATH
program-aided), not a different merge.
**Merge-weight sweep (cat, GSM8k-Hard, founders reused) — 0.5/0.5 was a strawman, and there is an
interior optimum.**
| math/code | composed | surplus | union-exceedance | executable |
|---|---|---|---|---|
| 0.5/0.5 | 0.407 | −0.020 | 0.073 | 0.827 |
| **0.3/0.7** | **0.453** | **+0.027** | **0.093** | 0.967 |
| 0.2/0.8 | 0.433 | +0.007 | 0.080 | 0.967 |
| 0.1/0.9 | 0.433 | +0.007 | 0.040 | 0.960 |
The surplus is positive over a range and peaks at an *interior* weight — E9's "optimal recombination
rate is intermediate", in real weights — and the executability cost of blending disappears once the
non-code parent is down-weighted (0.827 → 0.967, above even the code parent's 0.953). It is also
LoRA Soups' own result that *learned* CAT beats *static* CAT, arrived at independently. **Amendment
(adopted):** merge weights are chosen per generation on a **disjoint validation split** of the
composed target and reported on the test split — E10's directed recombination, which was fixed at
0.5/0.5 in §1.2 only to keep the operator contrast clean. The `linear` control arm keeps the same
treatment, so the operator contrast survives. `_split_pool` makes val/test disjointness structural
rather than a property of seeds, which matters at MATH-500's pool size.
**Still short of the gate: peak surplus +0.027 against a +0.05 threshold, and n = 150 gives
SE ≈ 0.04.** The bar is high because the *best parent* is at 0.427, where LoRA Soups' was 0.142.
So the second gate configuration (`configs/llm/compose_gate_math500.yaml`) moves the target to
MATH-500 rather than moving the threshold.
**Full gen-0 sweep, both operators (GSM8k-Hard, 150 items, best parent 0.427):**
| operator | 0.5/0.5 | 0.3/0.7 | 0.2/0.8 | 0.1/0.9 | range |
|---|---|---|---|---|---|
| cat | 0.407 | **0.453** | 0.433 | 0.433 | 0.046 |
| linear | 0.333 | 0.467 | **0.507** | 0.413 | 0.174 |
**H6 as pre-registered is FALSIFIED, and what replaces it is more interesting.** The prediction was
that blending "conserves" — no composition to lose. In fact linear blending produces the *largest*
union-exceedance (0.133 vs cat's 0.093) and the highest composed accuracy of any configuration
(0.507, surplus +0.080), *provided the weight is chosen*. What distinguishes the operators is
**variance, not mean**: concatenation is nearly flat in the blend ratio (range 0.046) and never
catastrophic, while blending swings by 0.174 — worst of all at equal weights (0.333, executability
0.680, the cross-terms wrecking the code parent's format), best of all at 0.2/0.8. That is E9's
structure in real weights: blind recombination → outbreeding depression; directed recombination →
gain; the union operator is the conservative strategy. Revised H6 (recorded before the campaign):
*the operator ordering is weight-dependent at generation 0; does it stay so across generations, or
does one operator degrade faster?* — measured by the `dry_cat` arm.
**Second target, MATH-500 (`results/llm_compose_gate_math500`, cat + selected weights): composed
0.200, best parent 0.158, surplus +0.042, union-exceedance 0.100.** The balanced-pair prediction
holds — a much weaker best parent (0.158 vs 0.427) leaves more headroom, and cat's surplus rises from
+0.027 to +0.042. The composition effect therefore reproduces on **two independent targets**, at the
cost of one extra evaluation pass since the founders are shared.
**Gate verdict: PASS on the amended configuration** (directed weight selection; GSM8k-Hard primary,
MATH-500 as the generality check). Campaign launched 2026-09-07: seed 1 local, seeds 2-3 as CX3 array
`4000472` (6 elements, 3 arms x 2 seeds).
## 5. Cost
Per generation-arm: 2 lineages × (2700 gated inheritance answers + train ~1000 × 3 epochs) +
composed eval on 300 GSM-Hard items with execution. At **1.5B**: ≈ 25 min. At **7B**: ≈ 75 min.
| base (from C1) | arms | seeds | G | total |
|---|---|---|---|---|
| 1.5B | 3 | 3 | 6 | **≈ 12 GPU-h** (local, overnight) |
| 3B | 3 | 3 | 6 | ≈ 25 GPU-h (local or 4 CX3 jobs) |
| 7B | 3 | 3 | 6 | ≈ 40 L40S-h (9 array elements, 6 h each) |
The optional `dry_composed` arm adds a third. Every figure in §3 is drawn from one parquet;
checkpoint/resume carries over from `society_v2.py`.
---
## 6. Anticipated failure modes
- **Gen-0 does not compose (C4 fails).** Most likely cause is the base being too weak for
program-aided math at all. C1's escalation should prevent it; if it survives C1, stop — the
experiment has no signal to measure the decay of.
- **Parents don't drift.** If gated self-consumption is *too* good, q stays ≈ 1 and there is nothing
to observe. Mitigation: the gate is a knob (v2 measured τ = 0.5 → 0.87 and ungated → 0.81); if q_6 >
0.9 in the dry arm at τ = 0.85, drop to ungated, which is the *more* faithful self-consumption
channel anyway. Pre-declared, not a post-hoc rescue.
- **ρ unmeasurable.** Math and code lineages answer disjoint prompt types, so behavioural agreement may
be uninformative. Fallback: ρ from LoRA-delta cosine (already implemented in `epistasis.py`) and
from agreement on the *shared* GSM-Hard prompts.
- **Execution verifier flakiness** — timeouts counted as failures, reported as a rate.
- **GSM-Hard contamination** in an Instruct base: report the base's zero-shot GSM-Hard number; if it
is implausibly high, switch to the perturbed-number variant.
---
## 7. Engineering checklist
- [ ] `src/llm/execute.py` — sandboxed subprocess execution verifier (timeout, no network, no
filesystem writes, deterministic), + tests.
- [ ] `src/llm/datasets.py` — MetaMathQA / Code Alpaca / GSM8k-Hard loaders, fixed subsets, cached.
- [ ] `src/llm/compose.py` — `kind: llm_compose`; two lineages, per-generation merge-and-measure,
`cat`/`linear`, grounding fraction, confidence gate, checkpoint/resume (port from `society_v2.py`).
- [ ] q_t / ρ_t instrumentation (reuse `epistasis.generate_with_confidence`, `delta_geometry`).
- [ ] `configs/llm/compose_calib_{c1..c5}.yaml`, `compose_s{1,2,3}.yaml`, `hpc/llm_compose.pbs`.
- [ ] `figures/plot_llm_compose.py` (4 panels: S_t per arm; q_t per lineage; ρ_t; observed vs Ĉ_t) and
`figures/stats_llm_compose.py` (H2–H6) — **written before unblinding**, as in v2.
- [ ] Smoke: 1.5B, G = 2, all arms, 50 eval items.
## 8. Outcome → manuscript
| Outcome | What changes |
|---|---|
| H1–H4 pass, H3 within tolerance | Fig. 1A's "open — the stated gap" cell is filled by a *different and better* experiment than the one specified: the closed form predicting a real LLM capability trajectory over generations. New figure; the recombination and grounding sections each gain their language-model rung. |
| H2 + H4 pass, H3 fails | The signs transfer, the quantitative law does not — report as such; the paper's predictive claim stays anchored on the biological model, and the LLM tier is confirmatory (which is what Fig. 1A already says of the other rows). |
| H1 fails at every base size | No campaign. The SI records the 0.5B transmission ceiling (v2) plus the failure to reproduce a published composition effect in our harness — an infrastructure result, honestly labelled. |
## 9. Decisions for GG
1. **Base escalation cap** — stop at 3B (cheap, local, likely enough) or allow 7B if C1 demands it?
2. **Optional `dry_composed` arm** (+33% cost): does composition survive when the *merged* model is
itself the reproducing lineage? It is the closest thing to the original society question.
3. **Skill pair** — math × code (best-anchored to the literature) or a second pair alongside
(manual × instruction-following) for generality at double the cost?
4. Whether the v2 no-go and this redesign are worth a short **SI subsection on negative results**, or
stay in the repository record only.