# Multi-seed LLM recombination (0.5B) — the claims with error bars

PNAS work-order Phase 3: removes the "one seed" objection on the three LLM recombination claims.
Protocol: **test sets fixed** (seed 1000+i per family), **training seed varied** (specialists cache
per-seed as `spec_<family>[_hard]_s<seed>`), so across-seed variance is training variance only.
Figure: `llm_seeds.png` (this dir) aggregates all three experiments, 95% CI over seeds.

### (A) Fisher–Muller, easy benchmark, 5 seeds (`llm_merge_seeds`)
| model | overall | worst-family |
|---|---|---|
| merge_ties | **0.647 ± 0.027** | **0.282 ± 0.020** |
| merge_soup | 0.632 ± 0.042 | 0.278 ± 0.028 |
| best specialist (strings) | 0.592 ± 0.009 | 0.078 ± 0.011 |
| base | 0.277 | 0.150 |

Both merges beat every specialist overall (ties: non-overlapping CIs; soup: marginal at 0.5B, as in
the single-seed run — decisive at 7B) and the **worst-family signature is unambiguous**: merges ≈0.28
vs ≤0.16 for any parent — only recombined models are competent everywhere.

### (B) Union vs fusion, hard benchmark, 3 seeds (`llm_moe_hard_seeds/`)
Routing (union) beats fusion **in every seed** (3/3 paired, both metrics; e.g. overall per seed:
routing 0.258/0.300/0.262 vs soup 0.212/0.238/0.071). The paired per-seed values also show *why*:
seed 3's soup failed catastrophically (0.071 overall, 0.000 worst-family) while routing was unaffected
(0.262/0.225). Seed-level sd: soup 0.090 vs routing 0.023 — suggestive of a genuine variance
difference, but with 3 seeds this is an observation, not an estimate; the robust statement is the
3/3 paired ordering plus the observed catastrophic-failure mode that routing is structurally immune
to. (Learned router still = oracle: lexically distinct families, known rider.)

### (C) Directed offspring selection, hard, 3 seeds (`llm_directed_hard_seeds/`)
directed_overall beats the a-priori soup **in every seed** (3/3 paired; 0.225/0.242/0.196 vs
0.212/0.238/0.071 — including rescuing soup's catastrophic seed); directed_balanced worst-family
0.158 vs soup 0.088. Directed selection both beats and stabilises the blend (same 3-seed caveat as
panel B); per-input routing remains above any single global blend, as before.

**Read together:** all three recombination claims hold under seed replication with consistent paired
ordering (route > directed-select > soup in every seed on headroom tasks), and the per-seed values
surface a failure mode — occasional catastrophic soup merges — that the union/selection operators
avoided in every observed case. Variance *estimates* await more seeds. Base: Qwen2.5-0.5B-Instruct;
statistical (per-seed) reproducibility per blueprint §4.