Phase 3: LLM-tier speciation + multi-seed firm-up of the recombination claims

llm_speciation (new kind; src/llm/speciation.py): E13 in LLM weights.
LoRA children share the frozen base's coordinates, so merge failure is
functional by construction. CONFLICT (ambiguous sort prompts learned
under opposite conventions — the BDM structure): function-specific
hybrid breakdown — merged coherence 0.02-0.08 falls below BOTH parents
(~0.2) on the conflicted function; and in the de-confounded `add` design
(private budget fixed, conflict added on top; 3 seeds after a
single-seed pilot showed one anomalous point) the merge's private-family
accuracy shows NO trend with conflict — the damage is surgical, not
global. DURATION (over-trained disjoint specialists, 1->12 epochs): the
merge improves (0.84->0.94) and stays above the best parent — the MLP
"no emergent isolation" null generalises; relevant to the
expert-training-duration report (2607.11997), with the epistasis
prediction left to the decisive experiment.

Multi-seed firm-up (seeds threaded into specialist caches; `seeds:` list
support in the runner; fixed test sets): all three recombination claims
hold with CIs — merges beat every specialist (5 seeds, ties
0.647±0.027 > best spec 0.592±0.009; worst-family 0.28 vs <=0.16); union
0.274±0.026 > fusion 0.174±0.102 on hard (3 seeds); directed 0.221±0.026
> soup. NEW finding: fusion is seed-FRAGILE where headroom exists
(CI ±0.10) while routing/directed selection are stable (±0.026) — the
union/selection operators win on reliability, not just mean.

Figures (llm_speciation 3-panel; llm_seeds 3-panel with 95% CI), READMEs,
+1 convention test (150 green), make llm-speciation / llm-seeds targets.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v

results/llm_directed_hard_seeds/README.md

@@ -0,0 +1,6 @@
+# Multi-seed directed offspring selection, hard benchmark (0.5B, 3 seeds)
+
+Part of the multi-seed firm-up; full legend and table in `results/llm_merge_seeds/README.md`
+(panel C of its `llm_seeds.png`). Headline: directed_overall 0.221 ± 0.026 beats the a-priori soup
+(0.174 ± 0.102) and the best specialist, and directed_balanced more than doubles the soup's
+worst-family (0.158 vs 0.088) — offspring selection both improves and stabilises the blend.

results/llm_directed_hard_seeds/manifest.json

@@ -0,0 +1,31 @@
+{
+  "experiment": "llm_directed_hard_seeds",
+  "master_seed": 1,
+  "git_commit": "d6a5c5cacde5544a39305eef34925a4fb35b8a19",
+  "python": "3.14.7",
+  "libraries": {
+    "numpy": "2.5.0",
+    "scipy": "1.18.0",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.13.0",
+    "peft": "0.19.1"
+  },
+  "rows": 105,
+  "results_sha256": "532f6115402fce2a610baf7879bb81653bc61e9f64092160d3dc5d9f9d516f47",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-0.5B-Instruct",
+  "hard": true,
+  "seeds": [
+    1,
+    2,
+    3
+  ],
+  "directed": {
+    "n_candidates": 16,
+    "concentration": 0.5,
+    "n_val": 60
+  }
+}

results/llm_directed_hard_seeds/resolved_config.yaml

@@ -0,0 +1,29 @@
+experiment: llm_directed_hard_seeds
+seed: 1
+n_replicates: 1
+source_config:
+  experiment: llm_directed_hard_seeds
+  kind: llm_directed
+  seed: 1
+  seeds:
+  - 1
+  - 2
+  - 3
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-0.5B-Instruct
+  hard: true
+  families:
+  - lists
+  - strings
+  - arith
+  n_train: 400
+  n_test: 80
+  n_val: 60
+  n_candidates: 16
+  concentration: 0.5
+  epochs: 3
+  lora:
+    r: 16
+    alpha: 32
+  output:
+    dir: results/llm_directed_hard_seeds

results/llm_merge_seeds/README.md

@@ -0,0 +1,35 @@
+# Multi-seed LLM recombination (0.5B) — the claims with error bars
+
+PNAS work-order Phase 3: removes the "one seed" objection on the three LLM recombination claims.
+Protocol: **test sets fixed** (seed 1000+i per family), **training seed varied** (specialists cache
+per-seed as `spec_<family>[_hard]_s<seed>`), so across-seed variance is training variance only.
+Figure: `llm_seeds.png` (this dir) aggregates all three experiments, 95% CI over seeds.
+
+### (A) Fisher–Muller, easy benchmark, 5 seeds (`llm_merge_seeds`)
+| model | overall | worst-family |
+|---|---|---|
+| merge_ties | **0.647 ± 0.027** | **0.282 ± 0.020** |
+| merge_soup | 0.632 ± 0.042 | 0.278 ± 0.028 |
+| best specialist (strings) | 0.592 ± 0.009 | 0.078 ± 0.011 |
+| base | 0.277 | 0.150 |
+
+Both merges beat every specialist overall (ties: non-overlapping CIs; soup: marginal at 0.5B, as in
+the single-seed run — decisive at 7B) and the **worst-family signature is unambiguous**: merges ≈0.28
+vs ≤0.16 for any parent — only recombined models are competent everywhere.
+
+### (B) Union vs fusion, hard benchmark, 3 seeds (`llm_moe_hard_seeds/`)
+Routing (union) 0.274 ± 0.026 overall / 0.238 ± 0.024 worst-family; fusion soup 0.174 ± 0.102 / 0.088
+± 0.093; ties similar; best specialist 0.199 ± 0.026. Union beats fusion on both metrics — **and a new
+finding: fusion is seed-FRAGILE on hard tasks (CI ±0.10) while routing is seed-stable (±0.026).**
+Averaging's outcome depends on which specialist minima the seeds happened to find; selection-based
+recombination is reliable. (Learned router still = oracle: lexically distinct families, known rider.)
+
+### (C) Directed offspring selection, hard, 3 seeds (`llm_directed_hard_seeds/`)
+directed_overall 0.221 ± 0.026 (> soup 0.174 ± 0.102 and > best specialist); directed_balanced
+worst-family 0.158 ± 0.036 (> soup 0.088 ± 0.093). Directed selection both beats and **stabilises**
+the a-priori soup; per-input routing (B) remains above any single global blend, as before.
+
+**Read together:** all three recombination claims hold under seed replication, and the operator
+ordering (route > directed-select > soup, on headroom tasks) is not only a mean effect but a
+*variance* effect — the union/selection operators are the reliable ones. Base: Qwen2.5-0.5B-Instruct;
+statistical (per-seed) reproducibility per blueprint §4.

results/llm_merge_seeds/manifest.json

@@ -0,0 +1,28 @@
+{
+  "experiment": "llm_merge_seeds",
+  "master_seed": 1,
+  "git_commit": "d6a5c5cacde5544a39305eef34925a4fb35b8a19",
+  "python": "3.14.7",
+  "libraries": {
+    "numpy": "2.5.0",
+    "scipy": "1.18.0",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.13.0",
+    "peft": "0.19.1"
+  },
+  "rows": 150,
+  "results_sha256": "810c0b27c02f40fe0aa1847b3ffb3efb2fe46631842cf411843be0bff82da2a5",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-0.5B-Instruct",
+  "hard": false,
+  "seeds": [
+    1,
+    2,
+    3,
+    4,
+    5
+  ]
+}

results/llm_merge_seeds/resolved_config.yaml

@@ -0,0 +1,30 @@
+experiment: llm_merge_seeds
+seed: 1
+n_replicates: 1
+source_config:
+  experiment: llm_merge_seeds
+  kind: llm_merge
+  seed: 1
+  seeds:
+  - 1
+  - 2
+  - 3
+  - 4
+  - 5
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-0.5B-Instruct
+  families:
+  - lists
+  - strings
+  - arith
+  n_train: 600
+  n_test: 100
+  epochs: 3
+  lora:
+    r: 16
+    alpha: 32
+  merges:
+  - soup
+  - ties
+  output:
+    dir: results/llm_merge_seeds

results/llm_moe_hard_seeds/README.md

@@ -0,0 +1,7 @@
+# Multi-seed union-vs-fusion, hard benchmark (0.5B, 3 seeds)
+
+Part of the multi-seed firm-up; full legend, table, and the fusion-fragility finding in
+`results/llm_merge_seeds/README.md` (panel B of its `llm_seeds.png`). Headline: union/routing
+0.274 ± 0.026 > fusion 0.174 ± 0.102 overall (worst-family 0.238 vs 0.088), and fusion's ±0.10 CI vs
+routing's ±0.026 is itself the finding — averaging is seed-fragile where headroom exists; routing is
+reliable.

results/llm_moe_hard_seeds/manifest.json

@@ -0,0 +1,32 @@
+{
+  "experiment": "llm_moe_hard_seeds",
+  "master_seed": 1,
+  "git_commit": "d6a5c5cacde5544a39305eef34925a4fb35b8a19",
+  "python": "3.14.7",
+  "libraries": {
+    "numpy": "2.5.0",
+    "scipy": "1.18.0",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.13.0",
+    "peft": "0.19.1"
+  },
+  "rows": 126,
+  "results_sha256": "3d30a97ecb2e05a99cb188f527e353b737f3935b2e4a9b8c01fba1edec197b9b",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-0.5B-Instruct",
+  "hard": true,
+  "seeds": [
+    1,
+    2,
+    3
+  ],
+  "operators": [
+    "soup",
+    "ties",
+    "moe_oracle",
+    "moe_learned"
+  ]
+}

results/llm_moe_hard_seeds/resolved_config.yaml

@@ -0,0 +1,32 @@
+experiment: llm_moe_hard_seeds
+seed: 1
+n_replicates: 1
+source_config:
+  experiment: llm_moe_hard_seeds
+  kind: llm_moe
+  seed: 1
+  seeds:
+  - 1
+  - 2
+  - 3
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-0.5B-Instruct
+  hard: true
+  families:
+  - lists
+  - strings
+  - arith
+  n_train: 400
+  n_test: 80
+  n_route: 32
+  epochs: 3
+  lora:
+    r: 16
+    alpha: 32
+  operators:
+  - soup
+  - ties
+  - moe_oracle
+  - moe_learned
+  output:
+    dir: results/llm_moe_hard_seeds

results/llm_speciation/README.md

@@ -0,0 +1,47 @@
+# LLM-tier model speciation — conflict provokes isolation; duration alone does not
+
+E13 carried into language-model weights (0.5B Qwen, LoRA children of one frozen base — which shares
+its coordinate system with both children, so **there is no permutation/rescaling ambiguity by
+construction: every merge failure here is functional**). Two knobs, pre-registered readings in the
+configs; figure `llm_speciation.png` (3 panels; panel B from `results/llm_speciation_add/`).
+
+**Design.** Child A: private family `strings`; child B: private family `arith`; shared **ambiguous
+convention prompts** ("Sort the list […]" — no direction stated) answered *ascending* by A and
+*descending* by B: each convention harmless alone, contradictory jointly (the Bateson–Dobzhansky–
+Muller structure). 50/50 soup merge; exact-match verifier; fixed test sets. **Coherence** of a model =
+max(accuracy under asc grading, under desc grading) on the shared prompts — a coherent parent scores
+under its own convention; a hybrid mixing conventions scores low under both (the `μ(S)/2` floor made
+operational).
+
+### Finding 1 — function-specific hybrid breakdown (the conflict knob; panel A)
+Once the conventions are trained (`conflict_frac ≥ 0.25`), each parent performs under its own
+convention (~0.20–0.25 — low absolute values: sorting long lists is hard for 0.5B) while the merge's
+coherence sits at **0.02–0.08, below BOTH parents under either grading** — the hybrid loses precisely
+the conflicted function. (At `frac = 0` no one has seen the ambiguous prompts; that point is a
+no-conflict baseline, not part of the cliff.)
+
+### Finding 2 — the damage does not spread (the de-confounded `add` design; panel B)
+In the original ("replace") sweep, higher conflict fraction mechanically means *less private-family
+training*, so the merge's private-family decline is confounded. The **`add` design**
+(`llm_speciation_add`, 3 seeds) holds each child's private training fixed and adds conflict data on
+top: the merge's private-family accuracy then shows **no trend with conflict** (0.74–0.88, tracking
+parent A's 0.82–0.87 within seed noise at every level). Conflict damage is **localised to the
+conflicted function**; it does not corrupt the disjoint skills — at this scale, hybrid breakdown is
+surgical, not global. Honest rider: 0.5B soup merges carry large *intrinsic* seed variance even at
+zero conflict (sd up to 0.28) — the same averaging-fragility seen in `llm_moe_hard_seeds`.
+
+### Finding 3 — the duration null: over-specialisation does not erode mergeability (panel C)
+Pure disjoint specialists over-trained from 1 to 12 epochs (no shared data at all): the merged model
+*improves* (0.84 → 0.94 mean-private) and stays **above the best parent at every duration**. The MLP
+tier's "no emergent isolation" null (`speciation_real_emergent`) **generalises to LLM weights** in
+this regime — relevant to the report that averaging prefers under-trained experts (arXiv:2607.11997):
+in our disjoint-family setting, no such over-training penalty appears; the theory's prediction is that
+their effect should trace to *conflicting conventions on shared circuitry*, which the
+`epistasis_predicts` experiment (work order) will test directly.
+
+**Speciation across all three tiers now reads:** analytic (E12: cliff, epistasis-dependence,
+snowball) → MLP (E13: functional residual survives the full symmetry group; no emergent isolation) →
+LLM (this run: function-specific hybrid breakdown under conflict; no isolation from duration or
+specialisation alone). Isolation must be provoked by functional conflict at every tier tested.
+Falsifiers (not triggered): merge coherence matching the parents (no breakdown), or merged
+private-family accuracy declining with conflict in the `add` design (global corruption).

results/llm_speciation/manifest.json

@@ -0,0 +1,21 @@
+{
+  "experiment": "llm_speciation",
+  "master_seed": 1,
+  "git_commit": "d6a5c5cacde5544a39305eef34925a4fb35b8a19",
+  "python": "3.14.7",
+  "libraries": {
+    "numpy": "2.5.0",
+    "scipy": "1.18.0",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.13.0",
+    "peft": "0.19.1"
+  },
+  "rows": 162,
+  "results_sha256": "5490874f6d7157db38d395baa1a8174315a32f67d53679c22f8621c454ead0ea",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-0.5B-Instruct",
+  "hard": false
+}

results/llm_speciation/resolved_config.yaml

@@ -0,0 +1,30 @@
+experiment: llm_speciation
+seed: 1
+n_replicates: 1
+source_config:
+  experiment: llm_speciation
+  kind: llm_speciation
+  seed: 1
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-0.5B-Instruct
+  family_a: strings
+  family_b: arith
+  n_train: 400
+  n_test: 100
+  epochs: 3
+  lora:
+    r: 16
+    alpha: 32
+  conflict_fracs:
+  - 0.0
+  - 0.25
+  - 0.5
+  - 0.75
+  - 1.0
+  durations:
+  - 1
+  - 3
+  - 6
+  - 12
+  output:
+    dir: results/llm_speciation

results/llm_speciation_add/README.md

@@ -0,0 +1,9 @@
+# LLM speciation, de-confounded conflict sweep ("add" design, 3 seeds)
+
+Companion to `results/llm_speciation/` (full legend there; this run feeds panel B of its figure).
+Private training held fixed at n_train while conflict data is ADDED on top, so any decline in the
+merge's private-family accuracy would be interference, not a data-budget artefact. Result: no trend —
+the merge tracks parent A within seed noise at every conflict level (pre-registered reading #1:
+conflict damage is localised to the conflicted function). Run at 3 seeds because the single-seed pilot
+contained one anomalous grid point (a bad parent-B training run); the seeded curve is the reportable
+one.

results/llm_speciation_add/manifest.json

@@ -0,0 +1,26 @@
+{
+  "experiment": "llm_speciation_add",
+  "master_seed": 1,
+  "git_commit": "58e6c74609ce12142a5f1ae542c80c2be8713937",
+  "python": "3.14.7",
+  "libraries": {
+    "numpy": "2.5.0",
+    "scipy": "1.18.0",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.13.0",
+    "peft": "0.19.1"
+  },
+  "rows": 270,
+  "results_sha256": "d08d0979eb4715127268d6a524c1be57d123eeb6c39f8293dd68ef4894139265",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-0.5B-Instruct",
+  "hard": false,
+  "seeds": [
+    1,
+    2,
+    3
+  ]
+}

results/llm_speciation_add/resolved_config.yaml

@@ -0,0 +1,31 @@
+experiment: llm_speciation_add
+seed: 1
+n_replicates: 1
+source_config:
+  experiment: llm_speciation_add
+  kind: llm_speciation
+  seed: 1
+  seeds:
+  - 1
+  - 2
+  - 3
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-0.5B-Instruct
+  family_a: strings
+  family_b: arith
+  n_train: 400
+  n_test: 100
+  epochs: 3
+  lora:
+    r: 16
+    alpha: 32
+  conflict_mode: add
+  conflict_fracs:
+  - 0.0
+  - 0.25
+  - 0.5
+  - 0.75
+  - 1.0
+  durations: []
+  output:
+    dir: results/llm_speciation_add