Phase 3: LLM-tier speciation + multi-seed firm-up of the recombination claims

llm_speciation (new kind; src/llm/speciation.py): E13 in LLM weights.
LoRA children share the frozen base's coordinates, so merge failure is
functional by construction. CONFLICT (ambiguous sort prompts learned
under opposite conventions — the BDM structure): function-specific
hybrid breakdown — merged coherence 0.02-0.08 falls below BOTH parents
(~0.2) on the conflicted function; and in the de-confounded `add` design
(private budget fixed, conflict added on top; 3 seeds after a
single-seed pilot showed one anomalous point) the merge's private-family
accuracy shows NO trend with conflict — the damage is surgical, not
global. DURATION (over-trained disjoint specialists, 1->12 epochs): the
merge improves (0.84->0.94) and stays above the best parent — the MLP
"no emergent isolation" null generalises; relevant to the
expert-training-duration report (2607.11997), with the epistasis
prediction left to the decisive experiment.

Multi-seed firm-up (seeds threaded into specialist caches; `seeds:` list
support in the runner; fixed test sets): all three recombination claims
hold with CIs — merges beat every specialist (5 seeds, ties
0.647±0.027 > best spec 0.592±0.009; worst-family 0.28 vs <=0.16); union
0.274±0.026 > fusion 0.174±0.102 on hard (3 seeds); directed 0.221±0.026
> soup. NEW finding: fusion is seed-FRAGILE where headroom exists
(CI ±0.10) while routing/directed selection are stable (±0.026) — the
union/selection operators win on reliability, not just mean.

Figures (llm_speciation 3-panel; llm_seeds 3-panel with 95% CI), READMEs,
+1 convention test (150 green), make llm-speciation / llm-seeds targets.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v

figures/plot_llm_seeds.py

@@ -0,0 +1,88 @@
+"""Multi-seed LLM robustness figure — the recombination claims with error bars.
+
+Aggregates the three multi-seed 0.5B experiments (fixed test sets, training seed varied) into one
+figure with 95% CIs over seeds:
+
+(A) llm_merge_seeds — Fisher–Muller: merged specialists vs the best single specialist, overall and
+    worst-family (5 seeds).
+(B) llm_moe_hard_seeds — union (routing) vs fusion (soup/ties) on the hard benchmark (3 seeds).
+(C) llm_directed_hard_seeds — directed offspring selection vs the a-priori soup, hard (3 seeds).
+
+Usage:  python figures/plot_llm_seeds.py
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+import numpy as np
+import matplotlib.pyplot as plt
+
+sys.path.insert(0, str(Path(__file__).parent))
+from _figlib import load_bundle, savefig  # noqa: E402
+
+
+def _agg(df, models, metric):
+    """Per-model mean and 95% CI over seeds for one metric."""
+    out = []
+    for m in models:
+        v = df[(df["model"] == m) & (df["metric"] == metric)].groupby("seed")["accuracy"].mean()
+        out.append((v.mean(), 1.96 * v.std(ddof=1) / max(1, np.sqrt(len(v)))))
+    return out
+
+
+def _panel(ax, df, models, labels, title):
+    x = np.arange(len(models))
+    for off, metric, color in ((-0.17, "overall", "#2c7fb8"), (0.17, "worst_family", "#d62728")):
+        vals = _agg(df, models, metric)
+        ax.bar(x + off, [v for v, _ in vals], 0.34, yerr=[e for _, e in vals],
+               capsize=3, color=color, label=metric)
+    ax.set_xticks(x); ax.set_xticklabels(labels, fontsize=8)
+    ax.set(ylabel="verifier accuracy", ylim=(0, 1.0), title=title)
+    ax.legend(frameon=False, fontsize=8)
+
+
+def _best_spec(df):
+    """Synthesise a best-single-specialist row set per seed (max over spec_* by overall)."""
+    specs = sorted(m for m in df["model"].unique() if m.startswith("spec_"))
+    rows = []
+    for s, sub in df.groupby("seed"):
+        ov = {m: sub[(sub["model"] == m) & (sub["metric"] == "overall")]["accuracy"].mean()
+              for m in specs}
+        best = max(ov, key=ov.get)
+        b = sub[sub["model"] == best].copy()
+        b["model"] = "best_specialist"
+        rows.append(b)
+    import pandas as pd
+    return pd.concat([df] + rows, ignore_index=True)
+
+
+def main() -> None:
+    fig, axes = plt.subplots(1, 3, figsize=(16, 4.8))
+
+    df, _ = load_bundle("results/llm_merge_seeds")
+    _panel(axes[0], _best_spec(df), ["base", "best_specialist", "merge_soup", "merge_ties"],
+           ["base", "best\nspecialist", "merge\n(soup)", "merge\n(ties)"],
+           "(A) Fisher–Muller with error bars\n(5 seeds, easy benchmark, 0.5B)")
+
+    df, _ = load_bundle("results/llm_moe_hard_seeds")
+    _panel(axes[1], _best_spec(df), ["best_specialist", "merge_soup", "merge_ties", "moe_oracle",
+                                     "moe_learned"],
+           ["best\nspecialist", "fusion\n(soup)", "fusion\n(ties)", "union\n(route,oracle)",
+            "union\n(route,learned)"],
+           "(B) union vs fusion, hard benchmark\n(3 seeds, 0.5B)")
+
+    df, _ = load_bundle("results/llm_directed_hard_seeds")
+    _panel(axes[2], df, ["merge_soup", "directed_overall", "directed_balanced"],
+           ["a-priori soup", "directed\n(overall)", "directed\n(balanced)"],
+           "(C) directed offspring selection, hard\n(3 seeds, 0.5B)")
+
+    fig.suptitle("The LLM recombination claims are seed-robust (fixed test sets; training seed varied; 95% CI)",
+                 y=1.03, fontsize=12)
+    fig.tight_layout()
+    savefig(fig, "results/llm_merge_seeds", "llm_seeds")
+
+
+if __name__ == "__main__":
+    main()

figures/plot_llm_speciation.py

@@ -0,0 +1,96 @@
+"""LLM-tier model speciation figure — conflict coherence cliff, de-confounded interference, duration null.
+
+(A) The conflict cliff, read where it is clean: on the shared ambiguous prompts, each parent performs
+under its own convention while the 50/50 merge scores below BOTH under either grading — the hybrid
+loses precisely the conflicted function (the mu(S) floor made visible). From the "replace" design
+(results/llm_speciation).
+
+(B) The de-confounded private-family readout ("add" design, results/llm_speciation_add: private
+training held fixed, conflict data added on top): whether the merge's private-family competence
+tracks its parents (conflict damage localised to the conflicted function) or falls below them
+(interference spreading to shared circuitry).
+
+(C) The duration (emergent) null: over-trained disjoint specialists keep merging well — the merged
+model's private-family accuracy stays above the best parent at every duration. The MLP tier's
+"no emergent isolation" null generalises to LLM weights in this regime.
+
+Because LoRA deltas share the frozen base's coordinates, none of this involves alignment ambiguity:
+every failure shown is functional by construction.
+
+Usage:  python figures/plot_llm_speciation.py
+"""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+import matplotlib.pyplot as plt
+
+sys.path.insert(0, str(Path(__file__).parent))
+from _figlib import load_bundle, savefig  # noqa: E402
+
+
+def _series(df, mode, model, metric):
+    sub = df[(df["mode"] == mode) & (df["model"] == model) & (df["metric"] == metric)]
+    g = sub.groupby("x")["accuracy"].agg(["mean", "std"]).reset_index().fillna(0.0)
+    return g["x"], g["mean"], g["std"]
+
+
+def main() -> None:
+    rep, _ = load_bundle("results/llm_speciation")
+    add, _ = load_bundle("results/llm_speciation_add")
+    fam_a = "strings" if (rep["metric"] == "strings").any() else "lists"
+    fam_b = "arith"
+
+    fig, axes = plt.subplots(1, 3, figsize=(16.5, 4.9))
+
+    # (A) coherence on the conflicted function (replace design).
+    ax = axes[0]
+    x, y, _ = _series(rep, "conflict", "parent_a", "ambig_asc")
+    ax.plot(x, y, "--o", color="#9ecae1", lw=1.5, label="parent A under its convention (asc)")
+    x, y, _ = _series(rep, "conflict", "parent_b", "ambig_desc")
+    ax.plot(x, y, "--o", color="#a1d99b", lw=1.5, label="parent B under its convention (desc)")
+    x, y, _ = _series(rep, "conflict", "merge_soup", "coherence")
+    ax.plot(x, y, "-s", color="#d62728", lw=2.2, label="merge under its BEST convention")
+    ax.set(xlabel="fraction of training carrying the conflicting convention",
+           ylabel="accuracy on the shared ambiguous prompts", ylim=(-0.02, None),
+           title="(A) the hybrid loses the conflicted function\n(below BOTH parents under either grading)")
+    ax.legend(frameon=False, fontsize=8)
+
+    # (B) de-confounded private families (add design).
+    ax = axes[1]
+    mode = "conflict_add"
+    for model, color, style, lw in (("merge_soup", "#d62728", "-s", 2.2),
+                                    ("parent_a", "#9ecae1", "--o", 1.5),
+                                    ("parent_b", "#a1d99b", "--o", 1.5)):
+        x, y, s = _series(add, mode, model, "mean_private")
+        ax.plot(x, y, style, color=color, lw=lw, label=f"{model}: private families (mean)")
+        ax.fill_between(x, y - s, y + s, color=color, alpha=0.15)   # +-1 sd over seeds
+    ax.set(xlabel="conflict data added on top of fixed private training",
+           ylabel="verifier accuracy", ylim=(-0.02, 1.02),
+           title="(B) conflict damage does NOT spread: private families\ntrack the parents at every conflict level (3 seeds, ±1 sd)")
+    ax.legend(frameon=False, fontsize=8)
+
+    # (C) duration null.
+    ax = axes[2]
+    x, y, _ = _series(rep, "duration", "merge_soup", "mean_private")
+    ax.plot(x, y, "-o", color="#d62728", lw=2.2, label="merge: private families (mean)")
+    x, y, _ = _series(rep, "duration", "parent_a", fam_a)
+    ax.plot(x, y, "--o", color="#9ecae1", lw=1.5, label=f"parent A on its own family ({fam_a})")
+    x, y, _ = _series(rep, "duration", "parent_b", fam_b)
+    ax.plot(x, y, "--o", color="#a1d99b", lw=1.5, label=f"parent B on its own family ({fam_b})")
+    ax.set(xlabel="specialist training duration (epochs)", ylabel="verifier accuracy",
+           ylim=(-0.02, 1.02),
+           title="(C) the emergent test: over-specialisation\ndoes not erode mergeability here")
+    ax.legend(frameon=False, fontsize=8)
+
+    fig.suptitle("LLM-tier model speciation: conflict provokes function-specific hybrid breakdown; "
+                 "no isolation emerges from duration alone (LoRA shares base coordinates — failures are "
+                 "functional by construction)", y=1.03, fontsize=11.5)
+    fig.tight_layout()
+    savefig(fig, "results/llm_speciation", "llm_speciation")
+
+
+if __name__ == "__main__":
+    main()