Phase 3: LLM-tier speciation + multi-seed firm-up of the recombination claims

llm_speciation (new kind; src/llm/speciation.py): E13 in LLM weights.
LoRA children share the frozen base's coordinates, so merge failure is
functional by construction. CONFLICT (ambiguous sort prompts learned
under opposite conventions — the BDM structure): function-specific
hybrid breakdown — merged coherence 0.02-0.08 falls below BOTH parents
(~0.2) on the conflicted function; and in the de-confounded `add` design
(private budget fixed, conflict added on top; 3 seeds after a
single-seed pilot showed one anomalous point) the merge's private-family
accuracy shows NO trend with conflict — the damage is surgical, not
global. DURATION (over-trained disjoint specialists, 1->12 epochs): the
merge improves (0.84->0.94) and stays above the best parent — the MLP
"no emergent isolation" null generalises; relevant to the
expert-training-duration report (2607.11997), with the epistasis
prediction left to the decisive experiment.

Multi-seed firm-up (seeds threaded into specialist caches; `seeds:` list
support in the runner; fixed test sets): all three recombination claims
hold with CIs — merges beat every specialist (5 seeds, ties
0.647±0.027 > best spec 0.592±0.009; worst-family 0.28 vs <=0.16); union
0.274±0.026 > fusion 0.174±0.102 on hard (3 seeds); directed 0.221±0.026
> soup. NEW finding: fusion is seed-FRAGILE where headroom exists
(CI ±0.10) while routing/directed selection are stable (±0.026) — the
union/selection operators win on reliability, not just mean.

Figures (llm_speciation 3-panel; llm_seeds 3-panel with 95% CI), READMEs,
+1 convention test (150 green), make llm-speciation / llm-seeds targets.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
Giorgio Gilestro 2026-09-06 15:39:15 +01:00
parent 58e6c74609
commit 5a23ddaf2a
31 changed files with 956 additions and 11 deletions

View file

@ -107,3 +107,22 @@ def test_merge_weights_requires_two_candidates():
import pytest
with pytest.raises(ValueError):
sample_merge_weights(3, 1, np.random.default_rng(0))
def test_convention_tasks_conflict_only_between_conventions():
# The BDM structure of llm_speciation: identical prompts, each convention internally consistent
# and verifiable, the two conventions contradictory on (almost) every prompt.
from llm.speciation import make_convention_tasks
from llm.tasks import verify
asc = make_convention_tasks(20, seed=5, convention="asc")
desc = make_convention_tasks(20, seed=5, convention="desc")
assert [a.prompt for a in asc] == [d.prompt for d in desc] # same inputs, graded two ways
assert all(verify(a.answer, a) for a in asc) # each convention self-consistent
assert all(verify(d.answer, d) for d in desc)
conflicting = sum(a.answer != d.answer for a, d in zip(asc, desc))
assert conflicting >= 18 # contradictory unless already sorted
assert all(not verify(a.answer, d) for a, d in zip(asc, desc) if a.answer != d.answer)
# deterministic: prompts and answers are a pure function of (seed, convention)
again = make_convention_tasks(20, seed=5, convention="asc")
assert [t.answer for t in again] == [t.answer for t in asc]