Giorgio Gilestro
5a23ddaf2a
llm_speciation (new kind; src/llm/speciation.py): E13 in LLM weights. LoRA children share the frozen base's coordinates, so merge failure is functional by construction. CONFLICT (ambiguous sort prompts learned under opposite conventions — the BDM structure): function-specific hybrid breakdown — merged coherence 0.02-0.08 falls below BOTH parents (~0.2) on the conflicted function; and in the de-confounded `add` design (private budget fixed, conflict added on top; 3 seeds after a single-seed pilot showed one anomalous point) the merge's private-family accuracy shows NO trend with conflict — the damage is surgical, not global. DURATION (over-trained disjoint specialists, 1->12 epochs): the merge improves (0.84->0.94) and stays above the best parent — the MLP "no emergent isolation" null generalises; relevant to the expert-training-duration report (2607.11997), with the epistasis prediction left to the decisive experiment. Multi-seed firm-up (seeds threaded into specialist caches; `seeds:` list support in the runner; fixed test sets): all three recombination claims hold with CIs — merges beat every specialist (5 seeds, ties 0.647±0.027 > best spec 0.592±0.009; worst-family 0.28 vs <=0.16); union 0.274±0.026 > fusion 0.174±0.102 on hard (3 seeds); directed 0.221±0.026 > soup. NEW finding: fusion is seed-FRAGILE where headroom exists (CI ±0.10) while routing/directed selection are stable (±0.026) — the union/selection operators win on reliability, not just mean. Figures (llm_speciation 3-panel; llm_seeds 3-panel with 95% CI), READMEs, +1 convention test (150 green), make llm-speciation / llm-seeds targets. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
32 lines
1.7 KiB
YAML
32 lines
1.7 KiB
YAML
experiment: llm_speciation
|
|
kind: llm_speciation
|
|
seed: 1
|
|
n_replicates: 1
|
|
|
|
# LLM-tier model speciation (E13 in language-model weights; PNAS work order Phase 3). Two LoRA
|
|
# children from the same frozen base — so there is NO permutation/rescaling ambiguity by construction:
|
|
# any merge failure is functional incompatibility, isolated architecturally. Two sweeps:
|
|
# conflict_fracs — the IMPOSED cliff: each child has a private disjoint family (A: strings,
|
|
# B: arith) plus a shared set of AMBIGUOUS sort prompts ("Sort the list [...]", no direction)
|
|
# answered ascending by A and descending by B (each convention harmless alone, contradictory
|
|
# jointly — the BDM structure). Prediction: the MERGED model's private-family competence degrades
|
|
# and its convention coherence (max of asc/desc grading) collapses as conflict grows, while each
|
|
# PARENT stays fine — hybrid breakdown in verifier units, echoing the MLP cliff.
|
|
# durations — the EMERGENT null: pure disjoint specialists over-trained (epochs swept), zero shared
|
|
# data. Arbitrates the MLP tier's null (no emergent isolation; the merge rescued specialists at
|
|
# every divergence) against the empirical report that averaging prefers under-trained experts
|
|
# (arXiv:2607.11997). Pre-registered readings: merged quality falls with duration while parents'
|
|
# own-family quality holds -> emergent incompatibility at the LLM tier; otherwise the null
|
|
# generalises. Either outcome is reportable; do not tune toward one.
|
|
|
|
base_model: Qwen/Qwen2.5-0.5B-Instruct
|
|
family_a: strings
|
|
family_b: arith
|
|
n_train: 400
|
|
n_test: 100
|
|
epochs: 3
|
|
lora: {r: 16, alpha: 32}
|
|
conflict_fracs: [0.0, 0.25, 0.5, 0.75, 1.0]
|
|
durations: [1, 3, 6, 12]
|
|
|
|
output: {dir: results/llm_speciation}
|