Phase 3: LLM-tier speciation + multi-seed firm-up of the recombination claims
llm_speciation (new kind; src/llm/speciation.py): E13 in LLM weights. LoRA children share the frozen base's coordinates, so merge failure is functional by construction. CONFLICT (ambiguous sort prompts learned under opposite conventions — the BDM structure): function-specific hybrid breakdown — merged coherence 0.02-0.08 falls below BOTH parents (~0.2) on the conflicted function; and in the de-confounded `add` design (private budget fixed, conflict added on top; 3 seeds after a single-seed pilot showed one anomalous point) the merge's private-family accuracy shows NO trend with conflict — the damage is surgical, not global. DURATION (over-trained disjoint specialists, 1->12 epochs): the merge improves (0.84->0.94) and stays above the best parent — the MLP "no emergent isolation" null generalises; relevant to the expert-training-duration report (2607.11997), with the epistasis prediction left to the decisive experiment. Multi-seed firm-up (seeds threaded into specialist caches; `seeds:` list support in the runner; fixed test sets): all three recombination claims hold with CIs — merges beat every specialist (5 seeds, ties 0.647±0.027 > best spec 0.592±0.009; worst-family 0.28 vs <=0.16); union 0.274±0.026 > fusion 0.174±0.102 on hard (3 seeds); directed 0.221±0.026 > soup. NEW finding: fusion is seed-FRAGILE where headroom exists (CI ±0.10) while routing/directed selection are stable (±0.026) — the union/selection operators win on reliability, not just mean. Figures (llm_speciation 3-panel; llm_seeds 3-panel with 95% CI), READMEs, +1 convention test (150 green), make llm-speciation / llm-seeds targets. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
parent
58e6c74609
commit
5a23ddaf2a
31 changed files with 956 additions and 11 deletions
6
results/llm_directed_hard_seeds/README.md
Normal file
6
results/llm_directed_hard_seeds/README.md
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
# Multi-seed directed offspring selection, hard benchmark (0.5B, 3 seeds)
|
||||
|
||||
Part of the multi-seed firm-up; full legend and table in `results/llm_merge_seeds/README.md`
|
||||
(panel C of its `llm_seeds.png`). Headline: directed_overall 0.221 ± 0.026 beats the a-priori soup
|
||||
(0.174 ± 0.102) and the best specialist, and directed_balanced more than doubles the soup's
|
||||
worst-family (0.158 vs 0.088) — offspring selection both improves and stabilises the blend.
|
||||
31
results/llm_directed_hard_seeds/manifest.json
Normal file
31
results/llm_directed_hard_seeds/manifest.json
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
{
|
||||
"experiment": "llm_directed_hard_seeds",
|
||||
"master_seed": 1,
|
||||
"git_commit": "d6a5c5cacde5544a39305eef34925a4fb35b8a19",
|
||||
"python": "3.14.7",
|
||||
"libraries": {
|
||||
"numpy": "2.5.0",
|
||||
"scipy": "1.18.0",
|
||||
"pandas": "3.0.3",
|
||||
"pyarrow": "24.0.0",
|
||||
"torch": "2.12.1",
|
||||
"transformers": "5.13.0",
|
||||
"peft": "0.19.1"
|
||||
},
|
||||
"rows": 105,
|
||||
"results_sha256": "532f6115402fce2a610baf7879bb81653bc61e9f64092160d3dc5d9f9d516f47",
|
||||
"layer": "2",
|
||||
"tier": "llm",
|
||||
"base_model": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||
"hard": true,
|
||||
"seeds": [
|
||||
1,
|
||||
2,
|
||||
3
|
||||
],
|
||||
"directed": {
|
||||
"n_candidates": 16,
|
||||
"concentration": 0.5,
|
||||
"n_val": 60
|
||||
}
|
||||
}
|
||||
29
results/llm_directed_hard_seeds/resolved_config.yaml
Normal file
29
results/llm_directed_hard_seeds/resolved_config.yaml
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
experiment: llm_directed_hard_seeds
|
||||
seed: 1
|
||||
n_replicates: 1
|
||||
source_config:
|
||||
experiment: llm_directed_hard_seeds
|
||||
kind: llm_directed
|
||||
seed: 1
|
||||
seeds:
|
||||
- 1
|
||||
- 2
|
||||
- 3
|
||||
n_replicates: 1
|
||||
base_model: Qwen/Qwen2.5-0.5B-Instruct
|
||||
hard: true
|
||||
families:
|
||||
- lists
|
||||
- strings
|
||||
- arith
|
||||
n_train: 400
|
||||
n_test: 80
|
||||
n_val: 60
|
||||
n_candidates: 16
|
||||
concentration: 0.5
|
||||
epochs: 3
|
||||
lora:
|
||||
r: 16
|
||||
alpha: 32
|
||||
output:
|
||||
dir: results/llm_directed_hard_seeds
|
||||
Loading…
Add table
Add a link
Reference in a new issue