epistasis_predicts: functional conflict, not weight geometry, predicts merge failure pre-merge
The decisive experiment from the external review. 39 LoRA parent pairs (0.5B, 3 seeds) on three axes decorrelated by construction: conflict (contradictory conventions on shared prompts, private budgets fixed), compat (same prompts, SAME convention — overlap without conflict), and duration (weight divergence, zero conflict). Six pre-merge predictors; primary outcome = merge penalty (parent potential − merged achieved). League table (Spearman vs penalty, n=39): functional measures predict (dis_raw +0.460, epi_conf +0.446, p<0.005); geometry collapses (delta_cos +0.03, delta_l2 +0.17 n.s.); gradient alignment weak (−0.35); performance ~0. The first grid's apparent geometry win (+0.60) was an overlap/volume artifact — the compat control axis (added for exactly this) exposed and killed it: same overlap and data volume, zero penalty. Honest riders in the README: confidence weighting does not beat raw disagreement as a rank predictor (pre-registered internal prediction not confirmed; it does double the conflict/compat level contrast), and |rho|~0.45 is bounded by 0.5B merge-outcome noise (7B is the firm-up). Also: micro-batched gradient accumulation (OOM fix on the shared 16GB GPU), exact r-space LoRA-delta geometry (brute-force-verified test, 151 green), systemd-run runbook lesson (tmux dies with the SSH session scope on this box). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
parent
5a23ddaf2a
commit
287d2326cc
16 changed files with 622 additions and 3 deletions
8
results/llm_epistasis_compat/README.md
Normal file
8
results/llm_epistasis_compat/README.md
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
# The overlap-without-conflict control axis
|
||||
|
||||
Companion to `results/llm_epistasis/` (full analysis and league table there). Both children train on
|
||||
the SAME ambiguous prompts with the SAME convention (asc/asc) — task overlap and added-data volume
|
||||
identical to the conflict axis, with zero conflict. Result: merge penalty ≈ 0.005, geometry
|
||||
predictors' apparent power collapses (delta_cos ρ +0.60 → +0.03 once these pairs enter the pool),
|
||||
functional measures correctly stay low. The control that separated "detects incompatibility" from
|
||||
"detects overlap".
|
||||
26
results/llm_epistasis_compat/manifest.json
Normal file
26
results/llm_epistasis_compat/manifest.json
Normal file
|
|
@ -0,0 +1,26 @@
|
|||
{
|
||||
"experiment": "llm_epistasis_compat",
|
||||
"master_seed": 1,
|
||||
"git_commit": "5a23ddaf2a906a14d9aeb2797cf8fef821a519f4",
|
||||
"python": "3.14.7",
|
||||
"libraries": {
|
||||
"numpy": "2.5.0",
|
||||
"scipy": "1.18.0",
|
||||
"pandas": "3.0.3",
|
||||
"pyarrow": "24.0.0",
|
||||
"torch": "2.12.1",
|
||||
"transformers": "5.13.0",
|
||||
"peft": "0.19.1"
|
||||
},
|
||||
"rows": 12,
|
||||
"results_sha256": "e2091aebfdfdeaa1d44c3ca8b9a60d2dfd13bfe546d811f19ce8eb1ccfc14e7b",
|
||||
"layer": "2",
|
||||
"tier": "llm",
|
||||
"base_model": "Qwen/Qwen2.5-0.5B-Instruct",
|
||||
"hard": false,
|
||||
"seeds": [
|
||||
1,
|
||||
2,
|
||||
3
|
||||
]
|
||||
}
|
||||
30
results/llm_epistasis_compat/resolved_config.yaml
Normal file
30
results/llm_epistasis_compat/resolved_config.yaml
Normal file
|
|
@ -0,0 +1,30 @@
|
|||
experiment: llm_epistasis_compat
|
||||
seed: 1
|
||||
n_replicates: 1
|
||||
source_config:
|
||||
experiment: llm_epistasis_compat
|
||||
kind: llm_epistasis
|
||||
seed: 1
|
||||
seeds:
|
||||
- 1
|
||||
- 2
|
||||
- 3
|
||||
n_replicates: 1
|
||||
base_model: Qwen/Qwen2.5-0.5B-Instruct
|
||||
family_a: strings
|
||||
family_b: arith
|
||||
n_train: 400
|
||||
n_test: 80
|
||||
epochs: 3
|
||||
n_probe_each: 30
|
||||
grad_k: 32
|
||||
lora:
|
||||
r: 16
|
||||
alpha: 32
|
||||
compat_fracs:
|
||||
- 0.25
|
||||
- 0.5
|
||||
- 0.75
|
||||
- 1.0
|
||||
output:
|
||||
dir: results/llm_epistasis_compat
|
||||
Loading…
Add table
Add a link
Reference in a new issue