epistasis_predicts: functional conflict, not weight geometry, predicts merge failure pre-merge

The decisive experiment from the external review. 39 LoRA parent pairs
(0.5B, 3 seeds) on three axes decorrelated by construction: conflict
(contradictory conventions on shared prompts, private budgets fixed),
compat (same prompts, SAME convention — overlap without conflict), and
duration (weight divergence, zero conflict). Six pre-merge predictors;
primary outcome = merge penalty (parent potential − merged achieved).

League table (Spearman vs penalty, n=39): functional measures predict
(dis_raw +0.460, epi_conf +0.446, p<0.005); geometry collapses
(delta_cos +0.03, delta_l2 +0.17 n.s.); gradient alignment weak (−0.35);
performance ~0. The first grid's apparent geometry win (+0.60) was an
overlap/volume artifact — the compat control axis (added for exactly
this) exposed and killed it: same overlap and data volume, zero penalty.
Honest riders in the README: confidence weighting does not beat raw
disagreement as a rank predictor (pre-registered internal prediction not
confirmed; it does double the conflict/compat level contrast), and
|rho|~0.45 is bounded by 0.5B merge-outcome noise (7B is the firm-up).

Also: micro-batched gradient accumulation (OOM fix on the shared 16GB
GPU), exact r-space LoRA-delta geometry (brute-force-verified test,
151 green), systemd-run runbook lesson (tmux dies with the SSH session
scope on this box).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v

results/llm_epistasis/README.md

@@ -0,0 +1,52 @@
+# The decisive experiment — predicting merge failure BEFORE merging
+
+The external review's bar (2026-08-11): population-genetic quantities must *predict*, not
+re-describe — forecast merge success **pre-merge** and beat existing predictors. Design: 39 parent
+pairs (0.5B LoRA children of one frozen base, 3 seeds) on **three axes decorrelated by
+construction** — `conflict` (contradictory conventions on shared ambiguous prompts, private budgets
+fixed), `compat` (the control: **same shared prompts, same convention** — task overlap *without*
+conflict; added after the first grid exposed a confound, see below), and `duration` (weight
+divergence with zero conflict, 1→12 epochs). Primary outcome (pre-registered):
+**merge penalty** = parent potential − merged achieved (the hybrid-load analogue). Figure:
+`llm_epistasis.png`.
+
+### The league table (Spearman ρ vs merge penalty, full three-axis pool, n = 39)
+| pre-merge predictor | ρ | p | reading |
+|---|---|---|---|
+| raw functional disagreement (`dis_raw`) | **+0.460** | 0.003 | predicts |
+| operational epistasis (`epi_conf`, confidence-weighted) | **+0.446** | 0.004 | predicts |
+| gradient alignment at the base (cf. 2601.22285) | −0.347 | 0.03 | weakly informative |
+| LoRA-delta L2 distance (geometry) | +0.165 | 0.32 | uninformative |
+| LoRA-delta cosine (geometry) | +0.030 | 0.86 | uninformative |
+| cross-family accuracy (performance) | −0.005 | 0.98 | uninformative |
+
+**Headline: functional conflict, measured before merging, predicts merge failure; weight geometry
+does not.** The duration axis spans the same weight-divergence range as the conflict axis
+(L2 ≈ 2.4–4.0) at ~zero penalty, and the compat axis adds the same *data volumes* and overlap at
+~zero penalty — so both geometric predictors collapse once overlap and volume are controlled.
+
+### The control that did the work (`llm_epistasis_compat/`)
+In the first grid (conflict + duration only), `delta_cos` scored ρ = +0.60 — apparently the best
+predictor. That was an **artifact**: every shared-data pair in that pool was a conflicted pair, so
+geometry could win as a mere task-overlap/volume detector. The `compat` axis (overlap without
+conflict) exposes it: penalty ≈ 0.005 there, and the geometry correlations collapse (+0.60 → +0.03).
+The functional measures behave correctly on the control — parents trained on the same convention
+*agree* on the shared prompts (epi_conf: 0.46 conflict vs 0.23 compat, a 2× contrast; raw
+disagreement 0.72 vs 0.47, only 1.5× — the confidence weighting removes complementation noise from
+the *level*, giving the cleaner axis separation).
+
+### Honest riders (pre-registered falsifier status)
+1. The internal prediction that confidence weighting would beat raw disagreement **as a rank
+   predictor is not confirmed**: `epi_conf` and `dis_raw` are statistically indistinguishable at
+   n = 39 (the weighting does improve the conflict-vs-compat *contrast* in levels). The paper reports
+   the functional-vs-geometric verdict, not a win for the refinement.
+2. Correlations are moderate (|ρ| ≈ 0.45), bounded by 0.5B merge-outcome noise (soup merges carry
+   large intrinsic seed variance — see `llm_moe_hard_seeds`); read as signs and ordering, not
+   magnitudes. 7B replication is the natural firm-up.
+3. Gradient alignment carries real signal (it differentiates conflicting conventions at the base) but
+   less than the functional measures in this design.
+
+**Bottom line for the paper:** the framework's claim — *epistasis (functional conflict), not
+divergence, sets merge compatibility* — survives its designed falsification test at this tier: the
+operational conflict measures predict, the divergence measures do not, and the case was made honest
+by a control that first *broke our own experiment's* favourite-looking geometric predictor.

results/llm_epistasis/manifest.json

@@ -0,0 +1,26 @@
+{
+  "experiment": "llm_epistasis",
+  "master_seed": 1,
+  "git_commit": "5a23ddaf2a906a14d9aeb2797cf8fef821a519f4",
+  "python": "3.14.7",
+  "libraries": {
+    "numpy": "2.5.0",
+    "scipy": "1.18.0",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.13.0",
+    "peft": "0.19.1"
+  },
+  "rows": 27,
+  "results_sha256": "2895adb3347c4550fa1bc052a32e86aa17f4891f1163a80ea118618a911e4bd5",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-0.5B-Instruct",
+  "hard": false,
+  "seeds": [
+    1,
+    2,
+    3
+  ]
+}

results/llm_epistasis/resolved_config.yaml

@@ -0,0 +1,36 @@
+experiment: llm_epistasis
+seed: 1
+n_replicates: 1
+source_config:
+  experiment: llm_epistasis
+  kind: llm_epistasis
+  seed: 1
+  seeds:
+  - 1
+  - 2
+  - 3
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-0.5B-Instruct
+  family_a: strings
+  family_b: arith
+  n_train: 400
+  n_test: 80
+  epochs: 3
+  n_probe_each: 30
+  grad_k: 32
+  lora:
+    r: 16
+    alpha: 32
+  conflict_fracs:
+  - 0.0
+  - 0.25
+  - 0.5
+  - 0.75
+  - 1.0
+  durations:
+  - 1
+  - 3
+  - 6
+  - 12
+  output:
+    dir: results/llm_epistasis