second review round: tempered claims, robust statistics, corrected technical statements

Analyses (figures/stats_llm_epistasis.py, committed + reproducible):
condition-clustered bootstrap CIs (functional measures exclude zero:
dis_raw [+0.04,+0.69], conf-weighted [+0.02,+0.68]; gradient alignment
[-0.59,-0.06]; geometry straddles zero), PAIRED predictor contrasts (not
individually significant — stated), leave-one-condition-out held-out
prediction (functional replicates, geometry ~0, performance baseline
unstable), three outcome references (ordering sensitive to reference —
reported, with the mechanism), between/within-axis decomposition
(within-conflict identification impossible by design; the compat axis
identifies), and seed-level paired reliability (routing/directed beat
soup 3/3 seeds incl. one catastrophic soup failure; CI-width fragility
claim withdrawn).

Renames and corrections: "decisive experiment" -> "controlled predictive
test"; "operational epistasis" -> "confidence-weighted functional
conflict (proposed proxy)"; "functional by construction" -> "controls a
major source of coordinate mismatch / conflict-associated" (module,
configs, READMEs, figures); SI proposition's "chord" defined precisely
(endpoint-loss interpolation, invariant) vs the path (not invariant) +
no-global-optimality caveat (removable = lower bound, residual = upper);
snowball count != performance cliff distinction added; claims table
gains four rows (grid finding / weighting NOT supported / functional-vs-
all-geometry not established / operator choice open); §1 ladder states
the prediction rung as a bounded small-model result.

paper/response-to-review-2.md: point-by-point, opening with the
bookkeeping correction (E13b/c were in the reviewed draft — revised
interpretation, not new results). READMEs rewritten around the four
analyses with the chronology (prospective/adaptive/post-hoc) disclosed.
151 tests green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v

configs/llm/speciation.yaml

@@ -4,8 +4,8 @@ seed: 1
 n_replicates: 1
 
 # LLM-tier model speciation (E13 in language-model weights; PNAS work order Phase 3). Two LoRA
-# children from the same frozen base — so there is NO permutation/rescaling ambiguity by construction:
-# any merge failure is functional incompatibility, isolated architecturally. Two sweeps:
+# children from the same frozen base — which controls a major source of coordinate mismatch (no
+# alignment step needed), allowing a cleaner test of conflict-associated merge failure. Two sweeps:
 #   conflict_fracs — the IMPOSED cliff: each child has a private disjoint family (A: strings,
 #     B: arith) plus a shared set of AMBIGUOUS sort prompts ("Sort the list [...]", no direction)
 #     answered ascending by A and descending by B (each convention harmless alone, contradictory

figures/plot_llm_epistasis.py

@@ -1,8 +1,9 @@
-"""The decisive-experiment figure — does pre-merge epistasis predict merge failure?
+"""The controlled predictive test — does a pre-merge functional-conflict measure predict merge damage?
 
-(A) The theory's predictor: operational epistasis (confidence-weighted bilateral disagreement,
-measured before merging) against the merge penalty (parent potential − merged achieved, the
-hybrid-load analogue). Conflict-axis pairs in red, duration-axis pairs in blue.
+(A) The framework-motivated predictor: confidence-weighted functional conflict (bilateral confident
+disagreement, measured before merging — a proposed proxy for merge-relevant interactions, not a
+measured epistasis) against the merge penalty (oracle parent potential − merged achieved; ordering is
+sensitive to the outcome reference — see stats_llm_epistasis.py).
 
 (B) The geometry predictor on the same outcome: weight divergence (LoRA-delta L2) — the
 matched-divergence contrast: the duration axis spans large weight divergence at ~zero penalty, while
@@ -27,7 +28,7 @@ from scipy.stats import spearmanr
 sys.path.insert(0, str(Path(__file__).parent))
 from _figlib import load_bundle, savefig  # noqa: E402
 
-PREDICTORS = [("epi_conf", "operational\nepistasis"),
+PREDICTORS = [("epi_conf", "conf-weighted\nfunctional conflict"),
               ("dis_raw", "raw\ndisagreement"),
               ("grad_cos", "gradient\nalignment"),
               ("delta_cos", "delta\ncosine"),
@@ -59,8 +60,8 @@ def main() -> None:
         pass
 
     fig, axes = plt.subplots(1, 3, figsize=(16.5, 4.9))
-    _scatter(axes[0], df, "epi_conf", "operational epistasis (pre-merge)",
-             "(A) the theory's predictor")
+    _scatter(axes[0], df, "epi_conf", "confidence-weighted functional conflict (pre-merge)",
+             "(A) the framework-motivated predictor")
     _scatter(axes[1], df, "delta_cos", "LoRA-delta cosine similarity (pre-merge)",
              "(B) the geometry predictor — does it detect\nincompatibility, or just task overlap?")
 
@@ -77,8 +78,9 @@ def main() -> None:
     ax.set(ylabel="|Spearman ρ| vs merge penalty", ylim=(0, 1),
            title="(C) predictor league table (pre-merge only)")
 
-    fig.suptitle("Predicting merge failure BEFORE merging: functional conflict, not weight divergence "
-                 "(conflict, overlap-without-conflict, and divergence decorrelated by construction; 3 seeds)", y=1.03, fontsize=12)
+    fig.suptitle("A controlled predictive test: across this task grid, pre-merge functional disagreement "
+                 "predicted merge penalties; the selected weight-geometry baselines did not "
+                 "(three axes decorrelated by construction; 13 conditions x 3 seeds)", y=1.03, fontsize=11.5)
     fig.tight_layout()
     savefig(fig, "results/llm_epistasis", "llm_epistasis")
 

figures/plot_llm_speciation.py

@@ -14,8 +14,9 @@ tracks its parents (conflict damage localised to the conflicted function) or fal
 model's private-family accuracy stays above the best parent at every duration. The MLP tier's
 "no emergent isolation" null generalises to LLM weights in this regime.
 
-Because LoRA deltas share the frozen base's coordinates, none of this involves alignment ambiguity:
-every failure shown is functional by construction.
+The shared frozen base controls a major source of coordinate mismatch (LoRA deltas share its
+coordinates), allowing a cleaner test of conflict-associated merging failure — though averaging can
+still fail for non-conflict reasons (nonlinear interaction, scaling, capacity).
 
 Usage:  python figures/plot_llm_speciation.py
 """
@@ -86,8 +87,8 @@ def main() -> None:
     ax.legend(frameon=False, fontsize=8)
 
     fig.suptitle("LLM-tier model speciation: conflict provokes function-specific hybrid breakdown; "
-                 "no isolation emerges from duration alone (LoRA shares base coordinates — failures are "
-                 "functional by construction)", y=1.03, fontsize=11.5)
+                 "no isolation emerges from duration alone (shared base controls coordinate mismatch — "
+                 "a cleaner test of conflict-associated failure)", y=1.03, fontsize=11.5)
     fig.tight_layout()
     savefig(fig, "results/llm_speciation", "llm_speciation")
 

figures/stats_llm_epistasis.py

@@ -0,0 +1,108 @@
+"""Robust statistics for the controlled predictive test (source of the README numbers).
+
+Implements the second external review's four requested analyses (2026-08-11), from committed
+artifacts only:
+
+1. condition-clustered bootstrap CIs for each predictor's Spearman rho, and PAIRED bootstrap
+   differences between predictors (a significant rho for one and not another is not a significant
+   difference — the paired contrast is the honest comparison);
+2. sample-structure disclosure (13 conditions x 3 seeds = 39 rows; parents are retrained per
+   condition x seed but share task-data seeds across conditions within a seed, so rows are not
+   independent — hence clustering by condition);
+3. between- vs within-axis decomposition (pooled correlations are partly axis discrimination);
+4. the outcome under three references: oracle parent potential (pre-registered primary),
+   best parent, and mean parent — reported because the predictor ordering is sensitive to it.
+
+Plus leave-one-condition-out (LOCO) held-out prediction per predictor.
+
+Usage:  python figures/stats_llm_epistasis.py
+"""
+
+from __future__ import annotations
+
+import numpy as np
+import pandas as pd
+from scipy.stats import spearmanr
+
+PREDICTORS = ["epi_conf", "dis_raw", "grad_cos", "delta_cos", "delta_l2", "cross_perf"]
+
+
+def load() -> pd.DataFrame:
+    a = pd.read_parquet("results/llm_epistasis/results.parquet")
+    b = pd.read_parquet("results/llm_epistasis_compat/results.parquet")
+    df = pd.concat([a, b], ignore_index=True)
+    df["cond"] = df["mode"] + "_" + df["x"].astype(str)
+    df["parent_a_overall"] = df[["pa_fam_a", "pa_fam_b", "pa_coh"]].mean(axis=1)
+    df["parent_b_overall"] = df[["pb_fam_a", "pb_fam_b", "pb_coh"]].mean(axis=1)
+    df["pen_oracle"] = df["merge_penalty"]                       # pre-registered primary
+    df["pen_best"] = df[["parent_a_overall", "parent_b_overall"]].max(axis=1) - df["merged_overall"]
+    df["pen_mean"] = df[["parent_a_overall", "parent_b_overall"]].mean(axis=1) - df["merged_overall"]
+    return df
+
+
+def clustered_bootstrap(df: pd.DataFrame, outcome: str = "pen_oracle", B: int = 4000, seed: int = 0):
+    """Percentile CIs for each predictor's rho, resampling CONDITIONS (13 clusters) with replacement."""
+    rng = np.random.default_rng(seed)
+    conds = df["cond"].unique()
+    groups = {c: df[df["cond"] == c] for c in conds}
+    boot = {p: np.empty(B) for p in PREDICTORS}
+    for i in range(B):
+        bs = pd.concat([groups[c] for c in rng.choice(conds, size=len(conds), replace=True)],
+                       ignore_index=True)
+        for p in PREDICTORS:
+            boot[p][i] = spearmanr(bs[p], bs[outcome])[0]
+    return boot
+
+
+def loco(df: pd.DataFrame, outcome: str = "pen_oracle"):
+    """Leave-one-condition-out held-out prediction (linear fit per predictor)."""
+    out = {}
+    for p in PREDICTORS:
+        pr, ac = [], []
+        for c in df["cond"].unique():
+            tr, te = df[df["cond"] != c], df[df["cond"] == c]
+            coef = np.polyfit(tr[p], tr[outcome], 1)
+            pr += list(np.polyval(coef, te[p])); ac += list(te[outcome])
+        rho, pv = spearmanr(pr, ac)
+        out[p] = (rho, pv, float(np.sqrt(np.mean((np.array(pr) - np.array(ac)) ** 2))))
+    return out
+
+
+def main() -> None:
+    df = load()
+    print(f"sample: {df['cond'].nunique()} conditions x {df['seed'].nunique()} seeds = {len(df)} rows")
+
+    print("\n== league table under three outcome references (Spearman rho) ==")
+    print(f"{'predictor':>11} {'oracle*':>8} {'best':>8} {'mean':>8}   (*pre-registered primary)")
+    for p in PREDICTORS:
+        r = [spearmanr(df[p], df[o])[0] for o in ["pen_oracle", "pen_best", "pen_mean"]]
+        print(f"{p:>11} {r[0]:+8.3f} {r[1]:+8.3f} {r[2]:+8.3f}")
+
+    boot = clustered_bootstrap(df)
+    print("\n== condition-clustered bootstrap 95% CIs (primary outcome) ==")
+    for p in PREDICTORS:
+        v = boot[p][~np.isnan(boot[p])]
+        print(f"{p:>11}: {spearmanr(df[p], df['pen_oracle'])[0]:+.3f}"
+              f"  [{np.percentile(v, 2.5):+.3f}, {np.percentile(v, 97.5):+.3f}]")
+
+    print("\n== paired bootstrap |rho| differences ==")
+    for a_, b_ in [("epi_conf", "dis_raw"), ("dis_raw", "delta_cos"),
+                   ("dis_raw", "grad_cos"), ("epi_conf", "delta_cos")]:
+        d = np.abs(boot[a_]) - np.abs(boot[b_]); d = d[~np.isnan(d)]
+        print(f"|rho({a_})| - |rho({b_})|: {np.mean(d):+.3f}"
+              f"  [{np.percentile(d, 2.5):+.3f}, {np.percentile(d, 97.5):+.3f}]")
+
+    print("\n== leave-one-condition-out held-out prediction ==")
+    for p, (rho, pv, rmse) in loco(df).items():
+        print(f"{p:>11}: LOCO rho={rho:+.3f} (p={pv:.3g})  rmse={rmse:.3f}")
+
+    print("\n== between- vs within-axis ==")
+    print("mean penalty by axis:", df.groupby("mode")["pen_oracle"].mean().round(3).to_dict())
+    c_df = df[df["mode"] == "conflict"]
+    for p in PREDICTORS:
+        r, pv = spearmanr(c_df[p], c_df["pen_oracle"])
+        print(f"{p:>11} (conflict axis only, n={len(c_df)}): {r:+.3f} (p={pv:.2g})")
+
+
+if __name__ == "__main__":
+    main()

paper/response-to-review-2.md

@@ -0,0 +1,146 @@
+# Response to the second review
+
+*This response accompanies a further revision. Every number below is reproduced by a committed script
+(`figures/stats_llm_epistasis.py`) from committed artifacts; the revised documents are
+`results/llm_epistasis/README.md` (rewritten around your four analyses), the manuscript, and
+`paper/si-notes.md`.*
+
+---
+
+## 0. A correction first (your §7)
+
+You are right, and we apologise for the bookkeeping error: the draft you reviewed **already
+contained** the full-symmetry alignment experiment, the conflict residual, and the
+compatible-specialisation null. Our previous letter's "new since the review" conflated three things
+that we now state separately: **new results** (the LLM-tier speciation runs, the multi-seed
+replication, and the controlled predictive test with its control axis), **new controls and analyses**
+(the compatible-overlap axis; the robust statistics in this letter), and **revised interpretation**
+(everything about E13b/c, which was experimental content you had already seen and whose *presentation*
+we changed). The experimental delta attributable to the review process is the first and second
+categories only.
+
+## 1. Your two-conclusion distinction — adopted as the evidential boundary
+
+We accept the boundary exactly as you drew it:
+
+- **Demonstrated:** a small-model, controlled predictive test in which pre-merge functional
+  disagreement predicted merge penalties where the selected weight-space measures did not.
+- **Not demonstrated:** that an epistasis-specific predictor adds value beyond ordinary functional
+  disagreement, or that the prediction improves operator choice.
+
+The experiment is now titled a **"controlled predictive test"** everywhere ("decisive experiment
+delivered" is gone), and the manuscript's §1 ladder describes the prediction rung in your conditional
+formulation, with its four boundary clauses stated in place: constructed grid, small scale, refinement
+not superior, operator choice open. The measure itself is renamed **"confidence-weighted functional
+conflict — a proposed proxy for merge-relevant interactions"**; we accept that bilateral confident
+contradiction measures incompatible endpoint behaviour, not non-additive interaction in the strict
+biological sense, and the paper no longer calls any measured quantity "epistasis." Your distinction —
+*the framework motivated the measurement and controls* vs *their success validates the specifically
+population-genetic mechanism* — is adopted verbatim; we claim the former.
+
+## 2. The four analyses — run
+
+**(1) Direct predictor comparison.** Condition-clustered bootstrap (13 clusters, B = 4000), 95% CIs
+for each predictor's ρ against the pre-registered primary outcome:
+
+| predictor | ρ | clustered 95% CI |
+|---|---|---|
+| raw functional disagreement | +0.460 | [+0.04, +0.69] |
+| confidence-weighted functional conflict | +0.446 | [+0.02, +0.68] |
+| gradient alignment | −0.347 | [−0.59, −0.06] |
+| delta L2 | +0.165 | [−0.27, +0.58] |
+| delta cosine | +0.030 | [−0.46, +0.51] |
+| cross-family accuracy | −0.005 | [−0.29, +0.31] |
+
+**Paired contrasts are not individually significant** (e.g. |ρ(dis_raw)| − |ρ(delta_cos)| = +0.23,
+CI [−0.23, +0.59]). Held-out prediction (leave-one-condition-out linear fits): functional measures
+replicate (dis_raw ρ = +0.396, p = 0.013; conf-weighted +0.352, p = 0.028); geometry ≈ 0; the
+performance baseline is unstable out-of-sample (−0.435). So the supported statement — now the
+conclusion in the README, the figure title, and the manuscript — is yours: *across this controlled
+grid, functional disagreement showed a detectable, held-out-robust association with merge penalty;
+LoRA-delta cosine and L2 showed no statistically detectable association; gradient alignment carried
+intermediate signal (its CI excludes zero), so this is not a clean functional-versus-all-geometric
+divide; head-to-head predictor differences are not individually significant; only these baselines were
+tested.* "Weight divergence does not predict merge failure" has been removed as over-broad.
+
+**(2) Sample structure.** 39 rows = **13 conditions × 3 seeds**. Parents are retrained per
+condition × seed, but children share task-data seeds across conditions within a seed (e.g. the
+duration-3 parent and the conflict-0 parent are trained on essentially the same data), so rows are not
+independent — hence clustering by condition in all uncertainty estimates, disclosed in the README.
+
+**(3) Between- vs within-axis.** You were right that the pooled correlation is substantially axis
+discrimination (mean penalty: conflict 0.061 vs compat 0.005 / duration 0.011). Within the conflict
+axis (n = 15): functional measures +0.59/+0.62 — but delta-L2 is +0.71 there, because *within that
+axis* conflict fraction, added-data volume, and delta growth are collinear: **within-axis
+identification is impossible by design**, and we now say so; the identification comes from the control
+axes, where the same volumes and L2 ranges occur at ~zero penalty. Scatterplots are coloured by axis
+in the figure. Predicting penalties for a held-out *conflict mechanism* (not just held-out conditions
+of the same mechanism) is listed as the next test — we agree it is the more valuable one.
+
+**(4) Outcome references.** Reported under all three: oracle parent potential (pre-registered
+primary), best parent, and mean parent. The ordering is **sensitive to the reference** — under the
+best-parent reference, delta-L2 correlates comparably to the functional measures (+0.48 vs +0.34).
+Our reading, stated in the README rather than hidden: that reference inherits parent-strength trends
+that track training volume, which geometry also tracks, coupling predictor and outcome through the
+reference rather than through merge damage. On structural coupling between the disagreement predictor
+and the oracle-potential outcome: probe and test sets are disjoint by construction, but we agree
+definitional coupling through parental complementarity cannot be fully excluded, which is one more
+reason all three references are now on the table.
+
+## 3. Hypothesis labels
+
+Your table is adopted essentially as written — the manuscript's claims-at-a-glance table now carries:
+pre-merge disagreement predicts penalty (*empirical within the controlled grid*, with the boundary
+clauses in the limits column); confidence weighting improves rank prediction (***not supported***);
+functional beats all geometry (*not established — selected baselines only*); operator choice (*open*);
+cliff, snowball, emergent DMIs (*hypotheses*, unchanged). We also added your snowball distinction
+where the snowball is discussed: super-linear growth in incompatibility *count* does not by itself
+entail a sharp *performance* cliff — that needs the count→effect-size→performance link, which the
+analytic model supplies under its assumptions and any neural test must establish separately.
+
+## 4. The two technical statements — corrected
+
+**"Endpoints and chord are invariant."** Accepted; the ambiguity was ours. The SI proposition now
+defines "chord" precisely as the α-linear interpolation **of the endpoint loss values** — the barrier
+baseline, a function of endpoints only, which *is* invariant — and states explicitly that the
+**weight-space interpolation path is generally not invariant** (that being precisely why alignment can
+lower a barrier). It also now carries your second point: exact recovery of a permuted-and-rescaled
+copy validates a special case and does not establish global optimality of the alignment for
+independently trained networks — so the "removable" share is a lower bound and the "residual" an upper
+bound, stated wherever the decomposition is used.
+
+**"Frozen base pins the coordinate system, so failure is functional by construction."** Accepted, and
+your replacement wording adopted verbatim across the module, configs, READMEs, and figure: *the shared
+frozen base controls a major source of coordinate mismatch, allowing a cleaner test of
+conflict-associated merging failure* — with the boundary stated (failures of delta-averaging can still
+reflect nonlinear interaction, scaling, or capacity). "Conflict-associated," not "functional by
+construction," throughout.
+
+## 5. Chronology and reliability
+
+**Chronology (now a section of the README).** Prospective: hypotheses, predictors, primary outcome and
+falsifiers were in the config before the first grid ran. Adaptive: the compatible-overlap control was
+added *after* geometry appeared to win, with its own pre-stated readings, run on the same seeds; no
+existing rows were re-run or altered. Post hoc: the clustered-bootstrap/LOCO/multi-reference analyses
+were added at your request after all data was collected. We agree this is transparent adaptive
+experimentation, not wholly prospective confirmation, and it is labelled as such.
+
+**Reliability.** The CI-width claim is withdrawn. The seed-level statement now reads: routing beat the
+soup **in every seed** (3/3 paired, both metrics), directed selection beat the soup 3/3, and one seed
+exhibited a catastrophic soup failure (0.071 overall, 0.000 worst-family) to which routing was immune
+(0.262/0.225); seed-level sds (0.090 vs 0.023) are reported as an observation, with the explicit note
+that three seeds do not support a variance estimate.
+
+## 6. On your bottom line
+
+We accept your formulation as the paper's claim for this line of work — it now closes the relevant
+results section nearly verbatim: *we separated overlap, divergence, and conflict experimentally; in
+this controlled setting, functional disagreement predicted merging damage when simple weight-distance
+measures did not; the proposed epistasis refinement and the emergent-speciation mechanism remain
+unconfirmed.* And we take the redirection about what would count next: not 0.5B → 7B alone, but
+**generalisation to unfamiliar conflict structures** (a held-out conflict mechanism, and real rather
+than constructed task pairs) and **a demonstrably better budget-matched merging decision**. Those two
+now head the open-problems list, above the scale replication.
+
+We would welcome a third pass if you have the appetite — particularly on whether the README's
+conditional conclusion and the chronology section read at the right strength.

paper/si-notes.md

@@ -12,10 +12,17 @@ condition, the cyclically-relabelled classes; `μ(S) ≈ conflict_frac` up to cl
 networks, positive per-unit rescalings — the full unit symmetry group of a plain ReLU MLP) satisfies
 `T(B)(x) = B(x)` for all `x` by construction.
 
-**Proposition 1 (endpoint invariance).** For every function-preserving `T`, the endpoint functions —
-and hence the endpoint losses/errors and the linear chord between them — are identical for the pair
-`(A, T(B))` and the pair `(A, B)`. Alignment can only re-coordinate the *interpolation path*, never
-the endpoints or the chord. *(Immediate from the definition of function-preserving.)*
+**Proposition 1 (endpoint invariance — with the term "chord" defined precisely).** Here "chord"
+means the α-linear interpolation **of the endpoint loss values**, `(1−α)·L(A) + α·L(B)` — the
+baseline in the barrier definition, a function of the endpoints only — NOT the weight-space
+interpolation path. For every function-preserving `T`, the endpoint functions, hence the endpoint
+losses and this chord, are identical for `(A, T(B))` and `(A, B)`. The **interpolation path itself is
+generally NOT invariant** — losses along `(1−α)·A + α·T(B)` change with `T`, which is precisely why
+alignment can lower a barrier. *(Immediate from the definition of function-preserving.)* Scope
+caveat: our aligner provably recovers a permuted-and-rescaled copy exactly — an important special
+case — but this does not establish global optimality of the alignment over the symmetry group for
+independently trained networks; the decomposition's "removable" share is therefore a lower bound, and
+the "residual" an upper bound, on their true values.
 
 **Proposition 2 (no merged model can serve both parents).** Let `h` be *any* single classifier (in
 particular, any interpolated/merged model, under any alignment). On every `x ∈ S`, `f_A(x) ≠ f_B(x)`,

paper/the-evolution-of-sex-for-ai.md

@@ -164,10 +164,15 @@ different things are easily conflated: **interpretation** (an existing result is
 in these terms — e.g., merged offspring beating their parents as Fisher–Muller), **explanation** (the
 transferred mechanism accounts for observations existing accounts leave open — e.g., which merge
 failures are coordinate artefacts and which are functional), and **prediction** (the framework
-forecasts an unmeasured outcome and improves a design decision — e.g., an epistasis measure taken
-*before* merging that beats geometry-based predictors of merge success). This paper is strongest on
-the first, makes concrete progress on the second, and states the third as its open, decisive test —
-proposed here with pre-registered falsifiers, not claimed as done. The organising shift we argue for
+forecasts an unmeasured outcome and improves a design decision). This paper is strongest on the
+first, makes concrete progress on the second, and reports a first, bounded step on the third: a
+**controlled predictive test** at small scale in which pre-merge *functional-disagreement* measures —
+chosen by the framework — showed a detectable, held-out-robust association with merge damage on a
+constructed task grid, while the selected weight-geometry baselines did not. We are precise about
+that result's boundary where it is reported: it is a small-model demonstration on a constructed grid;
+the proposed epistasis-specific refinement did not outperform plain disagreement; predictor
+differences are not individually significant head-to-head; and whether the prediction improves a
+budget-matched operator choice remains open. The organising shift we argue for
 is prior to any single mechanism: **treat multigenerational model populations as systems whose
 inheritance, diversity, and compatibility must be managed — not merely as collections of models to
 optimise.**
@@ -470,9 +475,15 @@ cannot satisfy two contradictory answer conventions — is information-theoretic
 genetics; what the genetic frame adds is *structure around it*: which divergences generate such
 conflicts, the prediction that epistasis rather than distance sets the cliff's position, and the
 snowball's super-linear onset — the latter two verified so far only in the analytic model, and
-therefore carried as **hypotheses at the neural tier, not results**. Second, our alignment removes the
-symmetries we enumerate for this architecture class; richer transformation families for other
-architectures could reapportion removable vs residual, though not below the conflict floor. Third,
+therefore carried as **hypotheses at the neural tier, not results**. (On the snowball, one more
+distinction: super-linear growth in the *number* of incompatibilities does not by itself entail a
+sharp *performance* cliff — that needs the link from incompatibility count through effect sizes to
+measured performance, which the analytic model supplies under its assumptions and any neural test
+must establish separately.) Second, our alignment removes the symmetries we enumerate for this
+architecture class, and exactly recovering a permuted-and-rescaled copy validates a special case
+rather than proving global optimality for independently trained networks — so the removable share is
+a lower bound and the residual an upper bound; richer transformation families for other architectures
+could reapportion the split, though not below the conflict floor. Third,
 "unmergeable" here means by aligned linear interpolation of weights — a barrier to that operator does
 not preclude every conceivable recombination method (routing, for one, sidesteps it by not blending).
 Emergent Dobzhansky–Muller incompatibilities in real weights remain the flagship *hypothesis* of this
@@ -770,7 +781,10 @@ falsifier, not yet established):
 | Outbreeding depression on rugged landscapes; operator design rule | Exact-model result; hypothesis at LLM scale | NK epistasis stands in for skill entanglement | E9–E10; directed selection rescues | Not yet mapped onto a real task-entanglement measure |
 | Optimal mate-pool breadth shrinks with ruggedness | Exact-model result; hypothesis for merging populations | Ring population, local selection | E14 | Phenomenon known to island-model evolutionary computation; our contribution is the mapping and the diversity/mean decomposition |
 | Merge failure decomposes into coordinate artefact + functional residual | Empirical (MLP tier; LLM tier in progress) | Alignment enumerates the architecture's unit symmetries | Full-symmetry residual ≈ 0 (compatible) vs ≈ naive (conflict); cliff in hybrid fitness | Scoped to aligned linear interpolation; conflict floor is information-theoretic, not genetic |
-| Epistasis (not divergence) sets the cliff; snowball onset | Exact-model result; **hypothesis** at the neural tier | BDM incompatibility structure | E12 | The decisive pre-merge prediction test is proposed, not run |
+| Epistasis (not divergence) sets the cliff; snowball onset | Exact-model result; **hypothesis** at the neural tier | BDM incompatibility structure | E12 | Snowball count ≠ performance cliff without the effect-size link; neural test outstanding |
+| Pre-merge functional disagreement predicts merge penalty | Empirical, within a controlled grid (0.5B, 13 conditions × 3 seeds) | Constructed conflict/overlap/duration axes; oracle-potential outcome (pre-registered; ordering sensitive to reference) | Clustered CIs exclude 0; held-out LOCO ρ≈0.4; selected geometry baselines ≈ 0 | Head-to-head predictor differences not individually significant; only selected baselines; generalisation to real task pairs open |
+| Confidence weighting improves rank prediction over raw disagreement | **Not supported** (pre-registered internal prediction) | — | Paired Δ\|ρ\| ≈ −0.02, CI [−0.13, +0.06] | Weighting does double the conflict-vs-compat level contrast |
+| The predictor improves budget-matched operator choice | **Open** | — | Soup-vs-route gap readout noise-dominated at 0.5B | The practical payoff; untested |
 | Emergent speciation without conflict | **Not observed** (pre-registered) | Shared ancestry, compatible tasks, tested divergences | E13b: residual 0.000; merge rescues specialists | Bounds the hypothesis; longer horizons/distribution shift/capacity pressure untested |
 | Grounding + sex + diversity jointly necessary | Exact-model result; hypothesis at LLM scale | Conformity stands in for self-consumption | E11 four-arm ablation, each arm failing distinctly | The full grounded LLM society is unbuilt |
 

results/llm_epistasis/README.md

@@ -1,52 +1,86 @@
-# The decisive experiment — predicting merge failure BEFORE merging
+# A controlled predictive test — pre-merge functional conflict and merge penalty
 
-The external review's bar (2026-08-11): population-genetic quantities must *predict*, not
-re-describe — forecast merge success **pre-merge** and beat existing predictors. Design: 39 parent
-pairs (0.5B LoRA children of one frozen base, 3 seeds) on **three axes decorrelated by
-construction** — `conflict` (contradictory conventions on shared ambiguous prompts, private budgets
-fixed), `compat` (the control: **same shared prompts, same convention** — task overlap *without*
-conflict; added after the first grid exposed a confound, see below), and `duration` (weight
-divergence with zero conflict, 1→12 epochs). Primary outcome (pre-registered):
-**merge penalty** = parent potential − merged achieved (the hybrid-load analogue). Figure:
+*(Renamed from "the decisive experiment" after the second external review: this is a **small-model
+controlled predictive test**, not the decisive experiment delivered. The operator-choice component is
+unfinished, the epistasis-specific refinement did not outperform plain disagreement, and
+generalisation beyond this constructed task grid is open.)*
+
+**Question.** Can merge failure be predicted **before merging**, and by what kind of measure? Design:
+39 rows = **13 conditions × 3 seeds** (0.5B LoRA children of one frozen base; parents are retrained
+per condition × seed but share task-data seeds across conditions within a seed, so rows are *not*
+independent — all uncertainty below is condition-clustered). Three axes decorrelated by construction:
+`conflict` (contradictory conventions on shared ambiguous prompts, private budgets fixed), `compat`
+(same shared prompts, same convention — overlap and volume without conflict), `duration` (weight
+divergence with zero conflict). Primary outcome (pre-registered): **merge penalty vs oracle parent
+potential** (per-component best of the parents); also reported vs best-parent and mean-parent
+references. All robust statistics reproduce via `figures/stats_llm_epistasis.py`; figure:
 `llm_epistasis.png`.
 
-### The league table (Spearman ρ vs merge penalty, full three-axis pool, n = 39)
-| pre-merge predictor | ρ | p | reading |
+### The predictors (measured pre-merge)
+- **Confidence-weighted functional conflict** (`epi_conf`) — *a proposed proxy for merge-relevant
+  interactions* (previously "operational epistasis"; renamed because bilateral confident
+  contradiction measures incompatible endpoint behaviour, not non-additive interaction effects in the
+  strict biological sense).
+- **Raw functional disagreement** (`dis_raw`) — the unweighted rate.
+- **Gradient alignment** at the shared base (the ML-literature predictor, cf. 2601.22285);
+  **LoRA-delta cosine / L2** (weight geometry, exact in r-space); **cross-family accuracy**
+  (performance baseline).
+
+### The supported conclusion (stated conditionally)
+> **Across this controlled task grid, pre-merge functional disagreement predicted merge penalties,
+> whereas LoRA-delta cosine and L2 showed no statistically detectable association.** Gradient
+> alignment carried intermediate signal, so the result is not a clean functional-versus-all-geometric
+> divide, and only these selected baselines were tested.
+
+| predictor | ρ (primary) | clustered 95% CI | held-out (LOCO) ρ |
 |---|---|---|---|
-| raw functional disagreement (`dis_raw`) | **+0.460** | 0.003 | predicts |
-| operational epistasis (`epi_conf`, confidence-weighted) | **+0.446** | 0.004 | predicts |
-| gradient alignment at the base (cf. 2601.22285) | −0.347 | 0.03 | weakly informative |
-| LoRA-delta L2 distance (geometry) | +0.165 | 0.32 | uninformative |
-| LoRA-delta cosine (geometry) | +0.030 | 0.86 | uninformative |
-| cross-family accuracy (performance) | −0.005 | 0.98 | uninformative |
+| raw functional disagreement | +0.460 | [+0.04, +0.69] | +0.396 (p=0.013) |
+| confidence-weighted functional conflict | +0.446 | [+0.02, +0.68] | +0.352 (p=0.028) |
+| gradient alignment | −0.347 | [−0.59, −0.06] | +0.10 (n.s.) |
+| delta L2 (geometry) | +0.165 | [−0.27, +0.58] | −0.09 (n.s.) |
+| delta cosine (geometry) | +0.030 | [−0.46, +0.51] | −0.14 (n.s.) |
+| cross-family accuracy | −0.005 | [−0.29, +0.31] | −0.435 (unstable out-of-sample) |
 
-**Headline: functional conflict, measured before merging, predicts merge failure; weight geometry
-does not.** The duration axis spans the same weight-divergence range as the conflict axis
-(L2 ≈ 2.4–4.0) at ~zero penalty, and the compat axis adds the same *data volumes* and overlap at
-~zero penalty — so both geometric predictors collapse once overlap and volume are controlled.
+**Paired comparisons are not individually significant** at this sample size (e.g.
+|ρ(dis_raw)| − |ρ(delta_cos)| = +0.23, CI [−0.23, +0.59]): the honest statement is that the
+functional measures are *individually detectably informative* (CIs exclude zero; held-out
+replication) while the geometry measures are *not distinguishable from zero* — not that functional
+significantly beats geometry head-to-head.
 
-### The control that did the work (`llm_epistasis_compat/`)
-In the first grid (conflict + duration only), `delta_cos` scored ρ = +0.60 — apparently the best
-predictor. That was an **artifact**: every shared-data pair in that pool was a conflicted pair, so
-geometry could win as a mere task-overlap/volume detector. The `compat` axis (overlap without
-conflict) exposes it: penalty ≈ 0.005 there, and the geometry correlations collapse (+0.60 → +0.03).
-The functional measures behave correctly on the control — parents trained on the same convention
-*agree* on the shared prompts (epi_conf: 0.46 conflict vs 0.23 compat, a 2× contrast; raw
-disagreement 0.72 vs 0.47, only 1.5× — the confidence weighting removes complementation noise from
-the *level*, giving the cleaner axis separation).
+### What the decomposition shows (and its limits)
+- **Between vs within.** Much of the pooled correlation is axis discrimination (mean penalty:
+  conflict 0.061 vs compat 0.005 / duration 0.011). Within the conflict axis (n=15) the functional
+  measures still track (+0.59/+0.62) — but so does delta-L2 (+0.71), because *within that axis*
+  conflict fraction, added-data volume, and delta growth are collinear: **within-axis identification
+  is impossible by design; the identification comes from the control axes**, where the same volumes
+  and L2 ranges occur with ~zero penalty.
+- **Outcome-reference sensitivity.** Under the *best-parent* reference the ordering changes
+  (delta-L2 +0.48 vs functional +0.34): that reference inherits parent-strength trends that track
+  training volume — which geometry also tracks — coupling predictor and outcome through the
+  reference rather than through merge damage. We keep the pre-registered oracle-potential primary,
+  and report the sensitivity rather than hide it.
+- **The compat control did the decisive work** — and it *first broke our own experiment*: in the
+  initial two-axis grid the best predictor was delta-cosine (ρ=+0.60), an overlap/volume artifact
+  that the control exposed (collapse to +0.03).
 
-### Honest riders (pre-registered falsifier status)
-1. The internal prediction that confidence weighting would beat raw disagreement **as a rank
-   predictor is not confirmed**: `epi_conf` and `dis_raw` are statistically indistinguishable at
-   n = 39 (the weighting does improve the conflict-vs-compat *contrast* in levels). The paper reports
-   the functional-vs-geometric verdict, not a win for the refinement.
-2. Correlations are moderate (|ρ| ≈ 0.45), bounded by 0.5B merge-outcome noise (soup merges carry
-   large intrinsic seed variance — see `llm_moe_hard_seeds`); read as signs and ordering, not
-   magnitudes. 7B replication is the natural firm-up.
-3. Gradient alignment carries real signal (it differentiates conflicting conventions at the base) but
-   less than the functional measures in this design.
+### Chronology (adaptive, not wholly prospective)
+1. Hypotheses, predictors, primary outcome, and falsifiers written into `configs/llm/epistasis.yaml`
+   **before** the first grid ran (conflict + duration axes).
+2. The first grid's geometry result prompted the `compat` control axis
+   (`configs/llm/epistasis_compat.yaml`, pre-registered readings in its comments) — an **amendment
+   after seeing data**, run on the same seeds; no conflict/duration rows were re-run or altered.
+3. The clustered-bootstrap / LOCO / multi-reference analyses were added at the second review's
+   request, after all data was collected.
 
-**Bottom line for the paper:** the framework's claim — *epistasis (functional conflict), not
-divergence, sets merge compatibility* — survives its designed falsification test at this tier: the
-operational conflict measures predict, the divergence measures do not, and the case was made honest
-by a control that first *broke our own experiment's* favourite-looking geometric predictor.
+### Pre-registered internal prediction: NOT confirmed
+Confidence weighting was predicted to beat raw disagreement as a rank predictor. It does not (paired
+Δ|ρ| = −0.02, CI [−0.13, +0.06]). The weighting does double the conflict-vs-compat *level* contrast
+(2.0× vs 1.5×), but the present evidence favours **functional disagreement generally, not the
+DMI-specific refinement**. Accordingly: the population-genetic framework motivated the measurement
+and the controls; their success does **not** validate the specifically population-genetic mechanism.
+
+### Open (unchanged by this experiment)
+Budget-matched operator choice (the soup-vs-route gap readout is noise-dominated at 0.5B);
+generalisation to unfamiliar conflict structures and real task pairs; 7B replication; whether any
+measured quantity deserves the name *epistasis* (non-additive interaction of combinations) rather
+than *conflict*.

results/llm_merge_seeds/README.md

@@ -18,18 +18,22 @@ the single-seed run — decisive at 7B) and the **worst-family signature is unam
 vs ≤0.16 for any parent — only recombined models are competent everywhere.
 
 ### (B) Union vs fusion, hard benchmark, 3 seeds (`llm_moe_hard_seeds/`)
-Routing (union) 0.274 ± 0.026 overall / 0.238 ± 0.024 worst-family; fusion soup 0.174 ± 0.102 / 0.088
-± 0.093; ties similar; best specialist 0.199 ± 0.026. Union beats fusion on both metrics — **and a new
-finding: fusion is seed-FRAGILE on hard tasks (CI ±0.10) while routing is seed-stable (±0.026).**
-Averaging's outcome depends on which specialist minima the seeds happened to find; selection-based
-recombination is reliable. (Learned router still = oracle: lexically distinct families, known rider.)
+Routing (union) beats fusion **in every seed** (3/3 paired, both metrics; e.g. overall per seed:
+routing 0.258/0.300/0.262 vs soup 0.212/0.238/0.071). The paired per-seed values also show *why*:
+seed 3's soup failed catastrophically (0.071 overall, 0.000 worst-family) while routing was unaffected
+(0.262/0.225). Seed-level sd: soup 0.090 vs routing 0.023 — suggestive of a genuine variance
+difference, but with 3 seeds this is an observation, not an estimate; the robust statement is the
+3/3 paired ordering plus the observed catastrophic-failure mode that routing is structurally immune
+to. (Learned router still = oracle: lexically distinct families, known rider.)
 
 ### (C) Directed offspring selection, hard, 3 seeds (`llm_directed_hard_seeds/`)
-directed_overall 0.221 ± 0.026 (> soup 0.174 ± 0.102 and > best specialist); directed_balanced
-worst-family 0.158 ± 0.036 (> soup 0.088 ± 0.093). Directed selection both beats and **stabilises**
-the a-priori soup; per-input routing (B) remains above any single global blend, as before.
+directed_overall beats the a-priori soup **in every seed** (3/3 paired; 0.225/0.242/0.196 vs
+0.212/0.238/0.071 — including rescuing soup's catastrophic seed); directed_balanced worst-family
+0.158 vs soup 0.088. Directed selection both beats and stabilises the blend (same 3-seed caveat as
+panel B); per-input routing remains above any single global blend, as before.
 
-**Read together:** all three recombination claims hold under seed replication, and the operator
-ordering (route > directed-select > soup, on headroom tasks) is not only a mean effect but a
-*variance* effect — the union/selection operators are the reliable ones. Base: Qwen2.5-0.5B-Instruct;
+**Read together:** all three recombination claims hold under seed replication with consistent paired
+ordering (route > directed-select > soup in every seed on headroom tasks), and the per-seed values
+surface a failure mode — occasional catastrophic soup merges — that the union/selection operators
+avoided in every observed case. Variance *estimates* await more seeds. Base: Qwen2.5-0.5B-Instruct;
 statistical (per-seed) reproducibility per blueprint §4.

results/llm_moe_hard_seeds/README.md

@@ -1,7 +1,7 @@
 # Multi-seed union-vs-fusion, hard benchmark (0.5B, 3 seeds)
 
-Part of the multi-seed firm-up; full legend, table, and the fusion-fragility finding in
-`results/llm_merge_seeds/README.md` (panel B of its `llm_seeds.png`). Headline: union/routing
-0.274 ± 0.026 > fusion 0.174 ± 0.102 overall (worst-family 0.238 vs 0.088), and fusion's ±0.10 CI vs
-routing's ±0.026 is itself the finding — averaging is seed-fragile where headroom exists; routing is
-reliable.
+Part of the multi-seed firm-up; full legend and per-seed table in `results/llm_merge_seeds/README.md`
+(panel B of its `llm_seeds.png`). Headline: routing beats fusion in every seed (3/3 paired, both
+metrics), and one seed exhibited a catastrophic soup failure (0.071 overall / 0.000 worst-family) that
+routing was immune to (0.262/0.225). With 3 seeds the variance contrast (sd 0.090 vs 0.023) is an
+observation, not an estimate.

results/llm_speciation/README.md

@@ -1,8 +1,10 @@
 # LLM-tier model speciation — conflict provokes isolation; duration alone does not
 
-E13 carried into language-model weights (0.5B Qwen, LoRA children of one frozen base — which shares
-its coordinate system with both children, so **there is no permutation/rescaling ambiguity by
-construction: every merge failure here is functional**). Two knobs, pre-registered readings in the
+E13 carried into language-model weights (0.5B Qwen, LoRA children of one frozen base — the shared
+base **controls a major source of coordinate mismatch**, allowing a cleaner test of
+conflict-associated merge failure; averaging can still fail for non-conflict reasons — nonlinear
+interaction, scaling, capacity — so failures here are *conflict-associated*, not automatically
+functional). Two knobs, pre-registered readings in the
 configs; figure `llm_speciation.png` (3 panels; panel B from `results/llm_speciation_add/`).
 
 **Design.** Child A: private family `strings`; child B: private family `arith`; shared **ambiguous

src/llm/epistasis.py

@@ -1,4 +1,4 @@
-"""The decisive experiment: does an operational EPISTASIS measure predict merge success PRE-merge?
+"""A controlled predictive test: does a pre-merge FUNCTIONAL-CONFLICT measure predict merge damage?
 
 The external review's bar (2026-08-11): population-genetic quantities must *predict*, not re-describe —
 forecast merge success **before merging**, and beat existing predictors. This module builds a grid of
@@ -11,7 +11,9 @@ The theory-derived predictor — and its built-in ablation. A raw functional-dis
 two parents is, by our own theory, the WRONG measure: disjoint specialists disagree hugely (one knows,
 one is ignorant) yet merge perfectly — harmless *complementation*. The Dobzhansky–Muller analogue is
 **bilateral confident contradiction**: both parents confidently produce different answers to the same
-input. So the operational epistasis measure is confidence-weighted disagreement,
+input. So the framework-motivated measure is **confidence-weighted functional conflict** — a proposed proxy
+for merge-relevant interactions (bilateral confident contradiction measures incompatible endpoint
+behaviour, not non-additive interaction effects in the strict biological sense):
 
     epi_conf = E_probe[ conf_A · conf_B · 1(ans_A != ans_B) ],   conf = exp(mean token logprob),
 

src/llm/speciation.py

@@ -1,8 +1,11 @@
 """LLM-tier model speciation (E13 in language-model weights) — the conflict cliff + the duration null.
 
-The real-LLM image of E13, with a structural bonus: LoRA deltas live in the frozen base's coordinate
-system, so there is **no permutation/rescaling ambiguity by construction** — any merge failure here is
-*functional* incompatibility, the residual isolated architecturally (no alignment step needed).
+The real-LLM image of E13, with a structural advantage: LoRA deltas live in the frozen base's
+coordinate system, which controls a major source of coordinate mismatch (no alignment step needed) and
+so allows a cleaner test of conflict-associated merge failure. Note the honest boundary: a failure of
+delta-averaging can still reflect nonlinear interaction, scaling, or capacity — shared coordinates
+remove the permutation explanation, they do not make every failure evidence of incompatible functional
+requirements.
 
 Two knobs, mirroring the MLP experiment: