MachineSex/paper/si-notes.md

# SI notes — drafts of formal statements for the PNAS manuscript

*Working drafts; folded into the SI Appendix at Phase 4. Each statement is written to be exactly as
strong as what is true — no more.*

## S1. The incompatibility floor: what no alignment can remove (E13c)

**Setting.** Models A and B are trained on the same input distribution; their target label functions
`f_A` and `f_B` agree except on a conflict set `S` of probability mass `μ(S)` (in E13's conflict
condition, the cyclically-relabelled classes; `μ(S) ≈ conflict_frac` up to class balance). A
*function-preserving transformation* `T` (any composition of hidden-unit permutations and, for ReLU
networks, positive per-unit rescalings — the full unit symmetry group of a plain ReLU MLP) satisfies
`T(B)(x) = B(x)` for all `x` by construction.

**Proposition 1 (endpoint invariance — with the term "chord" defined precisely).** Here "chord"
means the α-linear interpolation **of the endpoint loss values**, `(1−α)·L(A) + α·L(B)` — the
baseline in the barrier definition, a function of the endpoints only — NOT the weight-space
interpolation path. For every function-preserving `T`, the endpoint functions, hence the endpoint
losses and this chord, are identical for `(A, T(B))` and `(A, B)`. The **interpolation path itself is
generally NOT invariant** — losses along `(1−α)·A + α·T(B)` change with `T`, which is precisely why
alignment can lower a barrier. *(Immediate from the definition of function-preserving.)* Scope
caveat: our aligner provably recovers a permuted-and-rescaled copy exactly — an important special
case — but this does not establish global optimality of the alignment over the symmetry group for
independently trained networks; the decomposition's "removable" share is therefore a lower bound, and
the "residual" an upper bound, on their true values.

**Proposition 2 (no merged model can serve both parents).** Let `h` be *any* single classifier (in
particular, any interpolated/merged model, under any alignment). On every `x ∈ S`, `f_A(x) ≠ f_B(x)`,
so `h(x)` disagrees with at least one of them. Hence

  `ε_A(h) + ε_B(h) ≥ μ(S)`,  and therefore  `max(ε_A(h), ε_B(h)) ≥ μ(S)/2`,

where `ε_P(h)` is `h`'s error against parent `P`'s labels. A hybrid of two models whose conventions
conflict on mass `μ(S)` errs at rate at least `μ(S)/2` against at least one parent — **hybrid
disadvantage with an information-theoretic floor, independent of the alignment group, the
architecture, and the merging operator.** This is reproductive isolation in the fitness sense: past a
given functional conflict, *no* recombination operator produces an offspring loyal to both lineages.

**What remains empirical, and why the experiment is designed as it is.** Propositions 1–2 do *not*
bound the single-task path barrier (the loss along the interpolation between A and `T(B)` evaluated
on one parent's task): in principle a path could dip toward one parent's function. Whether it does is
exactly what E13 measures — and the measured answer is that it does not: the conflict-condition
barrier is unchanged by permutation alignment (`residual`) *and* by alignment modulo the full
permutation × positive-rescaling group (`residual_scale`), while the same aligner removes ~all of the
independent-init barrier (the positive control). Richer-symmetry results for transformers
(arXiv:2606.23607; neuron-identifiability approaches to linear mode connectivity, 2026) strengthen
the *removable* side of the decomposition and are therefore complementary: the more barrier a larger
group can remove for *compatible* models, the sharper the meaning of the residual that survives for
*incompatible* ones — and Proposition 2 caps what any of them could ever achieve on the conflict set.

**Terminology note for the paper.** "Residual (after alignment)" = the estimated functional
incompatibility; for ReLU MLPs we align modulo the full unit symmetry group, so the estimate is not
confounded by missed symmetries of that architecture class.

## S2. Emergent vs imposed incompatibility (E13b framing)

The conflict condition *imposes* contradiction (the two label maps disagree on `S`), which pins
`μ(S) > 0` and activates Proposition 2. A true Bateson–Dobzhansky–Muller incompatibility is
*emergent*: each lineage's substitutions are harmless on their own background (`μ(S) = 0` — the
training signals never contradict), and incompatibility, if any, arises only in the *combination*.
The `disjoint` (complementary class specialists) and `augment` (divergent input conventions)
conditions realise this: any residual barrier they develop cannot be attributed to label conflict and
is the emergent-speciation signal proper. Pre-registered readings: residual grows with divergence →
model speciation is emergent in real weights (E12's trajectory realised); residual stays at the
`shared`-control level → within this regime, trained networks are *more* merge-compatible than the
biological analogy predicts — an honest bound on the analogy, and itself a design-relevant result
(merging is safe absent functional conflict).

**Outcome (2026-08-11 run, 4 reps, t_div ≤ 3200): the second reading.** Residual 0.000 at every
divergence in both emergent conditions, and the merge *rescues* the forgetting `disjoint` specialists
(parents → 0.535/0.474 on the full task; merged ≈ 0.955 throughout — a sustained Fisher–Muller rescue
at zero barrier). Isolation in real weights required functional conflict in this regime; whether
long-horizon over-specialisation erodes mergeability at LLM scale (cf. arXiv:2607.11997) is the
`llm_speciation` question (Phase 3).