MachineSex/paper/si-notes.md

SI notes — drafts of formal statements for the PNAS manuscript

Working drafts; folded into the SI Appendix at Phase 4. Each statement is written to be exactly as strong as what is true — no more.

S1. The incompatibility floor: what no alignment can remove (E13c)

Setting. Models A and B are trained on the same input distribution; their target label functions f_A and f_B agree except on a conflict set S of probability mass μ(S) (in E13's conflict condition, the cyclically-relabelled classes; μ(S) ≈ conflict_frac up to class balance). A function-preserving transformation T (any composition of hidden-unit permutations and, for ReLU networks, positive per-unit rescalings — the full unit symmetry group of a plain ReLU MLP) satisfies T(B)(x) = B(x) for all x by construction.

Proposition 1 (endpoint invariance — with the term "chord" defined precisely). Here "chord" means the α-linear interpolation of the endpoint loss values, (1−α)·L(A) + α·L(B) — the baseline in the barrier definition, a function of the endpoints only — NOT the weight-space interpolation path. For every function-preserving T, the endpoint functions, hence the endpoint losses and this chord, are identical for (A, T(B)) and (A, B). The interpolation path itself is generally NOT invariant — losses along (1−α)·A + α·T(B) change with T, which is precisely why alignment can lower a barrier. (Immediate from the definition of function-preserving.) Scope caveat: our aligner provably recovers a permuted-and-rescaled copy exactly — an important special case — but this does not establish global optimality of the alignment over the symmetry group for independently trained networks; the decomposition's "removable" share is therefore a lower bound, and the "residual" an upper bound, on their true values.

Proposition 2 (no merged model can serve both parents). Let h be any single classifier (in particular, any interpolated/merged model, under any alignment). On every x ∈ S, f_A(x) ≠ f_B(x), so h(x) disagrees with at least one of them. Hence

ε_A(h) + ε_B(h) ≥ μ(S), and therefore max(ε_A(h), ε_B(h)) ≥ μ(S)/2,

where ε_P(h) is h's error against parent P's labels. A hybrid of two models whose conventions conflict on mass μ(S) errs at rate at least μ(S)/2 against at least one parent — hybrid disadvantage with an information-theoretic floor, independent of the alignment group, the architecture, and the merging operator. This is reproductive isolation in the fitness sense: past a given functional conflict, no recombination operator produces an offspring loyal to both lineages.

What remains empirical, and why the experiment is designed as it is. Propositions 1–2 do not bound the single-task path barrier (the loss along the interpolation between A and T(B) evaluated on one parent's task): in principle a path could dip toward one parent's function. Whether it does is exactly what E13 measures — and the measured answer is that it does not: the conflict-condition barrier is unchanged by permutation alignment (residual) and by alignment modulo the full permutation × positive-rescaling group (residual_scale), while the same aligner removes ~all of the independent-init barrier (the positive control). Richer-symmetry results for transformers (arXiv:2606.23607; neuron-identifiability approaches to linear mode connectivity, 2026) strengthen the removable side of the decomposition and are therefore complementary: the more barrier a larger group can remove for compatible models, the sharper the meaning of the residual that survives for incompatible ones — and Proposition 2 caps what any of them could ever achieve on the conflict set.

Terminology note for the paper. "Residual (after alignment)" = the estimated functional incompatibility; for ReLU MLPs we align modulo the full unit symmetry group, so the estimate is not confounded by missed symmetries of that architecture class.

S2. Emergent vs imposed incompatibility (E13b framing)

The conflict condition imposes contradiction (the two label maps disagree on S), which pins μ(S) > 0 and activates Proposition 2. A true Bateson–Dobzhansky–Muller incompatibility is emergent: each lineage's substitutions are harmless on their own background (μ(S) = 0 — the training signals never contradict), and incompatibility, if any, arises only in the combination. The disjoint (complementary class specialists) and augment (divergent input conventions) conditions realise this: any residual barrier they develop cannot be attributed to label conflict and is the emergent-speciation signal proper. Pre-registered readings: residual grows with divergence → model speciation is emergent in real weights (E12's trajectory realised); residual stays at the shared-control level → within this regime, trained networks are more merge-compatible than the biological analogy predicts — an honest bound on the analogy, and itself a design-relevant result (merging is safe absent functional conflict).

Outcome (2026-08-11 run, 4 reps, t_div ≤ 3200): the second reading. Residual 0.000 at every divergence in both emergent conditions, and the merge rescues the forgetting disjoint specialists (parents → 0.535/0.474 on the full task; merged ≈ 0.955 throughout — a sustained Fisher–Muller rescue at zero barrier). Isolation in real weights required functional conflict in this regime; whether long-horizon over-specialisation erodes mergeability at LLM scale (cf. arXiv:2607.11997) is the llm_speciation question (Phase 3).