MachineSex/paper/pnas/si.md

SI Appendix — The evolution of sex for artificial intelligence

Skeleton assembled at Phase 4; finalised at submission. Every numbered experiment has a committed config (configs/), artifact triple (results/<name>/results.parquet + resolved config + manifest with content hashes and git commit), a README with its legend and falsifier status, and a figure that regenerates from the parquet alone. reproduce.sh re-runs everything from the master seeds.

SI Text S1–S2: formal statements

S1. The incompatibility floor: what no alignment can remove (E13c)

Setting. Models A and B are trained on the same input distribution; their target label functions f_A and f_B agree except on a conflict set S of probability mass μ(S) (in E13's conflict condition, the cyclically-relabelled classes; μ(S) ≈ conflict_frac up to class balance). A function-preserving transformation T (any composition of hidden-unit permutations and, for ReLU networks, positive per-unit rescalings — the full unit symmetry group of a plain ReLU MLP) satisfies T(B)(x) = B(x) for all x by construction.

Proposition 1 (endpoint invariance — with the term "chord" defined precisely). Here "chord" means the α-linear interpolation of the endpoint loss values, (1−α)·L(A) + α·L(B) — the baseline in the barrier definition, a function of the endpoints only — NOT the weight-space interpolation path. For every function-preserving T, the endpoint functions, hence the endpoint losses and this chord, are identical for (A, T(B)) and (A, B). The interpolation path itself is generally NOT invariant — losses along (1−α)·A + α·T(B) change with T, which is precisely why alignment can lower a barrier. (Immediate from the definition of function-preserving.) Scope caveat: our aligner provably recovers a permuted-and-rescaled copy exactly — an important special case — but this does not establish global optimality of the alignment over the symmetry group for independently trained networks; the decomposition's "removable" share is therefore a lower bound, and the "residual" an upper bound, on their true values.

Proposition 2 (no merged model can serve both parents). Let h be any single classifier (in particular, any interpolated/merged model, under any alignment). On every x ∈ S, f_A(x) ≠ f_B(x), so h(x) disagrees with at least one of them. Hence

ε_A(h) + ε_B(h) ≥ μ(S), and therefore max(ε_A(h), ε_B(h)) ≥ μ(S)/2,

where ε_P(h) is h's error against parent P's labels. A hybrid of two models whose conventions conflict on mass μ(S) errs at rate at least μ(S)/2 against at least one parent — hybrid disadvantage with an information-theoretic floor, independent of the alignment group, the architecture, and the merging operator. This is reproductive isolation in the fitness sense: past a given functional conflict, no recombination operator produces an offspring loyal to both lineages.

What remains empirical, and why the experiment is designed as it is. Propositions 1–2 do not bound the single-task path barrier (the loss along the interpolation between A and T(B) evaluated on one parent's task): in principle a path could dip toward one parent's function. Whether it does is exactly what E13 measures — and the measured answer is that it does not: the conflict-condition barrier is unchanged by permutation alignment (residual) and by alignment modulo the full permutation × positive-rescaling group (residual_scale), while the same aligner removes ~all of the independent-init barrier (the positive control). Richer-symmetry results for transformers (arXiv:2606.23607; neuron-identifiability approaches to linear mode connectivity, 2026) strengthen the removable side of the decomposition and are therefore complementary: the more barrier a larger group can remove for compatible models, the sharper the meaning of the residual that survives for incompatible ones — and Proposition 2 caps what any of them could ever achieve on the conflict set.

Terminology note for the paper. "Residual (after alignment)" = the estimated functional incompatibility; for ReLU MLPs we align modulo the full unit symmetry group, so the estimate is not confounded by missed symmetries of that architecture class.

S2. Emergent vs imposed incompatibility (E13b framing)

The conflict condition imposes contradiction (the two label maps disagree on S), which pins μ(S) > 0 and activates Proposition 2. A true Bateson–Dobzhansky–Muller incompatibility is emergent: each lineage's substitutions are harmless on their own background (μ(S) = 0 — the training signals never contradict), and incompatibility, if any, arises only in the combination. The disjoint (complementary class specialists) and augment (divergent input conventions) conditions realise this: any residual barrier they develop cannot be attributed to label conflict and is the emergent-speciation signal proper. Pre-registered readings: residual grows with divergence → model speciation is emergent in real weights (E12's trajectory realised); residual stays at the shared-control level → within this regime, trained networks are more merge-compatible than the biological analogy predicts — an honest bound on the analogy, and itself a design-relevant result (merging is safe absent functional conflict).

Outcome (2026-08-11 run, 4 reps, t_div ≤ 3200): the second reading. Residual 0.000 at every divergence in both emergent conditions, and the merge rescues the forgetting disjoint specialists (parents → 0.535/0.474 on the full task; merged ≈ 0.955 throughout — a sustained Fisher–Muller rescue at zero barrier). Isolation in real weights required functional conflict in this regime; whether long-horizon over-specialisation erodes mergeability at LLM scale (cf. arXiv:2607.11997) is the llm_speciation question (Phase 3).

SI Table S1: the claims ledger (status / assumptions / evidence / limits)

Claim	Status	Key assumptions	Evidence	Known limits
Collapse = Wright–Fisher drift (minimal model)	Exact (diagnosis conceded to prior work)	Knowledge = categorical distribution; refit = resample	Closed forms reproduced to <0.5%	Real learners add a signed, architecture-specific estimator bias (measured)
Grounding = immigration; critical real-data fraction ≪ 1	Exact + empirical sign	Fresh samples from a fixed, non-drifting truth	Exact H_eq; g*≈0.048; sign holds in RNN/MLP/VAE and on MNIST	Deepest tail unrescuable at feasible budgets (m ∼ 1/p); sharp threshold softens in trained nets
"Merge, don't average" conservation	Exact for the output-mean operator	Rare-item regime; an oracle/verifier identifies the strongest source	E4 closed form + simulation; neural reproduction	Weight-averaging and routing are empirical cousins, not instances; budgets differ; bridge = the headroom rule
Offspring exceed every parent (Fisher–Muller)	Interpretation + empirical	Complementary (decorrelated) parents; verifiable fitness	E8 analytic; 7B LoRA merge beats every specialist on every family	LLM tier: 3 lexically-distinct families; multi-seed replication in progress
Outbreeding depression on rugged landscapes; operator design rule	Exact-model result; hypothesis at LLM scale	NK epistasis stands in for skill entanglement	E9–E10; directed selection rescues	Not yet mapped onto a real task-entanglement measure
Optimal mate-pool breadth shrinks with ruggedness	Exact-model result; hypothesis for merging populations	Ring population, local selection	E14	Phenomenon known to island-model evolutionary computation; our contribution is the mapping and the diversity/mean decomposition
Merge failure decomposes into coordinate artefact + functional residual	Empirical (MLP tier; LLM tier in progress)	Alignment enumerates the architecture's unit symmetries	Full-symmetry residual ≈ 0 (compatible) vs ≈ naive (conflict); cliff in hybrid fitness	Scoped to aligned linear interpolation; conflict floor is information-theoretic, not genetic
Epistasis (not divergence) sets the cliff; snowball onset	Exact-model result; hypothesis at the neural tier	BDM incompatibility structure	E12	Snowball count ≠ performance cliff without the effect-size link; neural test outstanding
Pre-merge functional disagreement predicts merge penalty	Empirical, within a controlled grid (0.5B, 13 conditions × 3 seeds)	Constructed conflict/overlap/duration axes; oracle-potential outcome (pre-registered; ordering sensitive to reference)	Clustered CIs exclude 0; held-out LOCO ρ≈0.4; selected geometry baselines ≈ 0	Head-to-head predictor differences not individually significant; only selected baselines; generalisation to real task pairs open
Confidence weighting improves rank prediction over raw disagreement	Not supported (pre-registered internal prediction)	—	Paired Δ|ρ| ≈ −0.02, CI [−0.13, +0.06]	Weighting does double the conflict-vs-compat level contrast
The predictor improves budget-matched operator choice	Open	—	Soup-vs-route gap readout noise-dominated at 0.5B	The practical payoff; untested
Emergent speciation without conflict	Not observed (pre-registered)	Shared ancestry, compatible tasks, tested divergences	E13b: residual 0.000; merge rescues specialists	Bounds the hypothesis; longer horizons/distribution shift/capacity pressure untested
Grounding + sex + diversity jointly necessary	Exact-model result; hypothesis at LLM scale	Conformity stands in for self-consumption	E11 four-arm ablation, each arm failing distinctly	The full grounded LLM society is unbuilt

SI Table S2: headline quantitative results

Headline quantitative results with sample sizes, uncertainty, and outcome definitions (full per-experiment tables and falsifier status in the per-experiment documentation).

Result	Setting / n	Outcome definition	Headline
Closed-form validation	Analytic tier; standing tests	Simulated vs closed-form H-decay, immigration equilibrium, multi-teacher union	Agreement < 0.5%
Grounding retention	Minimal model; 18+ replicates per point	Fraction of equilibrium diversity retained at grounding g (operational threshold)	g ≈ 0.05 retained ≥95% (tested setting); smooth in g
MNIST collapse & rescue	Conv-VAE, 4 replicates; frozen oracle (98.5% mode acc.)	Mode support / forward-KL over generations	Dry: 30→1 modes; 10% grounding: 30/30 held
Fisher–Muller in LLMs	5 seeds (0.5B), fixed tests; single 7B run	Merged vs best-specialist accuracy (overall; worst family)	Ties 0.647±0.027 vs 0.592±0.009; 7B 0.87 vs 0.77
Union vs blend (headroom)	3 seeds (0.5B hard); single 7B-hard run	Paired per-seed ordering, routing vs weight-average	Routing > blend in 3/3 seeds; one catastrophic blend failure avoided
Speciation decomposition	MLPs, 3 replicates	LMC error barrier residual after permutation+rescaling alignment	Same-task 0.001; conflict 0.497 (naive 0.502)
Emergent isolation	MLPs 4 reps to 6.4× base training; LLM 1→12 epochs	Residual barrier; merged vs parent accuracy	0.000 everywhere; merge rescues parents (≈0.955 vs ≈0.50)
Predictive test	13 conditions × 3 seeds (0.5B)	Merge penalty vs oracle parent potential (pre-registered; ±: clustered 95% CI)	Functional ρ +0.45/+0.46, CI excl. 0; LOCO ρ ≈ 0.4; geometry n.s.; paired differences n.s.

SI Methods (per tier — full details in the per-experiment READMEs and configs)

Analytic tier (E1–E14). Wright–Fisher simulator over K-item distributions; closed-form validation suite (tests/test_scientific_validation.py, <0.5% tolerances); learning kernel; multi-locus genotypes, NK landscapes, n-parent crossover (E7–E11); BDM speciation model (E12); mating structure (E14). Bitwise reproducible from master seeds.

Neural tier. Histogram bridge (exact reduction to the analytic tier — the harness gate); RNN/MLP/VAE collapse+grounding on a synthetic mode universe with an exact oracle; conv-VAE on MNIST with a frozen CNN oracle (98.5% mode accuracy; 30x30 confusion matrix recorded as the measurement floor); E13 speciation: no-BatchNorm MLPs, weight-average merges, LMC error barriers before/after alignment under the complete unit symmetry group (deterministic Re-Basin permutation matching composed with exact scale canonicalisation; sanity gate recovers a permuted-and-rescaled copy exactly); pre-registered emergent conditions (disjoint classes; shifted-view conventions).

Language-model tier. LoRA rank-16 specialists on procedural task families with an exact-match verifier; Qwen2.5-Instruct 0.5B/7B; operators: soup/TIES adapter arithmetic, per-input routing, Dirichlet offspring populations screened on held-out validation; multi-seed protocol (fixed tests, varied training seed); speciation knobs (conflicting conventions on ambiguous prompts; duration); the controlled predictive test (six pre-merge predictors; three decorrelated axes; robust statistics via figures/stats_llm_epistasis.py: condition-clustered bootstrap, paired contrasts, leave-one-condition-out prediction, three outcome references). Statistical (per-seed) reproducibility documented for GPU tiers.

SI Statistics

Output of figures/stats_llm_epistasis.py (clustered CIs, paired predictor contrasts, LOCO held-out prediction, outcome-reference sensitivity, within/between-axis decomposition) — reproduced verbatim at submission. Chronology of the predictive test (prospective / adaptive / post-hoc) as disclosed in results/llm_epistasis/README.md.

SI Figures

One per experiment, regenerated from committed artifacts: E1–E14, bridge/collapse/grounding/ architectures/recombination, kernel (sharpen/smooth), mnist_collapse (+ montage), speciation_real (decomposition/cliff/emergent), llm_merge(_hpc/_seeds), llm_moe(_hpc/_hard_hpc/_hard_seeds), llm_directed(_hpc/_hard_hpc/_hard_seeds), llm_speciation(_add), llm_epistasis(_compat).