E13b/c: harden real-weight speciation — full symmetry group + emergent-divergence null

E13c (the symmetry defense): alignment now runs modulo the FULL
function-preserving unit symmetry group of a ReLU MLP (per-unit positive
rescaling via canonicalise_scale, composed with Re-Basin permutations;
sanity gate recovers a permuted-and-rescaled copy exactly). Verdict: the
full group removes the independent-init barrier (residual 0.001) and
essentially none of the conflict barrier (0.502 -> 0.497) — the residual
is functional, not a missed symmetry (answers arXiv:2606.23607). The
cliff gains a hybrid-fitness readout: merged accuracy 0.97 -> 0.03 with
conflict. Floor proposition drafted (paper/si-notes.md S1): endpoint
invariance + max(eps_A, eps_B) >= mu(S)/2 for any merged model under any
alignment group.

E13b (emergent divergence): pre-registered second reading — with NO
conflicting training signal (disjoint class specialists; rolled-input
conventions), residual is 0.000 at every divergence to t_div=3200, and
the merge RESCUES the forgetting specialists (parents 0.535/0.474 ->
merged 0.955; a sustained Fisher-Muller rescue at zero barrier).
Speciation in real weights requires functional conflict; it does not
emerge from compatible specialisation on shared ancestry. LLM-scale
over-specialisation (cf. 2607.11997) deferred to Phase-3 llm_speciation.

3-panel figure, READMEs, +2 tests (149 green), make mnist wired.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
Giorgio Gilestro 2026-09-06 12:35:14 +01:00
parent 72d5e9e736
commit ea051a5f92
15 changed files with 435 additions and 77 deletions

67
paper/si-notes.md Normal file
View file

@ -0,0 +1,67 @@
# SI notes — drafts of formal statements for the PNAS manuscript
*Working drafts; folded into the SI Appendix at Phase 4. Each statement is written to be exactly as
strong as what is true — no more.*
## S1. The incompatibility floor: what no alignment can remove (E13c)
**Setting.** Models A and B are trained on the same input distribution; their target label functions
`f_A` and `f_B` agree except on a conflict set `S` of probability mass `μ(S)` (in E13's conflict
condition, the cyclically-relabelled classes; `μ(S) ≈ conflict_frac` up to class balance). A
*function-preserving transformation* `T` (any composition of hidden-unit permutations and, for ReLU
networks, positive per-unit rescalings — the full unit symmetry group of a plain ReLU MLP) satisfies
`T(B)(x) = B(x)` for all `x` by construction.
**Proposition 1 (endpoint invariance).** For every function-preserving `T`, the endpoint functions —
and hence the endpoint losses/errors and the linear chord between them — are identical for the pair
`(A, T(B))` and the pair `(A, B)`. Alignment can only re-coordinate the *interpolation path*, never
the endpoints or the chord. *(Immediate from the definition of function-preserving.)*
**Proposition 2 (no merged model can serve both parents).** Let `h` be *any* single classifier (in
particular, any interpolated/merged model, under any alignment). On every `x ∈ S`, `f_A(x) ≠ f_B(x)`,
so `h(x)` disagrees with at least one of them. Hence
`ε_A(h) + ε_B(h) ≥ μ(S)`, and therefore `max(ε_A(h), ε_B(h)) ≥ μ(S)/2`,
where `ε_P(h)` is `h`'s error against parent `P`'s labels. A hybrid of two models whose conventions
conflict on mass `μ(S)` errs at rate at least `μ(S)/2` against at least one parent — **hybrid
disadvantage with an information-theoretic floor, independent of the alignment group, the
architecture, and the merging operator.** This is reproductive isolation in the fitness sense: past a
given functional conflict, *no* recombination operator produces an offspring loyal to both lineages.
**What remains empirical, and why the experiment is designed as it is.** Propositions 12 do *not*
bound the single-task path barrier (the loss along the interpolation between A and `T(B)` evaluated
on one parent's task): in principle a path could dip toward one parent's function. Whether it does is
exactly what E13 measures — and the measured answer is that it does not: the conflict-condition
barrier is unchanged by permutation alignment (`residual`) *and* by alignment modulo the full
permutation × positive-rescaling group (`residual_scale`), while the same aligner removes ~all of the
independent-init barrier (the positive control). Richer-symmetry results for transformers
(arXiv:2606.23607; neuron-identifiability approaches to linear mode connectivity, 2026) strengthen
the *removable* side of the decomposition and are therefore complementary: the more barrier a larger
group can remove for *compatible* models, the sharper the meaning of the residual that survives for
*incompatible* ones — and Proposition 2 caps what any of them could ever achieve on the conflict set.
**Terminology note for the paper.** "Residual (after alignment)" = the estimated functional
incompatibility; for ReLU MLPs we align modulo the full unit symmetry group, so the estimate is not
confounded by missed symmetries of that architecture class.
## S2. Emergent vs imposed incompatibility (E13b framing)
The conflict condition *imposes* contradiction (the two label maps disagree on `S`), which pins
`μ(S) > 0` and activates Proposition 2. A true BatesonDobzhanskyMuller incompatibility is
*emergent*: each lineage's substitutions are harmless on their own background (`μ(S) = 0` — the
training signals never contradict), and incompatibility, if any, arises only in the *combination*.
The `disjoint` (complementary class specialists) and `augment` (divergent input conventions)
conditions realise this: any residual barrier they develop cannot be attributed to label conflict and
is the emergent-speciation signal proper. Pre-registered readings: residual grows with divergence →
model speciation is emergent in real weights (E12's trajectory realised); residual stays at the
`shared`-control level → within this regime, trained networks are *more* merge-compatible than the
biological analogy predicts — an honest bound on the analogy, and itself a design-relevant result
(merging is safe absent functional conflict).
**Outcome (2026-08-11 run, 4 reps, t_div ≤ 3200): the second reading.** Residual 0.000 at every
divergence in both emergent conditions, and the merge *rescues* the forgetting `disjoint` specialists
(parents → 0.535/0.474 on the full task; merged ≈ 0.955 throughout — a sustained FisherMuller rescue
at zero barrier). Isolation in real weights required functional conflict in this regime; whether
long-horizon over-specialisation erodes mergeability at LLM scale (cf. arXiv:2607.11997) is the
`llm_speciation` question (Phase 3).