E13b/c: harden real-weight speciation — full symmetry group + emergent-divergence null
E13c (the symmetry defense): alignment now runs modulo the FULL function-preserving unit symmetry group of a ReLU MLP (per-unit positive rescaling via canonicalise_scale, composed with Re-Basin permutations; sanity gate recovers a permuted-and-rescaled copy exactly). Verdict: the full group removes the independent-init barrier (residual 0.001) and essentially none of the conflict barrier (0.502 -> 0.497) — the residual is functional, not a missed symmetry (answers arXiv:2606.23607). The cliff gains a hybrid-fitness readout: merged accuracy 0.97 -> 0.03 with conflict. Floor proposition drafted (paper/si-notes.md S1): endpoint invariance + max(eps_A, eps_B) >= mu(S)/2 for any merged model under any alignment group. E13b (emergent divergence): pre-registered second reading — with NO conflicting training signal (disjoint class specialists; rolled-input conventions), residual is 0.000 at every divergence to t_div=3200, and the merge RESCUES the forgetting specialists (parents 0.535/0.474 -> merged 0.955; a sustained Fisher-Muller rescue at zero barrier). Speciation in real weights requires functional conflict; it does not emerge from compatible specialisation on shared ancestry. LLM-scale over-specialisation (cf. 2607.11997) deferred to Phase-3 llm_speciation. 3-panel figure, READMEs, +2 tests (149 green), make mnist wired. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
parent
72d5e9e736
commit
ea051a5f92
15 changed files with 435 additions and 77 deletions
67
paper/si-notes.md
Normal file
67
paper/si-notes.md
Normal file
|
|
@ -0,0 +1,67 @@
|
|||
# SI notes — drafts of formal statements for the PNAS manuscript
|
||||
|
||||
*Working drafts; folded into the SI Appendix at Phase 4. Each statement is written to be exactly as
|
||||
strong as what is true — no more.*
|
||||
|
||||
## S1. The incompatibility floor: what no alignment can remove (E13c)
|
||||
|
||||
**Setting.** Models A and B are trained on the same input distribution; their target label functions
|
||||
`f_A` and `f_B` agree except on a conflict set `S` of probability mass `μ(S)` (in E13's conflict
|
||||
condition, the cyclically-relabelled classes; `μ(S) ≈ conflict_frac` up to class balance). A
|
||||
*function-preserving transformation* `T` (any composition of hidden-unit permutations and, for ReLU
|
||||
networks, positive per-unit rescalings — the full unit symmetry group of a plain ReLU MLP) satisfies
|
||||
`T(B)(x) = B(x)` for all `x` by construction.
|
||||
|
||||
**Proposition 1 (endpoint invariance).** For every function-preserving `T`, the endpoint functions —
|
||||
and hence the endpoint losses/errors and the linear chord between them — are identical for the pair
|
||||
`(A, T(B))` and the pair `(A, B)`. Alignment can only re-coordinate the *interpolation path*, never
|
||||
the endpoints or the chord. *(Immediate from the definition of function-preserving.)*
|
||||
|
||||
**Proposition 2 (no merged model can serve both parents).** Let `h` be *any* single classifier (in
|
||||
particular, any interpolated/merged model, under any alignment). On every `x ∈ S`, `f_A(x) ≠ f_B(x)`,
|
||||
so `h(x)` disagrees with at least one of them. Hence
|
||||
|
||||
`ε_A(h) + ε_B(h) ≥ μ(S)`, and therefore `max(ε_A(h), ε_B(h)) ≥ μ(S)/2`,
|
||||
|
||||
where `ε_P(h)` is `h`'s error against parent `P`'s labels. A hybrid of two models whose conventions
|
||||
conflict on mass `μ(S)` errs at rate at least `μ(S)/2` against at least one parent — **hybrid
|
||||
disadvantage with an information-theoretic floor, independent of the alignment group, the
|
||||
architecture, and the merging operator.** This is reproductive isolation in the fitness sense: past a
|
||||
given functional conflict, *no* recombination operator produces an offspring loyal to both lineages.
|
||||
|
||||
**What remains empirical, and why the experiment is designed as it is.** Propositions 1–2 do *not*
|
||||
bound the single-task path barrier (the loss along the interpolation between A and `T(B)` evaluated
|
||||
on one parent's task): in principle a path could dip toward one parent's function. Whether it does is
|
||||
exactly what E13 measures — and the measured answer is that it does not: the conflict-condition
|
||||
barrier is unchanged by permutation alignment (`residual`) *and* by alignment modulo the full
|
||||
permutation × positive-rescaling group (`residual_scale`), while the same aligner removes ~all of the
|
||||
independent-init barrier (the positive control). Richer-symmetry results for transformers
|
||||
(arXiv:2606.23607; neuron-identifiability approaches to linear mode connectivity, 2026) strengthen
|
||||
the *removable* side of the decomposition and are therefore complementary: the more barrier a larger
|
||||
group can remove for *compatible* models, the sharper the meaning of the residual that survives for
|
||||
*incompatible* ones — and Proposition 2 caps what any of them could ever achieve on the conflict set.
|
||||
|
||||
**Terminology note for the paper.** "Residual (after alignment)" = the estimated functional
|
||||
incompatibility; for ReLU MLPs we align modulo the full unit symmetry group, so the estimate is not
|
||||
confounded by missed symmetries of that architecture class.
|
||||
|
||||
## S2. Emergent vs imposed incompatibility (E13b framing)
|
||||
|
||||
The conflict condition *imposes* contradiction (the two label maps disagree on `S`), which pins
|
||||
`μ(S) > 0` and activates Proposition 2. A true Bateson–Dobzhansky–Muller incompatibility is
|
||||
*emergent*: each lineage's substitutions are harmless on their own background (`μ(S) = 0` — the
|
||||
training signals never contradict), and incompatibility, if any, arises only in the *combination*.
|
||||
The `disjoint` (complementary class specialists) and `augment` (divergent input conventions)
|
||||
conditions realise this: any residual barrier they develop cannot be attributed to label conflict and
|
||||
is the emergent-speciation signal proper. Pre-registered readings: residual grows with divergence →
|
||||
model speciation is emergent in real weights (E12's trajectory realised); residual stays at the
|
||||
`shared`-control level → within this regime, trained networks are *more* merge-compatible than the
|
||||
biological analogy predicts — an honest bound on the analogy, and itself a design-relevant result
|
||||
(merging is safe absent functional conflict).
|
||||
|
||||
**Outcome (2026-08-11 run, 4 reps, t_div ≤ 3200): the second reading.** Residual 0.000 at every
|
||||
divergence in both emergent conditions, and the merge *rescues* the forgetting `disjoint` specialists
|
||||
(parents → 0.535/0.474 on the full task; merged ≈ 0.955 throughout — a sustained Fisher–Muller rescue
|
||||
at zero barrier). Isolation in real weights required functional conflict in this regime; whether
|
||||
long-horizon over-specialisation erodes mergeability at LLM scale (cf. arXiv:2607.11997) is the
|
||||
`llm_speciation` question (Phase 3).
|
||||
Loading…
Add table
Add a link
Reference in a new issue