Giorgio Gilestro
ea051a5f92
E13c (the symmetry defense): alignment now runs modulo the FULL function-preserving unit symmetry group of a ReLU MLP (per-unit positive rescaling via canonicalise_scale, composed with Re-Basin permutations; sanity gate recovers a permuted-and-rescaled copy exactly). Verdict: the full group removes the independent-init barrier (residual 0.001) and essentially none of the conflict barrier (0.502 -> 0.497) — the residual is functional, not a missed symmetry (answers arXiv:2606.23607). The cliff gains a hybrid-fitness readout: merged accuracy 0.97 -> 0.03 with conflict. Floor proposition drafted (paper/si-notes.md S1): endpoint invariance + max(eps_A, eps_B) >= mu(S)/2 for any merged model under any alignment group. E13b (emergent divergence): pre-registered second reading — with NO conflicting training signal (disjoint class specialists; rolled-input conventions), residual is 0.000 at every divergence to t_div=3200, and the merge RESCUES the forgetting specialists (parents 0.535/0.474 -> merged 0.955; a sustained Fisher-Muller rescue at zero barrier). Speciation in real weights requires functional conflict; it does not emerge from compatible specialisation on shared ancestry. LLM-scale over-specialisation (cf. 2607.11997) deferred to Phase-3 llm_speciation. 3-panel figure, READMEs, +2 tests (149 green), make mnist wired. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
85 lines
6.2 KiB
Markdown
85 lines
6.2 KiB
Markdown
# E13 — Real-weight model speciation (the alignment residual, now modulo the full symmetry group)
|
||
|
||
**Claim tested.** E12 predicts model *speciation* analytically: as two lineages diverge, recombination
|
||
(merging) fails, via Bateson–Dobzhansky–Muller incompatibilities. E13 confirms it in **real trained
|
||
weights**, separating the part of the merge barrier that is a mere **coordinate artefact** (removable
|
||
by re-coordinating hidden units) from the **residual** that no alignment can remove — the true
|
||
reproductive-isolation signal.
|
||
|
||
**E13c hardening (2026 PNAS campaign).** Recent work shows symmetry groups *richer than permutations*
|
||
remove more of the barrier between independently trained transformers (arXiv:2606.23607;
|
||
neuron-identifiability LMC). We therefore align modulo the **full function-preserving unit symmetry
|
||
group of a plain ReLU MLP** — per-unit positive rescaling (scale canonicalisation, exact) *composed
|
||
with* Git Re-Basin permutation matching (`neural/rebasin.py`; the sanity gate recovers a permuted
|
||
**and rescaled** copy to exact weight identity). Both residuals are reported: `residual` (permutation
|
||
only) and `residual_scale` (full group).
|
||
|
||
**Setup.** No-BatchNorm MLPs (784–512–512–10) on MNIST. Children forked/trained per condition;
|
||
weight-average merge; linear-mode-connectivity error barrier before/after alignment; midpoint
|
||
(merged-model) accuracy recorded alongside. 3 replicates (decomposition/cliff), 4 (emergent).
|
||
Statistically reproducible (seeded); the alignment itself is deterministic NumPy/scipy.
|
||
|
||
### Results — the decomposition (mean over divergence, reps)
|
||
| condition | naive barrier | residual (permutation) | **residual (full symmetry group)** | merged acc |
|
||
|---|---|---|---|---|
|
||
| `shared` (same task, shared fork) | 0.000 | 0.000 | **0.000** | 0.964 |
|
||
| `independent` (same task, different init) | 0.044 | 0.001 | **0.001** | 0.960 (= parents) |
|
||
| `conflict` (contradictory label maps) | 0.502 | 0.502 | **0.497** | **0.037 (inviable)** |
|
||
|
||
- **`independent`**: the barrier is a coordinate artefact — permutations already remove ~98%, and the
|
||
full symmetry group confirms (residual 0.001). The aligned merge performs **at parent level**
|
||
(0.960): same species, different basis.
|
||
- **`conflict`**: the full symmetry group removes essentially nothing (0.502 → 0.497). The residual is
|
||
**functional**, not a missed symmetry — and the hybrid is functionally dead (accuracy 0.037).
|
||
Because the same aligner erased the independent-init barrier, this cannot be a failure to align.
|
||
- Formal floor (SI note S1, `paper/si-notes.md`): for label maps conflicting on mass `μ(S)`, *any*
|
||
single merged model errs at rate ≥ `μ(S)/2` against at least one parent, under *any* alignment
|
||
group and merge operator — hybrid disadvantage is information-theoretic, and endpoints/chord are
|
||
invariant to all function-preserving transformations.
|
||
|
||
### Results — the isolation cliff (`speciation_real_cliff/`)
|
||
Sweeping the fraction of conflicting classes (residual = full-symmetry alignment; `t_div=800`):
|
||
|
||
| conflict fraction | 0.0 | 0.2 | 0.4 | 0.6 | 0.8 | 1.0 |
|
||
|---|---|---|---|---|---|---|
|
||
| residual barrier | 0.000 | 0.122 | 0.187 | 0.278 | 0.406 | 0.506 |
|
||
| **merged (hybrid) accuracy** | 0.968 | 0.764 | 0.586 | 0.396 | 0.199 | 0.034 |
|
||
|
||
`residual_scale ≈ residual` at every point (±0.005): the cliff is functional isolation under the full
|
||
symmetry group. Read as **hybrid fitness**, the merged model's accuracy falls 0.97 → 0.03 — the
|
||
real-weight image of E12's *compatible → outbreeding depression → hybrid inviability* trajectory.
|
||
|
||
### Results — emergent divergence does NOT speciate (`speciation_real_emergent/`, E13b)
|
||
The conflict condition *imposes* contradiction; a true BDM incompatibility is *emergent*. Two
|
||
pre-registered conditions with **no conflicting training signal anywhere**: `disjoint` (child A trains
|
||
only on classes 0–4, child B on 5–9) and `augment` (same labels, inputs rolled ±3 px), swept to
|
||
`t_div = 3200` (children trained 6.4× longer than the shared base):
|
||
|
||
- **Residual barrier = 0.000 at every divergence, both conditions** (naive barrier is 0 too — the
|
||
children never leave the shared basin).
|
||
- The `disjoint` parents decay to 0.535/0.474 on the full task (each forgets the other's classes),
|
||
while the **merged model holds ≈ 0.955 at every divergence** — a sustained ~40-point
|
||
**Fisher–Muller rescue** of two catastrophically-forgetting specialists, at zero barrier.
|
||
`augment` shows the same shape (parents 0.65/0.73, merge ≈ 0.90).
|
||
|
||
**Honest conclusion (the pre-registered second reading):** in this regime — shared ancestry, same
|
||
architecture, compatible tasks, divergence up to 3200 steps — **model speciation does not emerge
|
||
spontaneously; reproductive isolation requires functional conflict.** Trained networks are *more*
|
||
merge-compatible than the biological analogy predicts, and the design rule sharpens: *merge freely
|
||
across divergently-specialised lineages of shared ancestry — the danger is conflicting conventions,
|
||
not specialisation per se.* Scope caveat: small MLPs, one fork depth; whether long-horizon
|
||
over-specialisation at LLM scale erodes mergeability (as the expert-training-duration literature
|
||
suggests, arXiv:2607.11997) is exactly the Phase-3 `llm_speciation` question.
|
||
|
||
### Positioning
|
||
Git Re-Basin / Entezari (barriers as coordinate artefacts), the richer-symmetry LMC results
|
||
(2606.23607 and neuron-identifiability, 2026), Frankle (fork instability), Pari 2024 (route don't
|
||
fuse), Zhou 2026 / 2601.22285 (predict mergeability from divergence/geometry), 2603.09463
|
||
(merge-collapse capacity theory). E13's contribution is the synthesis they lack: a controlled
|
||
decomposition where alignment — *modulo the full symmetry group* — cleanly partitions the merge
|
||
barrier into a removable coordinate artefact and a **functional reproductive-isolation residual** that
|
||
rises with task conflict, is absent under compatible specialisation, and carries an
|
||
information-theoretic floor. **Falsifiers (none triggered):** alignment failing on `independent`
|
||
(would invalidate the residual); conflict showing no residual; the richer symmetry group dissolving
|
||
the conflict residual (it removed 0.005 of 0.502); emergent conditions showing residual attributable
|
||
to alignment failure.
|