E13b/c: harden real-weight speciation — full symmetry group + emergent-divergence null

E13c (the symmetry defense): alignment now runs modulo the FULL
function-preserving unit symmetry group of a ReLU MLP (per-unit positive
rescaling via canonicalise_scale, composed with Re-Basin permutations;
sanity gate recovers a permuted-and-rescaled copy exactly). Verdict: the
full group removes the independent-init barrier (residual 0.001) and
essentially none of the conflict barrier (0.502 -> 0.497) — the residual
is functional, not a missed symmetry (answers arXiv:2606.23607). The
cliff gains a hybrid-fitness readout: merged accuracy 0.97 -> 0.03 with
conflict. Floor proposition drafted (paper/si-notes.md S1): endpoint
invariance + max(eps_A, eps_B) >= mu(S)/2 for any merged model under any
alignment group.

E13b (emergent divergence): pre-registered second reading — with NO
conflicting training signal (disjoint class specialists; rolled-input
conventions), residual is 0.000 at every divergence to t_div=3200, and
the merge RESCUES the forgetting specialists (parents 0.535/0.474 ->
merged 0.955; a sustained Fisher-Muller rescue at zero barrier).
Speciation in real weights requires functional conflict; it does not
emerge from compatible specialisation on shared ancestry. LLM-scale
over-specialisation (cf. 2607.11997) deferred to Phase-3 llm_speciation.

3-panel figure, READMEs, +2 tests (149 green), make mnist wired.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
Giorgio Gilestro 2026-09-06 12:35:14 +01:00
parent 72d5e9e736
commit ea051a5f92
15 changed files with 435 additions and 77 deletions

View file

@ -15,9 +15,20 @@ empirical question this experiment answers:
- ``conflict`` the children learn *conflicting* label maps (B's labels cyclically shifted):
genuinely incompatible functions on shared capacity. A large barrier that alignment *cannot* remove
(residual stays high) true reproductive isolation. "Different species."
- ``disjoint`` (E13b, *emergent* divergence) child A keeps training only on classes 04, child B only
on 59: no contradiction anywhere (a true DobzhanskyMuller setting each lineage's changes are
harmless alone). Does a residual barrier *emerge* with divergence, without imposed conflict? And does
the *merged* model first rescue the two forgetting specialists (FisherMuller) then fail (speciation)
as divergence grows E12's compatible → depression → inviability curve, emergent in real weights?
- ``augment`` (E13b, conventions) same task and labels, but A trains on images rolled +3 px and B on
images rolled 3 px: representational conventions drift with zero output conflict.
The discriminating metric is the **residual** (barrier after alignment): ~0 for ``shared`` and
``independent`` (compatible the incompatibility, if any, is coordinate), large for ``conflict``.
Alignment is reported at two levels (E13c): permutation-only (Git Re-Basin, ``residual``) and
**scale-canonicalised + permutation** (``residual_scale``) the *full* function-preserving unit
symmetry group of a plain ReLU MLP so the residual cannot be attributed to a symmetry the aligner
missed (cf. arXiv:2606.23607). Merged-model (midpoint) accuracies are recorded alongside the barriers.
Divergence is swept via post-fork training steps ``t_div``. Small no-BatchNorm MLPs on MNIST the clean
Re-Basin regime. Statistically reproducible (seeded); NumPy/scipy alignment is deterministic.
@ -30,7 +41,7 @@ from typing import Any, Mapping
import numpy as np
import pandas as pd
from .rebasin import apply_perms, barrier, weight_matching
from .rebasin import apply_perms, barrier, canonicalise_scale, interpolate, weight_matching
from .train import seed_everything
@ -114,10 +125,19 @@ def run_speciation_real(cfg: Mapping[str, Any], seed: int) -> pd.DataFrame:
y2[ytr == int(c)] = int(c2)
return y2
def _rolled(px):
"""Images shifted horizontally by ``px`` pixels (a representational convention; labels intact)."""
return torch.roll(Xtr.reshape(-1, 28, 28), shifts=px, dims=2).reshape(-1, 784)
def subset(cond, child, conflict_frac):
"""(X, y) the child trains on for a given condition."""
if cond == "conflict" and child == 1:
return Xtr, _conflict_labels(conflict_frac) # B learns a conflicting label map
if cond == "disjoint": # emergent DMI: disjoint, compatible tasks
mask = (ytr < 5) if child == 0 else (ytr >= 5)
return Xtr[mask], ytr[mask]
if cond == "augment": # emergent conventions: same task, shifted views
return _rolled(3 if child == 0 else -3), ytr
return Xtr, ytr # shared / independent / conflict-A: normal task
# Two modes: (1) conditions x t_div decomposition; (2) a conflict-fraction isolation cliff.
@ -150,16 +170,36 @@ def run_speciation_real(cfg: Mapping[str, Any], seed: int) -> pd.DataFrame:
pA, pB = children
probe = _mlp(sizes, device); loss_fn = make_loss_fn(probe)
b_naive = barrier(pA, pB, loss_fn)
# E13: permutation-only alignment (Git Re-Basin) — the coordinate artefact.
perms = weight_matching(pA, pB, np.random.default_rng(ss + 5))
b_aligned = barrier(pA, apply_perms(pB, perms), loss_fn)
pB_perm = apply_perms(pB, perms)
b_aligned = barrier(pA, pB_perm, loss_fn)
# E13c: scale-canonicalise both, then match — the FULL ReLU unit symmetry group, so the
# residual cannot be blamed on a symmetry the aligner missed (arXiv:2606.23607).
cA, cB = canonicalise_scale(pA), canonicalise_scale(pB)
perms_c = weight_matching(cA, cB, np.random.default_rng(ss + 5))
cB_al = apply_perms(cB, perms_c)
b_scale = barrier(cA, cB_al, loss_fn)
# E13b money curve: the merged (midpoint) model vs its parents on the full task.
accA, accB = 1 - loss_fn(pA)[1], 1 - loss_fn(pB)[1]
acc_mid_naive = 1 - loss_fn(interpolate(pA, pB, 0.5))[1]
acc_mid_aligned = 1 - loss_fn(interpolate(pA, pB_perm, 0.5))[1]
acc_mid_scale = 1 - loss_fn(interpolate(cA, cB_al, 0.5))[1]
rows.append({
"condition": cond, "t_div": t_div, "conflict_frac": conflict_frac, "replicate": rep,
"barrier_naive": b_naive["error_barrier"],
"barrier_aligned": b_aligned["error_barrier"],
"removable": b_naive["error_barrier"] - b_aligned["error_barrier"],
"residual": b_aligned["error_barrier"],
"barrier_aligned_scale": b_scale["error_barrier"],
"removable_scale": b_naive["error_barrier"] - b_scale["error_barrier"],
"residual_scale": b_scale["error_barrier"],
"loss_barrier_naive": b_naive["loss_barrier"],
"loss_barrier_aligned": b_aligned["loss_barrier"],
"loss_barrier_scale": b_scale["loss_barrier"],
"acc_parent_a": accA, "acc_parent_b": accB,
"acc_merge_naive": acc_mid_naive,
"acc_merge_aligned": acc_mid_aligned,
"acc_merge_scale": acc_mid_scale,
"parent_acc": (accA + accB) / 2.0})
return pd.DataFrame(rows)