giorgio/MachineSex
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

E13b/c: harden real-weight speciation — full symmetry group + emergent-divergence null

Browse source 
E13c (the symmetry defense): alignment now runs modulo the FULL
function-preserving unit symmetry group of a ReLU MLP (per-unit positive
rescaling via canonicalise_scale, composed with Re-Basin permutations;
sanity gate recovers a permuted-and-rescaled copy exactly). Verdict: the
full group removes the independent-init barrier (residual 0.001) and
essentially none of the conflict barrier (0.502 -> 0.497) — the residual
is functional, not a missed symmetry (answers arXiv:2606.23607). The
cliff gains a hybrid-fitness readout: merged accuracy 0.97 -> 0.03 with
conflict. Floor proposition drafted (paper/si-notes.md S1): endpoint
invariance + max(eps_A, eps_B) >= mu(S)/2 for any merged model under any
alignment group.

E13b (emergent divergence): pre-registered second reading — with NO
conflicting training signal (disjoint class specialists; rolled-input
conventions), residual is 0.000 at every divergence to t_div=3200, and
the merge RESCUES the forgetting specialists (parents 0.535/0.474 ->
merged 0.955; a sustained Fisher-Muller rescue at zero barrier).
Speciation in real weights requires functional conflict; it does not
emerge from compatible specialisation on shared ancestry. LLM-scale
over-specialisation (cf. 2607.11997) deferred to Phase-3 llm_speciation.

3-panel figure, READMEs, +2 tests (149 green), make mnist wired.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
Giorgio Gilestro 2026-09-06 12:35:14 +01:00
parent 72d5e9e736
commit ea051a5f92
15 changed files with 435 additions and 77 deletions
Download patch file Download diff file Expand all files Collapse all files

43
tests/test_rebasin.py
View file

@ -5,7 +5,7 @@ from __future__ import annotations
import numpy as np
import pytest
from neural.rebasin import apply_perms, barrier, interpolate, weight_matching
from neural.rebasin import apply_perms, barrier, canonicalise_scale, interpolate, weight_matching
def _mlp(sizes, rng):
@ -64,3 +64,44 @@ def test_apply_perms_preserves_function():
perms = [rng.permutation(7), rng.permutation(7)]
X = rng.standard_normal((16, 4))
assert np.allclose(_forward(A, X), _forward(apply_perms(A, perms), X))
def _rescale(params, scales_per_layer):
# Apply the positive per-unit rescaling symmetry: unit i of hidden layer k scaled by c>0.
out = [(W.copy(), b.copy()) for W, b in params]
for k, scales in enumerate(scales_per_layer):
W, b = out[k]
out[k] = (W * scales[:, None], b * scales)
Wn, bn = out[k + 1]
out[k + 1] = (Wn / scales[None, :], bn)
return out
def test_canonicalise_scale_preserves_function_and_normalises():
rng = np.random.default_rng(6)
A = _mlp([5, 9, 9, 2], rng)
C = canonicalise_scale(A)
X = rng.standard_normal((24, 5))
assert np.allclose(_forward(A, X), _forward(C, X), atol=1e-8) # function-preserving (ReLU homogeneity)
for k in range(len(C) - 1): # every hidden unit's (W, b) is unit-norm
W, b = C[k]
assert np.allclose(np.sqrt((W ** 2).sum(axis=1) + b ** 2), 1.0)
def test_weight_matching_recovers_permutation_and_rescaling():
# A permuted AND positively-rescaled copy is functionally identical; permutation-only matching can
# miss it, but canonicalise-then-match must realign it to functional identity — the full ReLU
# symmetry group (the E13c referee-proofing gate).
rng = np.random.default_rng(7)
A = _mlp([6, 12, 12, 3], rng)
B = apply_perms(_rescale(A, [np.exp(rng.uniform(-2, 2, 12)), np.exp(rng.uniform(-2, 2, 12))]),
[rng.permutation(12), rng.permutation(12)])
X = rng.standard_normal((32, 6))
assert np.allclose(_forward(A, X), _forward(B, X), atol=1e-6) # symmetry-equivalent copy
cA, cB = canonicalise_scale(A), canonicalise_scale(B)
perms = weight_matching(cA, cB, np.random.default_rng(8))
B_aligned = apply_perms(cB, perms)
assert np.allclose(_forward(cA, X), _forward(B_aligned, X), atol=1e-5) # realigned exactly
# and the aligned weights themselves coincide (canonical form is unique up to permutation)
for (Wa, ba), (Wb, bb) in zip(cA, B_aligned):
assert np.allclose(Wa, Wb, atol=1e-6) and np.allclose(ba, bb, atol=1e-6)