E13b/c: harden real-weight speciation — full symmetry group + emergent-divergence null
E13c (the symmetry defense): alignment now runs modulo the FULL function-preserving unit symmetry group of a ReLU MLP (per-unit positive rescaling via canonicalise_scale, composed with Re-Basin permutations; sanity gate recovers a permuted-and-rescaled copy exactly). Verdict: the full group removes the independent-init barrier (residual 0.001) and essentially none of the conflict barrier (0.502 -> 0.497) — the residual is functional, not a missed symmetry (answers arXiv:2606.23607). The cliff gains a hybrid-fitness readout: merged accuracy 0.97 -> 0.03 with conflict. Floor proposition drafted (paper/si-notes.md S1): endpoint invariance + max(eps_A, eps_B) >= mu(S)/2 for any merged model under any alignment group. E13b (emergent divergence): pre-registered second reading — with NO conflicting training signal (disjoint class specialists; rolled-input conventions), residual is 0.000 at every divergence to t_div=3200, and the merge RESCUES the forgetting specialists (parents 0.535/0.474 -> merged 0.955; a sustained Fisher-Muller rescue at zero barrier). Speciation in real weights requires functional conflict; it does not emerge from compatible specialisation on shared ancestry. LLM-scale over-specialisation (cf. 2607.11997) deferred to Phase-3 llm_speciation. 3-panel figure, READMEs, +2 tests (149 green), make mnist wired. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
parent
72d5e9e736
commit
ea051a5f92
15 changed files with 435 additions and 77 deletions
|
|
@ -1,11 +1,20 @@
|
|||
"""E13 figure — real-weight model speciation with Git Re-Basin.
|
||||
"""E13 figure — real-weight model speciation: what alignment can and cannot merge, and what emerges.
|
||||
|
||||
(A) The barrier decomposition per condition: the linear-mode-connectivity error barrier between two
|
||||
merged MLPs, split into the part permutation alignment REMOVES (coordinate artefact) and the RESIDUAL it
|
||||
cannot (reproductive isolation). `shared` ≈ 0; `independent` (same task, different init) is almost all
|
||||
removable (residual ≈ 0 — same species, different basis); `conflict` (conflicting tasks) is almost all
|
||||
residual (real isolation). (B) The isolation cliff: residual barrier vs the fraction of conflicting
|
||||
classes — the real-weight image of E12's cliff, after alignment (so it is not a coordinate artefact).
|
||||
(A) The barrier decomposition per condition, at two alignment strengths: the linear-mode-connectivity
|
||||
error barrier between two merged MLPs, naive vs after Git Re-Basin permutation alignment vs after
|
||||
alignment modulo the FULL ReLU unit symmetry group (scale-canonicalisation + permutation, E13c).
|
||||
`independent` (same task, different init) is a coordinate artefact — either alignment removes ~all of
|
||||
it; `conflict` (contradictory label maps) survives both — real reproductive isolation, not a missed
|
||||
symmetry (cf. arXiv:2606.23607).
|
||||
|
||||
(B) The isolation cliff as hybrid fitness: sweeping the fraction of conflicting classes, the residual
|
||||
(full-symmetry) barrier rises while the merged (midpoint) model's accuracy falls 0.97 -> 0.03 — the
|
||||
real-weight image of E12's compatible -> depression -> inviability trajectory.
|
||||
|
||||
(C) Emergent divergence (E13b): children specialising on disjoint classes (or divergent input
|
||||
conventions) from a shared fork develop NO residual barrier at any divergence — instead the merge
|
||||
RESCUES the two forgetting specialists (Fisher-Muller), holding ~0.95 while the parents decay.
|
||||
Speciation requires functional conflict; it does not emerge from compatible specialisation here.
|
||||
|
||||
Usage: python figures/plot_speciation_real.py
|
||||
"""
|
||||
|
|
@ -25,40 +34,72 @@ from _figlib import load_bundle, savefig # noqa: E402
|
|||
def main() -> None:
|
||||
dec, _ = load_bundle("results/speciation_real")
|
||||
cliff, _ = load_bundle("results/speciation_real_cliff")
|
||||
emer, _ = load_bundle("results/speciation_real_emergent")
|
||||
|
||||
fig, axes = plt.subplots(1, 2, figsize=(13, 5))
|
||||
fig, axes = plt.subplots(1, 3, figsize=(17.5, 5))
|
||||
|
||||
# Panel A: removable (coordinate artefact) vs residual (isolation), stacked, per condition.
|
||||
# Panel A: naive / residual(perm) / residual(perm+scale) per condition.
|
||||
ax = axes[0]
|
||||
order = [c for c in ["shared", "independent", "conflict"] if c in set(dec["condition"])]
|
||||
g = dec.groupby("condition").agg(removable=("removable", "mean"),
|
||||
residual=("residual", "mean")).reindex(order)
|
||||
x = np.arange(len(order))
|
||||
ax.bar(x, g["removable"], 0.6, label="removable by alignment\n(coordinate artefact)", color="#9ecae1")
|
||||
ax.bar(x, g["residual"], 0.6, bottom=g["removable"], label="residual after alignment\n(reproductive isolation)",
|
||||
color="#d62728")
|
||||
g = dec.groupby("condition").agg(naive=("barrier_naive", "mean"),
|
||||
res_p=("residual", "mean"),
|
||||
res_s=("residual_scale", "mean")).reindex(order)
|
||||
x = np.arange(len(order)); w = 0.27
|
||||
ax.bar(x - w, g["naive"], w, label="naive (no alignment)", color="#9ecae1")
|
||||
ax.bar(x, g["res_p"], w, label="residual after permutation\n(Git Re-Basin)", color="#fc9272")
|
||||
ax.bar(x + w, g["res_s"], w, label="residual after FULL symmetry group\n(scale + permutation)", color="#d62728")
|
||||
ax.set_xticks(x); ax.set_xticklabels(order)
|
||||
ax.set(ylabel="linear-mode-connectivity error barrier",
|
||||
title="Merge barrier = coordinate artefact + residual isolation\n"
|
||||
"(same task even across inits is coordinate; conflict is real)")
|
||||
ax.legend(frameon=False, fontsize=8)
|
||||
title="(A) coordinate artefact vs functional isolation\n"
|
||||
"(conflict survives the full ReLU symmetry group)")
|
||||
ax.legend(frameon=False, fontsize=7.5)
|
||||
|
||||
# Panel B: the isolation cliff — residual barrier vs conflict fraction.
|
||||
# Panel B: the cliff — residual barrier and hybrid fitness vs conflict fraction.
|
||||
ax = axes[1]
|
||||
cg = cliff.groupby("conflict_frac").agg(res_m=("residual", "mean"), res_s=("residual", "std"),
|
||||
nai_m=("barrier_naive", "mean")).reset_index()
|
||||
ax.plot(cg["conflict_frac"], cg["nai_m"], "--o", color="#999", lw=1.4, label="naive barrier")
|
||||
ax.plot(cg["conflict_frac"], cg["res_m"], "-o", color="#d62728", lw=2, label="residual (after alignment)")
|
||||
ax.fill_between(cg["conflict_frac"], cg["res_m"] - cg["res_s"], cg["res_m"] + cg["res_s"],
|
||||
cg = cliff.groupby("conflict_frac").agg(res_s=("residual_scale", "mean"), sd=("residual_scale", "std"),
|
||||
nai=("barrier_naive", "mean"),
|
||||
hyb=("acc_merge_scale", "mean")).reset_index()
|
||||
ax.plot(cg["conflict_frac"], cg["nai"], "--o", color="#999", lw=1.4, label="naive barrier")
|
||||
ax.plot(cg["conflict_frac"], cg["res_s"], "-o", color="#d62728", lw=2,
|
||||
label="residual (full-symmetry alignment)")
|
||||
ax.fill_between(cg["conflict_frac"], cg["res_s"] - cg["sd"], cg["res_s"] + cg["sd"],
|
||||
color="#d62728", alpha=0.15)
|
||||
ax.set(xlabel="fraction of classes with conflicting labels", ylabel="error barrier",
|
||||
ylim=(-0.02, None),
|
||||
title="The reproductive-isolation cliff, in real weights\n"
|
||||
"(residual rises with task conflict — not removable by alignment)")
|
||||
ax.legend(frameon=False, fontsize=9)
|
||||
title="(B) the isolation cliff, in real weights\n(hybrid fitness falls as conflict rises)")
|
||||
ax2 = ax.twinx()
|
||||
ax2.plot(cg["conflict_frac"], cg["hyb"], "-s", color="#2c7fb8", lw=1.8, label="merged-model accuracy")
|
||||
ax2.set_ylabel("merged (hybrid) accuracy", color="#2c7fb8")
|
||||
ax2.tick_params(axis="y", labelcolor="#2c7fb8"); ax2.set_ylim(-0.02, 1.02)
|
||||
lines, labels = ax.get_legend_handles_labels()
|
||||
l2, la2 = ax2.get_legend_handles_labels()
|
||||
ax.legend(lines + l2, labels + la2, frameon=False, fontsize=7.5, loc="center left")
|
||||
|
||||
fig.suptitle("E13 — real-weight model speciation: what permutation alignment can and cannot merge",
|
||||
y=1.02, fontsize=13)
|
||||
# Panel C: emergent divergence — no isolation; the merge rescues the forgetting specialists.
|
||||
ax = axes[2]
|
||||
colors = {"shared": "#999999", "disjoint": "#2c7fb8", "augment": "#41ab5d"}
|
||||
for cond in ["shared", "disjoint", "augment"]:
|
||||
sub = emer[emer["condition"] == cond]
|
||||
if sub.empty:
|
||||
continue
|
||||
m = sub.groupby("t_div").agg(merge=("acc_merge_scale", "mean"),
|
||||
pa=("acc_parent_a", "mean"), pb=("acc_parent_b", "mean"),
|
||||
res=("residual_scale", "mean")).reset_index()
|
||||
ax.plot(m["t_div"], m["merge"], "-o", color=colors[cond], lw=2, label=f"{cond}: merged")
|
||||
if cond == "disjoint":
|
||||
ax.plot(m["t_div"], (m["pa"] + m["pb"]) / 2, "--", color=colors[cond], lw=1.2,
|
||||
label="disjoint: parents (forgetting)")
|
||||
max_res = float(emer[emer["condition"] != "shared"]["residual_scale"].max())
|
||||
ax.set_xscale("log")
|
||||
ax.set(xlabel="divergence (post-fork training steps)", ylabel="accuracy on the full task",
|
||||
ylim=(0, 1.02),
|
||||
title="(C) emergent divergence does NOT speciate —\n"
|
||||
f"the merge rescues the specialists (max residual = {max_res:.3f})")
|
||||
ax.legend(frameon=False, fontsize=7.5, loc="center left")
|
||||
|
||||
fig.suptitle("E13 — real-weight model speciation: isolation requires functional conflict; "
|
||||
"alignment (even modulo the full symmetry group) cannot remove it, and compatible "
|
||||
"specialists merge into a rescuing generalist", y=1.03, fontsize=12)
|
||||
fig.tight_layout()
|
||||
savefig(fig, "results/speciation_real", "speciation_real")
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue