giorgio/MachineSex
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

recombination: reproduce the E4 "merge, don't average" finding in real weights

Browse source 
src/neural/recombine.py mirrors Layer-1 run_coverage but trains K_T specialist RNNs on
assignments from the exact shared-switch retention construction (K_T/rho/q clean; union
matches the closed form), then recombines the measured teacher distributions two ways:
mean (naive pooling) vs oracle-guided max-merge (per-mode strongest teacher, M2N2-style),
each followed by size-n resampling.

Result (8 reps): at rho=0, union rises 0.49->0.96 (supply matches closed form); analytic
surviving_max rises 0.043->0.087 while surviving_mean stays flat ~0.045 — the conservation
law (averaging cancels the union gain, max-merge realises it). At rho=1 (identical
teachers) union and max are flat. The lesson holds in the neural setting; trained-weight
columns show the same signs but noisier (smoothing inflates baseline; deep tail barely
clears n=200 resampling). torch-gated test added. 93 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-07-04 21:49:44 +01:00
parent aca7b394a3
commit d22dd9d535
7 changed files with 263 additions and 7 deletions
Download patch file Download diff file Expand all files Collapse all files

23
tests/test_neural_torch.py
View file

@ -58,3 +58,26 @@ def test_rnn_dry_collapses_grounded_holds():
assert dry["forward_kl"].iloc[-1] > 1.5 # tail forgotten
assert grd["forward_kl"].iloc[-1] < dry["forward_kl"].iloc[-1] # grounding closer to truth
assert grd["heterozygosity"].iloc[-1] > dry["heterozygosity"].iloc[-1]
def test_recombination_schema_and_union_supply():
# run_recombination trains K_T specialists and reports mean vs max-merge coverage.
# Cheap check: schema is right and the construction-level union rises with K_T (the
# recombination *supply*; magnitudes of surviving coverage need the full multi-rep run).
from neural.recombine import run_recombination
cfg = {
"experiment": "recomb_test", "seed": 20260704, "n_replicates": 1,
"synthetic": {"K": 128, "R": 1, "zipf_s": 1.3, "tail_threshold": 1e-3,
"style_len": 3, "style_vocab": 5, "id_base": 2},
"model": {"kind": "rnn", "hidden": 96, "embed": 20, "epochs": 8, "lr": 2e-3,
"batch_size": 256, "n_eval": 5000},
"coverage": {"n": 200, "q": 0.5, "retain_thresh": 1e-3},
"sweep": [{"param": "K_T", "values": [1, 3]}, {"param": "rho", "values": [0.0]}],
}
df = run_recombination(cfg)
for col in ("union_coverage", "surviving_mean", "surviving_max",
"surviving_mean_target", "surviving_max_target"):
assert col in df.columns
u = df.groupby("K_T")["union_coverage"].mean()
assert u.loc[3] > u.loc[1] + 0.1 # union supply rises with teacher count