recombination: reproduce the E4 "merge, don't average" finding in real weights

src/neural/recombine.py mirrors Layer-1 run_coverage but trains K_T specialist RNNs on
assignments from the exact shared-switch retention construction (K_T/rho/q clean; union
matches the closed form), then recombines the measured teacher distributions two ways:
mean (naive pooling) vs oracle-guided max-merge (per-mode strongest teacher, M2N2-style),
each followed by size-n resampling.

Result (8 reps): at rho=0, union rises 0.49->0.96 (supply matches closed form); analytic
surviving_max rises 0.043->0.087 while surviving_mean stays flat ~0.045 — the conservation
law (averaging cancels the union gain, max-merge realises it). At rho=1 (identical
teachers) union and max are flat. The lesson holds in the neural setting; trained-weight
columns show the same signs but noisier (smoothing inflates baseline; deep tail barely
clears n=200 resampling). torch-gated test added. 93 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-07-04 21:49:44 +01:00
parent aca7b394a3
commit d22dd9d535
7 changed files with 263 additions and 7 deletions

View file

@ -0,0 +1,17 @@
{
"experiment": "recombination",
"master_seed": 20260704,
"git_commit": "aca7b394a3f4b1f487afdb01d70757956408d59f",
"python": "3.14.5",
"libraries": {
"numpy": "2.5.0",
"scipy": "1.18.0",
"pandas": "3.0.3",
"pyarrow": "24.0.0",
"torch": "2.12.1"
},
"rows": 64,
"results_sha256": "8e6cb83e4e5a0c711137d4f70c3d83192f845ceaa9194b3aeac427741808477b",
"layer": "1.5",
"model_kind": "rnn"
}

View file

@ -0,0 +1,44 @@
experiment: recombination
seed: 20260704
n_replicates: 8
source_config:
experiment: recombination
kind: recombination
seed: 20260704
n_replicates: 8
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 22
lr: 0.002
batch_size: 256
n_eval: 12000
coverage:
n: 200
q: 0.5
retain_thresh: 0.001
region_specialisation: false
sweep:
- param: K_T
values:
- 1
- 2
- 3
- 5
- param: rho
values:
- 0.0
- 1.0
output:
dir: results/recombination