recombination: reproduce the E4 "merge, don't average" finding in real weights

src/neural/recombine.py mirrors Layer-1 run_coverage but trains K_T specialist RNNs on
assignments from the exact shared-switch retention construction (K_T/rho/q clean; union
matches the closed form), then recombines the measured teacher distributions two ways:
mean (naive pooling) vs oracle-guided max-merge (per-mode strongest teacher, M2N2-style),
each followed by size-n resampling.

Result (8 reps): at rho=0, union rises 0.49->0.96 (supply matches closed form); analytic
surviving_max rises 0.043->0.087 while surviving_mean stays flat ~0.045 — the conservation
law (averaging cancels the union gain, max-merge realises it). At rho=1 (identical
teachers) union and max are flat. The lesson holds in the neural setting; trained-weight
columns show the same signs but noisier (smoothing inflates baseline; deep tail barely
clears n=200 resampling). torch-gated test added. 93 tests green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-07-04 21:49:44 +01:00
parent aca7b394a3
commit d22dd9d535
7 changed files with 263 additions and 7 deletions

View file

@ -184,13 +184,25 @@ C3 vertical claim deferred.*
lets RNN/MLP retain *more* tail than the exact histogram under grounding (they generalise to
unseen codewords) — an inductive-bias finding worth the write-up.
**2026-07-04 — `recombination` (load-bearing E4 replication).**
- `recombine.py` mirrors `run_coverage` but trains K_T specialist RNNs on assignments from the
exact shared-switch retention construction (K_T/rho/q clean; union matches the closed form), then
recombines the *measured* teacher distributions two ways: `mean` (naive pooling) vs `max`
(oracle-guided union / M2N2-style), each followed by size-n resampling. The neural merge is the
per-mode max over teacher distributions (oracle-guided), NOT weight-averaging of RNNs.
- **Result (8 reps):** at rho=0, **union rises 0.49→0.96** (supply matches closed form); analytic
**surviving_max rises 0.043→0.087 while surviving_mean stays flat ~0.045** — the conservation law
(averaging cancels the union gain; max-merge realises it). At rho=1 (identical teachers) union AND
max are flat — more identical teachers buy nothing. **The "merge, don't average" lesson holds in
the neural setting.** Trained-weight columns show the same signs but noisier: neural smoothing
inflates baseline survival and the deep tail barely clears n=200 resampling (compresses magnitude)
— the expected inductive-bias caveat. torch-gated test added. **93 tests green.**
## Remaining
- [ ] **`grounding` refinement:** re-run with forward-KL as the phase metric + ≥10 reps (and/or
smaller n) for a clean neural g*. Pin the falsifier ("g* ≪ 1 exists") before re-running.
- [ ] **`recombination` (load-bearing):** `recombine.py` — mean-mixture vs union-preserving merge.
The neural merge MUST be **oracle-guided mixture sampling** (sample from the teacher strongest on
each mode), NOT weight-averaging of recurrent nets. Reproduce the E4 "mean flat, max rises" finding.
- [ ] **`region_matched`** grounding (R>1), **`remint`** re-mint gate (optional).
- [ ] **VAE fidelity:** fix the prior-hole mismatch (KL-annealing / free-bits / larger latent) so it
clears the gen-0 gate, then add to `architectures`. Or document as a known limitation.