recombination: reproduce the E4 "merge, don't average" finding in real weights
src/neural/recombine.py mirrors Layer-1 run_coverage but trains K_T specialist RNNs on assignments from the exact shared-switch retention construction (K_T/rho/q clean; union matches the closed form), then recombines the measured teacher distributions two ways: mean (naive pooling) vs oracle-guided max-merge (per-mode strongest teacher, M2N2-style), each followed by size-n resampling. Result (8 reps): at rho=0, union rises 0.49->0.96 (supply matches closed form); analytic surviving_max rises 0.043->0.087 while surviving_mean stays flat ~0.045 — the conservation law (averaging cancels the union gain, max-merge realises it). At rho=1 (identical teachers) union and max are flat. The lesson holds in the neural setting; trained-weight columns show the same signs but noisier (smoothing inflates baseline; deep tail barely clears n=200 resampling). torch-gated test added. 93 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
aca7b394a3
commit
d22dd9d535
7 changed files with 263 additions and 7 deletions
|
|
@ -184,13 +184,25 @@ C3 vertical claim deferred.*
|
|||
lets RNN/MLP retain *more* tail than the exact histogram under grounding (they generalise to
|
||||
unseen codewords) — an inductive-bias finding worth the write-up.
|
||||
|
||||
**2026-07-04 — `recombination` (load-bearing E4 replication).**
|
||||
|
||||
- `recombine.py` mirrors `run_coverage` but trains K_T specialist RNNs on assignments from the
|
||||
exact shared-switch retention construction (K_T/rho/q clean; union matches the closed form), then
|
||||
recombines the *measured* teacher distributions two ways: `mean` (naive pooling) vs `max`
|
||||
(oracle-guided union / M2N2-style), each followed by size-n resampling. The neural merge is the
|
||||
per-mode max over teacher distributions (oracle-guided), NOT weight-averaging of RNNs.
|
||||
- **Result (8 reps):** at rho=0, **union rises 0.49→0.96** (supply matches closed form); analytic
|
||||
**surviving_max rises 0.043→0.087 while surviving_mean stays flat ~0.045** — the conservation law
|
||||
(averaging cancels the union gain; max-merge realises it). At rho=1 (identical teachers) union AND
|
||||
max are flat — more identical teachers buy nothing. **The "merge, don't average" lesson holds in
|
||||
the neural setting.** Trained-weight columns show the same signs but noisier: neural smoothing
|
||||
inflates baseline survival and the deep tail barely clears n=200 resampling (compresses magnitude)
|
||||
— the expected inductive-bias caveat. torch-gated test added. **93 tests green.**
|
||||
|
||||
## Remaining
|
||||
|
||||
- [ ] **`grounding` refinement:** re-run with forward-KL as the phase metric + ≥10 reps (and/or
|
||||
smaller n) for a clean neural g*. Pin the falsifier ("g* ≪ 1 exists") before re-running.
|
||||
- [ ] **`recombination` (load-bearing):** `recombine.py` — mean-mixture vs union-preserving merge.
|
||||
The neural merge MUST be **oracle-guided mixture sampling** (sample from the teacher strongest on
|
||||
each mode), NOT weight-averaging of recurrent nets. Reproduce the E4 "mean flat, max rises" finding.
|
||||
- [ ] **`region_matched`** grounding (R>1), **`remint`** re-mint gate (optional).
|
||||
- [ ] **VAE fidelity:** fix the prior-hole mismatch (KL-annealing / free-bits / larger latent) so it
|
||||
clears the gen-0 gate, then add to `architectures`. Or document as a known limitation.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue