Giorgio Gilestro
871bc39ec6
Revisiting Layer 1 against Layer 1.5 (and Riis 2026, arXiv:2604.08554):
neutral Wright-Fisher is a null that BOTH neural architectures deviate
from, in opposite directions. Add a learning kernel to the refit step,
p_{t+1} = T_theta(counts/n), with two population-genetics knobs -- reset u
(mutation toward a prior = smoothing) and temperature tau (sharpening =
mode-competition) -- both identity by default, so the histogram bridge and
all 68 scientific-validation/correctness tests are unchanged.
Result: neutral drift fails both neural models, oppositely.
- VAE regime (n=6000, K=30): neutral drift is inert (no collapse), yet the
real VAE collapsed to one mode. Sharpening tau=0.8 reproduces it -- the
estimator ADDS collapse pressure.
- RNN regime (n=200, K=256): neutral drives H->0, but the real RNN only
partially collapses. Mutation u=0.006 reproduces the H-floor -- the
estimator REMOVES collapse pressure. Honest caveat: uniform-mutation
overshoots the RNN's forward-KL, evidence its smoothing prior is
truth-like, not uniform (future refinement).
This mechanistically explains the architecture-generality result and the
softened neural g*, and develops the estimator axis Riis names as future
work. New: knowledge/kernel.py, configs/layer1/kernel_{sharpen,smooth}.yaml,
figures/plot_kernel.py (overlays analytic arms vs committed neural
endpoints), READMEs, tests/test_kernel.py (+6, 105 total green). Strategic
Riis positioning recorded in CLAUDE.md: concede "collapse=drift" as prior
art; lead with recombination, the kernel axis, and the Lamarckian society.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|---|---|---|
| .. | ||
| kernel.pdf | ||
| kernel.png | ||
| manifest.json | ||
| README.md | ||
| resolved_config.yaml | ||
kernel — the learning kernel: why real learners deviate from neutral drift
(This legend covers both results/kernel_sharpen/ and results/kernel_smooth/; the figure
kernel.png is written into both.)
Claim tested. Neutral Wright–Fisher drift (the histogram bridge, and the baseline of Riis
2026) is the null model of collapse. But Layer 1.5 showed real trained models deviate from it —
and in opposite directions. Can a single extension of Layer 1 — a parameterized learning
kernel on the refit step, p_{t+1} = T_θ(counts/n) — reproduce both deviations, and does neutral
drift genuinely fail without it?
Setup. The kernel (knowledge/kernel.py) has two population-genetics knobs, both reducing to
neutral drift at their defaults (so the histogram and every scientific-validation test are
unchanged): reset u — mutation toward a prior (p ← (1−u)p + u·π), i.e. smoothing; and
temperature τ — sharpening (p ∝ p^{1/τ}, τ<1 concentrates), i.e. mode-competition. Two
matched-to-neural regimes, 24 replicates each.
The four panels (kernel.png; blue = neutral, red = kernel-on, green dashed = the real neural model)
Top row — VAE regime (n=6000, K=30), pro-collapse:
- Heterozygosity. Neutral drift is inert — at
n=6000it barely moves (Hstays atH*). Yet the real VAE (green) collapsed toH≈0. Sharpening (τ=0.8) reproduces the collapse. Neutral drift is falsified; the estimator's mode-competition is required. - Support. Neutral holds ~all 30 modes; sharpening → 1 mode, matching the VAE.
Bottom row — RNN regime (n=200, K=256), anti-collapse:
3. Heterozygosity. Neutral drift drives H → 0, but the real RNN (green) only partially
collapses (H floors at ~0.68). Mutation u=0.006 reproduces the floor. The estimator here
removes collapse pressure.
4. Forward-KL. Neutral diverges; smoothing plateaus. Honest caveat: uniform-mutation plateaus
above the RNN's KL (~5 vs ~2) — evidence the RNN's smoothing target is truth-like, not
uniform (a refinement for future work). The sign is unambiguous.
Takeaway
Model collapse in real learners = neutral drift ⊕ an architecture-specific estimator-bias
operator that can point either way. The histogram sits at the neutral null (u=0, τ=1); the VAE
sharpens (adds collapse); the RNN/MLP smooth (add a diversity floor). This mechanistically
explains the Layer-1.5 architecture-generality result and the softened neural g*, and develops
the exact axis Riis (2026) names as future work ("different smoothing schemes… each induce their
own fixed-point geometry… a natural direction for further work"). u/τ are calibrated from a
single neural diagnostic and pinned in the configs. Falsifier (not triggered): if neutral drift
had already reproduced the neural curves, the estimator axis would be superfluous — instead it fails
in both regimes, oppositely.