Giorgio Gilestro
6f8cef1ac5
Removed from main (all preserved on the dev branch): the arXiv build and
its sources, design documents (blueprint, results summary, review responses,
essay drafts), tasks/ and CLAUDE.md, the cover letter and reference tooling,
two unused manuscript figures, and every experiment that feeds no figure or
number in the paper: the collapse null, the sexual-vs-asexual lineage, the
NK speciation variant, the 0.5B single-seed LLM prototypes, the compose and
society experiments with their calibration and pilot runs, and their
configs, runners, tests, figure scripts and PBS jobs. Their result bundles
are moved to results/_archive/ (ignored) so the parquets stay on disk.
Also: plot_llm_speciation reads the s{seed}/ layout; the mating-breadth
plot writes under its bundle name; Makefile targets reduced to the kept
experiments; REPRODUCING.md and README point to dev for the rest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
|
||
|---|---|---|
| .. | ||
| kernel.pdf | ||
| kernel.png | ||
| manifest.json | ||
| README.md | ||
| resolved_config.yaml | ||
kernel — the learning kernel: why real learners deviate from neutral drift
(This legend covers both results/figS2_kernel_sharpen/ and results/figS2_kernel_smooth/; the figure
kernel.png is written into both.)
Claim tested. Neutral Wright–Fisher drift (the histogram bridge, and the baseline of Riis
2026) is the null model of collapse. But Layer 1.5 showed real trained models deviate from it —
and in opposite directions. Can a single extension of Layer 1 — a parameterized learning
kernel on the refit step, p_{t+1} = T_θ(counts/n) — reproduce both deviations, and does neutral
drift genuinely fail without it?
Setup. The kernel (knowledge/kernel.py) has two population-genetics knobs, both reducing to
neutral drift at their defaults (so the histogram and every scientific-validation test are
unchanged): reset u — mutation toward a prior (p ← (1−u)p + u·π), i.e. smoothing; and
temperature τ — sharpening (p ∝ p^{1/τ}, τ<1 concentrates), i.e. mode-competition. Two
matched-to-neural regimes, 24 replicates each.
The four panels (kernel.png; blue = neutral, red = kernel-on, green dashed = the real neural model)
Top row — VAE regime (n=6000, K=30), pro-collapse:
- Heterozygosity. Neutral drift is inert — at
n=6000it barely moves (Hstays atH*). Yet the real VAE (green) collapsed toH≈0. Sharpening (τ=0.8) reproduces the collapse. Neutral drift is falsified; the estimator's mode-competition is required. - Support. Neutral holds ~all 30 modes; sharpening → 1 mode, matching the VAE.
Bottom row — RNN regime (n=200, K=256), anti-collapse:
3. Heterozygosity. Neutral drift drives H → 0, but the real RNN (green) only partially
collapses (H floors at ~0.68). Mutation u=0.006 reproduces the floor. The estimator here
removes collapse pressure.
4. Forward-KL. Neutral diverges; smoothing plateaus. Honest caveat: uniform-mutation plateaus
above the RNN's KL (~5 vs ~2) — evidence the RNN's smoothing target is truth-like, not
uniform (a refinement for future work). The sign is unambiguous.
Takeaway
Model collapse in real learners = neutral drift ⊕ an architecture-specific estimator-bias
operator that can point either way. The histogram sits at the neutral null (u=0, τ=1); the VAE
sharpens (adds collapse); the RNN/MLP smooth (add a diversity floor). This mechanistically
explains the Layer-1.5 architecture-generality result and the softened neural g*, and develops
the exact axis Riis (2026) names as future work ("different smoothing schemes… each induce their
own fixed-point geometry… a natural direction for further work"). u/τ are calibrated from a
single neural diagnostic and pinned in the configs. Falsifier (not triggered): if neutral drift
had already reproduced the neural curves, the estimator axis would be superfluous — instead it fails
in both regimes, oppositely.