Giorgio Gilestro
84124de143
Clarity pass over the main text (36-item audit), Discussion rewrite and cut, acknowledgements, Souly et al. as ref 62, lettered SI panels, model section moved under Results; plus the untracked curriculum/society/compose/smol configs, runners, figures, stats and tests that the SI already cites. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
2.1 KiB
llm_curriculum_v5_stop3 — forced stop at generation 3 (control for the declinable merge)
Config configs/llm/curriculum_v5_stop3.yaml: the v5 society arm (three lineages, six real-dataset
families in a Latin square, Qwen2.5-1.5B base, rank-16 adapters continued from the parent) with
recombination switched off from generation 3 (merge_until: 3, no veto). Generations 0–2 merge
obligately; 3–5 never. Pre-registered in tasks/prereg-llm-society-v4.md §8h before running.
Seed 1 local (batch 24 / train batch 2); seeds 2–3 on CX3 via hpc/llm_curriculum_controls.pbs
(batch 48 / train batch 4), as in the v5 campaign. Layout: results.parquet (seed 1), s2/, s3/.
Why: in the seed-1 veto run lineages declined 1/3 of merges at generations 0–2 and 3/3 at 3–5, so this is the fixed schedule the declinable merge's timing must be separated from.
Result (best lineage, all six families, generation 5; figures/stats_llm_curriculum.py)
| arm | s1 | s2 | s3 | mean |
|---|---|---|---|---|
| forced stop at 3 (this run) | 0.792 | 0.794 | 0.792 | 0.793 |
| declinable merge (veto) | 0.783 | 0.789 | 0.803 | 0.792 |
| never merge (isolated) | 0.814 | 0.783 | 0.792 | 0.796 |
| always merge (society) | 0.211 | 0.247 | 0.350 | 0.269 |
| contrast | s1 | s2 | s3 | mean ± 95% CI |
|---|---|---|---|---|
| veto − stop3 | −0.008 | −0.006 | +0.011 | −0.001 ± 0.012 |
| stop3 − isolated | −0.022 | +0.011 | 0.000 | −0.004 ± 0.019 |
| stop3 − society | +0.581 | +0.547 | +0.442 | +0.523 ± 0.082 |
Pre-registered reading (§8h, first branch): the forced stop matches the veto within ±0.03 in every
seed, so the declinable merge's final accuracy is explained by when it stopped merging. What the
veto adds is that it found that schedule by measurement rather than by having it set in advance;
the claim that per-decision evaluation adds value beyond timing is not supported here. Both
non-obligate schedules avoid the collapse of obligate merging (+0.52 over society).
Falsifier status: the paper's caveat "the control separating the modifier's timing from a fixed early stop was not run" is removed; the modifier claim is reduced accordingly.