Giorgio Gilestro
84124de143
Clarity pass over the main text (36-item audit), Discussion rewrite and cut, acknowledgements, Souly et al. as ref 62, lettered SI panels, model section moved under Results; plus the untracked curriculum/society/compose/smol configs, runners, figures, stats and tests that the SI already cites. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
35 lines
2.1 KiB
Markdown
35 lines
2.1 KiB
Markdown
# llm_curriculum_v5_stop3 — forced stop at generation 3 (control for the declinable merge)
|
||
|
||
Config `configs/llm/curriculum_v5_stop3.yaml`: the v5 `society` arm (three lineages, six real-dataset
|
||
families in a Latin square, Qwen2.5-1.5B base, rank-16 adapters continued from the parent) with
|
||
recombination switched off from generation 3 (`merge_until: 3`, no veto). Generations 0–2 merge
|
||
obligately; 3–5 never. Pre-registered in `tasks/prereg-llm-society-v4.md` §8h before running.
|
||
Seed 1 local (batch 24 / train batch 2); seeds 2–3 on CX3 via `hpc/llm_curriculum_controls.pbs`
|
||
(batch 48 / train batch 4), as in the v5 campaign. Layout: `results.parquet` (seed 1), `s2/`, `s3/`.
|
||
|
||
Why: in the seed-1 veto run lineages declined 1/3 of merges at generations 0–2 and 3/3 at 3–5, so
|
||
this is the fixed schedule the declinable merge's *timing* must be separated from.
|
||
|
||
## Result (best lineage, all six families, generation 5; `figures/stats_llm_curriculum.py`)
|
||
|
||
| arm | s1 | s2 | s3 | mean |
|
||
|---|---|---|---|---|
|
||
| forced stop at 3 (this run) | 0.792 | 0.794 | 0.792 | 0.793 |
|
||
| declinable merge (veto) | 0.783 | 0.789 | 0.803 | 0.792 |
|
||
| never merge (isolated) | 0.814 | 0.783 | 0.792 | 0.796 |
|
||
| always merge (society) | 0.211 | 0.247 | 0.350 | 0.269 |
|
||
|
||
| contrast | s1 | s2 | s3 | mean ± 95% CI |
|
||
|---|---|---|---|---|
|
||
| veto − stop3 | −0.008 | −0.006 | +0.011 | −0.001 ± 0.012 |
|
||
| stop3 − isolated | −0.022 | +0.011 | 0.000 | −0.004 ± 0.019 |
|
||
| stop3 − society | +0.581 | +0.547 | +0.442 | +0.523 ± 0.082 |
|
||
|
||
Pre-registered reading (§8h, first branch): the forced stop matches the veto within ±0.03 in every
|
||
seed, so the declinable merge's final accuracy is explained by *when* it stopped merging. What the
|
||
veto adds is that it found that schedule by measurement rather than by having it set in advance;
|
||
the claim that per-decision evaluation adds value beyond timing is not supported here. Both
|
||
non-obligate schedules avoid the collapse of obligate merging (+0.52 over `society`).
|
||
|
||
Falsifier status: the paper's caveat "the control separating the modifier's timing from a fixed
|
||
early stop was not run" is removed; the modifier claim is reduced accordingly.
|