Giorgio Gilestro
6f8cef1ac5
Removed from main (all preserved on the dev branch): the arXiv build and
its sources, design documents (blueprint, results summary, review responses,
essay drafts), tasks/ and CLAUDE.md, the cover letter and reference tooling,
two unused manuscript figures, and every experiment that feeds no figure or
number in the paper: the collapse null, the sexual-vs-asexual lineage, the
NK speciation variant, the 0.5B single-seed LLM prototypes, the compose and
society experiments with their calibration and pilot runs, and their
configs, runners, tests, figure scripts and PBS jobs. Their result bundles
are moved to results/_archive/ (ignored) so the parquets stay on disk.
Also: plot_llm_speciation reads the s{seed}/ layout; the mating-breadth
plot writes under its bundle name; Makefile targets reduced to the kept
experiments; REPRODUCING.md and README point to dev for the rest.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
35 lines
2.2 KiB
Markdown
35 lines
2.2 KiB
Markdown
# llm_curriculum_v5_stop3 — forced stop at generation 3 (control for the declinable merge)
|
||
|
||
Config `configs/llm/curriculum_v5_stop3.yaml`: the v5 `society` arm (three lineages, six real-dataset
|
||
families in a Latin square, Qwen2.5-1.5B base, rank-16 adapters continued from the parent) with
|
||
recombination switched off from generation 3 (`merge_until: 3`, no veto). Generations 0–2 merge
|
||
obligately; 3–5 never. Pre-registered in `tasks/prereg-llm-society-v4.md on the dev branch` §8h before running.
|
||
Seed 1 local (batch 24 / train batch 2); seeds 2–3 on CX3 via `hpc/llm_curriculum_controls.pbs`
|
||
(batch 48 / train batch 4), as in the v5 campaign. Layout: `results.parquet` (seed 1), `s2/`, `s3/`.
|
||
|
||
Why: in the seed-1 veto run lineages declined 1/3 of merges at generations 0–2 and 3/3 at 3–5, so
|
||
this is the fixed schedule the declinable merge's *timing* must be separated from.
|
||
|
||
## Result (best lineage, all six families, generation 5; `figures/stats_llm_curriculum.py`)
|
||
|
||
| arm | s1 | s2 | s3 | mean |
|
||
|---|---|---|---|---|
|
||
| forced stop at 3 (this run) | 0.792 | 0.794 | 0.792 | 0.793 |
|
||
| declinable merge (veto) | 0.783 | 0.789 | 0.803 | 0.792 |
|
||
| never merge (isolated) | 0.814 | 0.783 | 0.792 | 0.796 |
|
||
| always merge (society) | 0.211 | 0.247 | 0.350 | 0.269 |
|
||
|
||
| contrast | s1 | s2 | s3 | mean ± 95% CI |
|
||
|---|---|---|---|---|
|
||
| veto − stop3 | −0.008 | −0.006 | +0.011 | −0.001 ± 0.012 |
|
||
| stop3 − isolated | −0.022 | +0.011 | 0.000 | −0.004 ± 0.019 |
|
||
| stop3 − society | +0.581 | +0.547 | +0.442 | +0.523 ± 0.082 |
|
||
|
||
Pre-registered reading (§8h, first branch): the forced stop matches the veto within ±0.03 in every
|
||
seed, so the declinable merge's final accuracy is explained by *when* it stopped merging. What the
|
||
veto adds is that it found that schedule by measurement rather than by having it set in advance;
|
||
the claim that per-decision evaluation adds value beyond timing is not supported here. Both
|
||
non-obligate schedules avoid the collapse of obligate merging (+0.52 over `society`).
|
||
|
||
Falsifier status: the paper's caveat "the control separating the modifier's timing from a fixed
|
||
early stop was not run" is removed; the modifier claim is reduced accordingly.
|