Clarity pass over the main text (36-item audit), Discussion rewrite and cut, acknowledgements, Souly et al. as ref 62, lettered SI panels, model section moved under Results; plus the untracked curriculum/society/compose/smol configs, runners, figures, stats and tests that the SI already cites. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm |
||
|---|---|---|
| .. | ||
| s1 | ||
| s2 | ||
| s3 | ||
| curriculum_timing.pdf | ||
| curriculum_timing.png | ||
| README.md | ||
Conflict-arrival curricula — does the decline (and the collapse) follow conflict, or generation?
Claim tested (manuscript revision 2026-09-12). In the Latin-square population the fraction of
merges a lineage declines rose with generation, and the obligate-merge arm collapsed from generation
4. Three things rise together there: adapter age, skill count, and the arrival of the two families
whose answer conventions conflict (boolq yes/no, winogrande 1/2). The decorrelated curriculum
separated complementarity from generation but not conflict arrival. These two curricula move ONLY
conflict arrival: conflict-early puts boolq/winogrande in generations 1–2 of every lineage;
conflict-late puts them in generations 5–6; the four compatible families fill the rest in
rotated orders. Age and skill count rise one family per generation in both (tested in
tests/test_llm_curriculum.py). Pre-registered readings: declines and collapse at generations 2–3
in early and 5–6 in late → conflict arrival drives them; the same rise/collapse schedule in both →
age or skill count.
Setup. As curriculum_v5 (Qwen2.5-1.5B base, 3 lineages, 6 generations, 300 new + 150 replay
examples, rank-16 LoRA, 60 test items per family). Configs: curriculum_v5_{early,late}.yaml
(arms isolated + declinable society) and curriculum_v5_{early,late}_obl.yaml (obligate
society). Seeds 1–3 via hpc/llm_curriculum_timing.pbs (L40S, 20–40 min per element). Bundles:
results/llm_curriculum_v5_{early,late,early_obl,late_obl}/s{seed}/. Figure
curriculum_timing.png (this directory); numbers below from figures/stats_llm_curriculum.py.
Result — generation, not conflict arrival
- Declines rise with generation in both curricula on the same schedule (mean fraction declined, generations 1→6; conflict present from generation 2 in early, 6 in late): early 0.56, 0.33, 0.67, 0.67, 0.89, 0.78 · late 0.44, 0.67, 0.44, 0.56, 0.78, 0.89 (Latin 0.44 → 1.00; decorrelated 0.44 → 0.89).
- Conflict-timing test (partial Spearman of declined on a conflict-present indicator, controlling for generation; seed-clustered bootstrap): early + late pooled, n = 36: ρ(conflict | generation) = −0.09, 95% CI (−0.45, 0.15); ρ(generation | conflict) = +0.45. All four curricula pooled, n = 72: ρ(conflict | generation) = −0.04, CI (−0.30, 0.15); ρ(generation | conflict) = +0.39. Complementarity, pooled over four curricula with generation controlled: +0.05, CI (−0.08, 0.20).
- Obligate collapse also follows generation. Best-lineage all-families accuracy of the obligate arm, generations 1→6, mean of 3 seeds: early 0.63, 0.61, 0.72, 0.68, 0.50, 0.28 · late 0.75, 0.76, 0.71, 0.61, 0.57, 0.44 (Latin 0.68 … 0.27). Conflict-early dips when the pair arrives (generation 2), recovers to the others' level by generation 3, and collapses from generation 5; conflict-late collapses from generation 4 with its conflicting pair still to come, and ends less deep. Final obligate − isolated: early −0.517 ± 0.050, late −0.395 ± 0.053 (3/3 seeds each).
- Declinable arms finish level with never merging in both curricula: early 0.777 vs 0.796 (per seed −0.022, −0.039, +0.003), late 0.791 vs 0.781 (+0.019, +0.003, +0.006).
Reading. With conflict arrival moved by four generations, neither the decline curve nor the collapse moved with it. What remains confounded is adapter age with skill count (both rise one per generation by construction). The recombination-modifier reading (declines track complementarity) stays unsupported; the "conflict arrival explains the collapse" reading is now also unsupported at the level of timing, though conflicting conventions still set the per-pair error floor (Proposition S2) and the early-arrival dip is visible. Falsifier for the generation reading (not triggered): declines or collapse shifting by ≥ 3 generations between the two curricula.