From 72d5e9e736cec94947ed1d4513d5b7f43f50d0f7 Mon Sep 17 00:00:00 2001 From: Giorgio Gilestro Date: Sun, 6 Sep 2026 12:34:59 +0100 Subject: [PATCH] tasks: open the PNAS submission campaign (work order + plan) Post-hold reassessment (2026-08-11): fresh lit scan confirms the core claims (speciation/BDMI, evolution-of-sex framing, mating systems, headroom law) remain unclaimed; new concessions identified (First-Extinction 2509.20101, quantitative-trait 2407.17493, verifier-injection 2510.16657) and one methodological threat to E13 (richer symmetry groups, 2606.23607). Venue check ranks PNAS first (direct 2026 precedent: "Evolvable AI" + letters), over PLoS Comp Bio (scope risk). Five-phase plan: E13 hardening -> arXiv preprint -> llm_speciation + multi-seed -> PNAS manuscript -> submission mechanics. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v --- tasks/todo.md | 8 ++ tasks/workorder-pnas-submission.md | 163 +++++++++++++++++++++++++++++ 2 files changed, 171 insertions(+) create mode 100644 tasks/workorder-pnas-submission.md diff --git a/tasks/todo.md b/tasks/todo.md index 2963b0e..ede03eb 100644 --- a/tasks/todo.md +++ b/tasks/todo.md @@ -393,3 +393,11 @@ C3 vertical claim deferred.* wins where easy tasks let a strong base compose to ceiling. Vindicates E8 max>mean at 7B. - `configs/llm/{moe_hard,moe_hard_hpc,directed_hard_hpc}.yaml`, `hpc/llm_hard.pbs`, hard READMEs+figures, +1 test (131 green). `results/llm_{moe,directed}_hard_hpc/`. + +**2026-08-11 — PNAS submission campaign opened.** GG approved PNAS as target after the post-hold +re-assessment (fresh lit scan: speciation/sex-framing/mating-systems/headroom all still unclaimed; +new concessions First-Extinction 2509.20101 + qt-trait 2407.17493 + verifier-injection 2510.16657; +E13 exposed to richer-symmetry objection 2606.23607). Full plan: `tasks/workorder-pnas-submission.md`. +Phases: (1) E13 hardening (scale-aware alignment + emergent-divergence condition), (2) arXiv preprint, +(3) llm_speciation + multi-seed LLM arc, (4) PNAS-format manuscript (5 consolidated figures, dual +audience), (5) submission mechanics (Zenodo DOI, cover letter, editor/reviewer suggestions). diff --git a/tasks/workorder-pnas-submission.md b/tasks/workorder-pnas-submission.md new file mode 100644 index 0000000..177ebb7 --- /dev/null +++ b/tasks/workorder-pnas-submission.md @@ -0,0 +1,163 @@ +# Work order — PNAS submission: "The Evolution of Sex for Artificial Intelligence" + +*Created 2026-08-11. Target: PNAS Direct Submission research article (~6 pp main + SI Appendix), +preceded by an arXiv preprint (q-bio.PE × cs.LG). Decision basis: 2026-08-11 literature + venue scan +(see memory/competitive-landscape.md, August addendum). GG approved PNAS + the analysis.* + +**Governing constraints.** (1) Dual audience: every claim stated so a computer scientist and a +biologist can each follow it — keep the "two statements of the same fact" device and the +dictionary table; define every term at first use. (2) Honesty ledger: concede Riis + +First-Extinction + quantitative-trait collapse up front; claims are the *cure* and its theory, not +the diagnosis. (3) Reproducibility is the differentiator: every figure from committed artifacts, +code+data DOI at submission. + +--- + +## Phase 1 — Referee-proof the headline (E13 hardening) — Week 1 + +The speciation result (E12+E13) is both our most novel and most exposed claim +(arXiv:2606.23607, June 2026: a symmetry group larger than permutations removes most transformer +barriers). Patch before the preprint goes up. + +- [x] **E13c — scale-aware alignment (the symmetry defense).** *(Done 2026-08-11: canonicalise_scale + + gated tests; decomposition & cliff regenerated — conflict residual 0.502→0.497 under the full + group, independent 0.001; floor proposition drafted in paper/si-notes.md S1. Hybrid-fitness + readout added: cliff accuracy 0.97→0.03.)* + - Extend `neural/rebasin.py` with per-unit **positive rescaling** canonicalisation: for the + no-BatchNorm ReLU MLP, the full function-preserving unit symmetry group is permutation ∘ + positive diagonal scaling. Canonicalise both nets first (rescale each hidden unit to + ‖in-weights‖=1, pushing the norm into the out-weights), *then* permutation-match. Sanity gate: + a randomly permuted **and rescaled** copy must realign to functional identity (extend + `tests/test_rebasin.py`). + - Re-run `speciation_real` + `speciation_real_cliff` reporting `residual_perm` vs + `residual_perm_scale`. Expected: independent-init residual stays ≈0; conflict residual + unchanged. Update figure/README. + - Write the **theoretical floor proposition** (SI): two models with low loss on *contradictory* + label assignments over the same inputs cannot both be matched by any function-preserving + transformation — any single merged function errs on at least the disagreement mass, so the + conflict-condition barrier has an information-theoretic floor independent of the symmetry + group. Cite 2606.23607 and position: their result strengthens ours (the *removable* part may + grow with richer groups; the *functional* residual cannot vanish). Also cite the two June-2026 + LMC papers found in the direct arXiv sweep (2026-08-11): "Beyond Structural Symmetries: Linear + Mode Connectivity via Neuron Identifiability" (2606.03…) and "Functional Equivalence in + Attention … Applications to Linear Mode Connectivity" (2606.16…) — same objection family, same + defense. Direct arXiv API sweep also re-confirmed: speciation/DMI/Fisher–Muller/ratchet/ + evolution-of-sex/mating-for-models have zero ML hits on arXiv itself as of the newest listings. +- [x] **E13b — emergent speciation (the decisive missing experiment).** *(Done 2026-08-11: the + pre-registered SECOND reading — residual 0.000 at every t_div ≤ 3200 in both `disjoint` and + `augment`; the merge RESCUES the forgetting specialists (parents 0.535/0.474 → merged 0.955, + sustained Fisher–Muller). Speciation requires functional conflict in this regime; LLM-scale + over-specialisation deferred to Phase-3 llm_speciation. Figure panel C + READMEs updated.)* Current E13 *imposes* + contradiction; a true Dobzhansky–Muller incompatibility is *emergent* (each change harmless + alone, bad only in combination). Design (`speciation_real` new conditions, CPU/torch, cheap): + - `disjoint`: fork from the shared MNIST base; child A continues on classes 0–4 only, child B on + 5–9 only (no contradiction anywhere). Sweep divergence time `t_div`. Measure naive barrier and + scale+perm-aligned residual on the full task, **and merged-model accuracy vs best parent**. + - `conventions` (optional, purer DMI): same task, different augmentation conventions (A: + rotations, B: inversions) — representational drift with zero output conflict. + - **The money curve:** merged accuracy vs `t_div` should trace E12's *compatible → + outbreeding-depression → inviability* trajectory emergently: at low divergence the merge + *rescues* the two forgetting specialists (Fisher–Muller), at high divergence it fails + (speciation). If it comes out, this is a headline panel. Either outcome is publishable + (no growth ⇒ "models are safer to merge than biology predicts" — an honest bound). + - Tests + README + fold into figure. Pre-register the falsifier language before running. + +## Phase 2 — arXiv preprint (priority stake) — end of Week 1 + +- [ ] Citation refresh in `paper/the-evolution-of-sex-for-ai.md`: **new concessions** — + First-Extinction Law (2509.20101), quantitative-trait collapse (2407.17493), verifier-injection + (2510.16657); **precursor** — Livnat & Papadimitriou, *Sex as an algorithm* (CACM 2016); + **adjacent-to-differentiate** — merge-collapse theory (2603.09463), mergeability prediction + (2601.22285), non-local merging (2410.12766), model kinship (2410.12613), expert-duration + (2607.11997), symmetry-scaling (2606.23607), Harris (2604.05142), in-context diversity collapse + (2604.18005, 2603.24676). +- [ ] Fold E13b/c results into the speciation section (whatever they show — honestly). +- [ ] LaTeX conversion of the full v6 (article class is fine for arXiv; keep figures as-is), + build with tectonic, proof-read pass. +- [ ] Post to arXiv: q-bio.PE primary, cs.LG (or cs.NE) cross-list. PNAS permits preprints. + +## Phase 3 — New experiments for impact & robustness — Weeks 2–3 + +- [ ] **`llm_speciation` — the cliff at the LLM tier (new, highest-impact addition).** + Speciation demonstrated at all three tiers (analytic → MLP → LLM) makes the headline + unassailable. Structural bonus to state in the paper: LoRA deltas live in the frozen base's + coordinate system, so there is **no permutation ambiguity by construction** — any LoRA-merge + failure is *functional* incompatibility, the residual isolated architecturally. + - Conflict knob: two LoRA children from the same base learn contradictory conventions on a + shared family (e.g. "sort ascending" vs "sort descending"; answer-format conflicts) on a + fraction `conflict_frac` of prompts, plus their own disjoint families. Sweep `conflict_frac`, + soup-merge, evaluate on neutral held-out + both convention sets. Predict a monotone cliff. + - Emergent knob: take existing disjoint specialists, sweep **training duration** + (over-specialisation) → merged performance vs steps. Frames the Amazon observation + (2607.11997: optimal expert duration) as *our theory explaining their data* — a strong PNAS + move. + - 0.5B locally first; one 7B CX3 confirm if the sign is clean (`hpc/` PBS, minutes). +- [ ] **Multi-seed the LLM arc (robustness — removes the "one seed" objection).** + - Thread the seed into specialist cache keys (`spec_[_hard]_s`); verify nothing + else assumes the old names. + - 0.5B: seeds 1–5 × {merge, moe, directed} × {easy, hard}. 7B on CX3: seeds 1–3 × hard + {merge, moe, directed} (8–25 min walltimes → trivial). Aggregate figures with 95% CI; update + READMEs; the headroom law now carries error bars. +- [ ] **(Optional, SI-grade) ambiguous-families router stress test** — overlapping-skill families + where the router is no longer trivially perfect; documents the honest limit of union-by-routing. + Do only if time permits; otherwise keep the existing rider sentence. + +## Phase 4 — The PNAS manuscript — Weeks 3–4 + +- [ ] **Restructure** `paper/the-evolution-of-sex-for-ai.md` → PNAS research article + (`paper/pnas/`): Significance (~120 w, dual-audience), Abstract (~250 w), Intro (concede the + diagnosis; thesis: the cure is sex; where this sits), **Results ladder**: + 1. Collapse is drift, grounding is immigration (E1–E3: exact `H_eq`, `g*≈0.048`, tail threshold + `m·p*≳1`) — concede Riis/First-Extinction, keep the immigration delta. + 2. Merge, don't average — the conservation law (E4 + neural recombination). + 3. Sex: Fisher–Muller offspring exceed every parent; outbreeding depression on rugged + landscapes; directed sex as the AI advantage (E8–E10). + 4. **Model speciation** (E12 + hardened E13 + `llm_speciation`) — the headline. + 5. The jointly-necessary society (E11) + mating structure (E14, one panel). + 6. Real-weight confirmation tier: neural/MNIST (brief) + multi-seed LLM headroom law. + Discussion: the borrowed/ours ledger, the design-rules table (average/route/select/don't-merge × + landscape), limits (what LLM-scale still lacks: the full grounded society), what biology gets + back (in-silico tests of sex theory in regimes biology can't reach). Methods: brief + SI. +- [ ] **Manifesto sections → Discussion or drop** (institutions, four timescales, re-minting + philosophy, fitness-is-not-truth): compress each to ≤1 paragraph or move to SI "extended + discussion". The accessible companion doc stays as-is for outreach, not submission. +- [ ] **Dual-audience devices**: Table 1 = the population-genetics ↔ machine-learning dictionary; + every term defined in one clause at first use; keep "the ML statement / the genetics statement" + paired-paragraph device, compressed. +- [ ] **Figures: consolidate ~25 per-experiment figures into 5 main multi-panel figures** + (1 concept/dictionary+operator schematic; 2 collapse&grounding incl. the MNIST digit-decay + montage — the one image both audiences get instantly; 3 sex: Fisher–Muller/outbreeding/directed + + merge-don't-average; 4 speciation across three tiers; 5 society ablation + headroom law). + Everything else → SI figures. All regenerated from committed parquet. +- [ ] **SI Appendix**: results-summary.md as the skeleton; full methods, all closed forms + + tolerances, per-experiment configs/seeds, the E13 floor proposition, confusion matrices, + reproduce.sh instructions. +- [ ] Word budget: main text ≤ ~6,000 words; check PNAS current LaTeX template + submission + checklist at writing time. + +## Phase 5 — Submission mechanics — Week 5 + +- [ ] **Code & data availability**: public GitHub snapshot + Zenodo DOI (code + committed results + artifacts); PNAS data-availability statement; verify `reproduce.sh` end-to-end on a clean clone. +- [ ] **Cover letter**: the fit argument (Evolvable-AI 2026 precedent; geneticist's-lens + contribution; what biology gains); position vs Riis/Shumailov explicitly. +- [ ] **Editor & reviewer suggestions**: identify the editor who handled "Evolvable AI" (PNAS + 2527700123; N.C. Stenseth per scan — verify); suggest 3–5 reviewers mixing (i) an + evolution-of-sex theorist, (ii) a model-merging ML researcher, (iii) a model-collapse author. +- [ ] **Policy checks at submission time** (verify, don't assume): PNAS AI-assistance disclosure + wording; preprint policy; OA/page charges + whether Imperial has a read-publish agreement with + PNAS; ORCID; competing interests. +- [ ] Sync arXiv v2 with the submitted text. + +## Decision items for GG (not blocking Phases 1–3) + +- [ ] Title: keep *"The Evolution of Sex for Artificial Intelligence"* vs a more declarative PNAS + title (e.g. *"A population genetics of model merging: why AI societies should reproduce + sexually"*). Decide at Phase 4. +- [ ] Authorship & AI-assistance acknowledgement wording (per PNAS policy). +- [ ] Repo public at submission vs at acceptance (Zenodo DOI needed at submission either way). +- [ ] APC/OA budget approval. + +## Discovered During Work + +*(append here)*