giorgio/MachineSex
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Restructure: descriptive tier and experiment names, paper/manuscript

Browse source 
- paper/pnas -> paper/manuscript (venue-neutral)
- configs/layer1 -> configs/inheritance, src/knowledge -> src/inheritance
  (imported as `inheritance`), make layer1 -> make inheritance; layer2 alias dropped
- inheritance and trained-network bundles named after the manuscript figure
  they feed (fig2_grounding_sweep, figS3_rebaselining, ...), or descriptively
  where they feed none; configs keep their `experiment:` value so parquet
  hashes are unchanged, only output.dir moves
- figure scripts, SI figure sources, notebooks, REPRODUCING.md, README and the
  SI Methods/tables updated; make clean no longer deletes tracked manifests;
  reproduce.sh hashes the s{seed}/ layouts too

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
This commit is contained in:
Giorgio Gilestro 2026-09-13 17:00:40 +01:00
parent 84124de143
commit ab3dc10587
240 changed files with 477 additions and 476 deletions
Download patch file Download diff file Expand all files Collapse all files

51
REPRODUCING.md
View file

@ -5,7 +5,7 @@ that produced it. Every figure panel, every headline number, and the environment
in are listed below. If something in the paper is not traceable through this document, that is a
bug — please open an issue.
Manuscript: `paper/pnas/main.md` (built to `paper/pnas/main.pdf`).
Manuscript: `paper/manuscript/main.md` (built to `paper/manuscript/main.pdf`).
---
@ -50,7 +50,7 @@ Tier by tier, by hand:
```bash
make test # correctness + closed-form scientific validation
make layer1 # the inheritance model: E1-E12, E14, learning kernel
make inheritance # the inheritance model, every experiment at its committed seed
make neural # trained networks (needs a GPU)
make mnist # real-MNIST tier + the Fig. 2A montage asset (needs torchvision)
make llm # language-model prototypes (needs a GPU)
@ -64,16 +64,16 @@ make paper-figures # the manuscript figures Fig. 1-5 + rebuild the PDF body
## 4. The figure map
Every manuscript panel, the artifact it is plotted from, the config that produced that artifact, and
that config's declared seed. All panels are drawn by `paper/pnas/make_figs.py` (function per figure);
that config's declared seed. All panels are drawn by `paper/manuscript/make_figs.py` (function per figure);
`make paper-figures` regenerates all of them. Figures are a **pure function of committed artifacts** —
no panel re-simulates anything, with the single documented exception of the Fig. 2A montage asset.
| Panel | Drawn by | From artifact | Produced by config | Seed(s) |
|---|---|---|---|---|
| Fig. 1A, 1B | `fig1a()`, `fig1b()` | — (schematics; icons in `paper/pnas/figs/icons/`) | — | — |
| Fig. 2A | `fig2()` | `results/mnist_collapse/mnist_montage.png` | `configs/neural/mnist_collapse.yaml` → asset from `figures/mnist_montage.py` | 20260705 |
| Fig. 2B *(reference)* | `fig2()` | `results/E2/` | `configs/layer1/E2.yaml` | 20260704 |
| Fig. 3A | `fig3()` | — (schematic of the compared models; icons in `paper/pnas/figs/icons/`) | — | — |
| Fig. 1A, 1B | `fig1a()`, `fig1b()` | — (schematics; icons in `paper/manuscript/figs/icons/`) | — | — |
| Fig. 2A | `fig2()` | `results/fig2_mnist_collapse/mnist_montage.png` | `configs/neural/fig2_mnist_collapse.yaml` → asset from `figures/mnist_montage.py` | 20260705 |
| Fig. 2B *(reference)* | `fig2()` | `results/fig2_grounding_sweep/` | `configs/inheritance/fig2_grounding_sweep.yaml` | 20260704 |
| Fig. 3A | `fig3()` | — (schematic of the compared models; icons in `paper/manuscript/figs/icons/`) | — | — |
| Fig. 3B | `fig3()` | `results/llm_merge_seeds/` | `configs/llm/merge_seeds.yaml` | 1, 2, 3, 4, 5 |
| Fig. 3C | `fig3()` | `results/llm_moe_hard_hpc/s{1,2,3}/` | `configs/llm/moe_hard_hpc.yaml` (7B, HPC; seeds 2–3 via `hpc/llm_7b_seeds.pbs`); statistics `figures/stats_llm_7b_seeds.py` | 1, 2, 3 |
| Fig. 3D, 3D | `fig3()` | `results/llm_epistasis/` + `results/llm_epistasis_compat/` | `configs/llm/epistasis.yaml`, `configs/llm/epistasis_compat.yaml` | 1, 2, 3 |
@ -83,20 +83,26 @@ no panel re-simulates anything, with the single documented exception of the Fig.
| Fig. S14 | `figures/plot_curriculum_timing.py` | `results/llm_curriculum_v5_{early,late,early_obl,late_obl}/s{1,2,3}/` | `configs/llm/curriculum_v5_{early,late}[_obl].yaml` via `hpc/llm_curriculum_timing.pbs`; statistics `figures/stats_llm_curriculum.py` | 1, 2, 3 |
| Fig. S15 | `figures/plot_curriculum_cull.py` | `results/llm_curriculum_v5_cull/s{1,2,3}/` | `configs/llm/curriculum_v5_cull.yaml` via `hpc/llm_cull.pbs`; statistics `figures/stats_llm_curriculum.py` | 1, 2, 3 |
| Fig. S16 | `figures/plot_llm_smol.py` | `results/llm_{merge_seeds,moe_hard_seeds}_smol/` (+ the Qwen bundles) | `configs/llm/{merge_seeds,moe_hard_seeds}_smol.yaml` via `hpc/llm_smol.pbs`; statistics `figures/stats_llm_smol.py` | 1–5; 1–3 |
| Fig. 4D, 4D, 4E *(reference)* | `fig4()` | `results/E11/` | `configs/layer1/E11.yaml` | 20260705 |
| Fig. 4D, 4D, 4E *(reference)* | `fig4()` | `results/fig4_society_ablation/` | `configs/inheritance/fig4_society_ablation.yaml` | 20260705 |
| Fig. 5A | `fig5()` | `results/speciation_real/` | `configs/neural/speciation_real.yaml` | 13 |
| Fig. 5B | `fig5()` | `results/speciation_real_cliff/` | `configs/neural/speciation_real_cliff.yaml` | 13 |
| Fig. 5C, 5D | `fig5()` | `results/llm_speciation/s{1,2,3}/` | `configs/llm/speciation.yaml` (seeds 2–3 via `hpc/llm_speciation_seeds.pbs`); statistics `figures/stats_llm_speciation_seeds.py` | 1, 2, 3 |
| Fig. 5E, 5F *(reference)* | `fig5()` | `results/E12/` | `configs/layer1/E12.yaml` | 12 |
| Fig. 5E, 5F *(reference)* | `fig5()` | `results/fig5_speciation_bdm/` | `configs/inheritance/fig5_speciation_bdm.yaml` | 12 |
Panels marked *(reference)* are inheritance-model results included to set the expectation the
real-model panels are read against, not as findings in their own right.
**Inheritance-model results reported in SI only.** These have no real-model counterpart in the paper
and reproduce known results, so they are cited as reference values and plotted only in SI, from their
per-experiment figures: blending vs union (E4), Fisher–Muller super-parent (E8), outbreeding
depression (E9), directed recombination (E10), and mate-pool breadth (E14), drawn by
`figures/plot_E{4,8,9,10,14}.py` from `results/E{4,8,9,10,14}/`.
per-experiment figures: blending vs union (`figS8_multiparent_union`), Fisher–Muller super-parent
(`figS9_specialist_superparent`), outbreeding depression (`figS10_rugged_landscapes`), directed
recombination (`figS11_directed_recombination`), and mate-pool breadth (`figS13_mating_breadth`),
each drawn by `figures/plot_<bundle>.py` from `results/<bundle>/`.
**Bundle names.** Inheritance-model and trained-network bundles are named after the manuscript
figure they feed at submission (`fig2_*`, `figS4_*`); bundles that feed no figure carry a
descriptive name. The name is a label fixed at submission, so a later renumbering of the figures
does not rename the bundles.
**Replication.** Every panel is replicated: inheritance-model panels over 12–100 internal
replicates; Fig. 3B over five training seeds; Fig. 3C, 3D–E, 4A–B and 5C–D over three (Fig. 5C–D
@ -106,18 +112,15 @@ was single-seed until 2026-09-12; seeds 2–3 ran via `hpc/llm_speciation_seeds.
| Result | Artifact | Config | Seed |
|---|---|---|---|
| Collapse null (E1) | `results/E1/` | `configs/layer1/E1.yaml` | 20260704 |
| Region-matched grounding (E3) | `results/E3/` | `configs/layer1/E3.yaml` | 20260704 |
| Quality-diversity vs greedy (E5) | `results/E5/` | `configs/layer1/E5.yaml` | 20260704 |
| Re-minting / irreversibility (E6) | `results/E6/` | `configs/layer1/E6.yaml` | 20260704 |
| Advantage of sex, lineage (E7) | `results/E7/` | `configs/layer1/E7.yaml` | 20260705 |
| Incompatibilities on NK (E12_nk) | `results/E12_nk/` | `configs/layer1/E12_nk.yaml` | 12 |
| Learning kernel (estimator bias) | `results/kernel_sharpen/`, `results/kernel_smooth/` | `configs/layer1/kernel_{sharpen,smooth}.yaml` | 20260705 |
| Region-matched grounding (Fig. S5) | `results/figS5_aimed_grounding/` | `configs/inheritance/figS5_aimed_grounding.yaml` | 20260704 |
| Quality-diversity vs greedy (Fig. S12) | `results/figS12_quality_diversity/` | `configs/inheritance/figS12_quality_diversity.yaml` | 20260704 |
| Re-baselining / irreversibility (Fig. S3) | `results/figS3_rebaselining/` | `configs/inheritance/figS3_rebaselining.yaml` | 20260704 |
| Learning kernel (Fig. S2) | `results/figS2_kernel_sharpen/`, `results/figS2_kernel_smooth/` | `configs/inheritance/figS2_kernel_{sharpen,smooth}.yaml` | 20260705 |
| Histogram bridge gate | `results/bridge/` | `configs/neural/bridge.yaml` | 20260704 |
| Neural collapse / grounding / architectures / recombination | `results/{collapse,grounding,architectures,recombination}/` | `configs/neural/*.yaml` | 20260704 |
| Neural collapse / grounding / architectures / recombination | `results/{collapse,figS6_grounding_rnn,figS1_architectures,recombination}/` | `configs/neural/*.yaml` | 20260704 |
| Emergent-isolation null | `results/speciation_real_emergent/` | `configs/neural/speciation_real_emergent.yaml` | 813 |
| Budget-controlled speciation (add design) | `results/llm_speciation_add/` | `configs/llm/speciation_add.yaml` | 1, 2, 3 |
| LLM prototypes (0.5B) | `results/llm_{merge,moe,directed}/` | `configs/llm/{merge,moe,directed}.yaml` | 1 |
| 0.5B multi-seed runs | `results/llm_{merge_seeds,moe_hard_seeds,directed_hard_seeds}/` | `configs/llm/{merge_seeds,moe_hard_seeds,directed_hard_seeds}.yaml` | 1–5; 1–3; 1–3 |
| 7B firm-ups | `results/llm_*_hpc/` | `configs/llm/*_hpc.yaml` (run via `hpc/*.pbs`) | 1 |
### Per-experiment (exploratory) figures
@ -132,7 +135,7 @@ sensitivity).
## 5. Seeds and determinism
**Policy.** One master seed per config. All sub-randomness is derived from it via
`numpy.random.SeedSequence.spawn` (`src/knowledge/seeding.py`); no code touches global RNG state, and
`numpy.random.SeedSequence.spawn` (`src/inheritance/seeding.py`); no code touches global RNG state, and
every `rng` is passed explicitly. A run is a pure function of its resolved config.
**Biological-model tier: bitwise reproducible.** Re-running a config on the same lockfile
@ -185,7 +188,7 @@ same `uv.lock` environment, so the only difference from a local run is the GPU.
```bash
qsub hpc/llm_merge.pbs # 7B merge firm-up
qsub hpc/llm_hard.pbs # hard-benchmark moe + directed at 7B
qsub hpc/llm_society.pbs # the society campaign (array over seeds)
qsub hpc/llm_7b_seeds.pbs # seeds 2-3 of the three 7B experiments (array)
```
## 8. Notebooks
@ -206,6 +209,6 @@ qsub hpc/llm_society.pbs # the society campaign (array over seeds)
regenerated. The archived deposit (Zenodo DOI, on publication) includes the parquets so that the
paper's "regenerates from committed artifacts without re-simulation" holds from the archive.
- `figures/mnist_montage.py` re-runs a short dry lineage to draw its montage rather than reading a
parquet; it is an eyeball diagnostic whose quantitative counterpart is `results/mnist_collapse/`.
parquet; it is an eyeball diagnostic whose quantitative counterpart is `results/fig2_mnist_collapse/`.
- The composed society at language-model scale is an open experiment at the time of writing; see
`tasks/workorder-llm-society.md`.