Reproducibility pass: figure map, one-command reproduce.sh, notebooks, Makefile gaps

An audit of the figure pipeline found real sync gaps, now closed:

- `paper/pnas/make_figs.py` (which draws every manuscript figure) was invoked
  by NO Makefile target or script - a manual step. Added `make paper-figures`.
- `configs/llm/epistasis{,_compat}.yaml` were reachable from nothing at all,
  despite producing Fig. 3C-D. Added `make llm-epistasis` (+ its statistics).
- `make figures` never regenerated the MNIST montage that Fig. 2B embeds;
  it now runs with the `mnist` target (it needs torch - it re-simulates).
- Added `make llm-society`, `env-notebooks`, `notebooks`.

New REPRODUCING.md is the authoritative map: every manuscript panel -> the
artifact it plots -> the config that produced it -> that config's seed, plus
the determinism policy (biological tier bitwise; GPU tiers statistical), the
seed-provenance statement, and an artifact-hash verification snippet. All 44
committed bundles currently hash-match their manifests, and figure
regeneration is pixel-identical (verified by comparison).

reproduce.sh delivers the one-command reproduction the paper's Methods
promises, writing REPRODUCED.md with recomputed hashes per bundle.

Two executed notebooks: 01 builds the Wright-Fisher model from scratch and
checks both closed forms interactively (runs in ~1 min on a laptop); 02
verifies artifact hashes then regenerates and displays all seven manuscript
figures. Both execute end-to-end (`make notebooks`).

Also pins `.python-version` to 3.14: the interpreter was previously
unpinned, and a `uv sync` silently switched it to 3.11 mid-session (see
tasks/lessons.md). README rewritten - it still described a Layer-1-only repo
of E1-E6 and pointed at a figure_manifest.md that does not exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
Giorgio Gilestro 2026-09-07 15:50:50 +01:00
parent 8086504f44
commit c435cfba6e
18 changed files with 4204 additions and 28 deletions

View file

@ -39,3 +39,14 @@ pop-gen construct is **"the biological model"** (GG decision, 2026-09-07) — us
line "closed forms · bitwise-reproducible" carries that content). Support level formerly
"Exact" is now "Closed form". Keep "exact" only in technical noun phrases (exact-match
verifier, exact oracle, exact equilibrium, exact recovery).
## Never `uv sync` while a job is using the venv (2026-09-07)
Adding the `notebooks` extra mid-session ran `uv sync` **without** `--extra llm`, which rebuilt the
shared `.venv` — dropping the LLM stack *and* silently switching the interpreter 3.14 → 3.11 (uv
recreates with the system default when no `.python-version` is pinned). That killed the running
local society seed with a `FileNotFoundError` deep in `huggingface_hub` templates — a failure that
looked scientific but was pure environment churn.
Rules: (1) never mutate `.venv` while a background job is running against it — wait, or build a
throwaway venv elsewhere; (2) `uv sync` is *declarative* — always pass **every** extra the project
needs, or it removes the ones you omit; (3) the repo now pins `.python-version` (3.14) so the
interpreter can never drift silently.