Giorgio Gilestro
c435cfba6e
An audit of the figure pipeline found real sync gaps, now closed:
- `paper/pnas/make_figs.py` (which draws every manuscript figure) was invoked
by NO Makefile target or script - a manual step. Added `make paper-figures`.
- `configs/llm/epistasis{,_compat}.yaml` were reachable from nothing at all,
despite producing Fig. 3C-D. Added `make llm-epistasis` (+ its statistics).
- `make figures` never regenerated the MNIST montage that Fig. 2B embeds;
it now runs with the `mnist` target (it needs torch - it re-simulates).
- Added `make llm-society`, `env-notebooks`, `notebooks`.
New REPRODUCING.md is the authoritative map: every manuscript panel -> the
artifact it plots -> the config that produced it -> that config's seed, plus
the determinism policy (biological tier bitwise; GPU tiers statistical), the
seed-provenance statement, and an artifact-hash verification snippet. All 44
committed bundles currently hash-match their manifests, and figure
regeneration is pixel-identical (verified by comparison).
reproduce.sh delivers the one-command reproduction the paper's Methods
promises, writing REPRODUCED.md with recomputed hashes per bundle.
Two executed notebooks: 01 builds the Wright-Fisher model from scratch and
checks both closed forms interactively (runs in ~1 min on a laptop); 02
verifies artifact hashes then regenerates and displays all seven manuscript
figures. Both execute end-to-end (`make notebooks`).
Also pins `.python-version` to 3.14: the interpreter was previously
unpinned, and a `uv sync` silently switched it to 3.11 mid-session (see
tasks/lessons.md). README rewritten - it still described a Layer-1-only repo
of E1-E6 and pointed at a figure_manifest.md that does not exist.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
52 lines
4.1 KiB
Markdown
52 lines
4.1 KiB
Markdown
|
|
**2026-08-11 — Stop padding time estimates.** GG: "You generally massively overestimate the time it
|
|
takes to do some work... You're fast." Phase-1 items I scoped as "Week 1" took ~an hour. Rule: state
|
|
what will be done and in what order; give a duration only when compute-bound (training walltime), and
|
|
base it on measured runtimes, not human-project heuristics.
|
|
|
|
**2026-08-11 — Correspondence is not identity; interpretation is not prediction (manuscript reviews).**
|
|
The external review's core corrections, to internalise for all paper claims: (1) distinguish
|
|
interpretation / explanation / prediction and claim only the level the evidence supports; (2) a
|
|
minimal model being exactly Wright-Fisher does not make real training "literally" WF — our own
|
|
learning-kernel result says otherwise (cite it against ourselves); (3) name the operator every claim
|
|
is about (output-mean vs weight-average vs max-with-oracle vs routing are different objects with
|
|
different budgets); (4) don't write "nobody has / none imports" — invite no priority disputes; say
|
|
"to our knowledge" and state the positive contribution; (5) negative results (E13b) are strengths —
|
|
lead with them; (6) "control theory" needs states/controls/dynamics/rule or it's a "framework".
|
|
|
|
**2026-08-11 — tmux does NOT survive SSH disconnects on this machine.** The tmux server starts inside
|
|
the SSH session's systemd scope and gets reaped on logout (lost ~20 min of the epistasis grid; GG:
|
|
"SSH disconnected"). Reliable pattern here: `systemd-run --user --collect --unit=<name>
|
|
--working-directory="$PWD" bash -c '<cmd>'` — lands in user@.service (kept alive by the desktop
|
|
session), survives disconnects; check with `systemctl --user is-active <name>`, logs via redirect.
|
|
|
|
**2026-08-11 — Internal deliberations must not leak into reader-facing prose.** GG (on the
|
|
convergence paragraph): it "carries all the ghosts of our internal brainstorming that should be
|
|
completely alien to the readers (this is, alas, a recurring problem with your writing)." The pattern:
|
|
strategic concerns from the working process (priority anxieties -> "we cite for priority of
|
|
publication"; calibration debates -> "the honest statement"; positioning battles -> "nobody has")
|
|
surface as defensive or self-referential prose. Rule: before finalising any reader-facing passage,
|
|
ask "does understanding this sentence require knowing how we worked?" If yes, rewrite it as a plain
|
|
statement about the literature or the result. Confident papers situate; they do not litigate.
|
|
(Promoted to a general rule in the global ~/.claude/CLAUDE.md, 2026-08-11 — it applies to all
|
|
reader-facing prose in every project.)
|
|
|
|
## Terminology: "model" is overloaded in this project (2026-09-07)
|
|
In the PNAS manuscript and figures, "model" must mean an AI model. The settled term for the
|
|
pop-gen construct is **"the biological model"** (GG decision, 2026-09-07) — used everywhere
|
|
("the multi-locus biological model", "Biological-model tier"), never "minimal model",
|
|
"analytic model", or "exact model"; GG also vetoed "(exact)" as a tier label (the guarantee
|
|
line "closed forms · bitwise-reproducible" carries that content). Support level formerly
|
|
"Exact" is now "Closed form". Keep "exact" only in technical noun phrases (exact-match
|
|
verifier, exact oracle, exact equilibrium, exact recovery).
|
|
|
|
## Never `uv sync` while a job is using the venv (2026-09-07)
|
|
Adding the `notebooks` extra mid-session ran `uv sync` **without** `--extra llm`, which rebuilt the
|
|
shared `.venv` — dropping the LLM stack *and* silently switching the interpreter 3.14 → 3.11 (uv
|
|
recreates with the system default when no `.python-version` is pinned). That killed the running
|
|
local society seed with a `FileNotFoundError` deep in `huggingface_hub` templates — a failure that
|
|
looked scientific but was pure environment churn.
|
|
Rules: (1) never mutate `.venv` while a background job is running against it — wait, or build a
|
|
throwaway venv elsewhere; (2) `uv sync` is *declarative* — always pass **every** extra the project
|
|
needs, or it removes the ones you omit; (3) the repo now pins `.python-version` (3.14) so the
|
|
interpreter can never drift silently.
|