Giorgio Gilestro
a88289964a
Replaces the three-paragraph methods sketch with a scientific account of how
the study was run (M1-M7):
- M1 design principles: cheapest falsifying tier; match claim precision to
instrument precision; every tier gets an oracle independent of the model
being measured; falsifiers declared before running.
- M2 replication: what a replicate *is* differs by tier (independent lineage /
lineage incl. fresh init and data order / training seed with test sets held
fixed), and a table giving every experiment's replicate count with the
reasoning - why 200 for E4 (per-item binary outcomes), 60 for the bridge
gate (must detect any departure), 3-5 where the contrast is categorical,
and 1 for the 7B runs, labelled as single runs.
- M3-M5 per-tier procedures: parameter choices and their justification, the
correlated-parent construction, why the neural sandbox is synthetic (a
lossless identity code plus style entropy gives an exact oracle while still
forcing the model to learn a distribution), MNIST modes and the frozen-CNN
oracle with its confusion matrix as measurement floor, why no-BatchNorm MLPs
for the alignment analysis, and for the LLM tier: why Qwen 0.5B/7B (one
family so scale is the only variable), why procedural tasks rather than a
benchmark (exact verifier, contamination-free, controlled disjointness, a
difficulty knob), why LoRA (confines each parent to an additive low-rank
delta over an identical base, which is what makes weight-space
recombination well defined), the training algorithm, and the split scheme.
- M6 negative controls, including the one that removed a result: the
compatible-overlap axis collapsed the delta-cosine predictor from rho=+0.60
to +0.03.
- M7 statistical procedures.
Also: SI voice converted to first person and terminology synced to the
"biological model" rename; removed a process ghost from the preamble
("Skeleton assembled at Phase 4"); build.py now takes a document argument and
no longer eats documents that lack a title block, so the SI compiles via a new
si.tex wrapper (10 pp). `make paper` builds both PDFs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
82 lines
4.3 KiB
Makefile
82 lines
4.3 KiB
Makefile
# Layer 1 + Layer 1.5 automation. The uv venv (built from the committed uv.lock) is the
|
|
# reproducibility source of truth; every target runs inside it via `uv run`.
|
|
|
|
.PHONY: env env-neural env-mnist env-llm env-notebooks test layer1 layer2 neural mnist llm \
|
|
llm-epistasis llm-society figures paper-figures paper notebooks clean
|
|
|
|
env: ## build .venv from the committed lockfile
|
|
uv sync --extra dev
|
|
|
|
env-neural: ## add the Layer 1.5 torch stack (GPU; Stage C onward)
|
|
uv sync --extra dev --extra neural
|
|
|
|
env-mnist: ## add torchvision for the real-MNIST confirmation tier
|
|
uv sync --extra dev --extra neural --extra mnist
|
|
|
|
test: ## correctness tests + scientific-validation tests (the spine of trust)
|
|
uv run pytest
|
|
|
|
layer1: ## run experiments E1-E6 + the learning-kernel bridge (analytic)
|
|
for e in E1 E2 E3 E4 E5 E6 E7 E8 E9 E10 E11 E12 E12_nk E14 kernel_sharpen kernel_smooth; do uv run python -m knowledge.experiment configs/layer1/$$e.yaml; done
|
|
|
|
neural: ## run Layer 1.5 synthetic neural experiments (excludes the MNIST/torchvision tiers)
|
|
for c in configs/neural/*.yaml; do case "$$c" in *mnist*|*speciation_real*) ;; \
|
|
*) uv run python -m neural.experiment "$$c" ;; esac; done
|
|
|
|
mnist: ## run the torchvision tiers: MNIST collapse + E13 real-weight speciation (needs env-mnist)
|
|
uv run python -m neural.experiment configs/neural/mnist_collapse.yaml
|
|
uv run python -m neural.experiment configs/neural/speciation_real.yaml
|
|
uv run python -m neural.experiment configs/neural/speciation_real_cliff.yaml
|
|
uv run python -m neural.experiment configs/neural/speciation_real_emergent.yaml
|
|
MPLBACKEND=Agg uv run python figures/mnist_montage.py # the asset paper Fig. 2B embeds
|
|
|
|
env-llm: ## add the LLM stack for the Layer-2 prototype (GPU; transformers/peft)
|
|
uv sync --extra dev --extra neural --extra llm
|
|
|
|
llm: ## run the LLM prototypes: merge (fusion) + moe (union) + directed (offspring select)
|
|
uv run python -m llm.experiment configs/llm/merge.yaml
|
|
uv run python -m llm.experiment configs/llm/moe.yaml
|
|
uv run python -m llm.experiment configs/llm/directed.yaml
|
|
|
|
llm-speciation: ## LLM-tier speciation: conflict cliff (replace + de-confounded add) and duration null
|
|
uv run python -m llm.experiment configs/llm/speciation.yaml
|
|
uv run python -m llm.experiment configs/llm/speciation_add.yaml
|
|
|
|
llm-epistasis: ## the controlled predictive test (feeds paper Fig. 3C-D) + its robust statistics
|
|
uv run python -m llm.experiment configs/llm/epistasis.yaml
|
|
uv run python -m llm.experiment configs/llm/epistasis_compat.yaml
|
|
uv run python figures/stats_llm_epistasis.py
|
|
|
|
llm-society: ## the composed society at LLM scale (C3): pilot; the campaign runs on HPC
|
|
uv run python -m llm.experiment configs/llm/society.yaml
|
|
|
|
llm-seeds: ## multi-seed firm-up (heavy): merge x5, moe-hard x3, directed-hard x3
|
|
uv run python -m llm.experiment configs/llm/merge_seeds.yaml
|
|
uv run python -m llm.experiment configs/llm/moe_hard_seeds.yaml
|
|
uv run python -m llm.experiment configs/llm/directed_hard_seeds.yaml
|
|
|
|
layer2: neural ## alias: Layer 1.5 is the current Layer-2 deliverable (LLM rung deferred)
|
|
|
|
figures: ## regenerate per-experiment figures from committed results (pure; no re-simulation)
|
|
for e in E1 E2 E3 E4 E5 E6; do MPLBACKEND=Agg uv run python figures/plot_$$e.py; done
|
|
for p in figures/plot_*.py; do case "$$p" in */plot_E[1-6].py|*/_*) ;; \
|
|
*) [ -e "$$p" ] && MPLBACKEND=Agg uv run python "$$p" ;; esac; done
|
|
|
|
paper-figures: ## regenerate the manuscript figures (Fig. 1-7) and rebuild the PDF bodies
|
|
MPLBACKEND=Agg uv run python paper/pnas/make_figs.py
|
|
uv run python paper/pnas/build.py
|
|
uv run python paper/pnas/build.py si
|
|
|
|
paper: paper-figures ## figures + compile both PDFs (needs tectonic)
|
|
cd paper/pnas && tectonic main.tex && tectonic si.tex
|
|
|
|
env-notebooks: ## add Jupyter for the walkthrough notebooks
|
|
uv sync --extra dev --extra notebooks
|
|
|
|
notebooks: ## execute every notebook end-to-end (a reproduction check in itself)
|
|
for nb in notebooks/*.ipynb; do uv run jupyter nbconvert --to notebook --execute \
|
|
--inplace --ExecutePreprocessor.timeout=1800 "$$nb"; done
|
|
|
|
clean: ## remove caches and generated results (keeps committed manifests)
|
|
rm -rf .pytest_cache **/__pycache__
|
|
find results -type f ! -name '.gitkeep' -delete 2>/dev/null || true
|