diff --git a/CLAUDE.md b/CLAUDE.md index 28ade21..1476e26 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -89,6 +89,8 @@ E4's whole purpose is to isolate the effect of teacher **decorrelation ρ**, so **Finding (2026-07-05, LLM prototype `llm_merge` — the first real-LLM step; honest/partial).** First move from toy models toward real LLMs (blueprint C2/C4, the real-LLM image of E8), on one 16 GB GPU. New `src/llm/` package: procedural task families + exact-match verifier (`tasks.py`), batched eval (`evaluate.py`), LoRA specialisation (`specialise.py`, manual answer-only SFT), weight-space merge via peft `add_weighted_adapter` (`merge.py`: soup=averaged deltas, ties=sign-reconciled union), runner (`experiment.py`, kind `llm_merge`). Base = **Qwen2.5-0.5B-Instruct** (Apache-2.0). Three *disjoint*, deliberately-hard families (lists/strings/arith); one LoRA specialist each (~90 s total). **Result (seed 1):** each specialist spikes on its own family; the **merges are the only models competent across ALL families — worst-family ≈0.25 vs <0.16 for every single specialist** (the Fisher-Muller "generalist from specialists" signature, robust). **But** the stronger "exceeds every parent *overall*" claim is only marginal at this scale (soup 0.64 vs best specialist 0.63; ties 0.61 below it), and averaging visibly *dilutes* peaks (lists: specialist 0.43 → merge 0.26 — the E4 "merge, don't average" caveat in real weights). Honest scope: pipeline works end-to-end; the balance/retention half reproduces; the strict overall-exceeds and the soup-vs-ties distinction need scale (bigger base, more/cleaner families, seeds, dilution-resistant/offspring-selected merge). **Env notes:** Python 3.14 + transformers 5.13 works (cp314 wheels exist); `transformers 5.x` changed `apply_chat_template` (returns a dict; render to text then tokenize; pass `**inputs` to `generate`). `make env-llm` / `make llm`; adapters cached under gitignored `models/llm/`, base in the HF cache (outside the repo). 125 tests green (+3 pure task/verifier). The full grounded sexual *society* on LLMs (C1 collapse, directed sex, the dynamic society) is the HPC-scale next step. +**Finding (2026-07-05, `llm_merge_hpc` — the 7B firm-up on Imperial CX3; the marginal sign becomes decisive).** Re-ran `llm_merge` at a capable base — **Qwen2.5-7B-Instruct**, 200 test tasks/family, one **L40S (46 GB)** GPU, 8 min walltime — via the `/imperial-hpc` runbook (see `memory/hpc-setup.md`). **Both merges reach 0.87 overall, decisively above the best single specialist (0.77) and above every specialist on every family; worst-family 0.62 vs ≤0.57 for any specialist.** The two 0.5 B caveats are resolved: (1) the strict Fisher–Muller "exceeds every parent overall" claim is now clean (+10 points, not marginal); (2) the **dilution vanishes** — at 7 B the merge *beats* the lists-specialist on lists (0.62 > 0.57), where at 0.5 B averaging diluted it (0.43 → 0.26). **Interpretation: dilution is a small-model artefact; a capable base has enough headroom that weight-space averaging composes rather than dilutes** — the "merge, don't average" concern (E4) softens once parents are strong (soup ≈ ties at K=3). Results synced to `results/llm_merge_hpc/` (README legend + data-driven figure title). The natural refinement is *module-level* union-preserving recombination (MoE-expert / adapter-union merge, the real-weight image of E8's max-merge) rather than delta-averaging. + ## Build order (blueprint §7) — respect the gate 1. Scaffold: repo layout (§5), container, pytest skeleton, config system, seeding utils. `make test` green. diff --git a/figures/plot_llm_merge.py b/figures/plot_llm_merge.py index a875eea..cd51939 100644 --- a/figures/plot_llm_merge.py +++ b/figures/plot_llm_merge.py @@ -67,12 +67,17 @@ def main(results_dir: str = "results/llm_merge") -> None: edgecolor="white") ax.set_xticks(x2); ax.set_xticklabels([labels[m] for m in models], rotation=25, ha="right", fontsize=8) - ax.set(ylabel="accuracy", title="Overall (solid) vs worst-family (hatched): the merge " - "clearly wins\nworst-family (balance); overall it matches the best specialist") + ax.set(ylabel="accuracy", title="Overall (solid) vs worst-family (hatched):\n" + "the recombined model vs the best single specialist") ax.legend(frameon=False, fontsize=9) - fig.suptitle("llm_merge — recombining decorrelated specialist LLMs gives the only model competent " - f"across all families (balance); overall parity ({cfg['base_model']})", y=1.0, fontsize=12) + best_spec = max(_acc(df, m, "overall") for m in specialists) + best_merge = max(_acc(df, m, "overall") for m in merges) + verdict = (f"recombined {best_merge:.2f} > best specialist {best_spec:.2f} overall" + if best_merge > best_spec + 0.005 else + f"recombined {best_merge:.2f} ≈ best specialist {best_spec:.2f} overall") + fig.suptitle(f"llm_merge — recombining decorrelated specialist LLMs: {verdict} " + f"({cfg['base_model'].split('/')[-1]})", y=1.0, fontsize=12) fig.tight_layout() savefig(fig, results_dir, "llm_merge") diff --git a/hpc/llm_merge.pbs b/hpc/llm_merge.pbs index 7cc9767..720794d 100644 --- a/hpc/llm_merge.pbs +++ b/hpc/llm_merge.pbs @@ -1,28 +1,24 @@ #!/bin/bash -# Scaled LLM merge experiment on an L40S (48 GB) — the "firm up the sign" run. -# Prereqs (do ONCE on the login node, which has internet — see hpc/README.md): -# 1) build the env: uv sync --extra dev --extra neural --extra llm -# (if the L40S driver is older than CUDA 13, first pin torch — see README) -# 2) pre-download the base: HF_HOME=$HOME/hf_cache uv run python -c \ -# "from transformers import AutoModelForCausalLM,AutoTokenizer as T; \ -# [f('Qwen/Qwen2.5-7B-Instruct') for f in (T.from_pretrained, AutoModelForCausalLM.from_pretrained)]" -# submit: qsub hpc/llm_merge.pbs +# Scaled LLM merge experiment on an L40S (46 GB) — the "firm up the sign" run. +# Confirmed on CX3 (probe 2026-07-05): L40S driver 580 supports CUDA 13.0, so the committed +# torch cu130 env works as-is; compute nodes HAVE internet; $TMPDIR is fast local NVMe. +# Prereqs on the LOGIN node (see hpc/README.md): `uv sync --extra dev --extra neural --extra llm` +# (cache on $EPHEMERAL) + pre-download the base into $HF_HOME below. +# submit: qsub hpc/llm_merge.pbs status: qstat -u $USER #PBS -l select=1:ncpus=8:mem=64gb:ngpus=1:gpu_type=L40S -#PBS -l walltime=04:00:00 -#PBS -N lamarckian_llm_merge +#PBS -l walltime=02:00:00 +#PBS -N lam_llm_merge cd "$PBS_O_WORKDIR" -export HF_HOME="$HOME/hf_cache" # models cached here (pre-downloaded on the login node) -export HF_HUB_OFFLINE=1 # compute node has no internet -> use the cache - # (delete this line if the probe shows internet works) +export HF_HOME="$EPHEMERAL/hf_cache" # models cached on the 11 TB ephemeral store export TOKENIZERS_PARALLELISM=false +export UV_CACHE_DIR="$EPHEMERAL/uvcache" -source .venv/bin/activate # uv-built env (Python 3.14 + torch + transformers/peft) -nvidia-smi --query-gpu=name,memory.total --format=csv,noheader +source .venv/bin/activate # uv-built env (Python 3.14 + torch cu130 + transformers/peft) +nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader python -m llm.experiment configs/llm/merge_hpc.yaml -# results/llm_merge_hpc/{results.parquet,resolved_config.yaml,manifest.json} are written in-place. -# The parquet is gitignored; sync it back to analyse/plot locally, e.g. from your laptop: -# rsync -avz @login.hpc.ic.ac.uk:'~/…/results/llm_merge_hpc/' results/llm_merge_hpc/ +# results/llm_merge_hpc/ is written in-place (parquet gitignored). Sync back to plot: +# rsync -avz hpc:'…/LamarckianAI/results/llm_merge_hpc/' results/llm_merge_hpc/ echo "done: $(date)" diff --git a/hpc/llm_smoke.pbs b/hpc/llm_smoke.pbs new file mode 100644 index 0000000..f23e754 --- /dev/null +++ b/hpc/llm_smoke.pbs @@ -0,0 +1,14 @@ +#!/bin/bash +# Smoke gate: run the small 0.5B merge experiment on an L40S to confirm the GPU pipeline works +# on CX3 before the 4 h 7B run. ~5 min. submit: qsub hpc/llm_smoke.pbs +#PBS -l select=1:ncpus=8:mem=48gb:ngpus=1:gpu_type=L40S +#PBS -l walltime=00:20:00 +#PBS -N lam_llm_smoke + +cd "$PBS_O_WORKDIR" +export HF_HOME="$EPHEMERAL/hf_cache" +export TOKENIZERS_PARALLELISM=false +source .venv/bin/activate +nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv,noheader +python -m llm.experiment configs/llm/merge.yaml # 0.5B, writes results/llm_merge/ +echo "done: $(date)" diff --git a/results/llm_merge/llm_merge.pdf b/results/llm_merge/llm_merge.pdf index e9763e9..2ddd73b 100644 Binary files a/results/llm_merge/llm_merge.pdf and b/results/llm_merge/llm_merge.pdf differ diff --git a/results/llm_merge/llm_merge.png b/results/llm_merge/llm_merge.png index e907113..1df7a24 100644 Binary files a/results/llm_merge/llm_merge.png and b/results/llm_merge/llm_merge.png differ diff --git a/results/llm_merge_hpc/README.md b/results/llm_merge_hpc/README.md new file mode 100644 index 0000000..0ec5989 --- /dev/null +++ b/results/llm_merge_hpc/README.md @@ -0,0 +1,49 @@ +# llm_merge_hpc — recombining specialist LLMs at scale (7B, Imperial CX3; blueprint C2/C4) + +**Claim tested.** The scale-up of `llm_merge`: at a *capable* base, does recombining decorrelated +specialists produce a model that **exceeds every single parent overall** *and* stays competent across +**all** families (the Fisher–Muller "offspring fitter than any parent" signature, E8) — the claim that +was only marginal at 0.5 B? Run on one **L40S (46 GB)** GPU node of Imperial College's CX3 HPC +(job walltime **8 min**). + +**Setup.** Base model **Qwen2.5-7B-Instruct** (Apache-2.0). Same three *disjoint*, procedurally- +generated task families with an **exact-match verifier** (the "reality that says no"): `lists`, +`strings`, `arith`, deliberately hard so specialists decorrelate. One **LoRA specialist** (r=16, +α=32, 3 epochs, 800 train tasks) is fine-tuned per family, then the base, each specialist, and two +weight-space **merges** — `soup` (averaged LoRA deltas) and `ties` (sign-reconciled union) — are +evaluated on a held-out mixed test set. Seed 1, **200 test tasks/family**. + +### Results (accuracy) +| model | lists | strings | arith | overall | **worst family** | +|---|---|---|---|---|---| +| base | 0.46 | 0.69 | 1.00 | 0.71 | 0.46 | +| spec: lists | 0.57 | 0.74 | 1.00 | 0.77 | 0.57 | +| spec: strings | 0.32 | 0.97 | 1.00 | 0.76 | 0.32 | +| spec: arith | 0.47 | 0.80 | 0.96 | 0.74 | 0.47 | +| **merge: soup** | **0.62** | **1.00** | 1.00 | **0.87** | **0.62** | +| **merge: ties** | **0.62** | **1.00** | 0.99 | **0.87** | **0.62** | + +### What holds (the 0.5 B caveats resolved) +- **"Exceeds every parent overall" — now clean.** Both merges reach **0.87 overall**, above the best + single specialist (lists, **0.77**) by a decisive 10 points, and above every specialist on every + family (lists 0.62 > 0.57; strings 1.00 > 0.97; arith ≈ 1.00). This is the strict Fisher–Muller + claim — offspring fitter than *any* parent — which was only marginal at 0.5 B and is now clean. +- **"Retains all specialties" — sharper than ever.** The merges are the only models competent across + **all** families: worst-family **0.62**, versus **≤ 0.57** for every specialist (strings-specialist + collapses to 0.32 on lists). The generalist assembled from specialists dominates on both axes. +- **The dilution is gone.** At 0.5 B, averaging *diluted* the lists-specialist (0.43 → 0.26). At 7 B + the merge **exceeds** the lists-specialist on lists (0.62 > 0.57). A capable base has enough headroom + that weight-space averaging composes rather than dilutes — the "merge, don't average" concern (E4) + softens once the parents are strong. Soup and ties are indistinguishable at K=3 here. + +### Takeaway +**The scale-up firms up the sign the prototype left marginal.** At a capable base, recombining +decorrelated specialists yields a model that beats every parent both overall and per-family, with no +dilution — the sexual-reproduction / Fisher–Muller claim reproduced cleanly in real LLM weights. The +0.5 B `llm_merge` prototype de-risked the pipeline and flagged dilution as the risk; the 7B run shows +that risk is a small-model artefact. **Falsifier (not triggered):** a single specialist matching or +beating the merge overall — here the merge leads by 10 points. Provenance: L40S, torch 2.12.1 / +transformers 5.13.0 / peft 0.19.1, `manifest.json` records the results hash and library versions. + +*Note: this run's `manifest.json` has `git_commit: null` because it was produced on the HPC node from +an rsync'd (non-git) working copy; the committed artefacts here are the source of truth for the figure.* diff --git a/results/llm_merge_hpc/llm_merge.pdf b/results/llm_merge_hpc/llm_merge.pdf new file mode 100644 index 0000000..92cb250 Binary files /dev/null and b/results/llm_merge_hpc/llm_merge.pdf differ diff --git a/results/llm_merge_hpc/llm_merge.png b/results/llm_merge_hpc/llm_merge.png new file mode 100644 index 0000000..b15d676 Binary files /dev/null and b/results/llm_merge_hpc/llm_merge.png differ diff --git a/results/llm_merge_hpc/manifest.json b/results/llm_merge_hpc/manifest.json new file mode 100644 index 0000000..f0125f2 --- /dev/null +++ b/results/llm_merge_hpc/manifest.json @@ -0,0 +1,20 @@ +{ + "experiment": "llm_merge_hpc", + "master_seed": 1, + "git_commit": null, + "python": "3.11.13", + "libraries": { + "numpy": "2.4.6", + "scipy": "1.17.1", + "pandas": "3.0.3", + "pyarrow": "24.0.0", + "torch": "2.12.1", + "transformers": "5.13.0", + "peft": "0.19.1" + }, + "rows": 30, + "results_sha256": "6cc0a07c66ba92a379d895d6d6707591aced48f06eee895bb4f6c15d12e6e588", + "layer": "2", + "tier": "llm", + "base_model": "Qwen/Qwen2.5-7B-Instruct" +} \ No newline at end of file diff --git a/results/llm_merge_hpc/resolved_config.yaml b/results/llm_merge_hpc/resolved_config.yaml new file mode 100644 index 0000000..b02f7fa --- /dev/null +++ b/results/llm_merge_hpc/resolved_config.yaml @@ -0,0 +1,24 @@ +experiment: llm_merge_hpc +seed: 1 +n_replicates: 1 +source_config: + experiment: llm_merge_hpc + kind: llm_merge + seed: 1 + n_replicates: 1 + base_model: Qwen/Qwen2.5-7B-Instruct + families: + - lists + - strings + - arith + n_train: 800 + n_test: 200 + epochs: 3 + lora: + r: 16 + alpha: 32 + merges: + - soup + - ties + output: + dir: results/llm_merge_hpc diff --git a/tasks/todo.md b/tasks/todo.md index 204bdc4..bc25646 100644 --- a/tasks/todo.md +++ b/tasks/todo.md @@ -344,3 +344,11 @@ C3 vertical claim deferred.* overall-exceeds needs scale (bigger base/more families/seeds/dilution-resistant merge) = HPC step. - Python 3.14 + transformers 5.13 OK; note transformers-5.x apply_chat_template returns a dict. `make env-llm`/`make llm`; `figures/plot_llm_merge.py`, README, `tests/test_llm.py` (+3, 125 green). + +**2026-07-05 — LLM merge 7B firm-up on Imperial CX3 (`llm_merge_hpc`): marginal sign → decisive.** ✅ +- Ran on one L40S (46 GB) via `/imperial-hpc` runbook; 8 min walltime; Qwen2.5-7B-Instruct, 200 tests/family. +- **Both merges 0.87 overall > best specialist 0.77** (decisive +10 pts) and beat every specialist on + every family; worst-family 0.62 vs ≤0.57. Both 0.5B caveats resolved: overall-exceeds is now clean, + and dilution VANISHES (merge 0.62 > lists-spec 0.57 on lists) — dilution was a small-model artefact. +- `results/llm_merge_hpc/` (README legend, data-driven figure title). Next refinement: module-level + union-preserving recombination (MoE-expert/adapter-union = real-weight E8 max-merge), not delta-avg.