llm: first real-LLM prototype — recombining specialist LLMs (C2/C4)

First step from toy models toward real language models, on one 16 GB GPU.
New src/llm/ package: procedural task families + exact-match verifier
(tasks.py), batched eval (evaluate.py), LoRA specialisation (specialise.py,
manual answer-only SFT), weight-space merge via peft add_weighted_adapter
(merge.py: soup = averaged deltas, ties = sign-reconciled union), runner
(experiment.py, kind llm_merge). Base Qwen2.5-0.5B-Instruct (Apache-2.0);
three disjoint hard families (lists/strings/arith); one LoRA specialist each
(~90s total).

Result (seed 1), reported honestly:
- STRONG/robust: the merges are the ONLY models competent across ALL
  families -- worst-family ~0.25 vs <0.16 for every single specialist (the
  Fisher-Muller "generalist assembled from specialists" signature, in real
  LoRA weights).
- MARGINAL: "exceeds every parent overall" is only marginal at this scale
  (soup 0.64 vs best specialist 0.63; ties 0.61 below it).
- CAVEAT VISIBLE: averaging dilutes peaks (lists specialist 0.43 -> merge
  0.26) -- Layer-1's "merge, don't average" (E4) appearing in real weights.

The pipeline works end-to-end; the balance/retention half reproduces; the
strict overall-exceeds and soup-vs-ties distinction need scale (bigger base,
more/cleaner families, seeds, a dilution-resistant / offspring-selected
merge) -- the HPC step. Env: Python 3.14 + transformers 5.13 works;
note transformers-5.x apply_chat_template returns a dict. make env-llm /
make llm; adapters under gitignored models/llm/, base in the HF cache.
figures/plot_llm_merge.py, README, tests/test_llm.py (+3, 125 green).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

results/llm_merge/README.md

@@ -0,0 +1,48 @@
+# llm_merge — recombining specialist LLMs (the first real-LLM prototype; blueprint C2/C4)
+
+**Claim tested.** The first step from toy models toward real language models: does the sexual-
+reproduction result — recombining decorrelated specialists yields a model that exceeds/retains what
+any single parent has (E8) — appear in real LoRA-adapted LLM weights? This is a **prototype**, run
+on a single 16 GB consumer GPU, not the full society.
+
+**Setup.** Base model **Qwen2.5-0.5B-Instruct** (Apache-2.0). Three *disjoint*, procedurally-generated
+task families with an **exact-match verifier** (the "reality that says no"): `lists` (list ops),
+`strings` (string ops), `arith` (integer arithmetic), deliberately made hard so specialists
+decorrelate. One **LoRA specialist** is fine-tuned per family (~90 s for all three), then the base,
+each specialist, and two weight-space **merges** — `soup` (averaged LoRA deltas) and `ties`
+(sign-reconciled union) — are evaluated on a held-out mixed test set. Seed 1, 100 test tasks/family.
+
+### Results (accuracy)
+| model | lists | strings | arith | overall | **worst family** |
+|---|---|---|---|---|---|
+| base | 0.15 | 0.15 | 0.53 | 0.28 | 0.15 |
+| spec: lists | 0.43 | 0.16 | 0.71 | 0.43 | 0.16 |
+| spec: strings | 0.08 | **1.00** | 0.80 | 0.63 | 0.08 |
+| spec: arith | 0.11 | 0.22 | 0.78 | 0.37 | 0.11 |
+| **merge: soup** | 0.26 | 0.74 | 0.91 | 0.64 | **0.26** |
+| **merge: ties** | 0.23 | 0.71 | 0.90 | 0.61 | **0.23** |
+
+### What holds, and what doesn't (honest)
+- **Strong and robust — balance / "retains all specialties".** The merges are the *only* models
+  competent across **all** families: worst-family ≈ **0.25**, versus **< 0.16** for every single
+  specialist (the best specialist, strings, is at 0.08 on its worst family). Each specialist spikes on
+  its own family and is weak elsewhere; the merge is decent everywhere. This is the Fisher-Muller
+  "a generalist assembled from specialists" signature, in real LLM weights.
+- **Marginal / noisy — "exceeds any parent overall".** On *overall* accuracy the merge only *matches*
+  the best specialist (soup 0.64 vs strings-specialist 0.63; ties 0.61 is slightly below). At this
+  scale (a 0.5 B model, 3 families, one seed) the strict "offspring exceed every parent" claim is not
+  cleanly established.
+- **The dilution caveat, visible in the flesh.** On `lists`, the lists-specialist alone scores 0.43
+  but the merge only 0.23–0.26 — weight-averaging *diluted* that specialist's contribution. This is
+  exactly Layer-1's "merge, don't average" concern (E4) appearing in real weights; the finer
+  soup-vs-ties advantage is not resolved at K=3.
+
+### Takeaway
+The pipeline runs end-to-end on real LLMs on a 16 GB GPU (specialise → verify → merge → evaluate), and
+the **balance/retention** half of the sexual-reproduction claim reproduces clearly. The stronger
+"exceeds every parent" claim is marginal at this toy scale and is the thing a larger run should firm
+up — more, cleaner-decorrelated families; a bigger base; multiple seeds; and a merge that resists
+dilution (e.g. per-task-family weighting, or the offspring-selection of "directed sex"). That scaling
+is the natural HPC step; this prototype de-risks the machinery and shows the first sign in real
+weights. **Falsifier (partially triggered — reported honestly):** a single specialist matches the
+merge on *overall* here; the merge's advantage is currently specific to cross-family *balance*.

results/llm_merge/manifest.json

@@ -0,0 +1,20 @@
+{
+  "experiment": "llm_merge",
+  "master_seed": 1,
+  "git_commit": "6bca1db61e1130ac6899308cc18e520cd9872839",
+  "python": "3.14.5",
+  "libraries": {
+    "numpy": "2.5.0",
+    "scipy": "1.18.0",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.13.0",
+    "peft": "0.19.1"
+  },
+  "rows": 30,
+  "results_sha256": "bbc13776970c9bc1e2779b1abe0ff5dbee85ef2e1380eba6fe6e5c9ba7e012aa",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-0.5B-Instruct"
+}

results/llm_merge/resolved_config.yaml

@@ -0,0 +1,24 @@
+experiment: llm_merge
+seed: 1
+n_replicates: 1
+source_config:
+  experiment: llm_merge
+  kind: llm_merge
+  seed: 1
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-0.5B-Instruct
+  families:
+  - lists
+  - strings
+  - arith
+  n_train: 700
+  n_test: 100
+  epochs: 3
+  lora:
+    r: 16
+    alpha: 32
+  merges:
+  - soup
+  - ties
+  output:
+    dir: results/llm_merge