Manuscript revision and pending experiment work, snapshot before restructuring

Clarity pass over the main text (36-item audit), Discussion rewrite and cut,
acknowledgements, Souly et al. as ref 62, lettered SI panels, model section
moved under Results; plus the untracked curriculum/society/compose/smol
configs, runners, figures, stats and tests that the SI already cites.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm

results/llm_merge_hpc/README.md

@@ -47,3 +47,26 @@ transformers 5.13.0 / peft 0.19.1, `manifest.json` records the results hash and
 
 *Note: this run's `manifest.json` has `git_commit: null` because it was produced on the HPC node from
 an rsync'd (non-git) working copy; the committed artefacts here are the source of truth for the figure.*
+
+## Seeds 1–3 (2026-09-11)
+
+Seeds 2–3 were run on CX3 via `hpc/llm_7b_seeds.pbs` (seed 1 above was moved to `s1/`; the bundle
+layout is now `s{seed}/`). Fixed test sets, training seed varied. Per-seed values and mean ± 95% CI
+from `figures/stats_llm_7b_seeds.py`:
+```
+model       metric  n_seeds    s1    s2    s3  mean  ci95
+best_specialist      overall        3 0.773 0.840 0.808 0.807 0.038
+best_specialist worst_family        3 0.575 0.520 0.630 0.575 0.062
+     merge_soup      overall        3 0.873 0.877 0.870 0.873 0.004
+     merge_soup worst_family        3 0.625 0.635 0.640 0.633 0.009
+     merge_ties      overall        3 0.868 0.873 0.860 0.867 0.008
+     merge_ties worst_family        3 0.615 0.635 0.625 0.625 0.011
+
+                    contrast       metric  n_seeds    s1    s2     s3  mean  ci95 sign_agrees
+merge_soup − best_specialist      overall        3 0.100 0.037  0.062 0.066 0.036         3/3
+merge_soup − best_specialist worst_family        3 0.050 0.115  0.010 0.058 0.060         3/3
+merge_ties − best_specialist      overall        3 0.095 0.033  0.052 0.060 0.036         3/3
+merge_ties − best_specialist worst_family        3 0.040 0.115 -0.005 0.050 0.069         2/3
+```
+
+Reading: merged specialists beat the best single specialist overall in every seed (+0.066 ± 0.036); the seed-1 margin (+0.100) was the largest of the three because seed 1's best specialist was the weakest (0.773 vs 0.840, 0.808).

results/llm_merge_hpc/s1/manifest.json

@@ -0,0 +1,20 @@
+{
+  "experiment": "llm_merge_hpc",
+  "master_seed": 1,
+  "git_commit": null,
+  "python": "3.11.13",
+  "libraries": {
+    "numpy": "2.4.6",
+    "scipy": "1.17.1",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.13.0",
+    "peft": "0.19.1"
+  },
+  "rows": 30,
+  "results_sha256": "6cc0a07c66ba92a379d895d6d6707591aced48f06eee895bb4f6c15d12e6e588",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-7B-Instruct"
+}

results/llm_merge_hpc/s1/resolved_config.yaml

@@ -0,0 +1,24 @@
+experiment: llm_merge_hpc
+seed: 1
+n_replicates: 1
+source_config:
+  experiment: llm_merge_hpc
+  kind: llm_merge
+  seed: 1
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-7B-Instruct
+  families:
+  - lists
+  - strings
+  - arith
+  n_train: 800
+  n_test: 200
+  epochs: 3
+  lora:
+    r: 16
+    alpha: 32
+  merges:
+  - soup
+  - ties
+  output:
+    dir: results/llm_merge_hpc

results/llm_merge_hpc/s2/manifest.json

@@ -0,0 +1,21 @@
+{
+  "experiment": "llm_merge_hpc",
+  "master_seed": 2,
+  "git_commit": null,
+  "python": "3.11.7",
+  "libraries": {
+    "numpy": "2.4.6",
+    "scipy": "1.17.1",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.16.1",
+    "peft": "0.20.0"
+  },
+  "rows": 30,
+  "results_sha256": "a7345f168ae007ed570374bd86df715a60782bd60493aee53eac5d7e6fc1f393",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-7B-Instruct",
+  "hard": false
+}

results/llm_merge_hpc/s2/resolved_config.yaml

@@ -0,0 +1,24 @@
+experiment: llm_merge_hpc
+seed: 2
+n_replicates: 1
+source_config:
+  experiment: llm_merge_hpc
+  kind: llm_merge
+  seed: 2
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-7B-Instruct
+  families:
+  - lists
+  - strings
+  - arith
+  n_train: 800
+  n_test: 200
+  epochs: 3
+  lora:
+    r: 16
+    alpha: 32
+  merges:
+  - soup
+  - ties
+  output:
+    dir: results/llm_merge_hpc/s2

results/llm_merge_hpc/s3/manifest.json

@@ -0,0 +1,21 @@
+{
+  "experiment": "llm_merge_hpc",
+  "master_seed": 3,
+  "git_commit": null,
+  "python": "3.11.7",
+  "libraries": {
+    "numpy": "2.4.6",
+    "scipy": "1.17.1",
+    "pandas": "3.0.3",
+    "pyarrow": "24.0.0",
+    "torch": "2.12.1",
+    "transformers": "5.16.1",
+    "peft": "0.20.0"
+  },
+  "rows": 30,
+  "results_sha256": "b59cf7d62c07a21001208cad8dc79607e93e4bc6d66199081192f84a6b614db7",
+  "layer": "2",
+  "tier": "llm",
+  "base_model": "Qwen/Qwen2.5-7B-Instruct",
+  "hard": false
+}

results/llm_merge_hpc/s3/resolved_config.yaml

@@ -0,0 +1,24 @@
+experiment: llm_merge_hpc
+seed: 3
+n_replicates: 1
+source_config:
+  experiment: llm_merge_hpc
+  kind: llm_merge
+  seed: 3
+  n_replicates: 1
+  base_model: Qwen/Qwen2.5-7B-Instruct
+  families:
+  - lists
+  - strings
+  - arith
+  n_train: 800
+  n_test: 200
+  epochs: 3
+  lora:
+    r: 16
+    alpha: 32
+  merges:
+  - soup
+  - ties
+  output:
+    dir: results/llm_merge_hpc/s3