Giorgio Gilestro
84124de143
Clarity pass over the main text (36-item audit), Discussion rewrite and cut, acknowledgements, Souly et al. as ref 62, lettered SI panels, model section moved under Results; plus the untracked curriculum/society/compose/smol configs, runners, figures, stats and tests that the SI already cites. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
72 lines
5 KiB
Markdown
72 lines
5 KiB
Markdown
# llm_merge_hpc — recombining specialist LLMs at scale (7B, Imperial CX3; blueprint C2/C4)
|
||
|
||
**Claim tested.** The scale-up of `llm_merge`: at a *capable* base, does recombining decorrelated
|
||
specialists produce a model that **exceeds every single parent overall** *and* stays competent across
|
||
**all** families (the Fisher–Muller "offspring fitter than any parent" signature, E8) — the claim that
|
||
was only marginal at 0.5 B? Run on one **L40S (46 GB)** GPU node of Imperial College's CX3 HPC
|
||
(job walltime **8 min**).
|
||
|
||
**Setup.** Base model **Qwen2.5-7B-Instruct** (Apache-2.0). Same three *disjoint*, procedurally-
|
||
generated task families with an **exact-match verifier** (the "reality that says no"): `lists`,
|
||
`strings`, `arith`, deliberately hard so specialists decorrelate. One **LoRA specialist** (r=16,
|
||
α=32, 3 epochs, 800 train tasks) is fine-tuned per family, then the base, each specialist, and two
|
||
weight-space **merges** — `soup` (averaged LoRA deltas) and `ties` (sign-reconciled union) — are
|
||
evaluated on a held-out mixed test set. Seed 1, **200 test tasks/family**.
|
||
|
||
### Results (accuracy)
|
||
| model | lists | strings | arith | overall | **worst family** |
|
||
|---|---|---|---|---|---|
|
||
| base | 0.46 | 0.69 | 1.00 | 0.71 | 0.46 |
|
||
| spec: lists | 0.57 | 0.74 | 1.00 | 0.77 | 0.57 |
|
||
| spec: strings | 0.32 | 0.97 | 1.00 | 0.76 | 0.32 |
|
||
| spec: arith | 0.47 | 0.80 | 0.96 | 0.74 | 0.47 |
|
||
| **merge: soup** | **0.62** | **1.00** | 1.00 | **0.87** | **0.62** |
|
||
| **merge: ties** | **0.62** | **1.00** | 0.99 | **0.87** | **0.62** |
|
||
|
||
### What holds (the 0.5 B caveats resolved)
|
||
- **"Exceeds every parent overall" — now clean.** Both merges reach **0.87 overall**, above the best
|
||
single specialist (lists, **0.77**) by a decisive 10 points, and above every specialist on every
|
||
family (lists 0.62 > 0.57; strings 1.00 > 0.97; arith ≈ 1.00). This is the strict Fisher–Muller
|
||
claim — offspring fitter than *any* parent — which was only marginal at 0.5 B and is now clean.
|
||
- **"Retains all specialties" — sharper than ever.** The merges are the only models competent across
|
||
**all** families: worst-family **0.62**, versus **≤ 0.57** for every specialist (strings-specialist
|
||
collapses to 0.32 on lists). The generalist assembled from specialists dominates on both axes.
|
||
- **The dilution is gone.** At 0.5 B, averaging *diluted* the lists-specialist (0.43 → 0.26). At 7 B
|
||
the merge **exceeds** the lists-specialist on lists (0.62 > 0.57). A capable base has enough headroom
|
||
that weight-space averaging composes rather than dilutes — the "merge, don't average" concern (E4)
|
||
softens once the parents are strong. Soup and ties are indistinguishable at K=3 here.
|
||
|
||
### Takeaway
|
||
**The scale-up firms up the sign the prototype left marginal.** At a capable base, recombining
|
||
decorrelated specialists yields a model that beats every parent both overall and per-family, with no
|
||
dilution — the sexual-reproduction / Fisher–Muller claim reproduced cleanly in real LLM weights. The
|
||
0.5 B `llm_merge` prototype de-risked the pipeline and flagged dilution as the risk; the 7B run shows
|
||
that risk is a small-model artefact. **Falsifier (not triggered):** a single specialist matching or
|
||
beating the merge overall — here the merge leads by 10 points. Provenance: L40S, torch 2.12.1 /
|
||
transformers 5.13.0 / peft 0.19.1, `manifest.json` records the results hash and library versions.
|
||
|
||
*Note: this run's `manifest.json` has `git_commit: null` because it was produced on the HPC node from
|
||
an rsync'd (non-git) working copy; the committed artefacts here are the source of truth for the figure.*
|
||
|
||
## Seeds 1–3 (2026-09-11)
|
||
|
||
Seeds 2–3 were run on CX3 via `hpc/llm_7b_seeds.pbs` (seed 1 above was moved to `s1/`; the bundle
|
||
layout is now `s{seed}/`). Fixed test sets, training seed varied. Per-seed values and mean ± 95% CI
|
||
from `figures/stats_llm_7b_seeds.py`:
|
||
```
|
||
model metric n_seeds s1 s2 s3 mean ci95
|
||
best_specialist overall 3 0.773 0.840 0.808 0.807 0.038
|
||
best_specialist worst_family 3 0.575 0.520 0.630 0.575 0.062
|
||
merge_soup overall 3 0.873 0.877 0.870 0.873 0.004
|
||
merge_soup worst_family 3 0.625 0.635 0.640 0.633 0.009
|
||
merge_ties overall 3 0.868 0.873 0.860 0.867 0.008
|
||
merge_ties worst_family 3 0.615 0.635 0.625 0.625 0.011
|
||
|
||
contrast metric n_seeds s1 s2 s3 mean ci95 sign_agrees
|
||
merge_soup − best_specialist overall 3 0.100 0.037 0.062 0.066 0.036 3/3
|
||
merge_soup − best_specialist worst_family 3 0.050 0.115 0.010 0.058 0.060 3/3
|
||
merge_ties − best_specialist overall 3 0.095 0.033 0.052 0.060 0.036 3/3
|
||
merge_ties − best_specialist worst_family 3 0.040 0.115 -0.005 0.050 0.069 2/3
|
||
```
|
||
|
||
Reading: merged specialists beat the best single specialist overall in every seed (+0.066 ± 0.036); the seed-1 margin (+0.100) was the largest of the three because seed 1's best specialist was the weakest (0.773 vs 0.840, 0.808).
|