Giorgio Gilestro
809e45a5e0
First step from toy models toward real language models, on one 16 GB GPU. New src/llm/ package: procedural task families + exact-match verifier (tasks.py), batched eval (evaluate.py), LoRA specialisation (specialise.py, manual answer-only SFT), weight-space merge via peft add_weighted_adapter (merge.py: soup = averaged deltas, ties = sign-reconciled union), runner (experiment.py, kind llm_merge). Base Qwen2.5-0.5B-Instruct (Apache-2.0); three disjoint hard families (lists/strings/arith); one LoRA specialist each (~90s total). Result (seed 1), reported honestly: - STRONG/robust: the merges are the ONLY models competent across ALL families -- worst-family ~0.25 vs <0.16 for every single specialist (the Fisher-Muller "generalist assembled from specialists" signature, in real LoRA weights). - MARGINAL: "exceeds every parent overall" is only marginal at this scale (soup 0.64 vs best specialist 0.63; ties 0.61 below it). - CAVEAT VISIBLE: averaging dilutes peaks (lists specialist 0.43 -> merge 0.26) -- Layer-1's "merge, don't average" (E4) appearing in real weights. The pipeline works end-to-end; the balance/retention half reproduces; the strict overall-exceeds and soup-vs-ties distinction need scale (bigger base, more/cleaner families, seeds, a dilution-resistant / offspring-selected merge) -- the HPC step. Env: Python 3.14 + transformers 5.13 works; note transformers-5.x apply_chat_template returns a dict. make env-llm / make llm; adapters under gitignored models/llm/, base in the HF cache. figures/plot_llm_merge.py, README, tests/test_llm.py (+3, 125 green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
48 lines
3.4 KiB
Markdown
48 lines
3.4 KiB
Markdown
# llm_merge — recombining specialist LLMs (the first real-LLM prototype; blueprint C2/C4)
|
||
|
||
**Claim tested.** The first step from toy models toward real language models: does the sexual-
|
||
reproduction result — recombining decorrelated specialists yields a model that exceeds/retains what
|
||
any single parent has (E8) — appear in real LoRA-adapted LLM weights? This is a **prototype**, run
|
||
on a single 16 GB consumer GPU, not the full society.
|
||
|
||
**Setup.** Base model **Qwen2.5-0.5B-Instruct** (Apache-2.0). Three *disjoint*, procedurally-generated
|
||
task families with an **exact-match verifier** (the "reality that says no"): `lists` (list ops),
|
||
`strings` (string ops), `arith` (integer arithmetic), deliberately made hard so specialists
|
||
decorrelate. One **LoRA specialist** is fine-tuned per family (~90 s for all three), then the base,
|
||
each specialist, and two weight-space **merges** — `soup` (averaged LoRA deltas) and `ties`
|
||
(sign-reconciled union) — are evaluated on a held-out mixed test set. Seed 1, 100 test tasks/family.
|
||
|
||
### Results (accuracy)
|
||
| model | lists | strings | arith | overall | **worst family** |
|
||
|---|---|---|---|---|---|
|
||
| base | 0.15 | 0.15 | 0.53 | 0.28 | 0.15 |
|
||
| spec: lists | 0.43 | 0.16 | 0.71 | 0.43 | 0.16 |
|
||
| spec: strings | 0.08 | **1.00** | 0.80 | 0.63 | 0.08 |
|
||
| spec: arith | 0.11 | 0.22 | 0.78 | 0.37 | 0.11 |
|
||
| **merge: soup** | 0.26 | 0.74 | 0.91 | 0.64 | **0.26** |
|
||
| **merge: ties** | 0.23 | 0.71 | 0.90 | 0.61 | **0.23** |
|
||
|
||
### What holds, and what doesn't (honest)
|
||
- **Strong and robust — balance / "retains all specialties".** The merges are the *only* models
|
||
competent across **all** families: worst-family ≈ **0.25**, versus **< 0.16** for every single
|
||
specialist (the best specialist, strings, is at 0.08 on its worst family). Each specialist spikes on
|
||
its own family and is weak elsewhere; the merge is decent everywhere. This is the Fisher-Muller
|
||
"a generalist assembled from specialists" signature, in real LLM weights.
|
||
- **Marginal / noisy — "exceeds any parent overall".** On *overall* accuracy the merge only *matches*
|
||
the best specialist (soup 0.64 vs strings-specialist 0.63; ties 0.61 is slightly below). At this
|
||
scale (a 0.5 B model, 3 families, one seed) the strict "offspring exceed every parent" claim is not
|
||
cleanly established.
|
||
- **The dilution caveat, visible in the flesh.** On `lists`, the lists-specialist alone scores 0.43
|
||
but the merge only 0.23–0.26 — weight-averaging *diluted* that specialist's contribution. This is
|
||
exactly Layer-1's "merge, don't average" concern (E4) appearing in real weights; the finer
|
||
soup-vs-ties advantage is not resolved at K=3.
|
||
|
||
### Takeaway
|
||
The pipeline runs end-to-end on real LLMs on a 16 GB GPU (specialise → verify → merge → evaluate), and
|
||
the **balance/retention** half of the sexual-reproduction claim reproduces clearly. The stronger
|
||
"exceeds every parent" claim is marginal at this toy scale and is the thing a larger run should firm
|
||
up — more, cleaner-decorrelated families; a bigger base; multiple seeds; and a merge that resists
|
||
dilution (e.g. per-task-family weighting, or the offspring-selection of "directed sex"). That scaling
|
||
is the natural HPC step; this prototype de-risks the machinery and shows the first sign in real
|
||
weights. **Falsifier (partially triggered — reported honestly):** a single specialist matches the
|
||
merge on *overall* here; the merge's advantage is currently specific to cross-family *balance*.
|