Giorgio Gilestro
809e45a5e0
First step from toy models toward real language models, on one 16 GB GPU. New src/llm/ package: procedural task families + exact-match verifier (tasks.py), batched eval (evaluate.py), LoRA specialisation (specialise.py, manual answer-only SFT), weight-space merge via peft add_weighted_adapter (merge.py: soup = averaged deltas, ties = sign-reconciled union), runner (experiment.py, kind llm_merge). Base Qwen2.5-0.5B-Instruct (Apache-2.0); three disjoint hard families (lists/strings/arith); one LoRA specialist each (~90s total). Result (seed 1), reported honestly: - STRONG/robust: the merges are the ONLY models competent across ALL families -- worst-family ~0.25 vs <0.16 for every single specialist (the Fisher-Muller "generalist assembled from specialists" signature, in real LoRA weights). - MARGINAL: "exceeds every parent overall" is only marginal at this scale (soup 0.64 vs best specialist 0.63; ties 0.61 below it). - CAVEAT VISIBLE: averaging dilutes peaks (lists specialist 0.43 -> merge 0.26) -- Layer-1's "merge, don't average" (E4) appearing in real weights. The pipeline works end-to-end; the balance/retention half reproduces; the strict overall-exceeds and soup-vs-ties distinction need scale (bigger base, more/cleaner families, seeds, a dilution-resistant / offspring-selected merge) -- the HPC step. Env: Python 3.14 + transformers 5.13 works; note transformers-5.x apply_chat_template returns a dict. make env-llm / make llm; adapters under gitignored models/llm/, base in the HF cache. figures/plot_llm_merge.py, README, tests/test_llm.py (+3, 125 green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
3.4 KiB
llm_merge — recombining specialist LLMs (the first real-LLM prototype; blueprint C2/C4)
Claim tested. The first step from toy models toward real language models: does the sexual- reproduction result — recombining decorrelated specialists yields a model that exceeds/retains what any single parent has (E8) — appear in real LoRA-adapted LLM weights? This is a prototype, run on a single 16 GB consumer GPU, not the full society.
Setup. Base model Qwen2.5-0.5B-Instruct (Apache-2.0). Three disjoint, procedurally-generated
task families with an exact-match verifier (the "reality that says no"): lists (list ops),
strings (string ops), arith (integer arithmetic), deliberately made hard so specialists
decorrelate. One LoRA specialist is fine-tuned per family (~90 s for all three), then the base,
each specialist, and two weight-space merges — soup (averaged LoRA deltas) and ties
(sign-reconciled union) — are evaluated on a held-out mixed test set. Seed 1, 100 test tasks/family.
Results (accuracy)
| model | lists | strings | arith | overall | worst family |
|---|---|---|---|---|---|
| base | 0.15 | 0.15 | 0.53 | 0.28 | 0.15 |
| spec: lists | 0.43 | 0.16 | 0.71 | 0.43 | 0.16 |
| spec: strings | 0.08 | 1.00 | 0.80 | 0.63 | 0.08 |
| spec: arith | 0.11 | 0.22 | 0.78 | 0.37 | 0.11 |
| merge: soup | 0.26 | 0.74 | 0.91 | 0.64 | 0.26 |
| merge: ties | 0.23 | 0.71 | 0.90 | 0.61 | 0.23 |
What holds, and what doesn't (honest)
- Strong and robust — balance / "retains all specialties". The merges are the only models competent across all families: worst-family ≈ 0.25, versus < 0.16 for every single specialist (the best specialist, strings, is at 0.08 on its worst family). Each specialist spikes on its own family and is weak elsewhere; the merge is decent everywhere. This is the Fisher-Muller "a generalist assembled from specialists" signature, in real LLM weights.
- Marginal / noisy — "exceeds any parent overall". On overall accuracy the merge only matches the best specialist (soup 0.64 vs strings-specialist 0.63; ties 0.61 is slightly below). At this scale (a 0.5 B model, 3 families, one seed) the strict "offspring exceed every parent" claim is not cleanly established.
- The dilution caveat, visible in the flesh. On
lists, the lists-specialist alone scores 0.43 but the merge only 0.23–0.26 — weight-averaging diluted that specialist's contribution. This is exactly Layer-1's "merge, don't average" concern (E4) appearing in real weights; the finer soup-vs-ties advantage is not resolved at K=3.
Takeaway
The pipeline runs end-to-end on real LLMs on a 16 GB GPU (specialise → verify → merge → evaluate), and the balance/retention half of the sexual-reproduction claim reproduces clearly. The stronger "exceeds every parent" claim is marginal at this toy scale and is the thing a larger run should firm up — more, cleaner-decorrelated families; a bigger base; multiple seeds; and a merge that resists dilution (e.g. per-task-family weighting, or the offspring-selection of "directed sex"). That scaling is the natural HPC step; this prototype de-risks the machinery and shows the first sign in real weights. Falsifier (partially triggered — reported honestly): a single specialist matches the merge on overall here; the merge's advantage is currently specific to cross-family balance.