Giorgio Gilestro
e433e48860
Adds the "directed sex" operator (E10) the moe regime-flip pointed to: don't
commit to one a-priori blend — breed a population of recombinant offspring
(specialists merged at Dirichlet-sampled weights), score each on a held-out
validation split with the verifier, and keep the fittest, reported on a fresh
test split. Two breeding objectives: best-overall and best-worst-family.
src/llm/directed.py + kind llm_directed, reusing the cached specialists.
Result — refinements pay off in proportion to how far the uniform soup is from
optimal:
- 0.5B (soup dilutes): directed selection beats soup on the bred objective —
directed_overall 0.69 > soup 0.64; directed_balanced worst-family 0.37 > 0.26.
Riders: single-objective selection trades off the other axis (overall-breed
tanks lists to 0.17); a global blend still trails per-input routing (0.74).
- 7B (Imperial CX3, soup already composes to ceiling on near-saturated families,
strings/arith 1.00): directed ~= soup (0.868 ~ 0.873, marginally below via a
val/test overfit gap) — no fitter offspring to breed.
Through-line across all four LLM runs: "merge, don't average" and its refinements
(routing, directed selection) are weak-base / suboptimal-default phenomena — they
help at 0.5B and are inert at 7B. Honest limitation kept in the writeup: the 7B
families are near-saturated, which caps the headroom; a harder unsaturated
benchmark is the fair next test.
Also folds in the two llm_moe local manifest/config files missed in
|
||
|---|---|---|
| .. | ||
| llm_directed.pdf | ||
| llm_directed.png | ||
| manifest.json | ||
| README.md | ||
| resolved_config.yaml | ||
llm_directed — directed sex in weight space: breed offspring + select on the verifier (E10, 0.5B)
Claim tested. llm_moe left a clean gap: fusion can compose beyond the parents but the right
blend is unknown and base-dependent, while pure routing is capped at the best parent. E10's answer is
directed sex — biology can't preview offspring, an AI can: breed a population of recombinant
offspring (the specialists merged at many different weights), score each against the verifier
("reality") on a held-out validation split, and keep the fittest. Selection replaces betting on one
a-priori blend. Two breeding objectives: best validation overall, and best validation
worst-family (raw capability vs the balanced generalist).
Setup. Base Qwen2.5-0.5B-Instruct, the three cached llm_merge specialists, 16 offspring
(Dirichlet-weighted merges, concentration 0.5, pinning candidate 0 = uniform soup for reference),
scored on an 80-task/family validation split, winners reported on a fresh 100-task/family test
split (no selection-on-test leakage). Seed 1.
Results (test accuracy)
| model | lists | strings | arith | overall | worst-family |
|---|---|---|---|---|---|
| best specialist (strings) | 0.08 | 1.00 | 0.80 | 0.63 | 0.08 |
| merge_soup (uniform, candidate 0) | 0.26 | 0.74 | 0.91 | 0.64 | 0.26 |
| directed_overall (bred for overall) | 0.17 | 0.99 | 0.92 | 0.69 | 0.17 |
| directed_balanced (bred for worst-family) | 0.37 | 0.37 | 0.79 | 0.51 | 0.37 |
What holds, and the honest cost
- Offspring selection beats the single a-priori blend — on the objective you breed for.
directed_overallreaches 0.69 overall > soup 0.64 (and > best parent 0.63);directed_balancedreaches 0.37 worst-family > soup 0.26. Searching the recombination-weight space and letting the verifier choose beats committing to uniform averaging — the E10 "preview and keep the fittest" claim, in real weights. - Single-objective selection trades off the other axis (honest). Breeding for overall on
lexically-imbalanced families finds a strings+arith-heavy blend that sacrifices the rare
listsskill (0.17, below soup's 0.26); breeding for balance lifts worst-family to 0.37 but costs overall. Directed sex gives control over what you breed for — it does not hand you both for free. - A global blend still trails per-input routing at a weak base. At 0.5B the best directed global
merge (0.69 / 0.43-max) does not beat
llm_moe's per-input routing (0.74 / 0.43): when the base is weak, adapting the recombination per input beats any one fixed blend, however well selected. So directed sex over blends beats averaging, not routing — combining the two (route, then select among routed+blended offspring) is the natural next operator.
Takeaway
Directed sex — breed a population, select on the verifier — is confirmed in real LLM weights: it beats
the single uniform soup on whichever objective it optimises, the distinctly-AI advantage (offspring
preview + unbounded candidates) that biology lacks. The honest scope at 0.5B: selection buys one axis
at the other's expense, and a single global blend can't yet beat per-input routing. Whether searching
blends + selection can exceed even the strong 7B soup (which routing could not) is answered by
results/llm_directed_hpc/: it can't — directed ≈ soup (0.868 ≈ 0.873) because the 7B soup already
composes to the ceiling on these near-saturated families, leaving no fitter offspring to breed. So
directed sex helps exactly when the default blend is suboptimal (0.5B), and is inert when it is
already near-optimal (7B). Falsifier (not triggered at 0.5B): directed offspring ≤ uniform soup on
their bred objective — instead each beat it.