giorgio/MachineSex
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
MachineSex/results/llm_directed/README.md
Giorgio Gilestro e433e48860 llm_directed: directed sex (breed offspring + select on verifier) — E10 in real weights 
Adds the "directed sex" operator (E10) the moe regime-flip pointed to: don't
commit to one a-priori blend — breed a population of recombinant offspring
(specialists merged at Dirichlet-sampled weights), score each on a held-out
validation split with the verifier, and keep the fittest, reported on a fresh
test split. Two breeding objectives: best-overall and best-worst-family.
src/llm/directed.py + kind llm_directed, reusing the cached specialists.

Result — refinements pay off in proportion to how far the uniform soup is from
optimal:
- 0.5B (soup dilutes): directed selection beats soup on the bred objective —
  directed_overall 0.69 > soup 0.64; directed_balanced worst-family 0.37 > 0.26.
  Riders: single-objective selection trades off the other axis (overall-breed
  tanks lists to 0.17); a global blend still trails per-input routing (0.74).
- 7B (Imperial CX3, soup already composes to ceiling on near-saturated families,
  strings/arith 1.00): directed ~= soup (0.868 ~ 0.873, marginally below via a
  val/test overfit gap) — no fitter offspring to breed.

Through-line across all four LLM runs: "merge, don't average" and its refinements
(routing, directed selection) are weak-base / suboptimal-default phenomena — they
help at 0.5B and are inert at 7B. Honest limitation kept in the writeup: the 7B
families are near-saturated, which caps the headroom; a harder unsaturated
benchmark is the fair next test.

Also folds in the two llm_moe local manifest/config files missed in 8da0dac.
+3 directed unit tests (130 green). Results in results/llm_directed{,_hpc}/
(parquet gitignored).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-05 18:35:04 +01:00

3.7 KiB
Raw Blame History

llm_directed — directed sex in weight space: breed offspring + select on the verifier (E10, 0.5B)

Claim tested. llm_moe left a clean gap: fusion can compose beyond the parents but the right blend is unknown and base-dependent, while pure routing is capped at the best parent. E10's answer is directed sex — biology can't preview offspring, an AI can: breed a population of recombinant offspring (the specialists merged at many different weights), score each against the verifier ("reality") on a held-out validation split, and keep the fittest. Selection replaces betting on one a-priori blend. Two breeding objectives: best validation overall, and best validation worst-family (raw capability vs the balanced generalist).

Setup. Base Qwen2.5-0.5B-Instruct, the three cached llm_merge specialists, 16 offspring (Dirichlet-weighted merges, concentration 0.5, pinning candidate 0 = uniform soup for reference), scored on an 80-task/family validation split, winners reported on a fresh 100-task/family test split (no selection-on-test leakage). Seed 1.

Results (test accuracy)

model lists strings arith overall worst-family
best specialist (strings) 0.08 1.00 0.80 0.63 0.08
merge_soup (uniform, candidate 0) 0.26 0.74 0.91 0.64 0.26
directed_overall (bred for overall) 0.17 0.99 0.92 0.69 0.17
directed_balanced (bred for worst-family) 0.37 0.37 0.79 0.51 0.37

What holds, and the honest cost

  • Offspring selection beats the single a-priori blend — on the objective you breed for. directed_overall reaches 0.69 overall > soup 0.64 (and > best parent 0.63); directed_balanced reaches 0.37 worst-family > soup 0.26. Searching the recombination-weight space and letting the verifier choose beats committing to uniform averaging — the E10 "preview and keep the fittest" claim, in real weights.
  • Single-objective selection trades off the other axis (honest). Breeding for overall on lexically-imbalanced families finds a strings+arith-heavy blend that sacrifices the rare lists skill (0.17, below soup's 0.26); breeding for balance lifts worst-family to 0.37 but costs overall. Directed sex gives control over what you breed for — it does not hand you both for free.
  • A global blend still trails per-input routing at a weak base. At 0.5B the best directed global merge (0.69 / 0.43-max) does not beat llm_moe's per-input routing (0.74 / 0.43): when the base is weak, adapting the recombination per input beats any one fixed blend, however well selected. So directed sex over blends beats averaging, not routing — combining the two (route, then select among routed+blended offspring) is the natural next operator.

Takeaway

Directed sex — breed a population, select on the verifier — is confirmed in real LLM weights: it beats the single uniform soup on whichever objective it optimises, the distinctly-AI advantage (offspring preview + unbounded candidates) that biology lacks. The honest scope at 0.5B: selection buys one axis at the other's expense, and a single global blend can't yet beat per-input routing. Whether searching blends + selection can exceed even the strong 7B soup (which routing could not) is answered by results/llm_directed_hpc/: it can't — directed ≈ soup (0.868 ≈ 0.873) because the 7B soup already composes to the ceiling on these near-saturated families, leaving no fitter offspring to breed. So directed sex helps exactly when the default blend is suboptimal (0.5B), and is inert when it is already near-optimal (7B). Falsifier (not triggered at 0.5B): directed offspring ≤ uniform soup on their bred objective — instead each beat it.