# llm_directed_hpc — directed sex at scale (7B): no headroom once the soup already composes

**Claim tested.** At 0.5B, breeding offspring and selecting on the verifier beat the uniform soup
(`llm_directed`: directed 0.69 > soup 0.64). But `llm_merge_hpc` showed the 7B soup already *composes*
to 0.87, beating every specialist. So the honest question: does searching the recombination-weight
space + selecting on the verifier find anything better than that strong default — or is there simply no
headroom left? Run on one **L40S (46 GB)** GPU of Imperial's CX3 HPC, 9 min walltime, 24 offspring,
reusing the cached 7B specialists.

**Setup.** Base **Qwen2.5-7B-Instruct**, cached 7B specialists, **24 offspring** (Dirichlet-weighted
merges, concentration 0.5, candidate 0 = uniform soup), scored on a **100-task/family validation**
split, winners reported on a **fresh 200-task/family test** split. Seed 1.

### Results (test accuracy)
| model | lists | strings | arith | overall | worst-family |
|---|---|---|---|---|---|
| best specialist (lists) | 0.57 | 0.74 | 1.00 | 0.77 | 0.57 |
| merge_soup (uniform, candidate 0) | 0.62 | 1.00 | 1.00 | **0.873** | 0.625 |
| directed_overall (bred for overall) | 0.62 | 1.00 | 1.00 | 0.868 | 0.615 |
| directed_balanced (bred for worst-family) | 0.62 | 1.00 | 1.00 | 0.868 | 0.615 |

### The finding: directed selection is inert once the default is already near-optimal
- **Directed ≈ soup, and marginally below it (0.868 vs 0.873).** Both breeding objectives converged to
  a near-soup blend, and the validation-selected winner is a *hair* worse than the plain uniform soup
  on the held-out test set — a small val/test generalisation gap (selection overfits the 100-task/
  family validation split). Searching 24 offspring found nothing that beats candidate 0.
- **Why: no headroom.** The 7B soup already *composes* to the ceiling on these families — strings and
  arith are saturated at **1.00**, and lists (0.62) is the only slack, itself already above every
  specialist. When the default blend is already optimal, there is no fitter offspring to breed, so
  selection can only match it (or lose slightly to val noise).
- **Honest limitation.** These task families are *near-saturated* at 7B (2 of 3 at 1.00), which
  structurally caps the headroom any recombination refinement could exploit. A fair test of directed
  sex at scale needs a **harder, unsaturated** benchmark where the optimal blend is non-trivial — this
  run cannot distinguish "directed sex doesn't help at scale" from "these tasks are too easy at 7B."

### Takeaway — the through-line across all four LLM runs
The value of every recombination *refinement* (routing, directed selection) scales with **how far the
default uniform soup is from optimal**:
- **0.5B** — soup *dilutes* (far from optimal): routing beats soup (0.74 > 0.64), directed selection
  beats soup (0.69 > 0.64). Refinements pay off.
- **7B** — soup *composes* to near-ceiling on saturated tasks: routing < soup (0.84 < 0.87), directed
  ≈ soup (0.868 ≈ 0.873). No headroom; refinements are inert.

So "merge, don't average" and its refinements are **weak-base / suboptimal-default** phenomena. The
open question a capable base leaves is whether directed sex helps on *hard, unsaturated* tasks at scale
— the natural next benchmark. **Falsifier for this run (triggered as a null, reported honestly):**
directed offspring failed to exceed the uniform soup at 7B; here they tied/marginally trailed it
because the soup was already optimal on near-saturated families. Provenance in `manifest.json`
(L40S, torch 2.12.1 / transformers 5.13.0 / peft 0.19.1; `git_commit: null` — rsync'd node copy).