# llm_directed_hard_hpc — directed sex on HARD (unsaturated) tasks at 7B: offspring selection helps again

**Claim tested.** `llm_directed_hpc` found directed selection ≈ soup at 7B (0.868 ≈ 0.873) and flagged
the honest caveat that the easy families were *saturated* (no headroom for selection to exploit). This
run re-runs directed sex on the **hard task variant**, where the uniform soup is far from the ceiling —
the regime in which breeding + selection can actually improve on the default blend. One **L40S (46 GB)**
GPU, Imperial CX3; reuses the `spec_*_hard` specialists trained by `llm_moe_hard_hpc`.

**Setup.** Base **Qwen2.5-7B-Instruct**, `hard: true`, **24 offspring** (Dirichlet-weighted merges),
100-task/family validation split (selection), 200-task/family test split (report). Seed 1.

### Results (test accuracy — unsaturated)
| model | lists | strings | arith | overall | worst-family |
|---|---|---|---|---|---|
| best specialist (strings) | 0.155 | 0.665 | 0.455 | 0.425 | 0.155 |
| merge_soup (uniform, candidate 0) | 0.390 | 0.300 | 0.485 | 0.392 | 0.300 |
| **directed_overall** | 0.380 | 0.630 | 0.465 | **0.492** | **0.380** |
| **directed_balanced** | 0.380 | 0.630 | 0.465 | **0.492** | **0.380** |

### The finding: the 7B "no headroom" null was also saturation
- **Directed selection beats the uniform soup by +10 points (0.492 > 0.392).** On hard tasks there *is*
  a better blend than uniform averaging, and breeding 24 offspring + selecting on the verifier finds it
  — recovering most of the routing-level performance (0.492 vs routing 0.500) from a single deployable
  merged model. The `llm_directed_hpc` null (directed ≈ soup at 7B) was a saturation artefact, exactly
  as that run's honest caveat predicted.
- **Selection repairs fusion's dilution.** The winning offspring lifts strings from soup's diluted
  0.300 back to **0.630** (near the 0.665 specialist) while keeping lists' composition gain (0.380) —
  i.e. it finds a blend that composes where composition helps and avoids diluting the fragile skill.
  Both breeding objectives converged to the same winner (overall = balanced), which also improves
  worst-family (0.380 > soup 0.300).
- **Directed ≈ routing here.** A *single* searched-and-selected merged model (0.492) matches the
  per-input router (0.500) on hard tasks — offspring selection buys most of routing's benefit without
  needing a router at inference.

### Takeaway
On unsaturated tasks, directed sex (breed offspring + select on the verifier) beats the single a-priori
soup at 7B, resolving the earlier null: it was task saturation, not scale, that made selection inert.
Together with `llm_moe_hard_hpc` this completes the correction — **both** "merge, don't average" (union
> fusion) and "directed sex" (selection > single blend) are **headroom** phenomena that hold at 7B once
the tasks are hard enough to leave room, not weak-base-only effects. **Falsifier (not triggered):**
directed offspring ≤ uniform soup — instead they beat it by 10 points. Provenance in `manifest.json`
(`hard: true`, L40S, torch 2.12.1 / transformers 5.13.0 / peft 0.19.1).

## Seeds 1–3 (2026-09-11)

Seeds 2–3 were run on CX3 via `hpc/llm_7b_seeds.pbs` (seed 1 above was moved to `s1/`; the bundle
layout is now `s{seed}/`). Fixed test sets, training seed varied. Per-seed values and mean ± 95% CI
from `figures/stats_llm_7b_seeds.py`:
```
model       metric  n_seeds    s1    s2    s3  mean  ci95
       merge_soup      overall        3 0.392 0.405 0.428 0.408 0.021
       merge_soup worst_family        3 0.300 0.345 0.340 0.328 0.028
 directed_overall      overall        3 0.492 0.480 0.473 0.482 0.011
 directed_overall worst_family        3 0.380 0.405 0.430 0.405 0.028
directed_balanced      overall        3 0.492 0.480 0.473 0.482 0.011
directed_balanced worst_family        3 0.380 0.405 0.430 0.405 0.028

                      contrast       metric  n_seeds   s1    s2    s3  mean  ci95 sign_agrees
 directed_overall − merge_soup      overall        3 0.10 0.075 0.045 0.073 0.031         3/3
 directed_overall − merge_soup worst_family        3 0.08 0.060 0.090 0.077 0.017         3/3
directed_balanced − merge_soup      overall        3 0.10 0.075 0.045 0.073 0.031         3/3
directed_balanced − merge_soup worst_family        3 0.08 0.060 0.090 0.077 0.017         3/3
```

Reading: directed selection beats the a-priori soup in every seed (+0.073 ± 0.031 overall, +0.077 ± 0.017 worst-family). The two objectives selected the same offspring in all three seeds.