hard benchmark: the 7B "fusion wins / no headroom" results were saturation artefacts

The easy task families saturated 7B (strings & arith at 1.00), so the earlier
7B nulls — moe: fusion 0.87 > union 0.84; directed ~= soup — could not separate
"refinements don't help at scale" from "tasks too easy at 7B". Adds a hard task
variant (hard: true in tasks.py: multi-step lists, Caesar ciphers / letter
transforms, multi-step & larger arithmetic; same family labels and answer
formats, threaded through make_tasks/train_specialist/runners; hard specialists
cache separately as spec_*_hard) and re-runs both experiments at 7B on Imperial
CX3 (one L40S, 24 min, unsaturated: arith ~0.48, strings 0.67, lists 0.34).

Both nulls flip back to the 0.5B ordering:
- Union beats fusion again: routing 0.500 > fusion 0.40 (soup 0.392 / ties
  0.400), the same 10-pt margin as 0.5B. Fusion dilutes the fragile strings
  specialist so hard (0.665 -> soup 0.300) that soup even trails the best single
  specialist (0.425); routing keeps it intact (0.670).
- Directed selection beats soup again: 0.492 > 0.392 (+10 pts), recovering most
  of routing's benefit from one deployable merged model (lifts strings to 0.630).

Correction to the earlier interpretation: the llm_moe_hpc "regime flip" and the
llm_directed_hpc "no headroom" null were driven by TASK SATURATION, not base
capability. The operative variable is headroom — "merge, don't average" (union >
fusion) and "directed sex" (selection > single blend) hold whenever there is room
to lose to dilution: a weak base (0.5B) OR hard tasks at a strong base (7B-hard).
Fusion only wins in the degenerate corner where easy tasks let a strong base
compose to the 1.00 ceiling. Vindicates E8's max > mean in real 7B weights once
saturation is controlled.

Default (easy) task behaviour is unchanged (hard defaults False). +1 hard-task
test (131 green). Excludes the 0.5B smoke bundle (a pipeline gate, not a
deliverable). Results in results/llm_{moe,directed}_hard_hpc/ (parquet gitignored).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

results/llm_directed_hard_hpc/README.md

@@ -0,0 +1,42 @@
+# llm_directed_hard_hpc — directed sex on HARD (unsaturated) tasks at 7B: offspring selection helps again
+
+**Claim tested.** `llm_directed_hpc` found directed selection ≈ soup at 7B (0.868 ≈ 0.873) and flagged
+the honest caveat that the easy families were *saturated* (no headroom for selection to exploit). This
+run re-runs directed sex on the **hard task variant**, where the uniform soup is far from the ceiling —
+the regime in which breeding + selection can actually improve on the default blend. One **L40S (46 GB)**
+GPU, Imperial CX3; reuses the `spec_*_hard` specialists trained by `llm_moe_hard_hpc`.
+
+**Setup.** Base **Qwen2.5-7B-Instruct**, `hard: true`, **24 offspring** (Dirichlet-weighted merges),
+100-task/family validation split (selection), 200-task/family test split (report). Seed 1.
+
+### Results (test accuracy — unsaturated)
+| model | lists | strings | arith | overall | worst-family |
+|---|---|---|---|---|---|
+| best specialist (strings) | 0.155 | 0.665 | 0.455 | 0.425 | 0.155 |
+| merge_soup (uniform, candidate 0) | 0.390 | 0.300 | 0.485 | 0.392 | 0.300 |
+| **directed_overall** | 0.380 | 0.630 | 0.465 | **0.492** | **0.380** |
+| **directed_balanced** | 0.380 | 0.630 | 0.465 | **0.492** | **0.380** |
+
+### The finding: the 7B "no headroom" null was also saturation
+- **Directed selection beats the uniform soup by +10 points (0.492 > 0.392).** On hard tasks there *is*
+  a better blend than uniform averaging, and breeding 24 offspring + selecting on the verifier finds it
+  — recovering most of the routing-level performance (0.492 vs routing 0.500) from a single deployable
+  merged model. The `llm_directed_hpc` null (directed ≈ soup at 7B) was a saturation artefact, exactly
+  as that run's honest caveat predicted.
+- **Selection repairs fusion's dilution.** The winning offspring lifts strings from soup's diluted
+  0.300 back to **0.630** (near the 0.665 specialist) while keeping lists' composition gain (0.380) —
+  i.e. it finds a blend that composes where composition helps and avoids diluting the fragile skill.
+  Both breeding objectives converged to the same winner (overall = balanced), which also improves
+  worst-family (0.380 > soup 0.300).
+- **Directed ≈ routing here.** A *single* searched-and-selected merged model (0.492) matches the
+  per-input router (0.500) on hard tasks — offspring selection buys most of routing's benefit without
+  needing a router at inference.
+
+### Takeaway
+On unsaturated tasks, directed sex (breed offspring + select on the verifier) beats the single a-priori
+soup at 7B, resolving the earlier null: it was task saturation, not scale, that made selection inert.
+Together with `llm_moe_hard_hpc` this completes the correction — **both** "merge, don't average" (union
+> fusion) and "directed sex" (selection > single blend) are **headroom** phenomena that hold at 7B once
+the tasks are hard enough to leave room, not weak-base-only effects. **Falsifier (not triggered):**
+directed offspring ≤ uniform soup — instead they beat it by 10 points. Provenance in `manifest.json`
+(`hard: true`, L40S, torch 2.12.1 / transformers 5.13.0 / peft 0.19.1).