Giorgio Gilestro
84124de143
Clarity pass over the main text (36-item audit), Discussion rewrite and cut, acknowledgements, Souly et al. as ref 62, lettered SI panels, model section moved under Results; plus the untracked curriculum/society/compose/smol configs, runners, figures, stats and tests that the SI already cites. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
65 lines
4.5 KiB
Markdown
65 lines
4.5 KiB
Markdown
# llm_directed_hard_hpc — directed sex on HARD (unsaturated) tasks at 7B: offspring selection helps again
|
||
|
||
**Claim tested.** `llm_directed_hpc` found directed selection ≈ soup at 7B (0.868 ≈ 0.873) and flagged
|
||
the honest caveat that the easy families were *saturated* (no headroom for selection to exploit). This
|
||
run re-runs directed sex on the **hard task variant**, where the uniform soup is far from the ceiling —
|
||
the regime in which breeding + selection can actually improve on the default blend. One **L40S (46 GB)**
|
||
GPU, Imperial CX3; reuses the `spec_*_hard` specialists trained by `llm_moe_hard_hpc`.
|
||
|
||
**Setup.** Base **Qwen2.5-7B-Instruct**, `hard: true`, **24 offspring** (Dirichlet-weighted merges),
|
||
100-task/family validation split (selection), 200-task/family test split (report). Seed 1.
|
||
|
||
### Results (test accuracy — unsaturated)
|
||
| model | lists | strings | arith | overall | worst-family |
|
||
|---|---|---|---|---|---|
|
||
| best specialist (strings) | 0.155 | 0.665 | 0.455 | 0.425 | 0.155 |
|
||
| merge_soup (uniform, candidate 0) | 0.390 | 0.300 | 0.485 | 0.392 | 0.300 |
|
||
| **directed_overall** | 0.380 | 0.630 | 0.465 | **0.492** | **0.380** |
|
||
| **directed_balanced** | 0.380 | 0.630 | 0.465 | **0.492** | **0.380** |
|
||
|
||
### The finding: the 7B "no headroom" null was also saturation
|
||
- **Directed selection beats the uniform soup by +10 points (0.492 > 0.392).** On hard tasks there *is*
|
||
a better blend than uniform averaging, and breeding 24 offspring + selecting on the verifier finds it
|
||
— recovering most of the routing-level performance (0.492 vs routing 0.500) from a single deployable
|
||
merged model. The `llm_directed_hpc` null (directed ≈ soup at 7B) was a saturation artefact, exactly
|
||
as that run's honest caveat predicted.
|
||
- **Selection repairs fusion's dilution.** The winning offspring lifts strings from soup's diluted
|
||
0.300 back to **0.630** (near the 0.665 specialist) while keeping lists' composition gain (0.380) —
|
||
i.e. it finds a blend that composes where composition helps and avoids diluting the fragile skill.
|
||
Both breeding objectives converged to the same winner (overall = balanced), which also improves
|
||
worst-family (0.380 > soup 0.300).
|
||
- **Directed ≈ routing here.** A *single* searched-and-selected merged model (0.492) matches the
|
||
per-input router (0.500) on hard tasks — offspring selection buys most of routing's benefit without
|
||
needing a router at inference.
|
||
|
||
### Takeaway
|
||
On unsaturated tasks, directed sex (breed offspring + select on the verifier) beats the single a-priori
|
||
soup at 7B, resolving the earlier null: it was task saturation, not scale, that made selection inert.
|
||
Together with `llm_moe_hard_hpc` this completes the correction — **both** "merge, don't average" (union
|
||
> fusion) and "directed sex" (selection > single blend) are **headroom** phenomena that hold at 7B once
|
||
the tasks are hard enough to leave room, not weak-base-only effects. **Falsifier (not triggered):**
|
||
directed offspring ≤ uniform soup — instead they beat it by 10 points. Provenance in `manifest.json`
|
||
(`hard: true`, L40S, torch 2.12.1 / transformers 5.13.0 / peft 0.19.1).
|
||
|
||
## Seeds 1–3 (2026-09-11)
|
||
|
||
Seeds 2–3 were run on CX3 via `hpc/llm_7b_seeds.pbs` (seed 1 above was moved to `s1/`; the bundle
|
||
layout is now `s{seed}/`). Fixed test sets, training seed varied. Per-seed values and mean ± 95% CI
|
||
from `figures/stats_llm_7b_seeds.py`:
|
||
```
|
||
model metric n_seeds s1 s2 s3 mean ci95
|
||
merge_soup overall 3 0.392 0.405 0.428 0.408 0.021
|
||
merge_soup worst_family 3 0.300 0.345 0.340 0.328 0.028
|
||
directed_overall overall 3 0.492 0.480 0.473 0.482 0.011
|
||
directed_overall worst_family 3 0.380 0.405 0.430 0.405 0.028
|
||
directed_balanced overall 3 0.492 0.480 0.473 0.482 0.011
|
||
directed_balanced worst_family 3 0.380 0.405 0.430 0.405 0.028
|
||
|
||
contrast metric n_seeds s1 s2 s3 mean ci95 sign_agrees
|
||
directed_overall − merge_soup overall 3 0.10 0.075 0.045 0.073 0.031 3/3
|
||
directed_overall − merge_soup worst_family 3 0.08 0.060 0.090 0.077 0.017 3/3
|
||
directed_balanced − merge_soup overall 3 0.10 0.075 0.045 0.073 0.031 3/3
|
||
directed_balanced − merge_soup worst_family 3 0.08 0.060 0.090 0.077 0.017 3/3
|
||
```
|
||
|
||
Reading: directed selection beats the a-priori soup in every seed (+0.073 ± 0.031 overall, +0.077 ± 0.017 worst-family). The two objectives selected the same offspring in all three seeds.
|