llm_moe: the union operator (route/max-merge) vs fusion — and the regime flips at scale

Adds the union-preserving recombination operator that llm_merge lacked (E8's max,
not mean): keep each specialist LoRA intact and SELECT the right one per prompt
(MoE router: oracle, or training-free nearest-centroid over base embeddings) or
per module (max_merge = winner-take-all by delta norm). src/llm/moe.py, kind
llm_moe, reuses the cached specialists.

Result — a clean regime boundary for "merge, don't average":
- 0.5B: union wins. Routing 0.74 / worst-family 0.43 > soup 0.64 / 0.26, with no
  dilution (recovers each specialist's own-family peak). E8's max > mean in real
  weights, because at a weak base averaging dilutes.
- 7B (Imperial CX3, L40S, 9 min): the ordering INVERTS. Fusion wins — soup 0.87 >
  routing 0.84 > max_merge 0.78. Routing is capped at the best parent per family;
  fusion blends and, given a capable base, COMPOSES beyond any parent (soup lists
  0.62 > spec 0.57). Selection can't synthesise better than its best component;
  averaging-that-composes can.

So "merge, don't average" (E4/E8) is a weak-parent / small-model law, not
universal: union wins under dilution, fusion wins under composition. Refines E8
(its additive-landscape max>mean assumed no compositional headroom). The operator
to want is fusion-that-composes + offspring selection = the directed-sex ideal
(E10) — the natural next experiment.

Honest riders: the learned router is trivially perfect (lexically-distinct
families), and router-free max_merge is the weakest union (not input-adaptive).
+2 router unit tests (127 green). Results in results/llm_moe{,_hpc}/ (parquet
gitignored per the reproducibility contract).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

tasks/todo.md

@@ -352,3 +352,18 @@ C3 vertical claim deferred.*
   and dilution VANISHES (merge 0.62 > lists-spec 0.57 on lists) — dilution was a small-model artefact.
 - `results/llm_merge_hpc/` (README legend, data-driven figure title). Next refinement: module-level
   union-preserving recombination (MoE-expert/adapter-union = real-weight E8 max-merge), not delta-avg.
+
+**2026-07-05 — MoE-expert / union recombination (`llm_moe`): E8's `max` vs `mean` in real weights.**
+- `src/llm/moe.py`: router (oracle + training-free nearest-centroid over base embeddings) + MoE
+  generate + router-free per-module `max_merge`. `kind: llm_moe` reuses the cached specialists.
+- **0.5B result:** routing beats fusion decisively — overall 0.74/worst 0.43 vs soup 0.64/0.26, no
+  dilution (recovers each specialist's own-family peak). Riders: learned router trivially perfect
+  (1.00, lexically-separable families) and static `max_merge` a poor union (0.46, not input-adaptive).
+- `configs/llm/{moe,moe_hpc}.yaml`, `figures/plot_llm_moe.py`, README, +2 tests (127 green),
+  `hpc/llm_moe.pbs`.
+- **7B firm-up (`llm_moe_hpc`, CX3 L40S, 9 min): the ordering FLIPS.** At 7B fusion wins —
+  soup 0.87 > routing 0.84 > max_merge 0.78 (0.5B had routing 0.74 > soup 0.64). Routing is capped at
+  the best parent per family; fusion *composes beyond* it at a capable base (soup lists 0.62 >
+  spec 0.57). So "merge, don't average" is a **weak-base law**, not universal — union wins under
+  dilution (0.5B), fusion wins under composition (7B). Refines E8. Next: fusion + offspring-selection
+  (directed sex). `results/llm_moe_hpc/` README + regime-aware figure.