llm_society pilot v1 diagnosis + v2 knobs: raise inheritance fidelity, add elitism

Pilot v1: full and no_grounding indistinguishable - both arms pay a ~25%
distillation tax per generation and pin at the same mutation-selection floor,
so grounding has no differential to act on. v2 raises inheritance fidelity
(n_inherit 240->600, child epochs 2->3, founders 600x3) and adds two evenly-
applied knobs: elitism (top parent survives as an unmodified copy - overlapping
generations, compensating for lossy distillation where E11 had faithful
genotype copying) and n_parents (sharper truncation selection). v1 archived at
results/llm_society_pilot1; v2 running.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v

configs/llm/society.yaml

@@ -10,11 +10,13 @@ g: 0.5
 lam: 0.3
 n_test: 40
 n_val: 30
-n_conf: 60
+n_conf: 90
-n_inherit: 240
+n_inherit: 600
 n_candidates: 6
-epochs: 2
+elitism: 1
-spec_train: 300
+n_parents: 3
-spec_epochs: 2
+epochs: 3
+spec_train: 600
+spec_epochs: 3
 lora: {r: 16, alpha: 32}
 output: {dir: results/llm_society}

src/llm/society.py

@@ -178,6 +178,8 @@ def run_society_experiment(cfg: dict) -> pd.DataFrame:
     n_inherit = int(cfg.get("n_inherit", 240))
     n_cand = int(cfg.get("n_candidates", 6))
     epochs = int(cfg.get("epochs", 2))
+    elitism = int(cfg.get("elitism", 0))
+    n_parents = int(cfg.get("n_parents", max(2, N // 2)))
     spec_train = int(cfg.get("spec_train", 300))
     spec_epochs = int(cfg.get("spec_epochs", 2))
     hard = bool(cfg.get("hard", False))
@@ -226,7 +228,7 @@ def run_society_experiment(cfg: dict) -> pd.DataFrame:
             fitness = np.array([fv["overall"] for fv in fit_val])
             scores = s["g"] * fitness + (1.0 - s["g"]) * conf
 
-            parents = select_parents(scores, dist, max(2, N // 2), diversity=s["diversity"], lam=lam)
+            parents = select_parents(scores, dist, n_parents, diversity=s["diversity"], lam=lam)
 
             # ---- rows (reporting uses the verifier in every arm; the loop does not)
             for i in range(N):
@@ -253,10 +255,11 @@ def run_society_experiment(cfg: dict) -> pd.DataFrame:
             inherit_pool = sum([make_tasks(f, n_inherit // len(fams),
                                            seed=seed * 104729 + t * 17 + i, hard=hard)
                                 for i, f in enumerate(fams)], [])
+            n_bred = N - elitism
             child_sources: list[dict] = []                  # per child: parent names + weights
             if s["sex"]:
                 from .directed import sample_merge_weights
-                pairs = complementary_pairs(parents, dist, N)
+                pairs = complementary_pairs(parents, dist, n_bred)
                 for c, (pa, pb) in enumerate(pairs):
                     w = sample_merge_weights(2, n_cand, rng)
                     best_i, best_v = 0, -np.inf
@@ -285,7 +288,7 @@ def run_society_experiment(cfg: dict) -> pd.DataFrame:
                     model.delete_adapter(cname)
                     child_sources.append({"parents": (pa, pb), "answers": answers})
             else:
-                for c in range(N):
+                for c in range(n_bred):
                     p = parents[c % len(parents)]
                     model.set_adapter(f"a{p}")
                     answers = generate(model, tok, [x.prompt for x in inherit_pool])
@@ -293,9 +296,20 @@ def run_society_experiment(cfg: dict) -> pd.DataFrame:
             del model
             torch.cuda.empty_cache()
 
-            # ---- reproduce: fresh LoRA per child, trained on its source's own answers
+            # ---- reproduce: fresh LoRA per child, trained on its source's own answers.
+            # Elites (overlapping generations): the top-scoring parents survive as unmodified
+            # copies, applied identically in every arm — reproduction here is lossy distillation,
+            # so without a survivor the mutation load erases the champion each generation.
             new_agents = []
-            for c, src in enumerate(child_sources):
+            for e in range(elitism):
+                d = root / arm / f"gen{t + 1}" / f"elite{e}"
+                shutil.copytree(agents[parents[e]], d, dirs_exist_ok=True)
+                new_agents.append(str(d))
+                rows.append({"experiment": name, "arm": arm, "seed": seed, "generation": t + 1,
+                             "agent": e, "selected": False, "conformity": float("nan"),
+                             "score": float("nan"), "metric": "parents",
+                             "value": float(parents[e] * 100 + parents[e])})
+            for c, src in enumerate(child_sources, start=elitism):
                 data = [Task(x.family, x.prompt, _answer_of(a))
                         for x, a in zip(inherit_pool, src["answers"]) if _answer_of(a)]
                 d = root / arm / f"gen{t + 1}" / f"agent{c}"

tasks/workorder-llm-society.md

@@ -70,7 +70,14 @@ disagreement), consensus accuracy, conformity−truth gap. E11's three panels re
 - [x] `src/llm/society.py` (pure operators + GPU loop), `kind: llm_society`
 - [x] Pure-function tests (consensus, conformity, QD selection, pairing) — 155 green
 - [x] `configs/llm/society_smoke.yaml` → smoke green (exit 0; conformity 0.54→0.75, diversity 0.69→0.41 in one self-consumption generation — the signature is live)
-- [ ] `configs/llm/society.yaml` pilot (full + no_grounding) → sanity-check trajectories
+- [x] Pilot v1 (n_inherit=240, ep2, no elitism): **arms indistinguishable** — both drop ~0.1 in the
+  first distillation generation and plateau at the same mutation–selection floor (best ≈0.40, mean
+  ≈0.35, consensus 0.63→0.4 in both). Diagnosis: inheritance too lossy; the ~25%/generation
+  distillation tax swamps the selection differential. v1 archived at `results/llm_society_pilot1`.
+- [ ] Pilot v2 (n_inherit=600, ep3, founders 600×3, elitism=1 all arms, n_parents=3, n_conf=90) —
+  raises inheritance fidelity so selection has something to act on; elites = overlapping
+  generations, applied identically in every arm (documented honestly; reproduction here is lossy
+  distillation, unlike E11's faithful genotype copy)
 - [ ] GG gate: review pilot curves before the campaign
 - [ ] `hpc/llm_society.pbs` array job (arm × seed) → campaign
 - [ ] Figure `paper/pnas/make_figs.py` panel(s); fold into main.md (replaces the "open" cell)