MachineSex/tasks/prereg-llm-society-v2.md

Pre-registration — llm_society v2: the composed society at LLM scale

Status (2026-09-07, 21:30): calibrated; NO-GO at 0.5B (GG, §12a); 7B plan in §13 awaiting scope. Supersedes the design in workorder-llm-society.md (v1). Calibration record and every amendment are in §4a; the campaign was not submitted. Read §1 (what v1 got wrong), §4a (what calibration found), §12a (the decision), §13 (what next).

Why a v2. The v1 campaign (3 CX3 seeds landed 2026-09-07, results/llm_society_campaign/; seed 1 still running locally) did not reproduce E11: no_grounding degraded (0.575 → 0.436, worst arm in every seed), but full also declined (→ 0.506) and no_sex was flattest (0.558). The conformity−truth gap did not separate the arms. Read against the framework, v1 had three structural faults that the theory would have predicted, plus one power fault. All four are diagnosed in §1 and designed out in §3. The point of this document is to make the remaining predictions explicit before spending the compute, so the campaign can fail informatively.

---

0. The question and the claims it tests

Does a finite population of LLM agents under the four composed operators — grounded evaluation, directed recombination, diversity-preserving selection, lossy inheritance — climb to capability that no founder had and hold it, while each ablation fails in its own way? This is E11 at the language-model tier: the paper's "open — the stated gap" cell (Fig. 1A).

Claims exercised, and the analytic experiment each rests on:

Claim	Analytic source	LLM prediction (§5)
Recombination assembles a genotype no parent had (vertical)	E8 (Fisher–Muller, unbounded parents)	H1
Blending conserves the single-parent level; only union realises the gain	E4 (conservation law), llm_moe	H2
Ungrounded selection → self-consumption → confident, unfit consensus	E11 (conformity mechanism)	H3
Without recombination, capability is capped at the best founder	E8 control (ρ=1) + no mutation operator here	H4
Greedy selection collapses diversity faster; QD holds it	E5, E11	H5
A capability survives inheritance only if observed often enough	E2 (per-item floor 1−e^{−mp})	H6

Explicit non-goal. Grounding here is E11's selection-channel grounding (g·fitness + (1−g)·conformity), not E2's immigration into the training mix. No verified answer ever enters any child's training data, in any arm. Fig. 1A already states that LLM-scale training-mix grounding is established in prior work and not re-run; this campaign does not change that. A negative here is evidence against the selection mechanism only.

---

1. What v1 got wrong, read through the framework

#	Fault	What the theory says	Evidence in v1	Fix (§3)
F1	Near-clone founders. 8 agents over 3 families → 3 lists-, 3 strings-, 2 arith-specialists differing only by task draws.	E8 control: recombining ρ=1 parents buys nothing (flat at 6 for any K). Pigeonhole: 4 parents from 3 families always contains a same-family pair.	no_sex ≥ full: merging near-clones is pure perturbation cost.	L = 12 disjoint families, one founder per family, ρ = 0 by construction; verified at gen 0.
F2	Combinatorial space too small. 3 skills → 2³ = 8 competence states; founders occupy 3 of them.	E8/E11 use L = 12 (4096 genotypes). The vertical claim needs room for a child to be new.	Best possible gain over a founder was tiny.	L = 12 → the best founder holds 1/12 of the space.
F3	Blending operator in the dilution regime. Sex = 2-parent linear LoRA merge at 0.5B.	E4 conservation law; llm_moe 0.5B: soup dilutes lists 0.43 → 0.26. Headroom law: dilution wherever there is room to lose.	full declined while no_sex held.	Reproduction by union-preserving recombination (confidence-routed union of parents' answers → distil). Linear merge kept as an explicit control arm (sex_linear) — H2.
F4	Truncation before breeding. Top-4 of 8 selected as parents; children bred only from them.	E11 selects on survival over the pooled parents + offspring, never on breeding eligibility. Truncating parents discards half the alleles at gen 1 with no mutation operator to restore them (E6: loss is permanent).	Half the families were unreachable after gen 1.	Survival selection over the pool (§3.4). Elitism becomes emergent, as in E11.
F5	Underpowered evaluation. n_test = 40 → SE 0.079 per measurement.	—	Every contrast except vs no_grounding sat inside one SE.	n_test = 240 (20/family) → SE 0.032 overall.
F6	Weak grounding contrast. g = 0.5 vs E11's 0.85; G = 10 vs 80.	The conformity gap in E11 needs the population to converge; g = 0.5 leaves conformity with half the vote even in full.	Gap flat in all arms.	g = 0.85; G = 12 (§7 explains why 12 suffices here).
F7	Transmission floor never measured. n_inherit = 600 over 3 families chosen by feel.	E2: an item survives only if it is observed enough in the inheritance sample — the per-item floor. Pilot v1 measured a ~25%/gen "distillation tax" and fixed it by doubling data, without asking where the floor was.	—	Calibration C2 measures the retention curve and sets n_inherit from it.

---

2. Theoretical predictions → design constraints

Each constraint below is derived, not chosen.

2.1 Decorrelation (E8, E4). Union coverage of K parents is U = ρq + (1−ρ)(1−(1−q)^K); the gain over a single parent is proportional to (1−ρ). Founders must therefore be as decorrelated as the task space allows: one family each, no shared training items, and the gen-0 behavioural-distance matrix must show no pair below 0.5 disagreement (gate C1c).

2.2 Combinatorial headroom (E8). With one family per founder, q = 1/L. The doubling bound for 2-parent recombination gives ≥ ⌈log₂ L⌉ = 4 generations to reach full coverage under lossless inheritance; with per-generation retention r per family the plateau is set by r, not L. So L = 12 gives headroom; G must exceed 4 by enough to see the plateau: G = 12.

2.3 Operator (E4, llm_moe, llm_directed). At 0.5B on unsaturated families the framework predicts linear blending dilutes and union preserves. The society's reproduction operator must be union-preserving or the experiment re-measures a known result. The union is implemented in the inheritance data, not in weight space: for each inheritance prompt the child learns the answer of whichever parent is more confident (mean token log-probability of its own answer). This is E4's max operator applied per item, it is verifier-free (legal in the no_grounding arm), and it is directed sex in E10's sense — mate choice by complementarity plus per-item selection. Gate C4 checks that confidence tracks competence (the routing precondition); gate C3 checks the union child beats the linear child on a single 2-founder cross before any campaign money is spent.

2.4 Selection acts on survival (E11). E11 pools N parents with n_off offspring and keeps the top N by score + novelty·λ. Reproducing that exactly gives: elitism for free (a strong parent survives by out-scoring its children), no gen-1 truncation, and a directly comparable selection intensity (keep 12 of 24 = top ½; E11 keeps 60 of 180 = top ⅓ — pre-noted as a difference).

2.5 Conformity must be decoupled from truth for H3 to be testable. E11 initialises random genotypes, so its consensus is uninformative at gen 0. In the LLM, consensus is the modal answer over agents. With one expert per family, 11 of 12 agents answer any given family's prompt at roughly base level, so the modal answer ≈ the base model's answer, and conformity rewards being base-like. Prediction: consensus accuracy at gen 0 ≈ base overall (gate C5 measures it; it must be < 0.35, i.e. well below the best founder's own-family accuracy, otherwise conformity is a truth proxy and the no_grounding arm cannot fail by the predicted mechanism — see §6 F-alt).

2.6 The inheritance floor (E2). A family survives distillation only if the child sees enough correct examples of it. Retention as a function of examples-per-family, r(k), is measurable (C2). Set k* = min k with r ≥ 0.85, n_inherit = L·k*. Consequence for a diluted skill at accuracy a: effective correct examples = a·k*, so skills are lost at the distillation step, not the merge step (H6, a mechanism prediction that distinguishes this account from "merging destroyed it").

2.7 Headroom, not size (the llm_moe_hard lesson). Families must be unsaturated for the specialist (≤ 0.9) and non-trivial for the base (≥ 0.05, ≤ 0.4) at 0.5B. C1 rejects families outside that band. The three existing easy families straddle it (base lists 0.15 / strings 0.15 / arith 0.53; specialists 0.43 / 1.00 / 0.91) — strings saturates, arith's base is high. Both are candidates for replacement, decided by C1 not by preference.

2.8 No cross-family conflict (E12, llm_epistasis, llm_speciation). Recombination helps on additive landscapes and hurts under functional conflict. Families must not share a prompt shape with different answer conventions (e.g. two families that both say "sort" meaning different orders). Gate C1b: pairwise confidence-weighted functional conflict between founders (the llm_epistasis measure) below the grid's no-conflict axis level.

---

3. Design

3.1 Base and agents. Qwen2.5-0.5B-Instruct (frozen). Agent = LoRA r=16 α=32 on attention + MLP projections. N = 12.

3.2 Task space. L = 12 disjoint procedurally generated families, exact-match verifier, chosen from the candidate list in §4 C1 by the calibration band. Fixed pools per family: test 20 (reporting only), val 10 (grounded selection signal), conformity 10 fresh per generation (prompts only). Inheritance pool: k* per family, fresh per generation.

3.3 Founders. Agent i is a specialist on family i, trained on 600 × 3 epochs of that family only, seed master·100 + i. Gate C1c: gen-0 pairwise behavioural distance ≥ 0.5 for every pair.

3.4 One generation.

1. Produce & score. Every agent in the current population answers test, val, and conformity pools. Consensus = modal normalised answer per conformity prompt. fitness = val accuracy; conformity = agreement rate with consensus; novelty = mean behavioural distance to the rest. score = g·fitness + (1−g)·conformity, g = 0.85 (0 in no_grounding).
2. Mate. Form n_off = N pairs by complementarity (greedy maximum behavioural distance, each agent used at most twice), restricted to the current population.
3. Recombine (sex ON). For each pair, each parent answers the inheritance pool once (with mean token log-prob). Union child data: per prompt, the answer of the more confident parent. In sex_linear: 6 Dirichlet-weighted linear LoRA merges screened on the arm's own signal (val fitness, or conformity in ungrounded arms), winner answers the pool. Sex OFF: the child's data is a single parent's answers (parent chosen by score-proportional sampling).
4. Inherit. Each child is a fresh LoRA trained from the base on its data, epochs from C2.
5. Survive. Pool = N parents + N children. Keep the top N by score + λ·novelty (λ = 0.3; λ = 0 in no_diversity). Scores for children computed after training, on the same pools.

3.5 Arms (5).

arm	g	recombination	survival
full	0.85	union (confidence-routed)	QD (λ=0.3)
no_grounding	0	union; routing unchanged (verifier-free)	QD on conformity
no_sex	0.85	none (single-parent redistillation)	QD
no_diversity	0.85	union	greedy (λ=0)
sex_linear	0.85	2-parent linear merge, 6 screened candidates	QD

sex_linear is the H2 control and is the first arm to drop if budget forces it (§9).

3.6 Generations and seeds. G = 12. Seeds 1–4 (fixed before running; not re-drawn). Reporting uses the verifier in every arm; the verifier enters the loop only where g > 0.

---

4. Calibration phase (local 16 GB GPU, ≈ 3–4 h, before any submission)

Each gate has a pre-set pass criterion. Fail → stop, fix, re-run the gate. No campaign until all pass.

Gate	What	Pass criterion	Cost
C1a family band	Base and specialist (600×3) accuracy on each of ~15 candidate families (100 test items each)	Keep families with base ∈ [0.05, 0.40] and specialist ∈ [0.60, 0.90]; need ≥ 12	~60 min
C1b no conflict	Pairwise confidence-weighted functional conflict between the 12 founders (from llm_epistasis)	Every pair below the compat axis level of the epistasis grid	~15 min
C1c decorrelation	Gen-0 behavioural-distance matrix on 120 mixed prompts	min pairwise disagreement ≥ 0.5	(with C1b)
C2 transmission floor	Distil a child from a founder's own answers with k ∈ {25, 50, 100, 200} examples of its family (rest of the pool mixed), 2 and 3 epochs; measure retained own-family accuracy ÷ founder accuracy	Choose k* = min k with retention ≥ 0.85 at the chosen epochs; if no k ≤ 200 passes, the design is infeasible at 0.5B — stop	~45 min
C3 operator	One cross (two founders): union-distil child vs best-of-6 linear-merge-distil child; both families' accuracy	Union child ≥ 0.85 × each parent on that parent's family; union ≥ linear on the minimum of the two. If linear ≥ union, H2 is already falsified — record it and reconsider the operator before the campaign	~20 min
C4 routing precondition	For each founder: mean log-prob on own-family answers vs off-family; AUC	AUC ≥ 0.7 for ≥ 10 of 12 founders	(with C2)
C5 consensus anchoring	Consensus accuracy over the 12 founders at gen 0, 120 prompts	< 0.35 (conformity is not a truth proxy)	~5 min

C2 also fixes the cost model (§9) — n_inherit = 12·k*.

---

5. Pre-registered hypotheses, thresholds, falsifiers

Primary outcome metric: best-agent overall test accuracy at generation G (deployed capability, elite included), reported with the best newborn (child trained that generation) alongside, so a "climb" carried by a surviving founder is visible as such. Secondary: per-family accuracy of the best agent (the competence genotype), behavioural diversity, consensus accuracy, conformity−truth gap. Reference level B₀ = best founder overall at gen 0 (≈ (0.7 + 11·base)/12 ≈ 0.24 if base ≈ 0.2; measured, not assumed).

	Prediction (from)	Quantitative threshold	Falsified if
H1 vertical climb	E8: union recombination of decorrelated one-family founders assembles multi-family agents; sigmoidal, most of the climb in gens 1–5, plateau set by r	full best-agent at G ≥ B₀ + 0.20 and best newborn at G ≥ B₀ + 0.15; best agent competent (≥ 0.6) on ≥ 6 of 12 families; in ≥ 3 of 4 seeds	best-agent gain < 0.10 in ≥ 2 seeds
H2 operator	E4 conservation law in the dilution regime	full − sex_linear ≥ 0.10 at G (paired, per seed); sex_linear best agent competent on ≤ 3 families	sex_linear ≥ full in ≥ 2 seeds. Pre-stated regime caveat: this ordering is predicted to invert at 7B on easy tasks (headroom law); a 7B follow-up would test that, not this.
H3 self-consumption	E11 + §2.5: conformity rewards base-likeness; ungrounded selection regresses the population to base and homogenises it	no_grounding best agent at G ≤ B₀ + 0.05; conformity−truth gap (no_grounding − full) ≥ 0.30 at G; consensus accuracy in no_grounding non-increasing	no_grounding ≥ full − 0.05 on best agent, or gap difference < 0.10
H4 sex necessity	E8 ρ=1 control + no skill-acquisition operator without recombination	no_sex best agent at G ≤ B₀ + 0.05 in every seed (a ceiling, stronger than E11's ~1-point effect)	no_sex gains ≥ 0.10 over B₀ in any seed → an unmodelled acquisition route exists (base competence amplified by self-distillation); report it
H5 diversity	E5/E11: greedy converges earliest; QD holds	AUC of behavioural diversity full > no_diversity in ≥ 3 seeds; no_diversity diversity < 0.1 by gen ≤ 6. Low power on best fitness pre-declared (E11: 0.78 vs 0.74)	no ordering in diversity AUC
H6 where skills die	E2 floor: loss occurs at distillation when correct examples/family < k*·a	For families lost between t and t+1 in full, the source (union answer set) accuracy on that family at t is ≥ 0.6 in ≤ 20% of cases — i.e. skills that were competently supplied are retained; skills die because they arrived diluted	≥ 40% of lost families were supplied at ≥ 0.6 → the distillation channel itself is lossy beyond the calibrated floor; revisit C2

Analysis is per-seed paired contrasts (4 seeds), reported as mean ± 95% CI and sign count. No metric introduced after unblinding is called a result. All rows above are also plotted whether or not they pass.

---

6. Anticipated failure modes and how each is handled

• F-alt (conformity anchored to truth). If C5 shows consensus accuracy ≥ 0.35, the no_grounding arm may fail by drift rather than by confident-wrong consensus (v1 workorder falsifier 3). Then H3's gap threshold is not testable; run anyway, report the observed signature, and say so.
• Terminal degeneration. A source that emits < 8 usable answers: copy the parent unchanged (existing sentinel). Count and report occurrences per arm.
• Family extinction is permanent (E6). No mutation operator reintroduces a lost family. Pre-state: the number of families alive in the population is itself a reported curve; full is predicted to hold ≥ 10 of 12 to G, no_diversity fewer.
• Router failure (C4 fails). Fall back to self-consistency routing (two samples, prefer the parent whose answers agree); re-run C4. If still failing, the union operator has no verifier-free implementation at this scale — record and consider 7B.
• Elite lock-in. A founder that survives to G on score alone makes "best agent" flat. Best newborn is co-primary for exactly this reason.
• Screening noise in sex_linear. 6 candidates on 120 val items (SE 0.046) — adequate for choosing among merges that differ by ≥ 0.1, which is the dilution scale.
• Queue / wall-time loss. Per-generation checkpoint (rows flushed to parquet; adapters on disk) and --resume; each PBS array element = one (seed, arm), ≤ 6 h.
• Environment drift. Never uv sync on a machine with a running job (tasks/lessons.md).

---

7. Power

SE of an overall accuracy at 240 items, p ≈ 0.5: 0.032. Per-family at 20 items: 0.11 (per-family readouts are descriptive only). Predicted effects: H1 ≥ 0.20, H2 ≥ 0.10, H3 ≥ 0.20 on best agent and ≥ 0.30 on the gap, H4 a ceiling — all ≥ 3 SE. H5 on best fitness is predicted small and is not powered; its diversity readout is. Four seeds give a sign test at p = 1/16 one-sided for a 4/4 outcome; the primary analysis is the paired mean and CI, the sign count is descriptive.

Why G = 12 suffices: the climb needs ⌈log₂ 12⌉ = 4 doublings; the E11 no_grounding crash occurred by generation 6 of 80; diversity collapse in E11's ablations by 10–15. Twelve generations covers every predicted transition with margin to see the plateau.

---

8. Analysis and figure (fixed now)

Figure, E11 layout plus one panel: (A) best-agent and best-newborn overall accuracy per arm over generations, B₀ dashed; (B) behavioural diversity; (C) conformity−truth gap; (D) competence heat-map — families × generations for the full best agent, with sex_linear beside it. Mean ± 95% CI over seeds. Script figures/plot_llm_society.py, reading only the committed bundles.

Statistics: per-seed paired contrasts at G for H1–H4; AUC contrast for H5; the supplied-vs-retained tabulation for H6. figures/stats_llm_society.py.

---

9. Compute and schedule

Measured anchor: v1 seed (N=8, G=10, 4 arms, n_inherit=600, 3 epochs) ≈ 3.3 L40S-hours.

Per generation-arm at v2 defaults (N=12, 24-agent pool eval on 480 prompts; 12 parents answering the inheritance pool once each with log-probs; 12 children trained at 12·k* × epochs):

k* from C2	training	inheritance answers	eval	per gen-arm	per (seed, arm), G=12	campaign, 4 seeds × 5 arms
50	~6 min	~3 min	~4 min	~13 min	~2.6 h	~52 L40S-h
100	~12 min	~5 min	~4 min	~21 min	~4.2 h	~84 L40S-h

Submitted as a 20-element PBS array (seed × arm), 6 h wall-time each, checkpointed. Dropping sex_linear saves 20%. The local GPU runs one seed's full + no_grounding in parallel as a hedge.

7B. ≈ 4× per operation → ~200–340 L40S-h for the full grid: not a first shot. Pre-registered role for 7B: a 1–2 seed confirmation of full vs no_grounding and of the H2 inversion, run only if 0.5B passes H1 and H3. On the easy families 7B is predicted to be in the composition regime, so sex_linear should catch up with full there — a positive prediction of the headroom law, not a replication.

Schedule. Calibration day 1 (local). GG gate on C-results. Engineering (§10) days 1–2. Campaign submission day 2–3; wall-clock ≈ 1 day if the queue cooperates. Analysis + figure day 4.

---

10. Engineering checklist (before submission)

• tasks.py: ~15 candidate families with verifier formats (int / int-list / lowercase word / uppercase word); FAMILIES becomes config-driven.
• society.py: survival-over-pool selection replacing parent truncation; complementarity pairing over the whole population; confidence-routed union inheritance (needs generate() to return mean token log-prob); sex_linear arm; per-generation parquet flush + --resume; log the source per-family accuracy before distillation (for H6) and families-alive per generation.
• Tests for the pure pieces (union routing, pooled survival, pairing constraint) — extend the 155.
• configs/llm/society_v2_calib_*.yaml, society_v2_s{1..4}.yaml; hpc/llm_society_v2.pbs array.
• figures/plot_llm_society.py, figures/stats_llm_society.py written before unblinding, against the smoke bundle.
• Smoke: N=4, L=4, G=2, all 5 arms, exit 0, figure renders.

---

11. Outcome → manuscript

Outcome	What changes in the paper
H1 ∧ H3 pass (H2, H4, H5 whatever they are)	Fig. 1A cell "open — the stated gap" → filled; new figure (§8) enters as the LLM tier of the society; Table S2 row; the Discussion's prescriptive claim gains its LLM instantiation.
H1 fails, H2 passes	The vertical claim does not transfer at 0.5B but the operator law does: report as a bounded negative in SI with the H6 diagnosis; Fig. 1A cell becomes "tested at 0.5B: operator law holds, climb does not"; 7B confirmation becomes the open item.
H3 fails (no self-consumption signature) with C5 passed	The selection-channel grounding mechanism does not transfer; state it, keep the E11 result as biological-model-only; nothing prescriptive at LLM scale.
C-gates fail	No campaign. The gate result itself goes in the SI as the reason the tier was not run.

---

12. Decisions (GG, 2026-09-07)

1. Scale: 0.5B full grid; 7B only as the gated confirmation of §9. Decided.
2. sex_linear arm: dropped from the first campaign — founders are cached and shared, so it can be appended later at ~20% of the grid cost, and the operator law is already established by llm_moe (Fig. 3B). The code path stays (arm_settings("sex_linear")); H2 is therefore deferred, not tested, in this campaign. Four arms × four seeds = 16 array elements.
3. Family candidates: no vetoes; calibration C1 decides membership. Seventeen candidates are implemented in src/llm/families.py (the three originals + fourteen new). Word-order reversal and run-length encoding were dropped at implementation because the verifier cannot score multi-word or alphanumeric answers; sortletters, caesar and charfreq use random pseudo-words so a 600-item training set cannot cover the test space.
4. Go/no-go gate after calibration — GG reviews the C-table before anything is submitted. Standing.

4a. Calibration record and amendments

Stage A, pass 1 (2026-09-07, results/llm_society_v2_calib_a, founders 600×3). Only 6 of 17 families in band: setops, numtheory, mixedtoken, vectors, digits, alphabet. Out of band: lists and binary under-trained (specialist 0.44); strings 0.91, roman 0.97, prime 1.00 above the specialist ceiling; arith base 0.53 (the base already knows it — violates the sole-expert premise); liststats and charfreq weak (0.38; charfreq also fails the routing AUC, 0.44); sortletters 0.20, caesar 0.01, progression 0.10 unlearnable at this budget. C1b max pairwise conflict 0.348 (just under the gate), every top pair involving a failed specialist answering confidently wrong; C1c min distance 0.65. Smoke passed on all four arms (figure + stats script exercised).

Amendments before pass 2 — recorded here because they change what §4 promised:

1. Specialist upper bound 0.90 → 1.00. The bound encoded the llm_moe_hard headroom lesson, which concerns fusion composing to a ceiling so that soup matches routing. The society uses union inheritance, and the quantities under test are transmission and assembly, for which a founder at 0.97 is not a problem. The base bound (≤ 0.40) is kept strict: it protects the sole-expert premise and C5's consensus decoupling. Consequence: strings, roman, prime become eligible; arith stays out unless the count forces it (then flagged).
2. Founder budget 600×3 → 1200×3, uniform, to recover the under-trained lists and binary.
3. Prompt spaces enlarged for roman (1–999), binary (1–511), prime (≤ 400) so a 600-item training set cannot cover the test space (pass 1: 300 / 200 / 210 unique of 600).
4. Three candidates added — wordlen, lettercount, sumeven (counting and filtered sums; verifier-safe; large prompt spaces). Three dropped without retraining: sortletters, caesar, progression (specialist ≤ 0.20).

None of these touches a hypothesis, a threshold, or the campaign design; they change which families are eligible. Pass 2 is configs/llm/society_v2_calib_a2.yaml.

Stage A, pass 2 (results/llm_society_v2_calib_a2, founders 1200×3). Nine in band: strings 1.00, setops 0.85, numtheory 0.83, mixedtoken 0.74, digits 0.97, alphabet 0.97, prime 0.96, wordlen 0.92, lettercount 0.77 (specialist accuracies; all bases 0.08–0.30). Roman misses only the base floor (base 0.04, specialist 0.99). The borderline families did not converge with more training — vectors 0.61 → 0.53, lists 0.44 → 0.50, binary 0.44 → 0.22, charfreq 0.38 → 0.55, liststats 0.38 → 0.47 — so they are noise-level at 0.5B, not under-trained; sumeven is unlearnable (0.06). C1c min distance 0.68 (pass). C4: 9 of 10 candidates have confidence AUC ≥ 0.7 (mixedtoken 0.62).

C1b fails among the ten. Max pairwise conflict 0.439 (wordlen × lettercount); five pairs ≥ 0.35, all among the counting families (wordlen, lettercount, digits, strings, mixedtoken): two specialists that both confidently answer "How many …?" with different small integers is precisely E12's conflicting-convention hazard, and under confidence-routed union the wrong one can win the prompt. The largest subset with every pair < 0.35 has seven members. Dropping lettercount (the hub, 7 conflicts) leaves nine with max 0.396.

Re-deriving the gate from the grid, rather than from a midpoint. 0.35 was chosen as midway between the grid's no-conflict axis (0.20–0.26) and its conflict axis (0.46–0.52). The grid's own outcome data say where the break is: P(merge penalty > 0.02) is 0.29 for epi_conf < 0.35, 0.22 for [0.35, 0.41), and 0.77 for ≥ 0.41 (n = 17 / 9 / 13). Below 0.41 the measure does not predict a penalty; above it, it does. A gate at 0.41 is therefore the data-derived boundary, and the nine-family set passes it (max 0.396).

Options put to GG (2026-09-07 evening):

1. L = 9, gate 0.41 — strings, setops, numtheory, mixedtoken, digits, alphabet, prime, wordlen, roman. Two amendments: gate 0.35 → 0.41 (grid-derived, above), base floor waived for roman (the floor screened for unlearnable tasks; roman's specialist at 0.99 settles that). H1's family threshold scales to ≥ 5 of 9. Recommended.
2. L = 7, gate 0.35 as written — alphabet, mixedtoken, numtheory, prime, roman, setops, strings. Weaker combinatorics (128 states, ~2.8 doublings) but no gate amendment.
3. Prompt tags per family to suppress off-family confidence, then re-calibrate everything (~1.5 h). Removes the conflict by construction; also makes routing trivially lexical (the llm_moe rider).

GG decision (2026-09-07, 20:30): option 1. L = 9: strings, setops, numtheory, mixedtoken, digits, alphabet, prime, wordlen, roman. C1b gate amended 0.35 → 0.41 (grid-derived); base floor waived for roman. Consequential edits to §3/§5/§7: N = 9 agents; n_test 27/family (243 overall, SE 0.032); n_val 13/family; n_conf 13/family; H1's family threshold ≥ 5 of 9; B₀ ≈ (spec + 8·base)/9. Stage B launched on the nine (society_v2_calib_b.yaml; probe families setops / alphabet / digits; cross setops × alphabet).

Stage B, C2 transmission (results/llm_society_v2_calib_b) — FAILS as pre-registered. Retention of a founder's own family in a child distilled from the founder's own answers, k examples per family × 9 families, 2 or 3 epochs:

k	setops	alphabet	digits
25	0.31–0.38	0.30–0.34	0.59–0.69
50	0.67–0.73	0.50–0.64	~0.71
100	0.62–0.67	0.57–0.67	—
150	0.61–0.73	0.73–0.81	—

No k ≤ 150 reaches 0.85; the curve is flattening. This is not the E2 observation floor — the source supplied the family at 0.77–1.00 accuracy, so the items were observed. It is interference: a one-family founder's inheritance data is one competent family and eight families of confident garbage, and a fresh LoRA fits all nine. That is the mechanism behind pilot v1's "distillation tax", now measured at 20–40% per generation. Under the pre-registered rule the ungated design is infeasible at 0.5B.

Proposed amendment (measured before adoption, C2b — society_v2_calib_c2b.yaml): confidence-gated inheritance. The child learns only the prompts its source is confident on (exp mean token log-prob ≥ τ). Verifier-free; identical in every arm; makes the child agnostic rather than wrong off-expertise — E8's founder model. C2b measures retention by τ, the Youden τ* separating own- from off-family confidence (calibration uses family labels; the campaign uses the fixed τ), and the off-family harm of ungated inheritance. Adoption requires GG's sign-off because it changes §3.4 step 4. Implemented as conf_gate in society_v2.py (default None = ungated).

C2b (results/llm_society_v2_calib_b_transmission_conf) — the gate passes at τ = 0.5, and the mechanism is two-part. Source answers a 300-per-family pool (2700 prompts); child keeps prompts with source confidence ≥ τ; 3 epochs. Retention (child own-family ÷ founder):

	ungated	τ = 0.5	τ = 0.7	τ = 0.85
setops	0.69	0.93	0.93	0.94
alphabet	0.86	0.85	0.88	0.82
digits	0.89	0.83	0.76	—
mean	0.81	0.87	0.86	—

Two drivers, not one. (i) Pool size: at k = 300 alphabet and digits already retain ≥ 0.86 ungated (they were 0.73–0.81 at k = 150). (ii) Confidence gating rescues the family pool size does not (setops 0.69 → 0.93) and is neutral-to-slightly-negative where the specialist is confident everywhere (digits: off-family confidence median 0.67 vs 0.45 for setops — the gate discards data without separating). Youden τ* ≈ 0.95–0.97 in all three (own-family confidence median 1.00), so τ = 0.5 is a mild gate keeping 50–75% of the pool. Off-family harm: not confirmed. Ungated children score at base off-family (0.19–0.27 vs base 0.20–0.22), not below it; gated children slightly above. Interference costs own-family retention; it does not push off-expertise competence below the prior.

Adopted for the campaign (pending GG go/no-go on the full table): k_inherit = 300, conf_gate = 0.5, epochs = 3. Mean retention 0.87 ≥ 0.85 meets the pre-registered C2 criterion under the amended channel. Cost consequence (§9): per-generation inheritance answers 2700 prompts per parent and children train on ~1300–2000 kept examples → ≈ 30 min per generation-arm, ≈ 6 h per (seed, arm) at G = 12, ≈ 96 L40S-h for 16 elements; PBS walltime raised to 8 h.

Stage B, C3 cross setops × alphabet (results/llm_society_v2_calib_b_cross) — operator half passes, retention half fails for C2's reason. Parents 0.84 / 0.96. Union child 0.57 / 0.62 — holds both families, routed 53% of prompts to the alphabet parent. Linear-blend child (best of 6 screened) 0.66 / 0.30 — keeps one family and loses the other: E4's dilution, in the operator the campaign dropped. Union ≥ linear on the minimum (0.57 vs 0.30) ✓. Union ≥ 0.85 × parent ✗ (0.68× / 0.65×) — the same transmission tax as C2. The cross is re-run under the confidence gate if C2b passes (conf_gate is now a cross-stage option; both children gated by their own source's confidence).

C3 re-run under the gate (results/llm_society_v2_calib_b_cross_gated, k = 300, τ = 0.5) — fails for a NEW reason. Union child setops 0.46 / alphabet 0.73 (0.55× / 0.76× of parents) — both held, both diluted. Linear child 0.80 / 0.22 — one skill at 0.95×, the other lost. Read together with C2b (a one-skill child retains 0.87–0.93 under the same gate and budget): a single skill transmits; a two-skill child dilutes each skill by 25–45% even from union-preserved data. This is E4's conservation law relocated from the sample budget to the learning budget — a fixed adapter (r = 16) and fixed epochs split across skills. Under it H1 (≥ 5 families at ≥ 0.6 in one agent) is predicted to fail by construction, whatever the operators do. F8 in the fault ledger.

C3b (before deciding anything): does capacity or budget lift the two-skill child? Three variants of the gated cross — 6 epochs at r = 16; r = 64 (α = 128) at 3 epochs; both. Pass criterion as C3: union child ≥ 0.85 × parent on each family. If one passes, the campaign adopts that child budget (cost re-estimated). If none passes, the vertical claim cannot be tested at 0.5B with self-distilled inheritance, and the honest options are a reduced campaign (H3–H5 only, which do not need multi-skill children) or 7B.

C3b results (results/llm_society_v2_calib_c3b_*; union child accuracy and ×parent):

child budget	setops	alphabet	verdict
r16, 3 ep, τ 0.5 (C3 gated)	0.46 (0.55×)	0.73 (0.76×)	fail
r16, 6 ep, τ 0.5	0.73 (0.87×)	0.79 (0.82×)	at the gate within noise (SE ≈ 0.06 on the ratio)
r64, 3 ep, τ 0.5	0.72 (0.90×)	0.58 (0.67×)	fail; r64 founders weaker (0.80/0.87) and less confident (35% routed to alphabet)
r64, 6 ep, τ 0.5	0.35 (0.44×)	0.50 (0.57×)	fail — overfits

Budget, not capacity, is the lever; rank stays 16. The kept count explains the residual: at τ = 0.5 the union child kept 1941 of 2700 prompts, of which only ~600 are its two competent families — the mild gate passes the max of two parents' confidences, so ~70% of the child's data is confident garbage. C2b's own table had τ = 0.85 as the best mean retention (0.88) at a third of the data. C3c (last calibration run): the cross at τ = 0.85, 3 and 6 epochs.

Prediction update carried into the go/no-go, whatever C3c says. A two-skill child retains ≈ 0.85× per skill at best; H1 as written (best agent ≥ B₀ + 0.20, ≥ 5 of 9 families at ≥ 0.6) needs five or six skills co-resident at ≈ 0.6 in one r = 16 adapter, which the calibration does not support at 0.5B. The realistic bar the calibration does support — H1′: children holding 2–3 families beat every founder on overall accuracy (≥ B₀ + 0.05) and the best agent climbs monotonically for ≥ 3 generations — is recorded now, before the campaign, as the primary vertical readout, with H1 kept as the stretch criterion. H3, H4, H5 do not need multi-skill agents and are unchanged.

C3c results (results/llm_society_v2_calib_c3c_*, τ = 0.85): 3 epochs → union child 0.72 / 0.79 (0.86× / 0.82×), kept 1041 of 2700; 6 epochs → 0.69 / 0.75. The tight gate reproduces the 6-epoch mild-gate retention at half the training, and more budget beyond that buys nothing: a two-skill child plateaus at ≈ 0.85× / 0.8× of its parents. Final child budget: τ = 0.85, 3 epochs, k = 300, r = 16. Cost re-estimate (§9): inheritance answers (9 parents × 2700 prompts with log-probs) now dominate at ≈ 11 min per generation-arm; training ≈ 9 min; evaluation ≈ 3 min → ≈ 25 min per generation-arm, ≈ 5 h per (seed, arm), ≈ 80 L40S-h for 16 elements. Walltime 8 h.

Stage B, C5 consensus (results/llm_society_v2_calib_b_consensus) — passes. Consensus accuracy over the nine founders at gen 0 = 0.31 (< 0.35); corr(conformity, own accuracy) = +0.29; min pairwise distance 0.63. Conformity is not a truth proxy here, so H3 is testable by the predicted mechanism.

12a. Go/no-go (GG, 2026-09-07, 21:30): NO-GO at 0.5B; plan 7B.

Grounds: the calibration passed C1, C4, C5, and — under the amended inheritance channel — C2, but C3's retention half exposed a ceiling no budget moves: a two-skill child holds each skill at ≈ 0.85× / 0.8× of its parents, and the vertical claim needs five or six skills co-resident in one r = 16 adapter. At 0.5B the society experiment could test H3–H5 but not the claim the paper's stated gap is about. Nothing is submitted. What today produced is a measured transmission ceiling for self-distilled LoRA inheritance at 0.5B — three mechanistically distinct limits (near-clone founders; interference from confident off-expertise answers, 20–40%/generation, removable by a confidence gate; the multi-skill learning-budget plateau) — and it goes in the SI as the reason the 0.5B tier was not run (§11, row 4).

13. The 7B plan (for GG review; nothing runs without a go)

What changes at 7B, and why it is not a re-run. The premise "one founder is the sole expert on its family" requires a base that cannot do the family. Qwen2.5-7B-Instruct already scores 0.99 on easy arith, 0.69 on easy strings, and will be high on roman / binary / setops / digits; the base bound (≤ 0.40) will exclude most of the current nine. The hard variants exist for three families only (7B base: lists 0.34, strings 0.67, arith 0.48). Phase 0 is therefore task design: ≥ 9 disjoint families with 7B base ≤ 0.4 and specialist ≥ 0.8 — multi-step, cipher, and compositional variants of the current generators — plus the C1 band and conflict gate re-run at 7B. This is a day of work before any GPU time, and it cannot be done on the 16 GB local card (7B training needs the L40S), so every calibration step goes through the CX3 queue (183 queued at last check).

Cost anchors (L40S, from llm_merge_hpc / llm_moe_hpc / llm_hard): 7B generation ≈ 10 prompts/s (0.5B ≈ 40); 7B LoRA SFT ≈ 15 example-passes/s (0.5B ≈ 60). Per generation-arm at N = 9, k = 300, τ = 0.85, 3 epochs: inheritance answers 9 × 2700 / 10 ≈ 40 min; evaluation 18 × 373 / 10 ≈ 11 min; training 9 × (≈ 1000 × 3 / 15) ≈ 31 min → ≈ 80 min per generation-arm.

scope	elements	G	per element	total L40S-h	walltime
Phase 1 — calibration A + B at 7B	2 jobs	—	~2 h each	~4	4 h
Headline: full vs no_grounding, 3 seeds	6	10	~13 h	~80	2 × 8 h with resume, or one 16 h
H3 + H4: 3 arms, 3 seeds	9	10	~13 h	~120	as above
Full grid: 4 arms, 4 seeds	16	12	~16 h	~260	3 × 8 h with resume, or one 24 h

Checkpoint/resume already makes multi-requeue elements safe. A vLLM generation path would cut the dominant 40-minute term by 5–10× but adds a dependency and a second code path; noted, not proposed.

Gates carried over unchanged: C1 band (base ≤ 0.40 strict, specialist ≥ 0.60; upper bound 1.0), C1b conflict < 0.41, C1c distance ≥ 0.5, C2/C2b retention ≥ 0.85 (gated channel), C3 union ≥ 0.85× per family on a two-founder cross, C4 AUC ≥ 0.7 for ≥ 8 of 9, C5 consensus < 0.35. The 7B-specific prediction that decides whether to proceed past Phase 1: with the larger adapter margin at 7B, the two-skill cross should clear 0.85× on both families at 3 epochs. If it does not, the multi-skill plateau is not a 0.5B artefact and the vertical claim should be pursued with a different inheritance channel (e.g. inheriting weights, not answers — which is what llm_merge_hpc already showed composes at 7B) rather than with more scale.

Hypotheses: H1 restored as written (≥ 5 of 9 families at ≥ 0.6, ≥ B₀ + 0.20) — that is the point of going to 7B; H1′ kept as the fallback readout; H3–H6 unchanged; H2 deferred.

Decisions for GG before Phase 0 starts: (i) scope row from the table; (ii) whether Phase 0 task design is worth the day, given the alternative in the prediction paragraph above; (iii) whether the SI text for the 0.5B ceiling (§11 row 4) is drafted now or after 7B.

14. Build log

• 2026-09-07 — families.py (17 candidates, all self-verifying and deterministic), society_ops.py (pooled survival, capped complementary mating, confidence-routed union, score-proportional single parent), society_v2.py (kind: llm_society_v2; per-generation checkpoint + resume; founder lock for concurrent arm-jobs; H6 source diagnostics; families-alive), calibrate.py (kind: llm_society_calib, stages families / transmission / cross / consensus), configs (society_v2_calib_a/b, society_v2_smoke), hpc/llm_society_v2.pbs (16-element seed × arm array), figures/plot_llm_society.py (the §8 layout, written before unblinding), 9 new pure tests (164 green). v1 code path untouched and still green. Smoke → calibration A launched locally.