"""Supplementary-figure inventory: key -> (source PDF under results/, caption). Keys are cited in si.md as `*(FIG:s)*` in numerical order, so the LaTeX figure counter (prefixed "S" in si.tex) reproduces the numbers the main text quotes. build.py copies each source into figs/si/. Captions are LaTeX-ready (maths in $...$, percent escaped) and follow the main-figure format: a headline sentence, the set-up in plain words, then the panels by letter with the key number and its reading. """ SI_FIGURES: dict[str, tuple[str, str]] = { "s1": ("results/architectures/architectures.pdf", "Collapse, and its arrest by real data, in three kinds of generator. The generational loop of " "Fig.~2 (train a child only on its parent's output, with or without 5\\% real data) is run with an " "exact frequency count (a histogram, no network), a recurrent network and a feed-forward network, " "on a synthetic universe of 256 knowledge items whose true frequencies are known exactly; 200 " "samples per generation, 22 generations, 5 replicates. (A) Distance from the true distribution " "(forward KL divergence, which grows the more of the truth a model fails to cover) against " "generation: solid lines, with no real data, climb in every architecture; dashed lines, with 5\\% " "real data, stay low. (B) The same distance at the end of the run (error bars over replicates): " "real data lowers it in all three. (C) The fraction of rare items still alive at the end: real data " "raises it in all three. The histogram's bars in C are small because a frequency count drops a " "rare item outright once it is unseen, whereas the networks keep some alive by smoothing (the " "subject of Fig.~S2). A variational autoencoder was excluded because it failed the generation-0 " "fidelity check on this task."), "s2": ("results/kernel_sharpen/kernel.pdf", "Why trained networks deviate from the ideal copier, in opposite directions. The inheritance model " "assumes a child's frequencies are exactly those it sampled from its parent. Two knobs are added to " "that copying step: a smoothing knob (a small pull toward treating every item as possible; mutation " "rate $u$) and a sharpening knob (a temperature $\\tau < 1$ that concentrates probability on the " "commonest items). Blue: the ideal copier; red: the copier with one knob turned; green dashed: the " "level the real trained network reached with no real data; 24 replicates per regime. (A, B) The " "image network of Fig.~2 (6,000 samples per generation, 30 items): heterozygosity (A) and the number " "of distinct items still produced (B) against generation. The ideal copier barely drifts at this " "sample size, yet the real network collapsed to a single item; sharpening at $\\tau = 0.8$ " "reproduces the collapse. (C, D) The recurrent network (200 samples, 256 items): heterozygosity (C) " "and forward KL divergence (D). The ideal copier drives diversity to zero, yet the real network keeps " "a floor near 0.68; smoothing at $u = 0.006$ reproduces the floor, though it overshoots the " "network's divergence (about 5 against 1.8), so the network's implicit prior is closer to the truth " "than a uniform one. In population-genetic terms the smoothing knob is recurrent mutation (variants " "appear without being inherited) and the sharpening knob is positive frequency-dependent selection " "(the majority gains, nothing new appears); a trained network behaves as drift plus one of these two " "biases, set by its architecture."), "s3": ("results/E6/E6.pdf", "Re-baselining a collapsed population locks in its losses. A tempting shortcut is to declare a " "model's current output the new reference and discard the original data. In the inheritance model " "(500 items, 200 samples per generation, 400 generations, 100 replicates) the population's current " "frequencies are frozen as the new grounding reference at generations 150 and 300 (dotted " "verticals) and the original truth is kept only for measurement. Four arms: re-baseline while still " "diverse, under generous real data (green); re-baseline after collapse, under starved real data " "(red); the same starvation with re-baselining allowed only while heterozygosity is above 0.75 " "(blue); never re-baseline (grey). (A) Distance from the original truth against generation (bands " "over replicates): the red arm steps up at each re-baselining and never returns; the healthy arm " "shows small steps; the gated and never arms coincide. (B) Heterozygosity, with the gate's threshold " "dashed: the gated arm never re-baselines because it stays below the line. Once rare knowledge is " "gone from every copy it cannot be rebuilt (Muller's ratchet); a diversity gate prevents the " "shortcut from making the loss permanent."), "s4": ("results/E2/E2.pdf", "The full real-data sweep in the inheritance model (the experiment summarised in Fig.~2B). 1,000 " "knowledge items with a long tail of rare ones, 200 samples per generation, 500 generations, 100 " "lineages; each generation also receives $m$ fresh real samples, so the real-data share is " "$g = m/(n+m)$, swept from 0 to 0.4. (A) Heterozygosity against generation, one line per $g$: with " "no real data it declines steadily; with any real data it levels off. (B) The level it settles at " "against $g$ (points, simulation) with the exact prediction (dashed) and the real data's own " "diversity (dotted); the red line marks $g^* = 0.048$ (95\\% CI 0.047--0.050), where 95\\% of the " "real data's diversity is kept. The hollow point at $g = 0$ has not converged (its equilibrium is " "zero). (C) The fraction of the rare tail retained, counted by items (red) and by their share of " "the truth (purple): both rise with $g$ but stay below 0.1 even at $g = 0.4$. (D) Survival by band " "of rarity, from the rarest (band 0) to the least rare (band 3): the rarest recover last. Overall " "diversity is cheap to protect; a rare item persists only once about one real example of it arrives " "per generation, so protecting it costs about one over its frequency in real samples."), "s5": ("results/E3/E3.pdf", "Real data protects only the topics it covers. The 1,000 items are divided into ten topics " "(regions) and the same total budget of real data is spent either evenly over all ten or " "concentrated on one topic the experimenter wants to protect; 200 samples per generation, 400 " "generations, 100 replicates. (A) The fraction of that topic's rare items still alive against " "generation, with real data aimed at it (blue) or spread evenly (red), bands 95\\% CI: aimed " "grounding holds about half the topic's rare items, spread grounding lets it fall to about 0.07. " "(B) Survival per topic at the end, same colours, the protected topic marked by the dotted line: " "aimed grounding protects its topic and leaves the others with no surviving rare items; spread " "grounding gives every topic the same low survival. Per-topic heterozygosity is confounded by how " "much of the truth each topic carries, so rare-item survival is the readout. A fixed budget of real " "data should be aimed at the knowledge one wants to keep."), "s6": ("results/grounding/grounding.pdf", "The real-data response in a trained recurrent network. The sweep of Fig.~S4 repeated in a " "recurrent generator rather than the exact simulation: 256 items, 200 samples per generation, 30 " "generations, $g$ swept over nine values from 0 to 0.2, 18 replicates. (A) Distance from the truth " "(forward KL divergence) against generation, one line per $g$: more real data suppresses the " "climb. (B) The final distance against $g$ (error bars 95\\% CI), falling steadily from 2.08 with " "no real data to 0.75 at $g = 0.2$ (paired $t = 3.3$ at $g = 0.2$). (C) The fraction of the " "achievable improvement each $g$ buys: half of it arrives by $g = 0.040$ (red line; bootstrap 95\\% " "CI 0.004--0.116 shaded), close to the simulation's $g^* = 0.048$ (black dashed), but the full " "improvement needs $g$ near 0.19. (D) Three ways of measuring collapse on one 0--1 scale: " "heterozygosity relative to the truth (blue) is flat near 0.8; the count of surviving rare items " "(orange) rises and falls with no pattern; the divergence-based recovery (green) rises cleanly. The " "direction of the effect matches the simulation, the threshold softens, and counting surviving items " "is the wrong ruler for a smoothing network, which keeps inventing rare items that are not in the " "truth; distance from the truth is the measure used for such networks."), "s7": ("results/mnist_collapse/mnist_collapse.pdf", "Collapse and rescue on real handwritten digits, in numbers (the experiment whose drawings are in " "Fig.~2A). A convolutional variational autoencoder is retrained from scratch each generation on the " "previous generation's drawings plus a fraction $g$ of real MNIST digits; the 30 kinds of digit " "(digit $\\times$ stroke thickness, resampled to a long tail with about 18 rare kinds) are read out " "by a frozen classifier plus a thickness measure at 98.5\\% accuracy. Two arms, $g = 0$ (red) and " "$g = 0.1$ (green); 6,000 drawings per generation, 15 generations, 4 replicates, bands 95\\% CI. " "(A) Distance from the truth rises from about 0.5 to about 18 with no real data and stays near the " "floor with 10\\%. (B) The number of distinct kinds still drawn falls from 30 to about 1 with no real " "data; with 10\\% all 30 survive (dotted line). (C) The share of the rare kinds still alive falls to " "0.06 with no real data; with 10\\% all of it is kept. (D) Heterozygosity falls to zero with no real " "data and stays near 0.9 with 10\\% (the truth's value dotted). Everything the simulation predicts " "appears on real images with an independent judge; the dose of real data needed is about twice the " "simulation's, for the reason shown in Fig.~S2."), "s8": ("results/E4/E4.pdf", "Averaging several parents cancels the benefit of having several; keeping each parent's strongest " "contribution does not. In the inheritance model (500 items) $K_T$ parents each remember a random " "share of the rare items, with the similarity of their shares controlled directly by a correlation " "$\\rho$ (0 fully complementary, 1 identical); $K_T \\in \\{1, 2, 3, 5\\}$, " "$\\rho \\in \\{0, 0.25, 0.5, 0.75, 1\\}$, 200 replicates. A child is built either by averaging the " "parents' output frequencies or by keeping, for each item, the largest frequency any parent gives it " "(a union), and then resamples as every generation does. (A) The fraction of the rare tail held by " "at least one parent against $\\rho$, one curve per $K_T$: points are simulation, lines an exact " "formula, and they match. (B) The fraction that survives in the child: solid lines (union) rise with " "more and less similar parents; dashed lines (averaging) stay flat near 0.08 whatever the number of " "parents. (C) The same at $\\rho = 0$ against the number of parents (error bars 95\\% CI). Averaging " "dilutes each rare item by the number of parents, which exactly cancels the gain of having more " "parents to draw on (blending inheritance, the scheme Jenkin showed would swamp rare variants); the " "union realises the gain, and needs a judge to say which parent holds each item."), "s9": ("results/E8/E8.pdf", "Many complementary parents can produce an offspring better than any of them. A capability is a " "string of twelve yes/no positions (a genotype of twelve loci) and fitness is the number of correct " "positions; each parent is a specialist, confident and correct (0.9) on the positions it has " "mastered and unsure (0.45) elsewhere, and no parent has mastered them all. Which positions a parent " "masters is drawn so that the number of parents $K_T$ and their correlation $\\rho$ are independent " "knobs; the deployed capability is the fitness of the most probable genotype; 40 replicates, error " "bars 95\\% CI. (A) Capability against the number of parents when parents master different positions " "($\\rho = 0$): position-wise recombination (red) reaches the perfect score of 12 with eight " "parents; the best single parent (grey) sits near 8.7; the average of the parents (blue) reaches " "about 11.6 at twelve parents. (B) Recombination against the number of parents at " "$\\rho \\in \\{0, 0.5, 1\\}$: complementary parents climb to the optimum, identical parents stay " "flat near 6. This is the Fisher--Muller effect, unbounded because a model population is not limited " "to two parents; Fig.~3B is its counterpart in language models."), "s10": ("results/E9/E9.pdf", "When skills are entangled, blind recombination harms the offspring. The twelve-position genotypes " "now sit on a rugged landscape (Kauffman's NK model) in which a position's value depends on its " "neighbours, with ruggedness $K$ from 0 (positions independent) to 8 (highly entangled). Parents are " "local optima found by hill-climbing, the model of a trained specialist; offspring are made from them " "at recombination rates from 0 (copy a parent) to 0.5 (free shuffling); 24 replicate landscapes, 200 " "offspring per point. (A) Mean offspring fitness minus the best parent against recombination rate, " "one curve per $K$: on a smooth landscape the difference is zero; as $K$ grows the curves fall, more " "steeply at higher rates, to about $-0.23$ at $K = 8$ under free recombination. (B) The fitness of " "the best offspring in a brood (parental level dotted): on rugged landscapes it peaks at an " "intermediate rate and falls back toward the parents under free shuffling. This is outbreeding " "depression; the optimal amount of recombination shrinks as skills become more entangled."), "s11": ("results/E10/E10.pdf", "Choosing mates and screening offspring rescues recombination on rugged landscapes. On the " "landscapes of Fig.~S10 three strategies are compared, all reported as deployed fitness in " "$[0, 1]$; 24 replicate landscapes, error bars 95\\% CI: the best single parent (grey); random " "recombination, as in biology (blue: random parents, free recombination, offspring taken as they " "come); and directed recombination, which a model population can do and a living one cannot (red: " "complementary parents chosen, many offspring generated at rate 0.2, the fittest kept, for five " "rounds). (A) Capability against ruggedness $K$ with the global optimum dotted: random recombination " "falls from 0.66 at $K = 2$ to 0.51 at $K = 10$; directed recombination tracks the best parent and " "the optimum at every $K$. (B) The same as a difference from the best parent: directed stays at or " "above zero throughout; random falls to about $-0.2$. In language models this is ``breed many " "merges, keep the best'' (Table~S2)."), "s12": ("results/E5/E5.pdf", "Selecting for the best destroys diversity; rewarding novelty preserves it. Each generation of the " "inheritance model (500 items, 200 samples per generation, 400 generations, 100 replicates, the same " "real data in every arm) now selects which items to keep, under three rules: no selection; greedy, " "keeping the items of highest true probability; and quality-diversity, which rewards an item for " "being rare as well as good, weighting item $i$ by $f_i p_i^{-\\alpha}$ with " "$\\alpha \\in \\{0.5, 1, 2\\}$. (A) Heterozygosity against generation: greedy (red) collapses within " "a few generations to about 0.01; quality-diversity at $\\alpha = 1$ (orange) and $\\alpha = 2$ " "(blue) and no selection (green) hold a plateau above 0.85. (B) The settled heterozygosity against " "$\\alpha$ (orange), with greedy (red dashed) and no selection (green dashed) as references: it rises " "from about 0.48 at $\\alpha = 0.5$ to about 0.88 at $\\alpha = 2$. (C) The number of distinct items " "alive at the end: about 9 under greedy, 14 to 28 under quality-diversity, about 32 with no " "selection. Chasing the best outputs is a directional pressure on top of drift; diversity has to be " "an objective in its own right, which is the diversity-preservation ingredient of Fig.~4D--F."), "s13": ("results/E14/E14.pdf", "Who should mate with whom: the best mating breadth narrows as skills become more entangled. " "Forty-eight agents carrying twelve-locus genotypes sit on a ring and evolve for 60 generations on " "an NK landscape of ruggedness $K \\in \\{0, 3, 6, 10\\}$; an offspring's second parent is drawn " "from a neighbourhood of half-width $\\approx bN/2$, so the breadth $b$ runs from mating only with " "neighbours ($b = 0.03$) to mating with anyone ($b = 1$), and an offspring replaces the agent at its " "position only if fitter (mutation 0.003, crossover rate 0.5, 20 replicates, error bars 95\\% CI, " "breadth on a logarithmic axis). (A) The best fitness reached, relative to the optimum, against " "breadth, per $K$: on a smooth landscape every breadth reaches the optimum; at $K = 3$ the best " "breadth is 0.6, at $K = 6$ and 10 it is 0.35, and mating with everyone falls below it. (B) The " "population's mean fitness rises with breadth at every $K > 0$. (C) Standing diversity (mean " "pairwise Hamming distance) falls with breadth, fastest on rugged landscapes. Wide mixing spreads a " "good variant fast but homogenises the population, so on entangled problems it loses the ability to " "explore several solutions in parallel (Wright's argument for structured populations)."), "s14": ("results/llm_curriculum_v5_early/curriculum_timing.pdf", "Conflict arrival does not set the timing of declines or of collapse. Four syllabi of the " "six-generation language-model population of Fig.~4 (three lineages, Qwen2.5-1.5B base, three " "training seeds each; mean $\\pm$ 95\\% CI): the rotated syllabus, the syllabus with complementarity " "peaking mid-way, and two that differ only in when the two skills with clashing answer conventions " "(BoolQ yes/no, WinoGrande 1/2) arrive, in generations 1--2 (conflict-early) or 5--6 " "(conflict-late), the four compatible skills filling the rest in rotated orders so that adapter age " "and skill count rise one per generation in all four. The filled marker on each curve is the first " "generation at which both clashing skills are present in every lineage. (A) The fraction of proposed " "merges declined per generation in the declinable arm: declines rise with generation on the same " "schedule in every syllabus (partial Spearman with generation controlled: conflict present " "$\\rho = -0.09$, 95\\% CI $-0.45$ to $0.15$; generation $\\rho = 0.45$; early and late pooled, " "$n = 36$). (B) Best-lineage accuracy over all six skills in the obligate-merge arm: the " "conflict-early population dips when the pair arrives, recovers by generation 3 and collapses from " "generation 5; the conflict-late population collapses from generation 4 with its clashing pair still " "to come. Final accuracy 0.28 (early) and 0.39 (late) against 0.80 and 0.78 for never merging. " "Moving the conflict by four generations moved neither the declines nor the collapse."), "s15": ("results/llm_curriculum_v5_cull/curriculum_cull.pdf", "Survival of the fittest did not give merging lineages the edge. The rotated-syllabus population of " "Fig.~4 with truncation selection added: after every generation's test the lowest-scoring lineage is " "re-founded from the highest-scoring one, keeping its own place in the syllabus ($N = 3$); the " "never-merge and declinable-merge arms are run with this selection (dashed) beside the same arms " "without it; three training seeds, mean $\\pm$ 95\\% CI. (A) Best-lineage accuracy over all six " "skills: all four populations end within 0.01 of each other, near 0.80 (never merge + selection " "0.804, declinable + selection 0.793, below in 3/3 seeds by $0.011 \\pm 0.003$; 0.796 and 0.792 " "without selection). (B) Population mean over the three lineages: selection lifts the mean early " "(generation 2: 0.58 against 0.48 for the unselected never-merge arm) but the final means converge. " "(C) Selection acted every generation (one replacement per generation in every selected population, " "dashed) and declines in the selected declinable arm rose with generation as before. Recombination's " "early lead (generation 1: 0.68 against 0.60) is present with and without selection and gone by " "generation 5 in both; under a syllabus that delivers every skill to every lineage, sex and selection " "each reach the same ceiling sooner and neither raises it."), "s16": ("results/llm_merge_seeds_smol/llm_smol.pdf", "The two most-cited language-model results on a second, unrelated family of base models. The " "experiments of Fig.~3B and 3C re-run, with protocol, task families, test sets and seeds unchanged, " "on HuggingFaceTB/SmolLM2-1.7B-Instruct (Apache-2.0; Llama architecture; a different laboratory and " "pretraining corpus from Qwen), shown beside the Qwen2.5-0.5B-Instruct originals; bars are means " "over training seeds with 95\\% CI, overall accuracy (lighter) and worst-family accuracy (darker). " "(A) Easy tasks, five seeds per lineage: on SmolLM2 the averaged and interference-aware merges exceed " "the best single specialist in every seed (overall $+0.049 \\pm 0.022$ and $+0.097 \\pm 0.020$; " "worst family $+0.19$ and $+0.20$), matching the Qwen margins. (B) Hard tasks, three seeds per " "lineage: routing among intact specialists beats the weight average in every seed on both families, " "by a larger margin on SmolLM2 ($+0.162 \\pm 0.036$ overall, $+0.221 \\pm 0.029$ worst family), " "where the average falls below the best single specialist in every seed. The learned router equals " "the oracle router on both families because the families are lexically separable."), }