figures: publication-ready — unified, lettered, codename-free
paper/pnas/make_figs.py re-plots every panel directly from the committed results artifacts into six single-file figures (figs/fig1..fig6.pdf): no experiment codenames or suptitles (interpretation moved to captions), bold panel letters, plain-language axis labels and legend entries, one consistent style (8pt, no top/right spines). Panels: fig1 A-B (grounding equilibrium + MNIST montage with its baked-in title cropped), fig2 A-B (blending cancellation + Fisher-Muller), fig3 A-D (outbreeding, directed recombination, mating breadth champion + diversity), fig4 A-C (society ablation trajectories), fig5 A-F (speciation: analytic curve + cliff, MLP decomposition + conflict sweep, LLM coherence + duration null), fig6 A-D (seed-replicated merging, 7B-hard routing vs averaging, predictive-test scatter, predictor comparison). build.py now places the single PDFs; captions rewritten per lettered panel; in-text panel refs updated (5B->5C-D, 5C->5E-F); stale stacked copies removed. Document 20pp -> 18pp. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
This commit is contained in:
parent
6f1f8bf172
commit
96902e87f0
26 changed files with 469 additions and 96 deletions
|
|
@ -54,9 +54,8 @@ Selection on a fitness function & Verifier-anchored selection (``reality that ca
|
|||
In the minimal model, grounding from a fixed real source is immigration into a drifting population, and the equilibrium diversity has a closed form our simulator matches exactly. That equilibrium is \emph{smooth} in the grounding fraction (there is no phase transition in aggregate diversity), so the practical number is an operational threshold, and we define it as such: under the tested population size and Zipf source distribution, \texttt{g \(\approx\) 0.05} retained most (\(\geq\)95\%) of equilibrium diversity indefinitely, with the required fraction depending on sample size, source distribution, and the chosen retention target (dependencies in SI). The engineering point survives the definition: verified real data is cheap insurance at fractions far below one. But the same analysis yields a floor the field's average-loss framing misses: under unstratified sampling from the source, a capability of rarity \texttt{p} appears in a real-data batch of size \texttt{m} with probability \texttt{1 − e\textasciicircum{}{−m\(\cdot\)p}}, so \texttt{m\(\cdot\)p \(\approx\) 1} marks roughly a 63\% chance of one example per batch: a soft observation floor, with higher confidence priced accordingly, and with distinct consequences for continuous retention, stationary occupancy, and reintroduction after loss (immigration can restore an absent item; SI separates these). Protecting the rarest knowledge under unstratified grounding is therefore priced per item at cost \texttt{\(\propto\) 1/p}; targeted or stratified sampling changes that cost, and recombination can recover rare capabilities \emph{that are still retained across complementary parents} (next section). In trained networks the \emph{sign} of the grounding response transfers everywhere we looked, with two deviations, both traced to the estimator bias above: sharp thresholds soften, and support-counting metrics decouple from truth (forward-KL is the operative collapse metric for a smoothing learner). On real images (Fig. 1B), dry self-training collapses a convolutional VAE to one mode while \textasciitilde{}10\% grounding holds all thirty (the trained model needs roughly twice the exact-operator fraction, the measured price of the estimator bias).
|
||||
|
||||
\begin{figure*}[p]\centering % fig1
|
||||
\includegraphics[width=\textwidth,height=0.63\textheight,keepaspectratio]{figs/fig1_E2.pdf}\par\smallskip
|
||||
\includegraphics[width=\textwidth,height=0.63\textheight,keepaspectratio]{figs/fig1_mnist_montage.pdf}\par\smallskip
|
||||
\caption{Collapse is drift; grounding is immigration. (A, top) The grounding response in the minimal model: an operational threshold $g\!\approx\!0.05$ retained most equilibrium diversity in the tested setting (the equilibrium is smooth in $g$), while per-item observation obeys $1-e^{-m p}$. (B, bottom) The same signs on real images: a convolutional VAE retrained each generation on its own output collapses to a single blurred mode (rows: generations), while $\sim$10\% grounding holds all thirty class$\times$style modes.}\label{fig1}
|
||||
\includegraphics[width=\textwidth]{figs/fig1.pdf}
|
||||
\caption{Grounding is immigration. (A) Stationary diversity against the grounding fraction in the minimal inheritance model: simulation (points, 95\% CI) matches the exact immigration--drift equilibrium (dashed). The equilibrium is smooth in $g$; $g \approx 0.05$ marks the operational threshold retaining 95\% of source diversity in this setting (red line, bootstrap CI shaded); the hollow point at $g=0$ is a finite-time value (the true equilibrium is zero). (B) The same signs on real images: samples from a convolutional VAE retrained each generation on its own output (rows: generations 0--15 of an ungrounded lineage) collapse toward a single blurred mode; 10\% grounding holds all thirty modes (quantified in SI).}\label{fig1}
|
||||
\end{figure*}
|
||||
|
||||
\subsection*{Recombination: a conservation law, its operators, and offspring that exceed every parent}
|
||||
|
|
@ -68,16 +67,13 @@ The generative payoff is the \textbf{Fisher--Muller effect}: recombination assem
|
|||
Sex has risks and, for AI, an unfair advantage, both quantified on rugged (epistatic) NK landscapes (60) (Fig. 3). When skills are entangled, blind recombination produces offspring \emph{below} their parents (\textbf{outbreeding depression}), worsening with ruggedness, and the optimal recombination rate shrinks as entanglement grows. But an engineered population can do what biology cannot: recombine unbounded parents, choose complementary mates, and \emph{screen many candidate offspring against a verifier before keeping one}. This \textbf{directed sex} converts the outbreeding catastrophe into a reliable gain in the model (tracking or exceeding the best parent at every ruggedness) and replicates as a sign in language models: bred-and-screened merges beat the a-priori blend in every seed on headroom tasks, including one seed where the blend failed catastrophically and selection was immune (Fig. 6A). Finally, population \emph{structure} is itself a knob: sweeping the mate-pool breadth from monogamous (local) to promiscuous (panmictic) against ruggedness, wide mixing maximises the population mean while monotonically destroying diversity, and the best \emph{champion} shifts from wide breadth on smooth landscapes to intermediate breadth on rugged ones (Fig. 3C), the mating-system phenomenon known to structured-population search, mapped onto merging populations.
|
||||
|
||||
\begin{figure*}[p]\centering % fig2
|
||||
\includegraphics[width=\textwidth,height=0.63\textheight,keepaspectratio]{figs/fig2_E4.pdf}\par\smallskip
|
||||
\includegraphics[width=\textwidth,height=0.63\textheight,keepaspectratio]{figs/fig2_E8.pdf}\par\smallskip
|
||||
\caption{Recombination: blending inheritance and the Fisher--Muller effect. (A, top) Refitting a child to the mean of its parents' output distributions conserves expected rare-item mass at the single-parent level, cancelling the multi-parent gain to first order in the rare-item regime; a strongest-source (union) operator, with renormalisation and an oracle, realises it. (B, bottom) Multi-locus recombination of decorrelated specialists assembles a genotype fitter than any parent, climbing to the optimum as parents are added, while the best single parent and the blended average plateau below.}\label{fig2}
|
||||
\includegraphics[width=\textwidth]{figs/fig2.pdf}
|
||||
\caption{Recombination in the minimal model: blending inheritance and the Fisher--Muller effect. (A) Expected rare-capability survival in a child refit from $K$ uncorrelated parents: the output-mean (blending) stays at the single-parent level --- the first-order cancellation --- while the union operator (strongest source per item, renormalised, oracle-identified) rises with parent count. (B) Multi-locus recombination of decorrelated specialists produces offspring fitter than any parent, approaching the optimum as parents are added; the best single parent and the blended average plateau below (mean $\pm$ 95\% CI).}\label{fig2}
|
||||
\end{figure*}
|
||||
|
||||
\begin{figure*}[p]\centering % fig3
|
||||
\includegraphics[width=\textwidth,height=0.42\textheight,keepaspectratio]{figs/fig3_E9.pdf}\par\smallskip
|
||||
\includegraphics[width=\textwidth,height=0.42\textheight,keepaspectratio]{figs/fig3_E10.pdf}\par\smallskip
|
||||
\includegraphics[width=\textwidth,height=0.42\textheight,keepaspectratio]{figs/fig3_E14.pdf}\par\smallskip
|
||||
\caption{Rugged (epistatic) landscapes: risk, remedy, and structure. (A, top) Outbreeding depression: blind recombination of specialists drops offspring below their parents, worsening with ruggedness; the optimal recombination rate shrinks as skills entangle. (B, middle) Directed sex --- unbounded parents, chosen mates, verifier-screened offspring --- converts the catastrophe into a reliable gain at every ruggedness. (C, bottom) Mating structure: wide (promiscuous) mixing maximises the population mean but monotonically destroys diversity; the champion-optimal mate-pool breadth narrows as the landscape roughens.}\label{fig3}
|
||||
\includegraphics[width=\textwidth]{figs/fig3.pdf}
|
||||
\caption{Rugged (epistatic) landscapes: risk, remedy, and population structure. (A) Outbreeding depression: the mean offspring of blindly recombined specialist parents falls below the best parent, more steeply the more rugged the landscape (NK ruggedness $K$) and the higher the recombination rate. (B) Screening candidate offspring against a verifier (directed recombination) restores the gain at every ruggedness where blind recombination fails. (C) Mating structure: the best champion arises at wide mate-pool breadth on smooth landscapes and at intermediate breadth on rugged ones. (D) Wide breadth monotonically erodes population diversity at every ruggedness (mean $\pm$ 95\% CI, 20 replicates).}\label{fig3}
|
||||
\end{figure*}
|
||||
|
||||
\subsection*{The society: grounding, recombination, and diversity make complementary contributions}
|
||||
|
|
@ -85,23 +81,21 @@ Sex has risks and, for AI, an unfair advantage, both quantified on rugged (epist
|
|||
Composing the operators (Fig. 4) requires one definitional distinction first. In the inheritance model, grounding is \textbf{grounded inheritance}: external samples added to the reproduction process (the data channel). In the society model, grounding is \textbf{grounded evaluation}: selection weights true fitness against conformity to the population's own consensus, \texttt{g}\(\cdot\)true-fitness + (1−g)\(\cdot\)conformity, the analogue of scoring models by the crowd's approval (the fitness channel). These are related design ideas, since both couple the lineage to a non-drifting external signal, but they are different operators, and we name them separately. In the tested society (a finite agent population on a rugged NK landscape), a four-arm ablation separates the failure modes: the full system (grounded evaluation + directed recombination + diversity-preserving selection (61)) climbs to near the global optimum while keeping its specialists; removing grounded evaluation converges the population confidently on an unfit consensus (self-consumption); removing recombination strands it on local optima; removing diversity converges it prematurely to a worse answer. Each removal fails differently; the three implementations make complementary contributions \emph{under the tested conditions}; general joint necessity is not established (alternative mutation, restart, archive, or selection schemes could alter the picture). At language-model scale this composed loop remains unbuilt; it is the paper's largest stated gap.
|
||||
|
||||
\begin{figure*}[p]\centering % fig4
|
||||
\includegraphics[width=\textwidth,height=0.98\textheight,keepaspectratio]{figs/fig4_E11.pdf}\par\smallskip
|
||||
\caption{The society: grounded evaluation, recombination, and diversity preservation make complementary contributions in the tested model. A finite agent population on a rugged NK landscape, with selection weighting true fitness against conformity. Four-arm ablation: the full system climbs to near the global optimum; removing grounding collapses the population onto a confident, unfit consensus (self-consumption); removing recombination strands it on local optima; removing diversity converges it prematurely. Each ablation fails differently.}\label{fig4}
|
||||
\includegraphics[width=\textwidth]{figs/fig4.pdf}
|
||||
\caption{The tested society: grounded evaluation, recombination, and diversity preservation make complementary contributions. A finite agent population on a rugged NK landscape; selection weights true fitness against conformity to the population consensus. (A) Best real fitness: the full system approaches the global optimum; removing grounded evaluation collapses the population onto a confident, unfit consensus; removing recombination or diversity preservation strands it lower. (B) Population diversity. (C) The self-consumption signature: conformity minus true fitness (mean $\pm$ 95\% CI, 12 replicates).}\label{fig4}
|
||||
\end{figure*}
|
||||
|
||||
\subsection*{The limit of sex: model speciation}
|
||||
|
||||
Recombination presupposes compatible parents. In biology, lineages pushed far enough apart become separate species (\textbf{reproductive isolation}) through Bateson--Dobzhansky--Muller incompatibilities: changes harmless on their own background but deleterious in combination. A merged model is exactly the exposed hybrid. We built the analytic model (Fig. 5A): hybrid fitness tracks the parents while compatible, then peels off and crashes below the ancestor; the isolation cliff arrives earlier the denser the incompatibilities; and the incompatibility \emph{count} snowballs quadratically with divergence (52). We note that a super-linear count does not by itself entail a sharp performance cliff without the count-to-effect-size link, which the analytic model supplies under its assumptions and any neural test must establish separately.
|
||||
|
||||
In trained networks, the claim must survive a known alternative: merge barriers between independently trained networks are famously \emph{coordinate artefacts}, removable by re-aligning hidden units (62); richer symmetry groups remove more (63), with known failures beyond the shared-data regime (64). We therefore aligned under the composition of permutation matching and exact per-unit rescaling (the unit symmetry group of plain ReLU MLPs, as the search space) and decomposed the barrier (Fig. 5B): two networks trained from different initialisations on the \emph{same} task have a barrier that this alignment removes essentially entirely (residual \(\approx\) 0.001, the aligned merge performing at parent level): coordinate, not functional; two networks trained on \emph{conflicting} label maps have a barrier the same alignment leaves largely unchanged (0.502 \(\rightarrow\) 0.497), with the merged model functionally dead. The tested alignment removes the same-task barrier but leaves the conflict-associated barrier intact, supporting a functional-conflict interpretation without proving optimal alignment: exact recovery of a permuted-and-rescaled copy validates a special case, so the removable share is a lower bound and the residual an upper bound. Sweeping conflict traces the cliff as hybrid fitness, 0.97 \(\rightarrow\) 0.03. The conflict floor itself is information-theoretic (no single model can satisfy contradictory conventions; SI Appendix, Proposition S2), with the framework's role being the \emph{structure around it}: which divergences generate conflict, and what moves the cliff.
|
||||
In trained networks, the claim must survive a known alternative: merge barriers between independently trained networks are famously \emph{coordinate artefacts}, removable by re-aligning hidden units (62); richer symmetry groups remove more (63), with known failures beyond the shared-data regime (64). We therefore aligned under the composition of permutation matching and exact per-unit rescaling (the unit symmetry group of plain ReLU MLPs, as the search space) and decomposed the barrier (Fig. 5 C and D): two networks trained from different initialisations on the \emph{same} task have a barrier that this alignment removes essentially entirely (residual \(\approx\) 0.001, the aligned merge performing at parent level): coordinate, not functional; two networks trained on \emph{conflicting} label maps have a barrier the same alignment leaves largely unchanged (0.502 \(\rightarrow\) 0.497), with the merged model functionally dead. The tested alignment removes the same-task barrier but leaves the conflict-associated barrier intact, supporting a functional-conflict interpretation without proving optimal alignment: exact recovery of a permuted-and-rescaled copy validates a special case, so the removable share is a lower bound and the residual an upper bound. Sweeping conflict traces the cliff as hybrid fitness, 0.97 \(\rightarrow\) 0.03. The conflict floor itself is information-theoretic (no single model can satisfy contradictory conventions; SI Appendix, Proposition S2), with the framework's role being the \emph{structure around it}: which divergences generate conflict, and what moves the cliff.
|
||||
|
||||
The strongest constraint comes from the pre-registered \textbf{emergent test}: true BDM incompatibilities are emergent (each lineage's changes harmless alone), so we let children diverge with \emph{no conflicting signal anywhere}, using complementary class specialists and divergent input conventions, to 6.4\(\times\) the base training. \textbf{No isolation emerged} (residual 0.000 throughout); instead the merge \emph{rescued} the two catastrophically-forgetting specialists (parents \(\approx\) 0.50, merge \(\approx\) 0.955, a sustained Fisher--Muller rescue). The same double result appears at the language-model tier (Fig. 5C): conflicting conventions produce \textbf{function-specific} hybrid breakdown (the merge scores below both parents on the conflicted function, while a budget-controlled design shows the disjoint skills merge unharmed), and over-training disjoint specialists 1\(\rightarrow\)12 epochs (cf. the merging literature's expert-duration effect; 65) produces no isolation at all --- the merge improves. Across every tier tested, \textbf{isolation had to be provoked by functional conflict; specialisation alone did not speciate} --- a bound on the analogy that sharpens the design rule: what breaks merging is conflicting conventions on shared circuitry, not divergence per se.
|
||||
The strongest constraint comes from the pre-registered \textbf{emergent test}: true BDM incompatibilities are emergent (each lineage's changes harmless alone), so we let children diverge with \emph{no conflicting signal anywhere}, using complementary class specialists and divergent input conventions, to 6.4\(\times\) the base training. \textbf{No isolation emerged} (residual 0.000 throughout); instead the merge \emph{rescued} the two catastrophically-forgetting specialists (parents \(\approx\) 0.50, merge \(\approx\) 0.955, a sustained Fisher--Muller rescue). The same double result appears at the language-model tier (Fig. 5 E and F): conflicting conventions produce \textbf{function-specific} hybrid breakdown (the merge scores below both parents on the conflicted function, while a budget-controlled design shows the disjoint skills merge unharmed), and over-training disjoint specialists 1\(\rightarrow\)12 epochs (cf. the merging literature's expert-duration effect; 65) produces no isolation at all --- the merge improves. Across every tier tested, \textbf{isolation had to be provoked by functional conflict; specialisation alone did not speciate} --- a bound on the analogy that sharpens the design rule: what breaks merging is conflicting conventions on shared circuitry, not divergence per se.
|
||||
|
||||
\begin{figure*}[p]\centering % fig5
|
||||
\includegraphics[width=\textwidth,height=0.42\textheight,keepaspectratio]{figs/fig5_E12.pdf}\par\smallskip
|
||||
\includegraphics[width=\textwidth,height=0.42\textheight,keepaspectratio]{figs/fig5_speciation_real.pdf}\par\smallskip
|
||||
\includegraphics[width=\textwidth,height=0.42\textheight,keepaspectratio]{figs/fig5_llm_speciation.pdf}\par\smallskip
|
||||
\caption{Model speciation across three tiers. (A, top) Analytic: hybrid fitness traces compatible $\rightarrow$ outbreeding depression $\rightarrow$ inviability; the cliff arrives earlier the denser the incompatibilities; incompatibility count snowballs with divergence. (B, middle) Trained MLPs: the merge barrier decomposed under permutation-and-rescaling alignment --- same-task/different-init barriers are coordinate artefacts (removed by alignment); conflicting-task barriers survive in full, with hybrid fitness falling 0.97 $\rightarrow$ 0.03; divergence without conflict produced no isolation, the merge instead rescuing the forgetting specialists. (C, bottom) Language models: conflicting conventions produce function-specific hybrid breakdown; over-training disjoint specialists produces none --- at every tier tested, isolation had to be provoked by functional conflict.}\label{fig5}
|
||||
\includegraphics[width=\textwidth]{figs/fig5.pdf}
|
||||
\caption{Model speciation at three tiers. (A) Analytic model: hybrid fitness tracks the parents while lineages are compatible, then falls to inviability; the denser the incompatibilities, the earlier the fall. (B) The isolation cliff: probability of hybrid inviability against divergence, by incompatibility density. (C) Trained networks: the merge error barrier between two MLPs before and after permutation-and-rescaling alignment --- the same-task/different-start barrier is a coordinate artefact (removed by alignment); the conflicting-task barrier is left essentially unchanged. (D) Sweeping the fraction of conflicting classes: the residual barrier rises while merged-model accuracy falls from 0.97 to 0.03. (E) Language models (0.5B LoRA children of a shared base): on shared ambiguous prompts each parent performs under its own convention while the merged model falls below both --- function-specific hybrid breakdown. (F) Divergence without conflict: over-training disjoint specialists from 1 to 12 epochs produces no isolation; the merged model tracks or exceeds the parents throughout.}\label{fig5}
|
||||
\end{figure*}
|
||||
|
||||
\subsection*{A controlled predictive test: functional conflict, measured pre-merge, predicts merge damage}
|
||||
|
|
@ -111,10 +105,8 @@ The framework's prediction-level claim was put to a designed test (Fig. 6C). Thi
|
|||
The supported conclusion, stated conditionally: \textbf{across this controlled grid, pre-merge functional disagreement predicted merge penalties (clustered bootstrap CIs excluding zero; held-out leave-one-condition-out \(\rho\) \(\approx\) 0.35--0.40), whereas LoRA-delta cosine and L2 showed no statistically detectable association; gradient alignment carried intermediate signal.} Head-to-head predictor differences are not individually significant at this sample size; only these baselines were tested; and with three seeds, uncertainty about seed generalisation remains substantial --- though the seed sensitivity favours the functional measures (per-seed \(\rho\) stable at +0.37 to +0.53 in each seed alone, geometry \(\approx\) 0 in every seed, gradient alignment seed-unstable at −0.11 to −0.55). Two further results bound the claim: the initial two-axis grid's best predictor was delta-cosine (\(\rho\) = +0.60) --- an overlap artefact that the compatible-overlap control was added to expose, and did (collapse to +0.03); and the pre-registered internal prediction that confidence weighting would beat raw disagreement \textbf{failed} (they are statistically indistinguishable as rank predictors), so the present evidence favours functional disagreement generally, not the DMI-specific refinement. The framework motivated the measurement and the controls; their success does not validate the specifically population-genetic mechanism. Whether the prediction improves a budget-matched operator choice, and whether it generalises to unfamiliar conflict structures and real task pairs, are the experiment's open front.
|
||||
|
||||
\begin{figure*}[p]\centering % fig6
|
||||
\includegraphics[width=\textwidth,height=0.42\textheight,keepaspectratio]{figs/fig6_llm_seeds.pdf}\par\smallskip
|
||||
\includegraphics[width=\textwidth,height=0.42\textheight,keepaspectratio]{figs/fig6_llm_moe.pdf}\par\smallskip
|
||||
\includegraphics[width=\textwidth,height=0.42\textheight,keepaspectratio]{figs/fig6_llm_epistasis.pdf}\par\smallskip
|
||||
\caption{The language-model tier. (A, top) Seed-replicated recombination claims (fixed test sets, training seed varied, 95\% CI): merges beat every specialist; union-preserving routing and directed offspring selection beat the blend in every seed on headroom tasks, including one catastrophic blend failure they avoided. (B, middle) The headroom rule at 7B on hard (unsaturated) tasks: the weight-average dilutes a fragile specialist below the best single parent; routing preserves it. (C, bottom) The controlled predictive test: across a task grid with conflict, compatible-overlap, and duration axes decorrelated by construction, pre-merge functional disagreement predicts merge penalty (held-out $\rho \approx 0.4$) while weight-geometry baselines show no detectable association; paired predictor differences are not individually significant.}\label{fig6}
|
||||
\includegraphics[width=\textwidth]{figs/fig6.pdf}
|
||||
\caption{The language-model tier. (A) Seed-replicated merging (0.5B, five seeds, fixed test sets; mean $\pm$ 95\% CI): merged specialists exceed the best single specialist overall, and only merged models are competent on every task family. (B) Hard, unsaturated tasks at 7B (single run): the weight-average dilutes a fragile specialist below the best single parent; routing among intact specialists preserves it. (C) The controlled predictive test (13 conditions $\times$ 3 seeds): pre-merge confidence-weighted functional conflict against merge penalty, coloured by grid axis --- penalty concentrates on the conflict axis. (D) Predictor comparison, $|$Spearman $\rho|$ against merge penalty over the full grid: functional measures carry signal, the tested weight-geometry baselines do not; paired differences between predictors are not individually significant.}\label{fig6}
|
||||
\end{figure*}
|
||||
|
||||
\textbf{Table 2.} Headline quantitative results with sample sizes, uncertainty, and outcome definitions (full per-experiment tables and falsifier status in SI Appendix and per-experiment documentation).
|
||||
|
|
|
|||
|
|
@ -1,10 +1,9 @@
|
|||
r"""Build the PNAS-draft PDF from main.md (Markdown stays the source of truth).
|
||||
|
||||
Adapted from paper/arxiv/md2tex.py (same Markdown subset + pipe tables), with one addition: standalone
|
||||
`*(FIG:name)*` markers compose multi-panel figures by stacking existing per-experiment vector PDFs
|
||||
(LaTeX-level consolidation; bespoke unified figures are a submission-time polish, tracked in the work
|
||||
order). Captions define the panel letters positionally (A = top, ...) because the sub-figures carry
|
||||
their own internal panel labels.
|
||||
`*(FIG:name)*` markers place the publication figures produced by make_figs.py (unified, lettered,
|
||||
codename-free panels re-plotted from the committed artifacts). Run make_figs.py before building when
|
||||
results change.
|
||||
|
||||
Usage: python paper/pnas/build.py && (cd paper/pnas && tectonic main.tex)
|
||||
"""
|
||||
|
|
@ -20,62 +19,66 @@ HERE = Path(__file__).resolve().parent
|
|||
SRC = HERE / "main.md"
|
||||
OUT = HERE / "body.tex"
|
||||
|
||||
# figure name -> (list of source PDFs (stacked top->bottom), caption)
|
||||
# figure name -> (single publication PDF from make_figs.py, caption)
|
||||
FIGURES: dict[str, tuple[list[str], str]] = {
|
||||
"fig1": (["results/E2/E2.pdf", "results/mnist_collapse/mnist_montage.pdf"],
|
||||
"Collapse is drift; grounding is immigration. (A, top) The grounding response in the minimal "
|
||||
"model: an operational threshold $g\\!\\approx\\!0.05$ retained most equilibrium diversity in "
|
||||
"the tested setting (the equilibrium is smooth in $g$), while per-item observation obeys "
|
||||
"$1-e^{-m p}$. (B, bottom) "
|
||||
"The same signs on real images: a convolutional VAE retrained each generation on its own "
|
||||
"output collapses to a single blurred mode (rows: generations), while $\\sim$10\\% grounding "
|
||||
"holds all thirty class$\\times$style modes."),
|
||||
"fig2": (["results/E4/E4.pdf", "results/E8/E8.pdf"],
|
||||
"Recombination: blending inheritance and the Fisher--Muller effect. (A, top) Refitting a child "
|
||||
"to the mean of its parents' output distributions conserves expected rare-item mass at the "
|
||||
"single-parent level, cancelling the multi-parent gain to first order in the rare-item "
|
||||
"regime; a strongest-source (union) operator, with renormalisation and an oracle, realises it. (B, bottom) Multi-locus recombination of decorrelated "
|
||||
"specialists assembles a genotype fitter than any parent, climbing to the optimum as parents "
|
||||
"are added, while the best single parent and the blended average plateau below."),
|
||||
"fig3": (["results/E9/E9.pdf", "results/E10/E10.pdf", "results/E14/E14.pdf"],
|
||||
"Rugged (epistatic) landscapes: risk, remedy, and structure. (A, top) Outbreeding depression: "
|
||||
"blind recombination of specialists drops offspring below their parents, worsening with "
|
||||
"ruggedness; the optimal recombination rate shrinks as skills entangle. (B, middle) Directed "
|
||||
"sex --- unbounded parents, chosen mates, verifier-screened offspring --- converts the "
|
||||
"catastrophe into a reliable gain at every ruggedness. (C, bottom) Mating structure: wide "
|
||||
"(promiscuous) mixing maximises the population mean but monotonically destroys diversity; the "
|
||||
"champion-optimal mate-pool breadth narrows as the landscape roughens."),
|
||||
"fig4": (["results/E11/E11.pdf"],
|
||||
"The society: grounded evaluation, recombination, and diversity preservation make complementary "
|
||||
"contributions in the tested model. A finite agent population on a rugged NK landscape, with "
|
||||
"selection weighting true fitness against conformity. Four-arm ablation: the full system "
|
||||
"climbs to near the global optimum; removing grounding collapses the population onto a "
|
||||
"confident, unfit consensus (self-consumption); removing recombination strands it on local "
|
||||
"optima; removing diversity converges it prematurely. Each ablation fails differently."),
|
||||
"fig5": (["results/E12/E12.pdf", "results/speciation_real/speciation_real.pdf",
|
||||
"results/llm_speciation/llm_speciation.pdf"],
|
||||
"Model speciation across three tiers. (A, top) Analytic: hybrid fitness traces compatible "
|
||||
"$\\rightarrow$ outbreeding depression $\\rightarrow$ inviability; the cliff arrives earlier "
|
||||
"the denser the incompatibilities; incompatibility count snowballs with divergence. (B, "
|
||||
"middle) Trained MLPs: the merge barrier decomposed under permutation-and-rescaling alignment --- "
|
||||
"same-task/different-init barriers are coordinate artefacts (removed by alignment); "
|
||||
"conflicting-task barriers survive in full, with hybrid fitness falling 0.97 $\\rightarrow$ "
|
||||
"0.03; divergence without conflict produced no isolation, the merge instead rescuing the "
|
||||
"forgetting specialists. (C, bottom) Language models: conflicting conventions produce "
|
||||
"function-specific hybrid breakdown; over-training disjoint specialists produces none --- at "
|
||||
"every tier tested, isolation had to be provoked by functional conflict."),
|
||||
"fig6": (["results/llm_merge_seeds/llm_seeds.pdf", "results/llm_moe_hard_hpc/llm_moe.pdf",
|
||||
"results/llm_epistasis/llm_epistasis.pdf"],
|
||||
"The language-model tier. (A, top) Seed-replicated recombination claims (fixed test sets, "
|
||||
"training seed varied, 95\\% CI): merges beat every specialist; union-preserving routing and "
|
||||
"directed offspring selection beat the blend in every seed on headroom tasks, including one "
|
||||
"catastrophic blend failure they avoided. (B, middle) The headroom rule at 7B on hard "
|
||||
"(unsaturated) tasks: the weight-average dilutes a fragile specialist below the best single "
|
||||
"parent; routing preserves it. (C, bottom) The controlled predictive test: across a task grid "
|
||||
"with conflict, compatible-overlap, and duration axes decorrelated by construction, pre-merge "
|
||||
"functional disagreement predicts merge penalty (held-out $\\rho \\approx 0.4$) while "
|
||||
"weight-geometry baselines show no detectable association; paired predictor differences are "
|
||||
"not individually significant."),
|
||||
"fig1": (["paper/pnas/figs/fig1.pdf"],
|
||||
"Grounding is immigration. (A) Stationary diversity against the grounding fraction in the "
|
||||
"minimal inheritance model: simulation (points, 95\\% CI) matches the exact immigration--drift "
|
||||
"equilibrium (dashed). The equilibrium is smooth in $g$; $g \\approx 0.05$ marks the "
|
||||
"operational threshold retaining 95\\% of source diversity in this setting (red line, "
|
||||
"bootstrap CI shaded); the hollow point at $g=0$ is a finite-time value (the true equilibrium "
|
||||
"is zero). (B) The same signs on real images: samples from a convolutional VAE retrained each "
|
||||
"generation on its own output (rows: generations 0--15 of an ungrounded lineage) collapse "
|
||||
"toward a single blurred mode; 10\\% grounding holds all thirty modes (quantified in SI)."),
|
||||
"fig2": (["paper/pnas/figs/fig2.pdf"],
|
||||
"Recombination in the minimal model: blending inheritance and the Fisher--Muller effect. "
|
||||
"(A) Expected rare-capability survival in a child refit from $K$ uncorrelated parents: the "
|
||||
"output-mean (blending) stays at the single-parent level --- the first-order cancellation --- "
|
||||
"while the union operator (strongest source per item, renormalised, oracle-identified) rises "
|
||||
"with parent count. (B) Multi-locus recombination of decorrelated specialists produces "
|
||||
"offspring fitter than any parent, approaching the optimum as parents are added; the best "
|
||||
"single parent and the blended average plateau below (mean $\\pm$ 95\\% CI)."),
|
||||
"fig3": (["paper/pnas/figs/fig3.pdf"],
|
||||
"Rugged (epistatic) landscapes: risk, remedy, and population structure. (A) Outbreeding "
|
||||
"depression: the mean offspring of blindly recombined specialist parents falls below the best "
|
||||
"parent, more steeply the more rugged the landscape (NK ruggedness $K$) and the higher the "
|
||||
"recombination rate. (B) Screening candidate offspring against a verifier (directed "
|
||||
"recombination) restores the gain at every ruggedness where blind recombination fails. "
|
||||
"(C) Mating structure: the best champion arises at wide mate-pool breadth on smooth landscapes "
|
||||
"and at intermediate breadth on rugged ones. (D) Wide breadth monotonically erodes population "
|
||||
"diversity at every ruggedness (mean $\\pm$ 95\\% CI, 20 replicates)."),
|
||||
"fig4": (["paper/pnas/figs/fig4.pdf"],
|
||||
"The tested society: grounded evaluation, recombination, and diversity preservation make "
|
||||
"complementary contributions. A finite agent population on a rugged NK landscape; selection "
|
||||
"weights true fitness against conformity to the population consensus. (A) Best real fitness: "
|
||||
"the full system approaches the global optimum; removing grounded evaluation collapses the "
|
||||
"population onto a confident, unfit consensus; removing recombination or diversity "
|
||||
"preservation strands it lower. (B) Population diversity. (C) The self-consumption signature: "
|
||||
"conformity minus true fitness (mean $\\pm$ 95\\% CI, 12 replicates)."),
|
||||
"fig5": (["paper/pnas/figs/fig5.pdf"],
|
||||
"Model speciation at three tiers. (A) Analytic model: hybrid fitness tracks the parents while "
|
||||
"lineages are compatible, then falls to inviability; the denser the incompatibilities, the "
|
||||
"earlier the fall. (B) The isolation cliff: probability of hybrid inviability against "
|
||||
"divergence, by incompatibility density. (C) Trained networks: the merge error barrier between "
|
||||
"two MLPs before and after permutation-and-rescaling alignment --- the same-task/different-"
|
||||
"start barrier is a coordinate artefact (removed by alignment); the conflicting-task barrier "
|
||||
"is left essentially unchanged. (D) Sweeping the fraction of conflicting classes: the residual "
|
||||
"barrier rises while merged-model accuracy falls from 0.97 to 0.03. (E) Language models (0.5B "
|
||||
"LoRA children of a shared base): on shared ambiguous prompts each parent performs under its "
|
||||
"own convention while the merged model falls below both --- function-specific hybrid "
|
||||
"breakdown. (F) Divergence without conflict: over-training disjoint specialists from 1 to 12 "
|
||||
"epochs produces no isolation; the merged model tracks or exceeds the parents throughout."),
|
||||
"fig6": (["paper/pnas/figs/fig6.pdf"],
|
||||
"The language-model tier. (A) Seed-replicated merging (0.5B, five seeds, fixed test sets; mean "
|
||||
"$\\pm$ 95\\% CI): merged specialists exceed the best single specialist overall, and only "
|
||||
"merged models are competent on every task family. (B) Hard, unsaturated tasks at 7B (single "
|
||||
"run): the weight-average dilutes a fragile specialist below the best single parent; routing "
|
||||
"among intact specialists preserves it. (C) The controlled predictive test (13 conditions "
|
||||
"$\\times$ 3 seeds): pre-merge confidence-weighted functional conflict against merge penalty, "
|
||||
"coloured by grid axis --- penalty concentrates on the conflict axis. (D) Predictor "
|
||||
"comparison, $|$Spearman $\\rho|$ against merge penalty over the full grid: functional "
|
||||
"measures carry signal, the tested weight-geometry baselines do not; paired differences "
|
||||
"between predictors are not individually significant."),
|
||||
}
|
||||
|
||||
UNICODE = {"—": "---", "–": "--", "→": r"\(\rightarrow\)", "≈": r"\(\approx\)", "≥": r"\(\geq\)",
|
||||
|
|
@ -112,16 +115,11 @@ def inline(s: str) -> str:
|
|||
|
||||
def figure_env(name: str) -> str:
|
||||
pdfs, caption = FIGURES[name]
|
||||
(HERE / "figs").mkdir(exist_ok=True)
|
||||
lines = [f"\\begin{{figure*}}[p]\\centering % {name}"]
|
||||
for src in pdfs:
|
||||
dst = HERE / "figs" / (name + "_" + Path(src).name)
|
||||
shutil.copyfile(ROOT / src, dst)
|
||||
frac = min(0.98, 3.0 / len(pdfs) * 0.42)
|
||||
lines.append(f"\\includegraphics[width=\\textwidth,height={frac:.2f}\\textheight,"
|
||||
f"keepaspectratio]{{figs/{dst.name}}}\\par\\smallskip")
|
||||
lines.append(f"\\caption{{{caption}}}\\label{{{name}}}")
|
||||
lines.append("\\end{figure*}")
|
||||
src = ROOT / pdfs[0]
|
||||
lines = [f"\\begin{{figure*}}[p]\\centering % {name}",
|
||||
f"\\includegraphics[width=\\textwidth]{{figs/{src.name}}}",
|
||||
f"\\caption{{{caption}}}\\label{{{name}}}",
|
||||
"\\end{figure*}"]
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
|
|
|
|||
BIN
paper/pnas/figs/fig1.pdf
Normal file
BIN
paper/pnas/figs/fig1.pdf
Normal file
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
paper/pnas/figs/fig2.pdf
Normal file
BIN
paper/pnas/figs/fig2.pdf
Normal file
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
paper/pnas/figs/fig3.pdf
Normal file
BIN
paper/pnas/figs/fig3.pdf
Normal file
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
paper/pnas/figs/fig4.pdf
Normal file
BIN
paper/pnas/figs/fig4.pdf
Normal file
Binary file not shown.
Binary file not shown.
BIN
paper/pnas/figs/fig5.pdf
Normal file
BIN
paper/pnas/figs/fig5.pdf
Normal file
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
BIN
paper/pnas/figs/fig6.pdf
Normal file
BIN
paper/pnas/figs/fig6.pdf
Normal file
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
|
|
@ -294,7 +294,7 @@ In trained networks, the claim must survive a known alternative: merge barriers
|
|||
trained networks are famously *coordinate artefacts*, removable by re-aligning hidden units (62);
|
||||
richer symmetry groups remove more (63), with known failures beyond the shared-data regime (64). We therefore aligned under the composition of
|
||||
permutation matching and exact per-unit rescaling (the unit symmetry group of plain ReLU MLPs, as the
|
||||
search space) and decomposed the barrier (Fig. 5B): two networks trained from different
|
||||
search space) and decomposed the barrier (Fig. 5 C and D): two networks trained from different
|
||||
initialisations on the *same* task have a barrier that this alignment removes essentially entirely
|
||||
(residual ≈ 0.001, the aligned merge performing at parent level): coordinate, not functional; two
|
||||
networks trained on *conflicting* label maps have a barrier the same alignment leaves largely
|
||||
|
|
@ -312,7 +312,7 @@ emergent (each lineage's changes harmless alone), so we let children diverge wit
|
|||
signal anywhere*, using complementary class specialists and divergent input conventions, to 6.4× the base
|
||||
training. **No isolation emerged** (residual 0.000 throughout); instead the merge *rescued* the two
|
||||
catastrophically-forgetting specialists (parents ≈ 0.50, merge ≈ 0.955, a sustained Fisher–Muller
|
||||
rescue). The same double result appears at the language-model tier (Fig. 5C): conflicting conventions
|
||||
rescue). The same double result appears at the language-model tier (Fig. 5 E and F): conflicting conventions
|
||||
produce **function-specific** hybrid breakdown (the merge scores below both parents on the conflicted
|
||||
function, while a budget-controlled design shows the disjoint skills merge unharmed), and over-training
|
||||
disjoint specialists 1→12 epochs (cf. the merging literature's expert-duration effect; 65) produces
|
||||
|
|
|
|||
Binary file not shown.
381
paper/pnas/make_figs.py
Normal file
381
paper/pnas/make_figs.py
Normal file
|
|
@ -0,0 +1,381 @@
|
|||
"""Publication figures for the PNAS draft — unified, lettered, codename-free.
|
||||
|
||||
Re-plots every panel directly from the committed results artifacts into six single-file figures
|
||||
(figs/fig1.pdf .. fig6.pdf): no experiment codenames, no suptitles, no per-panel headline titles
|
||||
(interpretation lives in the captions), bold panel letters, one consistent style. The per-experiment
|
||||
figures under results/ remain the exploratory versions; these are the manuscript's.
|
||||
|
||||
Usage: python paper/pnas/make_figs.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
sys.path.insert(0, str(ROOT / "figures"))
|
||||
sys.path.insert(0, str(ROOT / "src"))
|
||||
import os
|
||||
os.chdir(ROOT) # load_bundle uses repo-relative paths
|
||||
from _figlib import load_bundle, mean_ci # noqa: E402
|
||||
|
||||
OUT = ROOT / "paper" / "pnas" / "figs"
|
||||
|
||||
plt.rcParams.update({
|
||||
"font.size": 8, "axes.labelsize": 8.5, "legend.fontsize": 7, "legend.frameon": False,
|
||||
"lines.markersize": 3.6, "axes.spines.top": False, "axes.spines.right": False,
|
||||
})
|
||||
|
||||
|
||||
def letter(ax, s, x=-0.14):
|
||||
ax.text(x, 1.02, s, transform=ax.transAxes, fontsize=12, fontweight="bold", va="bottom")
|
||||
|
||||
|
||||
def save(fig, name):
|
||||
OUT.mkdir(exist_ok=True)
|
||||
fig.savefig(OUT / f"{name}.pdf", bbox_inches="tight")
|
||||
plt.close(fig)
|
||||
print("wrote", OUT / f"{name}.pdf")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- fig 1: grounding + MNIST
|
||||
def fig1():
|
||||
from knowledge.analysis import critical_grounding, reduce_to_stationary
|
||||
from knowledge.metrics import heterozygosity
|
||||
from knowledge.truth import make_true_distribution
|
||||
|
||||
df, cfg = load_bundle("results/E2")
|
||||
n = cfg["dynamics"]["n"]
|
||||
td = make_true_distribution(cfg["truth"]["K"], 1, "zipf", cfg["truth"]["tail_frac"],
|
||||
cfg["truth"]["zipf_s"], 0, tail_threshold=cfg["truth"]["tail_threshold"])
|
||||
H_star = heterozygosity(td.p_star)
|
||||
last = int(cfg["generations"] * 0.8)
|
||||
stat = df[df["generation"] >= last]
|
||||
|
||||
fig, axes = plt.subplots(1, 2, figsize=(10.6, 3.5), gridspec_kw={"width_ratios": [1, 1.35]})
|
||||
|
||||
ax = axes[0]
|
||||
st = reduce_to_stationary(stat, value_col="heterozygosity", replicate_col="replicate", last_frac=1.0)
|
||||
gg, Hm, Hci = mean_ci(stat, "g", "heterozygosity")
|
||||
nz = gg > 0
|
||||
ax.errorbar(gg[nz], Hm[nz], yerr=Hci[nz], fmt="o", color="#1f77b4", capsize=2, label="simulation")
|
||||
ax.plot(gg[~nz], Hm[~nz], "o", mfc="white", mec="#1f77b4")
|
||||
m_of_g = stat.groupby("g")["m"].first().to_numpy()
|
||||
m_grid = np.linspace(0, m_of_g.max(), 400)
|
||||
|
||||
def H_eq(m):
|
||||
m = np.asarray(m, float)
|
||||
return np.where(m <= 0, 0.0, H_star * m * (2 * n + m - 1) / (n + 2 * n * m + m * m))
|
||||
|
||||
ax.plot(m_grid / (n + m_grid), H_eq(m_grid), "k--", lw=1, label="exact equilibrium")
|
||||
ax.axhline(H_star, ls=":", color="gray", lw=1, label="source diversity $H^*$")
|
||||
r = critical_grounding(st, H_star=H_star, frac=0.95, seed=7)
|
||||
ax.axvspan(r["ci_low"], r["ci_high"], color="#d62728", alpha=0.15)
|
||||
ax.axvline(r["g_star"], color="#d62728", lw=1.1,
|
||||
label=f"95%-retention threshold $g\\approx{r['g_star']:.3f}$")
|
||||
ax.set(xlabel="grounding fraction $g = m/(n+m)$", ylabel="stationary diversity $H$")
|
||||
ax.legend()
|
||||
letter(ax, "A")
|
||||
|
||||
ax = axes[1]
|
||||
from PIL import Image
|
||||
im = np.asarray(Image.open("results/mnist_collapse/mnist_montage.png"))
|
||||
crop = int(im.shape[0] * 0.085) # remove the baked-in title band
|
||||
ax.imshow(im[crop:], interpolation="bilinear")
|
||||
ax.set_axis_off()
|
||||
letter(ax, "B", x=-0.02)
|
||||
save(fig, "fig1")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- fig 2: blending vs union + Fisher–Muller
|
||||
def fig2():
|
||||
fig, axes = plt.subplots(1, 2, figsize=(10.6, 3.5))
|
||||
|
||||
df, _ = load_bundle("results/E4")
|
||||
r0 = df[(df["g"] == 0.0) & (df["rho"] == 0.0)]
|
||||
mx = r0.groupby("K_T")["surviving_max"].agg(["mean", "sem"])
|
||||
mn = r0.groupby("K_T")["surviving_mean"].agg(["mean", "sem"])
|
||||
ax = axes[0]
|
||||
ax.errorbar(mx.index, mx["mean"], yerr=1.96 * mx["sem"], fmt="-o", color="#1f77b4",
|
||||
capsize=2, label="union operator (strongest source)")
|
||||
ax.errorbar(mn.index, mn["mean"], yerr=1.96 * mn["sem"], fmt="--s", color="#d62728",
|
||||
capsize=2, label="output-mean (blending)")
|
||||
ax.set(xlabel="number of parents", ylabel="rare capabilities surviving in the child",
|
||||
xticks=sorted(r0["K_T"].unique()))
|
||||
ax.legend()
|
||||
letter(ax, "A")
|
||||
|
||||
df8, cfg8 = load_bundle("results/E8")
|
||||
L = cfg8["society"]["L"]
|
||||
d0 = df8[df8["rho"] == 0.0]
|
||||
ax = axes[1]
|
||||
for col, c, lab in [("best_parent", "#7f7f7f", "best single parent"),
|
||||
("average", "#1f77b4", "blended average"),
|
||||
("sexual", "#d62728", "recombined offspring")]:
|
||||
k, m, ci = mean_ci(d0, "K_T", col)
|
||||
ax.errorbar(k, m, yerr=ci, fmt="-o", color=c, capsize=2, label=lab)
|
||||
ax.axhline(L, ls=":", color="green", lw=1, label="optimum")
|
||||
ax.set(xlabel="number of parents", ylabel="offspring capability")
|
||||
ax.legend()
|
||||
letter(ax, "B")
|
||||
save(fig, "fig2")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- fig 3: rugged landscapes
|
||||
def fig3():
|
||||
fig, axes = plt.subplots(2, 2, figsize=(10.6, 6.8))
|
||||
|
||||
df9, _ = load_bundle("results/E9")
|
||||
Ks = sorted(df9["K"].unique())
|
||||
colors = plt.cm.viridis(np.linspace(0, 0.85, len(Ks)))
|
||||
bp = df9.groupby("K")["best_parent"].mean()
|
||||
ax = axes[0, 0]
|
||||
for K, c in zip(Ks, colors):
|
||||
s = df9[df9["K"] == K].groupby("rate")["mean_offspring"].mean() - bp[K]
|
||||
ax.plot(s.index, s.values, "-o", color=c, label=f"$K$={K}")
|
||||
ax.axhline(0, ls=":", color="gray", lw=1)
|
||||
ax.set(xlabel="recombination rate", ylabel="mean offspring − best parent")
|
||||
ax.legend(title="ruggedness", ncol=2)
|
||||
letter(ax, "A")
|
||||
|
||||
df10, _ = load_bundle("results/E10")
|
||||
ax = axes[0, 1]
|
||||
for col, c, lab in [("global_opt", "green", "global optimum"),
|
||||
("directed_sex", "#d62728", "screened recombination (directed)"),
|
||||
("best_parent", "#7f7f7f", "best single parent"),
|
||||
("random_sex", "#1f77b4", "blind recombination")]:
|
||||
k, m, ci = mean_ci(df10, "K", col)
|
||||
if col == "global_opt":
|
||||
ax.plot(k, m, ":", color=c, label=lab)
|
||||
else:
|
||||
ax.errorbar(k, m, yerr=ci, fmt="-o", color=c, capsize=2, label=lab)
|
||||
ax.set(xlabel="landscape ruggedness $K$", ylabel="offspring capability")
|
||||
ax.legend()
|
||||
letter(ax, "B")
|
||||
|
||||
df14, _ = load_bundle("results/E14")
|
||||
last = df14[df14["generation"] == df14["generation"].max()].copy()
|
||||
last["best_n"] = last["best_fitness"] / last["global_opt"]
|
||||
K14 = sorted(last["K"].unique())
|
||||
cmap = plt.get_cmap("viridis")
|
||||
c14 = {K: cmap(i / max(1, len(K14) - 1)) for i, K in enumerate(K14)}
|
||||
for ax, col, ylab, L in [(axes[1, 0], "best_n", "best fitness / optimum", "C"),
|
||||
(axes[1, 1], "diversity", "population diversity", "D")]:
|
||||
for K in K14:
|
||||
g = last[last["K"] == K].groupby("breadth")[col].agg(["mean", "sem"]).reset_index()
|
||||
ax.errorbar(g["breadth"], g["mean"], yerr=1.96 * g["sem"].fillna(0), fmt="-o",
|
||||
color=c14[K], capsize=2, label=f"$K$={K}")
|
||||
ax.set_xscale("log")
|
||||
ax.set(xlabel="mate-pool breadth (monogamous → panmictic)", ylabel=ylab)
|
||||
ax.legend(title="ruggedness")
|
||||
letter(ax, L)
|
||||
save(fig, "fig3")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- fig 4: the society
|
||||
def fig4():
|
||||
df, _ = load_bundle("results/E11")
|
||||
arms = [("full", "#2ca02c", "full system"),
|
||||
("no_sex", "#ff7f0e", "no recombination"),
|
||||
("no_diversity", "#9467bd", "no diversity preservation"),
|
||||
("no_grounding", "#d62728", "no grounded evaluation")]
|
||||
arms = [a for a in arms if a[0] in set(df["arm"].unique())]
|
||||
g_opt = df["global_opt"].mean()
|
||||
|
||||
fig, axes = plt.subplots(1, 3, figsize=(11.4, 3.2))
|
||||
panels = [("best_fitness", "best real fitness", "A"),
|
||||
("diversity", "population diversity", "B"),
|
||||
("conformity_true_gap", "conformity − true fitness", "C")]
|
||||
for ax, (col, ylab, L) in zip(axes, panels):
|
||||
for name, c, lab in arms:
|
||||
sub = df[df["arm"] == name]
|
||||
g, m, ci = mean_ci(sub, "generation", col)
|
||||
ax.plot(g, m, "-", color=c, lw=1.6, label=lab)
|
||||
ax.fill_between(g, m - ci, m + ci, color=c, alpha=0.15)
|
||||
if col == "best_fitness":
|
||||
ax.axhline(g_opt, ls=":", color="gray", lw=1, label="global optimum")
|
||||
ax.legend()
|
||||
ax.set(xlabel="generation", ylabel=ylab)
|
||||
letter(ax, L)
|
||||
save(fig, "fig4")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- fig 5: speciation, three tiers
|
||||
def fig5():
|
||||
fig, axes = plt.subplots(2, 3, figsize=(11.4, 6.6))
|
||||
|
||||
bdm, _ = load_bundle("results/E12")
|
||||
rhos = sorted(bdm["rho"].unique())
|
||||
colors = plt.cm.viridis(np.linspace(0.15, 0.85, len(rhos)))
|
||||
|
||||
def agg(df, keys, value):
|
||||
g = df.groupby(keys)[value].agg(["mean", "std", "count"]).reset_index()
|
||||
g["se"] = g["std"] / np.sqrt(g["count"].clip(lower=1))
|
||||
return g
|
||||
|
||||
ax = axes[0, 0]
|
||||
par = agg(bdm, "divergence", "parent_fitness")
|
||||
ax.plot(par["divergence"], par["mean"], "k--", lw=1.3, label="parents")
|
||||
for rho, c in zip(rhos, colors):
|
||||
g = agg(bdm[bdm["rho"] == rho], "divergence", "offspring_fitness")
|
||||
ax.plot(g["divergence"], g["mean"], "-o", color=c, label=f"hybrid, density {rho:g}")
|
||||
ax.fill_between(g["divergence"], g["mean"] - g["se"], g["mean"] + g["se"], color=c, alpha=0.15)
|
||||
ax.axhline(0, color="#999", lw=0.7, ls=":")
|
||||
ax.set(xlabel="parental divergence (substitutions)", ylabel="fitness")
|
||||
ax.legend()
|
||||
letter(ax, "A")
|
||||
|
||||
ax = axes[0, 1]
|
||||
for rho, c in zip(rhos, colors):
|
||||
g = agg(bdm[bdm["rho"] == rho], "divergence", "isolation")
|
||||
ax.plot(g["divergence"], g["mean"], "-o", color=c, label=f"{rho:g}")
|
||||
ax.set(xlabel="parental divergence (substitutions)", ylabel="P(hybrid inviable)", ylim=(-0.02, 1.02))
|
||||
ax.legend(title="incompatibility density")
|
||||
letter(ax, "B")
|
||||
|
||||
dec, _ = load_bundle("results/speciation_real")
|
||||
order = [c for c in ["shared", "independent", "conflict"] if c in set(dec["condition"])]
|
||||
g = dec.groupby("condition").agg(naive=("barrier_naive", "mean"),
|
||||
res=("residual_scale", "mean")).reindex(order)
|
||||
ax = axes[0, 2]
|
||||
x = np.arange(len(order)); w = 0.38
|
||||
ax.bar(x - w / 2, g["naive"], w, color="#9ecae1", label="before alignment")
|
||||
ax.bar(x + w / 2, g["res"], w, color="#d62728", label="after alignment (residual)")
|
||||
ax.set_xticks(x)
|
||||
ax.set_xticklabels(["same task,\nshared start", "same task,\ndifferent start", "conflicting\ntasks"])
|
||||
ax.set(ylabel="merge error barrier")
|
||||
ax.legend()
|
||||
letter(ax, "C")
|
||||
|
||||
cliff, _ = load_bundle("results/speciation_real_cliff")
|
||||
cg = cliff.groupby("conflict_frac").agg(res=("residual_scale", "mean"),
|
||||
hyb=("acc_merge_scale", "mean")).reset_index()
|
||||
ax = axes[1, 0]
|
||||
ax.plot(cg["conflict_frac"], cg["res"], "-o", color="#d62728", label="residual barrier")
|
||||
ax2 = ax.twinx()
|
||||
ax2.plot(cg["conflict_frac"], cg["hyb"], "-s", color="#2c7fb8", label="merged-model accuracy")
|
||||
ax2.set_ylabel("merged accuracy", color="#2c7fb8")
|
||||
ax2.tick_params(axis="y", labelcolor="#2c7fb8")
|
||||
ax2.spines["right"].set_visible(True)
|
||||
ax.set(xlabel="fraction of classes in conflict", ylabel="residual barrier")
|
||||
l1, la1 = ax.get_legend_handles_labels(); l2, la2 = ax2.get_legend_handles_labels()
|
||||
ax.legend(l1 + l2, la1 + la2, loc="center left")
|
||||
letter(ax, "D")
|
||||
|
||||
rep, _ = load_bundle("results/llm_speciation")
|
||||
|
||||
def series(df, mode, model, metric):
|
||||
sub = df[(df["mode"] == mode) & (df["model"] == model) & (df["metric"] == metric)]
|
||||
g = sub.groupby("x")["accuracy"].mean().reset_index()
|
||||
return g["x"], g["accuracy"]
|
||||
|
||||
ax = axes[1, 1]
|
||||
x_, y_ = series(rep, "conflict", "parent_a", "ambig_asc")
|
||||
ax.plot(x_, y_, "--o", color="#9ecae1", label="parent A, own convention")
|
||||
x_, y_ = series(rep, "conflict", "parent_b", "ambig_desc")
|
||||
ax.plot(x_, y_, "--o", color="#a1d99b", label="parent B, own convention")
|
||||
x_, y_ = series(rep, "conflict", "merge_soup", "coherence")
|
||||
ax.plot(x_, y_, "-s", color="#d62728", label="merge, best convention")
|
||||
ax.set(xlabel="fraction of training in conflict", ylabel="accuracy, shared prompts")
|
||||
ax.legend()
|
||||
letter(ax, "E")
|
||||
|
||||
ax = axes[1, 2]
|
||||
x_, y_ = series(rep, "duration", "merge_soup", "mean_private")
|
||||
ax.plot(x_, y_, "-o", color="#d62728", label="merged model")
|
||||
x_, y_ = series(rep, "duration", "parent_a", "strings")
|
||||
ax.plot(x_, y_, "--o", color="#9ecae1", label="parent A, own task")
|
||||
x_, y_ = series(rep, "duration", "parent_b", "arith")
|
||||
ax.plot(x_, y_, "--o", color="#a1d99b", label="parent B, own task")
|
||||
ax.set(xlabel="specialist training (epochs)", ylabel="accuracy", ylim=(0, 1.02))
|
||||
ax.legend()
|
||||
letter(ax, "F")
|
||||
save(fig, "fig5")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- fig 6: the language-model tier
|
||||
def fig6():
|
||||
import pandas as pd
|
||||
from scipy.stats import spearmanr
|
||||
|
||||
fig, axes = plt.subplots(2, 2, figsize=(10.6, 6.6))
|
||||
|
||||
dfm, _ = load_bundle("results/llm_merge_seeds")
|
||||
specs = sorted(m for m in dfm["model"].unique() if m.startswith("spec_"))
|
||||
rows = []
|
||||
for s, sub in dfm.groupby("seed"):
|
||||
ov = {m: sub[(sub["model"] == m) & (sub["metric"] == "overall")]["accuracy"].mean() for m in specs}
|
||||
b = sub[sub["model"] == max(ov, key=ov.get)].copy(); b["model"] = "best_specialist"
|
||||
rows.append(b)
|
||||
dfm = pd.concat([dfm] + rows, ignore_index=True)
|
||||
models = ["base", "best_specialist", "merge_soup", "merge_ties"]
|
||||
labels = ["base", "best\nspecialist", "merged\n(average)", "merged\n(interference-aware)"]
|
||||
ax = axes[0, 0]
|
||||
x = np.arange(len(models))
|
||||
for off, metric, c, lab in ((-0.19, "overall", "#2c7fb8", "overall"),
|
||||
(0.19, "worst_family", "#d62728", "worst task family")):
|
||||
vals, errs = [], []
|
||||
for m in models:
|
||||
v = dfm[(dfm["model"] == m) & (dfm["metric"] == metric)].groupby("seed")["accuracy"].mean()
|
||||
vals.append(v.mean()); errs.append(1.96 * v.std(ddof=1) / max(1, np.sqrt(len(v))))
|
||||
ax.bar(x + off, vals, 0.36, yerr=errs, capsize=2, color=c, label=lab)
|
||||
ax.set_xticks(x); ax.set_xticklabels(labels, fontsize=7)
|
||||
ax.set(ylabel="verifier accuracy")
|
||||
ax.legend()
|
||||
letter(ax, "A")
|
||||
|
||||
df7, _ = load_bundle("results/llm_moe_hard_hpc")
|
||||
|
||||
def acc7(model, metric):
|
||||
r = df7[(df7["model"] == model) & (df7["metric"] == metric)]["accuracy"]
|
||||
return float(r.iloc[0]) if len(r) else np.nan
|
||||
specs7 = sorted(m for m in df7["model"].unique() if m.startswith("spec_"))
|
||||
best7 = max(specs7, key=lambda m: acc7(m, "overall"))
|
||||
models7 = [best7, "merge_soup", "merge_ties", "moe_oracle"]
|
||||
labels7 = ["best\nspecialist", "merged\n(average)", "merged\n(interference-aware)", "routed\n(kept separate)"]
|
||||
ax = axes[0, 1]
|
||||
x = np.arange(len(models7))
|
||||
for off, metric, c, lab in ((-0.19, "overall", "#2c7fb8", "overall"),
|
||||
(0.19, "worst_family", "#d62728", "worst task family")):
|
||||
ax.bar(x + off, [acc7(m, metric) for m in models7], 0.36, color=c, label=lab)
|
||||
ax.set_xticks(x); ax.set_xticklabels(labels7, fontsize=7)
|
||||
ax.set(ylabel="verifier accuracy")
|
||||
ax.legend()
|
||||
letter(ax, "B")
|
||||
|
||||
a = pd.read_parquet("results/llm_epistasis/results.parquet")
|
||||
b = pd.read_parquet("results/llm_epistasis_compat/results.parquet")
|
||||
dfe = pd.concat([a, b], ignore_index=True)
|
||||
ax = axes[1, 0]
|
||||
for mode, c, mk, lab in (("conflict", "#d62728", "o", "conflicting conventions"),
|
||||
("duration", "#2c7fb8", "s", "divergence only"),
|
||||
("compat", "#41ab5d", "^", "overlap, no conflict")):
|
||||
sub = dfe[dfe["mode"] == mode]
|
||||
ax.scatter(sub["epi_conf"], sub["merge_penalty"], c=c, marker=mk, s=26, alpha=0.75, label=lab)
|
||||
ax.axhline(0, color="#999", lw=0.6)
|
||||
ax.set(xlabel="pre-merge functional conflict (confidence-weighted)",
|
||||
ylabel="merge penalty")
|
||||
ax.legend()
|
||||
letter(ax, "C")
|
||||
|
||||
preds = [("dis_raw", "raw\ndisagreement"), ("epi_conf", "conf-weighted\nconflict"),
|
||||
("grad_cos", "gradient\nalignment"), ("delta_cos", "weight\ncosine"),
|
||||
("delta_l2", "weight\ndistance"), ("cross_perf", "cross-task\naccuracy")]
|
||||
ax = axes[1, 1]
|
||||
rhos_ = [abs(spearmanr(dfe[c], dfe["merge_penalty"])[0]) for c, _ in preds]
|
||||
cols = ["#fc9272", "#d62728", "#9ecae1", "#9ecae1", "#9ecae1", "#9ecae1"]
|
||||
ax.bar(np.arange(len(preds)), rhos_, 0.6, color=cols)
|
||||
ax.set_xticks(np.arange(len(preds)))
|
||||
ax.set_xticklabels([l for _, l in preds], fontsize=6.5)
|
||||
ax.set(ylabel="|Spearman ρ| vs merge penalty", ylim=(0, 0.8))
|
||||
letter(ax, "D")
|
||||
save(fig, "fig6")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
for f in (fig1, fig2, fig3, fig4, fig5, fig6):
|
||||
f()
|
||||
Loading…
Add table
Add a link
Reference in a new issue