society: multi-locus recombination frame — the vertical claim (E7/E8)
Enter the Lamarckian society with a robust theoretical frame. The single-
locus, fixed-p* model can only express recovery toward a ceiling; the
society's load-bearing claim is vertical -- capability that EXCEEDS any
component. Generalize knowledge to a distribution over genotypes (L
biallelic loci, K=2^L, additive fitness = # correct loci), reusing all the
K-mode machinery. The one new operator is recombination: free recombination
sends p -> product of per-locus marginals (linkage equilibrium).
E8 (star, kind: society) -- the vertical claim / Fisher-Muller: decorrelated
PARENTS (specialists, expert on their loci, agnostic elsewhere) are
recombined; sexual merge assembles a genotype fitter than any parent,
climbing to the optimum (12/12, a genotype no parent had) as parent count
grows and rho->0, while the best single parent (~8.7) and the mean-mixture
"model soup" (~11.6) plateau below. Reuses make_retention_matrix (locus
mastery replaces tail-item retention).
E7 (kind: genotype_lineage) -- the advantage of sex: a single population
adapts toward the optimum; the sexual lineage adapts faster than asexual
(clonal interference) by keeping loci in linkage equilibrium (LD->0 vs LD
spike). Honest scope: a speed advantage, not a permanent Muller's-ratchet
gap (subtle to force); E8 carries the headline.
Metaphor shift (per GG): the society is sexual reproduction with UNBOUNDED
parents, not teacher->pupil. Teacher->pupil caps at the ceiling; n-parent
recombination is combinatorial and generative, and unlike biology there is
no two-parent limit. Collapse = asexual degradation; the cure = sex. This
unifies E4 (merge != average) + E6 (irreversibility) under evolution-of-sex
theory and reaches ground Riis's single-locus n-grams cannot.
New: knowledge/{genotype,genotype_lineage,society}.py, configs/layer1/{E7,
E8}.yaml, figures/plot_{E7,E8}.py, READMEs, tests/test_genotype.py (+7).
experiment.py dispatch (kind in {genotype_lineage, society}); make layer1
wired. 112 tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
871bc39ec6
commit
62c68d6c8c
22 changed files with 879 additions and 3 deletions
|
|
@ -80,6 +80,8 @@ E4's whole purpose is to isolate the effect of teacher **decorrelation ρ**, so
|
|||
|
||||
**Strategic positioning vs Riis 2026 (arXiv:2604.08554, "Drift and selection in LLM text ecosystems").** Riis independently formalizes **collapse = Wright–Fisher drift** (his Thm 1) with n-gram agents: minority-mass martingale, rare-first extinction, single-token dropout ≈ αe^{−α}, de Bruijn-polytope fixed points, plus descriptive-vs-normative *selection* (Thm 2). **Concede as prior art:** "collapse is literally Wright–Fisher", the martingale, rare-first loss, the WF/effective-population formalism — cite him; do **not** frame these as our contribution. **Crucial distinction that protects us:** his "mixed environment" *retains the lineage's own old synthetic tokens* — there is **no injection of fresh real data from a fixed `p*`**, so his headline is *pessimistic* (Thm 1c: extinction is independent of α — retention only changes speed). Our **grounding is immigration from a non-drifting external truth**, giving a stationary `H_eq>0` and a critical `g*≪1` that *prevents* collapse — the mechanism his closed loop lacks. **Our defensible novelty, ranked:** (1) **recombination + "merge, don't average" conservation law** (E4) — he has no model-merging operator; flagship; (2) **the learning-kernel / estimator-bias axis** — he *explicitly names it as future work*; we now build+measure it; (3) grounding threshold (solid anchor, but immigration–drift balance is classic — not a flagship); (4) architecture-generality in real weights + MNIST; (5) **the Lamarckian society + the vertical/cumulative C3 claim — wholly ours, not yet run.** Reposition the paper from *"collapse is drift"* (now contested) to **a population-genetic *control theory* for sustaining open-ended knowledge**: drift is the diagnosed disease (cite Riis), our contribution is the engineered remedies and their integration.
|
||||
|
||||
**Finding (2026-07-05, E7/E8 — the multi-locus society frame; raises the ceiling).** To express the *vertical* claim (capability that *exceeds* any component), knowledge is generalized from a single-locus fixed-`p*` distribution to a distribution over **genotypes** (`L` biallelic loci, `K=2^L`; fitness = # correct loci; reuses all the K-mode machinery). The one new operator is **recombination** (`knowledge/genotype.py`): free recombination sends `p → ⊗ per-locus marginals` (linkage equilibrium). Two experiments, both analytic. **E8 (the star, `kind: society`) — the vertical claim / Fisher–Muller:** decorrelated *parents* (specialists, expert on their loci, agnostic elsewhere) are recombined; **sexual merge assembles a genotype fitter than any parent, climbing to the optimum (12/12) as parent count grows and `ρ→0`, while the best single parent (~8.7) and the mean-mixture "model soup" (~11.6) plateau below.** Clean, dramatic, 40 reps; reuses `make_retention_matrix` (locus mastery replaces tail-item retention). **E7 (`kind: genotype_lineage`) — the advantage of sex:** a single population adapting toward the optimum; the sexual lineage adapts *faster* (clonal interference slows the asexual one) by keeping loci in linkage equilibrium (LD→0 vs LD spike). Honest scope: a **speed** advantage, not a permanent Muller's-ratchet gap (the single-population ratchet is subtle to force; E8 carries the headline). **Metaphor shift (GG, 2026-07-05):** the society is framed as **sexual reproduction with unbounded parents**, *not* teacher→pupil — teacher→pupil caps at the ceiling (recovery), n-parent recombination is combinatorial and *generative* (exceeds any parent), and unlike biology there is no two-parent limit. Collapse = asexual degradation; the cure = sex. This unifies E4 (merge≠average) + E6 (irreversibility) under evolution-of-sex theory and stakes ground Riis's single-locus n-grams cannot reach. Scope is bounded: fixed combinatorial space (`L≤12`, "effectively open-ended relative to n"), additive fitness (NK/epistasis is an optional extension).
|
||||
|
||||
## Build order (blueprint §7) — respect the gate
|
||||
|
||||
1. Scaffold: repo layout (§5), container, pytest skeleton, config system, seeding utils. `make test` green.
|
||||
|
|
|
|||
2
Makefile
2
Makefile
|
|
@ -16,7 +16,7 @@ test: ## correctness tests + scientific-validation tests (the spine
|
|||
uv run pytest
|
||||
|
||||
layer1: ## run experiments E1-E6 + the learning-kernel bridge (analytic)
|
||||
for e in E1 E2 E3 E4 E5 E6 kernel_sharpen kernel_smooth; do uv run python -m knowledge.experiment configs/layer1/$$e.yaml; done
|
||||
for e in E1 E2 E3 E4 E5 E6 E7 E8 kernel_sharpen kernel_smooth; do uv run python -m knowledge.experiment configs/layer1/$$e.yaml; done
|
||||
|
||||
neural: ## run Layer 1.5 synthetic neural experiments (excludes the heavy MNIST tier)
|
||||
for c in configs/neural/*.yaml; do case "$$c" in *mnist*) ;; \
|
||||
|
|
|
|||
29
configs/layer1/E7.yaml
Normal file
29
configs/layer1/E7.yaml
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
experiment: E7
|
||||
kind: genotype_lineage
|
||||
seed: 20260705
|
||||
n_replicates: 20
|
||||
|
||||
# (The advantage of sex — the dynamic mechanism behind E8): a single population adapts from all-wrong
|
||||
# toward a multi-locus optimum under selection + drift + mutation. Beneficial alleles arise in
|
||||
# different sub-lineages; recombination reassorts them into one genotype, while an asexual lineage
|
||||
# suffers clonal interference (the alleles compete and cannot combine). Expect the SEXUAL lineage
|
||||
# (recomb_rate=1) to climb toward the optimum faster than the ASEXUAL one (recomb_rate=0) — the
|
||||
# classical advantage of sex, and the reason a lone model lineage cannot do what a recombining
|
||||
# society can. Honest scope: a SPEED advantage, not a dramatic permanent gap (the single-population
|
||||
# ratchet is subtle); E8 carries the headline. Falsifier: sexual adapts no faster than asexual.
|
||||
|
||||
genotype:
|
||||
L: 12
|
||||
n: 150 # population/resample size (drift strength)
|
||||
mu: 0.02 # per-locus mutation (flip) rate
|
||||
base: 1.3 # multiplicative selection: fitness weight = base^(#correct loci)
|
||||
recomb_rate: 0.0 # overwritten per arm by the sweep
|
||||
init: wrong # start all-wrong (load L); adapt upward
|
||||
|
||||
generations: 120
|
||||
|
||||
sweep:
|
||||
- param: genotype.recomb_rate
|
||||
values: [0.0, 1.0] # asexual vs sexual
|
||||
|
||||
output: {dir: results/E7}
|
||||
29
configs/layer1/E8.yaml
Normal file
29
configs/layer1/E8.yaml
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
experiment: E8
|
||||
kind: society
|
||||
seed: 20260705
|
||||
n_replicates: 40
|
||||
|
||||
# (The vertical claim / Fisher-Muller — the society headline): can an offspring recombined from
|
||||
# MANY decorrelated parents be fitter than ANY parent? Each parent is a specialist: confident-
|
||||
# correct (hi) on the loci it has mastered, agnostic (~0.5) elsewhere; which loci each masters comes
|
||||
# from the exact shared-switch construction, so parent count K_T and decorrelation rho are clean
|
||||
# knobs. Deployed capability = fitness of the MODE genotype. Compare best single parent vs mean-
|
||||
# mixture ("model soup", combine-but-don't-recombine) vs sexual recombination (assemble the best
|
||||
# allele of each locus across all parents). Expect: sexual climbs to the optimum (=L, a genotype NO
|
||||
# parent had) as K_T grows and rho->0, while best-parent and average plateau far below. Unlike
|
||||
# biological sex there is no two-parent limit. Falsifier: sexual never exceeds the best parent, or
|
||||
# averaging matches sexual.
|
||||
|
||||
society:
|
||||
L: 12 # loci; the optimum (all-correct) has fitness 12 and no parent possesses it
|
||||
q: 0.5 # fraction of loci each parent masters (marginal mastery)
|
||||
hi: 0.9 # correct-allele prob on a mastered locus (confident expert)
|
||||
lo: 0.45 # correct-allele prob on an unmastered locus (agnostic, slightly wrong)
|
||||
|
||||
sweep:
|
||||
- param: K_T
|
||||
values: [1, 2, 3, 5, 8, 12] # number of parents (unbounded; grows the recombinant reach)
|
||||
- param: rho
|
||||
values: [0.0, 0.5, 1.0] # decorrelated -> identical parents (the control)
|
||||
|
||||
output: {dir: results/E8}
|
||||
62
figures/plot_E7.py
Normal file
62
figures/plot_E7.py
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
"""E7 figure — the advantage of sex: recombination adapts faster than clonal reproduction.
|
||||
|
||||
The dynamic mechanism behind E8. A single population adapts from all-wrong toward a multi-locus
|
||||
optimum under selection + drift + mutation. Beneficial alleles arise in different sub-lineages;
|
||||
sexual recombination reassorts them into one genotype, while an asexual lineage suffers clonal
|
||||
interference. The sexual lineage climbs faster — the classical advantage of sex (an honest *speed*
|
||||
advantage; both eventually plateau near the optimum in this tractable regime).
|
||||
|
||||
Two panels: (A) mean-fitness adaptation curves, asexual vs sexual, over generations; (B) linkage
|
||||
disequilibrium over generations — asexual holds beneficial alleles in disequilibrium (scattered
|
||||
across genotypes) while sexual drives it to ~0 (assembled), the mechanism of the speed gap.
|
||||
|
||||
Usage: python figures/plot_E7.py [results/E7]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import matplotlib.pyplot as plt
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _figlib import load_bundle, mean_ci, savefig # noqa: E402
|
||||
|
||||
|
||||
def main(results_dir: str = "results/E7") -> None:
|
||||
df, cfg = load_bundle(results_dir)
|
||||
L = cfg["genotype"]["L"]
|
||||
arms = [(0.0, "#7f7f7f", "asexual (clonal)"), (1.0, "#d62728", "sexual (recombining)")]
|
||||
|
||||
fig, axes = plt.subplots(1, 2, figsize=(13, 5))
|
||||
|
||||
ax = axes[0]
|
||||
for rate, c, lab in arms:
|
||||
sub = df[df["recomb_rate"] == rate]
|
||||
g, m, ci = mean_ci(sub, "generation", "mean_fitness")
|
||||
ax.plot(g, m, "-", color=c, lw=1.8, label=lab)
|
||||
ax.fill_between(g, m - ci, m + ci, color=c, alpha=0.2)
|
||||
ax.axhline(L, ls=":", color="green", lw=1, label=f"optimum ($L$={L})")
|
||||
ax.set(xlabel="generation", ylabel="mean fitness (# correct loci)",
|
||||
title="Advantage of sex: recombination adapts faster\n(clonal interference slows the asexual lineage)")
|
||||
ax.legend(frameon=False, fontsize=9)
|
||||
|
||||
ax = axes[1]
|
||||
for rate, c, lab in arms:
|
||||
sub = df[df["recomb_rate"] == rate]
|
||||
g, m, ci = mean_ci(sub, "generation", "ld")
|
||||
ax.plot(g, m, "-", color=c, lw=1.8, label=lab)
|
||||
ax.fill_between(g, m - ci, m + ci, color=c, alpha=0.2)
|
||||
ax.set(xlabel="generation", ylabel="mean linkage disequilibrium |D|",
|
||||
title="Mechanism: asexual scatters beneficial alleles (LD>0);\nsexual assembles them (LD→0)")
|
||||
ax.legend(frameon=False, fontsize=9)
|
||||
|
||||
fig.suptitle("E7 — the advantage of sex: recombination reassorts beneficial alleles that arose "
|
||||
"in different lineages", y=1.02, fontsize=12)
|
||||
fig.tight_layout()
|
||||
savefig(fig, results_dir, "E7")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(*sys.argv[1:])
|
||||
68
figures/plot_E8.py
Normal file
68
figures/plot_E8.py
Normal file
|
|
@ -0,0 +1,68 @@
|
|||
"""E8 figure — the vertical claim: n-parent recombination exceeds any parent (Fisher–Muller).
|
||||
|
||||
The society headline. Decorrelated *parents* are specialists (expert on some loci, agnostic on the
|
||||
rest); an *offspring* recombined from all of them can be fitter than any parent — capability that
|
||||
*exceeds* every component, not just recovers a ceiling. Unlike biological sex there is no two-parent
|
||||
limit, so capability climbs toward the optimum as the parent pool grows and decorrelates.
|
||||
|
||||
Two panels: (A) deployed capability (mode-genotype fitness) vs parent count at ρ=0 — sexual
|
||||
recombination reaches the optimum (a genotype no parent had) while the best single parent and the
|
||||
mean-mixture "model soup" plateau below; (B) the decorrelation control — sexual capability vs parent
|
||||
count for ρ ∈ {0, 0.5, 1}: decorrelated parents (ρ=0) climb to the optimum, identical parents (ρ=1)
|
||||
buy nothing. Reads only the committed bundle.
|
||||
|
||||
Usage: python figures/plot_E8.py [results/E8]
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import matplotlib.pyplot as plt
|
||||
import numpy as np
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _figlib import load_bundle, mean_ci, savefig # noqa: E402
|
||||
|
||||
|
||||
def main(results_dir: str = "results/E8") -> None:
|
||||
df, cfg = load_bundle(results_dir)
|
||||
L = cfg["society"]["L"]
|
||||
rhos = sorted(df["rho"].unique())
|
||||
|
||||
fig, axes = plt.subplots(1, 2, figsize=(13, 5))
|
||||
|
||||
# Panel A: best-parent vs average vs sexual, at rho=0.
|
||||
ax = axes[0]
|
||||
d0 = df[df["rho"] == 0.0]
|
||||
for col, c, lab in [("best_parent", "#7f7f7f", "best single parent"),
|
||||
("average", "#1f77b4", "average (model soup)"),
|
||||
("sexual", "#d62728", "sexual recombination")]:
|
||||
k, m, ci = mean_ci(d0, "K_T", col)
|
||||
ax.errorbar(k, m, yerr=ci, fmt="-o", color=c, capsize=3, label=lab)
|
||||
ax.axhline(L, ls=":", color="green", lw=1, label=f"optimum ($L$={L})")
|
||||
ax.set(xlabel="number of parents $K_T$", ylabel="deployed capability (mode fitness)",
|
||||
title="Recombination exceeds any parent (ρ=0):\nsexual reaches the optimum; soup & best-parent plateau")
|
||||
ax.legend(frameon=False, fontsize=9)
|
||||
|
||||
# Panel B: sexual capability vs K_T for each rho (decorrelation control).
|
||||
ax = axes[1]
|
||||
colors = plt.cm.viridis(np.linspace(0, 0.8, len(rhos)))
|
||||
for rho, c in zip(rhos, colors):
|
||||
sub = df[df["rho"] == rho]
|
||||
k, m, ci = mean_ci(sub, "K_T", "sexual")
|
||||
ax.errorbar(k, m, yerr=ci, fmt="-o", color=c, capsize=3, label=fr"ρ={rho:g}")
|
||||
ax.axhline(L, ls=":", color="green", lw=1, label=f"optimum ($L$={L})")
|
||||
ax.set(xlabel="number of parents $K_T$", ylabel="sexual-recombination capability",
|
||||
title="Decorrelation is the fuel:\nρ=0 climbs to the optimum; ρ=1 (clones) buy nothing")
|
||||
ax.legend(frameon=False, fontsize=9)
|
||||
|
||||
fig.suptitle("E8 — the vertical claim: an offspring recombined from many decorrelated parents "
|
||||
"is fitter than any parent (Fisher–Muller; no two-parent limit)", y=1.02, fontsize=12)
|
||||
fig.tight_layout()
|
||||
savefig(fig, results_dir, "E8")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main(*sys.argv[1:])
|
||||
BIN
results/E7/E7.pdf
Normal file
BIN
results/E7/E7.pdf
Normal file
Binary file not shown.
BIN
results/E7/E7.png
Normal file
BIN
results/E7/E7.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 157 KiB |
31
results/E7/README.md
Normal file
31
results/E7/README.md
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
# E7 — the advantage of sex: recombination adapts faster than clonal reproduction
|
||||
|
||||
**Claim tested.** The dynamic mechanism behind E8: *why* can a recombining society reach capability a
|
||||
lone lineage cannot? Because recombination reassorts beneficial variants that arise in different
|
||||
sub-lineages, while an asexual (clonal) lineage suffers **clonal interference** — the variants compete
|
||||
and cannot combine.
|
||||
|
||||
**Setup.** A single population (distribution over `2^L` genotypes, `L=12`) adapts from **all-wrong**
|
||||
toward the multi-locus optimum under the composed step: selection (fitness-proportional) + drift
|
||||
(resample `n=150`) + mutation (per-locus flips, `μ=0.02`) + recombination. Two arms — **asexual**
|
||||
(`recomb_rate=0`) vs **sexual** (`recomb_rate=1`). 20 replicates.
|
||||
|
||||
### Symbols
|
||||
- **asexual/clonal** = offspring are whole-genotype copies (Layer-1's regime) · **sexual** = loci reassorted across the population each generation.
|
||||
- **fitness** = number of correct loci (optimum = `L`) · **linkage disequilibrium |D|** = how far the loci are from statistical independence (correct alleles scattered across different genotypes).
|
||||
|
||||
### The two panels
|
||||
1. **Advantage of sex.** Mean fitness over generations: the **sexual lineage (red) climbs faster**
|
||||
than the asexual one (grey) through the adaptation phase (gen ~10–35). *Honest scope:* both plateau
|
||||
near the optimum by gen ~40 in this tractable regime — this is a **speed** advantage, not a
|
||||
permanent gap (the single-population Muller's ratchet is subtle to force; E8 carries the headline).
|
||||
2. **Mechanism.** Linkage disequilibrium over generations: the asexual lineage spikes to `|D|≈0.04`
|
||||
during adaptation (beneficial alleles held apart, scattered across genotypes), while the sexual
|
||||
lineage stays at `|D|≈0` — it *assembles* them. The LD gap is exactly why sexual adapts faster.
|
||||
|
||||
### Takeaway
|
||||
Recombination's advantage is real and classical: it combines good ideas that arose independently,
|
||||
which clonal reproduction cannot. This is the population-level reason a single evolving model lineage
|
||||
degrades or stalls where a recombining **society** climbs — and it grounds the E8 vertical result in
|
||||
the evolution-of-sex theory. **Falsifier (not triggered):** if the sexual lineage adapted no faster
|
||||
than the asexual one (and kept the same LD), recombination would do no work.
|
||||
14
results/E7/manifest.json
Normal file
14
results/E7/manifest.json
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"experiment": "E7",
|
||||
"master_seed": 20260705,
|
||||
"git_commit": "871bc39ec6628f82aed75d007fdf675880eebc97",
|
||||
"python": "3.14.5",
|
||||
"libraries": {
|
||||
"numpy": "2.5.0",
|
||||
"scipy": "1.18.0",
|
||||
"pandas": "3.0.3",
|
||||
"pyarrow": "24.0.0"
|
||||
},
|
||||
"rows": 4840,
|
||||
"results_sha256": "4836cd7045ad5419554e7edc65e4e12b23877a551f38969d65e09a4f77715faa"
|
||||
}
|
||||
23
results/E7/resolved_config.yaml
Normal file
23
results/E7/resolved_config.yaml
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
experiment: E7
|
||||
seed: 20260705
|
||||
n_replicates: 20
|
||||
source_config:
|
||||
experiment: E7
|
||||
kind: genotype_lineage
|
||||
seed: 20260705
|
||||
n_replicates: 20
|
||||
genotype:
|
||||
L: 12
|
||||
n: 150
|
||||
mu: 0.02
|
||||
base: 1.3
|
||||
recomb_rate: 0.0
|
||||
init: wrong
|
||||
generations: 120
|
||||
sweep:
|
||||
- param: genotype.recomb_rate
|
||||
values:
|
||||
- 0.0
|
||||
- 1.0
|
||||
output:
|
||||
dir: results/E7
|
||||
BIN
results/E8/E8.pdf
Normal file
BIN
results/E8/E8.pdf
Normal file
Binary file not shown.
BIN
results/E8/E8.png
Normal file
BIN
results/E8/E8.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 167 KiB |
35
results/E8/README.md
Normal file
35
results/E8/README.md
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
# E8 — the vertical claim: n-parent recombination exceeds any parent (Fisher–Muller)
|
||||
|
||||
**Claim tested.** The society's headline, and the thing no fixed-`p*` model could express: can an
|
||||
offspring recombined from **many decorrelated parents** be *fitter than any parent* — capability that
|
||||
**exceeds** every component, not merely recovers a ceiling?
|
||||
|
||||
**Setup.** A capability is a **genotype** of `L=12` biallelic loci; fitness = number of correct
|
||||
loci; the optimum (all-correct, fitness 12) is a genotype **no parent possesses**. Each *parent* is a
|
||||
specialist: confident-correct (`hi=0.9`) on the loci it has mastered, agnostic (`lo=0.45`) on the
|
||||
rest. Which loci each masters comes from the exact shared-switch construction, so **parent count
|
||||
`K_T`** and **decorrelation `ρ`** are clean, independently-swept knobs. Deployed capability = fitness
|
||||
of the **mode** (most-probable) genotype — what you would ship. 40 replicates.
|
||||
|
||||
### Symbols
|
||||
- **parent** = a specialist model; **offspring** = the recombined model; **`K_T`** = number of parents (unbounded — biological sex is stuck at 2; model merging is not).
|
||||
- **`ρ`** = correlation of which loci parents master (0 = complementary, 1 = identical clones).
|
||||
- **best parent** = fittest single specialist · **average** = mean-mixture "model soup" (combine, don't recombine) · **sexual** = union-preserving recombination (assemble the best allele of each locus).
|
||||
|
||||
### The two panels
|
||||
1. **Recombination exceeds any parent (ρ=0).** Deployed capability vs `K_T`: **sexual (red) climbs
|
||||
to the optimum (12)** as parents accumulate — a genotype none of them had — while the **best single
|
||||
parent (grey) plateaus at ~8.7** and the **model soup (blue) reaches ~11.6** but is beaten by
|
||||
sexual at every `K_T` (and badly at small `K_T`: at 2 parents, sexual 9.0 vs soup 7.2 vs best 6.9).
|
||||
2. **Decorrelation is the fuel.** Sexual capability vs `K_T` for `ρ ∈ {0, 0.5, 1}`: decorrelated
|
||||
parents (`ρ=0`) climb to the optimum; identical clones (`ρ=1`) buy nothing (flat at ~6). The
|
||||
benefit is *combinatorial reach across complementary parents*, not merely "more models".
|
||||
|
||||
### Takeaway
|
||||
This is the Fisher–Muller effect for AI: recombination assembles beneficial variants that live in
|
||||
*different* parents into an offspring fitter than any of them. It is the rigorous, un-preempted core
|
||||
of the Lamarckian society — collapse is asexual degradation; the cure is **sex, with no parent
|
||||
limit.** It reframes model merging from "averaging weights" to "meiotic reassortment", and it is the
|
||||
mechanism by which general capability can *climb while each specialty is re-earned and exceeded.* The
|
||||
dynamic version (why a lone lineage cannot do this) is E7. **Falsifier (not triggered):** if sexual
|
||||
never exceeded the best parent, or averaging matched it, the vertical claim would fail.
|
||||
14
results/E8/manifest.json
Normal file
14
results/E8/manifest.json
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"experiment": "E8",
|
||||
"master_seed": 20260705,
|
||||
"git_commit": "871bc39ec6628f82aed75d007fdf675880eebc97",
|
||||
"python": "3.14.5",
|
||||
"libraries": {
|
||||
"numpy": "2.5.0",
|
||||
"scipy": "1.18.0",
|
||||
"pandas": "3.0.3",
|
||||
"pyarrow": "24.0.0"
|
||||
},
|
||||
"rows": 720,
|
||||
"results_sha256": "fec5c4fcaa86999291dd77b015cd650ad8c28cb62f67eab54ca5be04bc18a711"
|
||||
}
|
||||
29
results/E8/resolved_config.yaml
Normal file
29
results/E8/resolved_config.yaml
Normal file
|
|
@ -0,0 +1,29 @@
|
|||
experiment: E8
|
||||
seed: 20260705
|
||||
n_replicates: 40
|
||||
source_config:
|
||||
experiment: E8
|
||||
kind: society
|
||||
seed: 20260705
|
||||
n_replicates: 40
|
||||
society:
|
||||
L: 12
|
||||
q: 0.5
|
||||
hi: 0.9
|
||||
lo: 0.45
|
||||
sweep:
|
||||
- param: K_T
|
||||
values:
|
||||
- 1
|
||||
- 2
|
||||
- 3
|
||||
- 5
|
||||
- 8
|
||||
- 12
|
||||
- param: rho
|
||||
values:
|
||||
- 0.0
|
||||
- 0.5
|
||||
- 1.0
|
||||
output:
|
||||
dir: results/E8
|
||||
|
|
@ -127,6 +127,46 @@ def run_experiment(cfg: dict) -> pd.DataFrame:
|
|||
return out
|
||||
|
||||
|
||||
_GENOTYPE_KEYS = ("genotype", "generations")
|
||||
|
||||
|
||||
def run_genotype_experiment(cfg: dict) -> pd.DataFrame:
|
||||
"""Run a genotype lineage across a sweep x replicates (E7, advantage of sex).
|
||||
|
||||
Mirrors ``run_experiment`` (paired replicate seeds) but assembles the base from the
|
||||
``genotype``/``generations`` blocks and calls ``run_genotype_lineage``. Sweeps use the same
|
||||
dotted-path ``_apply_param`` (e.g. ``genotype.recomb_rate`` for asexual vs sexual).
|
||||
"""
|
||||
from .genotype_lineage import run_genotype_lineage
|
||||
|
||||
base = {k: copy.deepcopy(cfg[k]) for k in _GENOTYPE_KEYS if k in cfg}
|
||||
sweeps = cfg.get("sweep", [])
|
||||
if isinstance(sweeps, dict):
|
||||
sweeps = [sweeps]
|
||||
params = [s["param"] for s in sweeps]
|
||||
value_lists = [list(s["values"]) for s in sweeps]
|
||||
combos = [({}, base)] if not sweeps else []
|
||||
for values in itertools.product(*value_lists):
|
||||
lin = copy.deepcopy(base)
|
||||
label: dict = {}
|
||||
for param, val in zip(params, values):
|
||||
label.update(_apply_param(lin, param, val))
|
||||
combos.append((label, lin))
|
||||
|
||||
seeds = spawn_seeds(int(cfg["seed"]), int(cfg["n_replicates"]))
|
||||
frames: list[pd.DataFrame] = []
|
||||
for label, lin in combos:
|
||||
for rep, ss in enumerate(seeds):
|
||||
df = run_genotype_lineage(lin, int(ss.generate_state(1)[0]))
|
||||
for col, val in label.items():
|
||||
df[col] = val
|
||||
df["replicate"] = rep
|
||||
frames.append(df)
|
||||
out = pd.concat(frames, ignore_index=True)
|
||||
out.insert(0, "experiment", cfg["experiment"])
|
||||
return out
|
||||
|
||||
|
||||
def run_coverage(cfg: dict) -> pd.DataFrame:
|
||||
"""E4 runner: multi-teacher recombination coverage (blueprint 2.5-E4 / 2.7.1).
|
||||
|
||||
|
|
@ -278,7 +318,16 @@ def run_and_save(config_path: str | Path) -> Path:
|
|||
config_path = Path(config_path)
|
||||
cfg = yaml.safe_load(config_path.read_text())
|
||||
out_dir = Path(cfg.get("output", {}).get("dir", f"results/{cfg['experiment']}"))
|
||||
df = run_coverage(cfg) if cfg.get("kind") == "coverage" else run_experiment(cfg)
|
||||
kind = cfg.get("kind", "lineage")
|
||||
if kind == "coverage":
|
||||
df = run_coverage(cfg)
|
||||
elif kind == "genotype_lineage":
|
||||
df = run_genotype_experiment(cfg) # E7: advantage of sex
|
||||
elif kind == "society":
|
||||
from .society import run_society # E8: multi-parent recombination
|
||||
df = run_society(cfg)
|
||||
else:
|
||||
df = run_experiment(cfg)
|
||||
save_artifacts(cfg, df, out_dir)
|
||||
return out_dir
|
||||
|
||||
|
|
|
|||
202
src/knowledge/genotype.py
Normal file
202
src/knowledge/genotype.py
Normal file
|
|
@ -0,0 +1,202 @@
|
|||
"""Multi-locus genotype space + recombination — the theoretical frame for the society.
|
||||
|
||||
The single-locus, fixed-`p*` model can only express *recovery toward a ceiling*. The society's
|
||||
vertical claim — capability that *exceeds* any component — needs combinatorial structure. A
|
||||
**genotype** is `L` biallelic loci (allele 1 = "correct", 0 = "wrong"); a model's knowledge is a
|
||||
distribution over the `2^L` genotypes (a K-vector with `K = 2^L`, so all of Layer 1's K-mode
|
||||
machinery — drift, grounding, selection, metrics — applies unchanged). Fitness is additive (the
|
||||
number of correct loci); the optimum is the all-correct genotype.
|
||||
|
||||
The one genuinely new operator is **recombination**. Free recombination replaces the joint genotype
|
||||
distribution by the product of its per-locus marginals (linkage equilibrium / Robbins proportions) —
|
||||
the population-level image of meiotic reassortment. This is what halts Muller's ratchet (it
|
||||
reconstitutes low-load genotypes from complementary high-load parents) and drives the Fisher–Muller
|
||||
effect (it assembles beneficial alleles that arose in different lineages into a genotype fitter than
|
||||
any parent). `rate=0` is asexual/clonal (Layer 1's regime); `rate=1` is free recombination.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
|
||||
from .metrics import heterozygosity
|
||||
|
||||
|
||||
def genotype_bits(L: int) -> np.ndarray:
|
||||
"""Return the ``(2^L, L)`` matrix of genotype bit-vectors (row g = g in binary, MSB first).
|
||||
|
||||
Args:
|
||||
L (int): Number of biallelic loci.
|
||||
|
||||
Returns:
|
||||
np.ndarray: ``(2^L, L)`` int8 array; ``bits[g, l]`` is allele of locus ``l`` in genotype ``g``.
|
||||
"""
|
||||
g = np.arange(1 << L, dtype=np.int64)
|
||||
shifts = np.arange(L - 1, -1, -1, dtype=np.int64)
|
||||
return ((g[:, None] >> shifts[None, :]) & 1).astype(np.int8)
|
||||
|
||||
|
||||
def additive_fitness(L: int) -> np.ndarray:
|
||||
"""Additive fitness vector: ``f(g)`` = number of correct (allele-1) loci in genotype ``g``.
|
||||
|
||||
Args:
|
||||
L (int): Number of loci.
|
||||
|
||||
Returns:
|
||||
np.ndarray: Length-``2^L`` fitness vector in ``{0, 1, ..., L}``; optimum = all-ones genotype.
|
||||
"""
|
||||
return genotype_bits(L).sum(axis=1).astype(float)
|
||||
|
||||
|
||||
def locus_marginals(p: np.ndarray, L: int) -> np.ndarray:
|
||||
"""Per-locus frequency of the correct (allele-1) variant under distribution ``p``.
|
||||
|
||||
Args:
|
||||
p (np.ndarray): Genotype distribution (length ``2^L``, sums to 1).
|
||||
L (int): Number of loci.
|
||||
|
||||
Returns:
|
||||
np.ndarray: Length-``L`` array; entry ``l`` is ``P(locus l is correct)``.
|
||||
"""
|
||||
return genotype_bits(L).T.astype(float) @ np.asarray(p, dtype=float) # (L, 2^L) @ (2^L,)
|
||||
|
||||
|
||||
def linkage_equilibrium(marginals: np.ndarray) -> np.ndarray:
|
||||
"""Build the product (linkage-equilibrium) genotype distribution from per-locus marginals.
|
||||
|
||||
``p_LE(g) = ∏_l [q_l if g_l==1 else (1−q_l)]`` — the joint under free recombination, where the
|
||||
loci are statistically independent (Robbins proportions).
|
||||
|
||||
Args:
|
||||
marginals (np.ndarray): Length-``L`` correct-allele frequencies ``q_l``.
|
||||
|
||||
Returns:
|
||||
np.ndarray: Length-``2^L`` product distribution (sums to 1).
|
||||
"""
|
||||
q = np.asarray(marginals, dtype=float)
|
||||
L = q.size
|
||||
bits = genotype_bits(L) # (2^L, L)
|
||||
per = np.where(bits == 1, q[None, :], 1.0 - q[None, :]) # (2^L, L)
|
||||
p = per.prod(axis=1)
|
||||
s = p.sum()
|
||||
return p / s if s > 0 else p
|
||||
|
||||
|
||||
def recombine(p: np.ndarray, L: int, rate: float) -> np.ndarray:
|
||||
"""Apply recombination at ``rate`` to a genotype distribution.
|
||||
|
||||
``rate=0`` returns ``p`` unchanged (asexual / clonal); ``rate=1`` returns the full
|
||||
linkage-equilibrium product of ``p``'s marginals (free recombination); intermediate rates
|
||||
interpolate ``(1−rate)·p + rate·p_LE``.
|
||||
|
||||
Args:
|
||||
p (np.ndarray): Genotype distribution (length ``2^L``).
|
||||
L (int): Number of loci.
|
||||
rate (float): Recombination rate in ``[0, 1]``.
|
||||
|
||||
Returns:
|
||||
np.ndarray: The recombined distribution (sums to 1).
|
||||
"""
|
||||
p = np.asarray(p, dtype=float)
|
||||
if rate <= 0.0:
|
||||
return p
|
||||
p_le = linkage_equilibrium(locus_marginals(p, L))
|
||||
out = p_le if rate >= 1.0 else (1.0 - rate) * p + rate * p_le
|
||||
s = out.sum()
|
||||
return out / s if s > 0 else out
|
||||
|
||||
|
||||
def recombine_teachers(teachers: list[np.ndarray], L: int, rate: float) -> np.ndarray:
|
||||
"""Merge ``K_T`` teacher genotype distributions (the multi-teacher / Fisher–Muller operator).
|
||||
|
||||
``rate=0`` = clonal mean-mixture (``mean(teachers)`` — keeps genotypes intact, cannot create a
|
||||
genotype no teacher had). ``rate=1`` = sexual merge: the linkage-equilibrium product of the
|
||||
*pooled* per-locus marginals, which assembles the best allele of each locus across teachers into
|
||||
genotypes none of them held — the Fisher–Muller effect and "merge, don't average" at the genotype
|
||||
level.
|
||||
|
||||
Args:
|
||||
teachers (list[np.ndarray]): ``K_T`` genotype distributions (each length ``2^L``).
|
||||
L (int): Number of loci.
|
||||
rate (float): Recombination rate in ``[0, 1]``.
|
||||
|
||||
Returns:
|
||||
np.ndarray: The merged genotype distribution (sums to 1).
|
||||
"""
|
||||
stack = np.asarray(teachers, dtype=float)
|
||||
mean_mix = stack.mean(axis=0)
|
||||
if rate <= 0.0:
|
||||
return mean_mix
|
||||
pooled_marginals = locus_marginals(mean_mix, L) # pooled per-locus correct-allele freq
|
||||
p_le = linkage_equilibrium(pooled_marginals)
|
||||
out = p_le if rate >= 1.0 else (1.0 - rate) * mean_mix + rate * p_le
|
||||
s = out.sum()
|
||||
return out / s if s > 0 else out
|
||||
|
||||
|
||||
def mutate(p: np.ndarray, L: int, mu: float) -> np.ndarray:
|
||||
"""Apply per-locus symmetric mutation at rate ``mu`` to a genotype distribution.
|
||||
|
||||
Each locus independently flips with probability ``mu``. At the distribution level this convolves
|
||||
``p`` with the per-locus flip kernel; implemented as ``L`` independent locus mixings. ``mu=0`` is
|
||||
a no-op. Provides the mutational pressure that Muller's ratchet grinds against.
|
||||
|
||||
Args:
|
||||
p (np.ndarray): Genotype distribution (length ``2^L``).
|
||||
L (int): Number of loci.
|
||||
mu (float): Per-locus flip probability in ``[0, 0.5]``.
|
||||
|
||||
Returns:
|
||||
np.ndarray: The mutated distribution (sums to 1).
|
||||
"""
|
||||
p = np.asarray(p, dtype=float)
|
||||
if mu <= 0.0:
|
||||
return p
|
||||
q = p.reshape([2] * L) if L > 0 else p
|
||||
for axis in range(L):
|
||||
flipped = np.flip(q, axis=axis)
|
||||
q = (1.0 - mu) * q + mu * flipped
|
||||
out = q.reshape(-1)
|
||||
s = out.sum()
|
||||
return out / s if s > 0 else out
|
||||
|
||||
|
||||
def locus_metrics(p: np.ndarray, L: int, fitness: np.ndarray, alive_eps: float = 1e-9) -> dict:
|
||||
"""Genotype-aware metrics for one generation (the multi-locus analogue of the K-mode row).
|
||||
|
||||
Args:
|
||||
p (np.ndarray): Genotype distribution (length ``2^L``).
|
||||
L (int): Number of loci.
|
||||
fitness (np.ndarray): Length-``2^L`` fitness (typically :func:`additive_fitness`).
|
||||
alive_eps (float): A genotype/allele is "present" if its probability exceeds this.
|
||||
|
||||
Returns:
|
||||
dict: ``mean_fitness``, ``best_fitness`` (max fitness of any present genotype),
|
||||
``opt_freq`` (mass on the all-correct optimum), ``min_load`` (fewest wrong loci among present
|
||||
genotypes = the Muller-ratchet state), ``mean_locus_correct`` (mean per-locus correct-allele
|
||||
freq), ``locus_H`` (mean per-locus heterozygosity), and ``ld`` (mean pairwise linkage
|
||||
disequilibrium |D|).
|
||||
"""
|
||||
p = np.asarray(p, dtype=float)
|
||||
present = p > alive_eps
|
||||
q = locus_marginals(p, L) # per-locus correct freq
|
||||
bits = genotype_bits(L)
|
||||
row = {
|
||||
"mean_fitness": float(fitness @ p),
|
||||
"best_fitness": float(fitness[present].max()) if present.any() else 0.0,
|
||||
"opt_freq": float(p[-1]), # genotype 2^L-1 = all-ones optimum
|
||||
"min_load": int(L - fitness[present].max()) if present.any() else L,
|
||||
"mean_locus_correct": float(q.mean()),
|
||||
"locus_H": float(np.mean([heterozygosity(np.array([1 - qi, qi])) for qi in q])),
|
||||
}
|
||||
# Mean pairwise linkage disequilibrium |D_ij| = |P(1_i,1_j) - q_i q_j| over locus pairs.
|
||||
if L >= 2:
|
||||
ds = []
|
||||
for i in range(L):
|
||||
for j in range(i + 1, L):
|
||||
p_ij = float(p[(bits[:, i] == 1) & (bits[:, j] == 1)].sum())
|
||||
ds.append(abs(p_ij - q[i] * q[j]))
|
||||
row["ld"] = float(np.mean(ds))
|
||||
else:
|
||||
row["ld"] = 0.0
|
||||
return row
|
||||
82
src/knowledge/genotype_lineage.py
Normal file
82
src/knowledge/genotype_lineage.py
Normal file
|
|
@ -0,0 +1,82 @@
|
|||
"""Single-population genotype evolution — the advantage of sex (E7).
|
||||
|
||||
A population (distribution over the ``2^L`` genotypes) adapts toward a multi-locus optimum under
|
||||
the composed generational step: **selection** (fitness-proportional, favouring correct alleles) +
|
||||
**drift** (finite resample of ``n``) + **mutation** (per-locus flips) + **recombination** (asexual
|
||||
``rate=0`` vs sexual ``rate>0``). Recombination reassorts beneficial alleles that arise in different
|
||||
sub-lineages into one genotype; without it (asexual) those alleles suffer *clonal interference* and
|
||||
adaptation is slower. So a sexual lineage climbs toward the optimum faster than an asexual one — the
|
||||
classical advantage of sex, and the dynamic counterpart of E8's one-shot multi-parent assembly.
|
||||
|
||||
Reuses ``step.apply_selection`` (fitness = number of correct loci) and the ``genotype`` operators;
|
||||
emits the same tidy per-generation DataFrame contract as ``run_lineage`` (with genotype-aware
|
||||
columns from ``genotype.locus_metrics``).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Mapping
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from .genotype import additive_fitness, locus_metrics, mutate, recombine
|
||||
from .step import apply_selection
|
||||
|
||||
|
||||
def run_genotype_lineage(cfg: Mapping[str, Any], seed: int) -> pd.DataFrame:
|
||||
"""Run one genotype lineage and return per-generation genotype metrics.
|
||||
|
||||
Args:
|
||||
cfg (Mapping): Config with a ``genotype`` block (``L``; drift ``n``; mutation ``mu``;
|
||||
selection ``base`` for multiplicative fitness ``base^#correct``; recombination
|
||||
``recomb_rate``; ``init`` in {``wrong``, ``uniform``, ``optimum``}) and
|
||||
``generations``.
|
||||
seed (int): Replicate seed; the run is a pure function of (cfg, seed).
|
||||
|
||||
Returns:
|
||||
pd.DataFrame: One row per generation 0..T with ``generation`` plus the
|
||||
``genotype.locus_metrics`` columns (``mean_fitness``, ``best_fitness``, ``opt_freq``,
|
||||
``min_load``, ``mean_locus_correct``, ``locus_H``, ``ld``).
|
||||
"""
|
||||
g = cfg["genotype"]
|
||||
L = int(g["L"])
|
||||
n = int(g["n"])
|
||||
mu = float(g.get("mu", 0.0))
|
||||
base = float(g.get("base", 1.0))
|
||||
rate = float(g.get("recomb_rate", 0.0))
|
||||
generations = int(cfg.get("generations", 100))
|
||||
K = 1 << L
|
||||
|
||||
report_fitness = additive_fitness(L) # # correct loci (0..L), for metrics
|
||||
sel_fitness = base ** report_fitness # multiplicative selection weight
|
||||
|
||||
init = g.get("init", "wrong")
|
||||
p = np.zeros(K)
|
||||
if init == "wrong":
|
||||
p[0] = 1.0 # all-wrong genotype (load L); adapt upward
|
||||
elif init == "optimum":
|
||||
p[-1] = 1.0 # all-correct (for degradation studies)
|
||||
elif init == "uniform":
|
||||
p[:] = 1.0 / K
|
||||
else:
|
||||
raise ValueError(f"unknown genotype init {init!r} (expected wrong|optimum|uniform)")
|
||||
|
||||
rng = np.random.default_rng(seed)
|
||||
rows: list[dict] = []
|
||||
|
||||
def record(t: int) -> None:
|
||||
row = {"generation": t}
|
||||
row.update(locus_metrics(p, L, report_fitness, alive_eps=1.0 / n))
|
||||
rows.append(row)
|
||||
|
||||
record(0)
|
||||
for t in range(1, generations + 1):
|
||||
p = apply_selection(p, sel_fitness, "greedy", 0.0) # fitness-proportional selection
|
||||
counts = rng.multinomial(n, p) # drift
|
||||
p = counts / counts.sum()
|
||||
p = mutate(p, L, mu) # per-locus mutation
|
||||
p = recombine(p, L, rate) # asexual (0) vs sexual (>0)
|
||||
record(t)
|
||||
|
||||
return pd.DataFrame(rows)
|
||||
95
src/knowledge/society.py
Normal file
95
src/knowledge/society.py
Normal file
|
|
@ -0,0 +1,95 @@
|
|||
"""Multi-parent recombination — the Fisher–Muller vertical claim (E8).
|
||||
|
||||
The society's headline: **an offspring recombined from many decorrelated parents can be fitter
|
||||
than any parent** — capability that *exceeds* every component, not just recovers a ceiling. This
|
||||
is the Fisher–Muller effect, and unlike biological sex it has **no two-parent limit**: an offspring
|
||||
here can have arbitrarily many parents (``recombine_teachers`` pools per-locus marginals over all of
|
||||
them).
|
||||
|
||||
Each parent is a *specialist*: confident-correct on the loci it has mastered, agnostic (~0.5) on the
|
||||
rest — the realistic picture of an expert. Which loci each parent masters comes from the exact
|
||||
shared-switch construction (``teachers.make_retention_matrix``), so parent count ``K_T`` and
|
||||
decorrelation ``rho`` are clean, independently-swept knobs (mastery of a *locus* replaces retention
|
||||
of a *tail item*). Three deployable capabilities are compared, all as the fitness of the *mode*
|
||||
(most-probable) genotype — what you would actually ship:
|
||||
|
||||
* **best_parent** — the single fittest specialist (no combination).
|
||||
* **average** — the mean-mixture "model soup" (combine, but don't recombine loci).
|
||||
* **sexual** — the union-preserving recombination (assemble the best allele of each locus).
|
||||
|
||||
Sexual climbs to the optimum as parents accumulate and decorrelate; the other two plateau.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import itertools
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from .genotype import additive_fitness, linkage_equilibrium, recombine_teachers
|
||||
from .seeding import spawn_seeds
|
||||
from .teachers import make_retention_matrix
|
||||
|
||||
|
||||
def make_specialist(mastery: np.ndarray, hi: float, lo: float) -> np.ndarray:
|
||||
"""Build a specialist's genotype distribution from its per-locus mastery mask.
|
||||
|
||||
Args:
|
||||
mastery (np.ndarray): Length-``L`` bool; True where this parent is expert.
|
||||
hi (float): Correct-allele probability on mastered loci (confident).
|
||||
lo (float): Correct-allele probability on unmastered loci (agnostic, ~0.5).
|
||||
|
||||
Returns:
|
||||
np.ndarray: Length-``2^L`` genotype distribution (product / linkage-equilibrium form).
|
||||
"""
|
||||
q = np.where(np.asarray(mastery, dtype=bool), hi, lo)
|
||||
return linkage_equilibrium(q)
|
||||
|
||||
|
||||
def _mode_fitness(p: np.ndarray, fitness: np.ndarray) -> float:
|
||||
"""Fitness of the most-probable (deployed consensus) genotype."""
|
||||
return float(fitness[int(np.argmax(p))])
|
||||
|
||||
|
||||
def run_society(cfg: dict) -> pd.DataFrame:
|
||||
"""Sweep parent count x decorrelation; compare best-parent vs average vs sexual capability.
|
||||
|
||||
Args:
|
||||
cfg (dict): Parsed experiment config with a ``society`` block (``L``, ``q`` mastery
|
||||
fraction, ``hi``, ``lo``), a ``sweep`` (``K_T`` x ``rho``), ``seed``, ``n_replicates``.
|
||||
|
||||
Returns:
|
||||
pd.DataFrame: One row per (K_T, rho, replicate) with ``best_parent``, ``average``,
|
||||
``sexual`` (mode-genotype fitness, 0..L) and ``L``.
|
||||
"""
|
||||
soc = cfg["society"]
|
||||
L = int(soc["L"])
|
||||
q = float(soc.get("q", 0.5))
|
||||
hi, lo = float(soc.get("hi", 0.9)), float(soc.get("lo", 0.45))
|
||||
fitness = additive_fitness(L)
|
||||
|
||||
sweeps = cfg["sweep"]
|
||||
if isinstance(sweeps, dict):
|
||||
sweeps = [sweeps]
|
||||
params = [s["param"] for s in sweeps]
|
||||
value_lists = [list(s["values"]) for s in sweeps]
|
||||
seeds = spawn_seeds(int(cfg["seed"]), int(cfg["n_replicates"]))
|
||||
|
||||
rows: list[dict] = []
|
||||
for combo in itertools.product(*value_lists):
|
||||
d = dict(zip(params, combo))
|
||||
K_T, rho = int(d["K_T"]), float(d["rho"])
|
||||
for rep, ss in enumerate(seeds):
|
||||
child = int(ss.generate_state(1)[0])
|
||||
rng = np.random.default_rng(child)
|
||||
mastery = make_retention_matrix(L, K_T, rho, q, rng).astype(bool) # (K_T, L)
|
||||
parents = [make_specialist(mastery[k], hi, lo) for k in range(K_T)]
|
||||
rows.append({
|
||||
"experiment": cfg["experiment"], "K_T": K_T, "rho": rho, "q": q, "L": L,
|
||||
"replicate": rep,
|
||||
"best_parent": max(_mode_fitness(p, fitness) for p in parents),
|
||||
"average": _mode_fitness(recombine_teachers(parents, L, 0.0), fitness),
|
||||
"sexual": _mode_fitness(recombine_teachers(parents, L, 1.0), fitness),
|
||||
})
|
||||
return pd.DataFrame(rows)
|
||||
|
|
@ -264,8 +264,30 @@ C3 vertical claim deferred.*
|
|||
Lamarckian society + vertical/cumulative C3 claim (not yet run). Reposition: from "collapse is drift"
|
||||
to a population-genetic CONTROL THEORY for sustaining open-ended knowledge.
|
||||
|
||||
## Remaining (all optional)
|
||||
**2026-07-05 — multi-locus society frame (E7/E8): raised the ceiling to enter the society.**
|
||||
|
||||
- Prompted by "enter the society with a robust theoretical frame." The single-locus fixed-`p*` model
|
||||
can't express "exceeding" a ceiling. Generalized knowledge to a distribution over **genotypes**
|
||||
(`knowledge/genotype.py`: `L` biallelic loci, `K=2^L`, additive fitness, recombination = product of
|
||||
per-locus marginals). Reuses all K-mode machinery + `make_retention_matrix` (locus mastery).
|
||||
- **E8 (star, `kind: society`, `knowledge/society.py`) — the vertical claim:** decorrelated parents
|
||||
recombined; **sexual merge reaches the optimum (12/12, a genotype no parent had)** as parent count
|
||||
grows / `ρ→0`, while best-parent (~8.7) and mean-mixture soup (~11.6) plateau. `configs/layer1/E8.yaml`,
|
||||
`plot_E8.py`, README. The Fisher–Muller effect for AI.
|
||||
- **E7 (`kind: genotype_lineage`, `knowledge/genotype_lineage.py`) — advantage of sex:** sexual lineage
|
||||
adapts faster than asexual (LD→0 vs LD spike). Honest: a speed advantage, not a permanent ratchet gap.
|
||||
- **Metaphor shift (GG):** sexual reproduction with **unbounded parents**, not teacher→pupil (which caps
|
||||
at the ceiling). Collapse = asexual degradation; cure = sex, no parent limit. Unifies E4+E6 under
|
||||
evolution-of-sex theory; beyond Riis's single-locus n-grams. `tests/test_genotype.py` (+7).
|
||||
Experiment dispatch (`kind` in {genotype_lineage, society}) + `make layer1` wired.
|
||||
|
||||
## Remaining (all optional / next)
|
||||
|
||||
- [ ] **The dynamic society:** an evolving *population of parents* (specialists) that ground +
|
||||
recombine + select over generations — capability climbing while specialties are re-earned (the full
|
||||
C3, dynamic). E7/E8 give the static + single-population pieces; the multi-parent *lineage* is next.
|
||||
- [ ] **NK/epistasis landscape** (sign epistasis can make recombination harmful — the honest limit of
|
||||
"sex always helps"); **multi-allelic loci**. Deepens the frame.
|
||||
- [ ] **Learning-kernel refinement:** truth-like smoothing prior (`prior="truth"`) + measurement floor
|
||||
for a quantitative RNN match; **multi-locus / linkage** modes (class×style) as the rigorous home for
|
||||
recombination. Both enrich predictive power and separate us further from Riis's single-locus n-grams.
|
||||
|
|
|
|||
90
tests/test_genotype.py
Normal file
90
tests/test_genotype.py
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
"""Multi-locus / recombination tests (pure NumPy) — the spine of the society frame.
|
||||
|
||||
Cover the genotype algebra (bits, additive fitness, linkage equilibrium), the recombination
|
||||
operators (identity at rate 0, product-of-marginals at rate 1), and the two headline behaviours:
|
||||
E8 — sexual recombination of decorrelated parents *exceeds* the best parent; E7 — a sexual lineage
|
||||
adapts faster than an asexual one.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import numpy as np
|
||||
|
||||
from knowledge.genotype import (
|
||||
additive_fitness, genotype_bits, linkage_equilibrium, locus_marginals, mutate,
|
||||
recombine, recombine_teachers,
|
||||
)
|
||||
from knowledge.genotype_lineage import run_genotype_lineage
|
||||
from knowledge.society import make_specialist, run_society
|
||||
from knowledge.teachers import make_retention_matrix
|
||||
|
||||
|
||||
def test_genotype_bits_and_additive_fitness():
|
||||
bits = genotype_bits(3)
|
||||
assert bits.shape == (8, 3)
|
||||
assert np.array_equal(bits[0], [0, 0, 0]) and np.array_equal(bits[7], [1, 1, 1])
|
||||
f = additive_fitness(3)
|
||||
assert f[0] == 0 and f[7] == 3 and f[1] == 1 # MSB-first: g=1 -> [0,0,1]
|
||||
|
||||
|
||||
def test_linkage_equilibrium_is_product_and_normalised():
|
||||
q = np.array([0.9, 0.5, 0.2])
|
||||
p = linkage_equilibrium(q)
|
||||
assert np.isclose(p.sum(), 1.0)
|
||||
# P(all-correct) = ∏ q; P(all-wrong) = ∏ (1-q)
|
||||
assert np.isclose(p[-1], np.prod(q)) and np.isclose(p[0], np.prod(1 - q))
|
||||
assert np.allclose(locus_marginals(p, 3), q) # marginals round-trip
|
||||
|
||||
|
||||
def test_recombine_identity_and_free():
|
||||
rng = np.random.default_rng(0)
|
||||
p = rng.dirichlet(np.ones(16)) # L=4
|
||||
assert np.array_equal(recombine(p, 4, 0.0), p) # rate 0 = asexual (unchanged)
|
||||
free = recombine(p, 4, 1.0) # rate 1 = product of marginals
|
||||
assert np.allclose(free, linkage_equilibrium(locus_marginals(p, 4)))
|
||||
assert np.isclose(free.sum(), 1.0)
|
||||
|
||||
|
||||
def test_mutate_normalises_and_spreads():
|
||||
L = 4
|
||||
p = np.zeros(1 << L); p[0] = 1.0 # point mass on all-wrong
|
||||
out = mutate(p, L, 0.1)
|
||||
assert np.isclose(out.sum(), 1.0)
|
||||
assert out[0] < 1.0 and (out > 0).sum() > 1 # mass leaks to neighbours
|
||||
|
||||
|
||||
def test_recombine_teachers_assembles_optimum_only_sexually():
|
||||
# Two complementary parents: one masters the low loci, the other the high loci. Sexual merge
|
||||
# assembles the all-correct optimum (a genotype NEITHER parent has as its mode); clonal cannot.
|
||||
L = 8
|
||||
lo = np.array([1, 1, 1, 1, 0, 0, 0, 0], dtype=bool)
|
||||
parents = [make_specialist(lo, 0.9, 0.45), make_specialist(~lo, 0.9, 0.45)]
|
||||
f = additive_fitness(L)
|
||||
best_parent = max(f[int(np.argmax(p))] for p in parents)
|
||||
sexual = f[int(np.argmax(recombine_teachers(parents, L, 1.0)))]
|
||||
clonal = f[int(np.argmax(recombine_teachers(parents, L, 0.0)))]
|
||||
assert sexual == L # sexual assembles the optimum
|
||||
assert sexual > best_parent # ... which exceeds either parent
|
||||
assert clonal <= best_parent + 0 # clonal (soup) does not assemble it
|
||||
|
||||
|
||||
def test_e8_sexual_exceeds_best_parent_and_soup():
|
||||
cfg = {"experiment": "e8t", "seed": 1, "n_replicates": 8,
|
||||
"society": {"L": 10, "q": 0.5, "hi": 0.9, "lo": 0.45},
|
||||
"sweep": [{"param": "K_T", "values": [8]}, {"param": "rho", "values": [0.0]}]}
|
||||
df = run_society(cfg)
|
||||
m = df.mean(numeric_only=True)
|
||||
assert m["sexual"] > m["best_parent"] + 1.5 # clearly exceeds the best parent
|
||||
assert m["sexual"] >= m["average"] # and is at least as good as the soup
|
||||
|
||||
|
||||
def test_e7_sexual_adapts_at_least_as_fast():
|
||||
base = {"genotype": {"L": 10, "n": 150, "mu": 0.02, "base": 1.3, "init": "wrong"},
|
||||
"generations": 25}
|
||||
asex = run_genotype_lineage({**base, "genotype": {**base["genotype"], "recomb_rate": 0.0}}, 0)
|
||||
sex = run_genotype_lineage({**base, "genotype": {**base["genotype"], "recomb_rate": 1.0}}, 0)
|
||||
mid = 12
|
||||
a = asex[asex["generation"] == mid]["mean_fitness"].iloc[0]
|
||||
s = sex[sex["generation"] == mid]["mean_fitness"].iloc[0]
|
||||
assert s >= a - 1e-9 # sexual adapts at least as fast mid-run
|
||||
assert sex["ld"].max() < asex["ld"].max() # ... by keeping loci in linkage equilibrium
|
||||
Loading…
Add table
Add a link
Reference in a new issue