Restructure: descriptive tier and experiment names, paper/manuscript
- paper/pnas -> paper/manuscript (venue-neutral)
- configs/layer1 -> configs/inheritance, src/knowledge -> src/inheritance
(imported as `inheritance`), make layer1 -> make inheritance; layer2 alias dropped
- inheritance and trained-network bundles named after the manuscript figure
they feed (fig2_grounding_sweep, figS3_rebaselining, ...), or descriptively
where they feed none; configs keep their `experiment:` value so parquet
hashes are unchanged, only output.dir moves
- figure scripts, SI figure sources, notebooks, REPRODUCING.md, README and the
SI Methods/tables updated; make clean no longer deletes tracked manifests;
reproduce.sh hashes the s{seed}/ layouts too
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
This commit is contained in:
parent
84124de143
commit
ab3dc10587
240 changed files with 477 additions and 476 deletions
33
results/figS8_multiparent_union/README.md
Normal file
33
results/figS8_multiparent_union/README.md
Normal file
|
|
@ -0,0 +1,33 @@
|
|||
# E4 — Recombination supplies the rare tail; only a union-preserving *merge* realises it
|
||||
|
||||
**Claim tested:** if several specialist models each remember a different slice of the rare tail, can
|
||||
combining them reconstruct the whole tail? And does *how* you combine them matter?
|
||||
|
||||
**Setup (Layer 1, pure math).** `K = 500` items. We build `K_T` teacher models that each retain the
|
||||
rare tail with probability `q`, and we control how **correlated** their retained tails are with a
|
||||
single knob `ρ` (rho): `ρ = 0` = fully complementary teachers, `ρ = 1` = identical teachers. Swept:
|
||||
`K_T ∈ {1,2,3,5}`, `ρ ∈ {0, 0.25, 0.5, 0.75, 1}`, grounding `g ∈ {0, 0.02, 0.05}`, 200 repeats.
|
||||
|
||||
### Symbols
|
||||
- **`K_T`** number of teacher models; **`ρ`** how correlated their retained tails are (0 = diverse, 1 = clones).
|
||||
- **union coverage** — fraction of the tail covered by *at least one* teacher (the raw supply).
|
||||
- **surviving coverage** — fraction that actually survives in the pupil after it retrains on the combination.
|
||||
- **mean-distill** — pupil trained on the pooled/averaged teacher outputs. **max-merge** — keep, per item, the strongest teacher (a union-preserving merge, à la M2N2).
|
||||
|
||||
### The three panels
|
||||
1. **Supply.** Union tail coverage vs `ρ`, one curve per `K_T`; solid lines are the exact closed form
|
||||
`U(K_T, ρ, q)`. More teachers and more diversity (lower `ρ`) supply more of the tail — and the
|
||||
simulation matches the formula exactly.
|
||||
2. **Realisation.** Surviving coverage vs `ρ`. **Solid = max-merge rises** with more/diverse teachers;
|
||||
**dashed = mean-distill stays flat.** Averaging dilutes each teacher's rare items back below the
|
||||
survival threshold — the gain is supplied but not realised.
|
||||
3. **The benefit needs the right operator (`ρ = 0`).** Surviving coverage vs `K_T` under both
|
||||
operators. Max-merge climbs with teacher count; mean-distill is flat — a **conservation law**:
|
||||
averaging's `1/K_T` dilution exactly cancels the union gain.
|
||||
|
||||
### Takeaway — "merge, don't average"
|
||||
Complementary specialists *contain* enough to rebuild the tail, but **naive multi-teacher distillation
|
||||
(averaging) throws it away**; you must combine them with a union-preserving merge. This is load-bearing
|
||||
for the paper's recombination claim and is re-tested in real neural weights in `results/recombination/`.
|
||||
**Falsifier (not triggered):** if mean-distill had also risen with `K_T`, or max never beat mean, the
|
||||
recombination story would collapse into "just average your models."
|
||||
BIN
results/figS8_multiparent_union/figS8_multiparent_union.pdf
Normal file
BIN
results/figS8_multiparent_union/figS8_multiparent_union.pdf
Normal file
Binary file not shown.
BIN
results/figS8_multiparent_union/figS8_multiparent_union.png
Normal file
BIN
results/figS8_multiparent_union/figS8_multiparent_union.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 191 KiB |
14
results/figS8_multiparent_union/manifest.json
Normal file
14
results/figS8_multiparent_union/manifest.json
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"experiment": "E4_multiteacher_decorrelation",
|
||||
"master_seed": 20260704,
|
||||
"git_commit": "a6eb9b75124779375fa1a0b3a64115ecd705b218",
|
||||
"python": "3.14.5",
|
||||
"libraries": {
|
||||
"numpy": "2.5.0",
|
||||
"scipy": "1.18.0",
|
||||
"pandas": "3.0.3",
|
||||
"pyarrow": "24.0.0"
|
||||
},
|
||||
"rows": 12000,
|
||||
"results_sha256": "a200ed088070015f41c983a367a252c09d06a865b45f117769df73fa00df52a2"
|
||||
}
|
||||
39
results/figS8_multiparent_union/resolved_config.yaml
Normal file
39
results/figS8_multiparent_union/resolved_config.yaml
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
experiment: E4_multiteacher_decorrelation
|
||||
seed: 20260704
|
||||
n_replicates: 200
|
||||
source_config:
|
||||
experiment: E4_multiteacher_decorrelation
|
||||
kind: coverage
|
||||
seed: 20260704
|
||||
n_replicates: 200
|
||||
truth:
|
||||
K: 500
|
||||
R: 1
|
||||
tail: zipf
|
||||
zipf_s: 1.1
|
||||
tail_frac: 0.5
|
||||
tail_threshold: 0.002
|
||||
coverage:
|
||||
n: 300
|
||||
q: 0.5
|
||||
sweep:
|
||||
- param: K_T
|
||||
values:
|
||||
- 1
|
||||
- 2
|
||||
- 3
|
||||
- 5
|
||||
- param: rho
|
||||
values:
|
||||
- 0.0
|
||||
- 0.25
|
||||
- 0.5
|
||||
- 0.75
|
||||
- 1.0
|
||||
- param: g
|
||||
values:
|
||||
- 0.0
|
||||
- 0.02
|
||||
- 0.05
|
||||
output:
|
||||
dir: results/figS8_multiparent_union
|
||||
Loading…
Add table
Add a link
Reference in a new issue