Giorgio Gilestro
a40ace1821
Analyses (figures/stats_llm_epistasis.py, committed + reproducible): condition-clustered bootstrap CIs (functional measures exclude zero: dis_raw [+0.04,+0.69], conf-weighted [+0.02,+0.68]; gradient alignment [-0.59,-0.06]; geometry straddles zero), PAIRED predictor contrasts (not individually significant — stated), leave-one-condition-out held-out prediction (functional replicates, geometry ~0, performance baseline unstable), three outcome references (ordering sensitive to reference — reported, with the mechanism), between/within-axis decomposition (within-conflict identification impossible by design; the compat axis identifies), and seed-level paired reliability (routing/directed beat soup 3/3 seeds incl. one catastrophic soup failure; CI-width fragility claim withdrawn). Renames and corrections: "decisive experiment" -> "controlled predictive test"; "operational epistasis" -> "confidence-weighted functional conflict (proposed proxy)"; "functional by construction" -> "controls a major source of coordinate mismatch / conflict-associated" (module, configs, READMEs, figures); SI proposition's "chord" defined precisely (endpoint-loss interpolation, invariant) vs the path (not invariant) + no-global-optimality caveat (removable = lower bound, residual = upper); snowball count != performance cliff distinction added; claims table gains four rows (grid finding / weighting NOT supported / functional-vs- all-geometry not established / operator choice open); §1 ladder states the prediction rung as a bounded small-model result. paper/response-to-review-2.md: point-by-point, opening with the bookkeeping correction (E13b/c were in the reviewed draft — revised interpretation, not new results). READMEs rewritten around the four analyses with the chronology (prospective/adaptive/post-hoc) disclosed. 151 tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BkRLcc18rwT2Lysu6PbG7v
32 lines
1.7 KiB
YAML
32 lines
1.7 KiB
YAML
experiment: llm_speciation
|
|
kind: llm_speciation
|
|
seed: 1
|
|
n_replicates: 1
|
|
|
|
# LLM-tier model speciation (E13 in language-model weights; PNAS work order Phase 3). Two LoRA
|
|
# children from the same frozen base — which controls a major source of coordinate mismatch (no
|
|
# alignment step needed), allowing a cleaner test of conflict-associated merge failure. Two sweeps:
|
|
# conflict_fracs — the IMPOSED cliff: each child has a private disjoint family (A: strings,
|
|
# B: arith) plus a shared set of AMBIGUOUS sort prompts ("Sort the list [...]", no direction)
|
|
# answered ascending by A and descending by B (each convention harmless alone, contradictory
|
|
# jointly — the BDM structure). Prediction: the MERGED model's private-family competence degrades
|
|
# and its convention coherence (max of asc/desc grading) collapses as conflict grows, while each
|
|
# PARENT stays fine — hybrid breakdown in verifier units, echoing the MLP cliff.
|
|
# durations — the EMERGENT null: pure disjoint specialists over-trained (epochs swept), zero shared
|
|
# data. Arbitrates the MLP tier's null (no emergent isolation; the merge rescued specialists at
|
|
# every divergence) against the empirical report that averaging prefers under-trained experts
|
|
# (arXiv:2607.11997). Pre-registered readings: merged quality falls with duration while parents'
|
|
# own-family quality holds -> emergent incompatibility at the LLM tier; otherwise the null
|
|
# generalises. Either outcome is reportable; do not tune toward one.
|
|
|
|
base_model: Qwen/Qwen2.5-0.5B-Instruct
|
|
family_a: strings
|
|
family_b: arith
|
|
n_train: 400
|
|
n_test: 100
|
|
epochs: 3
|
|
lora: {r: 16, alpha: 32}
|
|
conflict_fracs: [0.0, 0.25, 0.5, 0.75, 1.0]
|
|
durations: [1, 3, 6, 12]
|
|
|
|
output: {dir: results/llm_speciation}
|