Manuscript revision and pending experiment work, snapshot before restructuring
Clarity pass over the main text (36-item audit), Discussion rewrite and cut, acknowledgements, Souly et al. as ref 62, lettered SI panels, model section moved under Results; plus the untracked curriculum/society/compose/smol configs, runners, figures, stats and tests that the SI already cites. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Y64o8FKP7rCuXzC48pxpMm
This commit is contained in:
parent
e4804adabc
commit
84124de143
450 changed files with 52813 additions and 1202 deletions
31
configs/llm/compose_gate.yaml
Normal file
31
configs/llm/compose_gate.yaml
Normal file
|
|
@ -0,0 +1,31 @@
|
|||
# Generation-0 gate for the composition experiment (prereg v3 §4: C1 base, C3 specialists,
|
||||
# C4 replication). Trains the two founders and measures everything at t=0 without iterating.
|
||||
#
|
||||
# Base choice (C1, measured 2026-09-07): Qwen2.5-1.5B **base, not Instruct**. Zero-shot GSM-Hard
|
||||
# program-aided: Instruct-1.5B 0.500, Instruct-3B 0.417, base-3B 0.633 — all far too strong, since a
|
||||
# base that already has the skills makes the specialists' contribution vacuous. Base-1.5B scores
|
||||
# 0.067, within noise of the 0.059 Llama-2-7B starting point in LoRA Soups (COLING 2025), so the
|
||||
# published composition effect has room to appear.
|
||||
experiment: llm_compose_gate
|
||||
kind: llm_compose
|
||||
base_model: Qwen/Qwen2.5-1.5B
|
||||
seed: 1
|
||||
generations: 0 # gate only: train founders, measure t=0, stop
|
||||
arms: [dry]
|
||||
n_hard: 150 # composed target (GSM8k-Hard, program-aided) — SE ~0.04
|
||||
n_gsm8k: 100 # math own-skill probe
|
||||
n_mbpp: 80 # code own-skill probe (execution-verified)
|
||||
n_probe: 40 # shared prompts for rho
|
||||
k_inherit: 300
|
||||
epochs: 3
|
||||
conf_gate: 0.85
|
||||
g: 0.10
|
||||
spec_train: 1200
|
||||
spec_epochs: 3
|
||||
max_new_tokens: 320
|
||||
batch_size: 16
|
||||
score_batch_size: 4
|
||||
train_batch_size: 2 # 16 GB A4000; the fp32 logit upcast at vocab 152k is the constraint
|
||||
train_max_len: 448
|
||||
lora: {r: 16, alpha: 32}
|
||||
output: {dir: results/llm_compose_gate}
|
||||
Loading…
Add table
Add a link
Reference in a new issue