Layer 1.5: architecture-general neural existence proof
Re-scopes Layer 2 into a cheaper, architecture-general neural collapse proof
before the LLM rung. Realises the same Wright–Fisher abstractions in real trained
generative models on a fully-synthetic sandbox with an exact oracle, reusing
knowledge.metrics/truth/seeding and the output contract so neural curves overlay
the Layer-1 analytic curves.
- src/neural/: synthetic token-grammar sandbox (lossless identity + stochastic
style), ExactOracle, HistogramModel bridge, generation loop, experiment runner
- HARD GATE passed: histogram lineage reproduces Layer 1 exactly (neutral decay,
exact H_eq, tracks run_lineage) — tests/test_neural_validation.py
- torch models: autoregressive RNN + MLP (VAE implemented, not yet fidelity-
passing); determinism seeding derived from the SeedSequence stream
- N0 bridge (neural g*=0.047 ≈ Layer-1 0.048), N1 collapse-in-weights, N2 phase
boundary, N5 architecture-generality (collapse + grounding-rescue in histogram
+ RNN + MLP). Manifests/configs committed; parquet gitignored, hashes tracked
- additive backward-compatible save_artifacts extension; Makefile neural targets
Finding: neural smoothing partially resists H-collapse, so forward-KL and tail
survival are the sharp neural collapse metrics (H is smooth, per Layer 1).
92 tests green. Remaining (tasks/todo.md): N4 merge, N2 refine, N3/N6, VAE
fidelity, MNIST tier, figures. LLM/LoRA rung and C3 deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
1721d047fa
commit
840b6b00b3
35 changed files with 3679 additions and 23 deletions
275
results/N5/resolved_config.yaml
Normal file
275
results/N5/resolved_config.yaml
Normal file
|
|
@ -0,0 +1,275 @@
|
|||
experiment: N5_architecture_generality
|
||||
seed: 20260704
|
||||
n_replicates: 5
|
||||
source_config:
|
||||
experiment: N5_architecture_generality
|
||||
kind: gen_lineage
|
||||
seed: 20260704
|
||||
n_replicates: 5
|
||||
generations: 22
|
||||
synthetic:
|
||||
K: 256
|
||||
R: 1
|
||||
tail: zipf
|
||||
zipf_s: 1.3
|
||||
tail_frac: 0.5
|
||||
tail_threshold: 0.001
|
||||
init: truth
|
||||
style_len: 3
|
||||
style_vocab: 5
|
||||
id_base: 2
|
||||
model:
|
||||
kind: rnn
|
||||
hidden: 192
|
||||
embed: 24
|
||||
epochs: 25
|
||||
lr: 0.002
|
||||
batch_size: 256
|
||||
n_eval: 12000
|
||||
dynamics:
|
||||
n: 200
|
||||
grounding:
|
||||
m: 0
|
||||
policy: proportional
|
||||
remint:
|
||||
enabled: false
|
||||
period: null
|
||||
H_gate: null
|
||||
metrics:
|
||||
kl_floor: 1.0e-09
|
||||
support_eps: 1.0e-09
|
||||
sweep:
|
||||
- param: model.kind
|
||||
values:
|
||||
- histogram
|
||||
- rnn
|
||||
- mlp
|
||||
- param: g
|
||||
values:
|
||||
- 0.0
|
||||
- 0.05
|
||||
output:
|
||||
dir: results/N5
|
||||
grid:
|
||||
- label:
|
||||
kind: histogram
|
||||
g: 0.0
|
||||
m: 0
|
||||
neural_cfg:
|
||||
synthetic:
|
||||
K: 256
|
||||
R: 1
|
||||
tail: zipf
|
||||
zipf_s: 1.3
|
||||
tail_frac: 0.5
|
||||
tail_threshold: 0.001
|
||||
init: truth
|
||||
style_len: 3
|
||||
style_vocab: 5
|
||||
id_base: 2
|
||||
model:
|
||||
kind: histogram
|
||||
hidden: 192
|
||||
embed: 24
|
||||
epochs: 25
|
||||
lr: 0.002
|
||||
batch_size: 256
|
||||
n_eval: 12000
|
||||
dynamics:
|
||||
n: 200
|
||||
grounding:
|
||||
m: 0
|
||||
policy: proportional
|
||||
remint:
|
||||
enabled: false
|
||||
period: null
|
||||
H_gate: null
|
||||
generations: 22
|
||||
metrics:
|
||||
kl_floor: 1.0e-09
|
||||
support_eps: 1.0e-09
|
||||
- label:
|
||||
kind: histogram
|
||||
g: 0.05
|
||||
m: 11
|
||||
neural_cfg:
|
||||
synthetic:
|
||||
K: 256
|
||||
R: 1
|
||||
tail: zipf
|
||||
zipf_s: 1.3
|
||||
tail_frac: 0.5
|
||||
tail_threshold: 0.001
|
||||
init: truth
|
||||
style_len: 3
|
||||
style_vocab: 5
|
||||
id_base: 2
|
||||
model:
|
||||
kind: histogram
|
||||
hidden: 192
|
||||
embed: 24
|
||||
epochs: 25
|
||||
lr: 0.002
|
||||
batch_size: 256
|
||||
n_eval: 12000
|
||||
dynamics:
|
||||
n: 200
|
||||
grounding:
|
||||
m: 11
|
||||
policy: proportional
|
||||
remint:
|
||||
enabled: false
|
||||
period: null
|
||||
H_gate: null
|
||||
generations: 22
|
||||
metrics:
|
||||
kl_floor: 1.0e-09
|
||||
support_eps: 1.0e-09
|
||||
- label:
|
||||
kind: rnn
|
||||
g: 0.0
|
||||
m: 0
|
||||
neural_cfg:
|
||||
synthetic:
|
||||
K: 256
|
||||
R: 1
|
||||
tail: zipf
|
||||
zipf_s: 1.3
|
||||
tail_frac: 0.5
|
||||
tail_threshold: 0.001
|
||||
init: truth
|
||||
style_len: 3
|
||||
style_vocab: 5
|
||||
id_base: 2
|
||||
model:
|
||||
kind: rnn
|
||||
hidden: 192
|
||||
embed: 24
|
||||
epochs: 25
|
||||
lr: 0.002
|
||||
batch_size: 256
|
||||
n_eval: 12000
|
||||
dynamics:
|
||||
n: 200
|
||||
grounding:
|
||||
m: 0
|
||||
policy: proportional
|
||||
remint:
|
||||
enabled: false
|
||||
period: null
|
||||
H_gate: null
|
||||
generations: 22
|
||||
metrics:
|
||||
kl_floor: 1.0e-09
|
||||
support_eps: 1.0e-09
|
||||
- label:
|
||||
kind: rnn
|
||||
g: 0.05
|
||||
m: 11
|
||||
neural_cfg:
|
||||
synthetic:
|
||||
K: 256
|
||||
R: 1
|
||||
tail: zipf
|
||||
zipf_s: 1.3
|
||||
tail_frac: 0.5
|
||||
tail_threshold: 0.001
|
||||
init: truth
|
||||
style_len: 3
|
||||
style_vocab: 5
|
||||
id_base: 2
|
||||
model:
|
||||
kind: rnn
|
||||
hidden: 192
|
||||
embed: 24
|
||||
epochs: 25
|
||||
lr: 0.002
|
||||
batch_size: 256
|
||||
n_eval: 12000
|
||||
dynamics:
|
||||
n: 200
|
||||
grounding:
|
||||
m: 11
|
||||
policy: proportional
|
||||
remint:
|
||||
enabled: false
|
||||
period: null
|
||||
H_gate: null
|
||||
generations: 22
|
||||
metrics:
|
||||
kl_floor: 1.0e-09
|
||||
support_eps: 1.0e-09
|
||||
- label:
|
||||
kind: mlp
|
||||
g: 0.0
|
||||
m: 0
|
||||
neural_cfg:
|
||||
synthetic:
|
||||
K: 256
|
||||
R: 1
|
||||
tail: zipf
|
||||
zipf_s: 1.3
|
||||
tail_frac: 0.5
|
||||
tail_threshold: 0.001
|
||||
init: truth
|
||||
style_len: 3
|
||||
style_vocab: 5
|
||||
id_base: 2
|
||||
model:
|
||||
kind: mlp
|
||||
hidden: 192
|
||||
embed: 24
|
||||
epochs: 25
|
||||
lr: 0.002
|
||||
batch_size: 256
|
||||
n_eval: 12000
|
||||
dynamics:
|
||||
n: 200
|
||||
grounding:
|
||||
m: 0
|
||||
policy: proportional
|
||||
remint:
|
||||
enabled: false
|
||||
period: null
|
||||
H_gate: null
|
||||
generations: 22
|
||||
metrics:
|
||||
kl_floor: 1.0e-09
|
||||
support_eps: 1.0e-09
|
||||
- label:
|
||||
kind: mlp
|
||||
g: 0.05
|
||||
m: 11
|
||||
neural_cfg:
|
||||
synthetic:
|
||||
K: 256
|
||||
R: 1
|
||||
tail: zipf
|
||||
zipf_s: 1.3
|
||||
tail_frac: 0.5
|
||||
tail_threshold: 0.001
|
||||
init: truth
|
||||
style_len: 3
|
||||
style_vocab: 5
|
||||
id_base: 2
|
||||
model:
|
||||
kind: mlp
|
||||
hidden: 192
|
||||
embed: 24
|
||||
epochs: 25
|
||||
lr: 0.002
|
||||
batch_size: 256
|
||||
n_eval: 12000
|
||||
dynamics:
|
||||
n: 200
|
||||
grounding:
|
||||
m: 11
|
||||
policy: proportional
|
||||
remint:
|
||||
enabled: false
|
||||
period: null
|
||||
H_gate: null
|
||||
generations: 22
|
||||
metrics:
|
||||
kl_floor: 1.0e-09
|
||||
support_eps: 1.0e-09
|
||||
Loading…
Add table
Add a link
Reference in a new issue