Layer 1.5: architecture-general neural existence proof

Re-scopes Layer 2 into a cheaper, architecture-general neural collapse proof
before the LLM rung. Realises the same Wright–Fisher abstractions in real trained
generative models on a fully-synthetic sandbox with an exact oracle, reusing
knowledge.metrics/truth/seeding and the output contract so neural curves overlay
the Layer-1 analytic curves.

  - src/neural/: synthetic token-grammar sandbox (lossless identity + stochastic
    style), ExactOracle, HistogramModel bridge, generation loop, experiment runner
  - HARD GATE passed: histogram lineage reproduces Layer 1 exactly (neutral decay,
    exact H_eq, tracks run_lineage) — tests/test_neural_validation.py
  - torch models: autoregressive RNN + MLP (VAE implemented, not yet fidelity-
    passing); determinism seeding derived from the SeedSequence stream
  - N0 bridge (neural g*=0.047 ≈ Layer-1 0.048), N1 collapse-in-weights, N2 phase
    boundary, N5 architecture-generality (collapse + grounding-rescue in histogram
    + RNN + MLP). Manifests/configs committed; parquet gitignored, hashes tracked
  - additive backward-compatible save_artifacts extension; Makefile neural targets

Finding: neural smoothing partially resists H-collapse, so forward-KL and tail
survival are the sharp neural collapse metrics (H is smooth, per Layer 1).

92 tests green. Remaining (tasks/todo.md): N4 merge, N2 refine, N3/N6, VAE
fidelity, MNIST tier, figures. LLM/LoRA rung and C3 deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-07-04 21:02:49 +01:00
parent 1721d047fa
commit 840b6b00b3
35 changed files with 3679 additions and 23 deletions

16
results/N0/manifest.json Normal file
View file

@ -0,0 +1,16 @@
{
"experiment": "N0_bridge_histogram",
"master_seed": 20260704,
"git_commit": null,
"python": "3.14.5",
"libraries": {
"numpy": "2.5.0",
"scipy": "1.18.0",
"pandas": "3.0.3",
"pyarrow": "24.0.0"
},
"rows": 96480,
"results_sha256": "445fd5165fc69edbca87f79c0cf669b5879025637f4d36ddc769e940fb02f114",
"layer": "1.5",
"model_kind": "histogram"
}

View file

@ -0,0 +1,288 @@
experiment: N0_bridge_histogram
seed: 20260704
n_replicates: 60
source_config:
experiment: N0_bridge_histogram
kind: gen_lineage
seed: 20260704
n_replicates: 60
generations: 200
synthetic:
K: 200
R: 1
tail: zipf
zipf_s: 1.1
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 2
style_vocab: 4
id_base: 2
model:
kind: histogram
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
sweep:
- param: g
values:
- 0.0
- 0.005
- 0.01
- 0.02
- 0.05
- 0.1
- 0.2
- 0.4
output:
dir: results/N0
grid:
- label:
g: 0.0
m: 0
neural_cfg:
synthetic:
K: 200
R: 1
tail: zipf
zipf_s: 1.1
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 2
style_vocab: 4
id_base: 2
model:
kind: histogram
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 200
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.005
m: 1
neural_cfg:
synthetic:
K: 200
R: 1
tail: zipf
zipf_s: 1.1
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 2
style_vocab: 4
id_base: 2
model:
kind: histogram
dynamics:
n: 200
grounding:
m: 1
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 200
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.01
m: 2
neural_cfg:
synthetic:
K: 200
R: 1
tail: zipf
zipf_s: 1.1
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 2
style_vocab: 4
id_base: 2
model:
kind: histogram
dynamics:
n: 200
grounding:
m: 2
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 200
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.02
m: 4
neural_cfg:
synthetic:
K: 200
R: 1
tail: zipf
zipf_s: 1.1
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 2
style_vocab: 4
id_base: 2
model:
kind: histogram
dynamics:
n: 200
grounding:
m: 4
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 200
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.05
m: 11
neural_cfg:
synthetic:
K: 200
R: 1
tail: zipf
zipf_s: 1.1
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 2
style_vocab: 4
id_base: 2
model:
kind: histogram
dynamics:
n: 200
grounding:
m: 11
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 200
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.1
m: 22
neural_cfg:
synthetic:
K: 200
R: 1
tail: zipf
zipf_s: 1.1
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 2
style_vocab: 4
id_base: 2
model:
kind: histogram
dynamics:
n: 200
grounding:
m: 22
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 200
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.2
m: 50
neural_cfg:
synthetic:
K: 200
R: 1
tail: zipf
zipf_s: 1.1
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 2
style_vocab: 4
id_base: 2
model:
kind: histogram
dynamics:
n: 200
grounding:
m: 50
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 200
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.4
m: 133
neural_cfg:
synthetic:
K: 200
R: 1
tail: zipf
zipf_s: 1.1
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 2
style_vocab: 4
id_base: 2
model:
kind: histogram
dynamics:
n: 200
grounding:
m: 133
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 200
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09

17
results/N1/manifest.json Normal file
View file

@ -0,0 +1,17 @@
{
"experiment": "N1_collapse_in_weights",
"master_seed": 20260704,
"git_commit": null,
"python": "3.14.5",
"libraries": {
"numpy": "2.5.0",
"scipy": "1.18.0",
"pandas": "3.0.3",
"pyarrow": "24.0.0",
"torch": "2.12.1"
},
"rows": 520,
"results_sha256": "27438db70af3240524855e59c4a14d3fb63bf6fca2d2002cdcb1ac354559909b",
"layer": "1.5",
"model_kind": "rnn"
}

View file

@ -0,0 +1,194 @@
experiment: N1_collapse_in_weights
seed: 20260704
n_replicates: 5
source_config:
experiment: N1_collapse_in_weights
kind: gen_lineage
seed: 20260704
n_replicates: 5
generations: 25
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
sweep:
- param: g
values:
- 0.0
- 0.02
- 0.05
- 0.1
output:
dir: results/N1
grid:
- label:
g: 0.0
m: 0
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 25
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.02
m: 4
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 4
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 25
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.05
m: 11
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 11
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 25
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.1
m: 22
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 22
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 25
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09

17
results/N2/manifest.json Normal file
View file

@ -0,0 +1,17 @@
{
"experiment": "N2_grounding_phase_boundary_neural",
"master_seed": 20260704,
"git_commit": null,
"python": "3.14.5",
"libraries": {
"numpy": "2.5.0",
"scipy": "1.18.0",
"pandas": "3.0.3",
"pyarrow": "24.0.0",
"torch": "2.12.1"
},
"rows": 1085,
"results_sha256": "93590677221955189814ccf044b9dc8aee36b026e350c9dea7e8d23b137e514b",
"layer": "1.5",
"model_kind": "rnn"
}

View file

@ -0,0 +1,305 @@
experiment: N2_grounding_phase_boundary_neural
seed: 20260704
n_replicates: 5
source_config:
experiment: N2_grounding_phase_boundary_neural
kind: gen_lineage
seed: 20260704
n_replicates: 5
generations: 30
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
sweep:
- param: g
values:
- 0.0
- 0.005
- 0.01
- 0.02
- 0.05
- 0.1
- 0.2
output:
dir: results/N2
grid:
- label:
g: 0.0
m: 0
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 30
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.005
m: 1
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 1
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 30
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.01
m: 2
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 2
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 30
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.02
m: 4
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 4
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 30
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.05
m: 11
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 11
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 30
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.1
m: 22
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 22
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 30
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
g: 0.2
m: 50
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 128
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 50
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 30
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09

17
results/N5/manifest.json Normal file
View file

@ -0,0 +1,17 @@
{
"experiment": "N5_architecture_generality",
"master_seed": 20260704,
"git_commit": null,
"python": "3.14.5",
"libraries": {
"numpy": "2.5.0",
"scipy": "1.18.0",
"pandas": "3.0.3",
"pyarrow": "24.0.0",
"torch": "2.12.1"
},
"rows": 690,
"results_sha256": "4e8d7931e791493aa35ea7b114d81b90a3a58eabd92d8b1dbd87cf0a9e200ba1",
"layer": "1.5",
"model_kind": "rnn"
}

View file

@ -0,0 +1,275 @@
experiment: N5_architecture_generality
seed: 20260704
n_replicates: 5
source_config:
experiment: N5_architecture_generality
kind: gen_lineage
seed: 20260704
n_replicates: 5
generations: 22
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 192
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
sweep:
- param: model.kind
values:
- histogram
- rnn
- mlp
- param: g
values:
- 0.0
- 0.05
output:
dir: results/N5
grid:
- label:
kind: histogram
g: 0.0
m: 0
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: histogram
hidden: 192
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 22
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
kind: histogram
g: 0.05
m: 11
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: histogram
hidden: 192
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 11
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 22
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
kind: rnn
g: 0.0
m: 0
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 192
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 22
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
kind: rnn
g: 0.05
m: 11
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: rnn
hidden: 192
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 11
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 22
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
kind: mlp
g: 0.0
m: 0
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: mlp
hidden: 192
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 0
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 22
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09
- label:
kind: mlp
g: 0.05
m: 11
neural_cfg:
synthetic:
K: 256
R: 1
tail: zipf
zipf_s: 1.3
tail_frac: 0.5
tail_threshold: 0.001
init: truth
style_len: 3
style_vocab: 5
id_base: 2
model:
kind: mlp
hidden: 192
embed: 24
epochs: 25
lr: 0.002
batch_size: 256
n_eval: 12000
dynamics:
n: 200
grounding:
m: 11
policy: proportional
remint:
enabled: false
period: null
H_gate: null
generations: 22
metrics:
kl_floor: 1.0e-09
support_eps: 1.0e-09