giorgio/MachineSex
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

neural: real-MNIST external-validity tier (collapse + grounding)

Browse source 
Confirms model collapse and its arrest by grounding on REAL images, not
just the synthetic sandbox. A conv VAE (the canonical generative-collapse
model) is retrained each generation on its own generated digits, with a
fraction g of fresh real MNIST mixed in. Modes = digit class x stroke-
thickness bin (K=30, Zipf, ~18 tail modes); the oracle is a frozen CNN +
deterministic thickness at 98.5% mode accuracy (30x30 confusion matrix
recorded in the manifest as the measurement-noise floor).

Result (4 reps): dry (g=0) collapses to a single mode -- forward-KL
0.5->18, support 30->1, tail 1.0->0.06, H->0 -- while 10% grounding holds
all 30 modes (KL~0.6, full tail, H~0.9). Signs, not magnitudes (blueprint
3.5); the exact synthetic oracle stays the quantitative anchor. The VAE
needs ~10% grounding vs the synthetic histogram's ~5%, consistent with the
grounding finding that trained nets need more than the exact operator.

Plugs into the existing data-agnostic contract (metrics/grounding/output
reused verbatim): mnist_data (thickness bins, class x thickness bijection,
MnistSampler), mnist_oracle (ClassifierOracle + confusion matrix),
mnist_vae (ConvVAEGenerator), mnist_loop (run_mnist_lineage), kind=
mnist_lineage dispatch, MnistCfg/OracleCfg. Figures: plot_mnist (parquet-
only) + mnist_montage (eyeball diagnostic showing digits degenerate to one
blurry mode). make mnist / make env-mnist, kept out of the make neural
loop. 99 tests green (+5 torchvision-gated).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Giorgio Gilestro 2026-07-05 09:19:36 +01:00
parent 3b9f4f7893
commit 79bbc45f41
21 changed files with 2200 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

984
results/mnist_collapse/manifest.json Normal file
View file

@ -0,0 +1,984 @@
{
"experiment": "mnist_collapse",
"master_seed": 20260705,
"git_commit": "3b9f4f789358a442d510bf2d609aab0986834ac5",
"python": "3.14.5",
"libraries": {
"numpy": "2.5.0",
"scipy": "1.18.0",
"pandas": "3.0.3",
"pyarrow": "24.0.0",
"torch": "2.12.1",
"torchvision": "0.27.1"
},
"rows": 128,
"results_sha256": "d808003e0e01247923996c94eff18c430308448e23bb5d102aa860701145131e",
"layer": "1.5",
"tier": "mnist",
"model_kind": "convvae",
"oracle_ckpt_sha256": "7ff2cc619861daacf8168277c83cde29483554c92d732a24db9fdaa446fa9ff8",
"oracle_mode_accuracy": 0.9847,
"oracle_class_accuracy": 0.9847,
"confusion_matrix": [
[
365,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
0,
0,
0,
1,
0,
0,
1,
0,
1,
0,
0,
0,
0,
0
],
[
0,
302,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
2,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
0,
0,
307,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
0,
0,
0,
366,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
0,
0,
0,
0,
373,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
0,
0,
0,
0,
0,
390,
0,
2,
1,
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
1,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
1,
0,
0,
0,
0,
1,
335,
0,
0,
1,
0,
0,
0,
1,
0,
0,
0,
0,
0,
0,
0,
1,
0,
1,
1,
0,
0,
0,
0,
0
],
[
1,
0,
0,
0,
0,
0,
0,
308,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
0,
0,
0,
0,
0,
0
],
[
0,
0,
1,
0,
0,
1,
0,
0,
376,
0,
0,
2,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
0,
0,
0,
0,
0,
0,
0,
0,
0,
327,
0,
0,
0,
0,
0,
5,
0,
0,
0,
0,
0,
1,
0,
0,
0,
0,
0,
0,
0,
0
],
[
1,
0,
0,
0,
0,
0,
1,
0,
0,
0,
303,
0,
0,
0,
0,
0,
2,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
368,
0,
0,
0,
0,
0,
1,
0,
0,
0,
0,
0,
0,
0,
0,
1,
0,
0,
0
],
[
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
326,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
4,
0,
0
],
[
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
324,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
0
],
[
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
324,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
3
],
[
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
269,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
0,
0,
0,
0,
0,
0,
299,
0,
0,
1,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
0,
2,
0,
0,
0,
0,
0,
0,
0,
0,
1,
0,
0,
0,
0,
0,
0,
318,
0,
1,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
3,
0,
0,
1,
0,
1,
0,
0,
0,
0,
0,
0,
2,
1,
0,
3,
0,
0,
247,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
3,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
0,
0,
2,
0,
0,
315,
0,
0,
0,
0,
1,
0,
0,
0,
0,
0
],
[
0,
0,
1,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
1,
0,
0,
2,
0,
0,
374,
0,
0,
0,
0,
0,
0,
0,
0,
0
],
[
0,
0,
0,
1,
0,
0,
4,
0,
0,
2,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
327,
0,
0,
1,
0,
0,
0,
0,
0
],
[
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
335,
0,
1,
0,
0,
0,
2,
0
],
[
0,
1,
0,
0,
0,
0,
1,
2,
1,
0,
1,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
349,
0,
0,
0,
0,
0,
0
],
[
1,
0,
0,
0,
0,
0,
4,
0,
0,
1,
1,
0,
1,
0,
0,
4,
1,
0,
1,
2,
0,
0,
0,
0,
280,
0,
0,
0,
0,
0
],
[
2,
0,
0,
0,
0,
0,
0,
0,
0,
0,
2,
1,
0,
0,
1,
0,
2,
2,
0,
0,
0,
0,
0,
0,
0,
347,
0,
0,
0,
5
],
[
0,
0,
1,
0,
0,
1,
0,
0,
1,
0,
0,
5,
0,
0,
0,
0,
0,
1,
0,
0,
2,
0,
0,
2,
0,
0,
297,
0,
0,
6
],
[
0,
0,
0,
0,
2,
0,
0,
0,
0,
0,
0,
0,
1,
1,
0,
1,
0,
0,
0,
0,
0,
1,
0,
0,
1,
0,
0,
268,
0,
0
],
[
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
2,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
365,
0
],
[
0,
1,
1,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
2,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
0,
363
]
]
}