Giorgio Gilestro
840b6b00b3
Re-scopes Layer 2 into a cheaper, architecture-general neural collapse proof
before the LLM rung. Realises the same Wright–Fisher abstractions in real trained
generative models on a fully-synthetic sandbox with an exact oracle, reusing
knowledge.metrics/truth/seeding and the output contract so neural curves overlay
the Layer-1 analytic curves.
- src/neural/: synthetic token-grammar sandbox (lossless identity + stochastic
style), ExactOracle, HistogramModel bridge, generation loop, experiment runner
- HARD GATE passed: histogram lineage reproduces Layer 1 exactly (neutral decay,
exact H_eq, tracks run_lineage) — tests/test_neural_validation.py
- torch models: autoregressive RNN + MLP (VAE implemented, not yet fidelity-
passing); determinism seeding derived from the SeedSequence stream
- N0 bridge (neural g*=0.047 ≈ Layer-1 0.048), N1 collapse-in-weights, N2 phase
boundary, N5 architecture-generality (collapse + grounding-rescue in histogram
+ RNN + MLP). Manifests/configs committed; parquet gitignored, hashes tracked
- additive backward-compatible save_artifacts extension; Makefile neural targets
Finding: neural smoothing partially resists H-collapse, so forward-KL and tail
survival are the sharp neural collapse metrics (H is smooth, per Layer 1).
92 tests green. Remaining (tasks/todo.md): N4 merge, N2 refine, N3/N6, VAE
fidelity, MNIST tier, figures. LLM/LoRA rung and C3 deferred.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
82 lines
3.1 KiB
Python
82 lines
3.1 KiB
Python
"""The oracle — the neural analogue of Layer 1's "reality's no".
|
|
|
|
An oracle maps an observation to the mode it belongs to. For the fully-synthetic sandbox
|
|
the oracle is **exact** (it decodes the lossless identity segment), so the measured mode
|
|
distribution ``p_hat`` carries zero measurement noise — this is what lets a trained model's
|
|
collapse be read directly against the known ``p*``. (The MNIST tier will add a
|
|
``ClassifierOracle`` wrapping a frozen network plus its confusion matrix; that arrives with
|
|
Stage C and is confirmation-only.)
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from typing import Protocol, runtime_checkable
|
|
|
|
import numpy as np
|
|
|
|
from .config import SyntheticCfg
|
|
|
|
|
|
@runtime_checkable
|
|
class Oracle(Protocol):
|
|
"""Adjudicates which mode an observation belongs to."""
|
|
|
|
def classify(self, X: np.ndarray) -> np.ndarray:
|
|
"""Return the length-``n`` mode index for each row of ``X``."""
|
|
...
|
|
|
|
|
|
class ExactOracle:
|
|
"""Zero-error oracle for the synthetic sandbox: decodes the identity segment.
|
|
|
|
Args:
|
|
cfg (SyntheticCfg): The synthetic configuration whose grammar produced ``X``.
|
|
"""
|
|
|
|
def __init__(self, cfg: SyntheticCfg) -> None:
|
|
self.cfg = cfg
|
|
self.id_len = cfg.id_len
|
|
self.base = cfg.id_base
|
|
self.K = cfg.K
|
|
# Positional weights for base-`base` decoding, most-significant digit first.
|
|
self._weights = self.base ** np.arange(self.id_len - 1, -1, -1, dtype=np.int64)
|
|
|
|
def classify(self, X: np.ndarray) -> np.ndarray:
|
|
"""Decode mode indices from the identity segment of each observation.
|
|
|
|
Args:
|
|
X (np.ndarray): Token sequences of shape ``(n, seq_len)``.
|
|
|
|
Returns:
|
|
np.ndarray: Length-``n`` decoded indices. Grammar-valid data always decodes into
|
|
``[0, K)``; a *neural* model may emit an invalid codeword that decodes to
|
|
``>= K`` — such samples are dropped by :func:`measure_distribution` rather than
|
|
being clipped onto a real mode (which would bias ``p_hat``).
|
|
"""
|
|
X = np.asarray(X, dtype=np.int64)
|
|
ident = X[:, : self.id_len]
|
|
return ident @ self._weights
|
|
|
|
|
|
def measure_distribution(X: np.ndarray, oracle: Oracle, K: int) -> np.ndarray:
|
|
"""Measure the empirical mode distribution ``p_hat`` of a sample.
|
|
|
|
This is the neural readout of ``p_t``: classify every observation and normalise the
|
|
mode histogram. Modes absent from ``X`` receive zero mass (support shrinks exactly as
|
|
in Layer 1). Invalid codewords (decoded index ``>= K``, only producible by a neural
|
|
model) are dropped, so ``p_hat`` is renormalised over grammar-valid samples.
|
|
|
|
Args:
|
|
X (np.ndarray): Token sequences of shape ``(n, seq_len)``.
|
|
oracle (Oracle): The mode adjudicator.
|
|
K (int): Number of modes.
|
|
|
|
Returns:
|
|
np.ndarray: Length-``K`` probability vector summing to 1.
|
|
"""
|
|
modes = oracle.classify(X)
|
|
counts = np.bincount(modes, minlength=K)[:K].astype(float)
|
|
total = counts.sum()
|
|
if total <= 0:
|
|
raise ValueError("measure_distribution received an empty sample")
|
|
return counts / total
|