MachineSex

giorgio/MachineSex

Fork 0

Commit graph

Author	SHA1	Message	Date
Giorgio Gilestro	809e45a5e0	llm: first real-LLM prototype — recombining specialist LLMs (C2/C4) First step from toy models toward real language models, on one 16 GB GPU. New src/llm/ package: procedural task families + exact-match verifier (tasks.py), batched eval (evaluate.py), LoRA specialisation (specialise.py, manual answer-only SFT), weight-space merge via peft add_weighted_adapter (merge.py: soup = averaged deltas, ties = sign-reconciled union), runner (experiment.py, kind llm_merge). Base Qwen2.5-0.5B-Instruct (Apache-2.0); three disjoint hard families (lists/strings/arith); one LoRA specialist each (~90s total). Result (seed 1), reported honestly: - STRONG/robust: the merges are the ONLY models competent across ALL families -- worst-family ~0.25 vs <0.16 for every single specialist (the Fisher-Muller "generalist assembled from specialists" signature, in real LoRA weights). - MARGINAL: "exceeds every parent overall" is only marginal at this scale (soup 0.64 vs best specialist 0.63; ties 0.61 below it). - CAVEAT VISIBLE: averaging dilutes peaks (lists specialist 0.43 -> merge 0.26) -- Layer-1's "merge, don't average" (E4) appearing in real weights. The pipeline works end-to-end; the balance/retention half reproduces; the strict overall-exceeds and soup-vs-ties distinction need scale (bigger base, more/cleaner families, seeds, a dilution-resistant / offspring-selected merge) -- the HPC step. Env: Python 3.14 + transformers 5.13 works; note transformers-5.x apply_chat_template returns a dict. make env-llm / make llm; adapters under gitignored models/llm/, base in the HF cache. figures/plot_llm_merge.py, README, tests/test_llm.py (+3, 125 green). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-05 15:48:02 +01:00
Giorgio Gilestro	840b6b00b3	Layer 1.5: architecture-general neural existence proof Re-scopes Layer 2 into a cheaper, architecture-general neural collapse proof before the LLM rung. Realises the same Wright–Fisher abstractions in real trained generative models on a fully-synthetic sandbox with an exact oracle, reusing knowledge.metrics/truth/seeding and the output contract so neural curves overlay the Layer-1 analytic curves. - src/neural/: synthetic token-grammar sandbox (lossless identity + stochastic style), ExactOracle, HistogramModel bridge, generation loop, experiment runner - HARD GATE passed: histogram lineage reproduces Layer 1 exactly (neutral decay, exact H_eq, tracks run_lineage) — tests/test_neural_validation.py - torch models: autoregressive RNN + MLP (VAE implemented, not yet fidelity- passing); determinism seeding derived from the SeedSequence stream - N0 bridge (neural g*=0.047 ≈ Layer-1 0.048), N1 collapse-in-weights, N2 phase boundary, N5 architecture-generality (collapse + grounding-rescue in histogram + RNN + MLP). Manifests/configs committed; parquet gitignored, hashes tracked - additive backward-compatible save_artifacts extension; Makefile neural targets Finding: neural smoothing partially resists H-collapse, so forward-KL and tail survival are the sharp neural collapse metrics (H is smooth, per Layer 1). 92 tests green. Remaining (tasks/todo.md): N4 merge, N2 refine, N3/N6, VAE fidelity, MNIST tier, figures. LLM/LoRA rung and C3 deferred. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-04 21:02:49 +01:00
Giorgio Gilestro	a6eb9b7512	Layer 1 core: Wright-Fisher knowledge-transmission model with E1-E2 Scaffold plus the Layer 1 analytical core and the first two experiments. - knowledge/: truth, metrics, teachers (2.7.1 shared-switch construction), step, lineage, experiment, config, seeding (imported as `knowledge`). - Validation spine green: neutral decay (Pred 1), fixation (Pred 2), exact mutation-drift equilibrium (Pred 3), union coverage (Pred 5). 68 tests pass. - E1 reproduces tail-first collapse. E2 delivers the headline: a grounding phase boundary g* << 1, with stationary H tracking the exact H_eq closed form (g=0.005 -> 68% of truth diversity; g=0.05 -> 96%). - Reproducibility: uv venv from a hash-pinned uv.lock is the source of truth; every run writes results.parquet + resolved_config.yaml + manifest.json (lib versions, git commit, sha256). Figures and manifests tracked; the large regenerable parquet is gitignored. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>	2026-07-04 18:10:18 +02:00

Author

SHA1

Message

Date

Giorgio Gilestro

809e45a5e0

llm: first real-LLM prototype — recombining specialist LLMs (C2/C4)

First step from toy models toward real language models, on one 16 GB GPU.
New src/llm/ package: procedural task families + exact-match verifier
(tasks.py), batched eval (evaluate.py), LoRA specialisation (specialise.py,
manual answer-only SFT), weight-space merge via peft add_weighted_adapter
(merge.py: soup = averaged deltas, ties = sign-reconciled union), runner
(experiment.py, kind llm_merge). Base Qwen2.5-0.5B-Instruct (Apache-2.0);
three disjoint hard families (lists/strings/arith); one LoRA specialist each
(~90s total).

Result (seed 1), reported honestly:
- STRONG/robust: the merges are the ONLY models competent across ALL
  families -- worst-family ~0.25 vs <0.16 for every single specialist (the
  Fisher-Muller "generalist assembled from specialists" signature, in real
  LoRA weights).
- MARGINAL: "exceeds every parent overall" is only marginal at this scale
  (soup 0.64 vs best specialist 0.63; ties 0.61 below it).
- CAVEAT VISIBLE: averaging dilutes peaks (lists specialist 0.43 -> merge
  0.26) -- Layer-1's "merge, don't average" (E4) appearing in real weights.

The pipeline works end-to-end; the balance/retention half reproduces; the
strict overall-exceeds and soup-vs-ties distinction need scale (bigger base,
more/cleaner families, seeds, a dilution-resistant / offspring-selected
merge) -- the HPC step. Env: Python 3.14 + transformers 5.13 works;
note transformers-5.x apply_chat_template returns a dict. make env-llm /
make llm; adapters under gitignored models/llm/, base in the HF cache.
figures/plot_llm_merge.py, README, tests/test_llm.py (+3, 125 green).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

2026-07-05 15:48:02 +01:00

Giorgio Gilestro

840b6b00b3

Layer 1.5: architecture-general neural existence proof

Re-scopes Layer 2 into a cheaper, architecture-general neural collapse proof
before the LLM rung. Realises the same Wright–Fisher abstractions in real trained
generative models on a fully-synthetic sandbox with an exact oracle, reusing
knowledge.metrics/truth/seeding and the output contract so neural curves overlay
the Layer-1 analytic curves.

  - src/neural/: synthetic token-grammar sandbox (lossless identity + stochastic
    style), ExactOracle, HistogramModel bridge, generation loop, experiment runner
  - HARD GATE passed: histogram lineage reproduces Layer 1 exactly (neutral decay,
    exact H_eq, tracks run_lineage) — tests/test_neural_validation.py
  - torch models: autoregressive RNN + MLP (VAE implemented, not yet fidelity-
    passing); determinism seeding derived from the SeedSequence stream
  - N0 bridge (neural g*=0.047 ≈ Layer-1 0.048), N1 collapse-in-weights, N2 phase
    boundary, N5 architecture-generality (collapse + grounding-rescue in histogram
    + RNN + MLP). Manifests/configs committed; parquet gitignored, hashes tracked
  - additive backward-compatible save_artifacts extension; Makefile neural targets

Finding: neural smoothing partially resists H-collapse, so forward-KL and tail
survival are the sharp neural collapse metrics (H is smooth, per Layer 1).

92 tests green. Remaining (tasks/todo.md): N4 merge, N2 refine, N3/N6, VAE
fidelity, MNIST tier, figures. LLM/LoRA rung and C3 deferred.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-04 21:02:49 +01:00

Giorgio Gilestro

a6eb9b7512

Layer 1 core: Wright-Fisher knowledge-transmission model with E1-E2

Scaffold plus the Layer 1 analytical core and the first two experiments.

- knowledge/: truth, metrics, teachers (2.7.1 shared-switch construction),
  step, lineage, experiment, config, seeding (imported as `knowledge`).
- Validation spine green: neutral decay (Pred 1), fixation (Pred 2), exact
  mutation-drift equilibrium (Pred 3), union coverage (Pred 5). 68 tests pass.
- E1 reproduces tail-first collapse. E2 delivers the headline: a grounding
  phase boundary g* << 1, with stationary H tracking the exact H_eq closed
  form (g=0.005 -> 68% of truth diversity; g=0.05 -> 96%).
- Reproducibility: uv venv from a hash-pinned uv.lock is the source of truth;
  every run writes results.parquet + resolved_config.yaml + manifest.json
  (lib versions, git commit, sha256). Figures and manifests tracked; the
  large regenerable parquet is gitignored.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-07-04 18:10:18 +02:00

3 commits