MachineSex/results/E3

E3 — Grounding must overlap the content it protects

Claim tested: if your budget of real data is fixed, does it matter where you spend it? Is it enough to sprinkle real data uniformly, or must it target the knowledge you care about?

Setup (Layer 1, pure math). K = 1000 items divided into R = 10 disjoint regions (think subject areas). n = 200, 400 generations, 100 repeats. The same total real-data budget is spent two ways: uniform (spread evenly over all 10 regions) vs matched (concentrated on the "exercised" region we want to protect). One region is designated exercised (region 0 here) and has a rare tail we track.

Symbols

• region — a block of related items; a stand-in for a task family / subject area.
• matched vs uniform — real data aimed at the exercised region vs spread evenly, at equal total budget.
• tail items alive — how many rare items in a region still have any probability.

The two panels

1. Target region over time. Rare-item survival in the exercised region, generation by generation: blue = matched, red = uniform (bands = 95% CI). Matched holds the region's tail alive; uniform spreads too thin and lets it collapse.
2. Every region at steady state. Bar chart of stationary tail survival per region, matched (blue) vs uniform (red); the dotted line marks the exercised region. Matched wins big there — at the cost of the regions it deliberately ignores. Uniform is mediocre everywhere.

(Note: per-region diversity H is confounded by how much probability mass sits in a region, so this figure uses the honest, mass-independent metric — tail-item survival.)

Takeaway

Reality checks only protect what they actually cover. To keep a capability alive you must ground on that capability, not on data in general — grounding is local, not a global tonic. Key numbers: exercised-region tail survival ≈ 0.49 (matched) vs 0.07 (uniform) at equal budget. Falsifier (not triggered): if uniform had protected the region as well as matched, the targeting claim would die.