Skip to content

Serve OP1's adapter draws from DirectRNGStreams by default - #591

Draft
MaxGhenis wants to merge 15 commits into
masterfrom
openpop-op1-rng-inject-20261026
Draft

MaxGhenis wants to merge 15 commits into
masterfrom
openpop-op1-rng-inject-20261026

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Draft: waiting on independent review and CI. Follow-up to #566. Head 305b171b. OP1's adapters now take their person and tagged draws from DirectRNGStreams by default instead of the sealed ProjectionRNGRegistry. The projection is the same draw for draw; the direct class is cheaper. rng_streams="registry" keeps OP1's behavior from #566.

No sealed code changes. Against master 172f690f the branch changes nine files: open_population/harness.py, open_population/gates.py, tests/open_population/test_streams.py, the new tests/open_population/test_stream_injection.py, op1_report.md (new §6), op1_run_size.py, the new op1_run_size_injection.json, and the two tier files the recount sets (tests/tier_counts.json, tests/README-tiers.md). Engine, baselines, cola_track_a, track_b, gates.yaml, runs, scripts and tests/estimates are unchanged.

How it works

  • The engine loop builds its registry itself (engine/loop.py:216–217), so the substitution lives in OP1's wrapper, _AdapterStreams (open_population/harness.py).
  • At the first period it builds one DirectRNGStreams with the registry's draw index and horizon. Every slot of a period then receives one dataclasses.replace(context, rng_registry=<direct>); every other field is the loop's own object.
  • All eight slots are bound. The positional generator each slot receives is the loop's, unchanged.
  • _project, run_production and run_gate take rng_streams: Literal["registry", "direct"] = "direct". Any other value is refused before anything runs. OpenPopulationResult records the mode and the class that served the slots.
  • Under the default, the sealed observer's refusal (below) is re-raised as IdOrStreamCollision naming rng_streams="registry", chained from the original. Only that exact plain ValueError is re-raised; every other error passes through unchanged.

Invariants

  1. OP1's projection is identical whichever stream class serves the adapters' draws: every slice, every declared flow and the accounting history.
  2. At every valid stream address the direct stream equals the registry's (seed, bit_generator.state and draws); at every bad address both refuse with the same exception type and message.
  3. Within a run there is one direct instance and one context per period, and each slot's generator equals the loop's module generator for its period and module.
  4. The two classes share the registry's public surface. Every sealed reader of a context's streams that the census finds and that runs on invented inputs gives exactly the same output from either class, and the one sealed type check refuses the direct class rather than diverging. The static scan, a heuristic with stated limits, finds no other check that tells the classes apart in the code it reads.

The guarantee is behavioural: invariant 2 at every address, the reader runs, and invariant 1 end to end. The scan is a heuristic beside them; §6 of the report states its limits as four classes (values made at run time, introspection of run-time values, files read as data, and code it does not read).

Tests

  • test_stream_injection.py (new, 432 tests, unit tier):
    • end-to-end hash identity in both modes over sizes 1, 3 and 6 by draws 0, 3 and 17 at both 2027 and 2100 (18 configurations), plus 10 Hypothesis runs;
    • each sealed reader of a context's streams, run with both classes. A pinned census finds the 23 sealed places that use rng_registry, person_generator or composition_rngs_from_registry, each with the run that checks it or the reason none does. The ten runnable readers (mortality, SES mortality, earnings with three invented models, both claiming steps, di_aware_claiming, ASFR births, DI uniforms, the fertility capture and PeriodContext.person_generator) and composition_rngs_from_registry run on invented inputs with both classes, 12 Hypothesis examples each, and their outputs are compared exactly. Another test checks each reader's output depends on the draw index;
    • introspected public-surface parity, and no sealed attribute patched;
    • an AST scan of src/populace_dynamics/, a heuristic that tracks both classes. Specific checks name each kind of type check and each read of a place the catch-all trusts (__all__, docstrings, annotations, namespaces, source text, dynamic code, member introspection). Under them a catch-all reports every other use of either class, of an instance, of a module object that can reach one, and of their names as text once literal expressions are folded by a bounded folder;
    • 125 invented spellings, 123 of them also run on both classes to show each answers differently; 20 negatives; re-export tests; tests that messages carry only a path, a line and a kind;
    • accessor decorators trusted only on a property defined above them in the same class body (seven cases); literal code parsed five levels deep and reported as dynamic code past that (depths one to seven); seven working checks beside chains 3000 links long;
    • folder tests: 19 expressions past its limits, refused while holding under 1 MiB; nine limits checked at their edges; a source literal of any length kept as written; a 1 MiB budget per module, past which the scan stops with an error naming the module; one call to each of the 26 methods and 10 builtins it runs; and 300 Hypothesis expressions checked against Python;
    • a differential test and a Hypothesis property that pruning inert literal tables (done for speed) changes no scan result;
    • a binding test that compares every PeriodContext field except rng_registry with the loop's own context.
  • test_streams.py: the generator comparisons also check bit_generator.state, so the existing exhaustive grid and Hypothesis tests check state too.
  • The scan and the census skip, by path and before anything opens them, the two modules the project's data-access record keeps out of automated reads and the depletion-cut modules (bridge/depletion_cut*.py), so their input does not depend on who runs them. Tests pin the pattern and its three matches, and a fresh-interpreter test checks that no production or gate run imports any of them.

The one sealed type check

The scan finds one: mortality_observer.py:381 refuses any context whose registry is not exactly a ProjectionRNGRegistry. It raises ValueError before drawing, so it fails loudly. An OP1 run that composes this observer must pass rng_streams="registry"; run_production's and run_gate's docstrings say so, and no entry point composes it. The scan's hits must equal this one pinned entry exactly, so a new, moved or removed check fails the test.

Measurement

On the host, the three stream modes give the same projection hash at 25, 100 and 400 people, the same hashes as #566. At 400 people the default takes 3.13 s, against 26.50 s with every draw on the registry and 17.60 s for OP1 as merged. Peak RSS is 192 to 203 MB in every mode.

Plan

  • Build (Subfleet build lane).
  • Host run-size measurement in three stream modes.
  • Coordinator rulings applied (8634858c): keep the observer's loud refusal with a clearer OP1 error; full 2100 grid; depletion-cut exclusion.
  • Host rerun at 8634858c: tests/open_population 346 passed and 1 strict xfail; the seven-path command 665 passed, 2 skipped and 1 strict xfail. Six mutants of the new code are killed.
  • Review inject-r1 at 8634858c: REVISE. One major (static type checks the old scan missed) and three minor (a field mutant the binding test missed; an RSS figure; evidence wording).
  • Fixes in 33e383b4: the two-layer scan, the field-by-field binding test and the report corrections. Host rerun: tests/open_population 437 passed and 1 strict xfail; the seven-path command 756 passed, 2 skipped and 1 strict xfail; the real-tree scan still finds exactly the one pinned check. Thirteen more mutants are killed.
  • Review inject-r2 at 33e383b4: REVISE. One major (13 more static checks escaped the scan outside its stated limits), one minor (no bytes regression) and a nit.
  • Fixes in 5a4fc14c: the scan tracks both classes and checks reads of the places it trusts; a bounded literal folder; bytes spellings. Following the coordinator's suggestion, the guarantee is now behavioural: the reader census and runs above, with the scan presented as a heuristic. Host rerun: tests/open_population 614 passed and 1 strict xfail; the seven-path command 933 passed, 2 skipped and 1 strict xfail; the real-tree scan still finds exactly the one pinned check, and all 38 of r2's checks give hits. 32 scan and folder mutants and three stream mutants are killed.
  • Review inject-r3 at 5a4fc14c: APPROVE, with one minor finding (the folder's limits were stated without their scope) and two nits (accessor decorators trusted by name; nested literal code parsed five levels deep, unstated). It also noted that dropping partition from the folder failed no test and that long + chains raised RecursionError in the scan.
  • Fixes in 56fa57ca: the folder's scope stated, with a 1 MiB per-module budget; accessors trusted only on a property above them; a dynamic-code hit past five levels; a call to each folder method and builtin; stack-based walks; four spellings a mutant sweep found unpinned.
  • Tier-count slot: 619282e7 merges master 172f690f without conflicts (its tree is the one git merge-tree writes), and ec888be7 recounts. tests/open_population collects 457 more node ids than master (432 new, 25 in test_streams.py), all unit, so unit moves from 18,789 to 19,246 and the total to 25,816.
  • Host rerun at ec888be7 (305b171b changes only the report): tests/open_population 694 passed and 1 strict xfail; the seven-path command 1,022 passed, 2 skipped and 1 strict xfail (master's Track B change adds nine); collection 695, all unit; the real-tree scan still finds exactly the one pinned check, and every module's folded values weigh under a sixty-fourth of the budget; all 38 of r2's checks give hits; the scan at 5a4fc14c and now agree on 155 of 163 invented sources, the eight differences being the intended accessor and depth cases, and on the real tree; 24 of 25 mutants of the r3 fixes are killed (the survivor makes a path that starts None., which no module has), and all three stream mutants.
  • Delta review inject-r4 naming 305b171b (GPT-6.1 Sol), covering the r3 fixes, the merge and the recount.
  • Green CI at 305b171b, the tier-policy test included.

🤖 Generated with Claude Code

MaxGhenis and others added 7 commits October 25, 2026 23:03
The sealed loop builds one ProjectionRNGRegistry per run and puts it on
each period's context; its cost per person draw grows with the horizon.
OP1's harness wrapper now gives every slot (the exit and birth adapters
included) that context with one DirectRNGStreams of the same draw and
horizon in its place, built at the first period, after checking that the
loop's registry is the same object every period. The loop's positional
module generators are untouched, and no sealed file changes.

_project, run_production and run_gate take rng_streams ("direct" by
default, or "registry" for differential checks); OpenPopulationResult
records the mode and the class that served the draws.

tests/open_population/test_stream_injection.py pins the coordinator's
conditions: introspected interface parity; an alias-aware AST scan of
src/populace_dynamics/ outside open_population/ for type checks on the
registry, which skips the two closed modules by path before reading them
and reports only path, line and node kind; a fresh-process check that a
projection imports neither closed module; and projection-hash identity
between the two classes over three sizes, three draws and two horizons
(2027 and 2100), with a Hypothesis extension.

The scan finds one existing sealed check, mortality_observer.py:381
(type(registry) is not ProjectionRNGRegistry, present since e4e512e). It
raises rather than diverging; the test pins it and two behaviour tests show
the refusal, so an OP1 run composing that observer passes
rng_streams="registry".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Condition 3 of the injection ruling: OP1's slots now draw their person and
tagged streams from DirectRNGStreams, so the stream differential covers
every period 0 to n at horizons 0, 1, 7 and 75, all eight modules, draws
0, 1 and 9, and every method. Both generator comparisons now also check
bit_generator.state before and after drawing. A bad-address grid on the
same horizons compares refusal type and message, and a Hypothesis test
draws addresses, valid or not, across them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
op1_run_size.py now runs each invented size in three modes: registry
(adapters and emigration on ProjectionRNGRegistry), direct_emigration
(OP1 before the injection) and direct (the default). Each record names
the stream classes the adapters and the exit adapter were observed to
receive, the SHA-256 of streams.py, harness.py and exits.py, and the
process CPU time, and profiles the adapters' direct draws with disjoint
setup wrappers. The script stops if a mode runs another class or if the
modes' projection hashes differ.

The committed op1_run_size.json is unchanged; the host rerun replaces it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Section 6 of op1_report.md records the mechanism, the four invariants,
each of the coordinator's five conditions with its tests and evidence,
the one sealed type check the scan found (mortality_observer.py:381) and
its disposition, the commands and results of this build's runs, and a
restricted-files statement. Section 2 notes that it measured OP1 as
merged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Self-review found that a module binding overwritten on every fixed-point
pass could grow without end on a cycle such as "a = b.c; b = a", which
would hang the scan on such a module. Module bindings now keep their
first value, so each pass only adds names; a negative case pins it. A
comparison now counts when any of its operators, not only the first, is
an identity, equality or membership test; a chained spelling pins it.
The report's counts follow: 442 passed and 1 strict xfail.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
d49d081's message said the report's counts followed, but its report
edit had failed, so only the test change landed. This commit adds them:
32 spellings and nine negatives, a binding cycle among them; chained
comparisons; and 442 passed and 1 strict xfail, with all 76 of the new
module's tests rerun after that change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The main session reran op1_run_size.py on the host at 441b9f7 with
sizes 25, 100 and 400. The three stream modes give the same projection
hash at each size, matching section 2's hashes, and the direct default
takes 3.13 s at 400 people against 26.50 s on the registry.

The run goes to op1_run_size_injection.json so that op1_run_size.json
stays the evidence for section 2's two-mode run. Section 6 reports the
run and the host's test rerun.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
social-security-model Ready Ready Preview Oct 26, 2026 9:39pm UTC

Request Review

MaxGhenis and others added 2 commits October 26, 2026 02:15
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Keep the observer's loud refusal of the direct class, and re-raise it
  under the default as IdOrStreamCollision naming rng_streams="registry",
  chained from the sealed error. Only that exact plain ValueError is
  re-raised; run_production's and run_gate's docstrings say composing
  observe_mortality needs the registry mode.
- Compare the scan's hits with the pinned set exactly, as sorted lists.
- Skip the depletion-cut modules (bridge/depletion_cut*.py) by path, like
  the two closed modules. Tests pin the pattern and its three matches, check
  the tree's excluded set by directory listing, and check after a fresh
  projection that no depletion_cut module was imported.
- Run the 2100 identity grid over the full product of sizes and draws.
- Report the rulings, the host rerun (346 passed, 1 xfail) and six mutants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ct-r1)

- Scan: a catch-all now reports every other mention of the class, its name
  or its module's name outside imports, docstrings, __all__, annotations
  and construction. The specific checks follow checking-function aliases,
  class info in containers, unions, + joins, subscripts and named
  expressions, loop and attribute bindings, name and module comparisons,
  match values, quoted dispatch hints and private-attribute reads.
- The text filter is gone; inert literal containers are pruned after
  parsing, with a differential test and a Hypothesis property showing that
  pruning changes no result. The real tree still gives exactly the pinned
  hit.
- 58 invented spellings, all twelve of the review's among them; 56 run and
  answer differently for the two classes. 16 negatives.
- The binding test compares every PeriodContext field with the loop's own
  context, so a changed draw index, metadata or ordinals fails it.
- Section 6: the review's findings and fixes, the scan's coverage and
  limits, RSS within 1.4 MB, base 9cced8f and three evidence files, the
  generator helpers and entrant years stated exactly, refreshed line
  references and the host results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review inject-r2 found static checks the scan missed outside its stated
limits. The scan now tracks DirectRNGStreams as well as the registry,
treats each read of a place the catch-all trusts as a specific check, and
reads text built from literals through a bounded folder. All 38 of r2's
checks give hits, and the real-tree pin is unchanged.

Following the coordinator's suggestion, the guarantee is now behavioural.
A pinned census lists the 23 sealed places that reach a context's
streams. The ten runnable readers and the composition helper run on
invented inputs with both classes, under Hypothesis, and their outputs
are compared exactly. Section 6 of the report presents the scan as a
heuristic with four stated classes of limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 4 commits October 26, 2026 14:13
…rs, flag deep literal code

- The folder's limits now apply to each value it computes; a source
  literal is kept as written. A module's folded values share a 1 MiB
  budget, past which the scan stops with an error naming the module.
  Edge, literal and budget tests pin each limit.
- A property's setter, getter or deleter is trusted only on a property
  defined above it in the same class body; register stays trusted by
  spelling, and the report says so.
- Literal code is parsed five levels deep; a call whose code would be a
  sixth level is reported as dynamic code.
- Each folder method and builtin has a call checked against Python, and
  the Hypothesis grammar takes the one-argument and width methods.
- The class-info, instance and module-path walks keep their own stacks,
  so chains thousands of links long scan without RecursionError.
- Four spellings a mutant sweep found unpinned: class info read by a
  method and rebuilt, and an instance bound in a conditional and in a
  boolean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Against master 172f690, tests/open_population adds 457 node ids (432 in
test_stream_injection.py, 25 in test_streams.py) and removes none; all
collect as unit. unit moves from 18,789 to 19,246 and the total from
25,359 to 25,816.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Section 6 now names master 172f690 as the base, lists the two tier files
the recount sets, describes the merge and the recount, and records the
host run at ec888be: tests/open_population 694 passed and 1 strict
xfail, the seven-path command 1,022 passed, 2 skipped and 1 strict xfail,
24 of 25 scan mutants killed (the survivor makes a path no module has),
and the old-vs-new scan differential with only its intended differences.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — 305b171b Deployed Oct 26, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant