[rusholme]

What and why

Task 7 of the unattended run foundation-night-2026-10-21: rapid_docs records what the night built, validated and measured, and the reference and difference pages are rewritten for the foundation contract. Task 6's ledger line is TASK DONE 6, so everything is documented in full.

The pull requests documented: Caltech-IPAC/rapid#294 (difference determinism, ANALYZE official), #296 (the PSF kind, the read role's SELECT), IPAC-SW/rapid_systems#412 (Batch retries exit 75, three attempts, 24 GiB), Caltech-IPAC/rapid#298 (the cutover and the live loop over 69 epochs), #299 (the campaign: reference, 70 differences, chain over five epochs, the comparison with dev), #300 (the ten checks, the change group making the night official, the mid-night replacement, undo, two groups through the pooler).

Files

• system/foundation.md:
  • the PSF key gains kind, with the identity rule and dev's missing column stated; the import's PSFKIND check;
  • the publisher analyses official when its binding wrote there;
  • "A job on Batch": the retry rule as stated behaviour (exit 75 retried, three attempts, everything else ends the job, why, and the no-backoff limit);
  • the reference and difference sections reduced to one paragraph pointing at the stage pages;
  • "One schema file": the PSF kind was the first incremental change;
  • "On SMDC": a sixth part, the night: the PSF kind and read role SQL, the cutover as run, the live loop 69 deep, the campaign, one change group making it official, the mid-night replacement and what the model leaves as published, undo, two groups through the pooler;
  • new "Validation": the ten checks as the contract a night meets, with the 2026-10-21 outcomes;
  • "Measurements" rewritten for the night: per stage per epoch, the fan-out, the live chain's wall time, chain depth, change-group time per key at 145 keys, the checks' time, database size, storage, the retry; "Against the hour" updated and pointed at the latency-later ruling;
  • "Not decided here": the PSF kind and read role leave (decided); the ANALYZE official clause leaves (fixed in #294); the packets-after-replacement observation enters; undo after a later group, a non-live run's refused binding reported as FAILED by Batch, and CPU-family dependence enter; chain cost, forced photometry and change-group cost narrowed by the night's figures.
• system/reference.md, system/difference.md: rewritten for run(job): scope, inputs by key (the two PSF kinds), the steps in dev's order, declared outputs and the drawn rfid / pid, settings tables of the run's body, determinism, what the night measured, and each page's "Not decided here".
• system/stage-contract.md: inputs told apart by key, the PSF kind among them; exit 75 in the table; the retry paragraph replaces "Nothing retries by itself".
• system/decisions.md: four rulings of 2026-10-21 (12:34 to 12:38 PDT): decision-latency-later, decision-psf-kind, decision-read-role, decision-batch-retry.
• system/index.md: the scope cells of decisions.md, foundation.md, stage-contract.md, reference.md, difference.md.

Divergences from the brief, with the reason

• Rewrite, not fold. The brief offers either; rewriting the two stage pages and cutting the foundation page's two sections to a pointer leaves fewer words (the two pages go from 4,113 to about 2,200 words, the foundation sections from about 400 to 110) and keeps AGENTS.md's one-page-per-stage layout, with each statement on one page.
• Stage-page "Not decided here" bullets closed by the foundation, not by a ruling. On the old reference page: reference_sets (closed by the refimages key and the drawn rfid), registration concurrency beyond the advisory lock (no registration exists: the publisher and the identity sequence replace it), and the filter map existing twice (unified in rapidpipe.products.filters, which prep.py re-exports). The old difference page had none. Each is closed by what the foundation page and code now state; the remaining open items stay.
• The two-epoch measurement table is gone from the foundation page. The night's table replaces it; the two-epoch difference timings and the 0.99989 parity figure moved to the difference page beside the night's.
• No run ids, job ids, Batch ids or bucket names on the site, as foundation, stage-contract, decisions, index: milestones 4 to 6 on SMDC and Batch #89: the runbook in the pipeline repository carries them.
• The 18:45 PDT trim of the campaign's chain to five epochs is described as what ran, not recorded as a ruling: it is a choice about this run, not about the design.

Build

Not built on the host (no Sphinx; provisioning one is an install). docs.yml is the check. scripts/check-public-safety.sh passes locally; every {ref} decision target resolves; the changed pages carry no em dashes and none of AGENTS.md's excess-vocabulary words.

Measurements

Documentation only: no operation ran. The figures on the pages are tasks 1 to 6's.

Improvements seen (not applied)

• The specification's "Not decided here" still lists reference eligibility and selection pointing at the reference page's old wording about dev's launcher rule; it is still open, but the reference page now says only that the launcher chooses and records the frames.
• decisions.md's 2026-09-23 to 2026-09-26 carried rulings still cite difference and reference for registration and promotion rules those pages no longer state, since the run model is retired; a pass marking them superseded by the foundation would stop the cross-references pointing at absent text.
• load.md still states a psfs key without the kind; it describes the retired run model and is a candidate for the same rewrite.

What the next task must know

• The foundation page's "Validation" section is the night's contract; its ten checks are the runbook's.
• Status stays DRAFT on every page.

Left

• The other run-model pages (load, crossmatch, statistics, prune, alerts, export, finalize, maintain) as foundation, stage-contract, index: the foundation and thin chain as one page, replacing runs, products, checks and releases #88 listed.

🤖 Generated with Claude Code