docs: the older stage contract retired; finalize's page retired, stage and selftest gone from the tool page (mig-contracts)

system/decisions.md

@@ -1011,3 +1011,6 @@ The global default stays `zogy`, and OpenUniverse and GBTDS are unmeasured under
 
 (decision-naive-no-psfcat)=
 **The difference job makes no Photutils PSF-fit catalog for the naive subtraction**: a divergence from dev, which fits both naive catalogs, positive and negative, with no knob to turn them off. They are diagnostics nothing reads: they are not published, and no stage or script opens them. The naive image, its uncertainty and its SExtractor catalogs are still made, and `naive_diffimage_flag` is unchanged. A catalog not run sets no outcome bit, so `infobitssci` bits 4 and 5 are never set. Same bytes, measured on RimTimSim F213 field 4682737 SCA 2, epoch 4571, against run 99's reference and PSF rows (runs 122 on `:78` and 124 on `:79`, both CLEAN). The difference job takes 4,169 s before and **2,645 s** after; the naive catalogs had taken 1,247 s and 245 s. The 500 `sources` rows are equal in every non-identity column. The `diffimages` row is equal but for `checksum`, whose file differs only in its job-id and timestamp header cards; the pixels are equal. (diff-trim task 1, rapid PR 446, 2026-10-25)
+
+(decision-older-contract-retired)=
+**The older stage contract is retired; `run(job)` is the one contract**: `admit`, `finalize`, the shared runner `run_stage`, the per-stage settings files, the `rapidpipe stage` and `rapidpipe selftest` commands and `finalize`'s fixture are deleted from `rebuild`, and git keeps them. The loop never ran either stage. Retiring the code does not answer whether a post-processing stage (database ids in headers, the header stamp) returns: that stays the [migration](migration) page's question for the team. (mig-contracts task 1, rapid PR 464, 2026-10-25)

system/finalize.md

@@ -2,10 +2,11 @@
 
 **Status: DRAFT**
 
-This stage still runs on the older contract ([stage-contract](stage-contract),
-"Stages on the older contract"), and the loop does not run it. Where this
-page describes registration, custody, runs or promotion, the
-[foundation](foundation) page governs.
+The module this page describes was retired from the `rebuild` branch on
+2026-10-25 with the older stage contract it ran on (rapid PR 464); git keeps it.
+The loop never ran it. Whether a post-processing stage (database ids in
+headers, the header stamp) returns is the [migration](migration) page's
+question for the team. This page stays as the record of the port.
 
 The `finalize` stage stamps a difference image's header and republishes
 its bundle as new instances. It lands on the pipeline repository's

system/index.md

@@ -20,7 +20,7 @@ repository's history. The rebuild starts from this page.
 | [`decisions.md`](decisions) | DRAFT | The team's rulings, one dated line each with its author: Ben's rulings of 2026-10-24 on the second iteration (the bright-star residual flagged, not fixed; Spot as a probe), the association in the difference job; Ben's rulings of 2026-10-22 on what follows the wide-area ramp (housekeeping first, the other filters before injection, the reference bound by `mjdobs`, a reference per dataset, field and filter with the chain in admission order, injection outside the pipeline, the container-side producer last, the `catalogs` key with its dataset, difference outputs trimmed, GBTDS after injection, the PSF from the CRDS ePSF reference, Spot parked; the other filters as datasets of their own), his rulings of 2026-10-21 evening (the instance type recorded with no family pinned; the exercised HLWAS pipeline as the plan in force), his rulings of 2026-10-20 and 2026-10-21 on the foundation, its SMDC step and the night through the chain (latency later, the PSF kind, the read role, the Batch retry), his rulings of 2026-09-27, his rulings of 2026-10-06 on weekly maintenance on SMDC, earlier rulings, and the rulings carried from the build pending team review |
 | [`foundation.md`](foundation) | DRAFT | The foundation and the thin chain: runs with settings resolved from three layers, jobs, recorded inputs with the object catalog read at execution, scientific keys, official bindings and change groups, publish-last; the PSF key's kind and the `catalogs` key's dataset; the decisions with their reasons; the verbs, the `file://` and `s3://` prefixes with the working directory removed after upload, and a job on Batch with its retry rule, instance type, Batch job id and per-stage memory; Spot as a probe and its four rules; the thin chain, with the `intake`, `import`, `psf` and `packet` stages, association in the difference job, the reader, the quality gate, and the publication to Kafka from the packet job with the recovery sweep and the direct consumer check; the eight queries and the team's spatial shapes; the procedure proof; the SMDC procedure, the night run on it (cutover, live loop, campaign, one group making it official, a mid-night replacement, undo) and the HLWAS run from SOC L2 files to the topic (the live run beside the GBTDS one, one pointing, the loop, the ramp to 426 epochs) and the housekeeping after it (bytes, Batch, the pass, the reference bound), and the breadth run over the six other filters, one dataset each, and why, the fixes and the second iteration; the ten validation checks, the script that runs six of them, the check script for a run and their outcomes; the night's, the ramp's, the housekeeping's, the breadth run's, the fixes' and the second iteration's measurements, with the null test's residuals labelled as artefacts, the bright end flagged, completeness and purity of injected sources per magnitude, and Spot's first data points; what is not yet decided, cross-band association among it |
 | [`datasets.md`](datasets) | DRAFT | The simulations the rebuild reads, one section each (HLWAS r00342, GBTDS r00340, OpenUniverse TDS): their data, their quirks and each quirk's disposition on the dataset's settings overlay, a reader or a recorded line |
-| [`stage-contract.md`](stage-contract) | DRAFT | The `rapidpipe` package and its dependency direction; a stage as `run(job)` and its `JobContext`; settings from the run's body; the manifest; inputs told apart by key, the PSF kind among them; running a job locally, again or on Batch, with its image digest, instance type, Batch job id and peak memory; exit codes, a refusal because the run is not live exiting 0, and the Batch retry of exit 75; `admit` and `finalize`, still on the older contract |
+| [`stage-contract.md`](stage-contract) | DRAFT | The `rapidpipe` package and its dependency direction; a stage as `run(job)` and its `JobContext`; settings from the run's body; the manifest; inputs told apart by key, the PSF kind among them; running a job locally, again or on Batch, with its image digest, instance type, Batch job id and peak memory; exit codes, a refusal because the run is not live exiting 0, and the Batch retry of exit 75 |
 | [`reference.md`](reference) | DRAFT | The `reference` stage on `run(job)`: its scope and `l2files` inputs, the awaicgen coadd and SExtractor catalog in `dev`'s order, the header stamp, the three declared tables and the drawn `rfid`, its settings, and what it measured |
 | [`difference.md`](difference) | DRAFT | The `difference` stage on `run(job)`: its scope and four inputs with the two PSF kinds, the ZOGY, SFFT and naive steps in `dev`'s order, the declared `diffimages` row and source set with the drawn `pid`, the association with its objects and memberships, the bright-star footprint flag and the quality numbers, one differencer published, the gzipped files it keeps and why, its settings and determinism, and what it measured against `dev` |
 | [`intake.md`](intake) | DRAFT | The `intake` stage on `run(job)`: one SOC L2 ASDF file under the run's source root, the conversion ported from `dev`, the check that its science PSF was made from the file's own ePSF reference, the declared `l2files` row and the `exposures` row of the one SCA that expects it, with the packed `expid`, its settings (the reader, the DQ mask, the WCS translation and re-derivation) and the Gaia-offset metric, and what the ramp measured |
@@ -52,13 +52,12 @@ observability
 migration
 ```
 
-## Retired and older-contract pages
+## Retired pages
 
 These pages are not stages the loop runs. The first seven describe modules
-retired from `rebuild` with the run model, kept as the record of their port;
-`finalize` is still in the package on the older stage contract
-([stage contract](stage-contract), "Stages on the older contract"), with
-`admit`, and the loop runs neither.
+retired from `rebuild` with the run model; `finalize` was retired on
+2026-10-25 with the older stage contract it ran on, with `admit` (rapid PR 464).
+Each is kept as the record of its port.
 
 | Document | Status | Scope |
 |---|---|---|

system/migration.md

@@ -28,7 +28,7 @@ page or ruling states why.
 | `JOBPROCDATE` and the `<procdate>/req<reqid>/jid<N>/` layout | no processing date; `<RAPIDPIPE_OUTPUTS_ROOT>/<run>/<stage>/<job>/<attempt>/` | changed | bytes are immutable; a retry writes new files and never overwrites a referenced one ([foundation](foundation.md#why-it-is-this-way)) |
 | one test database per dataset (`rapidopsdb`, `socsimsdb`, `fakesourcesdb`, ...) | `dataset` in every scientific key; one live run per survey and dataset, all in one database | changed | one live run per survey and dataset, for every stage ([foundation](foundation.md#why-it-is-this-way)) |
 | registration stages 3, 6 and 9 (`parallelRegisterCompletedJobsInDB*.py`) and done files | the publisher, at the end of `job run`: one transaction loads the declared rows and attempts the binding; an empty expected product binds nothing unless the stage marks it `empty_ok` | changed | publish last ([foundation](foundation.md#publish-last)) |
-| post-processing pipeline (ppid 17, stages 7 and 9: database ids in headers, checksums) | `finalize` on the older stage contract, not run by the loop; `pid` and `rfid` drawn by the stage; checksums at publish | partial | no recorded reason; the port of `finalize` is open ([stage contract](stage-contract.md#not-decided-here)) |
+| post-processing pipeline (ppid 17, stages 7 and 9: database ids in headers, checksums) | none: `finalize`, never run by the loop, was retired with the older stage contract (rapid PR 464); `pid` and `rfid` drawn by the stage; checksums at publish | partial | no recorded reason; the port of `finalize` is open ([stage contract](stage-contract.md#not-decided-here)) |
 | prune (`pruneNotBestMerges.py`, `pruneNotBestSources.py`, stages 16 and 17) | a membership is official when its source set is; nothing is deleted | changed | a superseded source set's memberships leave with it ([foundation](foundation.md#why-it-is-this-way)) |
 | `pruneRedundantMerges.py` | the publisher reconciles two jobs' founded objects under per-field locks | changed | two epochs in flight at once are reconciled at publish ([foundation](foundation.md#why-it-is-this-way)); {ref}`association in the difference job <decision-association-in-difference>` |
 | `rapid_db.py` writer methods (`add_diffimage`, `start_job`, ...) | `rapidpipe.db.connection` and `rapidpipe.foundation` (the publisher, `official change`) | changed | stages read, compute in memory, and the publisher writes ([foundation](foundation.md#why-it-is-this-way)) |
@@ -95,7 +95,7 @@ page or ruling states why.
 | the `.ini` (`cdf/awsBatchSubmitJobs_launchSingleSciencePipeline.ini`) | `foundation-thin.toml`, a dataset overlay and `--set`, stored whole in `runs.settings` with `runs.layers` | changed | [foundation](foundation.md#runs); a dataset's quirk is a key on its overlay ([datasets](datasets)) |
 | `docker run -it --entrypoint bash` into the image, scripts by hand | `rapidpipe` from a Python environment; jobs on the Batch job definition `rapid-rebuild:<revision>`; the repository README's Getting started runs one epoch locally inside the pipeline image against a throwaway PostgreSQL 18 | changed | no recorded reason |
 | exit codes 64 and above are errors | `rapidpipe/exitcodes.py`: 0, 1, 2, 64, 65, 69, 70, 75; a tool's nonzero exit fails the job with 70; a transient failure exits 75; `loop` exits 1 for a failed or refused job and 64 when its first pass admits nothing | changed | {ref}`a tool's nonzero exit fails the job <decision-tool-exit-checked>`; [stage contract](stage-contract.md#exit-codes) |
-| per-stage `.out` log files in `/work` | one `rapidpipe.log` line shape to stderr, and `log/<stage>.log` beside the attempt's outputs, in UTC | changed | no recorded reason; [observability](observability.md#the-line) |
+| per-stage `.out` log files in `/work` | one `rapidpipe.log` line shape to stderr, in UTC, which Batch sends to CloudWatch | changed | no recorded reason; [observability](observability.md#the-line) |
 | test entries per date (`dev/*_tests.rst`) | `tests/unit`, `tests/db`, `tests/cli`; `scripts/validate_run.py` and `scripts/check_run.sh` per run | changed | no recorded reason |
 | branch from `dev`, pull request to `dev` | branch from `rebuild`, pull request to `rebuild`, green CI; a `ported-from:` pin on every ported module, and `scripts/science-drift.sh` listing unported `dev` commits | changed | {ref}`the dev-to-rebuild science flow <decision-science-flow>` |
 

system/stage-contract.md

@@ -31,15 +31,15 @@ import package `rapidpipe`. The word `rapid` alone means the project.
 | `rapidpipe.foundation` | The operations over `database/schema.sql`: runs and jobs, `change` and `undo`, `publish`, the stage contract (`JobContext`, `run_job`), and reading and writing `file://` and `s3://` prefixes. |
 | `rapidpipe.stages` | One module per stage. |
 | `rapidpipe.launch` | Submitting a recorded job to Batch. |
-| `rapidpipe.selftest` | The packaged fixture of `finalize`, the runner that drives the stages still on the older contract, and the fake toolkits the `reference` and `difference` unit tests use. |
 | `rapidpipe.cli` | The command-line tool: parses the command line and prints, with no orchestration of its own. |
 
 A unit test enforces the dependency order across every module,
 including lazy and relative imports: a unit imports only units strictly
-below it. From lowest to highest: the leaf modules (`exitcodes`, `log`
-and `revision`, with no `rapidpipe` import), `products`, then
+below it. From lowest to highest: the leaf modules (`exitcodes` and
+`log`, with no `rapidpipe` import), `products`, then
 `db`, `science` and `foundation` (which do not import one another),
-`stages`, `launch`, `selftest`, then `cli`. A stage module never imports
+`stages`, `launch`, then `cli`. The `reference` and `difference` unit
+tests' fake toolkits are under the repository's `tests/support/`. A stage module never imports
 another stage module.
 
 ## The stage contract
@@ -168,23 +168,17 @@ Off Batch nothing retries by itself. Every retry, whether Batch's, a
 and one that finds the job published stops at once. The
 [foundation](foundation) page states the rule and why.
 
-## Stages on the older contract
-
-`admit` and `finalize` still run as `rapidpipe stage <name> --run
---unit --attempt --inputs --outputs [--settings] [--dry-run]`, through
-the shared runner `run_stage` in `rapidpipe/stages/contract.py`, with
-their default settings under `rapidpipe/settings/<name>.toml` and
-`finalize`'s fixture under `rapidpipe selftest`. They write files and a
-manifest but publish nothing, and their exit codes are the six of
-`STAGE_EXIT_CODES` (0, 64, 65, 69, 70, 75). They cannot run on Batch:
-`job submit` runs only `run(job)` stages, and the loop runs neither. Each moves to `run(job)` when
-it is ported, declaring the rows its registration block carries today
-(`l2files` for `admit`, a `diffimages` row for `finalize`) as Parquet
-outputs, and computing
-limiting magnitude, overlapping fields and sky indices before
-publication.
-
-`reference` and `difference` ported the same way, and the port set the
+## The older contract, retired
+
+`admit` and `finalize` ran on an older contract, `rapidpipe stage <name>
+--run --unit --attempt --inputs --outputs`, through a shared runner,
+with per-stage settings files and a packaged fixture under `rapidpipe
+selftest`; they wrote files and a manifest and published nothing, and
+the loop ran neither. They, the runner, the `stage` and `selftest`
+commands and the fixture were retired on 2026-10-25 (rapid PR 464); git keeps
+them. `run(job)` is the one contract.
+
+`reference` and `difference` were ported to `run(job)`, and the port set the
 pattern: the module's settings file becomes the stage's table in the
 run's body, its old entry point, selftest module and fixtures are
 deleted rather than kept beside the new one, and the science modules
@@ -203,5 +197,7 @@ needs.
 
 ## Not decided here
 
-- The port of `admit` and `finalize` to `run(job)`.
+- Whether a post-processing stage (database ids in headers, the header
+  stamp `finalize` wrote) returns: the [migration](migration) page's
+  question for the team.
 - The C tool packaging beneath `rapidpipe.science`.

system/tool.md

@@ -28,13 +28,12 @@ in this repository's history.
 | `official undo <group-id> --who --why` | Applies the inverse of a change group and prints the new group's id |
 | `publish --run N [--settings FILE]` | Produces every official packet set of the run not yet stamped to the Kafka topic, stamping each once acknowledged; prints the counts |
 | `loop --run --filters --scas --pointings [--passes] --job-definition D [--job-queue Q] [--max-intake N] [--interval S] [--once]` | Admits what is new under the run's `[intake] source_root` for the selection and submits every ready job one stage forward on Batch, one summary line per pass, until a pass finds nothing new, pending or to submit ([loop](loop)) |
-| `stage run <name> ...`, `stage <name> ...`, `stage list`, `stage describe <name>` | Runs, lists or describes the stages on the older contract, `admit` and `finalize` ([stage contract](stage-contract), "Stages on the older contract"); the loop runs neither |
-| `selftest --stage finalize [--real-tools] [--work-dir D] [--output-location L]` | Runs `finalize`'s packaged fixture and checks its outputs; no other stage has one |
 
 The stages the loop submits (`psf`, `intake`, `reference`, `difference`,
 `packet`) and `import`, which a person runs by hand before the loop, are
-reached through `job local`, `job submit` and `job run`, never through
-`stage`.
+reached through `job local`, `job submit` and `job run`. The older
+contract's `stage` and `selftest` commands were retired on 2026-10-25
+(rapid PR 464).
 
 ## What the tool does not have
 
@@ -47,7 +46,7 @@ and the named queries in `database/queries.sql`. To cancel a Batch job:
 ## Exit codes
 
 Every `rapidpipe` process uses `rapidpipe/exitcodes.py` (`ExitCode`);
-the [stage contract](stage-contract) page lists the six a stage returns.
+the [stage contract](stage-contract) page lists what `job run` returns.
 
 | Code | Meaning |
 |---|---|