diff --git a/system/datasets.md b/system/datasets.md index 50fbc61..c695238 100644 --- a/system/datasets.md +++ b/system/datasets.md @@ -78,7 +78,8 @@ Two samples ran through the whole chain, nine F146 epochs each, the reference from the first six and three epochs differenced, every check clean: pointing 268.54 −28.54 SCA 8 (`r00340-gb1`), 515 packets, and pointing 266.71 −28.93 SCA 2 (`r00340-gb3`), 1,278 packets, every kept -source above S/N 20. Three differences fall below the gate's +source above S/N 20; run 139 on the same sample read 1,196 at equal +kept sources, cause unread. Three differences fall below the gate's `min_epochs`, so the gate sends a sample's sets unjudged. Under `reference_psf = "science"` the core, ring and registration stars are the reference mosaic catalog's (rapid PR 465). On `r00340-gb3`'s sample (run @@ -200,7 +201,7 @@ tables and time series per filter. | `dev` transposes the science PSF for this simulation | measured on one epoch: core −0.0694 → −0.0688, ring −0.0339 → −0.0385; not taken | | `dev`'s artefact repair threshold (10,000 DN/s) is below the stars' cores | the overlay: off, with `dev`'s rimtimsims gain, read noise and saturation | | The WCS | right as converted: Gaia DR3 at +0.03 to +0.12 / −0.01 to +0.08 px; no `crpix_shift` | -| F213's differences over-subtract star cores (core −0.050 to −0.075, 10k to 13k negative candidates above S/N 20 per epoch) | recorded; the quality gate holds every such set once the run has `min_epochs` differences (a five-epoch sample sends its first four unjudged); the next measurement for this dataset | +| F213's differences over-subtract star cores (core −0.050 to −0.075, 10k to 13k negative candidates above S/N 20 per epoch) | recorded; the quality gate holds every such set once the run has `min_epochs` differences (a five-epoch sample sends its first four unjudged). `reference_psf = "science"` on epoch 4571 brings the core inside its bound but the ring further outside (core −0.0694 → −0.0044, ring −0.0339 → −0.0557, 30 stars, run 133), so it is not taken; the overlay's comment records it (rapid PR 462). A reference-against-science width mismatch (1.39 against 1.30 px), the next lever a code knob | | Every epoch fills the 500-source cap, and a difference takes 28 to 72 min | recorded | | The 2025-05-13 truth's times and star positions match the images, its injected signal does not; its dim-EB time series carry the blended-EB ids | recorded; no completeness until a truth for this delivery exists | diff --git a/system/foundation.md b/system/foundation.md index 61aea42..a09257d 100644 --- a/system/foundation.md +++ b/system/foundation.md @@ -758,7 +758,9 @@ only the offset ranges the run's stamped sets recorded, overlapping ranges merged, with no consumer group; keeps a set's messages by key and headers; requires every stamped packet to be seen; and decodes and byte-compares against the outbox one set per field, the field's latest. -Zero kept is a failure, so a run with no stamped set fails. The +A run with no stamped packet set, or whose stamped sets hold no +packet, fails (rapid PR 458), so a run whose sets the gate holds fails +the check. The client's own record checksum is switched off, since it is pure Python and held the read to about 50 messages a second. A run's check takes about 4 s at one pointing and 388 s over 33,650 packets, whatever the @@ -1193,7 +1195,8 @@ took 666 s against 1,512 s. a time, and the consumer check, streams their output, and prints one line, `Check run : CLEAN|FAIL validate / reader / consumer / `, exiting non-zero -when any of the three fails. Every run of the breadth part was checked +when any of the three fails; through the consumer check it reads a run +that sent no packet as FAIL. Every run of the breadth part was checked with it, and every one was clean, as was every run of the second iteration: the injected footprint's on-demand run validated in 475 s and read 33,650 packets back in 388 s. diff --git a/system/loop.md b/system/loop.md index 544d225..8273c19 100644 --- a/system/loop.md +++ b/system/loop.md @@ -126,14 +126,15 @@ night uses a dataset name of its own. 1/1 consumer M/M ...`, and exits 0. Any failure prints `FAIL` and exits non-zero. -**What the exit codes mean**, as on `rebuild` at `d4866eab`: +**What the exit codes mean**, as on `rebuild` at `efc03fc6`: | Code | `loop` | `job local` | |---|---|---| | 0 | the selection drained with no failed or refused job (or one `--once` pass saw none) | the job published; under a run that is not live, the refusal is on stderr and on its row | | 1 | it drained around failed jobs, or jobs published with their binding refused. Each is named on stderr and in the last pass's `failed`, and the rest of the selection drained around them. Read the row's `refusal` (`rp job show `), fix the cause, rerun with `rp job submit --job --job-definition $JOBDEF`, then start the loop again | the job published but its binding was refused under a live run | | 64 | a usage error: the run does not exist, the selection cannot be parsed, its first pass found no file of the selection under the source root, or the dataset has no single official Gaia catalog when a packet job is due | the run or an input does not exist, or the arguments are wrong | -| 70 | an unexpected error, traceback on stderr | the stage failed: an exception, or a tool's nonzero exit (`ToolFailed`, with the tool's name, code and last 20 lines on the row) | +| 65 | not returned | the stage rejected a declared input (difference, reference or import), `CommandExit: ` on the row | +| 70 | an unexpected error, traceback on stderr | the stage failed: an exception (a stage's own `KeyError` among them, with its traceback), or a tool's nonzero exit (`ToolFailed`, with the tool's name, code and last 20 lines on the row) | | 75 | a transient failure: the database was unavailable, or an AWS call was throttled or timed out. Start the loop again | the same; Batch repeats such a job on its own | ## The selection diff --git a/system/stage-contract.md b/system/stage-contract.md index f9754e6..4380a46 100644 --- a/system/stage-contract.md +++ b/system/stage-contract.md @@ -152,8 +152,9 @@ definition. |---|---| | 0 | The job was published, or was already published; or it was published and its automatic binding was refused only because its run is not live, the refusal on its row and on standard error | | 1 | The job was published, but its automatic binding was refused for another reason, such as an expected binding no longer open; its rows are an unbound candidate | -| 64 | Usage: bad arguments, no outputs root, a stage with no `run(job)`, a run or input that does not exist, or a database configuration that cannot be read | -| 70 | The stage or the publication raised, a tool's nonzero exit included (`ToolFailed`); the job is marked `failed` | +| 64 | Usage: bad arguments, no outputs root, a stage with no `run(job)`, a run or input that does not exist (the foundation's `NotFound`), or a database configuration that cannot be read | +| 65 | A declared input the stage rejects (`CommandExit(65, reason)`): difference's, reference's and import's input checks | +| 70 | The stage or the publication raised, a tool's nonzero exit (`ToolFailed`), a stage's own lookup error, a missing dependency or a failed product write included, with the traceback; the job is marked `failed` | | 75 | A temporary dependency failure: the database unavailable or failing mid-query after the connection's own retries, or a storage or AWS call (the parameter store's included) throttled, timed out, unreachable or answered with HTTP 503 | A failed job's row records the code the process exits with, and its diff --git a/system/tool.md b/system/tool.md index 7d7abcb..eceb92a 100644 --- a/system/tool.md +++ b/system/tool.md @@ -53,10 +53,10 @@ the [stage contract](stage-contract) page lists what `job run` returns. | 0 | Success; also a job published under a run that is not live, the refusal on its row and on standard error | | 1 | A refused change group or a job published with its binding refused; `job wait` when a job failed; `loop`, `--once` included, when a job of the run failed or was published with its binding refused | | 2 | `job wait`'s `--timeout` passed first | -| 64 | Usage: every argparse parse failure (help exits 0), a missing run, job or group, an unreadable file, `RAPIDPIPE_OUTPUTS_ROOT` unset, a database configuration error, and `loop` when its first pass finds no stem of the selection under the source root | -| 65 | A declared input absent, corrupt or incompatible (a stage's own invocation) | +| 64 | Usage: every argparse parse failure (help exits 0), a missing run, job or group (the foundation's `NotFound`, no other lookup error), an unreadable file, `RAPIDPIPE_OUTPUTS_ROOT` unset, a database configuration error, and `loop` when its first pass finds no stem of the selection under the source root | +| 65 | A declared input rejected by the stage reading it: difference, reference or import (`CommandExit: ` on the row) | | 69 | Reserved; nothing in this build returns it | -| 70 | An unexpected error, logged with its traceback; a failed external tool (`ToolFailed`) | +| 70 | An unexpected error, logged with its traceback: a stage's own `KeyError` or `IndexError`, a missing dependency, a failed product write; a failed external tool (`ToolFailed`) | | 75 | A transient failure: the database unavailable or failing mid-query, a storage or AWS call throttled, timed out or unreachable; Batch retries it | A failed job's row records the same code its process exits with.