decisions: the loop's reference from the rows whose overlapfields list the field, as dev (ref-overlap)

system/decisions.md

@@ -999,10 +999,12 @@ The global default stays `zogy`, and OpenUniverse and GBTDS are unmeasured under
 **The difference's PSF-fit catalog fills its non-finite pixels from their neighbours for the finder and passes them as `mask=`**: a divergence from dev (dev runs DAOStarFinder and `PSFPhotometry` on the masked difference with its NaNs in place, no knob). Under photutils 3.0.0, DAOStarFinder convolves and measures with the NaNs in place, mask or not, so a star within about 4 px of one masked pixel was never found; part 12 step 6 measured a quarter of SCA 7's area blinded. The fill is `interpolate_replace_nans` with a Gaussian of σ 1 px (a hole wider than the kernel is 0), not a zero fill, which split a star with a masked peak into four rows; the fit ignores the masked pixels. Same bytes (`r00342-inj1` `001_0001` F146 passes 4 to 6, ZOGY, against run 72's and run 81's references): SCA 7 detected below AB 25 0.728 → **0.959** (within 5 px of a mask 9 → 81 of 93, beyond 275 → 293 of 297), false positives per epoch 174 (highest 186) → 165 (167), runs 115 → 117; SCA 8, the control, 542 → 545 of 600, false positives 214 (233) → 214 (233), runs 103 → 116. A star whose own peak pixel is masked is still missed. (finder-mask task 1, rapid PR 439, 2026-10-25)
 
 (decision-reference-by-field)=
-**The loop builds a field's reference only from the L2 rows whose own field is that field**: a divergence from dev, recorded here for the first time (`rapidpipe.foundation.loop.plan`, no knob). dev's launcher (`awsBatchSubmitJobs_launchSingleReferenceImagePipeline.py`, `get_overlapping_l2files`) takes every L2 file whose `overlapfields` lists the field, ordered by `mjdobs` and then by distance from the tile centre. On fixed-pointing simulations the two rules agree. On flight program 1049's four dithered F158 WFI02 epochs, 0.04 degrees apart, they do not:
+**The loop builds a field's reference from the L2 rows whose `overlapfields` list the field, as dev does**: this replaces the earlier rule, which took only the rows whose own field is that field (`rapidpipe.foundation.loop.plan`, no knob). dev's launcher (`awsBatchSubmitJobs_launchSingleReferenceImagePipeline.py`, `get_overlapping_l2files`) takes every L2 file whose `overlapfields` lists the field.
 
-- The frames' centres fall in three fields: epoch 1 in 3014319, epochs 2 and 3 in 3014320, epoch 4 in 3010224.
-- Every earlier epoch lists epoch 4's field in its `overlapfields`.
-- Under the loop's rule no later epoch has two earlier frames in its own field, so with `min_frames` 2 the loop builds no reference epoch 4 can be differenced against. It builds field 3014320's from epochs 2 and 3 and holds the other two fields.
+- **The frames.** A reference is planned for a field and filter that a selected row has as its own field; with a bound, a row at or after it. Its frames are the selected official rows of the filter before the bound whose `overlapfields` list the field, of any SCA, in `mjdobs` order.
+- **What is not ported.** dev's second sort key, distance from the tile centre, is not ported, so this is membership parity, not full parity. The reference stage coadds in `mjdobs` order, so the order matters only past `max_frames` (25).
+- **Why it changed.** On flight program 1049's four dithered F158 WFI02 epochs, 0.04 degrees apart, the frames' centres fall in three fields: epoch 1 in 3014319, epochs 2 and 3 in 3014320, epoch 4 in 3010224. Under the earlier rule the loop built no reference epoch 4 could be differenced against, and run 118's had to be submitted by hand.
+- **Flight, measured.** Under this rule the loop alone (run 126) builds field 3010224's reference from epochs 1 to 3 and differences epoch 4 against it. Every measured number of epoch 4 matches run 118's hand-built reference: core +0.0013, ring −0.0050, usable fraction 0.781, 328 packets. No reference is built for the two fields nothing is differenced against.
+- **HLWAS, measured.** On a fixed pointing the rule picks the same frames. The injected sample, `r00342-inj1` `001_0001` SCA 8 F146 passes 4 to 6 under ZOGY (run 127), builds its reference from passes 1 to 3, the frames run 81's reference holds. It detects 546 of 600 against run 116's 545, with false positives per epoch 213 mean and 233 highest against 214 and 233.
 
-Run 118's reference for field 3010224 was submitted by hand with dev's selection, epochs 1 to 3, and epoch 4 was differenced against it. It covers 78.4% of epoch 4's pixels: 28.9% by two or more frames, 53% to 54% by epoch 1 or 3 alone. The code is unchanged. (flight-first task 1, 2026-10-25)
+(flight-first task 1, 2026-10-25; ref-overlap task 1, rapid PR 450, 2026-10-25)

system/loop.md

@@ -201,12 +201,15 @@ Its order:
    taken, at most `--max-intake` per pass. Each names its filter's and
    SCA's science PSF. The first intake of an exposure expects the
    shared `exposures` key; the others leave it out.
-3. **reference**: one job per field and filter with at least
-   `[reference.selection] min_frames` official L2 rows, no official
-   reference and none pending or failed, and only while no intake of
-   the run is pending, so a reference coadds every epoch the drain
-   admits rather than the first two to land. The rows are one SCA's,
-   the first in order with enough frames; the job's scope records which.
+3. **reference**: one job per field and filter that a selected L2 row
+   has as its own field, with no official reference and none pending or
+   failed, and only while no intake of the run is pending, so a
+   reference coadds every epoch the drain admits rather than the first
+   two to land. Its frames are the selected official rows of the
+   filter, of any SCA, whose `overlapfields` list the field, at least
+   `[reference.selection] min_frames` of them, as `dev` selects them
+   ({ref}`ruling <decision-reference-by-field>`). The job's scope records
+   the first SCA among the rows it serves.
    With a reference bound set, the rules change as described under
    [The reference bound](#the-reference-bound).
 4. **difference**: one job per official L2 row whose field has an
@@ -295,16 +298,17 @@ seconds after the last one started, or at once if the pass ran longer.
 ## The reference bound
 
 A run may set `[reference.selection] before_mjd`, a modified Julian
-date. With it set, a field's reference is built only from the field's
-epochs before the bound, and only the epochs at or after it are
-differenced, so no epoch is subtracted from a reference that holds it.
+date. With it set, a field's reference is built only from the epochs
+before the bound that overlap the field, and only for a field an epoch
+at or after the bound has as its own. Only the epochs at or after it
+are differenced, so no epoch is subtracted from a reference that holds it.
 The bound is a date rather than a list of passes because a survey such
 as GBTDS has no passes ({ref}`ruling <decision-reference-bound>`). Absent, every
 epoch may build the reference and every epoch is differenced, as
 before.
 
-With the bound set, a reference job names only the field's official L2
-rows with `mjdobs` before it, and waits until the selection's intake has
+With the bound set, a reference job names only those official L2 rows
+with `mjdobs` before it, and waits until the selection's intake has
 drained: every listed file the selection admits is published and no
 intake of the run is pending. No intake pending is not enough on its
 own, because a pass admits at most `--max-intake` files and a field's
@@ -313,7 +317,7 @@ bound get no difference, association or packet; the field's chain
 starts at the bound.
 
 A field that, once the intake has drained, has fewer than `min_frames`
-official rows before the bound, none at all included, gets no
+official rows before the bound that overlap it, none at all included, gets no
 reference. The pass reports it under `held`, with the field and the
 reason, on every pass. A held field does not keep the loop from
 draining, and the verb exits 0: the gap is recorded, with no timeout.