Skip to content

fix: judge a missing post-run sensor by whether it changed since completion - #176

Merged
dwsmith1983 merged 1 commit into
mainfrom
fix/postrun-missing-check
Nov 22, 2026
Merged

dwsmith1983 merged 1 commit into
mainfrom
fix/postrun-missing-check

Conversation

@dwsmith1983

Copy link
Copy Markdown
Owner

POST_RUN_SENSOR_MISSING decided whether a post-run sensor had arrived from its updatedAt, read only as epoch milliseconds. A sensor with an RFC3339 updatedAt (the format the docs use), epoch seconds, or none raised the alert although it had arrived. The check also ignored per-period keys such as row-count#2026-03-10, and skipped any date with no baseline row, so it never fired when the sensor did not exist at completion, the case the alert is for.

A sensor record is one row per run date, so a change to it after the run is the signal and when it changed does not matter.

  • Arrival. A post-run sensor has arrived when it exists and either differs from its entry in the completion-time baseline or had no entry (did not exist at completion). The whole stored record is compared, so a changed updatedAt counts like any other field; the time of the write is not otherwise used.
  • Stored form. The baseline entry went through one more store write and read than the current sensor, so DynamoDB set types (a Python set written by boto3) never compared equal and the alert never fired. The current sensor is now put through the same encoding before the comparison. If that fails, a warning names the pipeline and sensor key and the sensor counts as arrived.
  • Keys. <key>#YYYY-MM-DD and <key>#YYYYMMDD are matched the way the baseline capture matches them, and a date with no baseline row is checked.

Accepted limits, stated in the docs where they affect users:

  • A sensor rewritten with identical values has not arrived.
  • A sensor written between the run completing and the baseline capture counts as present at completion.
  • A sensor whose baseline capture failed (BASELINE_CAPTURE_FAILED) has no entry and counts as arrived.
  • A per-period row-count#<date> stands in for a plain row-count even when the plain one changed; with both date forms present, which one is used is not fixed.
  • A set-typed field is compared in the order DynamoDB returns it.
  • Only today's UTC daily key is checked, as before.

…letion

POST_RUN_SENSOR_MISSING decided arrival from the sensor's updatedAt and
read it only as epoch milliseconds, so an RFC3339 or epoch-seconds
timestamp, or none, raised the alert although the sensor had arrived.
It also ignored per-period keys and skipped any date without a baseline
row, so it never fired when the sensor did not exist at completion.

A post-run sensor has now arrived when it exists and either differs from
its entry in the completion-time baseline or had no entry. The current
sensor is put through the same store encoding as the baseline before
the comparison, so set-typed values compare as the baseline holds them.
Per-period keys are matched the way the baseline capture matches them,
and a date with no baseline row is checked.
@github-actions github-actions Bot added tests Test changes lambda Lambda handlers docs Documentation labels Nov 22, 2026
@dwsmith1983 dwsmith1983 self-assigned this Nov 22, 2026
@dwsmith1983
dwsmith1983 merged commit fc1efcf into main Nov 22, 2026
12 checks passed
@dwsmith1983
dwsmith1983 deleted the fix/postrun-missing-check branch November 22, 2026 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation lambda Lambda handlers tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant