Skip to content

fix: refuse a rerun while the date's run is in progress - #171

Merged
dwsmith1983 merged 1 commit into
mainfrom
fix/rerun-refuse-running
Nov 22, 2026
Merged

dwsmith1983 merged 1 commit into
mainfrom
fix/rerun-refuse-running

Conversation

@dwsmith1983

Copy link
Copy Markdown
Owner

A rerun request for a date whose run was still in progress reset the trigger lock and started a second execution beside the first. Both then launched the job, and whichever finished last decided whether the date read as COMPLETED or FAILED_FINAL, with both JOB_COMPLETED and RETRY_EXHAUSTED possible for one date.

  • Refused. A request for a RUNNING date, manual or automatic, ends as RERUN_REJECTED with a run is in progress for <date> and a rerun-rejected job row. No budget is spent and the trigger row is untouched. The calendar and budget checks still come first and the circuit breaker after.
  • Atomic. ResetTriggerLock now requires the row not to be RUNNING, so a run that takes the date between the read and the reset is also refused rather than doubled.
  • Log. The rejection log carries the request's reason, so an automatic request refused for a running date can be told from a manual one.
  • Dry run. DRY_RUN_RERUN_REJECTED reports the same refusal with reason run in progress.
  • Dead runs. A date whose execution died while its row stayed RUNNING is now closed only by the watchdog's stale-trigger check: once the lock TTL (sfn_timeout_seconds plus 30 minutes, 4h30m by default) has passed since the latest attempt, it publishes SFN_TIMEOUT and sets the row to FAILED_FINAL, and the request can be written again. The docs say so; there is no override.

name_length_test.go reran a RUNNING date, which is the case this refuses, so it now starts from FAILED_FINAL. Two ResetTriggerLock tests on a mock that only matched the old condition string are replaced by a test on the in-memory store that covers RUNNING, COMPLETED, FAILED_FINAL and a missing row.

A rerun request for a RUNNING date reset the trigger lock and started a
second execution beside the first, and whichever finished last decided
the date's status. Such a request, manual or automatic, now ends as
RERUN_REJECTED ("a run is in progress for <date>") with no budget spent,
and ResetTriggerLock refuses a RUNNING row so a run that takes the date
mid-request is refused too. Dry run reports the same refusal. A date
whose execution died stays RUNNING until the watchdog's stale check
closes it with SFN_TIMEOUT, which the docs now describe.
@github-actions github-actions Bot added tests Test changes lambda Lambda handlers docs Documentation labels Nov 22, 2026
@dwsmith1983 dwsmith1983 self-assigned this Nov 22, 2026
@dwsmith1983
dwsmith1983 merged commit d551581 into main Nov 22, 2026
12 checks passed
@dwsmith1983
dwsmith1983 deleted the fix/rerun-refuse-running branch November 22, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation lambda Lambda handlers tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant