Skip to content

feat: log every failed attempt that will be retried - #2

Merged
klaidliadon merged 1 commit into
masterfrom
feat/log-retried-attempts
Oct 19, 2026
Merged

klaidliadon merged 1 commit into
masterfrom
feat/log-retried-attempts

Conversation

@klaidliadon

Copy link
Copy Markdown
Collaborator

Goal

An operator can see a task failing and retrying while it happens, instead of only after it runs out of attempts. Each failed attempt that will be retried now logs a warning with the attempt number, the budget, the backoff before the next try, and the error.

Problem it solves

  • Retries are invisible. Today the executor logs a task failure only when it parks the run: a permanent error, or the last attempt in the budget. A task that fails twice and then succeeds leaves no log line, and neither does one that is still working through its budget. The only trace is attempt_count and last_error on the task row.
  • Fault-injection tests have nothing to search for. A test that injects a transient fault and expects the task to retry can only check the database afterwards. With a warning per attempt, it can check the logs.

The warning fires once per failed attempt, for both the in-process backoff and the longer backoff that releases the claim. Parking keeps its existing alert-level line, so the last attempt logs once, not twice.

- only a parked task was logged, so retries were invisible until the
  budget ran out

Claude-Session: https://claude.ai/code/session_01JeE7nT7xaWppRm9ngjdD6B
@klaidliadon
klaidliadon requested a review from marino39 October 19, 2026 10:59
@klaidliadon
klaidliadon merged commit 8b8039a into master Oct 19, 2026
2 checks passed
@klaidliadon
klaidliadon deleted the feat/log-retried-attempts branch October 19, 2026 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants