Skip to content

fix(cli): reconcile provisional container exits - #3204

Open
shiju-nv wants to merge 1 commit into
NVIDIA:mainfrom
shiju-nv:fix/canonical-main-exit-race
Open

fix(cli): reconcile provisional container exits#3204
shiju-nv wants to merge 1 commit into
NVIDIA:mainfrom
shiju-nv:fix/canonical-main-exit-race

Conversation

@shiju-nv

@shiju-nv shiju-nv commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Treat a driver-reported ContainerExited without a canonical main-process result as provisional for a bounded five-second reconciliation window. This prevents sandbox create --no-keep from deleting a fast finite-command sandbox before the supervisor can publish its authoritative exit status, while preserving the concrete driver error when no supervisor result arrives.

Related Issue

No issue required: this is an obvious localized bug fix for the intermittent Release Dev Rust Docker E2E failure and completes the CLI side of the lifecycle reconciliation introduced by PR #3101.

Changes

  • Recognize Error sandboxes with Ready=False, reason ContainerExited, and no exit code as provisional during sandbox create.
  • Continue watching until the canonical main-process result arrives, the existing provisioning deadline expires, or a fixed five-second reconciliation deadline expires.
  • Keep the first reconciliation deadline fixed so repeated status events and logs cannot extend the wait.
  • Preserve immediate failure for other infrastructure errors and retain the concrete ContainerExited diagnostic when reconciliation expires.
  • Add deterministic integration coverage for both a later successful canonical result with --no-keep cleanup and a missing-result deadline.

Testing

  • mise run pre-commit passes
  • Unit tests added/updated
  • E2E tests added/updated (the existing Docker lifecycle test is the CI validation target)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (not applicable; no architecture or public API change)

Keep watching briefly when a compute driver reports ContainerExited without a canonical main-process result. This prevents --no-keep cleanup from deleting the sandbox before the supervisor publishes the authoritative exit status.

Signed-off-by: Shiju <shiju@nvidia.com>
@shiju-nv
shiju-nv force-pushed the fix/canonical-main-exit-race branch from ecb539f to 223c245 Compare September 5, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant