Skip to content

Report the turn boundary to Runners - #202

Merged
vitramir merged 4 commits into
mainfrom
hibernation-idle-report
Oct 31, 2026
Merged

vitramir merged 4 commits into
mainfrom
hibernation-idle-report

Conversation

@vitramir

Copy link
Copy Markdown
Contributor

The agynd half of workload hibernation's turn boundary, as specified in agynd-cli.md § 7 Turn Boundary (agynio/architecture#192). The wake path (§ 8) is #201.

A turn used to end in silence: the keepalive stops when the agent CLI goes quiet, and Runners learned of it only when the activity sweep found the workload past its keepalive grace. Anything that waits on an idle agent — hibernation first among them — waited that long too.

After the turn's inbox items are acked, agynd now calls RunnersGateway.ReportWorkloadIdle with the workload's id. The call flips agent_state to idle at once, and is the moment the Orchestrator may hibernate the workload: between one turn and the next, if the inbox is still empty when it looks.

  • After the ack, never before. Every handler (codex, claude, agn) acks the item as its last act and returns nil only once it has; the report sits behind that return in syncMessages, so the ack stays the last word of a turn and a workload is never frozen between finishing an item and acknowledging it. A turn that fails — the ack included — reports nothing.
  • Never fatal. A report that never arrives costs nothing but latency, because the sweep reaches the same state on its own schedule. A failure is logged; the turn has already succeeded and stays that way. Five-second timeout, as the touch has.
  • Not in holder mode. A sandbox runs no turn, so there is no report; its idleness is the Terminal Proxy's to observe.

The RPC landed in agynio/api#199 and is published on buf.build/agynio/api; buf generate buf.build/agynio/api --path agynio/api/gateway/v1 --include-imports, which CI and the Dockerfile run, picks it up. The repository commits no generated code, so there is none here. (buf dep update has nothing to do in this repository — it holds no .proto files of its own — and was not needed.) The e2e stub gateway answers the new call so the daemon's e2e logs stay clean.

Tests cover the ordering (ack, then report, with the workload id), a failing report leaving the turn successful, a failing ack producing no report, holder mode producing none, and the platform client. go build, go vet (also -tags e2e), gofmt and go test ./... pass. Under -race the daemon package reports two pre-existing races between shells_test.go's stubs and the shell-title refresh goroutine, which this change does not touch; they reproduce with the shell tests alone (-run Shell -count=10), with or without this change.

A turn used to end in silence. The keepalive stops when the agent CLI
goes quiet, and Runners learned of it only when the activity sweep
found the workload past its keepalive grace, which is how long any
decision that waits on an idle agent -- hibernation first among them --
had to wait as well.

After the turn's inbox items are acked, agynd now calls
ReportWorkloadIdle. The call flips the workload's agent_state to idle
at once, and is the moment the Orchestrator may hibernate the workload:
between one turn and the next, if the inbox is still empty when it
looks. The report follows the ack and never precedes it, so the ack is
the last word of a turn and a workload is never frozen between finishing
an item and acknowledging it. Every handler acks as its last act and
returns nil only once it has; the report sits behind that return, where
it is the same for codex, claude and agn.

A report that never arrives costs nothing but latency, because the
sweep reaches the same state on its own schedule, so a failure is
logged and never fatal. In holder mode there is no turn, so there is no
report; a sandbox's idleness is the Terminal Proxy's to observe.

The RPC is on the Gateway's RunnersGateway since agynio/api#199. The
repository commits no generated code; CI generates the bindings from
the BSR module, as before.
A hibernated workload resumes with every connection it held dead on the
far side, which cannot tell a frozen peer from a vanished one. The
Notifications stream is the one that would wait on its dead peer until
TCP gave up, so on a wake the subscription is dropped and reopened, the
inbox is read once -- what arrived during the sleep is in it, and the
stream that would have announced it died with the sleep -- and a turn
in flight is touched now rather than at the next tick. Nothing is
replayed to the agent CLI.

Two things say the workload has been woken. The runner sends SIGCONT to
the container's init once the guest's clock is set, which a process
without a handler never sees, so agynd asks for it. And the clock says
so on its own: the runner sets the wall clock after every restore while
the monotonic clock slept through the pause, so the keepalive compares
the two between ticks, and a wall clock more than a tick ahead is a
restore. The signal makes the wake prompt; the clock makes it certain.

The keepalive loop handles the wake rather than Run, which is inside
the turn for as long as a turn takes. The runner signals twice, a few
seconds apart, and the clock confirms what the signal announced; a
repeat within two ticks of a handled wake finds it handled and does
nothing, and one pending inbox read is as good as any number. The two
ticks are measured on the wall clock, because the monotonic one does
not move across a sleep and two wakes with a sleep between them must
both count.

A wake that lands while the subscriber is waiting out a failure cuts
the wait short: the wait was for the failure, and a wake is not one.
@vitramir
vitramir merged commit f409c12 into main Oct 31, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant