Repository navigation
Add workload hibernation to the runner, runners and agents APIs - #199
Merged
Merged
Conversation
Runner: HibernateWorkload and WakeWorkload (the wake carries fresh image pull credentials), HIBERNATING / HIBERNATED / WAKING in the reported WorkloadStatus, and a typed status plus snapshot_size_bytes on InspectWorkloadResponse and WorkloadListItem. The inspect shape's status was a free-form string and the list item had none; "present when HIBERNATED" needs a status of the enum's type to refer to. Runners: the same three statuses on Workload.status; hibernated_at, woken_at and snapshot_size_bytes on Workload, accepted by UpdateWorkload; snapshot_size_bytes on ReportWorkloadState; WAKE_FAILED and SNAPSHOT_LOST failure reasons; ReportWorkloadIdle on RunnersService and RunnersGateway. Agents: hibernate_after and hibernation_ttl on Agent, Create and Update, shaped like idle_timeout; HIBERNATED on SandboxStatus. Request field names follow their neighbours rather than the spec's prose: the runner-side requests take workload_id like StopWorkloadRequest, and ReportWorkloadIdleRequest takes id like TouchWorkloadRequest.
|
The latest Buf updates on your PR. Results from workflow buf-pr / buf (pull_request).
|
The spec decided against a null meaning for hibernate_after and hibernation_ttl. Whether an agent's workloads are hibernated at all is its own boolean, defaulting to true, shaped like Environment.persistent_shells: plain on the read shape, optional on Create and Update so an unset field means the default and "leave it alone" respectively. The two durations stay shaped like idle_timeout.
This was referenced Oct 30, 2026
vitramir
added a commit
to agynio/agynd-cli
that referenced
this pull request
Oct 31, 2026
* Report the turn boundary to Runners A turn used to end in silence. The keepalive stops when the agent CLI goes quiet, and Runners learned of it only when the activity sweep found the workload past its keepalive grace, which is how long any decision that waits on an idle agent -- hibernation first among them -- had to wait as well. After the turn's inbox items are acked, agynd now calls ReportWorkloadIdle. The call flips the workload's agent_state to idle at once, and is the moment the Orchestrator may hibernate the workload: between one turn and the next, if the inbox is still empty when it looks. The report follows the ack and never precedes it, so the ack is the last word of a turn and a workload is never frozen between finishing an item and acknowledging it. Every handler acks as its last act and returns nil only once it has; the report sits behind that return, where it is the same for codex, claude and agn. A report that never arrives costs nothing but latency, because the sweep reaches the same state on its own schedule, so a failure is logged and never fatal. In holder mode there is no turn, so there is no report; a sandbox's idleness is the Terminal Proxy's to observe. The RPC is on the Gateway's RunnersGateway since agynio/api#199. The repository commits no generated code; CI generates the bindings from the BSR module, as before. * Reconnect when the workload wakes from hibernation (#201) A hibernated workload resumes with every connection it held dead on the far side, which cannot tell a frozen peer from a vanished one. The Notifications stream is the one that would wait on its dead peer until TCP gave up, so on a wake the subscription is dropped and reopened, the inbox is read once -- what arrived during the sleep is in it, and the stream that would have announced it died with the sleep -- and a turn in flight is touched now rather than at the next tick. Nothing is replayed to the agent CLI. Two things say the workload has been woken. The runner sends SIGCONT to the container's init once the guest's clock is set, which a process without a handler never sees, so agynd asks for it. And the clock says so on its own: the runner sets the wall clock after every restore while the monotonic clock slept through the pause, so the keepalive compares the two between ticks, and a wall clock more than a tick ahead is a restore. The signal makes the wake prompt; the clock makes it certain. The keepalive loop handles the wake rather than Run, which is inside the turn for as long as a turn takes. The runner signals twice, a few seconds apart, and the clock confirms what the signal announced; a repeat within two ticks of a handled wake finds it handled and does nothing, and one pending inbox read is as good as any number. The two ticks are measured on the wall clock, because the monotonic one does not move across a sleep and two wakes with a sleep between them must both count. A wake that lands while the subscriber is waiting out a failure cuts the wait short: the wait was for the failure, and a wake is not one. * Run the holder test with the wake channel Run now takes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Mirrors the interface surface of workload hibernation into the contracts so the vm-runner, Runners service, Orchestrator,
agyndand Agents service can be built against them.What changes
agynio/api/runner/v1HibernateWorkload(HibernateWorkloadRequest{workload_id})andWakeWorkload(WakeWorkloadRequest{workload_id, image_pull_credentials});Unimplementedon a runner that does not advertisehibernateWorkloadStatusgainsHIBERNATING,HIBERNATED,WAKINGInspectWorkloadResponseandWorkloadListItemgain a typedstatusandsnapshot_size_bytes(present whenHIBERNATED)agynio/api/runners/v1(andRunnersGateway)WorkloadStatusgains the same three valuesWorkloadgainshibernated_at,woken_at(nullable timestamps) andsnapshot_size_bytes(nullable int64);UpdateWorkloadaccepts all threeReportWorkloadStatecarriessnapshot_size_bytesWorkloadFailureReasongainsWAKE_FAILEDandSNAPSHOT_LOSTReportWorkloadIdle(ReportWorkloadIdleRequest{id})onRunnersServiceandRunnersGatewayagynio/api/agents/v1Agent,CreateAgentRequestandUpdateAgentRequestgainhibernate(bool, default true;optionalon Create and Update likepersistent_shells),hibernate_afterandhibernation_ttl(duration strings shaped likeidle_timeout;hibernation_ttlminimum1h)SandboxStatusgainsHIBERNATEDChoices worth a look
Request fields follow their neighbours rather than the spec's prose. The runner-side requests take
workload_id, likeStopWorkloadRequestand every other request inrunner.protothat names a workload;ReportWorkloadIdleRequesttakesid, likeTouchWorkloadRequest, the call it sits beside.InspectWorkloadResponse.statusandWorkloadListItem.statusare not in the spec's field list but are implied by it: the spec says the runner reportsstatusonInspectWorkloadandListWorkloadsfrom the enum vocabulary, and the inspect shape's existingstate_statusis a free-form string while the list item had no status at all. Without a typed field, "present whenHIBERNATED" has nothing to refer to.Notes
buf lint,buf breakingagainstmain,buf buildandbuf formatall pass.