diff --git a/docs-site/content/kagent/1.x/operations/operational-considerations.md b/docs-site/content/kagent/1.x/operations/operational-considerations.md index d9f557b7..c049d8f7 100644 --- a/docs-site/content/kagent/1.x/operations/operational-considerations.md +++ b/docs-site/content/kagent/1.x/operations/operational-considerations.md @@ -97,6 +97,23 @@ Leader election keeps the replicas from conflicting. One replica holds a Kuberne > [!IMPORTANT] > If an installation runs on the development database, extra controller replicas do not make the installation highly available. That database is a single pod backed by one PersistentVolumeClaim, so losing it stops every replica at once. Run an external PostgreSQL instance before counting controller replicas toward availability. +## Recover missed Session pauses {#session-quiescence-recovery} + +An idle Session's {{< gloss "Actor" >}}Actor{{< /gloss >}} is paused or suspended so that the capacity it holds returns to its pool. The controller starts this within moments of a turn finishing. The setting in this section governs only how quickly the controller recovers when it misses that moment. + +When a task settles, the controller commits the task history to PostgreSQL. It then signals itself to pause or suspend the Session's Actor. The signal is local to one controller process, so a controller that exits between the commit and the signal leaves the work undone. Another replica finds the record on its next recovery scan, which runs every `1m` by default. + +This work is not leader-only, unlike the reconciliation that [Run multiple controller replicas](#run-multiple-controller-replicas) describes. Every replica claims from the same PostgreSQL queue, and PostgreSQL grants each record to one claimant, so no two replicas pause the same Session. Each replica runs four claims at a time, which bounds how much of this work one replica takes on. + +Set `KAGENT_SESSION_QUIESCENCE_POLL_INTERVAL` to change the recovery scan interval. A longer interval delays a pause, a suspend, and the {{< gloss "Snapshot" >}}snapshot{{< /gloss >}} that follows one, but only after a missed signal. A shorter interval scans more often. The value must be a positive Go duration, and the controller retries blocked work and database errors within one second regardless of the interval. + +```yaml +controller: + env: + - name: KAGENT_SESSION_QUIESCENCE_POLL_INTERVAL + value: 30s +``` + ## Node operations A {{< gloss "Worker" >}}Worker{{< /gloss >}} is an ordinary Kubernetes pod, so every node event reaches the {{< gloss "Actor" >}}Actors{{< /gloss >}} that this Worker hosts. What the event costs depends on what each Actor was doing when the node went away.