Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -41,22 +41,24 @@ kagent compiles each ActorTemplate to the sandbox class of the WorkerPool that h

Substrate's density model rests on one fact about agent workloads: an Actor spends most of its time idle, waiting on a person or a large language model (LLM) to respond, not actively computing. Substrate exploits that by suspending idle Actors and reclaiming their Worker, then resuming them on demand when traffic arrives. Suspending and resuming allows a WorkerPool to run far more Actors than it has Workers for at any given moment.

The following diagram traces an Actor through one suspend-and-resume cycle, and shows the second path that opens up once the resulting snapshot is tagged.
The following diagram traces an Actor through one suspend-and-resume cycle, and shows how a tag can pin the resulting snapshot.
</br></br>

{{< reuse "kagent-docs/snippets/snapshot-cycle-diagram.md" >}}

A **WorkerPool** keeps **Workers** running and ready, and one Worker hosts the **Actor** while its conversation is active. Suspending that Actor writes its full state to an immutable **ActorSnapshot** and frees the Worker that it was running on.

The diagram forks at that snapshot, because a snapshot serves two purposes.
The diagram reaches a tagged snapshot, because a snapshot serves two purposes.

- **Resume** restores the same Actor onto **any free Worker in the pool**, which is not necessarily the Worker that it ran on before. Because the snapshot captures the Actor's full state, the conversation continues from where it left off. Every idle agent takes this path.
- A **{{< gloss "Tag" >}}Tag{{< /gloss >}}** pins that snapshot, and a **New Actor** can be seeded from the tag at the moment that it is created. Resuming an existing Actor never goes through a tag.
- A **{{< gloss "Tag" >}}Tag{{< /gloss >}}** pins that snapshot so that Agent Substrate does not collect the snapshot. Resuming an existing Actor never goes through a tag.

A tag gives a snapshot a stable, human-meaningful name, so callers do not need to track Substrate's internal snapshot identity. A tag names one snapshot permanently, and only its visibility scope can change afterward. A tag also acts as a retention pin, so Substrate does not delete a snapshot while a tag still names it.

For example, an agent partway through a long incident investigation reaches a state worth keeping. Creating a [checkpoint]({{< link path="about/substrate-runtime/suspend-and-resume#checkpoints" >}}) tags the snapshot that the agent most recently suspended to, which holds that one snapshot in place while the agent carries on and writes newer ones. Without the tag, Substrate collects that snapshot once a newer one supersedes it.

Checkpoint forks are temporarily unavailable in the current kagent release. Because the snapshots that kagent takes on suspend include process memory, `ForkSession` rejects checkpoint forks with `FAILED_PRECONDITION` instead of restoring processes that still hold the source Session's identifiers.

<!-- REVIEW (performance reviewer): an earlier draft stated a 100 ms target at the ninety-fifth percentile, measured from traffic arrival to the moment the Actor can receive it. Removed pending confirmation of the approved measurement and whether it applies to kagent. Do not restore it as a latency guarantee. -->

<!-- REVIEW (runtime/storage reviewers): expand this section with pause versus suspend, golden snapshots, where snapshots are stored, and what resume restores, and extend the snapshot-cycle diagram to show those distinctions. Do not equate a filesystem snapshot with full execution-state restoration without confirmation. -->
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,14 @@ An {{< gloss "ActorTemplate" >}}ActorTemplate{{< /gloss >}}'s snapshot configura
- **`Full`**: Captures process memory, the root filesystem changes layered on top of the container image, and any attached durable volumes. A `Full` snapshot holds everything that is needed to resume the Actor hot, with its in-memory state intact.
- **`Data`**: Captures only the contents of attached durable volumes. Process memory and the rest of the root filesystem are discarded, which makes the snapshot much cheaper to write and store.

Scopes describe only what a snapshot captures, and they are configured per trigger. The `onPause` setting selects what a pause captures on the node, and `onCommit` selects what a suspend uploads to snapshot storage. What `onCommit` captures must be a subset of what `onPause` captures.
Scopes describe only what a snapshot captures. In the current Agent Substrate release, one snapshot scope applies to both pause snapshots on the node and suspend snapshots in snapshot storage.

A **DurableDir volume** is the per-Actor application data surface. Its contents are preserved by the `Data` scope, so they survive a suspend and resume cycle independently of process memory. How many volumes an ActorTemplate can declare depends on its sandbox class. A `microvm` template can declare several, because they are subdirectories of a single shared filesystem. A `gvisor` template is limited to one, until {{< gloss "gVisor" >}}gVisor{{< /gloss >}} accepts more than a single durable mount.

When an Actor resumes from a `Data`-scope snapshot, the ActorTemplate's `onResume.fromData` setting decides where the rest of the guest state comes from. The default is `ColdBoot`, which starts the containers fresh from the container image with the durable volume contents restored over them.

> [!NOTE]
> These scopes describe what Agent Substrate supports, not choices that you make. kagent compiles every ActorTemplate with the same snapshot configuration: `Full` on pause, `Data` on commit, `ColdBoot` on resume, and a single DurableDir volume named `data`. The only snapshot setting that you author is the storage location, on the Harness.
> These scopes describe what Agent Substrate supports, not choices that you make. kagent currently compiles every ActorTemplate with `Full` snapshots, so suspended Actors keep their process memory. The only snapshot setting that you author is the storage location on the Harness.

## Golden and per-Actor snapshots

Expand Down Expand Up @@ -88,4 +88,4 @@ Creating a checkpoint attaches an Agent Substrate {{< gloss "Tag" >}}Tag{{< /glo

A Session must be at a turn boundary to be checkpointed, because the turn boundary is captured. The request names the terminal task to pin. It fails when a newer turn has already started, and again while that turn's snapshot is still being written.

A checkpoint also records how far the conversation had advanced, and it lets you start a second Session from the state it pinned. That second Session, a {{< gloss "Fork" >}}fork{{< /gloss >}}, continues the conversation from the point that the checkpoint pinned, and new turns append only to the fork. A fork receives fresh conversation and task identifiers rather than inheriting the source's. To create a checkpoint and fork a Session from it, work through the [Agent Substrate example]({{< link path="examples/agent-substrate" >}}).
A checkpoint also records how far the conversation had advanced. Forking a checkpoint is temporarily unavailable because kagent's current checkpoints include process memory. `ForkSession` rejects these checkpoints with `FAILED_PRECONDITION` so that a restored Actor does not resume processes that still hold the source Session's identifiers. Forking returns when Agent Substrate lifecycle v2 lets kagent take `Data` snapshots on suspend again. To create and inspect a checkpoint, work through the [Agent Substrate example]({{< link path="examples/agent-substrate" >}}).
2 changes: 1 addition & 1 deletion docs-site/content/kagent/1.x/about/what-is-kagent.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ kagent 1.0 separates an agent's capabilities from its runtime, then runs the two
{{< feature-card title="Agent lifecycle via CRDs" desc="Define, version, and roll out Harnesses and AgentTemplates with kubectl and GitOps, the same workflow as every other workload." >}}
{{< feature-card title="Sandboxed by default" desc="Every Session runs on a Substrate Actor, isolated from the host kernel by a gVisor sandbox. Run untrusted, model-directed code safely." >}}
{{< feature-card title="Suspend and resume" desc="Idle Sessions suspend and free their compute, then resume on demand. Run far more agents than you have capacity for at any one moment." >}}
{{< feature-card title="Checkpoint and fork" desc="Pin a snapshot of a conversation at a turn boundary, then branch a second Session from it that keeps the revision it started with." >}}
{{< feature-card title="Checkpoints" desc="Pin a snapshot of a conversation at a turn boundary so that Agent Substrate keeps the state for inspection and retention." >}}
{{< feature-card title="Pluggable agent runtimes" desc="A Harness selects the engine behind an agent: kagent's own Go and Python engines, the Codex or Claude coding agents, or any image of your own that speaks kagent's A2A contract." >}}
{{< feature-card title="Tools over MCP" desc="Bind an agent to any Model Context Protocol (MCP) server with a RemoteMCPServer resource. An installation registers two servers already, including 124 tools for Kubernetes, Helm, Istio, and Argo Rollouts." >}}
{{< feature-card title="Agent tools" desc="Bind another AgentTemplate as a tool, so an agent can hand work to a specialist. A Shared binding nests that agent inside its parent's Actor, one level deep." >}}
Expand Down
29 changes: 10 additions & 19 deletions docs-site/content/kagent/1.x/agents/agent-memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,36 +26,27 @@ Memories are scoped to the agent and to the user who created them, and carry the
## Before you begin

> [!IMPORTANT]
> Memory requires an **external** PostgreSQL database with the [pgvector](https://github.com/pgvector/pgvector) extension installed. kagent's bundled database uses the stock `postgres` image, which does not ship pgvector, so it cannot store memories. Point kagent at your own database first.
> Memory requires an **external** PostgreSQL database with the [pgvector](https://github.com/pgvector/pgvector) extension installed. The development database that `kagent install` creates uses the stock `postgres` image, which does not ship pgvector, so the database cannot store memories. Point kagent at your own database first.

1. Store the connection string for your database in a Kubernetes Secret, in the same namespace as the kagent controller. The key that you choose becomes the file name that kagent reads, so `db-url` produces the path `/var/secrets/db-url`.
1. Store the connection string for your database in a Kubernetes Secret, in the same namespace as the kagent controller.
```bash
kubectl create secret generic my-postgres-url-secret -n kagent \
--from-literal=db-url='postgres://<username>:<password>@<host>:5432/<database>?sslmode=require'
--from-literal=connectionString='postgres://<username>:<password>@<host>:5432/<database>?sslmode=require'
```

2. Save the following values, which point kagent at your database, mount the Secret into the controller, and run the migration that creates the pgvector extension.
2. Save the following values, which point kagent at your database and run the migration that creates the pgvector extension.
```yaml
cat > kagent-memory-values.yaml <<EOF
database:
postgres:
# Path to a file holding the connection string, which takes precedence over url
urlFile: /var/secrets/db-url
# Secret holding the connection string.
connectionStringSecretRef:
name: my-postgres-url-secret
key: connectionString
# Use the login user directly. To assume a role instead, create the role in PostgreSQL and set its name here.
role: ""
# Runs the vector migration, which creates the pgvector extension and the memory tables
vectorEnabled: true
bundled:
# Use your own database instead of the bundled one
enabled: false
controller:
# Mounts the Secret so that the file at urlFile exists in the controller pod
volumes:
- name: db-secret
secret:
secretName: my-postgres-url-secret
volumeMounts:
- name: db-secret
mountPath: /var/secrets
readOnly: true
EOF
```

Expand Down
2 changes: 1 addition & 1 deletion docs-site/content/kagent/1.x/examples/a2a-byo.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,5 +251,5 @@ An agent of your own takes the same path as the example image, with two differen
{{< cards >}}
{{< card link=`{{< link path="agents/bring-your-own-agent" >}}` title="Bring your own agent" subtitle="Read the full A2A contract and the limits of the byo runtime." >}}
{{< card link=`{{< link path="examples/a2a-agents" >}}` title="Call an agent over A2A" subtitle="Talk to this Agent with grpcurl instead of the CLI." >}}
{{< card link=`{{< link path="examples/agent-substrate" >}}` title="Agent Substrate" subtitle="Watch the Actor behind your image suspend, checkpoint, and fork." >}}
{{< card link=`{{< link path="examples/agent-substrate" >}}` title="Agent Substrate" subtitle="Watch the Actor behind your image suspend and checkpoint." >}}
{{< /cards >}}
Loading