-
Notifications
You must be signed in to change notification settings - Fork 0
D4: Add bounded production agent adapters #47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mchwang
wants to merge
44
commits into
codex/agent-isolation-d3
Choose a base branch
from
codex/agent-adapters-d4
base: codex/agent-isolation-d3
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
44 commits
Select commit
Hold shift + click to select a range
489066e
Add D1 invocation contract and independent task clones
mchwang d110d81
Harden clone containment traversal and deadline validation
mchwang 17d71d0
Add D2 pinned restricted agent containers
mchwang 124800b
Harden D2 container validation and Linux setup
mchwang e02995f
Close D2 profile and validation trust gaps
mchwang 4b964a9
Require exact D2 capability and mount profiles
mchwang 0845b6d
Trust D2 helper images and settle cleanup
mchwang 5ccd2aa
Bind D2 resources to their owners
mchwang b975c7f
Close D2 namespace and cleanup gaps
mchwang aa1bdbc
Allow clone ownership regression to settle
mchwang c438462
Pin the D2 Codex state path
mchwang a4bd301
Seal D2 inputs and complete resource checks
mchwang 353efb0
Add vendor-only egress and phase policy
mchwang 1579033
Enforce invocation-scoped agent policy
mchwang 945e796
Bind adapters and proxy checks to invocation
mchwang 5494bb2
Block agent DNS and harden proxy validation
mchwang 7acc50d
Close remaining network lifecycle gaps
mchwang 981e831
Stabilize live Claude marker probe
mchwang 682e6ae
Add bounded production agent adapters
mchwang 5dd3a85
Harden adapter capture settlement
mchwang 1ab4dab
Pin adapter output to dedicated tmpfs
mchwang 4c68faf
Close remaining adapter lifecycle races
mchwang 3e57ff7
Validate pinned output identities
mchwang 080499c
Make cleanup and decode settlement retryable
mchwang 58a0d42
Abort and await bounded adapter capture
mchwang 38b020d
Retain adapter cleanup ownership
mchwang 9a8ff6a
Allow loaded Docker cleanup observation
mchwang 7c4e098
Secure deferred output acknowledgement
mchwang abffe7b
Retain colliding cleanup recovery
mchwang c68aad2
Preserve adapter setup ownership
mchwang 796f3ca
Serialize agent isolation CI tests
mchwang 7b25c0e
Bound cancellation with monotonic deadlines
mchwang 066e92e
Carry invocation ownership through settlement
mchwang 5b4de85
Bound failed network setup cleanup
mchwang 267018d
Guard active profile and close deadline
mchwang 22f362e
Apply adapter timeout across setup
mchwang 0c91f87
Bound adapter cleanup and final stderr
mchwang da1976e
Keep decoder settlement timers alive
mchwang ace826c
Retain recovery profile ownership
mchwang 0a0ce54
Revalidate container at launch boundary
mchwang 02ce3e1
Validate profile capability before cleanup
mchwang 158e07d
Preserve cleanup cancellation reasons
mchwang 5506d65
Update cleanup cancellation regression
mchwang a371bc7
Authenticate profiles at disposal boundary
mchwang File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| name: Agent isolation | ||
| on: | ||
| push: | ||
| branches: ['codex/agent-isolation-d2'] | ||
| pull_request: | ||
| paths: | ||
| - 'agents/**' | ||
| - 'git/clone.ts' | ||
| - 'test/agent-*.test.ts' | ||
| - '.github/workflows/agent-isolation.yml' | ||
| permissions: | ||
| contents: read | ||
| jobs: | ||
| real-docker: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '26.7.0' | ||
| cache: npm | ||
| - run: npm ci --ignore-scripts | ||
| - run: npm run typecheck | ||
| - run: npx vitest run --no-file-parallelism test/agent-contract.test.ts test/agent-clone.test.ts test/agent-container.test.ts test/agent-network.test.ts test/agent-policy.test.ts test/agent-adapter.test.ts test/agent-supervisor.test.ts |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| import type { InvocationHandle } from '../contract.ts'; | ||
| import { createContainerProfile, ProfileCreationCleanupError } from '../container/profile.ts'; | ||
| import { createVendorNetwork, removeVendorNetwork, VendorNetworkCreationCleanupError, | ||
| type VendorNetwork } from '../network/network.ts'; | ||
| import { createClaudeCommand, createPhasePolicy } from '../policy.ts'; | ||
| import { retainNetworkCleanup, retainSetupCleanup, startProfileInvocation } from './supervisor.ts'; | ||
| import { createAdapterInvocationBudget, type AgentAdapterOptions, type AgentAdapterRequest } from './types.ts'; | ||
|
|
||
| export function parseClaudeOutput(raw: Buffer): { text: string; providerFailed: boolean } { | ||
| const envelope = JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(raw)) as | ||
| { result?: unknown; is_error?: unknown }; | ||
| if (typeof envelope.result !== 'string' || typeof envelope.is_error !== 'boolean') | ||
| throw new Error('Claude returned a malformed output envelope.'); | ||
| return Object.freeze({ text: envelope.result, providerFailed: envelope.is_error }); | ||
| } | ||
|
|
||
| export function startClaudeInvocation(request: AgentAdapterRequest, | ||
| oauthToken: string, options: AgentAdapterOptions = {}): InvocationHandle { | ||
| if (!oauthToken || oauthToken.includes('\0')) throw new Error('Claude OAuth token is malformed.'); | ||
| const policy = createPhasePolicy(request.invocation); | ||
| const remaining = createAdapterInvocationBudget(request.invocation, options.timeoutMs); | ||
| let network: VendorNetwork; | ||
| try { network = createVendorNetwork(request.invocation, request.imageId, Math.min(60_000, remaining())); } | ||
| catch (error) { | ||
| if (error instanceof VendorNetworkCreationCleanupError) | ||
| return retainSetupCleanup(request.invocation, error.retryCleanup, error.startupError, error, | ||
| 'network creation cleanup'); | ||
| throw error; | ||
| } | ||
| try { | ||
| const profile = createContainerProfile({ ...request, policy, network, | ||
| command: createClaudeCommand(policy, request.prompt), claudeToken: oauthToken, | ||
| timeoutMs: Math.min(60_000, remaining()) }); | ||
| return startProfileInvocation(profile, { ...options, secrets: { CLAUDE_CODE_OAUTH_TOKEN: oauthToken }, | ||
| invocationBudget: remaining, | ||
| decode: (_profile, raw) => parseClaudeOutput(raw) }); | ||
| } catch (error) { | ||
| if (error instanceof ProfileCreationCleanupError) { | ||
| const retryCleanup = (networkTimeoutMs = 30_000) => { | ||
| const failures: unknown[] = []; | ||
| try { error.retryCleanup(); } catch (cleanupError) { failures.push(cleanupError); } | ||
| try { removeVendorNetwork(network, networkTimeoutMs); } catch (cleanupError) { failures.push(cleanupError); } | ||
| if (failures.length) throw new AggregateError(failures, 'Adapter setup cleanup did not settle.'); | ||
| }; | ||
| try { retryCleanup(Math.min(30_000, remaining())); } | ||
| catch (cleanupError) { return retainSetupCleanup(request.invocation, () => retryCleanup(), | ||
| error.startupError, cleanupError, 'profile and network cleanup'); } | ||
| throw error.startupError; | ||
| } | ||
| try { removeVendorNetwork(network, Math.min(30_000, remaining())); } | ||
| catch (cleanupError) { return retainNetworkCleanup(request.invocation, network, error, cleanupError); } | ||
| throw error; | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| import type { InvocationHandle } from '../contract.ts'; | ||
| import { createContainerProfile, ProfileCreationCleanupError } from '../container/profile.ts'; | ||
| import { createVendorNetwork, removeVendorNetwork, VendorNetworkCreationCleanupError, | ||
| type VendorNetwork } from '../network/network.ts'; | ||
| import { createCodexCommand, createPhasePolicy } from '../policy.ts'; | ||
| import { readBoundedContainerFile, retainNetworkCleanup, retainSetupCleanup, | ||
| startProfileInvocation } from './supervisor.ts'; | ||
| import { createAdapterInvocationBudget, type AgentAdapterOptions, type AgentAdapterRequest } from './types.ts'; | ||
|
|
||
| export const CODEX_OUTPUT_FILE = '/run/codeboost-output/final.txt'; | ||
|
|
||
| export async function readCodexOutput(container: string, maximumBytes: number, timeoutMs = 30_000, | ||
| signal?: AbortSignal) { | ||
| const output = await readBoundedContainerFile(container, CODEX_OUTPUT_FILE, maximumBytes, timeoutMs, signal); | ||
| const text = new TextDecoder('utf-8', { fatal: true }).decode(output); | ||
| return Object.freeze({ text, additionalBytes: output.length }); | ||
| } | ||
|
|
||
| export function startCodexInvocation(request: AgentAdapterRequest, | ||
| authFile: string, options: AgentAdapterOptions = {}): InvocationHandle { | ||
| if (!authFile || authFile.includes('\0')) throw new Error('Codex auth path is malformed.'); | ||
| const policy = createPhasePolicy(request.invocation); | ||
| const remaining = createAdapterInvocationBudget(request.invocation, options.timeoutMs); | ||
| let network: VendorNetwork; | ||
| try { network = createVendorNetwork(request.invocation, request.imageId, Math.min(60_000, remaining())); } | ||
| catch (error) { | ||
| if (error instanceof VendorNetworkCreationCleanupError) | ||
| return retainSetupCleanup(request.invocation, error.retryCleanup, error.startupError, error, | ||
| 'network creation cleanup'); | ||
| throw error; | ||
| } | ||
| try { | ||
| const profile = createContainerProfile({ ...request, policy, network, | ||
| command: createCodexCommand(policy, request.prompt), codexAuthFile: authFile, deferredOutput: true, | ||
| timeoutMs: Math.min(60_000, remaining()) }); | ||
| return startProfileInvocation(profile, { ...options, | ||
| invocationBudget: remaining, | ||
| decode: (current, _raw, maximum, timeoutMs, signal) => | ||
| readCodexOutput(current.name, maximum, timeoutMs, signal) }); | ||
| } catch (error) { | ||
| if (error instanceof ProfileCreationCleanupError) { | ||
| const retryCleanup = (networkTimeoutMs = 30_000) => { | ||
| const failures: unknown[] = []; | ||
| try { error.retryCleanup(); } catch (cleanupError) { failures.push(cleanupError); } | ||
| try { removeVendorNetwork(network, networkTimeoutMs); } catch (cleanupError) { failures.push(cleanupError); } | ||
| if (failures.length) throw new AggregateError(failures, 'Adapter setup cleanup did not settle.'); | ||
| }; | ||
| try { retryCleanup(Math.min(30_000, remaining())); } | ||
| catch (cleanupError) { return retainSetupCleanup(request.invocation, () => retryCleanup(), | ||
| error.startupError, cleanupError, 'profile and network cleanup'); } | ||
| throw error.startupError; | ||
| } | ||
| try { removeVendorNetwork(network, Math.min(30_000, remaining())); } | ||
| catch (cleanupError) { return retainNetworkCleanup(request.invocation, network, error, cleanupError); } | ||
| throw error; | ||
|
mchwang marked this conversation as resolved.
|
||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.