Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,9 @@ This section lists what is on `main` and not yet on npm.
- Slack and Discord signature checks accept a timestamp at most a minute ahead of local time (was five minutes either way) and read it in seconds only (a milliseconds value was taken as "now"); the HMAC `timestampHeader` of `WebhookAuth` keeps its seconds-or-milliseconds reading with the same one-minute future limit.
- `githubChannel({ deliveries })` takes a `DeliveryStore` (`seen(id)`), so `X-GitHub-Delivery` replay protection can be shared across replicas and restarts; the default stays in memory for an hour. See [GitHub](docs/channels.md).
- The `@modelcontextprotocol/sdk` peer range is now `^1.31.0` (was `^1.30.1`), which has the fix for GHSA-6qxp-vccf-f47h that MCP OAuth relies on.
- Approve-with-edits on a sub-agent's *grouped* approval validates the edited args against the tool's schema like a top-level or single sub-agent decision does: before, `snapshot.subagent` stored `decision.args` raw, so an invalid edit (e.g. a string where `z.number()` wants a number) ran the tool with garbage args; it now throws `LOUSHO_TOOL_ARGS_INVALID` and leaves the approval pending. (Eve2-F4)
- Aborting a run during a tool call no longer hangs `send()` / `stream()` / `approvals.resolve()` forever on a tool that ignores its `abortSignal`: the tool batch and the approved-tool wait of `resumeAfterApproval()` are raced against the run's signal the way the model call already was (Eve CORE-F15), the run ends `finishReason: 'aborted'` with a 'cancelled' result for calls still running (a result already landing as the signal fires is still kept), and the tool keeps running in the background. (Eve2-F5)
- A run that re-pauses on several approvals after a resume persists every pending approval id in its checkpoint: `markAwaitingApproval()` wrote only `approvalId`, so durable discovery (`pausedApprovalIds`, `agent.approvals.list()`) named 1 of N approvals; it now carries the group's `approvalIds` like `markGroupPending()` does. (Eve2-F6)

### Changed
- **Behavior change (CLI):** every `lousho` usage error exits 2 (an unknown command, an unknown flag, a missing or invalid value, an extra argument, a missing `--target`, a bad `lousho init` flag); 1 is left for a command that ran and failed. Before, only `eval` and `traces` exited 2 and the others 1. See [CLI](docs/cli.md).
Expand Down
7 changes: 5 additions & 2 deletions docs/runs.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,11 @@ How it behaves:
- The run does not wait for a provider that ignores the signal: the model
call is abandoned at the abort (its late reply, and anything it streams
afterwards, is dropped) and the run ends `'aborted'`. A tool that ignores
its `abortSignal` is waited for; the run then ends `'aborted'` without
another model call. `defineTool({ timeoutMs })` bounds a tool's call: one
its `abortSignal` is not waited for either: the run ends `'aborted'`
without another model call, the still-running call gets a 'cancelled'
result, and the tool keeps running in the background (a result already
landing as the signal fires is still kept). `defineTool({ timeoutMs })`
bounds a tool's call: one
still running then fails with a tool error the model sees, and a run
aborted while such a call runs stops waiting for it.
- `result.abortReason` says why: the signal's reason as `{ name, message }`,
Expand Down
7 changes: 5 additions & 2 deletions llms-full.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14855,8 +14855,11 @@ How it behaves:
- The run does not wait for a provider that ignores the signal: the model
call is abandoned at the abort (its late reply, and anything it streams
afterwards, is dropped) and the run ends `'aborted'`. A tool that ignores
its `abortSignal` is waited for; the run then ends `'aborted'` without
another model call. `defineTool({ timeoutMs })` bounds a tool's call: one
its `abortSignal` is not waited for either: the run ends `'aborted'`
without another model call, the still-running call gets a 'cancelled'
result, and the tool keeps running in the background (a result already
landing as the signal fires is still kept). `defineTool({ timeoutMs })`
bounds a tool's call: one
still running then fails with a tool error the model sees, and a run
aborted while such a call runs stops waiting for it.
- `result.abortReason` says why: the signal's reason as `{ name, message }`,
Expand Down
4 changes: 3 additions & 1 deletion src/execution/AgentExecutor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1537,7 +1537,9 @@ export class AgentExecutor {
persist: () => saveStepCheckpoint(options, state),
},
// LOU-V10: a steer stops the batch from starting more calls; running ones finish.
withSteerSignal(options.signal, options.inputQueue?.startBatch())
// Eve2-F5: the run's signal also ends the wait for a tool that ignores it.
withSteerSignal(options.signal, options.inputQueue?.startBatch()),
options.signal
);
options.inputQueue?.endPhase();

Expand Down
59 changes: 59 additions & 0 deletions src/execution/abortEdges.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,65 @@ describe('abort with a provider that ignores the signal (Eve CORE-F15)', () => {
expect(result.finishReason).toBe('aborted');
expect(model.calls).toHaveLength(1);
});

it('a tool that never settles does not hold an aborted run open (Eve2-F5)', async () => {
const controller = new AbortController();
let release!: (value: string) => void;
const wedged = defineTool({
name: 'wedged',
description: 'wedged',
input: z.object({}),
execute: () => {
controller.abort();
return new Promise<string>((resolve) => {
release = resolve;
});
},
});
const model = mockModel([{ toolCalls: [{ name: 'wedged' }] }, 'done']);
const started = Date.now();
const result = await createAgent({ provider: model, tools: [wedged] }).send('go', { signal: controller.signal });

expect(Date.now() - started).toBeLessThan(1500);
expect(result.finishReason).toBe('aborted');
expect(result.abortReason?.name).toBe('AbortError');
expect(model.calls).toHaveLength(1);
// The transcript stays well-formed: the unanswered call has a cancelled result.
const toolMessages = result.messages.filter((m) => m.role === 'tool');
expect(toolMessages).toHaveLength(1);
expect(JSON.parse(String(toolMessages[0].content))).toMatchObject({ kind: 'not-run' });

// A wedged tool that settles after the run ended is not recorded on top of it.
release('too late');
await new Promise((resolve) => setTimeout(resolve, 20));
expect(result.messages.filter((m) => m.role === 'tool')).toHaveLength(1);
});

it('a wedged approved tool does not hold a resume open on abort (Eve2-F5)', async () => {
const controller = new AbortController();
const wedged = defineTool({
name: 'wedged',
description: 'wedged',
input: z.object({}),
needsApproval: true,
execute: () => {
controller.abort();
return new Promise<string>(() => {});
},
});
const agent = createAgent({ provider: mockModel([{ toolCalls: [{ name: 'wedged' }] }, 'done']), tools: [wedged] });
const paused = await agent.send('go');
expect(paused.finishReason).toBe('awaiting-approval');

const started = Date.now();
const resumed = await agent.approvals.resolve({ id: paused.approvalId!, approved: true }, { signal: controller.signal });

expect(Date.now() - started).toBeLessThan(1500);
expect(resumed.finishReason).toBe('aborted');
expect(resumed.abortReason?.name).toBe('AbortError');
// The decided call still has a result, so the transcript stays well-formed.
expect(resumed.messages.some((m) => m.role === 'tool')).toBe(true);
});
});

describe('mockModel honours the signal (Eve CORE-F15)', () => {
Expand Down
47 changes: 47 additions & 0 deletions src/execution/approvalResumeDurability.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,53 @@ describe('approval resume durability', () => {
await expect(agent.approvals.resolve({ id: paused.approvalId!, approved: true })).rejects.toMatchObject({ code: 'LOUSHO_APPROVAL_NOT_FOUND' });
});

it("a resumed run that pauses again on a step's several calls records every pending approval id (Eve2-F6)", async () => {
const store = memoryStore();
const log: string[] = [];
const make = (name: string) =>
defineTool({
name,
description: name,
input: z.object({ n: z.number() }),
needsApproval: true,
execute: async ({ n }) => {
log.push(`${name}(${n})`);
return 'ok';
},
});
const first = {
toolCalls: [
{ name: 'wire_money', args: { n: 1 }, id: 'call_wire' },
{ name: 'delete_db', args: { n: 2 }, id: 'call_delete' },
],
};
const second = {
toolCalls: [
{ name: 'wire_money', args: { n: 3 }, id: 'call_wire2' },
{ name: 'delete_db', args: { n: 4 }, id: 'call_delete2' },
],
};
const worker = createAgent({ provider: mockModel([first, second, 'worker done']), tools: [make('wire_money'), make('delete_db')], description: 'Does risky things' });
const lead = createAgent({
provider: mockModel([{ toolCalls: [{ name: 'task', args: { agent: 'worker', prompt: 'do it', description: 'risky' }, id: 'call_task' }] }, 'lead done']),
subagents: { worker },
store,
});
const session = lead.session({ id: 's1' });
const paused = await session.send('go');
expect(paused.approvalIds).toHaveLength(2);

await lead.approvals.resolve({ id: paused.approvalIds![0], approved: true });
const repaused = await lead.approvals.resolve({ id: paused.approvalIds![1], approved: true });

// The sub-agent's next step paused on two calls too: the durable reader sees both, not just the first.
expect(repaused.finishReason).toBe('awaiting-approval');
expect(repaused.approvalIds).toHaveLength(2);
const pending = await session.pending();
expect(pending).toMatchObject({ status: 'awaiting-approval', approvalId: repaused.approvalIds![0], approvalIds: repaused.approvalIds });
expect(log).toEqual(['wire_money(1)', 'delete_db(2)']);
});

it("discardPending() withdraws the dropped turn's approvals", async () => {
const store = memoryStore();
const refund = defineTool({ name: 'refund', description: 'refund', input: z.object({}), needsApproval: true, execute: async () => 'refunded' });
Expand Down
1 change: 1 addition & 0 deletions src/execution/generateStep.ts
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,7 @@ export interface GeneratedStep {
/**
* LOU-V10: `call`, or its rejection with `signal`'s reason as soon as
* `signal` aborts - a provider that ignores the signal is not waited for.
* `call` keeps running either way.
*/
function abortable<T>(call: Promise<T>, signal: AbortSignal | undefined): Promise<T> {
if (!signal) return call;
Expand Down
81 changes: 72 additions & 9 deletions src/execution/resume.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import {
withGroupDecisions,
} from './ApprovalGate';
import { validateToolArguments } from './toolArgsValidation';
import { TOOL_ABORT_GRACE_MS } from './toolBatch';
import { AgentExecutor, ExecuteOptions, ExecutionResult, extendRunOptions } from './AgentExecutor';
import { observeRun, partialSink, runEventsOf, streamResumed, type AgentRun, type ToolSettled } from './agentRun';
import { Checkpoint, CheckpointStore, RUN_CONFIG_KEY, pausedApprovalIds } from './checkpoint';
Expand Down Expand Up @@ -549,7 +550,14 @@ async function decideGroupMember(
for (const member of group) if (member.decision) decided.set(member.pending.id, member.decision);
decided.set(pending.id, mine);
const waiting = await claimWaitingSiblings(group, decided, approvalStore, pending.id);
if (waiting.length === 0) return { record: { pending, snapshot: withGroupDecisions(snapshot, decided) } };
if (waiting.length === 0) {
// Eve2-F4: a decision made on a sub-agent's record stored its edited
// args unchecked (the lead's registry holds no such tool) - the
// sub-agent's own resume is where they can be validated, before any
// call of the step runs.
await validateGroupArgs(group, decided, toolRegistry);
return { record: { pending, snapshot: withGroupDecisions(snapshot, decided) } };
}
for (const sibling of waiting) await approvalStore.save(sibling.pending, withGroupDecisions(sibling.snapshot, decided));
return { paused: groupPaused(snapshot, waiting) };
} catch (error) {
Expand Down Expand Up @@ -591,6 +599,29 @@ function adoptDecisions(snapshot: ExecutionSnapshot, decided: Map<string, GroupD
for (const other of snapshot.approvalGroup ?? []) if (other.decision && !decided.has(other.pending.id)) decided.set(other.pending.id, other.decision);
}

/**
* Eve2-F4: the edited `args` of every approved decision in `decided`, checked
* against its tool's schema before the group runs - a decision recorded
* without validation (a sub-agent's, whose lead could not check them against
* the sub-agent's tools) is caught here, in the run that owns the registry.
* The parsed args replace the decision's, exactly like withEditedArgs() does
* for a single call. Throws `LOUSHO_TOOL_ARGS_INVALID`; the caller puts the
* claimed record back, so the approval stays pending.
*/
async function validateGroupArgs(
group: ApprovalGroupMember[],
decided: Map<string, GroupDecision>,
toolRegistry: ToolRegistry
): Promise<void> {
for (const member of group) {
const decision = decided.get(member.pending.id);
if (!decision?.approved || decision.args === undefined) continue;
const toolDesc = toolRegistry.get(member.pending.toolName);
if (!toolDesc) continue;
decided.set(member.pending.id, { ...decision, args: (await validateToolArguments(member.pending.toolName, toolDesc, decision.args)) as Record<string, unknown> });
}
}

/** Eve TOOLS-F12: the run's result while the `waiting` calls of its step are undecided. */
function groupPaused(snapshot: ExecutionSnapshot, waiting: ResolvedApproval[]): ExecutionResult {
const ids = waiting.map((sibling) => sibling.pending.id);
Expand Down Expand Up @@ -1306,6 +1337,8 @@ async function markAwaitingApproval(
businessState,
status: 'awaiting-approval',
approvalId: paused.approvalId,
// Eve2-F6: a re-pause on several of a step's calls keeps every pending id, like markGroupPending() does.
...(paused.approvalIds !== undefined && paused.approvalIds.length > 1 && { approvalIds: paused.approvalIds }),
agentFingerprint: snapshot.agentFingerprint,
runConfig: snapshot.agent.metadata?.[RUN_CONFIG_KEY],
...(snapshot.principal && { principal: snapshot.principal }),
Expand Down Expand Up @@ -1439,6 +1472,29 @@ interface SettledCall {
replacedByHook?: string;
}

/**
* Eve2-F5: like `abortable()` (a tool that ignores the signal does not hold
* the run open forever), except a settlement landing while the run unwinds
* still wins - the abort side drains for TOOL_ABORT_GRACE_MS first, so a
* tool released just as the run aborted keeps its real result (B4), exactly
* like ToolBatch.settled()'s drained wait.
*/
function abortableToolCall<T>(call: Promise<T>, signal: AbortSignal | undefined): Promise<T> {
if (!signal) return call;
return new Promise<T>((resolve, reject) => {
let timer: ReturnType<typeof setTimeout> | undefined;
const onAbort = () => {
timer = setTimeout(() => reject(signal.reason), TOOL_ABORT_GRACE_MS);
};
signal.addEventListener('abort', onAbort, { once: true });
if (signal.aborted) onAbort();
call.then(resolve, reject).finally(() => {
signal.removeEventListener('abort', onAbort);
clearTimeout(timer);
});
});
}

/**
* Executes the approved tool itself. The try/catch here handles ONLY the
* tool's own execution failure - NOT the postToolCall hook call that
Expand Down Expand Up @@ -1470,14 +1526,21 @@ async function executeApprovedTool(
// sandbox seam the way it previously did.
const sandbox = executeOptions.sandbox ?? NoopSandbox;
return {
result: await executeToolWithSandboxGuard(
pending.toolName,
toolDesc,
args,
sandbox,
executeOptions.signal,
{ toolCallId: pending.toolCallId, messages, approval, sessionId, principal: executeOptions.principal, tokens: executeOptions.tokens, onPartial: partialsOf(pending, executeOptions) },
scope
// Eve2-F5: raced against the run's signal - a tool that ignores it
// would otherwise hold the resume open forever. The rejection lands
// in the catch below like the tool's own abort error (the tool keeps
// running, documented for ExecuteOptions.signal).
result: await abortableToolCall(
executeToolWithSandboxGuard(
pending.toolName,
toolDesc,
args,
sandbox,
executeOptions.signal,
{ toolCallId: pending.toolCallId, messages, approval, sessionId, principal: executeOptions.principal, tokens: executeOptions.tokens, onPartial: partialsOf(pending, executeOptions) },
scope
),
executeOptions.signal
),
};
} catch (error) {
Expand Down
Loading
Loading