Repository navigation
Description
Before submitting
- I searched existing issues and did not find a duplicate.
- I included enough detail to reproduce or investigate the problem.
Area
apps/server
Steps to reproduce
Use a clean checkout of the commit in Version or commit with the repository's locked dependencies installed. Save the script below as repro.sh (anywhere outside the checkout) and run it from the checkout root, passing the checkout root as its argument:
tmp="$(mktemp -d)"
# save the script below as "$tmp/repro.sh"
REPRO_TMP_PARENT="$tmp" sh "$tmp/repro.sh" "$PWD"The script builds a temporary Vite+ test outside the checkout, with private HOME, TMPDIR and T3CODE_HOME, one test worker and no real provider account, and removes its fixture on exit. It reuses the repository's Codex replay testkit (CodexAdapterV2.testkit.ts, ProviderReplayHarness.ts) with the real CodexAdapterV2, provider-event ingestion, orchestrator, SQLite event/projection services and the production limitRecoveryCommand decision; only the external Codex app-server transport is replayed. For each scenario it:
- Dispatches an ordinary thread message, replaying the repository's simple Codex transcript preamble.
- Sends
account/rateLimits/updatedforlimitId: "codex",planType: "team", primary (300-minute window) at 0% or 100%, secondary (weekly) at 14%, and eitherindividualLimit: { limit: "0", used: "0.0", remainingPercent: 0, resetsAt: 1796083200 }or noindividualLimit. - Sends
turn/completedwith statusfailedandcodexErrorInfo: "usageLimitExceeded"(orcompletedfor the success control). When the server should read the account, the transcript expectsaccount/rateLimits/readand answers it with primary at 100%. - Arms recovery through a real
thread.metadata.updatecommand with the stored reset andautoResume: true. - Asks the production recovery decision what it would do 1 ms after the plan (primary) reset and 1 ms after the credit reset.
The six scenarios are: plan room with and without the cap, primary exhausted with and without the cap, a successful turn with the cap, and a manual "Continue." message after a failed, armed turn with the cap. Each replay must consume its complete transcript with no transport mismatch. The fixed clock is 2026-11-11T00:00:00.000Z, so the plan reset is 2026-11-11T05:00:00.000Z and the credit reset (1796083200) is 2026-12-01T00:00:00.000Z; these are fixture timestamps, not dates from a live incident.
repro.sh
#!/bin/sh
set -eu
repo=${1:-$(pwd)}
repo=$(cd "$repo" && pwd)
fixture=$(mktemp -d "${REPRO_TMP_PARENT:-${TMPDIR:-/tmp}}/credit-reset-verify.XXXXXX")
trap 'rm -rf "$fixture"' EXIT HUP INT TERM
mkdir -p "$fixture/fake-home" "$fixture/tmp" "$fixture/t3-home"
export HOME="$fixture/fake-home" TMPDIR="$fixture/tmp" T3CODE_HOME="$fixture/t3-home"
export VP_REMOTE_CACHE=off VP_REMOTE_CACHE_URL= T3_VERIFY_REPO="$repo" T3_VERIFY_FIXTURE="$fixture"
ln -s "$repo/apps/server/node_modules" "$fixture/node_modules"
cat > "$fixture/package.json" <<'JSON'
{"type":"module"}
JSON
cat > "$fixture/vite.config.ts" <<'TS'
import { defineConfig } from 'vite-plus';
export default defineConfig({root: process.env.T3_VERIFY_FIXTURE, test: {environment: 'node', include: ['verify.test.ts'], testTimeout: 60000, maxWorkers: 1, fileParallelism: false}});
TS
cat > "$fixture/verify.test.ts.template" <<'TS'
import { readFileSync, writeFileSync } from 'node:fs';
import { assert, it } from '@effect/vitest';
import * as Effect from 'effect/Effect';
import * as Layer from 'effect/Layer';
import * as Ref from 'effect/Ref';
import * as CodexReplay from 'effect-codex-app-server/replay';
import * as DateTime from 'effect/DateTime';
import { TestClock } from 'effect/testing';
import { CommandId, ProviderDriverKind } from '@t3tools/contracts';
import * as IdAllocator from '@t3tools/provider-core/server/IdAllocator';
import { materializeReplayTranscriptWorkspace } from '@t3tools/provider-testing/replayTranscript';
import * as CodexTestkit from '__ROOT__/apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.testkit.ts';
import { CodexOrchestratorReplayHarness } from '__ROOT__/apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.testkit.ts';
import { runOrchestratorV2ProviderReplayScenario } from '__ROOT__/apps/server/src/orchestration-v2/testkit/ProviderReplayHarness.ts';
import { materializeFixtureInput, CODEX_MODEL_SELECTION, SIMPLE_PROMPT } from '__ROOT__/apps/server/src/orchestration-v2/testkit/fixtures/shared.ts';
import { limitRecoveryCommand } from '__ROOT__/apps/server/src/orchestration-v2/UsageLimitRecoveryWorker.ts';
import { codexRateLimitsToLimits, codexUsageLimitResetAt } from '__ROOT__/apps/server/src/provider/codexUsageLimits.ts';
import { exhaustedUsageWindow } from '__ROOT__/packages/shared/src/usageLimits.ts';
const repo = process.env.T3_VERIFY_REPO!;
const workspace = process.env.T3_VERIFY_FIXTURE!;
// Fixed clock and credit reset copied from the candidate's reported fixture.
const checkedAt = '2026-11-11T00:00:00.000Z';
const now = Date.parse(checkedAt);
const sessionSeconds = now / 1000 + 5 * 60 * 60;
const sessionReset = new Date(sessionSeconds * 1000).toISOString();
const creditReset = new Date(1796083200 * 1000).toISOString();
const source = readFileSync(repo + '/apps/server/src/orchestration-v2/testkit/fixtures/simple/codex_transcript.ndjson', 'utf8').trim().split('\n').map(JSON.parse);
const packageVersion = JSON.parse(readFileSync(repo + '/apps/server/package.json', 'utf8')).version;
const preamble = source.slice(1, source.findIndex(e => e.label === 'turn/started') + 1)
.filter(e => ['initialize', 'initialized', 'thread/start', 'turn/start', 'turn/started'].includes(e.label));
preamble.find(e => e.type === 'expect_outbound' && e.label === 'initialize').frame.params.clientInfo.version = packageVersion;
const nativeThreadId = preamble.find(e => e.type === 'emit_inbound' && e.label === 'thread/start').frame.result.thread.id;
const nativeTurn = preamble.find(e => e.label === 'turn/started').frame.params.turn;
it.effect('compares credit caps through the real Codex adapter, event ingestion, orchestrator, projection and recovery worker decision', () => Effect.gen(function* () {
const rows: any[] = [];
for (const scenario of [
{name:'room-with-credit-cap', credit:true, used:0, fail:true},
{name:'room-without-credit-cap', credit:false, used:0, fail:true},
{name:'exhausted-with-credit-cap', credit:true, used:100, fail:true},
{name:'exhausted-without-credit-cap', credit:false, used:100, fail:true},
{name:'room-with-credit-cap-success', credit:true, used:0, fail:false, manual:false},
{name:'credit-cap-manual-continuation', credit:true, used:100, fail:true, manual:true},
]) {
yield* TestClock.setTime(now);
const snapshot = {
limitId:'codex', planType:'team', rateLimitReachedType:null,
primary:{usedPercent:scenario.used, resetsAt:sessionSeconds, windowDurationMins:300},
secondary:{usedPercent:14, resetsAt:sessionSeconds + 7*24*60*60, windowDurationMins:10080},
individualLimit: scenario.credit ? {limit:'0', used:'0.0', remainingPercent:0, resetsAt:1796083200} : null,
};
const provider = {driver:'codex', usageLimits:codexRateLimitsToLimits({snapshot, checkedAt})} as any;
const blocked = exhaustedUsageWindow(provider, now, CODEX_MODEL_SELECTION.model)?.id ?? null;
const helperReset = codexUsageLimitResetAt(snapshot);
const needsRead = scenario.fail && helperReset === null;
const failureReset = scenario.fail ? (scenario.credit ? creditReset : sessionReset) : null;
const expectedReset = 'manual' in scenario && scenario.manual ? null : failureReset;
const entries:any[] = structuredClone(preamble);
entries.push({type:'emit_inbound',label:'account/rateLimits/updated',frame:{method:'account/rateLimits/updated',params:{rateLimits:snapshot}}});
entries.push({type:'emit_inbound',label:'turn/completed',frame:{method:'turn/completed',params:{threadId:nativeThreadId,turn:{...nativeTurn,status:scenario.fail?'failed':'completed',completedAt:sessionSeconds-1,error:scenario.fail?{message:'Provider stopped this request.',codexErrorInfo:'usageLimitExceeded'}:null}}}});
if(needsRead) {
entries.push({type:'expect_outbound',label:'account/rateLimits/read',frame:{id:4,method:'account/rateLimits/read',params:null}});
entries.push({type:'emit_inbound',label:'account/rateLimits/read',frame:{id:4,result:{rateLimits:{...snapshot,primary:{...snapshot.primary,usedPercent:100}},rateLimitsByLimitId:null}}});
}
if ('manual' in scenario && scenario.manual) {
const start=structuredClone(preamble.find(e=>e.type==='expect_outbound'&&e.label==='turn/start'));
start.frame.id=4; start.frame.params.input=[{type:'text',text:'Continue.'}];
const turn={...nativeTurn,id:'manual-continuation-turn',status:'inProgress',error:null};
entries.push(start,{type:'emit_inbound',label:'turn/start',frame:{id:4,result:{turn}}},{type:'emit_inbound',label:'turn/started',frame:{method:'turn/started',params:{threadId:nativeThreadId,turn}}},{type:'emit_inbound',label:'turn/completed',frame:{method:'turn/completed',params:{threadId:nativeThreadId,turn:{...turn,status:'completed',completedAt:sessionSeconds+1}}}});
}
const transcript = yield* CodexOrchestratorReplayHarness.decodeTranscript(materializeReplayTranscriptWorkspace({provider:'codex',protocol:'codex.app-server',version:source[0].version,scenario:scenario.name,entries}, workspace));
const materialized = yield* materializeFixtureInput({scenario:scenario.name, fixtureInput:{steps:[{type:'message',text:SIMPLE_PROMPT},...('manual' in scenario && scenario.manual ? [{type:'message' as const,text:'Continue.'}] : [])]},driver:ProviderDriverKind.make('codex'),modelSelection:CODEX_MODEL_SELECTION});
const idAllocator = yield* IdAllocator.IdAllocatorV2;
const threadId=materialized.projectionThreadIds[0];
const runId=idAllocator.derive.run({threadId,ordinal:1});
const arm:any = scenario.fail ? {type:'thread.metadata.update',commandId:CommandId.make('arm-'+scenario.name),threadId,limitRecovery:{runId,resetAt:failureReset,autoResume:true,snooze:false}} : null;
const driverRef = yield* Ref.make<any>(undefined);
const harness = {...CodexOrchestratorReplayHarness, makeProviderAdapterRegistryLayer: (transcript:any) => Layer.effectContext(Effect.gen(function*(){
const driver = yield* CodexReplay.makeReplayDriver(transcript);
yield* Ref.set(driverRef,driver);
return yield* Layer.build(CodexTestkit.layer({transcript,driver}));
}))};
const firstIdle=materialized.steps.findIndex(s=>s.type==='await_thread_idle');
const scenarioSteps = 'manual' in scenario && scenario.manual
? [...materialized.steps.slice(0,firstIdle+1),{type:'dispatch' as const,command:arm},{type:'advance_clock' as const,duration:'1 millis' as const},...materialized.steps.slice(firstIdle+1)]
: [...materialized.steps,...(arm?[{type:'dispatch' as const,command:arm}]:[])];
const result = yield* Effect.scoped(runOrchestratorV2ProviderReplayScenario({
name:scenario.name,transcript,commands:[...materialized.commands,...(arm?[arm]:[])],
steps:scenarioSteps,
projectionThreadIds:materialized.projectionThreadIds,runtimePolicyOverride:{cwd:workspace},
},harness));
const driver=yield* Ref.get(driverRef);
const replayState=yield* Ref.get(driver.state);
assert.equal(replayState.failure,null);
assert.equal(replayState.cursor,entries.length);
const projection=result.projections.get(threadId)!;
const error=projection.turnItems.find(i=>i.type==='error');
const shell=result.shellSnapshot.threads.find(t=>t.id===threadId)!;
const afterPlanReset = limitRecoveryCommand(shell as any,true,sessionSeconds*1000+1);
const afterCreditReset = limitRecoveryCommand(shell as any,true,1796083200*1000+1);
const row={scenario:scenario.name,blockedWindow:blocked,helperReset,accountReadExpectedAndConsumed:needsRead,runStatus:projection.runs.at(-1)!.status,allRunStatuses:projection.runs.map(r=>r.status),failure:error?.type==='error'?error.failure:null,shellReset:shell.usageLimitResetAt,limitRecovery:projection.thread.limitRecovery == null ? null : {resetAt:projection.thread.limitRecovery.resetAt,autoResume:projection.thread.limitRecovery.autoResume,snooze:projection.thread.limitRecovery.snooze},replayConsumed:replayState.cursor===entries.length,afterPlanReset:afterPlanReset?.type??null,afterCreditReset:afterCreditReset?.type??null};
rows.push(row);
console.log(JSON.stringify(row));
assert.equal(projection.runs[0].status,scenario.fail?'failed':'completed');
if ('manual' in scenario && scenario.manual) {
assert.equal(projection.runs[1].status,'completed');
assert.equal(shell.usageLimitResetAt,null);
assert.equal(error?.type==='error'?error.failure.resetAt:null,creditReset);
assert.equal(afterPlanReset,null);
continue;
}
assert.equal(shell.usageLimitResetAt,expectedReset);
if(scenario.fail) {
assert.equal(projection.thread.limitRecovery?.resetAt,expectedReset);
assert.equal(afterPlanReset?.type??null,scenario.credit?null:'message.dispatch');
assert.equal(afterCreditReset?.type,'message.dispatch');
} else { assert.equal(error,undefined); assert.equal(projection.thread.limitRecovery,null); }
}
assert.equal(rows[0].blockedWindow,null);
assert.equal(rows[1].blockedWindow,null);
assert.equal(rows[2].blockedWindow,'primary');
assert.equal(rows[3].blockedWindow,'primary');
writeFileSync(workspace + '/results.json', JSON.stringify(rows,null,2)+'\n');
console.log('DEFECT PRESENT: real failed-turn projection arms the monthly credit reset and waits past the plan reset; no-cap controls use the plan reset; a successful turn remains completed.');
}).pipe(Effect.provide(IdAllocator.layer)));
TS
python3 - "$fixture/verify.test.ts.template" "$fixture/verify.test.ts" "$repo" <<'PY'
import sys
from pathlib import Path
Path(sys.argv[2]).write_text(Path(sys.argv[1]).read_text().replace('__ROOT__', sys.argv[3]))
PY
cd "$repo"
"$repo/node_modules/.bin/vp" test run --config "$fixture/vite.config.ts" verify.test.ts
cat "$fixture/results.json"
printf '%s\n' "DEFECT PRESENT: all six integrated scenarios matched; credit-cap failures wait past the plan reset and controls do not."Expected behavior
A seat-based Codex thread stopped on its plan allowance should become eligible for automatic continuation when the blocking plan window resets; a zero monthly credit cap should not move that continuation to monthly credit renewal. The seat-based usage contract in packages/shared/src/usageLimits.ts (
t3code/packages/shared/src/usageLimits.ts
Lines 58 to 64 in d9805b2
| case "codex": { | |
| // Seat-based plans spend credits only past their plan windows, so the credit cap blocks only plans without them. | |
| const hasPlanWindows = windows.some( | |
| (window) => window.id === "primary" || window.id === "secondary", | |
| ); | |
| return exhausted.find((window) => window.id !== "individual" || !hasPlanWindows); | |
| } |
individual window when plan windows exist, and the team-plan regression test added by #34243 (t3code/apps/server/src/provider/codexUsageLimits.test.ts
Lines 44 to 80 in d9805b2
| it("keeps a seat-based workspace with a zero credit cap usable while its plan windows have room", () => { | |
| const usageLimits = codexRateLimitsToLimits({ | |
| checkedAt, | |
| snapshot: { | |
| limitId: "codex", | |
| planType: "team", | |
| rateLimitReachedType: null, | |
| primary: { usedPercent: 0, resetsAt: 1_784_000_000, windowDurationMins: 300 }, | |
| secondary: { usedPercent: 14, resetsAt: 1_784_500_000, windowDurationMins: 10080 }, | |
| individualLimit: { remainingPercent: 0, resetsAt: 1_796_083_200 }, | |
| }, | |
| }); | |
| expect( | |
| exhaustedUsageWindow( | |
| { | |
| instanceId: ProviderInstanceId.make("codex"), | |
| driver: ProviderDriverKind.make("codex"), | |
| enabled: true, | |
| installed: true, | |
| version: null, | |
| status: "ready", | |
| auth: { status: "authenticated" }, | |
| checkedAt, | |
| models: [], | |
| slashCommands: [], | |
| skills: [], | |
| usageLimits, | |
| }, | |
| Date.parse(checkedAt), | |
| "gpt-6.1-sol", | |
| ), | |
| ).toBeUndefined(); | |
| }); | |
| it("maps primary and secondary onto the session and weekly windows", () => { | |
| expect( | |
| codexRateLimitsToLimits({ |
Actual behavior
Summary. When a seat-based Codex account (here planType: "team") carries a zero monthly credit cap (individualLimit with limit: "0", remainingPercent: 0) and a turn fails with usageLimitExceeded, the server records the cap's monthly reset as the failure's reset time instead of the plan window's reset. The thread shell exposes that monthly date as usageLimitResetAt, recovery arms against it, and the usage-limit recovery worker does not continue the thread when the plan window resets; it continues only after the monthly credit reset. This happens both when the last snapshot still shows plan room and when the primary window is at 100%. Merged #34243 (which fixed #34234) explicitly listed this reset helper under "Not in this change", so this is the acknowledged follow-up, not a new discovery. Evidence comes from a protocol-frame replay through the real Codex adapter and orchestrator; no live Codex account, real provider quota exhaustion or rendered client was used.
Observed in the replay at both tested builds, with identical output:
- Plan room, with the cap: the stored failure has
resetAt: "2026-12-01T00:00:00.000Z", the thread shell'susageLimitResetAtis the same monthly date, andlimitRecoverystores it withautoResume: true. The adapter sends noaccount/rateLimits/read. The recovery decision returns no command just after the plan reset andmessage.dispatchonly after the credit reset. - Plan room, without the cap (control): the adapter consumes
account/rateLimits/read, gets primary at 100%, and records2026-11-11T05:00:00.000Z. The recovery decision returnsmessage.dispatchjust after the plan reset. - Primary at 100%, with the cap: the shared exhausted-window check reports
primary, yet the failure, shell and recovery all store the monthly date and the worker waits past the plan reset. Without the cap (control) they store the plan reset and the worker dispatches after it. No account read occurs in either variant, so this pair isolates the reset selection. - Successful turn with the cap (control): the run completes with no failure, shell reset or recovery, so the cap alone does not create a failure; a genuine provider
usage_limitfailure is required. - Manual continuation with the cap (control): after the failed, armed turn, an ordinary "Continue." message completes a second run before the saved monthly deadline (
allRunStatuses: [failed, completed], shell reset cleared).
Source path at the baseline: codexUsageLimitResetAt (
t3code/apps/server/src/provider/codexUsageLimits.ts
Lines 189 to 203 in d9805b2
| /** All exhausted windows must reset before a continuation can run. */ | |
| export function codexUsageLimitResetAt( | |
| snapshot: CodexRateLimitSnapshot | undefined, | |
| ): string | null { | |
| if (!snapshot) return null; | |
| const windows = codexRateLimitsToWindows(snapshot).filter((window) => window.usedPercent >= 100); | |
| if (windows.length === 0 || windows.some((window) => !window.resetsAt)) return null; | |
| return windows.reduce<string | null>( | |
| (latest, window) => | |
| latest === null || Date.parse(window.resetsAt!) > Date.parse(latest) | |
| ? window.resetsAt! | |
| : latest, | |
| null, | |
| ); | |
| } |
individual, as exhausted and returns the latest reset. The adapter's terminal path (t3code/apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.ts
Lines 6047 to 6059 in d9805b2
| input.context, | |
| `terminal-failure:${input.context.providerTurnId}`, | |
| ), | |
| status: terminalStatus, | |
| failure: | |
| failure.class === "usage_limit" | |
| ? { | |
| ...failure, | |
| resetAt: failureResetAt(failure, yield* Ref.get(rateLimitSnapshot)), | |
| } | |
| : failure, | |
| ...(input.providerRetry === undefined | |
| ? {} |
usage_limit failure. Because the value is non-null, awaitsReset is false (t3code/apps/server/src/orchestration-v2/Adapters/CodexAdapterV2.ts
Lines 6120 to 6155 in d9805b2
| event: CodexRootTerminalEvent, | |
| ) { | |
| const awaitsReset = | |
| event.status === "failed" && | |
| event.failure.class === "usage_limit" && | |
| event.failure.resetAt == null && | |
| // The account read only reports the ordinary allowance, never a sharing cap's reset. | |
| !classifyCodexManagedError(event.failure.code)?.usageLimit && | |
| codexUsageLimitResetAt(yield* Ref.get(rateLimitSnapshot)) === null; | |
| if (!awaitsReset) yield* publishRootTerminal(context, event); | |
| const continued = continuedRuns.get(context.providerTurnId) ?? [context]; | |
| if (event.status !== "completed") { | |
| yield* Ref.update( | |
| terminalizedNonCompletedNativeTurns, | |
| (nativeTurns) => | |
| new Set([...nativeTurns, ...continued.map((turn) => turn.nativeTurnId)]), | |
| ); | |
| } | |
| for (const turn of continued) { | |
| continuedRuns.delete(turn.providerTurnId); | |
| } | |
| const nativeThreadId = context.providerThread.nativeThreadRef?.nativeId; | |
| const rootProviderThread = | |
| nativeThreadId == null ? undefined : rootProviderThreads.get(nativeThreadId); | |
| if (nativeThreadId != null && rootProviderThread !== undefined) { | |
| rootProviderThreads.set(nativeThreadId, { ...rootProviderThread, status: "idle" }); | |
| } | |
| // Notification handlers block response routing, so the read and terminal get a fiber. | |
| if (awaitsReset) { | |
| yield* readUsageLimitReset.pipe( | |
| Effect.andThen( | |
| turnTerminalizationPermit.withPermits(1)(publishRootTerminal(context, event)), | |
| ), | |
| Effect.forkIn(scope, { startImmediately: true }), | |
| ); | |
| } |
UsageLimitRecoveryWorker (t3code/apps/server/src/orchestration-v2/UsageLimitRecoveryWorker.ts
Lines 17 to 88 in d9805b2
| export function limitRecoveryCommand( | |
| thread: ProjectionStore.ProjectionLimitRecoveryCandidate, | |
| autoResume: boolean, | |
| nowMs: number, | |
| snooze = false, | |
| ): OrchestrationV2Command | null { | |
| if ( | |
| thread.status !== "failed" || | |
| thread.lastErrorClass !== "usage_limit" || | |
| !thread.latestRunId || | |
| !thread.usageLimitResetAt || | |
| thread.archivedAt !== null || | |
| thread.settledOverride === "settled" || | |
| thread.pendingRuntimeRequest !== null | |
| ) | |
| return null; | |
| const resetMs = Date.parse(thread.usageLimitResetAt); | |
| // An already-expired window reported with a fresh failure cannot start a retry loop. | |
| if ( | |
| !Number.isFinite(resetMs) || | |
| resetMs <= DateTime.toEpochMillis(thread.latestRunCompletedAt ?? thread.updatedAt) | |
| ) | |
| return null; | |
| const identity = `${thread.id}:${thread.latestRunId}:${resetMs}`; | |
| const recovery = thread.limitRecovery; | |
| if (recovery?.runId !== thread.latestRunId || recovery.resetAt !== thread.usageLimitResetAt) { | |
| if (nowMs - resetMs > MAX_OVERDUE_ARM_MS) return null; | |
| if (!autoResume && (!snooze || resetMs <= nowMs)) return null; | |
| return { | |
| type: "thread.metadata.update", | |
| commandId: CommandId.make(`limit-arm:${identity}`), | |
| threadId: thread.id, | |
| limitRecovery: { | |
| runId: thread.latestRunId, | |
| resetAt: thread.usageLimitResetAt, | |
| autoResume, | |
| snooze: snooze && resetMs > nowMs, | |
| }, | |
| }; | |
| } | |
| if ( | |
| !recovery.autoResume || | |
| resetMs > nowMs || | |
| (thread.snoozedUntil != null && DateTime.toEpochMillis(thread.snoozedUntil) > nowMs) | |
| ) | |
| return null; | |
| const deliveryIdentity = `${identity}:${recovery.requestId ?? "legacy"}`; | |
| return { | |
| type: "message.dispatch", | |
| commandId: CommandId.make(`limit-resume:${deliveryIdentity}`), | |
| messageId: MessageId.make(`limit-resume:${deliveryIdentity}`), | |
| threadId: thread.id, | |
| usageLimitContinuationOfRunId: thread.latestRunId, | |
| ...(recovery.requestId === undefined | |
| ? {} | |
| : { usageLimitRecoveryRequestId: recovery.requestId }), | |
| text: "Continue where you left off.", | |
| attachments: [], | |
| dispatchMode: { type: "start_immediately" }, | |
| createdBy: "agent", | |
| creationSource: "server", | |
| }; | |
| } | |
| // Events that can make a thread a candidate or change its command; streaming items cannot. | |
| const affectsLimitRecovery = (event: OrchestrationV2DomainEvent) => | |
| event.type.startsWith("thread.") || | |
| event.type === "run.created" || | |
| event.type === "run.updated" || | |
| event.type === "runtime-request.updated" || | |
| event.type.startsWith("provider-session.") || | |
| (event.type === "turn-item.updated" && event.payload.type === "error"); |
Read from source only, not observed in a rendered client: UsageLimitRecoveryBanner (
t3code/apps/web/src/components/chat/UsageLimitRecoveryBanner.tsx
Lines 50 to 83 in d9805b2
| const canSchedule = | |
| !settled && !requestPending && resetAt !== null && Date.parse(resetAt) > Date.parse(stoppedAt); | |
| const resumes = | |
| canSchedule && recovery?.runId === runId && recovery.resetAt === resetAt && recovery.autoResume; | |
| const workspaceLimit = errorCode === CHATGPT_WORKSPACE_USAGE_LIMIT_CODE; | |
| const chatGptLimit = workspaceLimit || errorCode === CHATGPT_USAGE_LIMIT_CODE; | |
| const recoveryActions = canSchedule ? ( | |
| <RecoveryActions key={`${runId}:${resetAt}`} {...props} /> | |
| ) : null; | |
| const displayAt = | |
| resumes && resetAt && snoozedUntil && Date.parse(snoozedUntil) > Date.parse(resetAt) | |
| ? snoozedUntil | |
| : resetAt; | |
| const resetDescription = displayAt | |
| ? `${resumes ? "Resumes" : "Resets"} ${formatUpcomingTimestamp(displayAt, timestampFormat)}` | |
| : "Reset time unavailable; retry manually"; | |
| if (!chatGptLimit) { | |
| return { | |
| id: `usage-limit-recovery:${runId}`, | |
| variant: "warning", | |
| group: "environment", | |
| icon: <GaugeIcon />, | |
| title: "Usage limit reached", | |
| description: resetDescription, | |
| actions: recoveryActions, | |
| }; | |
| } | |
| // OpenAI's partner guidelines specify this ChatGPT notice and its actions. | |
| return { | |
| id: `usage-limit-recovery:${runId}`, | |
| variant: "warning", | |
| group: "environment", | |
| icon: <OpenAI aria-hidden="true" />, | |
| title: "ChatGPT usage limit reached", |
Hypothesis, not verified: the failure-reset selector kept the individual-credit rule after #34243 gave the model-selection check its seat-based exception.
Evidence
- Expected source:
packages/shared/src/usageLimits.tslines 58-64 (seat-based exception for theindividualwindow) and the team-plan zero-credit test atapps/server/src/provider/codexUsageLimits.test.tslines 44-80; fix(shared): a zero-credit cap no longer marks seat-based Codex out of usage #34243's "Not in this change" note describing this exact follow-up. - Failure source: observed output of
repro.shabove, throughcodexUsageLimitResetAt(apps/server/src/provider/codexUsageLimits.tslines 189-203) →CodexAdapterV2.tslines 6047-6059 and 6120-6155 → stored error item, thread shell andthread.limitRecovery→limitRecoveryCommandinapps/server/src/orchestration-v2/UsageLimitRecoveryWorker.tslines 17-88. - Evidence provenance: observed
- Local verification: reproduced
- Reproduction completeness: complete
The reproduction replays Codex app-server protocol frames through the real adapter and orchestrator; no production function is replaced or patched. It does not use a live Codex account, a real provider quota exhaustion or a rendered client, and the banner text is read from source. The individualLimit payload includes the wire schema's limit, used, remainingPercent and resetsAt fields, matching the shape reported in #34234.
The full script output is under Logs or stack traces; it is identical at both tested builds.
Restoration check
With the script above, a usage-limit failure with the zero-credit cap stores the plan reset that the matching no-cap control stores (2026-11-11T05:00:00.000Z in both pairs: from the account read when the snapshot shows plan room, from the primary window when it is exhausted), the thread shell and armed recovery carry that date, and the recovery decision returns message.dispatch just after the plan reset. The no-cap controls must keep their current results, the successful-turn control must stay completed with no failure or recovery, and an account whose only blocking limit is the credit cap (no plan windows) must still wait for the credit reset.
References
- Codex seat-based workspace with a 0-credit monthly cap is treated as out of usage, new threads switch to the fallback model #34234 (Codex seat-based workspace with a 0-credit monthly cap is treated as out of usage, new threads switch to the fallback model #34234), closed: the zero-credit cap made seat-based Codex look out of usage for model selection. Merged fix(shared): a zero-credit cap no longer marks seat-based Codex out of usage #34243 (fix(shared): a zero-credit cap no longer marks seat-based Codex out of usage #34243) fixed that and states under "Not in this change": "codexUsageLimitResetAt (apps/server/src/provider/codexUsageLimits.ts), used by CodexAdapterV2 after a usage_limit turn failure, still counts a 0-credit 'individual' window as hit, so a seat-based user who hits the 5-hour limit may get a continuation reset date on the 1st of next month instead of the session reset." Closed, unmerged fix(server): Codex seat workspaces with a zero credit cap are no longer out of usage #34239 proposed an alternative filter.
- Related, not duplicates: Redeeming a Codex reset credit leaves T3 stuck: Usage keeps showing exhausted limits and limit-held threads never resume #33770 (closed) concerns reset-credit redemption and refresh; [Bug]: Usage-limit auto-resume ignores allowances restored before the scheduled reset #22307 concerns allowances restored before a recorded deadline; [Bug]: Usage-limit auto-resume gives up permanently when the continuation is rejected right after the reset #33708 concerns a continuation rejected just after a correct reset; [Bug]: Orchestrator V2 Usage Reset Tracker Broken #14169 concerns missing reset dates. Open PR fix(server): Codex usage-limit stops keep their reset time #18721 changes this helper's 99% and stale-window handling but not the
individualwindow, and open PR fix(server): Codex accounts that can spend credits keep text generation and new threads #27824 changes credit-based continuation but not the reset selection.
Impact
Minor bug or occasional failure
Assessed P2 (supported). For seat-based Codex accounts with a zero monthly credit cap, a genuine usage-limit failure arms unattended recovery for the monthly credit reset instead of the plan reset, so automatic continuation can be delayed until monthly renewal; the replay shows the stored state and the recovery decision, while live-account frequency is unknown and no data is lost.
Version or commit
0.1.9, commit d9805b2; repeated on main at 8024e50 with identical results
Environment
macOS arm64, Node v26.12.1, repository-pinned dependencies. Codex app-server protocol replay through the real server adapter and orchestrator; no live provider account or rendered client.
Logs or stack traces
Full script output
Test Files 1 passed (1)
Tests 1 passed (1)
[
{
"scenario": "room-with-credit-cap",
"blockedWindow": null,
"helperReset": "2026-12-01T00:00:00.000Z",
"accountReadExpectedAndConsumed": false,
"runStatus": "failed",
"allRunStatuses": [
"failed"
],
"failure": {
"class": "usage_limit",
"message": "Provider stopped this request.",
"code": "usageLimitExceeded",
"retryable": null,
"resetAt": "2026-12-01T00:00:00.000Z"
},
"shellReset": "2026-12-01T00:00:00.000Z",
"limitRecovery": {
"resetAt": "2026-12-01T00:00:00.000Z",
"autoResume": true,
"snooze": false
},
"replayConsumed": true,
"afterPlanReset": null,
"afterCreditReset": "message.dispatch"
},
{
"scenario": "room-without-credit-cap",
"blockedWindow": null,
"helperReset": null,
"accountReadExpectedAndConsumed": true,
"runStatus": "failed",
"allRunStatuses": [
"failed"
],
"failure": {
"class": "usage_limit",
"message": "Provider stopped this request.",
"code": "usageLimitExceeded",
"retryable": null,
"resetAt": "2026-11-11T05:00:00.000Z"
},
"shellReset": "2026-11-11T05:00:00.000Z",
"limitRecovery": {
"resetAt": "2026-11-11T05:00:00.000Z",
"autoResume": true,
"snooze": false
},
"replayConsumed": true,
"afterPlanReset": "message.dispatch",
"afterCreditReset": "message.dispatch"
},
{
"scenario": "exhausted-with-credit-cap",
"blockedWindow": "primary",
"helperReset": "2026-12-01T00:00:00.000Z",
"accountReadExpectedAndConsumed": false,
"runStatus": "failed",
"allRunStatuses": [
"failed"
],
"failure": {
"class": "usage_limit",
"message": "Provider stopped this request.",
"code": "usageLimitExceeded",
"retryable": null,
"resetAt": "2026-12-01T00:00:00.000Z"
},
"shellReset": "2026-12-01T00:00:00.000Z",
"limitRecovery": {
"resetAt": "2026-12-01T00:00:00.000Z",
"autoResume": true,
"snooze": false
},
"replayConsumed": true,
"afterPlanReset": null,
"afterCreditReset": "message.dispatch"
},
{
"scenario": "exhausted-without-credit-cap",
"blockedWindow": "primary",
"helperReset": "2026-11-11T05:00:00.000Z",
"accountReadExpectedAndConsumed": false,
"runStatus": "failed",
"allRunStatuses": [
"failed"
],
"failure": {
"class": "usage_limit",
"message": "Provider stopped this request.",
"code": "usageLimitExceeded",
"retryable": null,
"resetAt": "2026-11-11T05:00:00.000Z"
},
"shellReset": "2026-11-11T05:00:00.000Z",
"limitRecovery": {
"resetAt": "2026-11-11T05:00:00.000Z",
"autoResume": true,
"snooze": false
},
"replayConsumed": true,
"afterPlanReset": "message.dispatch",
"afterCreditReset": "message.dispatch"
},
{
"scenario": "room-with-credit-cap-success",
"blockedWindow": null,
"helperReset": "2026-12-01T00:00:00.000Z",
"accountReadExpectedAndConsumed": false,
"runStatus": "completed",
"allRunStatuses": [
"completed"
],
"failure": null,
"shellReset": null,
"limitRecovery": null,
"replayConsumed": true,
"afterPlanReset": null,
"afterCreditReset": null
},
{
"scenario": "credit-cap-manual-continuation",
"blockedWindow": "primary",
"helperReset": "2026-12-01T00:00:00.000Z",
"accountReadExpectedAndConsumed": false,
"runStatus": "completed",
"allRunStatuses": [
"failed",
"completed"
],
"failure": {
"class": "usage_limit",
"message": "Provider stopped this request.",
"code": "usageLimitExceeded",
"retryable": null,
"resetAt": "2026-12-01T00:00:00.000Z"
},
"shellReset": null,
"limitRecovery": {
"resetAt": "2026-12-01T00:00:00.000Z",
"autoResume": true,
"snooze": false
},
"replayConsumed": true,
"afterPlanReset": null,
"afterCreditReset": null
}
]
DEFECT PRESENT: all six integrated scenarios matched; credit-cap failures wait past the plan reset and controls do not.
Screenshots, recordings, or supporting files
No response
Workaround
Once the plan allowance is back, manually continuing the failed thread works; the replay shows the orchestrator accepts an ordinary continuation before the saved monthly deadline, though real provider acceptance still depends on the actual allowance.
Activity
maria-rcks commentedon Nov 15, 2026
Note
Written by
claude-fable-5-2on behalf of MariaClosing: this account is denounced in
.github/VOUCHED.tdfor filing automated issues faster than maintainers can triage them. A fix PR that links this issue is not affected by this close.