Skip to content

fix(server): preserve one-shot cooldown bypasses - #108

Merged
chrisohearn merged 3 commits into
integration/mcp-events-20261109from
r4/failover-bypass-followup
Nov 13, 2026
Merged

chrisohearn merged 3 commits into
integration/mcp-events-20261109from
r4/failover-bypass-followup

Conversation

@chrisohearn

@chrisohearn chrisohearn commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to #88's delta review, specifically its fastMode P2 and all three P3 findings: automatic runs consuming pending picks, repeated scheduled bypass while queued runs remain held, and missing dispatch-side scheduled-pick and multi-continuation tests.

Ordinary web sends could bypass an account cooldown because fastMode: false differed from an omitted option. Automatic runs could consume a pending explicit pick, while a lingering scheduled pick could bypass repeatedly.

Changes

  • Compare sent selections with draftModelPickMatchesThread, so Fast off and unset mean the same thing in either direction.
  • Reuse the existing persisted thread bypass flag for scheduled picks. Only a resendable prompt using the selected model consumes it, once. A held queued run no longer lets a schedule repeatedly bypass cooldown routing.
  • Share the existing resendability policy between failover and bypass consumption. Notification wakes and delegated completions preserve the pending pick on both dispatch and queued start, including when another thread-metadata event is written.
  • Keep the existing failover, restart, claimed-run, and provider-specific message guards. No new schema, migration, dependency, live settings change, restart, merge, or deployment.

Verification

Head bb1c50cd59ef4b2ffc58ae2e117f9cd41fd347ea, including fresh integration 56e297dccc5bc07cce51e273c35c933e50ecfb4c. The integration merge changed only host-guard files, not the checked runtime source. Pushed without force.

All 23 focused tests pass: 15 provider-switch/routing/failover cases, 3 runtime cases covering scheduled picks, multiple old-model server continuations, two notification wakes and delegated completions, and 5 failover/cooldown unit cases. Scoped formatting, lint and diff checks pass on the four changed files. Every heavy command used the shared lock, and Vitest used two workers with no file parallelism.

Five regression cases failed before the fix at the intended assertions: Fast off versus omitted in both directions, a repeated direct scheduled send with a held queue, a notification consuming the pick, and a queued delegated completion consuming it. Existing test cases and tables were extended; no new test file was added.

Hosted CI is the typecheck gate. Local dependencies remain stale after the upstream merge, as already diagnosed and accepted by the coordinator; no reinstall or unrelated type repair was attempted.

Acceptance commands

These isolated tests use replay adapters and temporary state, not the live database or paid provider turns. Run from the repository root:

export PATH="$PWD/node_modules/.bin:$PATH"
flock -w 1800 /home/cohearn/.cache/t3-r4/heavy.lock vp test run apps/server/src/orchestration-v2/testkit/ProviderSwitch.integration.test.ts -t 're-sends a message on the fallback|keeps queued .* routing safe' --maxWorkers=2 --no-file-parallelism
flock -w 1800 /home/cohearn/.cache/t3-r4/heavy.lock vp test run apps/server/src/orchestration-v2/runtimeLayer.test.ts -t 'keeps a scheduled model change|starts every message queued behind a scheduled model change|starts a queued delegated completion on a selection set' --maxWorkers=2 --no-file-parallelism
flock -w 1800 /home/cohearn/.cache/t3-r4/heavy.lock vp test run apps/server/src/orchestration-v2/ProviderFailover.test.ts apps/server/src/provider/ProviderCooldown.test.ts --maxWorkers=2 --no-file-parallelism

Implemented with GPT-6.1 Sol in Codex CLI.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-13T22:09:17.540484Z 5d8bca8 New commits
🔒 Security Review ✅ Completed 2026-11-13T22:08:46.079064Z 5d8bca8 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb1c50cd59

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +7827 to +7829
(resendablePrompt &&
command.modelSelection !== undefined &&
!draftModelPickMatchesThread(command.modelSelection, projection.thread.modelSelection)) ||

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Compare explicit picks against the scheduled default

When next-turn selection B is pending while the persisted thread selection is A, a resendable dispatch explicitly naming A is compared with A here, so bypassProviderCooldown is false. If A is cooling, resolveProviderCooldownTarget silently rewrites this explicitly selected run to the fallback even though A differs from the effective next-turn default B. Compare the explicit pick with nextTurnModelSelection(projection.thread) so this deliberate departure retains its one-shot bypass.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5d8bca8. Explicit selections are now compared with nextTurnModelSelection(thread), so choosing persisted A against scheduled B keeps its bypass while A is cooling. The scheduled-explicit-current regression failed before the repair and passes now; all 30 focused cases and scoped format/lint pass.

Comment on lines 3136 to +3139
...threadAtStart,
providerInstanceId: queuedRun.providerInstanceId,
modelSelection: queuedRun.modelSelection,
bypassProviderCooldown: false,
...(consumesProviderCooldownBypass ? { bypassProviderCooldown: false } : {}),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear the bypass when replacing a scheduled pick

When scheduled selection B is pending and a later explicitly named queued run on A starts after the scheduling boundary, threadAtRunStart clears scheduledModelSelection, but consumesProviderCooldownBypass is false because A does not match B. This conditional therefore preserves the now-orphaned bypassProviderCooldown: true, allowing the next ordinary A prompt to bypass a cooldown even though no explicit pick remains. Clear the associated bypass whenever the scheduled selection is cleared or superseded.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5d8bca8. threadAtRunStart clears bypassProviderCooldown whenever it clears or supersedes scheduledModelSelection, on both dispatch and queued starts. The scheduled-superseded regression proves the flag is cleared when a later explicit A replaces scheduled B; it failed before the repair and passes now.

Comment on lines +7822 to +7825
const consumesProviderCooldownBypass =
resendablePrompt &&
projection.thread.bypassProviderCooldown === true &&
draftModelPickMatchesThread(modelSelection, nextTurnModelSelection(projection.thread));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Exclude recovery runs from consuming the pending pick

When an automatic failover or restart continuation targets the same model as a pending scheduled pick, its copied user prompt passes resendable and this predicate consumes the thread bypass before the later guards exempt the recovery run from cooldown routing. These runs continue an earlier turn rather than taking the scheduled next turn, so the user's subsequent prompt can be redirected away from the cooling model. Exclude failover and restart continuations from bypass consumption, as resolveProviderCooldownTarget already does.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5d8bca8. Failover and restart continuations cannot consume a pending bypass on dispatch or queued start, and schedule cleanup treats them as continuations too. The two failover and two restart cases assert an armed pick before recovery and its preservation afterward; all four failed before the repair and pass now.

Comment on lines +2446 to +2450
const consumesProviderCooldownBypass =
queuedMessage !== undefined &&
ProviderFailover.resendable(queuedMessage) &&
projection.thread.bypassProviderCooldown === true &&
draftModelPickMatchesThread(requestedSelection, nextTurnModelSelection(projection.thread));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the bypass across older explicit queued runs

If an explicitly modeled queued run predates a scheduled change but happens to name the same selection, model equality here consumes the scheduled bypass even though threadAtRunStart correctly keeps the schedule pending through that older run. The queued run already has its own explicit-pick bypass, so when the selected provider is cooling, the first default turn after the preexisting queue is incorrectly routed to the fallback. Only consume the scheduled bypass for a run that actually follows or supersedes the scheduled selection boundary.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 5d8bca8. An explicit queued run at or before queuedThroughOrdinal keeps its own run bypass and cannot claim the later scheduled bypass. The scheduled-older-explicit regression verifies that the pick survives that run and is consumed once by the next default prompt; it failed before the repair and passes now.

Merge current integration and repair the four PR108 findings together. Compare explicit picks with the effective default, clear discarded schedule bypasses, preserve picks through recovery, and respect the queued scheduling boundary. Verified 30 focused cases and scoped format/lint under the shared resource lock; hosted CI owns typecheck.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5d8bca8a79

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +7948 to +7952
const consumesProviderCooldownBypass =
!recoveryRun &&
resendablePrompt &&
projection.thread.bypassProviderCooldown === true &&
draftModelPickMatchesThread(modelSelection, nextTurnModelSelection(projection.thread));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Clear the inherited bypass when an explicit pick replaces it

When an idle thread already has a one-shot bypass for selection B and the user sends a resendable prompt explicitly selecting C, this match is false, so consumesProviderCooldownBypass remains false while the following expression correctly gives the C run its own bypass. Because there is no scheduled selection, threadAtRunStart then preserves the thread-level bypass; after C becomes the thread default, the next ordinary C prompt consumes that stale flag and skips cooldown fallback. Clear the inherited thread flag when an explicit pick supersedes the effective selection while retaining the new run's local bypass.

Useful? React with 👍 / 👎.

@chrisohearn
chrisohearn merged commit 36df3be into integration/mcp-events-20261109 Nov 13, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants