Skip to content

Commit aac7be6

Browse files
feat(server): continue long idle threads in a fresh provider session
A long thread re-sends its whole history on every call. Once its prompt cache has expired, the next request is uncached anyway, so that is the cheapest moment to drop old history. With the new setting on, a person's message to a thread over 100k tokens, after an hour idle, skips resume and continues through the existing context handoff. Commands, agent messages, and threads with background work are left alone. The setting is off by default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 52c1eb3 commit aac7be6

8 files changed

Lines changed: 183 additions & 10 deletions

File tree

‎apps/mobile/src/features/settings/SettingsServerControlsRouteScreen.tsx‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -467,6 +467,20 @@ function ServerSettingsDetail(props: { readonly page: SettingsPage }) {
467467
))}
468468
</SettingsSection>
469469
) : null}
470+
<SettingsSection title="Long threads">
471+
<SettingsSwitchRow
472+
icon="clock"
473+
label="Fresh session after a long break"
474+
subtitle={
475+
projectSelected
476+
? "Environment-wide setting. Select All projects to change it."
477+
: "After an hour away, continue a long thread in a new provider session with recent context. Lowers cached input cost."
478+
}
479+
value={uniform("freshSessionOnColdReturn")}
480+
disabled={disabledFor("freshSessionOnColdReturn")}
481+
onValueChange={(value) => write({ freshSessionOnColdReturn: value })}
482+
/>
483+
</SettingsSection>
470484
<SettingsSection title="Skills">
471485
<SettingsSwitchRow
472486
icon="square.grid.2x2"

‎apps/mobile/src/features/settings/settings-search.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,9 @@ const SETTINGS_SEARCH_TERMS = {
5959
"assisted learning",
6060
"conversation summaries",
6161
"pstack",
62+
"fresh session",
63+
"long break",
64+
"cached input",
6265
],
6366
SettingsFollowUp: ["follow-ups", "queue", "steer", "while the agent is running"],
6467
SettingsArchive: ["archive", "archived", "restore", "unarchive"],

‎apps/server/src/orchestration-v2/ProviderTurnStartService.test.ts‎

Lines changed: 77 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,7 @@ import * as Option from "effect/Option";
4949
import * as Path from "effect/Path";
5050
import * as PlatformError from "effect/PlatformError";
5151
import * as Schema from "effect/Schema";
52+
import * as TestClock from "effect/testing/TestClock";
5253

5354
import * as ServerConfig from "../config.ts";
5455
import { withWorkspaceLease } from "../workspace/workspaceLease.ts";
@@ -83,11 +84,14 @@ const normalize = (target: string) => {
8384
}
8485
return `/${parts.join("/")}`;
8586
};
86-
const layerWorktreeLocation = (worktreesDir = managedWorktreesDir) =>
87+
const layerWorktreeLocation = (
88+
worktreesDir = managedWorktreesDir,
89+
settings?: Parameters<typeof ServerSettings.layerTest>[0],
90+
) =>
8791
Layer.mergeAll(
8892
Path.layer,
8993
Layer.mock(ServerConfig.ServerConfig)({ worktreesDir } as ServerConfig.ServerConfig["Service"]),
90-
ServerSettings.layerTest(),
94+
ServerSettings.layerTest(settings),
9195
);
9296

9397
function startWithMissingWorktree(
@@ -286,6 +290,7 @@ it.each([
286290
});
287291

288292
function makeLocalCommandHarness(input: {
293+
readonly settings?: Parameters<typeof ServerSettings.layerTest>[0];
289294
readonly pstack?: PstackPack.PstackPack["Service"];
290295
readonly text: string;
291296
readonly knowledgeStore?: Effect.Success<typeof KnowledgePersistence.makeStore>;
@@ -1030,7 +1035,7 @@ function makeLocalCommandHarness(input: {
10301035
}),
10311036
}),
10321037
IdAllocator.layer,
1033-
layerWorktreeLocation(input.worktreesDir),
1038+
layerWorktreeLocation(input.worktreesDir, input.settings),
10341039
input.fileSystem === undefined
10351040
? FileSystem.layerNoop({ exists: () => Effect.succeed(true) })
10361041
: Layer.succeed(FileSystem.FileSystem, input.fileSystem),
@@ -2297,6 +2302,75 @@ effectIt.effect(
22972302
}),
22982303
);
22992304

2305+
effectIt.effect.each([
2306+
{ name: "a long idle thread", enabled: true, usedTokens: 120_000, idleMinutes: 61, fresh: true },
2307+
{ name: "the setting off", enabled: false, usedTokens: 120_000, idleMinutes: 61, fresh: false },
2308+
{ name: "a recent turn", enabled: true, usedTokens: 120_000, idleMinutes: 59, fresh: false },
2309+
{ name: "a small thread", enabled: true, usedTokens: 99_999, idleMinutes: 61, fresh: false },
2310+
{
2311+
name: "a command message",
2312+
enabled: true,
2313+
usedTokens: 120_000,
2314+
idleMinutes: 61,
2315+
text: "/compact",
2316+
fresh: false,
2317+
},
2318+
{
2319+
name: "a message not from a person",
2320+
enabled: true,
2321+
usedTokens: 120_000,
2322+
idleMinutes: 61,
2323+
createdBy: "system" as const,
2324+
fresh: false,
2325+
},
2326+
] as const)("starting a turn with $name picks fresh session: $fresh", (scenario) =>
2327+
Effect.gen(function* () {
2328+
const harness = makeLocalCommandHarness({
2329+
settings: { freshSessionOnColdReturn: scenario.enabled },
2330+
text: "text" in scenario ? scenario.text : "Keep going on the migration",
2331+
previousTurn: { runStatus: "completed", turnStatus: "completed" },
2332+
recoveryNativeId: "fresh-native-conversation",
2333+
});
2334+
const projection = harness.projection();
2335+
const previousAt = "2026-09-04T12:00:00.000Z";
2336+
const previousAttemptId = RunAttemptId.make("previous-failed-attempt");
2337+
harness.setProjection({
2338+
...projection,
2339+
messages: projection.messages.map((message) => ({
2340+
...message,
2341+
...("createdBy" in scenario ? { createdBy: scenario.createdBy } : {}),
2342+
})),
2343+
attempts: [
2344+
...projection.attempts,
2345+
{
2346+
...projection.attempts[0]!,
2347+
id: previousAttemptId,
2348+
runId: RunId.make("previous-failed-run"),
2349+
rootNodeId: NodeId.make("previous-root"),
2350+
nativeThreadId: "existing-native-conversation",
2351+
providerThreadId: projection.providerThreads[0]!.id,
2352+
status: "completed",
2353+
},
2354+
],
2355+
providerTurns: projection.providerTurns.map((turn) => ({
2356+
...turn,
2357+
tokenUsage: { usedTokens: scenario.usedTokens, updatedAt: previousAt },
2358+
})),
2359+
});
2360+
yield* TestClock.setTime(Date.parse(previousAt) + scenario.idleMinutes * 60_000);
2361+
2362+
yield* harness.start;
2363+
2364+
// Commands run without a root run, which also means no fresh session.
2365+
const nativeId =
2366+
harness.startRootRun.mock.calls[0]?.[0].providerThread.nativeThreadRef?.nativeId;
2367+
expect(nativeId === "fresh-native-conversation").toBe(scenario.fresh);
2368+
expect(harness.events.some((event) => event.type === "context-handoff.updated")).toBe(
2369+
scenario.fresh,
2370+
);
2371+
}),
2372+
);
2373+
23002374
effectIt.effect.each(["prepared", "uncertain"] as const)(
23012375
"recovers a %s knowledge receipt through the native handoff fallback without resuming the ambiguous conversation",
23022376
(status) =>

‎apps/server/src/orchestration-v2/ProviderTurnStartService.ts‎

Lines changed: 44 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,11 @@ export class ProviderTurnStartServiceV2 extends Context.Service<
136136
ProviderTurnStartServiceV2Shape
137137
>()("t3/orchestration-v2/ProviderTurnStartService/ProviderTurnStartServiceV2") {}
138138

139+
// Past both limits, a replay of real threads shows a fresh session repaying its uncached first
140+
// request: the cache has already expired, and the smaller prompt is cheaper on every later call.
141+
const COLD_RETURN_IDLE_MS = 60 * 60_000;
142+
const COLD_RETURN_MIN_CONTEXT_TOKENS = 100_000;
143+
139144
export const layer: Layer.Layer<
140145
ProviderTurnStartServiceV2,
141146
never,
@@ -159,6 +164,7 @@ export const layer: Layer.Layer<
159164
ProviderTurnStartServiceV2,
160165
Effect.gen(function* () {
161166
const pstack = yield* PstackPack.PstackPack;
167+
const serverSettings = yield* ServerSettings.ServerSettingsService;
162168
const eventSink = yield* EventSink.EventSinkV2;
163169
const turnAnalytics = yield* ProviderUsageAnalytics.ProviderTurnAnalytics;
164170
const contextHandoffService = yield* ContextHandoffService.ContextHandoffServiceV2;
@@ -1040,6 +1046,27 @@ export const layer: Layer.Layer<
10401046
return retryHandoff;
10411047
},
10421048
);
1049+
const isColdReturn = Effect.fn("ProviderTurnStartService.isColdReturn")(function* () {
1050+
const enabled = yield* serverSettings.getSettings.pipe(
1051+
Effect.map((settings) => settings.freshSessionOnColdReturn),
1052+
Effect.orElseSucceed(() => false),
1053+
);
1054+
if (!enabled) return false;
1055+
const measured = latestNativeContextUsage(projection, providerThread);
1056+
if (measured === undefined) return false;
1057+
if (measured.usage.usedTokens < COLD_RETURN_MIN_CONTEXT_TOKENS) return false;
1058+
const reportedAt = DateTime.make(measured.reportedAt);
1059+
if (Option.isNone(reportedAt)) return false;
1060+
const idleMs =
1061+
DateTime.toEpochMillis(yield* DateTime.now) - DateTime.toEpochMillis(reportedAt.value);
1062+
if (idleMs < COLD_RETURN_IDLE_MS) return false;
1063+
const hasPendingWork = yield* (
1064+
session.hasPendingBackgroundWorkForThread?.(providerThread, { includeSubagents: true }) ??
1065+
session.hasPendingBackgroundWork ??
1066+
Effect.succeed(false)
1067+
);
1068+
return !hasPendingWork;
1069+
});
10431070
const prepareRecoveryHandoff = Effect.fn("ProviderTurnStartService.prepareRecoveryHandoff")(
10441071
function* () {
10451072
const transferId = yield* idAllocator.allocate.contextTransfer({
@@ -1244,15 +1271,23 @@ export const layer: Layer.Layer<
12441271
handoff.delivery?.nativeThreadId === providerThread.nativeThreadRef?.nativeId &&
12451272
handoff.delivery?.status === "pending",
12461273
);
1274+
const coldReturn =
1275+
!uncertainDelivery &&
1276+
message.createdBy === "user" &&
1277+
!isCompactionCommandMessage(message) &&
1278+
!goalCommand &&
1279+
(yield* isColdReturn());
12471280
const resumed = yield* Effect.result(
1248-
uncertainDelivery
1281+
uncertainDelivery || coldReturn
12491282
? Effect.fail(
12501283
new ProviderAdapter.ProviderAdapterTurnStartError({
12511284
driver: session.driver,
12521285
threadId: projection.thread.id,
12531286
providerThreadId: providerThread.id,
12541287
runId,
1255-
cause: "Uncertain native history injection",
1288+
cause: coldReturn
1289+
? "Fresh session after a long idle"
1290+
: "Uncertain native history injection",
12561291
}),
12571292
)
12581293
: session.resumeThread({
@@ -1322,11 +1357,13 @@ export const layer: Layer.Layer<
13221357
driver: session.driver,
13231358
providerThreadId: providerThread.id,
13241359
runId,
1325-
reason: uncertainDelivery
1326-
? "uncertain_history_delivery"
1327-
: resumed._tag === "Success"
1328-
? "handoff_budget_exhausted"
1329-
: "resume_failed",
1360+
reason: coldReturn
1361+
? "cold_return"
1362+
: uncertainDelivery
1363+
? "uncertain_history_delivery"
1364+
: resumed._tag === "Success"
1365+
? "handoff_budget_exhausted"
1366+
: "resume_failed",
13301367
errorTag: resumed._tag === "Failure" ? resumed.failure._tag : "ContextHandoffBudgetError",
13311368
});
13321369
const replacement = yield* loadFromProvider(

‎apps/web/src/components/settings/SettingsPanels.tsx‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -523,6 +523,9 @@ export function useSettingsRestore(onRestored?: () => void) {
523523
? ["Show skills in slash menu"]
524524
: []),
525525
...(settings.pstackEnabled !== DEFAULT_UNIFIED_SETTINGS.pstackEnabled ? ["pstack"] : []),
526+
...(settings.freshSessionOnColdReturn !== DEFAULT_UNIFIED_SETTINGS.freshSessionOnColdReturn
527+
? ["Fresh session after a long break"]
528+
: []),
526529
...(settings.composerRichTextEnabled !== DEFAULT_UNIFIED_SETTINGS.composerRichTextEnabled
527530
? ["Rich text composer"]
528531
: []),
@@ -642,6 +645,7 @@ export function useSettingsRestore(onRestored?: () => void) {
642645
proactivePanelsEnabled: DEFAULT_UNIFIED_SETTINGS.proactivePanelsEnabled,
643646
showSkillsInSlashMenu: DEFAULT_UNIFIED_SETTINGS.showSkillsInSlashMenu,
644647
pstackEnabled: DEFAULT_UNIFIED_SETTINGS.pstackEnabled,
648+
freshSessionOnColdReturn: DEFAULT_UNIFIED_SETTINGS.freshSessionOnColdReturn,
645649
composerRichTextEnabled: DEFAULT_UNIFIED_SETTINGS.composerRichTextEnabled,
646650
sendShortcut: DEFAULT_UNIFIED_SETTINGS.sendShortcut,
647651
followUpBehavior: DEFAULT_UNIFIED_SETTINGS.followUpBehavior,
@@ -1212,6 +1216,33 @@ export function ChatSettingsPanel() {
12121216
/>
12131217
}
12141218
/>
1219+
<SettingsRow
1220+
{...searchableSetting("fresh-session-on-cold-return")}
1221+
description="After an hour away, continue a thread over about 100k tokens in a new provider session with recent context. Lowers cached input cost, though the agent may re-read some files."
1222+
resetAction={
1223+
settings.freshSessionOnColdReturn !==
1224+
DEFAULT_UNIFIED_SETTINGS.freshSessionOnColdReturn ? (
1225+
<SettingResetButton
1226+
label="Fresh session after a long break"
1227+
onClick={() =>
1228+
updateSettings({
1229+
freshSessionOnColdReturn: DEFAULT_UNIFIED_SETTINGS.freshSessionOnColdReturn,
1230+
})
1231+
}
1232+
/>
1233+
) : null
1234+
}
1235+
control={
1236+
<ScopedSwitch
1237+
settingKeys={["freshSessionOnColdReturn"]}
1238+
checked={settings.freshSessionOnColdReturn}
1239+
onCheckedChange={(checked) =>
1240+
updateSettings({ freshSessionOnColdReturn: Boolean(checked) })
1241+
}
1242+
aria-label="Fresh session after a long break"
1243+
/>
1244+
}
1245+
/>
12151246
</AdvancedSettings>
12161247
</SettingsSection>
12171248

‎apps/web/src/components/settings/settingsSearch.ts‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -614,6 +614,12 @@ export const SETTINGS_SEARCH_ITEMS = [
614614
to: "/settings/chat",
615615
searchTerms: ["lauren tan skills workflows plugin pack architect setup models"],
616616
},
617+
{
618+
id: "fresh-session-on-cold-return",
619+
title: "Fresh session after a long break",
620+
to: "/settings/chat",
621+
searchTerms: ["cached input cost tokens long thread idle context handoff trim"],
622+
},
617623
{
618624
id: "composer-rich-text",
619625
title: "Rich text composer",

‎docs/user/portable-handoffs.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,12 @@ to retrieve saved messages and activity, including the remainder of a long item.
2121
constraint, you can still repeat it in your next message. Unless the project has conversation
2222
summaries turned on, a handoff is a budgeted selection, not a summary.
2323

24+
## Fresh sessions after a long break
25+
26+
Long threads get more expensive with every message, because the provider re-reads the whole history each time. With **Settings > Advanced > Fresh session after a long break** turned on, a message you send to a thread over about 100,000 tokens, after an hour or more idle, continues in a new provider session. It receives the same handoff described above, and your thread and its history in T3 Code stay as they were.
27+
28+
The first request costs more because nothing is cached, and the agent may re-read some files, so this suits threads you keep returning to. Runs with background work still in progress are left alone. The setting is off by default.
29+
2430
## Conversation summaries
2531

2632
With conversation summaries on, older history that would not fit in a handoff can arrive as short

‎packages/contracts/src/settings.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1297,6 +1297,7 @@ export const ServerSettings = Schema.Struct({
12971297
enableAgentDeviceAccess: Schema.Boolean.pipe(Schema.withDecodingDefault(Effect.succeed(false))),
12981298
/** Offer the bundled pstack skills in every provider composer. */
12991299
pstackEnabled: Schema.Boolean.pipe(Schema.withDecodingDefault(Effect.succeed(false))),
1300+
freshSessionOnColdReturn: Schema.Boolean.pipe(Schema.withDecodingDefault(Effect.succeed(false))),
13001301
/** Off by default: turning it on moves skill folders out of the user's Claude and Codex homes. */
13011302
skillsLibraryEnabled: Schema.Boolean.pipe(Schema.withDecodingDefault(Effect.succeed(false))),
13021303
/** Off by default: each settled run in an opted-in project costs a text generation call. */
@@ -1749,6 +1750,7 @@ export const ServerSettingsPatch = Schema.Struct({
17491750
),
17501751
enableAgentDeviceAccess: Schema.optionalKey(Schema.Boolean),
17511752
pstackEnabled: Schema.optionalKey(Schema.Boolean),
1753+
freshSessionOnColdReturn: Schema.optionalKey(Schema.Boolean),
17521754
skillsLibraryEnabled: Schema.optionalKey(Schema.Boolean),
17531755
enableAssistedLearning: Schema.optionalKey(Schema.Boolean),
17541756
enableConversationSummaries: Schema.optionalKey(Schema.Boolean),

0 commit comments

Comments
 (0)