Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,9 @@ The load-bearing consequence: the provider id **is** CoDev's authorship marker f

Every agent CoDev configures has to be *told* the window of the model it's talking to. The gateway serves custom models none of them recognize, and each guesses differently when unconfigured: Codex assumes a 272K fallback, OpenCode assumes context `0` (which disables compaction outright), Continue falls back to a generic default. `src/lib/model-limits.ts` is the single source of truth; the four writers in `configure.ts` translate it into each agent's own knob and hold no window constants of their own. The flat `GATEWAY_CONTEXT_WINDOW` / `GATEWAY_COMPACT_*` constants that used to live in `const.ts` are gone — they encoded the assumption that every gateway model shares one 196608-token window, which stopped being true once the gateway served both a 1M-token and a 200K-token model.

`ModelLimits` is `{ context, trigger, output? }`: the true window, the absolute token count where auto-compaction should fire, and an optional output cap. **`trigger` is explicit rather than a percentage** — the gap between window and trigger is a per-model judgement call, not a constant. `limitsFor(modelId)` resolves **remote → table → `DEFAULT_LIMITS`**, where remote is the gateway's own numbers cached in auth.json and `DEFAULT_LIMITS` (200K/160K) covers anything unrecognized. The 200K sibling of the 1M model is deliberately *absent* from the table: the default already describes it, and an entry that merely restates the default is one more thing to keep in sync. That 1M model's id lives in `model-limits.ts` as `M3_ID`, `atob`-encoded for the same reason `FALLBACK_MODEL` is in `const.ts` — the upstream vendor's name stays out of the shipped bundle, so keep it that way when adding table entries.
`ModelLimits` is `{ context, trigger, output? }`: the true window, the absolute token count where auto-compaction should fire, and an optional output cap. **The table carries windows only** — `limitsFromWindow` derives the trigger as `COMPACT_PCT` (90%) of the window, whatever the window's source, so adding a model means adding one number. `limitsFor(modelId)` resolves **remote → table → `DEFAULT_LIMITS`**, where remote is the gateway's own numbers cached in auth.json and `DEFAULT_LIMITS` (200K/180K) covers anything unrecognized. A model the deployment serves at that same 200K window needs no entry: the default already describes it, and an entry that merely restates the default is one more thing to keep in sync. Model ids in the table are plain strings (#259 dropped the `atob`-encoded `M3_ID`); `FALLBACK_MODEL` in `const.ts` is still encoded and stays that way.

**A table entry has a twin in [codev-code](https://github.com/quickbeard/codev-code) (`packages/codev-gateway/src/model-limits.ts`).** CoDev Code's own gateway sign-in configures itself without going through the hub, so it carries its own copy of the same table. The two are deliberately separate — the hub must keep configuring agents on machines with an older CoDev Code — but a model added to one and not the other gets different windows depending on which flow wrote the config. Add it to both.

Each agent takes a different shape, and the differences are the whole reason this module exists:

Expand Down
11 changes: 6 additions & 5 deletions src/lib/model-limits.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,14 @@ export const DEFAULT_LIMITS: ModelLimits = limitsFromWindow(DEFAULT_WINDOW);
// Known gateway models' context windows. Keyed by the exact id `/v1/models`
// reports, which is what lands in every agent config.
//
// M3's window is capped at 262144 on the current deployment (the model
// supports more, the gateway's resources don't). The 200K sibling is
// deliberately absent: DEFAULT_WINDOW already describes it correctly, and an
// entry that merely restates the default is one more thing to keep in sync.
// M3's and GLM-5.3-Flash's windows are capped at 262144 on the current
// deployment (the models support more, the gateway's resources don't). Models
// served at the 200K default are deliberately absent: DEFAULT_WINDOW already
// describes them correctly, and an entry that merely restates the default is
// one more thing to keep in sync.
const TABLE: Record<string, number> = {
"MiniMax/MiniMax-M3": 262144,
"zai-org/GLM-4.7-cc": 200000,
"zai-org/GLM-5.3-Flash": 262144,
};

// Windows reported by the gateway, cached in auth.json by the model-choice
Expand Down
18 changes: 12 additions & 6 deletions tests/lib/configure.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,8 @@ describe("configureClaudeCode", () => {
expect(read().CLAUDE_CODE_AUTO_COMPACT_WINDOW).toBe("200000");
expect(read().CLAUDE_AUTOCOMPACT_PCT_OVERRIDE).toBe("80");

configureClaudeCode({ apiKey: "sk", model: "zai-org/GLM-4.7-cc" });
// And a model on the 200K default lands on the same two numbers.
configureClaudeCode({ apiKey: "sk", model: "other/model" });
expect(read().CLAUDE_CODE_AUTO_COMPACT_WINDOW).toBe("200000");
expect(read().CLAUDE_AUTOCOMPACT_PCT_OVERRIDE).toBe("80");
});
Expand Down Expand Up @@ -459,7 +460,7 @@ describe("configureOpenCode", () => {
configureOpenCode({
apiKey: "sk-xyz",
model: "MiniMax/MiniMax-M3",
models: ["MiniMax/MiniMax-M3", "zai-org/GLM-4.7-cc"],
models: ["MiniMax/MiniMax-M3", "other/model"],
});

const filePath = join(tempDir, ".config", "opencode", "opencode.json");
Expand All @@ -469,13 +470,13 @@ describe("configureOpenCode", () => {

// True windows, so the TUI's "% context used" gauge stays honest.
expect(map["MiniMax/MiniMax-M3"].limit.context).toBe(262144);
expect(map["zai-org/GLM-4.7-cc"].limit.context).toBe(200000);
expect(map["other/model"].limit.context).toBe(200000);
// ...and each fires where it should, off one shared reserve. Both
// models' 90% target + reserve exceeds their window, so declaredInput's
// clamp pins input at the window — triggers stay per-model and never
// past the ceiling.
expect(map["MiniMax/MiniMax-M3"].limit.input - reserved).toBe(222144);
expect(map["zai-org/GLM-4.7-cc"].limit.input - reserved).toBe(160000);
expect(map["other/model"].limit.input - reserved).toBe(160000);
});

test("writes every fetched model into the provider's models map", async () => {
Expand Down Expand Up @@ -962,7 +963,12 @@ describe("configureCodex", () => {
expect(readCodexToml().model_context_window).toBe(262144);
expect(readCodexToml().model_auto_compact_token_limit).toBe(235930);

configureCodex({ apiKey: "sk-codex", model: "zai-org/GLM-4.7-cc" });
configureCodex({ apiKey: "sk-codex", model: "zai-org/GLM-5.3-Flash" });
expect(readCodexToml().model_context_window).toBe(262144);
expect(readCodexToml().model_auto_compact_token_limit).toBe(235930);

// A model with no table entry keeps the 200K default's own numbers.
configureCodex({ apiKey: "sk-codex", model: "other/model" });
expect(readCodexToml().model_context_window).toBe(200000);
expect(readCodexToml().model_auto_compact_token_limit).toBe(180000);
});
Expand Down Expand Up @@ -1100,7 +1106,7 @@ describe("configureContinue", () => {
configureContinue({
apiKey: "sk",
model: "MiniMax/MiniMax-M3",
models: ["MiniMax/MiniMax-M3", "zai-org/GLM-4.7-cc"],
models: ["MiniMax/MiniMax-M3", "other/model"],
});

const raw = readContinueYaml();
Expand Down
21 changes: 11 additions & 10 deletions tests/lib/model-limits.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,9 @@ describe("limitsFor", () => {
context: 262144,
trigger: 235930,
});
expect(limitsFor("zai-org/GLM-4.7-cc")).toEqual({
context: 200000,
trigger: 180000,
expect(limitsFor("zai-org/GLM-5.3-Flash")).toEqual({
context: 262144,
trigger: 235930,
});
});

Expand Down Expand Up @@ -81,9 +81,9 @@ describe("limitsFor", () => {

test("a model the gateway says nothing about still gets the table entry", () => {
writeCachedLimits({ "other/model": { context: 12345, trigger: 9876 } });
expect(limitsFor("zai-org/GLM-4.7-cc")).toEqual({
context: 200000,
trigger: 180000,
expect(limitsFor("zai-org/GLM-5.3-Flash")).toEqual({
context: 262144,
trigger: 235930,
});
});

Expand Down Expand Up @@ -148,7 +148,9 @@ describe("claudeWindow / claudeCompactPct", () => {
// 235930/262144 would be 90, but 235930/200000 is 118 — the raw ratio is
// meaningless once the window is clamped, so it must be bounded.
expect(claudeCompactPct(limitsFor("MiniMax/MiniMax-M3"))).toBe(80);
expect(claudeCompactPct(limitsFor("zai-org/GLM-4.7-cc"))).toBe(80);
// A model already at the ceiling is capped by the same guard: its own
// 90% target is above Claude Code's 80%.
expect(claudeCompactPct(DEFAULT_LIMITS)).toBe(80);
});

test("never exceeds 80%, which Claude Code's Rzq discards anything above", () => {
Expand Down Expand Up @@ -194,11 +196,10 @@ describe("declaredInput", () => {
declaredInput({ context: 1000000, trigger: 900000 }) - COMPACT_RESERVED,
).toBe(900000);
const m3 = limitsFor("MiniMax/MiniMax-M3");
const glm = limitsFor("zai-org/GLM-4.7-cc");
expect(declaredInput(m3)).toBe(262144);
expect(declaredInput(m3) - COMPACT_RESERVED).toBe(222144);
expect(declaredInput(glm)).toBe(200000);
expect(declaredInput(glm) - COMPACT_RESERVED).toBe(160000);
expect(declaredInput(DEFAULT_LIMITS)).toBe(200000);
expect(declaredInput(DEFAULT_LIMITS) - COMPACT_RESERVED).toBe(160000);
});

test("clamps to the true window so the budget is never overstated", () => {
Expand Down
Loading