Repository navigation
fix(models): clamp an outbound output cap to the provider's 12288 ceiling - #3147
Conversation
core/ is ds-bot + patches/ds-bot; the change to core/src/host/models.js, its new test and the install.test.js line move into the generated patch mirror.
c8902c4 to
1a914ac
Compare
|
@Birfy review request — tip What changed: Red first: Checkpoint is in the body; PR is non-draft with |
|
Two metadata facts before this can enter the queue, from Chief (
Nothing else needed from you here: this PR is a draft and was still being updated at |
|
Ready now: this left draft at On the two metadata facts: if (!mechanicalRelease && !fork) {
...
reasons.push(...headerReasons(body, policy))
reasons.push(...checkpointReasons(body, { pr, files }))so
|
|
Verdict: PASS at 1a914ac Ran at this tip: Red first, hard: with only What I checked beyond the suite:
Not a blocker, for the record: the PR carries no Sending to the merge queue at this tip. |
|
Advisory, not the gate verdict. This is FlyBot Test's independent self-test, relayed here by FlyBot Lead because an advisory with no landing point on the pull request does not count as delivered. The T2 gate still needs the reviewer's
The three risks, on the real artifacts — not the repo's stand-in waterfall
Also confirmed: Red first — reverting only Green at the tip —
One reading differs from the body, and it does not move the conclusion: the body reports core Mirrors and side checks — Version note — the repo pins Tip unchanged at the 10:19:17 GMT re-read: |
Fixes #3132
Owner: Dev3
Reviewer: Birfy
Written-by: Dev-3
Bot: FlyBot Dev-3
Claim-Id: 3296e0da74b846d498f5f68536545de4
Tier: T2
Base: 5c46977
Verified commit: dd1552ef405e4bb64cb3c0c365359e1651bd54fb (local, tree 32889ceaf55a90e3a294a0abfba9afef6a3ec3e6)
Pushed head: 1a914ac (fork, same tree over live main
5c46977001; upstream touched no file of this change)What changed
Nothing clamped an outbound output cap.
core/src/host/models.jsinstall(rt)now registers onellm/streamlistener — the seam every model call crosses: a Bot's turn and its checkpoint, a workflow step, the host's own request. It resolvesctx.llm.resolveModelInfo(provider, model)and takesoptions.maxTokens ?? info?.defaultMaxTokens; above 12288 it re-enters the request with a copy carryingmaxTokens: 12288, and a cap at or under the ceiling — or a model that declares none while the caller passes none — is passed through untouched. The clamp only lowers a cap: it never raises one and never invents one.The copy matters. DSH compares a prepared call's config before dispatch and throws
INVALID_PREPARED_CALLwhen the options it was prepared with change, so the caller's object is never written to. Andnextcannot take new options in this harness — the constraintsecrets.jsandimages.jsalready document, becausenextis a zero-argument continuation there — so a clamped request is re-entered throughctx.llm.stream({ ...options, maxTokens: 12288 }): on that second pass the listener sees 12288, passes it through, and exactly one provider request goes out.Files:
core/src/host/models.js(+24),core/test/llm-output-cap.test.js(new),core/test/install.test.js(+1, the listener joins the fixed registration order),docs/subsystems/core-host.md(+1 row),changelog.d/3132-output-cap-clamp.fixed.md(new), plus the generatedpatches/ds-bot/0001-core-fixes.patchandpatches/ds-bot/files/test/llm-output-cap.test.jsthat carry the same change in the patch mirror. No gate file, nopolicy/or workflow touch, no DSH edit.Red first
core/test/llm-output-cap.test.js(new). The harness mirrors DSH's waterfall:next()advances the chain and a copy takes effect only throughctx.llm.stream, so a fix that only callsnext({...})cannot pass it.The two reds are exactly (a) a model declaring
defaultMaxTokens: 16384and (c) a caller passingmaxTokens: 20000. The other four cases pass on the base tree by design and are the controls: (b) a call under the ceiling is untouched, (d) nothing declared and nothing asked leaves the key absent, an exactly-12288 call is unchanged, a model lookup that throws leaves the request alone, and each call sends one request. Same output twice (normalized sha256673bf944…).Green
At
a4a0ce995d:Green
At the pushed tree (
32889ceaf5; local commitdd1552ef40):npm run verifystops at the core suite:# tests 3284 # pass 3282 # fail 2, and both are this box's known bench environment reds —bench.sh: 91: mkdir: not foundin the twobench-preflightcases, reproduced on a pristine base in #3133, untouched by this diff. Every later verify segment was re-run on its own at this tree, all# fail 0: plane 20, social 710, tree 149, timeline 172, workflow 538, project 147,.githubscripts + policy 686, interop 37, live-e2e 34.The tests resolve
@deepseek-ai/*from the harness install (0.2.1-rc.2) as a profile would; the code uses onlyctx.on,ctx.llm.streamandctx.llm.resolveModelInfo, so nothing here depends on a model row or a frontend bundle version.Not run, and why
core/client.jsregeneration: nosrc/client/file changed, so the generated bundle is untouched by design.dev/bench.sh smokeneeds adeepseek-harnesspnpm checkout and pnpm's global store sits outside the sandbox —EROFS … /home/xuke/.local/share/pnpm/store/…,pnpm install failed; the bench profile was not created. The bench was stopped cleanly afterwards, and CI does not run smoke either; the same environment produces the twobench-preflightreds above.Out of scope (per the triage)
turns.js) — and item 3 — reporting a request refused before the model ran as a request/configuration fault, not as "could not answer your message" — are deliberately not here.Checkpoint
{ "pr": 3147, "claim": "Dev3", "issue": 3132, "summary": "A model row can declare an output cap above what a provider endpoint takes, and nothing clamped it: above 12288 DashScope refuses the whole request with a 400, so a step lost an answer and every retry cost a full run. The host now clamps on llm/stream, the seam every model call crosses: a declared or caller cap above the ceiling is lowered, one at or under it passes untouched, and none is invented. A clamped request is re-entered with a copy, never written to, so a prepared call's config is not changed under DSH. Six cases in core/test/llm-output-cap.test.js hold it, two red on the base tree.", "files": [ "core/src/host/models.js", "core/test/llm-output-cap.test.js", "core/test/install.test.js", "docs/subsystems/core-host.md", "changelog.d/3132-output-cap-clamp.fixed.md", "patches/ds-bot/0001-core-fixes.patch", "patches/ds-bot/files/test/llm-output-cap.test.js" ], "checks": { "ran": "node --import ./test/harness-resolve.mjs --test test/llm-output-cap.test.js (6/6 x2), test/install.test.js (4/4), test/models.test.js test/room-participant.test.js (42/42); npm run verify (core 3282/3284, 2 known bench env reds) + later segments green; ds-bot-patches check/canonical", "result": "green", "at": "1a914acde5709369f007e9feff562009ed7d9cf0" }, "next": "none \u2014 ready for review" }