Skip to content

fix(ai): a hosted gateway call is priced, not counted local - #3663

Merged
dan-petty merged 2 commits into
mainfrom
fix/3656-hosted-gateway-priced-local
Nov 5, 2026
Merged

dan-petty merged 2 commits into
mainfrom
fix/3656-hosted-gateway-priced-local

Conversation

@dan-petty

Copy link
Copy Markdown
Owner

A call to a hosted gateway group (claude-sonnet-5-5, claude-haiku-5-5, deepseek-v4.1-flash-1017) was recorded as a local call at $0, and its tokens counted as local savings. The hosted groups set no api_base, so the gateway reply names a deployment model and no backend, and is_local judged the call by the gateway host, a local host. A reply of the gateway provider that names a deployment model and no backend is now hosted, and the lifetime report groups the served usage by the deployment model. Claude and Gemini calls are now priced by genai-prices. genai-prices holds no price for deepseek-v4.1-flash-1017, so the warning for an unpriced hosted model now names the command devops ai cost set-price. Rows written before this change keep the source local.

Closes #3656

Change: openspec/changes/archive/2026-11-05-3656-hosted-gateway-priced-local/

A hosted gateway group sets no api_base, so its reply names a deployment
model and no backend. is_local now treats such a reply of the gateway
provider as hosted, and the lifetime report groups the served usage by the
deployment model. The warning for an unpriced hosted model names the
command that sets a price.

Refs #3656
@dan-petty dan-petty added type/bug Defect, regression, or incorrect runtime behavior scope/ai AI review pipeline, personas, prompts, LLM client, or RAG labels Nov 5, 2026
@dan-petty

Copy link
Copy Markdown
Owner Author

Fail-to-pass: devops ci fail-to-pass --base origin/main reports 8 failed on base (8 failed and 143 passed in tests/ai/benchmark/test_k8s_llm_gateway.py, tests/ai/spend/test_ai_spend_ledger.py, tests/ai/spend/test_ai_spend_pricing.py; 0 errored). The CI check Fail-to-Pass Check passes.

The 8 tests: is_local of a reply with a deployment model and no backend, the price of the Claude and Gemini groups behind a loopback gateway (3 cases), the warning and the price override of the DeepSeek group, the lifetime report, and the two older tests that pin the text of the warning. test_every_local_deployment_names_its_backend_so_the_ledger_counts_it_as_local passes on base on purpose: the gateway config is already correct, and the test fails if a local deployment loses its api_base.

Full local gate (devops ci --check): lint, format, typecheck, security, docs, openspec, build and the other 19 gates pass. The test gate has 4 failures and 1 error, in tests/crypto/test_ssh.py, tests/telemetry/test_config_env_overrides.py, tests/commands/test_devcontainer_install_tools_dry_run.py and tests/ai/test_tool_executor_contract.py. They fail the same way with this change stashed, so they come from the state of the dev container (an installed popeye, the name of the checkout directory, a /tmp/config.yaml, tool install status), not from this change.

What this does not do: it sets no price for deepseek-v4.1-flash-1017. The three known prices differ (gateway config 0.05 and 0.192 USD per million tokens, OpenRouter list 0.30 and 1.20, genai-prices deepseek/deepseek-v4.1-flash 0.0352 to 0.0448 and 0.08). A person sets the price with devops ai cost set-price openrouter/deepseek/deepseek-v4.1-flash-1017:floor <prompt_rate> <completion_rate>. Until then the group has the source unknown and one warning.

Ledger rows that were written before this change keep the source local. The caching follow-up is #3658.

@dan-petty
dan-petty marked this pull request as ready for review November 5, 2026 17:24
@dan-petty

Copy link
Copy Markdown
Owner Author

Scope note. This pull request fixes the calls whose gateway reply names a deployment model: the chat_dispatch rows. The agent_run rows of the same ledger (301 rows, 20.18 M prompt tokens against 10.53 M for chat_dispatch) record no reply header, so this change does not reach them. They are #3665. Merging this pull request does not close that problem.

@dan-petty
dan-petty merged commit acdc8a9 into main Nov 5, 2026
23 checks passed
@dan-petty
dan-petty deleted the fix/3656-hosted-gateway-priced-local branch November 5, 2026 18:06
dan-petty pushed a commit that referenced this pull request Nov 5, 2026
🤖 I have created a release *beep* *boop*
---


##
[0.62.7](v0.62.6...v0.62.7)
(2026-11-05)


### Bug Fixes

* **ai:** a hosted gateway call is priced, not counted local
([#3663](#3663))
([acdc8a9](acdc8a9))
* **roadmap:** plan-check measures each task group
([#3669](#3669))
([3791bd9](3791bd9))
* **roadmap:** the jobs trust the items of the two bots
([#3664](#3664))
([41a362d](41a362d))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: dan-petty-release[bot] <346998822+dan-petty-release[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope/ai AI review pipeline, personas, prompts, LLM client, or RAG type/bug Defect, regression, or incorrect runtime behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(ai): a hosted gateway call is recorded as a local call at $0

1 participant