From 957fb650602d00dcbc97a49e64d48e308e12c1b6 Mon Sep 17 00:00:00 2001 From: Daniel Petty Date: Thu, 5 Nov 2026 17:13:16 +0000 Subject: [PATCH 1/2] fix(ai): a hosted gateway call is priced, not counted local A hosted gateway group sets no api_base, so its reply names a deployment model and no backend. is_local now treats such a reply of the gateway provider as hosted, and the lifetime report groups the served usage by the deployment model. The warning for an unpriced hosted model names the command that sets a price. Refs #3656 --- .../.openspec.yaml | 2 + .../proposal.md | 31 +++++++ .../specs/ai-spend-pricing/spec.md | 27 ++++++ .../3656-hosted-gateway-priced-local/tasks.md | 18 ++++ src/devops_cli/ai/spend/ledger.py | 13 ++- src/devops_cli/ai/spend/pricing.py | 25 ++++-- src/devops_cli/lang/en/messages.py | 3 +- tests/ai/benchmark/test_k8s_llm_gateway.py | 13 +++ tests/ai/spend/test_ai_spend_ledger.py | 25 ++++++ tests/ai/spend/test_ai_spend_pricing.py | 84 ++++++++++++++++++- 10 files changed, 228 insertions(+), 13 deletions(-) create mode 100644 openspec/changes/3656-hosted-gateway-priced-local/.openspec.yaml create mode 100644 openspec/changes/3656-hosted-gateway-priced-local/proposal.md create mode 100644 openspec/changes/3656-hosted-gateway-priced-local/specs/ai-spend-pricing/spec.md create mode 100644 openspec/changes/3656-hosted-gateway-priced-local/tasks.md diff --git a/openspec/changes/3656-hosted-gateway-priced-local/.openspec.yaml b/openspec/changes/3656-hosted-gateway-priced-local/.openspec.yaml new file mode 100644 index 000000000..1491b921e --- /dev/null +++ b/openspec/changes/3656-hosted-gateway-priced-local/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-11-05 diff --git a/openspec/changes/3656-hosted-gateway-priced-local/proposal.md b/openspec/changes/3656-hosted-gateway-priced-local/proposal.md new file mode 100644 index 000000000..99fe64830 --- /dev/null +++ b/openspec/changes/3656-hosted-gateway-priced-local/proposal.md @@ -0,0 +1,31 @@ +# Proposal + +## Why + +Issue #3656: a call to a hosted gateway group (`claude-sonnet-5-5`, `claude-haiku-5-5`, `deepseek-v4.1-flash-1017`) is recorded as a local call at $0, and its tokens count as local savings. The ledger of the owner holds 354 such rows. The hosted groups set no `api_base`, so the gateway reply names no backend, and `is_local` then decides from the server that the call was sent to. That server is the gateway, a local host. The reply does name the deployment model (`x-litellm-model-name`), and `is_local` does not read it. + +A second gap: genai-prices holds no price for `deepseek-v4.1-flash-1017`. The warning for it does not say how to set a price. + +## Feasibility + +- The local ledger (`.data/ai/spend.db`, 2026-11-05) groups the rows with a deployment model that starts with `anthropic/` or `openrouter/`: 354 rows, `served_by` NULL in all, `pricing_source` `local` in all. Every row with a `served_by` is a cluster backend. The query is in issue #3656. +- `genai_prices.calc_price` (genai-prices 0.1.13) prices `claude-sonnet-5-5` and `claude-haiku-5-5` at the provider `anthropic` (2.0 and 10.0 USD per million tokens for Sonnet, the prices of the gateway group), and `gemini-3.8-flash` at the provider `google`. It raises `LookupError` for `deepseek-v4.1-flash-1017` under the providers `litellm` and `openrouter`. +- The upstream data (`https://raw.githubusercontent.com/pydantic/genai-prices/refs/heads/main/prices/new_data/v2/data.json`, 2026-11-05) lists `deepseek/deepseek-v4.1-flash` under `openrouter` and no `-1017` snapshot. Its price (0.0352 to 0.0448 USD input, 0.08 USD output per million) is not the price of the group (0.05 and 0.192 in `k8s/llm/gateway/config.yaml`), so a lookup by the base name would price the group wrongly. +- `devops ai gateway render` and `tests/ai/benchmark/test_k8s_llm_gateway.py` already treat a deployment with no `api_base` as hosted. + +## Existing Tool Considered + +genai-prices prices the Claude and Gemini groups once the ledger asks it. For the DeepSeek group the open mechanism is the price override that the repository already has (`devops ai cost set-price`), keyed by the deployment model. The change builds no price table and no model-name matching. It makes the warning name the command. It deletes no code. + +## What Changes + +- `is_local` takes `served_model`. A reply of the `gateway` provider that names a deployment model and no backend is hosted. `calculate_request_cost` passes the deployment model, and the lifetime report groups the served usage by it, so a hosted group no longer counts in local savings or sets `local` on its tokens. +- The warning for an unpriced hosted model adds the command `devops ai cost set-price `. +- `tests/ai/benchmark/test_k8s_llm_gateway.py` fails when an in-cluster deployment sets no `api_base`: the ledger would count its calls as hosted. +- The spec delta adds two requirements to `ai-spend-pricing`. +- The change fits one pull request. + +## Impact + +- Rows that the ledger recorded before this change keep the source `local`. No migration: a row records what the process knew when it wrote it. +- The DeepSeek group has the source `unknown` and a $0 cost until a person sets a price with the command that the warning names. The three known prices differ (the gateway config, the OpenRouter list, genai-prices), so the change chooses none. diff --git a/openspec/changes/3656-hosted-gateway-priced-local/specs/ai-spend-pricing/spec.md b/openspec/changes/3656-hosted-gateway-priced-local/specs/ai-spend-pricing/spec.md new file mode 100644 index 000000000..36002f030 --- /dev/null +++ b/openspec/changes/3656-hosted-gateway-priced-local/specs/ai-spend-pricing/spec.md @@ -0,0 +1,27 @@ +## ADDED Requirements + +### Requirement: A gateway reply that names a deployment model and no backend is hosted +A reply of the `gateway` provider that names a deployment model in the `x-litellm-model-name` header and names no backend in the `x-litellm-model-api-base` header SHALL count as a hosted call, whatever host the gateway has. A reply that names a backend SHALL stay decided by that backend. The lifetime report SHALL apply the same rule to the rows of the ledger, and a row of a hosted call SHALL NOT count in local savings. + +#### Scenario: Hosted groups behind a loopback gateway +- **WHEN** replies to `claude-haiku-5-5`, `claude-sonnet-5-5` and `gemini-3.8-flash` through a gateway on `127.0.0.1` name `anthropic/claude-haiku-5-5`, `anthropic/claude-sonnet-5-5` and `gemini/gemini-3.8-flash` and no backend +- **THEN** the pricing sources are `genai_prices:anthropic`, `genai_prices:anthropic` and `genai_prices:google`, and each cost is more than $0 + +#### Scenario: A reply that names a local backend +- **WHEN** a reply names a deployment model and a cluster Ollama as its backend +- **THEN** the call is local + +#### Scenario: The report counts a hosted group as hosted +- **WHEN** the ledger holds one call that names `anthropic/claude-haiku-5-5` and no backend, and one call that names neither, both sent to `localhost:4000` through the `gateway` provider +- **THEN** the report counts one local request + +### Requirement: The warning for an unpriced hosted model names the command that sets a price +The warning that names a hosted model that neither genai-prices nor a price override prices SHALL give the command `devops ai cost set-price ` with the model of the warning. + +#### Scenario: The DeepSeek group has no price +- **WHEN** two replies name `openrouter/deepseek/deepseek-v4.1-flash-1017:floor` and no backend, and no price override exists +- **THEN** both rows have the source `unknown` and a cost of $0, and one warning names the command `devops ai cost set-price` + +#### Scenario: A price override of the deployment model +- **WHEN** a price override of `deepseek-v4.1-flash-1017:floor` is 0.05 USD input and 0.192 USD output per million tokens +- **THEN** a call of 1,000,000 prompt and 1,000,000 completion tokens costs 0.242 USD with the source `custom_override` diff --git a/openspec/changes/3656-hosted-gateway-priced-local/tasks.md b/openspec/changes/3656-hosted-gateway-priced-local/tasks.md new file mode 100644 index 000000000..b94b7204b --- /dev/null +++ b/openspec/changes/3656-hosted-gateway-priced-local/tasks.md @@ -0,0 +1,18 @@ +# Tasks + +## 1. Count a hosted gateway group as hosted + +Kind: behavior. + +Reads: `src/devops_cli/ai/spend/pricing.py` (`is_local`, `calculate_request_cost`), `src/devops_cli/ai/spend/ledger.py` (`_QUERY_SERVED_USAGE`, `_tally_and_annotate_servers`), `src/devops_cli/lang/en/messages.py` (`hosted_model_unpriced`). Changes: the same files, `tests/ai/spend/test_ai_spend_pricing.py`, `tests/ai/spend/test_ai_spend_ledger.py` and `tests/ai/benchmark/test_k8s_llm_gateway.py`. + +- [x] 1.1 Add the failing tests: `is_local` of a reply with a deployment model and no backend, the price of the Claude and Gemini groups behind a loopback gateway, the warning and the override of the DeepSeek group, the lifetime report, and the gateway config check that each local deployment sets `api_base` +- [x] 1.2 Make `is_local` read the deployment model, pass it from `calculate_request_cost` and the report, and name the command in the warning + - Verification: `uv run devops ci test tests/ai/spend tests/ai/benchmark/test_k8s_llm_gateway.py` +- [x] 1.3 Regenerate the documentation with `uv run devops docs generate --sync-readme`. No generated document holds the message, so no file changes + - Verification: `uv run devops ci docs` + +## Workflow follow-up + +- Pending a person: set the price of the group `deepseek-v4.1-flash-1017` with `devops ai cost set-price`. +- Pending a person: review the diff in tuicr (tuicr -r origin/main..HEAD). diff --git a/src/devops_cli/ai/spend/ledger.py b/src/devops_cli/ai/spend/ledger.py index 6989cb85a..b28f8b725 100644 --- a/src/devops_cli/ai/spend/ledger.py +++ b/src/devops_cli/ai/spend/ledger.py @@ -146,19 +146,21 @@ """ # The backend that served a call decides whether it ran locally, so local usage is tallied by -# server, provider and serving backend (#1631). +# server, provider and serving backend (#1631). A gateway reply that names a deployment model and +# no backend is hosted, so the tally also keeps the deployment model (#3656). _QUERY_SERVED_USAGE = """ SELECT server, provider, served_by, + served_model, COUNT(*) as req_count, COALESCE(SUM(prompt_tokens), 0) as p_tokens, COALESCE(SUM(completion_tokens), 0) as c_tokens, COALESCE(SUM(CASE WHEN request_type = :embedding THEN prompt_tokens ELSE 0 END), 0) as e_tokens FROM ai_spend_records WHERE (:cutoff IS NULL OR timestamp >= :cutoff) -GROUP BY server, provider, served_by; +GROUP BY server, provider, served_by, served_model; """ _QUERY_MODEL_BREAKDOWN = """ @@ -319,6 +321,7 @@ class _ServedUsage(NamedTuple): server: str provider: str served_by: str | None + served_model: str | None usage: _Usage @@ -339,7 +342,10 @@ def _tally_and_annotate_servers( hosted: set[tuple[str, str]] = set() for group in served: if group.server != CONST_RESPONSE_CACHE_SERVER and is_local( - server=group.server, provider=group.provider, served_by=group.served_by + server=group.server, + provider=group.provider, + served_by=group.served_by, + served_model=group.served_model, ): local.append(group.usage) else: @@ -663,6 +669,7 @@ def _query_served_usage( server=r["server"], provider=r["provider"], served_by=r["served_by"], + served_model=r["served_model"], usage=_Usage( requests=int(r["req_count"] or 0), prompt_tokens=int(r["p_tokens"] or 0), diff --git a/src/devops_cli/ai/spend/pricing.py b/src/devops_cli/ai/spend/pricing.py index f07ea3e4d..da3925a62 100644 --- a/src/devops_cli/ai/spend/pricing.py +++ b/src/devops_cli/ai/spend/pricing.py @@ -15,6 +15,7 @@ from devops_cli.ai.spend.models import ModelPricing from devops_cli.config.constants import ( + CONST_AI_GATEWAY_PROVIDER, CONST_GENAI_PRICES_LITELLM_PROVIDER, CONST_LOCAL_PROVIDER_NAMES, CONST_LOCAL_TRANSPORT_LABELS, @@ -274,26 +275,34 @@ def _is_local_gateway_configured(settings: _SettingsOnce) -> bool: def is_local( - server: str | None = None, provider: str | None = None, served_by: str | None = None + server: str | None = None, + provider: str | None = None, + served_by: str | None = None, + served_model: str | None = None, ) -> bool: """Return True if the call ran on local hardware. `served_by` is the backend that served the call, as the LLM gateway names it in its reply (`ai/client/network.py` reads the header). When the reply names it, it decides: a hosted route behind a local gateway is hosted, and a local backend behind a gateway on a public host is - local (#1631). LiteLLM names the API base of a route only when it knows it, so a gateway - route sets `api_base`. When the reply names no backend, the provider and the server the call - was sent to decide. The call loads the settings at most once, and only for a host that its - address and its name do not decide (#1420). + local (#1631). LiteLLM names the API base of a route only when it knows it, so a local route + sets `api_base` and a hosted group sets none. So a gateway reply that names a deployment + model, `served_model`, and no backend is hosted, whatever host the gateway has (#3656). When + the reply names neither, the provider and the server the call was sent to decide. The call + loads the settings at most once, and only for a host that its address and its name do not + decide (#1420). """ settings = _SettingsOnce() if served_by and served_by.strip(): return _is_local_server(served_by, settings) + is_gateway = provider is not None and provider.strip().lower() == CONST_AI_GATEWAY_PROVIDER + if is_gateway and served_model and served_model.strip(): + return False if _is_local_provider(provider): return True if _is_local_server(server, settings): return True - if provider and provider.strip().lower() == "gateway" and not server: + if is_gateway and not server: return _is_local_gateway_configured(settings) return False @@ -584,7 +593,9 @@ def calculate_request_cost( it and not at `model`, the route name that the client asked for, which genai-prices does not know. genai-prices takes the provider from the prefix of the reference (#2031). """ - if is_local(server=server, provider=provider, served_by=served_by): + if is_local( + server=server, provider=provider, served_by=served_by, served_model=served_model + ): return RequestCost(0.0, CONST_PRICING_SOURCE_LOCAL) if served_model: model, provider = served_model, CONST_GENAI_PRICES_LITELLM_PROVIDER diff --git a/src/devops_cli/lang/en/messages.py b/src/devops_cli/lang/en/messages.py index 645ba6aa7..64f3a3b34 100644 --- a/src/devops_cli/lang/en/messages.py +++ b/src/devops_cli/lang/en/messages.py @@ -301,7 +301,8 @@ class AIMessages: ) hosted_model_unpriced: str = ( "genai-prices and the price overrides hold no price for model {model} under provider " - "{provider}. Its requests are recorded at $0 with the pricing source unknown." + "{provider}. Its requests are recorded at $0 with the pricing source unknown. To set a " + "price, run 'devops ai cost set-price {model} '." ) reference_model_unpriced: str = ( "genai-prices and the price overrides hold no price for {model} ({setting}) under " diff --git a/tests/ai/benchmark/test_k8s_llm_gateway.py b/tests/ai/benchmark/test_k8s_llm_gateway.py index f57b72141..e3db44e2a 100644 --- a/tests/ai/benchmark/test_k8s_llm_gateway.py +++ b/tests/ai/benchmark/test_k8s_llm_gateway.py @@ -2042,6 +2042,19 @@ def _hosted_violations(config: dict[str, Any], deployment: dict[str, Any]) -> li return problems +def test_every_local_deployment_names_its_backend_so_the_ledger_counts_it_as_local() -> None: + """A gateway reply names a deployment model and no backend only for a deployment with no + `api_base`, and the spend ledger counts such a call as hosted (#3656). So each deployment of + the in-cluster groups sets `api_base`, and each hosted group sets none.""" + config = _full_gateway_config() + + no_backend = { + m["model_name"] for m in config["model_list"] if not m["litellm_params"].get("api_base") + } + + assert no_backend == set(HOSTED_GROUPS) + + def test_the_hosted_groups_call_providers_with_optional_keys_and_join_no_pool() -> None: """The hosted groups follow their rules (#3113).""" deployment = _load_kind(GATEWAY_DIR / "deployment.yaml", "Deployment") diff --git a/tests/ai/spend/test_ai_spend_ledger.py b/tests/ai/spend/test_ai_spend_ledger.py index 475a0ecbc..f8adf3bd8 100644 --- a/tests/ai/spend/test_ai_spend_ledger.py +++ b/tests/ai/spend/test_ai_spend_ledger.py @@ -1036,6 +1036,31 @@ def test_the_report_counts_a_gateway_call_as_local_by_the_backend_that_served_it ) == (1, 1_000_000, 2.5, [("localhost:4000", False)]) +def test_the_report_counts_a_hosted_group_that_names_no_backend_as_hosted(tmp_path: Path) -> None: + """A hosted group sets no `api_base`, so its reply names a deployment model and no backend. + Of two such calls through one loopback gateway, the call of a model with a deployment model + is hosted and does not count in local savings, and the call that names none stays with the + host that decides.""" + ledger = SpendLedger(db_path=tmp_path / "spend.db") + for model, served_model, cost in ( + ("claude-haiku-5-5", "anthropic/claude-haiku-5-5", 0.6), + ("devops-chat", None, 0.0), + ): + ledger.record_request( + provider="gateway", + server="localhost:4000", + served_model=served_model, + model=model, + prompt_tokens=1_000_000, + completion_tokens=100_000, + cost_usd=cost, + ) + + report = ledger.get_lifetime_report(reference_model="gpt-4o") + + assert (report.local_requests, report.local_prompt_tokens) == (1, 1_000_000) + + def _local_chat_and_embedding_ledger(db_path: Path) -> SpendLedger: """A ledger with one local chat request and one local embedding request, of 1M tokens each.""" ledger = SpendLedger(db_path=db_path) diff --git a/tests/ai/spend/test_ai_spend_pricing.py b/tests/ai/spend/test_ai_spend_pricing.py index 4207d8b5a..e186771fe 100644 --- a/tests/ai/spend/test_ai_spend_pricing.py +++ b/tests/ai/spend/test_ai_spend_pricing.py @@ -721,6 +721,86 @@ def test_a_gateway_call_is_local_by_the_backend_that_served_it( ] == [True, True, False, True, False] +def test_a_gateway_reply_that_names_a_deployment_model_and_no_backend_is_hosted( + config: Callable[..., Path], +) -> None: + """Verify a hosted group sets no `api_base`, so its reply names a deployment model and no + backend. That call is hosted behind a loopback gateway, a configured gateway host and a + gateway with no server, while a reply that names a local backend stays local.""" + from devops_cli.ai.spend.pricing import is_local + + config(k8s={"domain": "example.com"}, ai={"gateway_url": "https://gw.example.com/v1"}) + claude = "anthropic/claude-haiku-5-5" + cluster_ollama = "http://ollama-16gib.llm.svc.cluster.local:11434" + + assert [ + is_local("localhost:4000", "gateway", served_model=claude), + is_local("gw.example.com", "gateway", served_model=claude), + is_local(None, "gateway", served_model=claude), + is_local("localhost:4000", "gateway", served_by=cluster_ollama, served_model=claude), + is_local("localhost:4000", "gateway", served_model=" "), + ] == [False, False, False, True, True] + + +@pytest.mark.parametrize( + ("model", "served_model", "source"), + [ + ("claude-haiku-5-5", "anthropic/claude-haiku-5-5", "genai_prices:anthropic"), + ("claude-sonnet-5-5", "anthropic/claude-sonnet-5-5", "genai_prices:anthropic"), + ("gemini-3.8-flash", "gemini/gemini-3.8-flash", "genai_prices:google"), + ], +) +def test_a_hosted_group_behind_a_loopback_gateway_is_priced_by_genai_prices( + tmp_path: Path, model: str, served_model: str, source: str +) -> None: + """Verify the Claude and Gemini groups of the gateway cost money: the reply names the + deployment model and no backend, and genai-prices prices the model.""" + registry = PricingRegistry(data_dir=tmp_path) + + cost = registry.calculate_request_cost( + model, + 1_000_000, + 100_000, + server="127.0.0.1:4200", + provider="gateway", + served_model=served_model, + ) + + assert (cost.pricing_source, cost.cost_usd > 0) == (source, True) + + +def test_a_hosted_group_that_nothing_prices_warns_with_the_command_that_sets_a_price( + tmp_path: Path, caplog: pytest.LogCaptureFixture +) -> None: + """Verify the DeepSeek group, which genai-prices does not list, is hosted at $0 with the + source `unknown`, and that its one warning names the command that sets a price, and a price + override of the deployment model prices it.""" + deepseek = "openrouter/deepseek/deepseek-v4.1-flash-1017:floor" + registry = PricingRegistry(data_dir=tmp_path) + call = { + "model": "deepseek-v4.1-flash-1017", + "server": "127.0.0.1:4200", + "provider": "gateway", + "served_model": deepseek, + } + + with caplog.at_level(logging.WARNING, logger="devops_cli.ai.spend.pricing"): + unpriced = [registry.calculate_request_cost(**call, prompt_tokens=10, completion_tokens=1)] + unpriced.append( + registry.calculate_request_cost(**call, prompt_tokens=10, completion_tokens=1) + ) + registry.set_custom_pricing("deepseek-v4.1-flash-1017:floor", 0.05, 0.192) + priced = registry.calculate_request_cost( + **call, prompt_tokens=1_000_000, completion_tokens=1_000_000 + ) + + assert ( + [(c.cost_usd, c.pricing_source) for c in unpriced], + [r.getMessage().count("devops ai cost set-price") for r in caplog.records], + (priced.cost_usd, priced.pricing_source), + ) == ([(0.0, "unknown")] * 2, [1], (0.242, "custom_override")) + + def _settings_loads_of(call: Callable[[], object]) -> int: """The number of times `call` runs `load_settings`, counted by a profile hook, so the test patches no devops-cli name (#1420).""" @@ -885,7 +965,7 @@ def price(model: str, server: str, provider: str) -> RequestCost: RequestCost(2.5, "genai_prices:openai"), RequestCost(0.0, "local"), [ - "genai-prices and the price overrides hold no price for model gpt-9-preview under provider openai. Its requests are recorded at $0 with the pricing source unknown." + "genai-prices and the price overrides hold no price for model gpt-9-preview under provider openai. Its requests are recorded at $0 with the pricing source unknown. To set a price, run 'devops ai cost set-price gpt-9-preview '." ], ) @@ -942,7 +1022,7 @@ def price(served_model: str | None, served_by: str = "https://api.openai.com") - ], RequestCost(0.0, "local"), [ - "genai-prices and the price overrides hold no price for model openai/gpt-9-preview under provider litellm. Its requests are recorded at $0 with the pricing source unknown." + "genai-prices and the price overrides hold no price for model openai/gpt-9-preview under provider litellm. Its requests are recorded at $0 with the pricing source unknown. To set a price, run 'devops ai cost set-price openai/gpt-9-preview '." ], ) From 6d27adcbaab55b5797213e0d47196ff9e9d5b934 Mon Sep 17 00:00:00 2001 From: Daniel Petty Date: Thu, 5 Nov 2026 17:15:49 +0000 Subject: [PATCH 2/2] docs(openspec): archive 3656-hosted-gateway-priced-local --- .../.openspec.yaml | 0 .../proposal.md | 0 .../specs/ai-spend-pricing/spec.md | 0 .../tasks.md | 0 openspec/specs/ai-spend-pricing/spec.md | 26 +++++++++++++++++++ 5 files changed, 26 insertions(+) rename openspec/changes/{3656-hosted-gateway-priced-local => archive/2026-11-05-3656-hosted-gateway-priced-local}/.openspec.yaml (100%) rename openspec/changes/{3656-hosted-gateway-priced-local => archive/2026-11-05-3656-hosted-gateway-priced-local}/proposal.md (100%) rename openspec/changes/{3656-hosted-gateway-priced-local => archive/2026-11-05-3656-hosted-gateway-priced-local}/specs/ai-spend-pricing/spec.md (100%) rename openspec/changes/{3656-hosted-gateway-priced-local => archive/2026-11-05-3656-hosted-gateway-priced-local}/tasks.md (100%) diff --git a/openspec/changes/3656-hosted-gateway-priced-local/.openspec.yaml b/openspec/changes/archive/2026-11-05-3656-hosted-gateway-priced-local/.openspec.yaml similarity index 100% rename from openspec/changes/3656-hosted-gateway-priced-local/.openspec.yaml rename to openspec/changes/archive/2026-11-05-3656-hosted-gateway-priced-local/.openspec.yaml diff --git a/openspec/changes/3656-hosted-gateway-priced-local/proposal.md b/openspec/changes/archive/2026-11-05-3656-hosted-gateway-priced-local/proposal.md similarity index 100% rename from openspec/changes/3656-hosted-gateway-priced-local/proposal.md rename to openspec/changes/archive/2026-11-05-3656-hosted-gateway-priced-local/proposal.md diff --git a/openspec/changes/3656-hosted-gateway-priced-local/specs/ai-spend-pricing/spec.md b/openspec/changes/archive/2026-11-05-3656-hosted-gateway-priced-local/specs/ai-spend-pricing/spec.md similarity index 100% rename from openspec/changes/3656-hosted-gateway-priced-local/specs/ai-spend-pricing/spec.md rename to openspec/changes/archive/2026-11-05-3656-hosted-gateway-priced-local/specs/ai-spend-pricing/spec.md diff --git a/openspec/changes/3656-hosted-gateway-priced-local/tasks.md b/openspec/changes/archive/2026-11-05-3656-hosted-gateway-priced-local/tasks.md similarity index 100% rename from openspec/changes/3656-hosted-gateway-priced-local/tasks.md rename to openspec/changes/archive/2026-11-05-3656-hosted-gateway-priced-local/tasks.md diff --git a/openspec/specs/ai-spend-pricing/spec.md b/openspec/specs/ai-spend-pricing/spec.md index b865e6a94..6d312e5ed 100644 --- a/openspec/specs/ai-spend-pricing/spec.md +++ b/openspec/specs/ai-spend-pricing/spec.md @@ -158,3 +158,29 @@ When a chat call that returns one reply fails after its status line, because of #### Scenario: An error frame after a ping - **WHEN** a chat call gets a stream with a `: ping` comment and then an error frame with `code` `"500"` - **THEN** the spend ledger has one row with the request type `chat_dispatch` and finish reason `error` + +### Requirement: A gateway reply that names a deployment model and no backend is hosted +A reply of the `gateway` provider that names a deployment model in the `x-litellm-model-name` header and names no backend in the `x-litellm-model-api-base` header SHALL count as a hosted call, whatever host the gateway has. A reply that names a backend SHALL stay decided by that backend. The lifetime report SHALL apply the same rule to the rows of the ledger, and a row of a hosted call SHALL NOT count in local savings. + +#### Scenario: Hosted groups behind a loopback gateway +- **WHEN** replies to `claude-haiku-5-5`, `claude-sonnet-5-5` and `gemini-3.8-flash` through a gateway on `127.0.0.1` name `anthropic/claude-haiku-5-5`, `anthropic/claude-sonnet-5-5` and `gemini/gemini-3.8-flash` and no backend +- **THEN** the pricing sources are `genai_prices:anthropic`, `genai_prices:anthropic` and `genai_prices:google`, and each cost is more than $0 + +#### Scenario: A reply that names a local backend +- **WHEN** a reply names a deployment model and a cluster Ollama as its backend +- **THEN** the call is local + +#### Scenario: The report counts a hosted group as hosted +- **WHEN** the ledger holds one call that names `anthropic/claude-haiku-5-5` and no backend, and one call that names neither, both sent to `localhost:4000` through the `gateway` provider +- **THEN** the report counts one local request + +### Requirement: The warning for an unpriced hosted model names the command that sets a price +The warning that names a hosted model that neither genai-prices nor a price override prices SHALL give the command `devops ai cost set-price ` with the model of the warning. + +#### Scenario: The DeepSeek group has no price +- **WHEN** two replies name `openrouter/deepseek/deepseek-v4.1-flash-1017:floor` and no backend, and no price override exists +- **THEN** both rows have the source `unknown` and a cost of $0, and one warning names the command `devops ai cost set-price` + +#### Scenario: A price override of the deployment model +- **WHEN** a price override of `deepseek-v4.1-flash-1017:floor` is 0.05 USD input and 0.192 USD output per million tokens +- **THEN** a call of 1,000,000 prompt and 1,000,000 completion tokens costs 0.242 USD with the source `custom_override`