Skip to content

perf(usage): settle unpriced cost rows once and add cost-tracking switch - #563

Open
lztxwd6666 wants to merge 1 commit into
SaladDay:mainfrom
lztxwd6666:perf/usage-cost-maintenance
Open

lztxwd6666 wants to merge 1 commit into
SaladDay:mainfrom
lztxwd6666:perf/usage-cost-maintenance

Conversation

@lztxwd6666

Copy link
Copy Markdown

Problem

On long-lived databases, proxy_request_logs can accumulate tens of thousands of zero-cost rows whose models are not in model_pricing (new models, custom relay names, mixed casing). Every Database::init() runs backfill_missing_usage_costs(), which selects all such rows on every invocation, and for each row expands model-name candidates plus exact/prefix LIKE queries against the whole pricing table. Pricing hits are cached but misses are not, so the same missing model is re-resolved thousands of times.

With a ~28k-row backlog, common CLI actions such as list/save/switch take several seconds because a single switch initializes the DB multiple times.

Fix

  • Cache pricing misses during a backfill pass, so repeated unpriced models are only resolved once per pass.
  • Settle genuinely unpriced rows once by writing pricing_model = "unknown" and leaving the cost at 0. Full sweeps skip these settled rows, keeping startup fast.
  • Keep costs recoverable: a targeted backfill (when a pricing row is added/edited) revisits unknown rows for that model, so later pricing can backfill historical usage.
  • Add a cost-tracking switch: settings cost-tracking {show,enable,disable} plus costTrackingEnabled in settings --json. Defaults to true, so existing users and data are unaffected. When disabled, proxy logging skips cost calculation and backfills become no-ops while token counts continue to be recorded.

Tests

  • Added backfill_settles_unpriced_rows_once_and_leaves_cost_at_zero and targeted_backfill_revisits_settled_unpriced_rows
  • cargo test --lib usage_stats — 26 passed
  • cargo test --lib proxy::usage — 30 passed
  • cargo fmt --check — clean
  • cargo check — clean

cargo test --lib has 8 pre-existing environment-related failures in proxy/daemon tests; reproduced on unmodified HEAD in this WSL2 environment.

Compatibility

No schema migration. Existing databases settle their backlog on the first backfill pass. Existing custom prices and historical non-zero costs are never overwritten.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T08:53:41.598490Z a3220ca PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a3220cab6d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +1638 to +1639
"UPDATE proxy_request_logs SET pricing_model = ?1 WHERE request_id = ?2",
params![UNPRICED_MARKER, log.request_id],

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the unpriced status for session-cost projections

For a token-bearing session row with no catalog price, writing a non-empty pricing_model changes its meaning to existing readers: services/session_cost/projection.rs:186-197 treats any non-empty value as evidence of successful pricing. The Sessions overlay consequently reports Some(0.0) instead of None for an entirely unpriced session, or presents a misleading partial total for a mixed session. This transition is reproducible with the existing aggregate SQL. Store settlement separately or update the session-cost trust predicate to recognize unknown as unpriced.

Useful? React with 👍 / 👎.

Comment on lines +1548 to +1549
} else {
"AND (pricing_model IS NULL OR TRIM(pricing_model) = '')"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Revisit settled misses after built-in pricing updates

When a later release adds pricing for a model used in already-settled logs, Database::init() calls ensure_model_pricing_seeded() and then startup usage maintenance, but maintenance uses this full-sweep branch and seeding never invokes the targeted backfill. Historical Claude/Codex rows therefore retain zero costs after the catalog gains a valid price; before this change, the next startup recalculated them. Invalidate or revisit settled misses when built-in pricing changes so upgrades can recover historical costs without requiring a manual pricing edit.

Useful? React with 👍 / 👎.

match only_model_id {
Some(model) => {
let sql = format!("{BASE_SQL} AND (model = ?1 OR request_model = ?1)");
let sql = format!("{base_sql} AND (model = ?1 OR request_model = ?1)");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Match settled rows using the pricing resolver's normalization

If a settled row stores vendor/New-Model and a price is subsequently added for canonical new-model, this exact predicate selects no rows even though the pricing resolver successfully normalizes and resolves that model. The new marker also excludes the row from subsequent full sweeps, removing the startup fallback that previously recovered it. Thus even the targeted pricing-edit path leaves historical usage at zero for namespaced or differently cased models. Use the resolver's candidate normalization when selecting affected settled rows.

Useful? React with 👍 / 👎.

Comment on lines +201 to +205
let cost = if crate::settings::cost_tracking_enabled() {
calculate_cost(
&context.app_type,
&usage,
lookup_model_pricing(state.db.as_ref(), pricing_model).as_ref(),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor the cost-tracking switch in session importers

With costTrackingEnabled set to false, normal session sync still calls insert_session_log_entry() in services/session_usage.rs, which unconditionally performs pricing lookup and cost calculation at lines 1287-1303; the Codex and Gemini importers have equivalent paths. Users importing local sessions therefore continue calculating and recording nonzero costs despite the setting's documented proxy/session scope. Apply the switch to session-import cost calculation as well, while retaining token inserts.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant