Skip to content

microskills: charge the proposer's call when its CLI reports no usage - #314

Open
Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:fix-microskills-proposer-cost
Open

Lanre Shittu (Shizoqua) wants to merge 1 commit into
microsoft:mainfrom
Shizoqua:fix-microskills-proposer-cost

Conversation

@Shizoqua

Copy link
Copy Markdown
Contributor

Every paid side call is charged to the ledger as overhead. The loop guard's judge and redirect writer charge the usage their runner recovered, else an estimate on the same basis (pricing.runner_effective_or_approx: the prompt at 1x and the reply at 5x). The micro-skills proposer charged only recovered usage (if eff is not None), so a call whose CLI reported none added nothing to opt_tokens, and the ledger's overhead and overhead share left it out. Cursor's side calls never report usage; the others do not when the usage block is missing.

Fix: the proposer is charged through runner_effective_or_approx too, as the judge is. Recovered usage, an on-device call's 0, and a failed call (not charged) are as before. The experiments' proposer and cost-basis tests pass unchanged.

Tests, in a new tests/test_proposer_cost.py:

  • a call with no usage is charged the estimate (0 on main);
  • it is the same estimate runner_effective_or_approx gives, as the judge charges (fails on main);
  • recovered usage and an on-device 0 are charged as before;
  • a failed call is still not charged.
python -m pytest tests -n 4 --dist worksteal -rfEs --tb=line
1405 passed, 41 skipped

The skips need Windows, a real agent CLI (Codex, Hermes, opencode) or aider, or a non-root user.

Fixes #312

Every paid side call is charged to the ledger as overhead. The loop guard's judge and redirect
writer charge the usage their runner recovered, else an estimate on the same basis
(`pricing.runner_effective_or_approx`: the prompt at 1x and the reply at 5x). The micro-skills
proposer charged only recovered usage (`if eff is not None`), so a call whose CLI reported none
added nothing to `opt_tokens`, and the ledger's overhead and overhead share left it out. Cursor's
side calls never report usage; the others do not when the usage block is missing.

The proposer is now charged through `runner_effective_or_approx` too. Recovered usage, an
on-device call's 0 and a failed call (not charged) are as before.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

microskills: the proposer's call costs nothing in the ledger when its CLI reports no usage

1 participant