Repository navigation
Python: count Gemini thinking tokens in the usage of Google AI completions - #14671
Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
alexcastgonz (pcbeingused333) wants to merge 1 commit into
Conversation
…tions Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
alexcastgonz (pcbeingused333)
deployed
to
github-app-auth
October 28, 2026 03:57 — with
GitHub Actions
Active
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues are reported for the update to include Gemini thinking tokens in completion-token accounting.
No open issues 🎉
What changed in this PR
Corrects completion-token accounting in the Python Google AI connector by including Gemini thinking tokens, so response usage and downstream telemetry reflect billed output tokens. The Vertex AI connector is unchanged.
Changes:
- Adds a shared helper that sums candidate and thinking tokens while handling missing counts.
- Uses the helper in both chat and text completion response metadata.
- Adds unit tests for token-count edge cases and both service integrations.
| File | Description |
|---|---|
python/tests/unit/connectors/ai/google/google_ai/services/test_google_ai_utils.py |
Tests the helper and chat/text usage accounting. |
python/semantic_kernel/connectors/ai/google/google_ai/services/utils.py |
Adds the completion-token counting helper. |
python/semantic_kernel/connectors/ai/google/google_ai/services/google_ai_text_completion.py |
Includes thinking tokens in text completion usage. |
python/semantic_kernel/connectors/ai/google/google_ai/services/google_ai_chat_completion.py |
Includes thinking tokens in chat completion usage. |
Review effort: Balanced
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
With a thinking Gemini model through the Google AI connector, the
usagein the response metadata under-reports the completion tokens. Gemini reports the thinking tokens inusage_metadata.thoughts_token_count, separately fromcandidates_token_count, and bills them as output tokens, butGoogleAIChatCompletionandGoogleAITextCompletiontookcompletion_tokensfromcandidates_token_countalone. The telemetry that reads this usage, and any caller that tracks cost from it, missed every thinking token.Description
A new
get_completion_token_counthelper ingoogle_ai/services/utils.pyreturnscandidates_token_count + thoughts_token_count(either may be missing, and it isNoneonly when both are). Both Google AI completion services use it in_get_metadata_from_response, which streaming also goes through. The Vertex AI connector is not changed.Tests in
test_google_ai_utils.pycover the helper (both counts, one missing, zero, none, no usage metadata) and check that the chat and text services report 50 completion tokens for 10 candidate and 40 thinking tokens. The Google AI unit tests pass (93 passed), and ruff check and format are clean on the touched paths. A live Gemini call was not made.Contribution Checklist
This change was made with Claude Code.
🤖 Generated with Claude Code