Skip to content

Python: count Gemini thinking tokens in the usage of Google AI completions - #14671

Open
alexcastgonz (pcbeingused333) wants to merge 1 commit into
microsoft:mainfrom
pcbeingused333:fix/google-ai-usage-thoughts-tokens
Open

alexcastgonz (pcbeingused333) wants to merge 1 commit into
microsoft:mainfrom
pcbeingused333:fix/google-ai-usage-thoughts-tokens

Conversation

@pcbeingused333

Copy link
Copy Markdown
Contributor

Motivation and Context

With a thinking Gemini model through the Google AI connector, the usage in the response metadata under-reports the completion tokens. Gemini reports the thinking tokens in usage_metadata.thoughts_token_count, separately from candidates_token_count, and bills them as output tokens, but GoogleAIChatCompletion and GoogleAITextCompletion took completion_tokens from candidates_token_count alone. The telemetry that reads this usage, and any caller that tracks cost from it, missed every thinking token.

Description

A new get_completion_token_count helper in google_ai/services/utils.py returns candidates_token_count + thoughts_token_count (either may be missing, and it is None only when both are). Both Google AI completion services use it in _get_metadata_from_response, which streaming also goes through. The Vertex AI connector is not changed.

Tests in test_google_ai_utils.py cover the helper (both counts, one missing, zero, none, no usage metadata) and check that the chat and text services report 50 completion tokens for 10 candidate and 40 thinking tokens. The Google AI unit tests pass (93 passed), and ruff check and format are clean on the touched paths. A live Gemini call was not made.

Contribution Checklist

This change was made with Claude Code.

🤖 Generated with Claude Code

…tions

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 28, 2026 03:57
@semantic-kernel-automation semantic-kernel-automation Bot added the python Pull requests for the Python Semantic Kernel label Oct 28, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are reported for the update to include Gemini thinking tokens in completion-token accounting.

No open issues 🎉

What changed in this PR

Corrects completion-token accounting in the Python Google AI connector by including Gemini thinking tokens, so response usage and downstream telemetry reflect billed output tokens. The Vertex AI connector is unchanged.

Changes:

  • Adds a shared helper that sums candidate and thinking tokens while handling missing counts.
  • Uses the helper in both chat and text completion response metadata.
  • Adds unit tests for token-count edge cases and both service integrations.
File Description
python/​tests/​unit/​connectors/​ai/​google/​google_ai/​services/​test_google_ai_utils.py Tests the helper and chat/text usage accounting.
python/​semantic_kernel/​connectors/​ai/​google/​google_ai/​services/​utils.py Adds the completion-token counting helper.
python/​semantic_kernel/​connectors/​ai/​google/​google_ai/​services/​google_ai_text_completion.py Includes thinking tokens in text completion usage.
python/​semantic_kernel/​connectors/​ai/​google/​google_ai/​services/​google_ai_chat_completion.py Includes thinking tokens in chat completion usage.

Review effort: Balanced

This branch was successfully deployed

1 active deployment
github-app-auth — 3d04c6e7 Deployed Oct 28, 2026 by pcbeingused333 via add_label #29474
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Pull requests for the Python Semantic Kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants