fix(telemetry): request the cloud-platform scope for GCP OTel exporters - #7025
Open
chelsealong wants to merge 1 commit into
Open
fix(telemetry): request the cloud-platform scope for GCP OTel exporters#7025chelsealong wants to merge 1 commit into
chelsealong wants to merge 1 commit into
Conversation
google.auth.default() is called with no scopes to build the Cloud Trace / Cloud Monitoring / Cloud Logging exporters. GCE/GKE/Cloud Run metadata credentials tolerate that, but a service-account key file has requires_scopes=True and no scopes, so its token refresh fails with invalid_scope before any export request is made -- only the exporters are affected, not the running application. Request the cloud-platform scope explicitly wherever ADC is resolved for these exporters. Fixes google#7024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes: #7024
Problem
With
get_fast_api_app(..., otel_to_cloud=True)and Application DefaultCredentials supplied as a service-account key file, every Cloud Trace
and Cloud Monitoring export fails at the token grant, before any HTTP
request is made:
The running application is unaffected — only the exporters fail — so the
symptom is a continuous stream of tracebacks in the container log with no
telemetry ever arriving.
Root cause
google.auth.default()is called with no scopes at the three places thatresolve ADC for the GCP OTel exporters:
src/google/adk/cli/api_server.py—_setup_gcp_telemetry, the path thatfires from
get_fast_api_app.src/google/adk/telemetry/google_cloud.py— thegoogle_auth is Nonefallback in
get_gcp_exportersand in_get_gcp_otlp_metric_exporter.GCE/GKE/Cloud Run metadata-server credentials tolerate an unscoped request,
so this is invisible in those environments. A service-account key file has
requires_scopes = Trueand no scopes, so its token refresh is rejected.Fix
Request the
https://www.googleapis.com/auth/cloud-platformscopeexplicitly at all three call sites, via
google.auth.default(scopes=[...])(which routes through
google.auth.credentials.with_scopes_if_requiredinternally, so credentials that don't need a scope are left untouched).
Testing Plan
Unit Tests:
Added assertions to
test_get_gcp_exportersandtest_get_gcp_otlp_metric_exporter_uses_default_credentialsintests/unittests/telemetry/test_google_cloud.py, and a newtest_setup_gcp_telemetry_requests_cloud_platform_scopeintests/unittests/cli/test_fast_api.py, each assertinggoogle.auth.defaultis called withscopes=[CLOUD_PLATFORM_SCOPE].Confirmed the new/updated assertions fail without the fix
(
git checkout HEAD~1 -- src/google/adk/cli/api_server.py src/google/adk/telemetry/google_cloud.py):And passes with the fix restored:
Full unit suite also passes:
Formatting/lint checked with the repo's configured tools:
Manual End-to-End (E2E) Tests:
Not run — reproducing the failure requires an off-GCP host with
GOOGLE_APPLICATION_CREDENTIALSpointed at a service-account key file, whichisn't available in this environment. The unit tests above assert the exact
google.auth.default(scopes=[...])call the issue's reproduction scriptshows fixes the
RefreshError.Additional context
This PR intentionally addresses only the
invalid_scopefailure describedin the issue. The issue also separately mentions a follow-on
INVALID_ARGUMENT: prometheus_target resource type must have an instance specifiederror fromget_gcp_resource()on non-GCP hosts, which thereporter offered to file separately — left out of scope here.
Developed with AI assistance (Claude Code), reviewed and tested by a human before submission.