Skip to content

feat(observability): add high-level request and job logs for package … - #5622

Open
WeiQuan0605 wants to merge 1 commit into
mainfrom
observability-higher-level-logs
Open

WeiQuan0605 wants to merge 1 commit into
mainfrom
observability-higher-level-logs

Conversation

@WeiQuan0605

@WeiQuan0605 WeiQuan0605 commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor
  • A short explanation of the proposed change:
    Adds structured observability logs at the HTTP request and background job level for the package upload, package download, and resource matching flows. Complements the lower-level blobstore and storage-cli logs added in PR #5575 by providing a higher-level view of what triggered each operation.

New log events emitted:

Event Logger When
package_upload.request.start/accepted/failed cc.api.packages POST /v3/packages/:guid/upload
package_download.request.start/done cc.api.packages GET /v3/packages/:guid/download
package_upload.action.start/enqueued/failed cc.action.package_upload Job enqueue inside PackageUpload
package_upload.job.start/done/failed cc.background PackageBits job execution
resource_match.request.start/done cc.api.resource_matches POST /v3/resource_matches
resource_match.exists.call cc.resource_pool Per-resource blobstore check (debug level)

ResourceMatchPresenter now accepts a pre-parsed array to avoid double Oj.load per request.

  • An explanation of the use cases your change solves

    • Correlate a cf push request ID across HTTP handler → job enqueue → job execution → blobstore operations
    • Measure end-to-end package upload latency (request accept time vs. job processing time)
    • Observe resource matching cache hit rates and per-size-bucket timing without enabling debug logging globally
    • Distinguish slow uploads from slow resource matching when diagnosing cf push performance
  • Links to any other associated PRs
    Add structured logs and Prometheus metrics for blobstore and packer operations #5575

  • I have reviewed the contributing guide

  • I have viewed, signed, and submitted the Contributor License Agreement

  • I have made this pull request to the main branch

  • I have run all the unit tests using bundle exec rake

  • I have run CF Acceptance Tests

…upload, resource matching, and package download

- package_upload.request.start/accepted/failed in packages_controller.rb
- package_upload.action.start/enqueued/failed in package_upload.rb (queue from config, no extra DB query)
- package_upload.job.start/done/failed in package_bits.rb
- resource_match.request.start/done in resource_matches_controller.rb
- resource_match.exists.call downgraded to logger.debug to avoid O(n) log spam
- resource_match done log includes cache_hits, cache_misses, cache_hit_rate, avg_resource_match_time
- download_start_time measured before audit event recording
- ResourceMatchPresenter accepts pre-parsed array to avoid double Oj.load per request

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant