Skip to content

Reuse benchmark builds within each CI pipeline run - #2222

Merged
LoopedBard3 merged 2 commits into
aspnet:mainfrom
LoopedBard3:loopedbard3-fasthttp-build-caching
Oct 26, 2026
Merged

LoopedBard3 merged 2 commits into
aspnet:mainfrom
LoopedBard3:loopedbard3-fasthttp-build-caching

Conversation

@LoopedBard3

@LoopedBard3 LoopedBard3 commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Avoid rebuilding the same benchmark applications for every matrix entry. Fortunes FastHttp currently spends roughly 42 seconds rebuilding the application for each size/rate combination, despite the load phase being only 15 seconds of warmup plus 15 seconds of measurement.

  • Pass Crank's existing build-reuse options directly in the container, gRPC, HttpClient, SignalR, and WebSocket templates. No separate cache profile or test harness is introduced.
  • Scope source and build keys to Build.BuildId, refreshing sources/builds for each pipeline run while reusing compatible variants on each agent.
  • Also scope cached Docker image tags to the build ID for container applications/databases and gRPC Go servers/clients, preventing interleaved pipeline runs from overwriting each other's images. Non-Docker jobs are unchanged by the image hooks.
  • Keep separate identities for Dockerfile variants, gRPC implementations/frameworks, and WebSocket compression builds. Share SignalR's client build between warmup and load.
  • Preserve benchmark durations, runtime settings, scheduling, and result-storage configuration. Applications and database containers still start fresh for each benchmark.

The branch is based on aspnet/Benchmarks main.

Validation

Configuration-only validation used the existing Crank CLI without running benchmark agents:

  • Expanded and JSON-validated all 541 matrix messages for two pipeline build IDs, including the shared database argument fragment and quoted configuration hooks.
  • Resolved 60 real configurations covering 30 variants across two build IDs. Verified compatible native build fingerprints for shared keys, source/build key isolation, Docker image-tag isolation for all four affected roles, and unchanged non-Docker image names.
  • Passed all 43 existing pod-scheduler tests, including generated pipeline snapshots.
  • Passed staged whitespace checks.

Deployment considerations

No live benchmark timing run has been performed; actual wall-time savings still need to be measured after deployment. Retained builds and images rely on the agents' existing cleanup/restart policy; this change does not add a cache TTL.

Cache application and supporting builds for container, gRPC, HttpClient, SignalR, and WebSocket matrices using run-scoped source and build keys. Isolate Docker and compression variants, document the cache lifecycle, and add configuration regression coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d203afa3-f8a9-47b3-9353-e38e91a647e9

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Shared Docker tags allow interleaved pipeline runs to reuse another run’s images despite run-scoped cache keys.

1 open issue
What changed in this PR

This PR adds per-pipeline build reuse to five benchmark matrices, reducing repeated compilation while preserving benchmark durations and result-storage settings.

Changes:

  • Adds run-scoped source and build cache profiles and wires them into CI templates.
  • Separates build variants by Docker image, gRPC implementation, framework, and WebSocket compression.
  • Adds configuration regression tests and documents cache lifetime and excluded suites.
File Description
scenarios/​containers.benchmarks.yml Gives ASP.NET Docker variants distinct image names.
build/​websocket-scenarios.yml Enables run-scoped WebSocket build reuse.
build/​tests/​test_build_reuse.py Tests profile wiring, cache isolation, and compatible reuse.
build/​signalr-scenarios.yml Enables SignalR server and shared client build reuse.
build/​README.md Documents caching behavior, lifetime, and validation.
build/​job-variables.yml Defines the shared cache-profile configuration URL.
build/​httpclient-scenarios.yml Enables build reuse for GET and POST matrices.
build/​grpc-scenarios.yml Adds implementation identities and enables build reuse.
build/​containers-scenarios.yml Enables container application and database build reuse.
build/​build-reuse.profile.yml Defines cache identities and input validation for five suites.

Review effort: Balanced

Comment thread build/build-reuse.profile.yml Resolved
Remove the shared cache profile and standalone configuration test harness. Keep per-run source and build identities directly in existing templates, and suffix cached container and gRPC Go image names with the pipeline build ID to prevent interleaved runs from sharing mutable tags.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: d203afa3-f8a9-47b3-9353-e38e91a647e9
@LoopedBard3
LoopedBard3 marked this pull request as ready for review October 22, 2026 19:03

@DrewScoggins DrewScoggins left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@LoopedBard3
LoopedBard3 merged commit 8ececa4 into aspnet:main Oct 26, 2026
2 checks passed
@LoopedBard3
LoopedBard3 deleted the loopedbard3-fasthttp-build-caching branch October 26, 2026 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants