Repository navigation
Add SetupCcache: use the shared ccache stores from CI jobs - #49
Merged
Merged
Conversation
willson556
force-pushed
the
setup-ccache
branch
from
October 15, 2026 20:48
8b4093b to
e0bd3fb
Compare
willson556
force-pushed
the
setup-ccache
branch
from
October 15, 2026 21:00
e0bd3fb to
c83f776
Compare
Member
Author
How to use the Graphite Merge QueueAdd the label merge-queue to this PR to add it to the merge queue. You must have a Graphite account in order to use the merge queue. Sign up using this link. An organization admin has enabled the Graphite Merge Queue in this repository. Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue. This stack of pull requests is managed by Graphite. Learn more about stacking. |
willson556
marked this pull request as ready for review
October 15, 2026 21:28
staflsystemsci
Bot
requested review from
a team,
fcheein,
gretl-silva-stafl,
mament0,
mns4595,
mzhu48,
nehalkpatel and
thebhef
and removed request for
a team
October 15, 2026 21:28
|
Review SLA Nobody owes a review on this PR right now. Due times are in business hours, 10:00 to 17:00 Pacific: 4 h for a PR under 250 added lines, 7 h otherwise, and 4 h for a re-review. Mid-stack, an assignee owes a response on the lowest PR they haven't approved. Dashboard |
nehalkpatel
approved these changes
Oct 15, 2026
fcheein
approved these changes
Oct 16, 2026
Merge activity
|
Step 1 of the shared ccache store rollout (CI RFC Proposal 2). Both stores are up and verified: Stafl-EMB03 on the LAN for self-hosted runners, and `ccache1.ci.emb.staflsystems.com` on AWS for GitHub-hosted ones. This adds the action that points CI jobs at them. Merging it changes nothing until a workflow calls it with `mode: remote`. **What it does:** - **Picks the store from `runner.environment`:** self-hosted runners use the LAN store, and GitHub-hosted runners use the AWS one. URLs and passwords come from the `AWS_CCACHE_URL`, `LAN_CCACHE_URL`, `AWS_CCACHE_HTTP_PASSWORD` and `LAN_CCACHE_HTTP_PASSWORD` org variables and secrets. - **Tunnels to AWS over TLS:** ccache's HTTP backend can't speak HTTPS. So on GitHub-hosted runners the action starts a ghostunnel client on `localhost:8080` that forwards over TLS, checked against the system trust store. ghostunnel is pinned to v1.11.3 and its sha256 is verified. - **Modes:** `mode: actions` (the default) keeps today's Actions-cache behavior through `hendrikmuhs/ccache-action`. Callers pass `vars.CCACHE_MODE`, so setting it back to `actions` is the rollback. In `remote` mode, `hendrikmuhs/ccache-action` still runs, with restore and save off. That keeps its `ccache -z` at the start and `ccache -s` at the end, which `CIMetrics --ccache` (#48) reads. - **`write-only: 'true'`** sets ccache's `recache`, for pushes to main. In `actions` mode it skips the restore, as the BMS2000 repos' `restore: github.ref != 'refs/heads/main'` does today. - **`fork: 'true'`** installs the IAR-capable fork. It replaces the "setup ccache" step that is copied into every BMS2000 repo. - **Fails open:** if the store can't be reached, the job warns and builds without it. **ccache versions:** the jobs run Ubuntu's 4.5.1 (CI image), 4.9 and the 4.12 fork. I checked the ccache source for how each one reads the store settings: - 4.5.1 names the setting `secondary_storage`, and 4.7 renamed it `remote_storage` but still accepts the old name. So the action sets `CCACHE_SECONDARY_STORAGE`. - `remote_only` only exists from 4.7. 4.5.1 errors on an unknown key in a config file but ignores an unknown `CCACHE_*` environment variable. So the action sets `CCACHE_REMOTE_ONLY=1` in the environment, and nothing goes in `ccache_config.conf`. **Smoke test:** CustomGithubActions is public, so it can't run one. The org's ccache variables and secrets are for private repositories only, and the self-hosted runner groups don't allow public repositories. The first run here got empty store settings on GitHub-hosted runners, and its `ubuntu-iar` job was never picked up. So: - `ci.yml` here runs `shellcheck` on `SetupCcache/` (stacked on #52, which adds `ci.yml` and `CI Status`). - The end-to-end test lives in the private [StaflSystems/ci-smoke-tests](https://github.com/StaflSystems/ci-smoke-tests), which checks out this repo at a given ref: `gh workflow run setup-ccache-smoke.yml -R StaflSystems/ci-smoke-tests -f ref=setup-ccache`. For each store and ccache version, it builds 20 run-unique files three times, each in its own step, which also checks that the tunnel outlives the step that started it: 1. All misses, with no store errors. 2. After `ccache -C`, every file hits, and the store served at least one hit per file. ccache 4.5.1 counts a manifest and an object hit per file, so it reports 40. 3. With the store unreachable, the build must still pass. **Smoke results** ([run 38816301601](https://github.com/StaflSystems/ci-smoke-tests/actions/runs/38816301601)): all three jobs passed all three builds. - AWS, ccache 4.9.1: 20/20 hits from the store, through a tunnel that outlived its step. - AWS, ccache 4.5.1: 20/20 hits. `CCACHE_SECONDARY_STORAGE` works and `CCACHE_REMOTE_ONLY` is ignored, as expected. - LAN (10.10.1.22), IAR fork (4.12): 20/20 hits from Stafl-EMB03. - With the store unreachable, every job built with all misses and passed. An earlier LAN run failed because Stafl-EMB03 hadn't picked up its DHCP reservation: its DHCP client ID wasn't its MAC. With `dhcp-identifier: mac` in its netplan config, it's now at 10.10.1.22, which is what `LAN_CCACHE_URL` points to. **Next:** once this passes, the A/B (rollout step 2): StaflLib's `host-Debug` and `arm-cm7-iar-Debug` pass `mode: remote`. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
graphite-app
Bot
force-pushed
the
ci-status
branch
from
October 16, 2026 17:31
6f011ad to
b51b72b
Compare
graphite-app
Bot
force-pushed
the
setup-ccache
branch
from
October 16, 2026 17:32
822baff to
19bccac
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Step 1 of the shared ccache store rollout (CI RFC Proposal 2). Both stores are up and verified: Stafl-EMB03 on the LAN for self-hosted runners, and
ccache1.ci.emb.staflsystems.comon AWS for GitHub-hosted ones. This adds the action that points CI jobs at them. Merging it changes nothing until a workflow calls it withmode: remote.What it does:
runner.environment: self-hosted runners use the LAN store, and GitHub-hosted runners use the AWS one. URLs and passwords come from theAWS_CCACHE_URL,LAN_CCACHE_URL,AWS_CCACHE_HTTP_PASSWORDandLAN_CCACHE_HTTP_PASSWORDorg variables and secrets.localhost:8080that forwards over TLS, checked against the system trust store. ghostunnel is pinned to v1.11.3 and its sha256 is verified.mode: actions(the default) keeps today's Actions-cache behavior throughhendrikmuhs/ccache-action. Callers passvars.CCACHE_MODE, so setting it back toactionsis the rollback. Inremotemode,hendrikmuhs/ccache-actionstill runs, with restore and save off. That keeps itsccache -zat the start andccache -sat the end, whichCIMetrics --ccache(CIMetrics: report ccache hit rates with --ccache #48) reads.write-only: 'true'sets ccache'srecache, for pushes to main. Inactionsmode it skips the restore, as the BMS2000 repos'restore: github.ref != 'refs/heads/main'does today.fork: 'true'installs the IAR-capable fork. It replaces the "setup ccache" step that is copied into every BMS2000 repo.ccache versions: the jobs run Ubuntu's 4.5.1 (CI image), 4.9 and the 4.12 fork. I checked the ccache source for how each one reads the store settings:
secondary_storage, and 4.7 renamed itremote_storagebut still accepts the old name. So the action setsCCACHE_SECONDARY_STORAGE.remote_onlyonly exists from 4.7. 4.5.1 errors on an unknown key in a config file but ignores an unknownCCACHE_*environment variable. So the action setsCCACHE_REMOTE_ONLY=1in the environment, and nothing goes inccache_config.conf.Smoke test: CustomGithubActions is public, so it can't run one. The org's ccache variables and secrets are for private repositories only, and the self-hosted runner groups don't allow public repositories. The first run here got empty store settings on GitHub-hosted runners, and its
ubuntu-iarjob was never picked up. So:ci.ymlhere runsshellcheckonSetupCcache/(stacked on Add a CI Status check that runs on every PR #52, which addsci.ymlandCI Status).gh workflow run setup-ccache-smoke.yml -R StaflSystems/ci-smoke-tests -f ref=setup-ccache.For each store and ccache version, it builds 20 run-unique files three times, each in its own step, which also checks that the tunnel outlives the step that started it:
ccache -C, every file hits, and the store served at least one hit per file. ccache 4.5.1 counts a manifest and an object hit per file, so it reports 40.Smoke results (run 38816301601): all three jobs passed all three builds.
CCACHE_SECONDARY_STORAGEworks andCCACHE_REMOTE_ONLYis ignored, as expected.An earlier LAN run failed because Stafl-EMB03 hadn't picked up its DHCP reservation: its DHCP client ID wasn't its MAC. With
dhcp-identifier: macin its netplan config, it's now at 10.10.1.22, which is whatLAN_CCACHE_URLpoints to.Next: once this passes, the A/B (rollout step 2): StaflLib's
host-Debugandarm-cm7-iar-Debugpassmode: remote.🤖 Generated with Claude Code