Repository navigation
Build-matched agent image, launch-time version check, and one gated, verifiable release flow - #129
Merged
Merged
Conversation
…k build Container commands defaulted to `crosslink-agent:latest`. Under Corvidae that tag does not exist (no release has been tagged since the move), and where it does exist it moves on every tag, prereleases included, while a cached copy is never refreshed, so the crosslink inside the container could lag behind or run ahead of the CLI that launched it. The two write to the same hub. The build script now chooses the default tag: an explicit CROSSLINK_AGENT_IMAGE_TAG at build time wins; a build from crosslink's own checkout uses `<version>` when HEAD is exactly `v<version>` with a clean tree and `nightly` otherwise; any other build, such as a crates.io source install, uses `<version>`. One constant carries it to every call site: kickoff's `--image` default, `container start`, the container auth fallback, swarm and sentinel. `container start` gains `--image` and honours CROSSLINK_CONTAINER_IMAGE like the auth commands. Floating tags (`nightly`, `latest`) are launched with `--pull=always`; pinned tags and digests are not. The pull-failure hint explains the build-matched tag when the default image is missing. The container guide describes the default and no longer calls registry tags immutable; only a digest pin is.
…atest only for verified stable releases A `v*` tag pushed both `:<version>` and `:latest` before the smoke test ran, moved `:latest` for prereleases, and the smoke test logged in before pulling, so neither a private package nor a missing tag could fail it. container-image.yml now: - checks the tag against crosslink/Cargo.toml before anything is pushed; - pushes only `:<version>` on a tag push; - runs the smoke test without logging in, first checking that every pushed tag resolves anonymously; - promotes `:latest` afterwards in a separate job, by the verified digest, and only for non-prerelease versions (re-pointing it to an older digest is the rollback); - refuses manual publishes from anything but develop, so a dispatch can no longer overwrite `:nightly` from a feature branch or rebuild an old tag. Release builds of the binaries (release-builds.yml and the image's own binaries) set CROSSLINK_AGENT_IMAGE_TAG to the tag's version, passed through cross, so their default image is the release image even from a shallow checkout without tags. A new daily image-canary workflow checks, anonymously, that `:nightly`, the newest release's `:<version>` (and `:latest` when it is stable), and the frozen ghcr.io/forecast-bio/crosslink-agent:latest that 0.9.0-beta.1 builds default to can still be pulled. crates.io publishing is unchanged here; its ownership and credential are tracked separately.
Cargo treats a rerun-if-changed path that does not exist as changed, so on
a checkout without packed-refs (the shallow checkouts CI uses) the build
script reran and the binary was rebuilt on every cargo invocation. On
Windows the readiness tests then failed: the second test group's rebuild
could not replace crosslink.exe while a daemon from the first group still
had it open ("Access is denied").
… fail on bad overrides, and keep offline launches working From the review of this PR: - A build counts as a release, and defaults to `:<version>`, only on positive evidence: a published package (`.cargo_vcs_info.json`), a checkout of crosslink at its `v<version>` tag, or an explicit CROSSLINK_AGENT_IMAGE_TAG. A build with no git information is no longer guessed to be a release; it defaults to `:nightly`. The clean-tree condition is dropped: watching the index to keep it current would rebuild on git's own index refreshes. - An explicit CROSSLINK_AGENT_IMAGE_TAG must match the OCI tag grammar and must not be `latest` or `local`; otherwise the build fails instead of falling back silently. CROSSLINK_AGENT_IMAGE_REPOSITORY lets fork builds default to their own registry, validated the same way. - The build script watches the HEAD, refs and packed-refs that git reports for the checkout (so linked worktrees work), and only paths that exist. - The rules live in src/build_support/agent_image.rs, included by build.rs and table-tested from the kickoff tests. - `--pull=always` made every launch of a floating tag fail without network access, even with the image cached. Floating tags are now refreshed with an explicit pull first; if that fails and a local copy exists, the launch warns and uses it, and otherwise the error carries the runtime's reason. - The container guide states what development builds actually run, that `:nightly` follows develop, and that `cargo install --git` needs `--image`.
…iner command CROSSLINK_CONTAINER_IMAGE was honoured only by `container start` and `container auth`; kickoff took its image from a clap default, and swarm and sentinel used the build default with no override at all, so the same environment chose different images depending on the command. One resolver now serves all of them: `--image`, then CROSSLINK_CONTAINER_IMAGE, then this build's default. Blank values count as unset and image names starting with `-` are refused. Kickoff (when it launches a container) and `container start` print the image and where it came from. Together with the build-time CROSSLINK_AGENT_IMAGE_REPOSITORY override, a fork's builds can default to the fork's own registry.
… at launch The only skew check, check_staleness, read a binary-hash label that only a local `container build` set, hashed a host binary against the image's Linux binary (never equal across platforms), and inspected the default image even when another was chosen; it never fired for published images. Images now carry org.opencontainers.image.version with the exact `crosslink --version` of their binary (`container build` sets it from the packaged binary; CI sets it in the release workflow). After refreshing the image, kickoff and `container start` compare that label with this CLI's version: a different version warns, because the container and the CLI write the same hub; the same version from a different commit (a development build against :nightly) is reported at info level; images without the label are skipped.
…test promotion, pinned image From the review of this PR, plus the release-binary and reproducibility items folded into it: - A gate job decides what a run may publish before anything is built: a v* tag must match crosslink/Cargo.toml (read with tomllib, like cargo) and must not already be published, since pinned tags are cached and a second image under one tag would split users; manual runs publish only from develop and can rehearse promotion against :promotion-rehearsal; the unreachable manual-<sha> branch is gone. - Binaries bake the gate's agent image tag explicitly (the version on a tag, nightly otherwise) instead of inferring it. - Only the publishing jobs hold packages/contents write, and their actions are pinned by commit. Images carry org.opencontainers.image.version (the binary's exact version), revision and source labels, and a GitHub build attestation. - The smoke test pulls the built digest without logging in, checks that every pushed tag resolves anonymously to that digest, and checks the version label against the binary. - promote-latest moves :latest only to the verified digest, only when the version is the highest stable v* tag (backports and reruns never move it backwards), one run at a time, verifying the source before and the tag after. Release runs are never cancelled part-way. - release-builds.yml becomes a reusable workflow called on release tags (it keeps its pull-request check), adds Linux aarch64 and macOS aarch64 builds, and the release job creates the GitHub release with the binaries, plugin archives, SHA256SUMS and build attestations only after the image passed its smoke test. - The canary checks every release since the build-matched default with prereleases ordered below their release, and pins the frozen forecast-bio image to its digest so a silent repoint fails. - The Dockerfile pins the base image by digest and gh, Codex, Claude Code, uv and gosu by version, and verifies gosu by checksum. crates.io publishing (publish.yml) is unchanged and still independent of this flow until its ownership and credential are settled.
…ository-move checklist There was no release runbook (tracker #170 planned one and the 0.5.2 changelog says it exists; it did not), and each repository move dropped part of publishing. RELEASING.md describes what a v* tag publishes and in which order, what to check, how to roll back, the guarantees of each image tag, updating pinned image tools, the repository settings releases depend on, and what to carry over when the repository or owner moves.
magnificentlycursed
changed the title
Rewriting this PR's [Unreleased] release-flow entry in 4b9f988 kept only the text before it, dropping every release section from 0.9.0-beta.1 back (1,089 lines down to 43). The file is now the base file with this PR's additions inserted under [Unreleased] > Added, plus an entry for the launch-time version check that was missing.
…bel, and a network-only offline fallback
From the second review round of this PR:
- build.rs honours an explicit CROSSLINK_VERSION (it must start with the
crate version), so CI can give every architecture the same version
string; the arm64 binary built with cross had none, because the cross
container cannot see the repository's git metadata.
- The version label is dev.crosslink.version. org.opencontainers.image.version
is inherited from the Ubuntu base ("24.04"), so unlabelled images would
have raised false version warnings.
- A failed pull falls back to a local copy only when the network failed and
only for floating tags. A registry that answers "manifest unknown",
"denied" or "unauthorized" has withdrawn or hidden the image, and a cached
copy no longer outlives that.
- A pinned tag or digest missing locally is pulled explicitly before launch,
so its failures carry the same hints and the version check runs on its
first launch too.
- The image is resolved only for container launches; a local launch (and
every swarm and sentinel launch) no longer reads the image settings.
- Tests: transport-versus-refusal classification, the full preparation
matrix with a fake runtime, the exact inspect arguments of the version
check; the namespace test that the always-set repository variable made
tautological is removed.
…out the shared cache, and verify what binaries default to From the second review round of this PR: - The gate requires the tagged commit to be on main and proceeds only when the registry answers "not found" for :<version>; a missing token or any other answer stops the release instead of republishing. - The gate sets one version string (crate version + commit) that every binary is built with, passed through cross, and the image's dev.crosslink.version label comes from it; the arm64 binary built with cross previously reported no commit, which would have failed its smoke leg after the release image was already pushed. - Release images build with no cache; the registry cache is written by develop pushes and is no longer trusted for releases. - The smoke test checks, on both architectures, that the binary defaults to this image and that the bundled tools (Claude Code, Codex, gh, uv) run; release-builds checks each binary's default image and version. - Builds also set the image repository from the publishing owner, so a fork's binaries default to the fork's image. - Rehearsal dispatches get their own concurrency group, so a develop push no longer cancels them. - The Dockerfile builds with pipefail, so a failed installer download fails the build, and says which inputs still float. - `just build-image` labels its image with the version it builds. - CI runs actionlint (with shellcheck) on every workflow. - RELEASING.md: the gate's new conditions, deleting a failed stray tag, re-running only failed jobs, what "never republished" covers, and that the intended rulesets are not configured yet.
A job name built from needs outputs is shown unexpanded when the job is skipped, as it is on every pull request.
On Linux a test thread forking a process while another test's fake runtime
script is open for writing keeps a write handle in that child until it
execs, and executing the script meanwhile fails with ETXTBSY ("Failed to
run …/c pull" on CI). The fake runtimes now probe until the script can be
spawned before a test uses them.
…build version check From the third review round of this PR: - Pull failures are classified by whether the registry refused the image (manifest unknown, denied, unauthorized, not found), matched with the image reference removed. Only a refusal stops a launch that has a local copy of a floating tag. Network errors, registry outages (5xx), rate limits and unrecognised failures fall back to the local copy with a warning, and the advice names which case it is. Round 2's list of network phrases missed "context deadline exceeded", "no route to host" and server errors, so cached :nightly launches failed with wrong advice. - An explicit CROSSLINK_VERSION must be the crate version optionally followed by +<build>; a mere prefix match accepted 0.9.01 for 0.9.0. - `--image` without a container now warns that it is ignored. - Tests: refusal classification including words inside the image reference, the resolver's precedence, sources and refusals, and that a local launch never reads the image settings.
…d before the push, rehearsals leave :nightly alone From the third review round of this PR: - A re-run of the publish job (Re-run failed jobs, the runbook's recovery) no longer rebuilds and overwrites :<version>: when the version is already published on a re-run attempt, the job reuses its digest for the smoke test, promotion and attestation; on a first attempt an existing version stops the job. - Before anything is pushed, both the amd64 and the arm64 binaries are checked for the gate's version string and for defaulting to this image, so an arm64 mismatch can no longer burn a release version after the push. - A rehearsal dispatch pushes only :nightly-<sha> and writes no cache, so it cannot race a develop push or move :nightly backwards. - Release runs skip the cargo cache as well as the image cache. - The tool check runs under pipefail, so a missing gh fails it. - release-builds passes its inputs to the check through env. - RELEASING.md says plainly that, until rulesets exist, the release checks guard against accidents only (they run from the tagged commit's own workflow), and describes the re-run behaviour and artifact retention.
This was referenced Oct 9, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #125, #122, #123, #124, #127 and #128. Delivers the image and release half of #126; its crates.io ordering waits on #121, and its repository settings are a separate decision.
Container agents now run the crosslink their CLI was built for, and a release publishes its image, binaries and GitHub release in one gated, verifiable order. This grew out of the publishing research and was revised through three rounds of adversarial review (Platform Engineer, Solution Architect, Software and Quality Engineer, Security). Round 1 raised 34 findings, round 2 two blockers (both fixed: a truncated changelog and an arm64 version mismatch), round 3 none; every round's findings were fixed or recorded as deferred (tracker #815) or as decisions (crates.io, repository rulesets).
Default image matches the CLI build (#125)
Container commands defaulted to
crosslink-agent:latest. That tag does not exist under Corvidae, moved with every tag including prereleases, and was never refreshed once cached, so the crosslink in the container could differ from the CLI that launched it. Both write the same hub.:<version>only on positive evidence:cargo install crosslink);v<version>tag;CROSSLINK_AGENT_IMAGE_TAG.:nightly.latest, orlocalfails the build instead of silently falling back.CROSSLINK_AGENT_IMAGE_REPOSITORYlets a fork's builds default to its own registry.crosslink/src/build_support/agent_image.rs, used bybuild.rsand table-tested.container start,container auth, swarm and sentinel all resolve the image the same way,--imagethenCROSSLINK_CONTAINER_IMAGEthen the build default, and report which one they used.:nightlyand:latestare refreshed before every launch. If the registry cannot be reached or does not answer and a local copy exists, the launch warns and uses it; if the registry refuses the image (withdrawn, private), it stops.Version check at launch (#127)
The old staleness check never fired. Images now carry
org.opencontainers.image.version, the exactcrosslink --versionof their binary, set by CI and bycontainer build. At launch, kickoff andcontainer startcompare it with the CLI's version:One gated release flow (#126, #122, #123)
A
v*tag runscontainer-image.ymlend to end:Gate. The tag must match
Cargo.tomland must not already be published. Versions are never republished. Manual runs publish only fromdevelop, and can rehearse the promotion against:promotion-rehearsal.Image. Only
:<version>is pushed, with version, revision and source labels and a build attestation.Smoke test. Runs without logging in. It checks that every pushed tag resolves to the built digest, pulls that digest on both architectures, and checks the version label against the binary.
:latest. Moves to the verified digest only for the highest stable release, one promotion at a time, checked before and after.GitHub release.
release-builds.yml, now a reusable workflow, builds:The release is created with these, plus
SHA256SUMSand attestations, only after the image passed.Only the publishing jobs hold write permissions, and their actions are pinned by commit. A daily canary checks anonymously:
:nightly;:<version>, and:latest;ghcr.io/forecast-bio/crosslink-agent:latest, which 0.9.0-beta.1 builds use, still names the same digest.Reproducible image (#128)
The Dockerfile now pins:
Two builds of the same commit contain the same tools. Released images are never rebuilt; newer tools ship in a patch release.
Runbook (#124)
RELEASING.mdcovers:The git-flow knowledge page was corrected to match.
Verification
cargo fmt --checkandcargo clippy -- -D warnings -W clippy::unwrap_used -W clippy::expect_usedare clean, and actionlint (with shellcheck) is clean on all three workflows.:nightly, and an explicit override changes it.latest) fails the build, and a fork repository override takes effect.:nightly.container auth statusrefreshes:nightlyand honoursCROSSLINK_CONTAINER_IMAGE.develop, dispatch and rehearsal events:developwithrehearse_promotionexercises the gate, smoke test and promotion first;RELEASING.mddescribes it.Not in this PR
publish.ymlstill runs independently and fails on tag pushes from this repository.mainanddevelop.🤖 Generated with Claude Code