Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 127 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@
# arm64 goes through QEMU and costs tens of minutes on a JVM image, which is exactly why this is
# tag-triggered rather than on any per-merge path.
#
# It publishes every image docker.yml publishes as :edge, the run worker and the run unit's images
# included, so SPIRE_VERSION=<version> pins every image compose.ghcr.yml names. Until 2026-11-15 it
# published the four reviewer images only, and a version tag left the factory on whatever it last pulled.
#
# It can also be run by hand, and a manual run is ALWAYS a dry run. That exists because this workflow
# is otherwise untestable: it triggers only on a tag, so nothing that changes it — a bumped action, an
# edited step — is exercised until a real release is already under way, which is the worst possible
Expand Down Expand Up @@ -78,13 +82,24 @@ jobs:
echo "SPIRE_OIDC_ORCHESTRATOR_SECRET=dev-only-orchestrator-secret"
echo "SPIRE_OIDC_GATEWAY_SECRET=dev-only-gateway-secret"
echo "SPIRE_OIDC_WORKER_SECRET=dev-only-worker-secret"
# compose interpolates every service before it filters by profile, so the run worker's
# required secret must exist although the gate never starts the factory profile. Without
# it `up` refuses the whole file; e2e.yml carries the same line for the same reason.
echo "SPIRE_OIDC_RUN_WORKER_SECRET=dev-only-run-worker-secret"
echo "KEYCLOAK_ADMIN_USER=dev-admin"
echo "KEYCLOAK_ADMIN_PASSWORD=TEST-keycloak-admin"
echo "SPIRE_TRUSTED_PROXIES=172.16.0.0/12"
echo "SPIRE_VERSION=edge"
} > deploy/.env
grep -qE '^SPIRE_ENCRYPTION_KEYSET=.+' deploy/.env || { echo "keyset generation produced nothing"; exit 1; }

# The gate starts the reviewer stack only: the factory profile mounts the host's Docker socket and
# its run units need a forge and a model to do anything. This step still renders the file WITH that
# profile, so a variable the run worker requires, or a unit image reference that no longer
# interpolates at SPIRE_VERSION, fails the gate rather than the first operator who enables it.
- name: The factory profile interpolates
run: docker compose -f deploy/compose.ghcr.yml --env-file deploy/.env --profile factory config -q

- name: Up
run: docker compose -f deploy/compose.ghcr.yml --env-file deploy/.env up -d

Expand Down Expand Up @@ -119,25 +134,129 @@ jobs:
strategy:
fail-fast: false
Comment thread
artyomsv marked this conversation as resolved.
Resolved
matrix:
# Every image docker.yml publishes as :edge, published here at the version: the four services,
# the run worker, the dashboard, the run unit's three companions and the two reference agents.
# DockerWorkflowBuildsEveryImageTest fails when the two lists differ, because an image left out
# here is one an operator cannot pin to SPIRE_VERSION=<version> while compose.ghcr.yml names it.
#
# The fields mean what they mean in docker.yml. apkUpgrade sends APK_UPGRADE_BUST (docker.yml
# says at length why it is a security control); gradleDist names the module whose installDist
# must run first. catalogueScript names the agent image's build script, which reads the model
# catalogue: see the Model catalogue step.
#
# Keep no blank line and no comment between entries: ApkUpgradeIsNotCachedTest parses this
# list as source text.
include:
- name: spire-gateway
context: .
dockerfile: Dockerfile
service: gateway
apkUpgrade: true
gradleDist: ''
catalogueScript: ''
- name: spire-orchestrator
context: .
dockerfile: Dockerfile
service: orchestrator
apkUpgrade: true
gradleDist: ''
catalogueScript: ''
- name: spire-review-worker
context: .
dockerfile: Dockerfile
service: review-worker
apkUpgrade: true
gradleDist: ''
catalogueScript: ''
- name: spire-run-worker
context: .
dockerfile: Dockerfile
service: run-worker
apkUpgrade: true
gradleDist: ''
catalogueScript: ''
- name: spire-ui
context: ./spire-ui
dockerfile: ./spire-ui/Dockerfile
service: ''
apkUpgrade: true
gradleDist: ''
catalogueScript: ''
- name: spire-publisher
context: ./spire-publisher
dockerfile: ./spire-publisher/Dockerfile
service: ''
apkUpgrade: true
gradleDist: spire-publisher
catalogueScript: ''
- name: spire-model-proxy
context: ./spire-model-proxy
dockerfile: ./spire-model-proxy/Dockerfile
service: ''
apkUpgrade: true
gradleDist: spire-model-proxy
catalogueScript: ''
- name: spire-egress-proxy
context: ./deploy/egress
dockerfile: ./deploy/egress/Dockerfile
service: ''
apkUpgrade: true
gradleDist: ''
catalogueScript: ''
- name: spire-agent-codex
context: ./deploy/agent
dockerfile: ./deploy/agent/codex/Dockerfile
service: ''
apkUpgrade: false
gradleDist: ''
catalogueScript: deploy/agent/build-codex.sh
- name: spire-agent-claude-code
context: ./deploy/agent
dockerfile: ./deploy/agent/claude-code/Dockerfile
service: ''
apkUpgrade: false
gradleDist: ''
catalogueScript: deploy/agent/build-claude-code.sh
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

# The agent images carry their model catalogue as a label, and a label cannot come from a RUN's
# output (build-codex.sh says why). In docker.yml the script builds, reads and rebuilds; a
# multi-arch push cannot run what it builds, so here the script builds the amd64 image on the
# runner, the label is read back from it, and the value goes to the multi-arch build below as
# AGENT_MODELS. Calling the script keeps one recipe: the trim of `codex debug models` is not
# restated here, and a vendor change to that command fails the release as it fails docker.yml.
# The arm64 image installs the same pinned CLI release, so one catalogue describes both.
#
# Before setup-buildx, on purpose: that action makes its docker-container builder the default,
# which would not load the script's image into the local store, and the read would find none.
# The Claude Code script reads a file rather than the binary; it goes the same way so both
# agents' catalogues come from the scripts an operator runs.
- name: Model catalogue
id: catalogue
if: matrix.catalogueScript != ''
run: |
bash "${{ matrix.catalogueScript }}" "release-catalogue/${{ matrix.name }}:amd64"
models="$(docker inspect --format '{{ index .Config.Labels "dev.codespire.agent.models" }}' \
"release-catalogue/${{ matrix.name }}:amd64")"
if [ -z "$models" ] || [ "$models" = "<no value>" ]; then
echo "the build script produced an image with no model catalogue"; exit 1
fi
echo "models=$models" >> "$GITHUB_OUTPUT"

# Same JDK and Gradle setup as docker.yml, for the two images that copy a distribution. The
# distribution is JVM bytecode, so one installDist serves both architectures.
- if: matrix.gradleDist != ''
uses: actions/setup-java@77dd5b2ce225ea11469928ac317850245a74ae78 # v6.2.0
with:
distribution: temurin
java-version: '25'
- if: matrix.gradleDist != ''
uses: gradle/actions/setup-gradle@8da46b539cd8fe6ab27ca75128919a18d757604c # v6.4.1
- name: Distribution
if: matrix.gradleDist != ''
run: ./gradlew --no-daemon --console=plain ":${{ matrix.gradleDist }}:installDist"

- uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4.4.0
- uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4.4.1
- uses: docker/login-action@81c6f93d4bc991e510b342a00b419b3bb4efc5c6 # v4.7.0
Expand Down Expand Up @@ -168,7 +287,14 @@ jobs:
with:
context: ${{ matrix.context }}
file: ${{ matrix.dockerfile }}
build-args: ${{ matrix.service && format('SERVICE={0}', matrix.service) || '' }}
# One line per argument, each empty for the entries it does not apply to; the action skips
# empty items. APK_UPGRADE_BUST is docker.yml's security control: without a per-build value
# the upgrade layer is restored from the gha cache docker.yml fills, and a release would ship
# the OS packages of whichever build first wrote that layer.
build-args: |
${{ matrix.service && format('SERVICE={0}', matrix.service) || '' }}
${{ matrix.apkUpgrade && format('APK_UPGRADE_BUST={0}', github.run_id) || '' }}
${{ steps.catalogue.outputs.models && format('AGENT_MODELS={0}', steps.catalogue.outputs.models) || '' }}
# Both architectures even on a dry run: arm64 is what exercises QEMU, so dropping it would
# make the dry run silent about the one action it is most needed to check.
platforms: linux/amd64,linux/arm64
Expand Down
18 changes: 11 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,11 +274,13 @@ its ticket told. M5 is next; harness reviews are being built alongside it.
suite has not been re-measured since, and builds now run in CI rather than on the host.
- **The automated GitLab run-unit gap is still open.** RunUnitSpec has no network field, so a
run unit cannot reach the e2e stack's GitLab. A live GitHub run does not close it.
- **The factory images are published on merge to master but not yet live-pulled.** docker.yml builds the run
worker, publisher, model proxy, egress proxy, Codex agent and Claude Code agent as public GHCR packages (each links to
this public repository by its source label; the Claude Code one by operator decision 2026-11-15, removed if Anthropic
asks; ADR-032 amended 2026-11-14 and 2026-11-15); no deployment has run a unit from them yet, and they are amd64
only, unsigned and without an SBOM.
- **The factory images are partly live-pulled.** docker.yml builds the run worker, publisher, model proxy, egress
proxy, Codex agent and Claude Code agent as public GHCR packages (each links to this public repository by its source
label; the Claude Code one by operator decision 2026-11-15, removed if Anthropic asks; ADR-032 amended 2026-11-14 and
2026-11-15), and release.yml publishes them at a version, multi-arch and signed. On 2026-11-14/15 a dev-stack review
of spire-test #106 ran on the Codex agent, publisher, model proxy and egress proxy pulled at `sha-66dd409`; the
run worker and Claude Code images, the packaged run worker's Docker socket on Linux (uid 1001), any arm64 image and
a real release (no tag has run release.yml with them) are not live-proven (docs/UNVERIFIED.md).
- **Claude Code ran on one model and one CLI version** (`claude-sonnet-5-5`, 2.1.319): its charge is not reconciled
with the vendor, only the measured canaries are proven absent, and a re-review, a reply and a rate limit on it have
no live run (docs/UNVERIFIED.md, the 3b entry).
Expand All @@ -303,9 +305,11 @@ cd spire-ui && npm install && npm run dev # React dashboard :34000 (UI_PORT)
```

**The factory's images** are on GHCR (`ghcr.io/artyomsv/spire-{run-worker,publisher,model-proxy,egress-proxy,agent-codex,agent-claude-code}`,
`:edge` and `:sha-<short>` from docker.yml; `compose.ghcr.yml` defaults to them). The Claude Code agent is public by operator
`:edge` and `:sha-<short>` from docker.yml, amd64; `:<version>` and `:latest` from release.yml on a `v*` tag, amd64 + arm64,
signed; `compose.ghcr.yml` defaults to them). The Claude Code agent is public by operator
decision and removed if Anthropic asks (its CLI's terms do not address redistribution; LICENSING.md). The run worker pulls a unit image only when its daemon
lacks it, so pin `SPIRE_VERSION` or pull `edge` by hand on upgrade. For the dev stack and `compose.yml` they are still
lacks it, so pin `SPIRE_VERSION` (a release version or `sha-<short>`) or pull `edge` by hand on upgrade. release.yml's
manual run is a dry run (builds both architectures, pushes and signs nothing). For the dev stack and `compose.yml` they are still
built locally (SMOKE-TEST Mode Q, Mode R §7):

```bash
Expand Down
6 changes: 3 additions & 3 deletions deploy/.env.example
Original file line number Diff line number Diff line change
Expand Up @@ -162,9 +162,9 @@ SPIRE_TRUSTED_PROXIES=172.16.0.0/12
SPIRE_PUBLIC_HOST=

# --- Image tag ---
# `edge` tracks master. Pin a commit (sha-<short>, every master build publishes one) for a
# deployment you intend to keep. A version tag works for the run worker and the unit images only once
# release.yml publishes them (a follow-up PR); until then pin sha-<short>. compose.ghcr.yml takes the run unit's images
# `edge` tracks master. For a deployment you intend to keep, pin a release version (the tag without
# its v, e.g. 1.2.3: release.yml publishes every image at it, multi-arch and signed) or a commit
# (sha-<short>, every master build publishes one, amd64 only). compose.ghcr.yml takes the run unit's images
# at this tag too, and the run worker pulls a unit image only when its daemon lacks it: on `edge`
# an upgrade must pull them by hand (deploy/README.md), or units keep running the first `edge` the
# daemon ever pulled beside a newer run worker.
Expand Down
22 changes: 16 additions & 6 deletions deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,10 +208,20 @@ service being refused by another, and the gateway's live database privileges.

## Images

`:edge` tracks `master`, and every master build also publishes `:sha-<short>`; pin a `sha-<short>`
tag for anything you intend to keep (a version tag works for the run worker and the unit images only once
`release.yml` publishes them, in a follow-up PR). Each image carries its licence as an OCI label and as a
file (LICENSING.md, "Published images").
`:edge` tracks `master`, and every master build also publishes `:sha-<short>` (both amd64 only). A `v*`
tag publishes every image below as `:<version>` (the tag without its `v`) and `:latest`, for amd64 and arm64,
cosign-signed by digest with an SBOM and provenance (`release.yml`). For anything you intend to keep, pin
`SPIRE_VERSION` to a release version, or to a `sha-<short>` between releases. Each image carries its
licence as an OCI label and as a file (LICENSING.md, "Published images").

To check a release image's signature (keyless, bound to the release workflow's identity). No real tag has published
the factory images yet (a dry run built them, 2026-11-15), so the first one is the first proof:

```bash
cosign verify ghcr.io/artyomsv/spire-run-worker:<version> \
--certificate-identity-regexp '^https://github.com/artyomsv/code-spire/.github/workflows/release.yml@refs/tags/v' \
--certificate-oidc-issuer https://token.actions.githubusercontent.com
```

| Image | What it is | Licence | Visibility |
|---|---|---|---|
Expand All @@ -233,8 +243,8 @@ to a registry only you can read; set `SPIRE_FACTORY_AGENT_IMAGE_CLAUDE_CODE` to
`compose.ghcr.yml` names the unit images at `SPIRE_VERSION`, the run worker's own tag. **The run
worker pulls a unit image only when its Docker daemon does not hold it**, so a moving tag is pulled
once and never refreshed: after an upgrade on `edge`, units keep running the first `edge` that daemon
ever pulled, beside a newer run worker. Either pin `SPIRE_VERSION` to a `sha-<short>` tag
(a version tag works only once `release.yml` publishes these images; a new tag is a new image, so it is pulled), or, on `edge`, pull the unit images on the run worker's
ever pulled, beside a newer run worker. Either pin `SPIRE_VERSION` to a release version or a `sha-<short>`
tag (a new tag is a new image, so it is pulled), or, on `edge`, pull the unit images on the run worker's
daemon at every upgrade:

```bash
Expand Down
5 changes: 3 additions & 2 deletions deploy/compose.ghcr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,8 +203,9 @@ services:
# sandbox); this file names the published one, compose.yml the locally built one.
#
# The worker pulls a unit image only when its daemon does not hold it, so a moving tag is pulled
# once and never again. Pin SPIRE_VERSION to a sha-<short> tag (a version tag works only once release.yml publishes these images); on `edge`, pull these
# three and the agent image by hand on every upgrade (deploy/README.md).
# once and never again. Pin SPIRE_VERSION to a release version (release.yml publishes every image here at it)
# or a sha-<short> tag; on `edge`, pull these three and the agent image by hand on every upgrade
# (deploy/README.md).
SPIRE_PUBLISHER_IMAGE: ${SPIRE_PUBLISHER_IMAGE:-ghcr.io/artyomsv/spire-publisher:${SPIRE_VERSION:-edge}}
# Which run units hold their model credential in a model proxy (OFF, REVIEWS, ALL), and its image: the
# proxied runs fail as IMAGE_UNAVAILABLE without the image, builds run on. See deploy/.env.example.
Expand Down
Loading
Loading