ci(deploy): deploy pre-built images tagged by commit SHA by alanpeixinho · Pull Request #2131 · kernelci/dashboard

alanpeixinho · 2026-09-21T21:15:22Z

What it is

Staging and production deploy pre-built GHCR images by commit SHA instead of :latest or on-server builds.

Publish GHCR Images — Manual ref; images tagged with resolved SHA; :latest only on push to main.
Staging — CI passes github.sha; pull + up --no-build; optional manual deploy with image_tag.
Production — Required image_tag (SHA); pulls that tag; DASHBOARD_VERSION from deployed commit.
Staging compose image paths match .env.example; docs updated.
Do not run Deploy production Dashboard for this PR.

How to test

1. Publish (no staging, no prod)

Push this branch.
Repo Actions → Publish GHCR Images → Run workflow.
Use workflow from this branch. Set ref to this branch (required).
Open the run → Summary: Commit SHA: <sha> is the commit you built.
Confirm tags in GHCR (https://github.com/orgs/kernelci/packages):

dashboard-backend, dashboard-frontend, dashboard-proxy each have a tag equal to that full SHA.
On this manual run, the latest tag digest should be unchanged (compare digest before vs after).

2. Negative staging (no host change)

Actions → Deploy Staging and E2E → Run workflow from this branch (we cannot run this via UI until the PR is merged, so we need to run this action via cli with gh workflow run "Deploy Staging and E2E" \ --repo kernelci/dashboard \ --ref chore/deploy-sha-images \ -f image_tag=<commit tag from the published image>).
Set image_tag to a fake SHA.
Open the run → job check-migrations should fail on actions/checkout (ref not found).
Job deploy-staging must not run (no SSH).

3. Manual staging (optional — live staging)

Same workflow as (2), image_tag = SHA from (1). This updates staging.

Run Summary: Deploying staging with IMAGE_TAG=<sha>.
Same Summary, table Staging deployment status, column Image:

ghcr.io/kernelci/dashboard/dashboard-backend:<sha>
same for dashboard-frontend and dashboard-proxy.

Do not use the staging site version string; that is git describe, not the image tag.

4. After merge to `main` (live staging; existing CI behavior)

Two runs, same merge commit. Actions at https://github.com/kernelci/dashboard/actions.
CI

Left: CI for KernelCI Dashboard.
Open the run with event push (not pull_request). Title is the merge commit.
Run header shows the commit SHA (github.sha).
Jobs: call-deploy-staging / Deploy Staging and E2E must run (skipped = you opened the PR CI run).
Open job deploy-staging → run Summary:

Deploying staging with IMAGE_TAG=<sha> equals the header SHA.
Staging deployment status → Image column uses …/dashboard-*: <same sha>, not :latest.
Publish

Left: Publish GHCR Images.
Open the run for that same commit.
Summary: Commit SHA: <sha> matches IMAGE_TAG from the CI staging job.
Discord only fires if staging fails; success is Actions Summary only.

Closes #2087.

* Publish GHCR Images: optional ref on manual dispatch; tag images with resolved commit SHA; push `latest` only on push to main * Staging: required `image_tag`; pull GHCR images with `--no-build`; manual `workflow_dispatch`; CI passes `github.sha` after tests * Production: required `image_tag` (replaces unused `tag` default); export `IMAGE_TAG` on deploy; checkout deploy SHA on server * Align staging `docker-compose.yml` image paths with `IMAGE_OWNER` / `IMAGE_REPOSITORY` from `.env.example` * Update DEPLOYMENT.md and README for SHA-based deploy flow Closes kernelci#2087

felipebergamin

Two things that will bite us on the actual deploy path — left them inline.

felipebergamin · 2026-09-23T15:45:19Z

    uses: ./.github/workflows/deploy-staging.yaml
    secrets: inherit
+    with:
+      image_tag: ${{ github.sha }}


This is going to race with Publish GHCR Images. Both still start on their own when something lands on main, and CI never waits for the images to actually be in the registry.

That used to be fine because staging built on the box. Now it pulls this SHA with --no-build, so if tests finish before all three images are pushed, deploy just fails. A retry/wait on pull would keep build and deploy as separate workflows and still make this reliable.

Good catch — addressed in 7c53d27.

Staging deploy now retries docker compose pull (up to ~10 min) so it waits for the SHA tags in GHCR instead of racing Publish. Build and Deploy stay separate.

felipebergamin · 2026-09-23T15:45:19Z

            cp ~/.env-staging dashboard-staging/.env &&
            cd dashboard-staging &&
-            git checkout ${GITHUB_SHA} &&
+            git checkout ${IMAGE_TAG} &&


This clone only has main, so git checkout will fail for a SHA that isn't on that branch. That's the hotfix path from #2087 — publish from a branch, then deploy that SHA. Actions checkout can fetch it; the server can't. Same thing in the production workflow.

Fetching the SHA first (git fetch origin "$IMAGE_TAG" then checkout) would cover that and also save us from cloning the whole history on every deploy.

Also fixed in 7c53d27 (staging + production).

We now git fetch origin "$IMAGE_TAG" before checkout, and clone with --depth 1 so hotfix SHAs not on main still resolve without pulling full history.

Retry staging pulls until SHA tags exist so CI does not race Publish. Fetch the image_tag on the host before checkout so hotfix commits work.

felipebergamin · 2026-09-25T14:15:54Z

    outputs:
      digest: ${{ steps.build.outputs.digest }}
    steps:
      - uses: actions/checkout@v4


since we are here I think we can update the action version.

Suggested change

- uses: actions/checkout@v4

- uses: actions/checkout@v7

Done in efdad26. Publish now uses actions/checkout@v7.

felipebergamin · 2026-09-25T14:19:19Z

+  resolve-ref:
+    if: github.repository == 'kernelci/dashboard' || github.event_name == 'workflow_dispatch'
+    runs-on: ubuntu-latest
+    outputs:
+      sha: ${{ steps.git.outputs.sha }}
+    steps:
+      - name: Resolve commit SHA
+        id: git
+        env:
+          GH_TOKEN: ${{ github.token }}
+          REF: ${{ github.event.inputs.ref || github.sha }}
+        run: |
+          set -euo pipefail
+          sha=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${REF}" --jq .sha)
+          echo "sha=${sha}" >> "$GITHUB_OUTPUT"
+


do we need this? I'm not sure but I think we can pass the ref directly to the checkout action.

Done in efdad26. We check out inputs.ref (or github.sha on push) and take the image tag from git rev-parse HEAD, so a branch or tag still becomes one commit SHA. The job stays so a bad ref fails once before the three image builds.

felipebergamin · 2026-09-25T14:36:22Z

+    inputs:
+      ref:
+        description: Git ref to build (branch, tag, or commit SHA)
+        required: true
+        type: string



Maybe we can use the github selector instead of allowing an open input. The ref selected there will be in ${{ github.ref }} (e.g. ref/tags/release-tag) and ${{ github.ref_name }} (e.g. release-tag). The commit sha will be ${{ github.sha }}

Dont we loose the power to go to any commit with that? I believe it only brings tags and branches.

felipebergamin · 2026-09-25T14:41:54Z

    steps:
      - uses: actions/checkout@v4
+        with:
+          ref: ${{ needs.resolve-ref.outputs.sha }}


if I'm not mistaken we can use ${{ github.sha }}.
maybe we don't even need to provide this arg, it defaults to github.ref

https://github.com/actions/checkout#usage

The image we want is inputs.ref, which can be a different SHA than the one from github.sha.

felipebergamin · 2026-09-25T14:48:29Z

-          tags: |
-            ${{ steps.meta.outputs.prefix }}/dashboard-frontend:latest
-            ${{ steps.meta.outputs.prefix }}/dashboard-frontend:${{ github.sha }}
+          tags: ${{ steps.meta.outputs.prefix }}/dashboard-frontend:${{ needs.resolve-ref.outputs.sha }}${{ github.event_name == 'push' && format(',{0}/dashboard-frontend:latest', steps.meta.outputs.prefix) || '' }}


I think we can use a short sha to make easier to read the image tags.

- name: Get short SHA run: echo "SHORT_SHA=${GITHUB_SHA::7}" >> "$GITHUB_ENV" - name: Show short SHA run: echo "${{ env.SHORT_SHA }}"

I might be too much conservative (especially because we are a small repo), but I still prefer long hashes due to reduced collision chance.
Do you consider too problematic to keep long hash here?

felipebergamin · 2026-09-25T14:50:33Z

+      image_tag:
+        description: GHCR image tag (Git commit SHA) to deploy
        required: true
-        default: 'main'
+        type: string


I'd replace this by github.ref_name

github.ref_name is the branch we are running the workflow from image_tag is the one we are using to build the image.

felipebergamin · 2026-09-25T14:51:19Z

      - name: Checkout code
        uses: actions/checkout@v4
        with:
+          ref: ${{ github.event.inputs.image_tag }}


Same as above: checkout image_tag, not github.ref_name.

felipebergamin · 2026-09-25T14:57:49Z

          SSH_HOST: ${{ secrets.STAGING_HOST }}
          SSH_KEY: ${{ secrets.STAGING_KEY }}
-          DASHBOARD_VERSION: ${{ github.event.inputs.tag }}
+          IMAGE_TAG: ${{ github.event.inputs.image_tag }}


again, I'd use github.(ref | ref_name | sha) instead of relying in the input exactly as provided

The host pulls image_tag. github.sha and github.ref_name are the workflow run, not that image.

felipebergamin · 2026-09-25T14:58:40Z

+    inputs:
+      image_tag:
+        description: GHCR image tag (Git commit SHA) to deploy
+        required: true
+        type: string


Same as production. Staging can deploy a SHA other than the branch that started the workflow.

Checkout the requested ref and tag images with git rev-parse HEAD. Bump actions/checkout to v7. Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>

alanpeixinho marked this pull request as ready for review September 22, 2026 21:12

alanpeixinho added 2 commits September 23, 2026 09:58

ci(deploy): fail publish early on an unresolvable ref

bfb9c73

alanpeixinho force-pushed the chore/deploy-sha-images branch from e9a1955 to bfb9c73 Compare September 23, 2026 12:58

alanpeixinho mentioned this pull request Sep 23, 2026

chore/deployment ingester docs #2122

Open

felipebergamin reviewed Sep 23, 2026

View reviewed changes

ci(deploy): wait for GHCR images and fetch deploy SHA

7c53d27

Retry staging pulls until SHA tags exist so CI does not race Publish. Fetch the image_tag on the host before checkout so hotfix commits work.

felipebergamin reviewed Sep 25, 2026

View reviewed changes

ci(deploy): resolve publish SHA from the checked-out ref

efdad26

Checkout the requested ref and tag images with git rev-parse HEAD. Bump actions/checkout to v7. Signed-off-by: Alan Peixinho <alan.peixinho@profusion.mobi>

Conversation

alanpeixinho commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it is

How to test

1. Publish (no staging, no prod)

2. Negative staging (no host change)

3. Manual staging (optional — live staging)

4. After merge to main (live staging; existing CI behavior)

Uh oh!

felipebergamin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

alanpeixinho commented Sep 21, 2026 •

edited

Loading

4. After merge to `main` (live staging; existing CI behavior)