Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 34 additions & 12 deletions .github/workflows/release-golang.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,14 @@ name: Release (Go)

# Tags a release of the Go module (languages/golang) that `go get` can use.
#
# The module embeds three WASI guests built from Rust (encrypt/wasm,
# encrypt/eql/wasm, auth/wasm). They are gitignored on main, and Go fetches a
# module version as the files committed at its tag, so a tag on a main commit
# would build and then fail at `NewClient` with ErrGuestNotBuilt. This
# The module embeds five WASI guests built from Rust: the three production
# guests (encrypt/wasm, encrypt/eql/wasm, auth/wasm) and the two
# deterministic-kms test builds that package encrypttest embeds
# (encrypt/encrypttest/wasm), which give an application's tests a client with
# no ZeroKMS. They are gitignored on main, and Go fetches a module version as
# the files committed at its tag, so a tag on a main commit would build and
# then fail at `NewClient` with ErrGuestNotBuilt, and at
# `encrypttest.NewClient` with its own not-built error. This
# workflow builds the guests at the commit it runs on, commits them on a
# release-only commit that no branch points at, tags that commit
# `languages/golang/v<version>` (the prefix Go requires for a module in a
Expand Down Expand Up @@ -64,6 +68,8 @@ env:
languages/golang/encrypt/wasm/stack_encrypt_guest.wasm
languages/golang/encrypt/eql/wasm/stack_encrypt_guest_eql.wasm
languages/golang/auth/wasm/stack_auth_guest.wasm
languages/golang/encrypt/encrypttest/wasm/stack_encrypt_guest_deterministic.wasm
languages/golang/encrypt/encrypttest/wasm/stack_encrypt_guest_eql_deterministic.wasm

concurrency:
group: release-golang
Expand Down Expand Up @@ -171,8 +177,30 @@ jobs:
mise run wasm:guest:build:eql
mise run wasm:auth-guest:build

# Hashed before the test builds and the tests run, and checked again
# after them below, so the guests committed are the ones tested.
# The test builds ship too: package encrypttest embeds them. Every
# stack-encrypt build writes the same cargo output and differs only in
# where its task copies it, so the production guests are hashed before
# the test builds run and checked after: a task that copied a seeded
# build over a production guest stops the release here, rather than
# being hashed and tagged as production.
- name: Build the test guests
run: |
set -euo pipefail
read -ra guests <<< "$GUESTS"
production=()
for guest in "${guests[@]}"; do
case "$guest" in
*/encrypttest/wasm/*) ;;
*) production+=("$guest") ;;
esac
done
sha256sum "${production[@]}" > "$RUNNER_TEMP/production.sha256"
mise run wasm:guest:build:deterministic
mise run wasm:guest:build:eql:deterministic
sha256sum -c "$RUNNER_TEMP/production.sha256"

# Hashed before the tests run, and checked again after them below, so
# the guests committed are the ones tested.
- name: Hash the guests
run: |
set -euo pipefail
Expand All @@ -181,12 +209,6 @@ jobs:
sha256sum "${guests[@]}" > release-out/guests.sha256
cat release-out/guests.sha256

# The deterministic test builds the hermetic Go tests load.
- name: Build the test guests
run: |
mise run wasm:guest:build:deterministic
mise run wasm:guest:build:eql:deterministic

- name: Go tests against the guests being released
run: mise run go:test

Expand Down
27 changes: 14 additions & 13 deletions .github/workflows/tests-golang.yml
Original file line number Diff line number Diff line change
Expand Up @@ -212,11 +212,12 @@ jobs:
- name: stack-encrypt guest release build and import-surface gate
run: mise run wasm:guest:build

# The deterministic-kms TEST build, under encrypt/testdata where no
# build or embed sees it: the Go tests open the record fixture Rust
# sealed through it, and run round trips with no ZeroKMS. The tests
# skip when it is absent, so a build that forgets this step would pass
# with less coverage, which is why the job builds it unconditionally.
# The deterministic-kms TEST build, under encrypt/encrypttest/wasm,
# which package encrypttest embeds: the Go tests open the record
# fixture Rust sealed through it, and run round trips with no ZeroKMS.
# The encrypt package's tests skip when it is absent, so a build that
# forgets this step would pass with less coverage, which is why the job
# builds it unconditionally. encrypttest's own tests fail without it.
- name: stack-encrypt guest deterministic test build
run: mise run wasm:guest:build:deterministic

Expand All @@ -243,7 +244,7 @@ jobs:
# same bytes rather than a stale or rebuilt guest.
- name: Record the guests' checksums
run: |
for guest in encrypt/wasm/stack_encrypt_guest.wasm encrypt/testdata/stack_encrypt_guest_deterministic.wasm encrypt/eql/wasm/stack_encrypt_guest_eql.wasm encrypt/testdata/stack_encrypt_guest_eql_deterministic.wasm auth/wasm/stack_auth_guest.wasm; do
for guest in encrypt/wasm/stack_encrypt_guest.wasm encrypt/encrypttest/wasm/stack_encrypt_guest_deterministic.wasm encrypt/eql/wasm/stack_encrypt_guest_eql.wasm encrypt/encrypttest/wasm/stack_encrypt_guest_eql_deterministic.wasm auth/wasm/stack_auth_guest.wasm; do
(cd languages/golang && openssl dgst -sha256 "$guest" | awk '{print $NF}' > "$guest.sha256" && echo "$guest sha256 $(cat "$guest.sha256")")
done

Expand All @@ -254,12 +255,12 @@ jobs:
path: |
languages/golang/encrypt/wasm/stack_encrypt_guest.wasm
languages/golang/encrypt/wasm/stack_encrypt_guest.wasm.sha256
languages/golang/encrypt/testdata/stack_encrypt_guest_deterministic.wasm
languages/golang/encrypt/testdata/stack_encrypt_guest_deterministic.wasm.sha256
languages/golang/encrypt/encrypttest/wasm/stack_encrypt_guest_deterministic.wasm
languages/golang/encrypt/encrypttest/wasm/stack_encrypt_guest_deterministic.wasm.sha256
languages/golang/encrypt/eql/wasm/stack_encrypt_guest_eql.wasm
languages/golang/encrypt/eql/wasm/stack_encrypt_guest_eql.wasm.sha256
languages/golang/encrypt/testdata/stack_encrypt_guest_eql_deterministic.wasm
languages/golang/encrypt/testdata/stack_encrypt_guest_eql_deterministic.wasm.sha256
languages/golang/encrypt/encrypttest/wasm/stack_encrypt_guest_eql_deterministic.wasm
languages/golang/encrypt/encrypttest/wasm/stack_encrypt_guest_eql_deterministic.wasm.sha256
languages/golang/auth/wasm/stack_auth_guest.wasm
languages/golang/auth/wasm/stack_auth_guest.wasm.sha256
if-no-files-found: error
Expand Down Expand Up @@ -433,7 +434,7 @@ jobs:
- name: The guests are the ones Linux built and checked
run: |
cd languages/golang
for guest in encrypt/wasm/stack_encrypt_guest.wasm encrypt/testdata/stack_encrypt_guest_deterministic.wasm encrypt/eql/wasm/stack_encrypt_guest_eql.wasm encrypt/testdata/stack_encrypt_guest_eql_deterministic.wasm auth/wasm/stack_auth_guest.wasm; do
for guest in encrypt/wasm/stack_encrypt_guest.wasm encrypt/encrypttest/wasm/stack_encrypt_guest_deterministic.wasm encrypt/eql/wasm/stack_encrypt_guest_eql.wasm encrypt/encrypttest/wasm/stack_encrypt_guest_eql_deterministic.wasm auth/wasm/stack_auth_guest.wasm; do
want=$(cat "$guest.sha256")
got=$(openssl dgst -sha256 "$guest" | awk '{print $NF}')
if [ "$want" != "$got" ]; then
Expand Down Expand Up @@ -518,7 +519,7 @@ jobs:
- name: The guests are the ones wasi-check built and checked
run: |
cd languages/golang
for guest in encrypt/testdata/stack_encrypt_guest_deterministic.wasm encrypt/testdata/stack_encrypt_guest_eql_deterministic.wasm; do
for guest in encrypt/encrypttest/wasm/stack_encrypt_guest_deterministic.wasm encrypt/encrypttest/wasm/stack_encrypt_guest_eql_deterministic.wasm; do
want=$(cat "$guest.sha256")
got=$(openssl dgst -sha256 "$guest" | awk '{print $NF}')
if [ "$want" != "$got" ]; then
Expand Down Expand Up @@ -595,7 +596,7 @@ jobs:
- name: The guests are the ones the wasi-check job built and checked
run: |
cd languages/golang
for guest in encrypt/wasm/stack_encrypt_guest.wasm encrypt/testdata/stack_encrypt_guest_deterministic.wasm encrypt/eql/wasm/stack_encrypt_guest_eql.wasm encrypt/testdata/stack_encrypt_guest_eql_deterministic.wasm auth/wasm/stack_auth_guest.wasm; do
for guest in encrypt/wasm/stack_encrypt_guest.wasm encrypt/encrypttest/wasm/stack_encrypt_guest_deterministic.wasm encrypt/eql/wasm/stack_encrypt_guest_eql.wasm encrypt/encrypttest/wasm/stack_encrypt_guest_eql_deterministic.wasm auth/wasm/stack_auth_guest.wasm; do
want=$(cat "$guest.sha256")
got=$(openssl dgst -sha256 "$guest" | awk '{print $NF}')
if [ "$want" != "$got" ]; then
Expand Down
9 changes: 7 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,12 @@ languages/golang/auth/wasm/*.wasm
languages/golang/encrypt/wasm/*.sha256
languages/golang/encrypt/eql/wasm/*.sha256
languages/golang/auth/wasm/*.sha256
# The deterministic-kms TEST build of the stack-encrypt guest, from `mise run
# wasm:guest:build:deterministic`; under testdata so no binary embeds it.
# The deterministic-kms TEST builds of the stack-encrypt guest, from `mise run
# wasm:guest:build:deterministic` and `wasm:guest:build:eql:deterministic`.
# Package encrypttest embeds them; the release commits them on the tag.
languages/golang/encrypt/encrypttest/wasm/*.wasm
languages/golang/encrypt/encrypttest/wasm/*.sha256
Comment on lines +124 to +125

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor The old rules for languages/golang/encrypt/testdata/*.wasm and *.sha256 are deleted, so a checkout that built the deterministic guests before this PR now shows both old builds (2.3 MB and 3.5 MB) as untracked, and git add -A languages/golang stages them. Keep the two old encrypt/testdata lines, with a comment saying they cover the path from before the move, or have the :deterministic tasks delete any stale copy there.

Source: code-review · claude-opus-5-5


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1b7fa30. The two encrypt/testdata rules are back, with a comment saying they cover the builds made there before the move.

# Where those builds went before they moved to encrypttest/wasm. A checkout
# that built them there still has them, and they must not show as untracked.
languages/golang/encrypt/testdata/*.wasm
languages/golang/encrypt/testdata/*.sha256
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ Every npm package except EQL lives under `languages/typescript/`: packages in `l
links are provenance only.
- `packages/stack-auth`, `packages/stack-profile`, `packages/stack-kms`, `packages/stack-encrypt`, `packages/stack-encrypt-derive`, `packages/stack-guest-abi`: The Rust crates imported from `cipherstash/cipherstash-suite` with their history — `stack-auth` and `stack-profile` (published to crates.io, one version group), `stack-kms` (published to crates.io from 0.1.0, its own version group, re-exported by `stack-encrypt` as `stack_encrypt::kms`), `stack-encrypt` and `stack-encrypt-derive` (published to crates.io from 0.1.0, one version group; `eql-bindings`' `stack-encrypt` feature depends on them from the registry), and `stack-guest-abi` (`publish = false`). They are the members of the **root Cargo workspace**, with the three node binding crates below. See "Working on the Rust crates".
- `languages/typescript/packages/auth`, `languages/typescript/packages/profile`, `languages/typescript/packages/stack-auth-wasm`: The node bindings of those crates. `@cipherstash/auth` (napi-rs v2) and its six `platforms/*` packages are published to npm from this repository by `release.yml` (`auth-artifacts`, `publish-auth`); a change to what it ships, the `stack-auth` crate included, needs an `@cipherstash/auth` changeset (`require-auth-npm-changeset.yml`). `@cipherstash/profile` and its platforms are private and never published; `@cipherstash/stack-auth-wasm` is private and builds the wasm that `@cipherstash/auth` ships. Their `build` and `test` scripts never invoke cargo; `build:native`, `build:debug` and `test:cargo` do.
- `languages/golang`: The Go module — the SDK `encrypt` with its generated-code support `encrypt/gensupport` and the policy packages `encrypt/policy` and `encrypt/policy/protosource`; the credential package `auth`; the generator `stashgen` and its command `cmd/stashgen`; and `internal` (the shared guest plumbing and the `record` wire model). A wazero host with no cgo. Its two WASI guests (`encrypt/guest`, `auth/guest`) are detached Cargo workspaces built by `mise run wasm:guest:build` and `mise run wasm:auth-guest:build`; `mise run wasm:guest:build:deterministic` builds the seeded TEST build the hermetic Go tests use; the `.wasm` files they embed are gitignored. The stack-encrypt guest has a second build with the EQL types (`mise run wasm:guest:build:eql`, cargo feature `eql`, linking `eql-bindings` by path; `wasm:guest:build:eql:deterministic` is its test build), embedded by `encrypt/eql` and registered on import, so a program whose generated code names an EQL type runs it (ADR-0007, amended). `encrypt/eql`'s types and `Types` table are generated from the EQL catalog by `eql-codegen` (`eql_gen.go`, written by `mise run types:generate` in `packages/eql` and drift-gated by `types:check` and `cargo test -p eql-codegen`). Generated `*_stash.go` files are committed and CI fails when `go generate ./...` changes one. Releases are cut by hand with `.github/workflows/release-golang.yml` (`gh workflow run release-golang.yml -f version=X.Y.Z[-pre] --ref <branch>`): because the guests are gitignored on main and Go fetches a version as the files committed at its tag, the workflow builds them, commits them on a release-only commit no branch points at, tags it `languages/golang/vX.Y.Z` and pushes the tag alone, then attaches the EQL v4 SQL bundle to the GitHub release. The ref must carry the EQL v4 bundle, or the build stops at "Build the EQL SQL bundles". `scripts/go-release-version.mjs` refuses, before anything is built, every version Go would refuse (non-canonical semver, `+build` metadata, a v2+ major on a module path with no `/vN` suffix, a version shaped like a pseudo-version), a final version dispatched from anywhere but `main`, and a final version while `languages/golang/go.mod` pins a `github.com/cipherstash/vitaminc/…` module at a pseudo-version (vitaminc's Go bindings have no tags, and a TEMPORARY pin on an unreleased commit must be gone before the module is tagged); a pre-release may come from a branch and may carry such a pin. The build job also refuses a version the Go proxy already lists in `@v/list`, which catches a tag deleted after the proxy fetched it; it never asks the proxy for the version itself before the tag exists, because the proxy would cache "not found" for up to 30 minutes and `verify` would fail after the push. The build checks the `Cargo.lock` of the EQL workspace and both guest workspaces with `--locked` first, since none of those builds passes it. It runs in a job with no write token and hands the guests to a `publish` job that holds `contents: write` and runs only git and gh. `publish` fetches the commit with plain unauthenticated git rather than `actions/checkout`, whose `token` input defaults to the job's writable token, and refuses a release commit whose tree is not the one the build tested. `publish` takes the paths it commits from the workflow's fixed `GUESTS` list, never from the artifact, and refuses a `guests.sha256` that is not exactly one SHA-256 line for each of those three guests: the build job runs third-party code after writing that manifest, so a path read from it could put any file on the tag (`scripts/__tests__/go-release-artifact.test.mjs`). A version is immutable once the Go proxy has fetched it, so a tag is never moved or deleted: if `publish` or `verify` fails after the tag is pushed, use "Re-run failed jobs" on the same run, which keeps a tag that run pushed and creates or completes the release; a new dispatch of the same version is refused. Never tag a main commit by hand: `go get` would fetch a module with no guests.
- `languages/golang`: The Go module — the SDK `encrypt` with its generated-code support `encrypt/gensupport`, its test package `encrypt/encrypttest` (context assertions, and `NewClient`: an offline client over the deterministic test builds, for an application's tests) and the policy packages `encrypt/policy` and `encrypt/policy/protosource`; the credential package `auth`; the generator `stashgen` and its command `cmd/stashgen`; and `internal` (the shared guest plumbing and the `record` wire model). A wazero host with no cgo. Its two WASI guests (`encrypt/guest`, `auth/guest`) are detached Cargo workspaces built by `mise run wasm:guest:build` and `mise run wasm:auth-guest:build`; `mise run wasm:guest:build:deterministic` and `wasm:guest:build:eql:deterministic` build the seeded TEST builds into `encrypt/encrypttest/wasm`, which `encrypttest` embeds and the hermetic Go tests read; the `.wasm` files are gitignored. The stack-encrypt guest has a second build with the EQL types (`mise run wasm:guest:build:eql`, cargo feature `eql`, linking `eql-bindings` by path; `wasm:guest:build:eql:deterministic` is its test build), embedded by `encrypt/eql` and registered on import, so a program whose generated code names an EQL type runs it (ADR-0007, amended). `encrypt/eql`'s types and `Types` table are generated from the EQL catalog by `eql-codegen` (`eql_gen.go`, written by `mise run types:generate` in `packages/eql` and drift-gated by `types:check` and `cargo test -p eql-codegen`). Generated `*_stash.go` files are committed and CI fails when `go generate ./...` changes one. Releases are cut by hand with `.github/workflows/release-golang.yml` (`gh workflow run release-golang.yml -f version=X.Y.Z[-pre] --ref <branch>`): because the guests are gitignored on main and Go fetches a version as the files committed at its tag, the workflow builds them, commits them on a release-only commit no branch points at, tags it `languages/golang/vX.Y.Z` and pushes the tag alone, then attaches the EQL v4 SQL bundle to the GitHub release. The ref must carry the EQL v4 bundle, or the build stops at "Build the EQL SQL bundles". `scripts/go-release-version.mjs` refuses, before anything is built, every version Go would refuse (non-canonical semver, `+build` metadata, a v2+ major on a module path with no `/vN` suffix, a version shaped like a pseudo-version), a final version dispatched from anywhere but `main`, and a final version while `languages/golang/go.mod` pins a `github.com/cipherstash/vitaminc/…` module at a pseudo-version (vitaminc's Go bindings have no tags, and a TEMPORARY pin on an unreleased commit must be gone before the module is tagged); a pre-release may come from a branch and may carry such a pin. The build job also refuses a version the Go proxy already lists in `@v/list`, which catches a tag deleted after the proxy fetched it; it never asks the proxy for the version itself before the tag exists, because the proxy would cache "not found" for up to 30 minutes and `verify` would fail after the push. The build checks the `Cargo.lock` of the EQL workspace and both guest workspaces with `--locked` first, since none of those builds passes it. It runs in a job with no write token and hands the guests to a `publish` job that holds `contents: write` and runs only git and gh. `publish` fetches the commit with plain unauthenticated git rather than `actions/checkout`, whose `token` input defaults to the job's writable token, and refuses a release commit whose tree is not the one the build tested. `publish` takes the paths it commits from the workflow's fixed `GUESTS` list, never from the artifact, and refuses a `guests.sha256` that is not exactly one SHA-256 line for each of those five guests (the three production guests and the two test builds `encrypttest` embeds): the build job runs third-party code after writing that manifest, so a path read from it could put any file on the tag (`scripts/__tests__/go-release-artifact.test.mjs`). A version is immutable once the Go proxy has fetched it, so a tag is never moved or deleted: if `publish` or `verify` fails after the tag is pushed, use "Re-run failed jobs" on the same run, which keeps a tag that run pushed and creates or completes the release; a new dispatch of the same version is refused. Never tag a main commit by hand: `go get` would fetch a module with no guests.
- `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README)
- `e2e/migration-guide`: Unpublished, separately installed npm examples for the Protect/EQL v2 to Stack/EQL v3 guide. See its README before running; Postgres requires a disposable database and DynamoDB persistence is in memory.
- `languages/typescript/examples/*`: Working apps (basic, prisma, supabase-worker)
Expand Down
Loading
Loading