Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .changeset/auth-keyset-records.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
'@cipherstash/auth': minor
---

Add `keysetCreate` and `keysetRewrap` (Node only). They create and rewrap keyset records for the key vendor backends (AWS KMS, Google Cloud KMS, Azure Key Vault, HashiCorp Vault Transit and the static key backend), through the same Rust vendor clients that the Rust and Go SDKs use. ZeroKMS users do not need them. `stash keyset create` and `stash keyset rewrap` call them.

- Each vendor's credentials come from its environment chain.
- `backend` takes the short names (`aws`, `gcp`, `azure`, `vault`, `static`) and the full names that the result gives (`aws-kms`, `gcp-kms`, `azure-key-vault`, `vault-transit`).
- `mode` is `derived`, the only key mode in this release. Another mode is refused with `INVALID_OPTIONS`.
- `keysetRewrap` wraps the record's root key under another vendor key of the same backend. It does not re-key stored values, and every stored value still opens.
- Both return a `Result`; a failure is a `KeysetFailure`.

For Google Cloud KMS, set `GOOGLE_OAUTH_ACCESS_TOKEN` (for example from `gcloud auth print-access-token`) or run on Google Cloud. Service account key files are not read, so the binary has no RSA signing code.
13 changes: 13 additions & 0 deletions .changeset/stash-keyset-commands.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
'stash': minor
---

Add `stash keyset create` and `stash keyset rewrap`, for keysets held by a key vendor backend (AWS KMS, Google Cloud KMS, Azure Key Vault, HashiCorp Vault Transit, or the static key backend). ZeroKMS users never need them.

- `stash keyset create --backend <aws|gcp|azure|vault|static> --vendor-key <ref> [--name <name>]` makes a keyset record and prints it as `CS_KEYSET_RECORD` (or `CS_KEYSET_RECORD_<NAME>`). `--backend` also takes the full names that the output gives (`aws-kms`, `gcp-kms`, `azure-key-vault`, `vault-transit`). The record is not secret. Pass its value to `encrypt.WithKeysetRecords` (Go) or `KeysetRecord::parse` (Rust).
- `--mode` is optional. `derived` is the default and the only key mode in this release.
- `stash keyset rewrap <record | --in file> [--vendor-key <ref>]` wraps a record's root key again, after a key rotation in the vendor or to move to another vendor key of the same backend. The keyset id does not change. Stored data is not re-keyed, and every stored value still opens.
- Credentials come from each vendor's environment chain, never from a flag. A static key is named by its environment variable (`--key-env`); without one, a new key is generated and printed as `CS_STATIC_KEY`.
- `--out <file>` writes the block with mode 0600 (`--force` to overwrite), and `--json` emits one NDJSON event.

The `stash-cli` skill documents both commands, and the `stash-zerokms` skill says that key vendor backends exist for the Rust crates and the Go SDK, and that TypeScript does not support them yet.
22 changes: 22 additions & 0 deletions .github/workflows/integration-drizzle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,17 @@ on:
# require-auth-npm-changeset.yml.
- 'packages/stack-auth/Cargo.toml'
- 'packages/stack-auth/src/**'
# The key vendor crates the auth binding compiles in (stash keyset).
- 'packages/stack-kms-vendor/Cargo.toml'
- 'packages/stack-kms-vendor/src/**'
- 'packages/stack-kms-aws/Cargo.toml'
- 'packages/stack-kms-aws/src/**'
- 'packages/stack-kms-vault/Cargo.toml'
- 'packages/stack-kms-vault/src/**'
- 'packages/stack-kms-gcp/Cargo.toml'
- 'packages/stack-kms-gcp/src/**'
- 'packages/stack-kms-azure/Cargo.toml'
- 'packages/stack-kms-azure/src/**'
- 'languages/typescript/packages/auth/**'
- 'languages/typescript/packages/stack-auth-wasm/**'
# The Rust that produces every EQL payload these suites round-trip.
Expand Down Expand Up @@ -175,6 +186,17 @@ on:
# require-auth-npm-changeset.yml.
- 'packages/stack-auth/Cargo.toml'
- 'packages/stack-auth/src/**'
# The key vendor crates the auth binding compiles in (stash keyset).
- 'packages/stack-kms-vendor/Cargo.toml'
- 'packages/stack-kms-vendor/src/**'
- 'packages/stack-kms-aws/Cargo.toml'
- 'packages/stack-kms-aws/src/**'
- 'packages/stack-kms-vault/Cargo.toml'
- 'packages/stack-kms-vault/src/**'
- 'packages/stack-kms-gcp/Cargo.toml'
- 'packages/stack-kms-gcp/src/**'
- 'packages/stack-kms-azure/Cargo.toml'
- 'packages/stack-kms-azure/src/**'
- 'languages/typescript/packages/auth/**'
- 'languages/typescript/packages/stack-auth-wasm/**'
# The Rust that produces every EQL payload these suites round-trip.
Expand Down
22 changes: 22 additions & 0 deletions .github/workflows/integration-protect-ffi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,17 @@ on:
# require-auth-npm-changeset.yml.
- 'packages/stack-auth/Cargo.toml'
- 'packages/stack-auth/src/**'
# The key vendor crates the auth binding compiles in (stash keyset).
- 'packages/stack-kms-vendor/Cargo.toml'
- 'packages/stack-kms-vendor/src/**'
- 'packages/stack-kms-aws/Cargo.toml'
- 'packages/stack-kms-aws/src/**'
- 'packages/stack-kms-vault/Cargo.toml'
- 'packages/stack-kms-vault/src/**'
- 'packages/stack-kms-gcp/Cargo.toml'
- 'packages/stack-kms-gcp/src/**'
- 'packages/stack-kms-azure/Cargo.toml'
- 'packages/stack-kms-azure/src/**'
- 'languages/typescript/packages/auth/**'
- 'languages/typescript/packages/stack-auth-wasm/**'
- '.github/actions/require-cs-secrets/**'
Expand Down Expand Up @@ -155,6 +166,17 @@ on:
# require-auth-npm-changeset.yml.
- 'packages/stack-auth/Cargo.toml'
- 'packages/stack-auth/src/**'
# The key vendor crates the auth binding compiles in (stash keyset).
- 'packages/stack-kms-vendor/Cargo.toml'
- 'packages/stack-kms-vendor/src/**'
- 'packages/stack-kms-aws/Cargo.toml'
- 'packages/stack-kms-aws/src/**'
- 'packages/stack-kms-vault/Cargo.toml'
- 'packages/stack-kms-vault/src/**'
- 'packages/stack-kms-gcp/Cargo.toml'
- 'packages/stack-kms-gcp/src/**'
- 'packages/stack-kms-azure/Cargo.toml'
- 'packages/stack-kms-azure/src/**'
- 'languages/typescript/packages/auth/**'
- 'languages/typescript/packages/stack-auth-wasm/**'
- '.github/actions/require-cs-secrets/**'
Expand Down
22 changes: 22 additions & 0 deletions .github/workflows/integration-supabase.yml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,17 @@ on:
# require-auth-npm-changeset.yml.
- 'packages/stack-auth/Cargo.toml'
- 'packages/stack-auth/src/**'
# The key vendor crates the auth binding compiles in (stash keyset).
- 'packages/stack-kms-vendor/Cargo.toml'
- 'packages/stack-kms-vendor/src/**'
- 'packages/stack-kms-aws/Cargo.toml'
- 'packages/stack-kms-aws/src/**'
- 'packages/stack-kms-vault/Cargo.toml'
- 'packages/stack-kms-vault/src/**'
- 'packages/stack-kms-gcp/Cargo.toml'
- 'packages/stack-kms-gcp/src/**'
- 'packages/stack-kms-azure/Cargo.toml'
- 'packages/stack-kms-azure/src/**'
- 'languages/typescript/packages/auth/**'
- 'languages/typescript/packages/stack-auth-wasm/**'
# The Rust that produces every EQL payload these suites round-trip.
Expand Down Expand Up @@ -154,6 +165,17 @@ on:
# require-auth-npm-changeset.yml.
- 'packages/stack-auth/Cargo.toml'
- 'packages/stack-auth/src/**'
# The key vendor crates the auth binding compiles in (stash keyset).
- 'packages/stack-kms-vendor/Cargo.toml'
- 'packages/stack-kms-vendor/src/**'
- 'packages/stack-kms-aws/Cargo.toml'
- 'packages/stack-kms-aws/src/**'
- 'packages/stack-kms-vault/Cargo.toml'
- 'packages/stack-kms-vault/src/**'
- 'packages/stack-kms-gcp/Cargo.toml'
- 'packages/stack-kms-gcp/src/**'
- 'packages/stack-kms-azure/Cargo.toml'
- 'packages/stack-kms-azure/src/**'
- 'languages/typescript/packages/auth/**'
- 'languages/typescript/packages/stack-auth-wasm/**'
# The Rust that produces every EQL payload these suites round-trip.
Expand Down
23 changes: 23 additions & 0 deletions .github/workflows/require-auth-npm-changeset.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ name: "Require @cipherstash/auth changeset"
# and the npm API does not move: the npm binary is built from that crate, and
# nothing else would release it.
#
# The key vendor crates are listed too: the binding compiles them in for
# `stash keyset` (CIP-5312), so a change to one changes the npm binary.
#
# Changes that never ship (tests, examples, devDependencies) are filtered out
# by `check-auth-npm-changeset.mjs --shipped`.
#
Expand All @@ -18,6 +21,16 @@ on:
paths:
- packages/stack-auth/Cargo.toml
- packages/stack-auth/src/**
- packages/stack-kms-vendor/Cargo.toml
- packages/stack-kms-vendor/src/**
- packages/stack-kms-aws/Cargo.toml
- packages/stack-kms-aws/src/**
- packages/stack-kms-vault/Cargo.toml
- packages/stack-kms-vault/src/**
- packages/stack-kms-gcp/Cargo.toml
- packages/stack-kms-gcp/src/**
- packages/stack-kms-azure/Cargo.toml
- packages/stack-kms-azure/src/**
- languages/typescript/packages/auth/**
- languages/typescript/packages/stack-auth-wasm/**
- scripts/check-auth-npm-changeset.mjs
Expand Down Expand Up @@ -84,6 +97,16 @@ jobs:
git diff --name-only --no-renames "${BASE_SHA}...HEAD" -- \
packages/stack-auth/Cargo.toml \
packages/stack-auth/src \
packages/stack-kms-vendor/Cargo.toml \
packages/stack-kms-vendor/src \
packages/stack-kms-aws/Cargo.toml \
packages/stack-kms-aws/src \
packages/stack-kms-vault/Cargo.toml \
packages/stack-kms-vault/src \
packages/stack-kms-gcp/Cargo.toml \
packages/stack-kms-gcp/src \
packages/stack-kms-azure/Cargo.toml \
packages/stack-kms-azure/src \
languages/typescript/packages/auth \
languages/typescript/packages/stack-auth-wasm
)"; then
Expand Down
7 changes: 3 additions & 4 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,11 +98,10 @@ jobs:
- '!docs/**'
# The Go module, and the Rust crates no binding these jobs build
# depends on. The auth and profile bindings compile stack-auth and
# stack-profile, which stay in.
# stack-profile, and the auth binding compiles the key vendor
# crates for `stash keyset` and, through them, stack-kms,
# stack-encrypt and stack-encrypt-derive; all of those stay in.
- '!languages/golang/**'
- '!packages/stack-encrypt/**'
- '!packages/stack-encrypt-derive/**'
- '!packages/stack-kms/**'
- '!packages/stack-guest-abi/**'
- '!packages/stack-auth/fuzz/**'
# EQL's SQL sources, test suites, codegen and tasks. What the
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ Every npm package except EQL lives under `languages/typescript/`: packages in `l
links are provenance only.
- `packages/stack-auth`, `packages/stack-profile`, `packages/stack-kms`, `packages/stack-encrypt`, `packages/stack-encrypt-derive`, `packages/stack-guest-abi`: The Rust crates imported from `cipherstash/cipherstash-suite` with their history — `stack-auth` and `stack-profile` (published to crates.io, one version group), `stack-kms` (published to crates.io from 0.1.0, its own version group, re-exported by `stack-encrypt` as `stack_encrypt::kms`), `stack-encrypt` and `stack-encrypt-derive` (published to crates.io from 0.1.0, one version group; `eql-bindings`' `stack-encrypt` feature depends on them from the registry), and `stack-guest-abi` (`publish = false`). They are the members of the **root Cargo workspace**, with the three node binding crates below. See "Working on the Rust crates".
- `packages/stack-kms-vendor`, `packages/stack-kms-aws`, `packages/stack-kms-vault`, `packages/stack-kms-gcp`, `packages/stack-kms-azure`: The key vendor backends (ADR-0013 and ADR-0014 in `packages/stack-encrypt/docs/adr/`). `stack-kms-vendor` is the vendor seam (`KeyVendorBackend`), keyset records, the derived data key mode, the static key backend and the `VendorTransport` HTTP seam (feature `http`: the native reqwest transport). Its `test-support` feature holds the test code every vendor crate shares: the contract suite (`contract`), `FakeVendor`, and the recorded-fixture harness (`replay`: fixture types, `Replay`, `Recording`); a vendor crate's `tests/common/mod.rs` holds only its own emulator, keys and constants. `stack-kms-aws` is the AWS KMS backend: a hand-written JSON client with SigV4 signing in-crate and the AWS credential providers; its LocalStack test runs in `tests-crates.yml`'s `localstack` job. `stack-kms-vault` is the HashiCorp Vault Transit backend: a hand-written Transit client with token, AppRole and Kubernetes auth; its `vault server -dev` test runs in `tests-crates.yml`'s `vault` job. `stack-kms-gcp` is the Google Cloud KMS backend: a hand-written Cloud KMS REST client with CRC32C checks, service account (RS256 JWT, the pure-Rust `rsa` crate, which has the open advisory RUSTSEC-2023-0071) and metadata server credentials; the service account login is the default feature `service-account`, and a build without it holds no RSA code; Google has no Cloud KMS emulator, so its live test runs only in the opt-in, credentialed `integration-gcp-kms.yml`, which is inert until the `STACK_KMS_GCP_LIVE_KEY` repository variable is set. `stack-kms-azure` is the Azure Key Vault and Managed HSM backend: a hand-written Key Vault REST client (`wrapkey`/`unwrapkey`, RSA-OAEP-256 or A256KW, the associated data bound by `stack_kms_vendor::bound` because Key Vault has none) with Entra ID client secret, workload identity and managed identity credentials, and no RSA code of its own; Microsoft has no Key Vault emulator, so its live test runs only in the opt-in, credentialed `integration-azure-key-vault.yml`, which is inert until the `STACK_KMS_AZURE_LIVE_KEY` repository variable is set. Both live workflows call one job, `_test-kms-live.yml`; each keeps its triggers, opt-in and fork-PR condition, and its variable and secret names. All five are root workspace members, publishable in the `stack-kms-vendor` version group, and staged in `release-plz.toml` (`release = false`) until their first version is published by hand. The `stack-kms-vendor` crate docs hold the capability matrix of every key backend (key lifetime, vendor calls, what the vendor sees and can refuse, credentials, rotation, known limits); change it with the code it describes. TypeScript (`@cipherstash/stack`) cannot use a key vendor backend yet.
- `languages/typescript/packages/auth`, `languages/typescript/packages/profile`, `languages/typescript/packages/stack-auth-wasm`: The node bindings of those crates. `@cipherstash/auth` (napi-rs v2) and its six `platforms/*` packages are published to npm from this repository by `release.yml` (`auth-artifacts`, `publish-auth`); a change to what it ships, the `stack-auth` crate included, needs an `@cipherstash/auth` changeset (`require-auth-npm-changeset.yml`). `@cipherstash/profile` and its platforms are private and never published; `@cipherstash/stack-auth-wasm` is private and builds the wasm that `@cipherstash/auth` ships. Their `build` and `test` scripts never invoke cargo; `build:native`, `build:debug` and `test:cargo` do.
- `languages/typescript/packages/auth`, `languages/typescript/packages/profile`, `languages/typescript/packages/stack-auth-wasm`: The node bindings of those crates. `@cipherstash/auth` (napi-rs v2) and its six `platforms/*` packages are published to npm from this repository by `release.yml` (`auth-artifacts`, `publish-auth`); a change to what it ships, the `stack-auth` crate and the five key vendor crates included (the binding compiles them in for `keysetCreate` and `keysetRewrap`, which `stash keyset` calls), needs an `@cipherstash/auth` changeset (`require-auth-npm-changeset.yml`). `@cipherstash/profile` and its platforms are private and never published; `@cipherstash/stack-auth-wasm` is private and builds the wasm that `@cipherstash/auth` ships. Their `build` and `test` scripts never invoke cargo; `build:native`, `build:debug` and `test:cargo` do.
- `languages/golang`: The Go module — the SDK `encrypt` with its generated-code support `encrypt/gensupport` and the policy packages `encrypt/policy` and `encrypt/policy/protosource`; the credential package `auth`; the generator `stashgen` and its command `cmd/stashgen`; and `internal` (the shared guest plumbing and the `record` wire model). A wazero host with no cgo. Its two WASI guests (`encrypt/guest`, `auth/guest`) are detached Cargo workspaces built by `mise run wasm:guest:build` and `mise run wasm:auth-guest:build`; `mise run wasm:guest:build:deterministic` builds the seeded TEST build the hermetic Go tests use; the `.wasm` files they embed are gitignored. The stack-encrypt guest has a second build with the EQL types (`mise run wasm:guest:build:eql`, cargo feature `eql`, linking `eql-bindings` by path; `wasm:guest:build:eql:deterministic` is its test build), embedded by `encrypt/eql` and registered on import, so a program whose generated code names an EQL type runs it (ADR-0007, amended). `encrypt/eql`'s types and `Types` table are generated from the EQL catalog by `eql-codegen` (`eql_gen.go`, written by `mise run types:generate` in `packages/eql` and drift-gated by `types:check` and `cargo test -p eql-codegen`). Generated `*_stash.go` files are committed and CI fails when `go generate ./...` changes one. Releases are cut by hand with `.github/workflows/release-golang.yml` (`gh workflow run release-golang.yml -f version=X.Y.Z[-pre] --ref <branch>`): because the guests are gitignored on main and Go fetches a version as the files committed at its tag, the workflow builds them, commits them on a release-only commit no branch points at, tags it `languages/golang/vX.Y.Z` and pushes the tag alone, then attaches the EQL v4 SQL bundle to the GitHub release. The ref must carry the EQL v4 bundle, or the build stops at "Build the EQL SQL bundles". `scripts/go-release-version.mjs` refuses, before anything is built, every version Go would refuse (non-canonical semver, `+build` metadata, a v2+ major on a module path with no `/vN` suffix, a version shaped like a pseudo-version), a final version dispatched from anywhere but `main`, and a final version while `languages/golang/go.mod` pins a `github.com/cipherstash/vitaminc/…` module at a pseudo-version (vitaminc's Go bindings have no tags, and a TEMPORARY pin on an unreleased commit must be gone before the module is tagged); a pre-release may come from a branch and may carry such a pin. The build job also refuses a version the Go proxy already lists in `@v/list`, which catches a tag deleted after the proxy fetched it; it never asks the proxy for the version itself before the tag exists, because the proxy would cache "not found" for up to 30 minutes and `verify` would fail after the push. The build checks the `Cargo.lock` of the EQL workspace and both guest workspaces with `--locked` first, since none of those builds passes it. It runs in a job with no write token and hands the guests to a `publish` job that holds `contents: write` and runs only git and gh. `publish` fetches the commit with plain unauthenticated git rather than `actions/checkout`, whose `token` input defaults to the job's writable token, and refuses a release commit whose tree is not the one the build tested. `publish` takes the paths it commits from the workflow's fixed `GUESTS` list, never from the artifact, and refuses a `guests.sha256` that is not exactly one SHA-256 line for each of those three guests: the build job runs third-party code after writing that manifest, so a path read from it could put any file on the tag (`scripts/__tests__/go-release-artifact.test.mjs`). A version is immutable once the Go proxy has fetched it, so a tag is never moved or deleted: if `publish` or `verify` fails after the tag is pushed, use "Re-run failed jobs" on the same run, which keeps a tag that run pushed and creates or completes the release; a new dispatch of the same version is refused. Never tag a main commit by hand: `go get` would fetch a module with no guests.
- `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README)
- `languages/typescript/examples/*`: Working apps (basic, prisma, supabase-worker)
Expand Down
Loading
Loading