diff --git a/.changeset/auth-keyset-records.md b/.changeset/auth-keyset-records.md new file mode 100644 index 0000000000..30f6787e6c --- /dev/null +++ b/.changeset/auth-keyset-records.md @@ -0,0 +1,7 @@ +--- +'@cipherstash/auth': minor +--- + +Add `keysetCreate` and `keysetRewrap` (Node only). They create and rewrap keyset records for the key vendor backends (AWS KMS, Google Cloud KMS, Azure Key Vault, HashiCorp Vault Transit and the static key backend), through the same Rust vendor clients that the Rust and Go SDKs use. Each vendor's credentials come from its environment chain. Both return a `Result`; a failure is a `KeysetFailure`. `stash keyset create` and `stash keyset rewrap` call them. ZeroKMS users do not need them. `keysetRewrap` does not re-key stored values, so a record in the wrapped key mode may move only to a new version of its own vendor key; a move to another key fails with `REWRAP_REFUSED` before any vendor call. A record in the derived key mode may move to any key. + +For Google Cloud KMS, set `GOOGLE_OAUTH_ACCESS_TOKEN` (for example from `gcloud auth print-access-token`) or run on Google Cloud. Service account key files are not read, so that the binary has no RSA signing code. diff --git a/.changeset/stash-key-vendor-rewrap-docs.md b/.changeset/stash-key-vendor-rewrap-docs.md new file mode 100644 index 0000000000..460cc4e430 --- /dev/null +++ b/.changeset/stash-key-vendor-rewrap-docs.md @@ -0,0 +1,13 @@ +--- +'stash': patch +--- + +`stash keyset rewrap` now refuses to move a `wrapped`-mode record to another +vendor key, with the code `rewrap_refused`, before any vendor call. Each stored +value keeps its data key wrapped under the vendor key that sealed it, so such a +move would leave the stored values unable to open. A `wrapped` record may still +move to a new version of its own key (after a rotation), and a `derived` record +may move to any key. The command help, the note it prints, the README and the +`stash-cli` skill state the rule. The `stash-zerokms` skill now says that key +vendor backends exist for the Rust crates and the Go SDK, that TypeScript does +not support them yet, and that `stash keyset` creates their keyset records. diff --git a/.changeset/stash-keyset-commands.md b/.changeset/stash-keyset-commands.md new file mode 100644 index 0000000000..11acab0e62 --- /dev/null +++ b/.changeset/stash-keyset-commands.md @@ -0,0 +1,12 @@ +--- +'stash': minor +--- + +Add `stash keyset create` and `stash keyset rewrap`, for keysets held by a key vendor backend (AWS KMS, Google Cloud KMS, Azure Key Vault, HashiCorp Vault Transit, or the static key backend). ZeroKMS users never need them. + +- `stash keyset create --backend --vendor-key --mode [--name ]` makes a keyset record and prints it as `CS_KEYSET_RECORD` (or `CS_KEYSET_RECORD_`). The record is not secret. Pass its value to `encrypt.WithKeysetRecords` (Go) or `KeysetRecord::parse` (Rust). +- `stash keyset rewrap [--vendor-key ]` wraps a record's key again, after a key rotation in the vendor or to move to another vendor key. The keyset id does not change, and stored data is not re-keyed, so a `wrapped` record may move only to a new version of its own vendor key; a move to another key is refused. A `derived` record may move to any key. +- Credentials come from each vendor's environment chain, never from a flag. A static key is named by its environment variable (`--key-env`); without one, a new key is generated and printed as `CS_STATIC_KEY`. +- `--out ` writes the block with mode 0600 (`--force` to overwrite), and `--json` emits one NDJSON event. + +The `stash-cli` skill documents both commands. diff --git a/.github/workflows/integration-azure-key-vault.yml b/.github/workflows/integration-azure-key-vault.yml new file mode 100644 index 0000000000..80868905d3 --- /dev/null +++ b/.github/workflows/integration-azure-key-vault.yml @@ -0,0 +1,116 @@ +name: Integration — stack-kms-azure (Azure Key Vault) + +# stack-kms-azure against real Azure Key Vault and, optionally, Managed HSM +# (tests/live.rs). Microsoft publishes no Key Vault emulator, so unlike the +# LocalStack and Vault jobs in tests-crates.yml this needs a real key and real +# credentials. Every other stack-kms-azure test runs in tests-crates.yml, +# against fixtures and an in-process emulator. +# +# OPT-IN. The job runs only when the repository variable +# STACK_KMS_AZURE_LIVE_KEY is set, so the workflow is inert until someone sets +# up the Azure side. To turn it on: +# +# 1. Create an RSA key (2048 bits or more) in a Key Vault, for this test +# only. Optionally, create an oct-HSM AES-256 key in a Managed HSM too +# (a Managed HSM is billed by the hour, so this part is optional). +# 2. Create an app registration (service principal) with a client secret. +# Give it the role "Key Vault Crypto Service Encryption User" on the key +# and nothing else ("Managed HSM Crypto Service Encryption User" on the +# HSM key, if there is one). +# 3. Set the repository variable STACK_KMS_AZURE_LIVE_KEY to the key +# identifier (https://.vault.azure.net/keys/), and, for the +# HSM, STACK_KMS_AZURE_LIVE_HSM_KEY +# (https://.managedhsm.azure.net/keys/). Set the repository +# secrets STACK_KMS_AZURE_TENANT_ID, STACK_KMS_AZURE_CLIENT_ID and +# STACK_KMS_AZURE_CLIENT_SECRET. +# +# Once the variable is set, a missing or empty secret FAILS the job (the +# pre-flight below): a credentialed job that skips for want of a secret reads +# exactly like one that passes. +# +# Fork PRs have no secrets, so the job skips them; every other event runs. +# Enforced by scripts/__tests__/workflow-dispatch-job-conditions.test.mjs. + +on: + push: + branches: [main] + paths: + - packages/stack-kms-azure/** + - packages/stack-kms-vendor/** + - packages/stack-kms/** + - Cargo.lock + - .github/workflows/integration-azure-key-vault.yml + pull_request: + # Repeated verbatim: GitHub Actions does not support YAML anchors/aliases. + paths: + - packages/stack-kms-azure/** + - packages/stack-kms-vendor/** + - packages/stack-kms/** + - Cargo.lock + - .github/workflows/integration-azure-key-vault.yml + workflow_dispatch: {} + +permissions: + contents: read + +defaults: + run: + shell: bash + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + live: + name: stack-kms-azure against Azure Key Vault + runs-on: blacksmith-16vcpu-ubuntu-2204 + timeout-minutes: 30 + if: >- + vars.STACK_KMS_AZURE_LIVE_KEY != '' + && (github.event_name != 'pull_request' + || github.event.pull_request.head.repo.full_name == github.repository) + env: + MISE_ENV: test + STACK_KMS_AZURE_LIVE_KEY: ${{ vars.STACK_KMS_AZURE_LIVE_KEY }} + STACK_KMS_AZURE_LIVE_HSM_KEY: ${{ vars.STACK_KMS_AZURE_LIVE_HSM_KEY }} + steps: + # Fast pre-flight: fail in seconds if a secret was rotated or cleared, + # before the toolchain install and the compile. + - name: Require the Azure service principal + env: + TENANT_ID: ${{ secrets.STACK_KMS_AZURE_TENANT_ID }} + CLIENT_ID: ${{ secrets.STACK_KMS_AZURE_CLIENT_ID }} + CLIENT_SECRET: ${{ secrets.STACK_KMS_AZURE_CLIENT_SECRET }} + run: | + missing=0 + for name in TENANT_ID CLIENT_ID CLIENT_SECRET; do + if [ -z "${!name}" ]; then + echo "::error::STACK_KMS_AZURE_LIVE_KEY is set but the secret STACK_KMS_AZURE_${name} is empty" + missing=1 + fi + done + exit "$missing" + + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + working_directory: . + cache: true + # Must name runs-on's image; see scripts/__tests__/workflow-mise-cache-image.test.mjs + cache_key_prefix: mise-v1-ubuntu-2204 + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + + # The secrets reach only this step's environment. + - name: stack-kms-azure against Azure Key Vault + env: + AZURE_TENANT_ID: ${{ secrets.STACK_KMS_AZURE_TENANT_ID }} + AZURE_CLIENT_ID: ${{ secrets.STACK_KMS_AZURE_CLIENT_ID }} + AZURE_CLIENT_SECRET: ${{ secrets.STACK_KMS_AZURE_CLIENT_SECRET }} + run: mise run test:live:stack-kms-azure diff --git a/.github/workflows/integration-drizzle.yml b/.github/workflows/integration-drizzle.yml index f5a3254ce5..a7512c8f24 100644 --- a/.github/workflows/integration-drizzle.yml +++ b/.github/workflows/integration-drizzle.yml @@ -67,6 +67,17 @@ on: # require-auth-npm-changeset.yml. - 'packages/stack-auth/Cargo.toml' - 'packages/stack-auth/src/**' + # The key vendor crates the auth binding compiles in (stash keyset). + - 'packages/stack-kms-vendor/Cargo.toml' + - 'packages/stack-kms-vendor/src/**' + - 'packages/stack-kms-aws/Cargo.toml' + - 'packages/stack-kms-aws/src/**' + - 'packages/stack-kms-vault/Cargo.toml' + - 'packages/stack-kms-vault/src/**' + - 'packages/stack-kms-gcp/Cargo.toml' + - 'packages/stack-kms-gcp/src/**' + - 'packages/stack-kms-azure/Cargo.toml' + - 'packages/stack-kms-azure/src/**' - 'languages/typescript/packages/auth/**' - 'languages/typescript/packages/stack-auth-wasm/**' # The Rust that produces every EQL payload these suites round-trip. @@ -175,6 +186,17 @@ on: # require-auth-npm-changeset.yml. - 'packages/stack-auth/Cargo.toml' - 'packages/stack-auth/src/**' + # The key vendor crates the auth binding compiles in (stash keyset). + - 'packages/stack-kms-vendor/Cargo.toml' + - 'packages/stack-kms-vendor/src/**' + - 'packages/stack-kms-aws/Cargo.toml' + - 'packages/stack-kms-aws/src/**' + - 'packages/stack-kms-vault/Cargo.toml' + - 'packages/stack-kms-vault/src/**' + - 'packages/stack-kms-gcp/Cargo.toml' + - 'packages/stack-kms-gcp/src/**' + - 'packages/stack-kms-azure/Cargo.toml' + - 'packages/stack-kms-azure/src/**' - 'languages/typescript/packages/auth/**' - 'languages/typescript/packages/stack-auth-wasm/**' # The Rust that produces every EQL payload these suites round-trip. diff --git a/.github/workflows/integration-gcp-kms.yml b/.github/workflows/integration-gcp-kms.yml new file mode 100644 index 0000000000..a216c0c840 --- /dev/null +++ b/.github/workflows/integration-gcp-kms.yml @@ -0,0 +1,111 @@ +name: Integration — stack-kms-gcp (Google Cloud KMS) + +# stack-kms-gcp against real Google Cloud KMS (tests/live.rs). Google +# publishes no Cloud KMS emulator, so unlike the LocalStack and Vault jobs in +# tests-crates.yml this needs a real crypto key and real credentials. Every +# other stack-kms-gcp test runs in tests-crates.yml, against fixtures and an +# in-process emulator. +# +# OPT-IN. The job runs only when the repository variable +# STACK_KMS_GCP_LIVE_KEY is set, so the workflow is inert until someone sets +# up the Google Cloud side. To turn it on: +# +# 1. Create a symmetric ENCRYPT_DECRYPT crypto key for this test only. +# 2. Create a service account with roles/cloudkms.cryptoKeyEncrypterDecrypter +# on that key and nothing else, and make a JSON key for it. +# 3. Set the repository variable STACK_KMS_GCP_LIVE_KEY to the crypto key +# name (projects/

/locations//keyRings//cryptoKeys/), and the +# repository secret STACK_KMS_GCP_SERVICE_ACCOUNT_KEY to the JSON key. +# +# Once the variable is set, a missing or empty secret FAILS the job (the +# pre-flight below): a credentialed job that skips for want of a secret reads +# exactly like one that passes. +# +# Fork PRs have no secrets, so the job skips them; every other event runs. +# Enforced by scripts/__tests__/workflow-dispatch-job-conditions.test.mjs. + +on: + push: + branches: [main] + paths: + - packages/stack-kms-gcp/** + - packages/stack-kms-vendor/** + - packages/stack-kms/** + - Cargo.lock + - .github/workflows/integration-gcp-kms.yml + pull_request: + # Repeated verbatim: GitHub Actions does not support YAML anchors/aliases. + paths: + - packages/stack-kms-gcp/** + - packages/stack-kms-vendor/** + - packages/stack-kms/** + - Cargo.lock + - .github/workflows/integration-gcp-kms.yml + workflow_dispatch: {} + +permissions: + contents: read + +defaults: + run: + shell: bash + +concurrency: + group: ${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + live: + name: stack-kms-gcp against Cloud KMS + runs-on: blacksmith-16vcpu-ubuntu-2204 + timeout-minutes: 30 + if: >- + vars.STACK_KMS_GCP_LIVE_KEY != '' + && (github.event_name != 'pull_request' + || github.event.pull_request.head.repo.full_name == github.repository) + env: + MISE_ENV: test + STACK_KMS_GCP_LIVE_KEY: ${{ vars.STACK_KMS_GCP_LIVE_KEY }} + steps: + # Fast pre-flight: fail in seconds if the secret was rotated or + # cleared, before the toolchain install and the compile. + - name: Require the Cloud KMS service account key + env: + SERVICE_ACCOUNT_KEY: ${{ secrets.STACK_KMS_GCP_SERVICE_ACCOUNT_KEY }} + run: | + if [ -z "$SERVICE_ACCOUNT_KEY" ]; then + echo "::error::STACK_KMS_GCP_LIVE_KEY is set but the secret STACK_KMS_GCP_SERVICE_ACCOUNT_KEY is empty" + exit 1 + fi + + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + working_directory: . + cache: true + # Must name runs-on's image; see scripts/__tests__/workflow-mise-cache-image.test.mjs + cache_key_prefix: mise-v1-ubuntu-2204 + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + + # The key goes to a file only this user can read, outside the + # checkout, and is removed after the test. + - name: Write the service account key + env: + SERVICE_ACCOUNT_KEY: ${{ secrets.STACK_KMS_GCP_SERVICE_ACCOUNT_KEY }} + run: | + umask 077 + printf '%s' "$SERVICE_ACCOUNT_KEY" > "$RUNNER_TEMP/stack-kms-gcp-live.json" + echo "GOOGLE_APPLICATION_CREDENTIALS=$RUNNER_TEMP/stack-kms-gcp-live.json" >> "$GITHUB_ENV" + + - name: stack-kms-gcp against Cloud KMS + run: mise run test:live:stack-kms-gcp + + - name: Remove the service account key + if: always() + run: rm -f "$RUNNER_TEMP/stack-kms-gcp-live.json" diff --git a/.github/workflows/integration-protect-ffi.yml b/.github/workflows/integration-protect-ffi.yml index 909f5bca89..bb3c068bc9 100644 --- a/.github/workflows/integration-protect-ffi.yml +++ b/.github/workflows/integration-protect-ffi.yml @@ -106,6 +106,17 @@ on: # require-auth-npm-changeset.yml. - 'packages/stack-auth/Cargo.toml' - 'packages/stack-auth/src/**' + # The key vendor crates the auth binding compiles in (stash keyset). + - 'packages/stack-kms-vendor/Cargo.toml' + - 'packages/stack-kms-vendor/src/**' + - 'packages/stack-kms-aws/Cargo.toml' + - 'packages/stack-kms-aws/src/**' + - 'packages/stack-kms-vault/Cargo.toml' + - 'packages/stack-kms-vault/src/**' + - 'packages/stack-kms-gcp/Cargo.toml' + - 'packages/stack-kms-gcp/src/**' + - 'packages/stack-kms-azure/Cargo.toml' + - 'packages/stack-kms-azure/src/**' - 'languages/typescript/packages/auth/**' - 'languages/typescript/packages/stack-auth-wasm/**' - '.github/actions/require-cs-secrets/**' @@ -155,6 +166,17 @@ on: # require-auth-npm-changeset.yml. - 'packages/stack-auth/Cargo.toml' - 'packages/stack-auth/src/**' + # The key vendor crates the auth binding compiles in (stash keyset). + - 'packages/stack-kms-vendor/Cargo.toml' + - 'packages/stack-kms-vendor/src/**' + - 'packages/stack-kms-aws/Cargo.toml' + - 'packages/stack-kms-aws/src/**' + - 'packages/stack-kms-vault/Cargo.toml' + - 'packages/stack-kms-vault/src/**' + - 'packages/stack-kms-gcp/Cargo.toml' + - 'packages/stack-kms-gcp/src/**' + - 'packages/stack-kms-azure/Cargo.toml' + - 'packages/stack-kms-azure/src/**' - 'languages/typescript/packages/auth/**' - 'languages/typescript/packages/stack-auth-wasm/**' - '.github/actions/require-cs-secrets/**' diff --git a/.github/workflows/integration-supabase.yml b/.github/workflows/integration-supabase.yml index 90917bd8bb..dedeba0331 100644 --- a/.github/workflows/integration-supabase.yml +++ b/.github/workflows/integration-supabase.yml @@ -60,6 +60,17 @@ on: # require-auth-npm-changeset.yml. - 'packages/stack-auth/Cargo.toml' - 'packages/stack-auth/src/**' + # The key vendor crates the auth binding compiles in (stash keyset). + - 'packages/stack-kms-vendor/Cargo.toml' + - 'packages/stack-kms-vendor/src/**' + - 'packages/stack-kms-aws/Cargo.toml' + - 'packages/stack-kms-aws/src/**' + - 'packages/stack-kms-vault/Cargo.toml' + - 'packages/stack-kms-vault/src/**' + - 'packages/stack-kms-gcp/Cargo.toml' + - 'packages/stack-kms-gcp/src/**' + - 'packages/stack-kms-azure/Cargo.toml' + - 'packages/stack-kms-azure/src/**' - 'languages/typescript/packages/auth/**' - 'languages/typescript/packages/stack-auth-wasm/**' # The Rust that produces every EQL payload these suites round-trip. @@ -154,6 +165,17 @@ on: # require-auth-npm-changeset.yml. - 'packages/stack-auth/Cargo.toml' - 'packages/stack-auth/src/**' + # The key vendor crates the auth binding compiles in (stash keyset). + - 'packages/stack-kms-vendor/Cargo.toml' + - 'packages/stack-kms-vendor/src/**' + - 'packages/stack-kms-aws/Cargo.toml' + - 'packages/stack-kms-aws/src/**' + - 'packages/stack-kms-vault/Cargo.toml' + - 'packages/stack-kms-vault/src/**' + - 'packages/stack-kms-gcp/Cargo.toml' + - 'packages/stack-kms-gcp/src/**' + - 'packages/stack-kms-azure/Cargo.toml' + - 'packages/stack-kms-azure/src/**' - 'languages/typescript/packages/auth/**' - 'languages/typescript/packages/stack-auth-wasm/**' # The Rust that produces every EQL payload these suites round-trip. diff --git a/.github/workflows/require-auth-npm-changeset.yml b/.github/workflows/require-auth-npm-changeset.yml index d71644af04..3180e96617 100644 --- a/.github/workflows/require-auth-npm-changeset.yml +++ b/.github/workflows/require-auth-npm-changeset.yml @@ -6,6 +6,9 @@ name: "Require @cipherstash/auth changeset" # and the npm API does not move: the npm binary is built from that crate, and # nothing else would release it. # +# The key vendor crates are listed too: the binding compiles them in for +# `stash keyset` (CIP-5312), so a change to one changes the npm binary. +# # Changes that never ship (tests, examples, devDependencies) are filtered out # by `check-auth-npm-changeset.mjs --shipped`. # @@ -18,6 +21,16 @@ on: paths: - packages/stack-auth/Cargo.toml - packages/stack-auth/src/** + - packages/stack-kms-vendor/Cargo.toml + - packages/stack-kms-vendor/src/** + - packages/stack-kms-aws/Cargo.toml + - packages/stack-kms-aws/src/** + - packages/stack-kms-vault/Cargo.toml + - packages/stack-kms-vault/src/** + - packages/stack-kms-gcp/Cargo.toml + - packages/stack-kms-gcp/src/** + - packages/stack-kms-azure/Cargo.toml + - packages/stack-kms-azure/src/** - languages/typescript/packages/auth/** - languages/typescript/packages/stack-auth-wasm/** - scripts/check-auth-npm-changeset.mjs @@ -84,6 +97,16 @@ jobs: git diff --name-only --no-renames "${BASE_SHA}...HEAD" -- \ packages/stack-auth/Cargo.toml \ packages/stack-auth/src \ + packages/stack-kms-vendor/Cargo.toml \ + packages/stack-kms-vendor/src \ + packages/stack-kms-aws/Cargo.toml \ + packages/stack-kms-aws/src \ + packages/stack-kms-vault/Cargo.toml \ + packages/stack-kms-vault/src \ + packages/stack-kms-gcp/Cargo.toml \ + packages/stack-kms-gcp/src \ + packages/stack-kms-azure/Cargo.toml \ + packages/stack-kms-azure/src \ languages/typescript/packages/auth \ languages/typescript/packages/stack-auth-wasm )"; then diff --git a/.github/workflows/tests-crates.yml b/.github/workflows/tests-crates.yml index 425dca68cb..57bc651578 100644 --- a/.github/workflows/tests-crates.yml +++ b/.github/workflows/tests-crates.yml @@ -20,6 +20,11 @@ on: - packages/stack-auth/** - packages/stack-profile/** - packages/stack-kms/** + - packages/stack-kms-vendor/** + - packages/stack-kms-aws/** + - packages/stack-kms-vault/** + - packages/stack-kms-gcp/** + - packages/stack-kms-azure/** - packages/stack-encrypt/** - conformance/stack-encrypt/** - packages/stack-encrypt-derive/** @@ -44,6 +49,11 @@ on: - packages/stack-auth/** - packages/stack-profile/** - packages/stack-kms/** + - packages/stack-kms-vendor/** + - packages/stack-kms-aws/** + - packages/stack-kms-vault/** + - packages/stack-kms-gcp/** + - packages/stack-kms-azure/** - packages/stack-encrypt/** - conformance/stack-encrypt/** - packages/stack-encrypt-derive/** @@ -151,12 +161,106 @@ jobs: # with no `version`, or a crate that uses API its dependency's packaged # source does not have yet. stack-profile and stack-auth are in the list # because stack-kms and stack-encrypt depend on them: without them, cargo - # resolves the published versions, which lag the tree. No token. + # resolves the published versions, which lag the tree. stack-kms-vendor, + # stack-kms-aws, stack-kms-vault, stack-kms-gcp and stack-kms-azure are publishable but staged in release-plz.toml (their + # first version is published by hand); packaging them here keeps them + # publishable meanwhile. No token. # `--allow-dirty` tolerates files earlier steps leave in the tree. Once a # version is on crates.io, cargo warns rather than fails on a dry run of # it. - name: Verify the stack-* crates package cleanly for crates.io - run: cargo publish --locked --dry-run --allow-dirty -p stack-profile -p stack-auth -p stack-kms -p stack-encrypt-derive -p stack-encrypt + run: cargo publish --locked --dry-run --allow-dirty -p stack-profile -p stack-auth -p stack-kms -p stack-encrypt-derive -p stack-encrypt -p stack-kms-vendor -p stack-kms-aws -p stack-kms-vault -p stack-kms-gcp -p stack-kms-azure + + # stack-kms-aws against a real AWS KMS API: LocalStack, as a service + # (ADR-0011). The `rust` job runs the same test binary, where it skips + # itself for want of an endpoint; this job sets the endpoint, and the mise + # task refuses to run without one, so a skip cannot read as a pass here. + localstack: + name: stack-kms-aws against LocalStack + runs-on: blacksmith-16vcpu-ubuntu-2204 + timeout-minutes: 30 + env: + MISE_ENV: test + STACK_KMS_AWS_LOCALSTACK_ENDPOINT: http://localhost:4566 + services: + localstack: + # Pinned by digest. 4.4 is a community release that needs no auth + # token. + image: localstack/localstack:4.4@sha256:b52c16663c70b7234f217cb993a339b46686e30a1a5d9279cb5feeb2202f837c + ports: + - 4566:4566 + env: + SERVICES: kms + options: >- + --health-cmd "curl -fs http://localhost:4566/_localstack/health" + --health-interval 5s + --health-timeout 5s + --health-retries 30 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + working_directory: . + cache: true + # Must name runs-on's image; see scripts/__tests__/workflow-mise-cache-image.test.mjs + cache_key_prefix: mise-v1-ubuntu-2204 + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + + - name: stack-kms-aws against LocalStack + run: mise run test:localstack:stack-kms-aws + + # stack-kms-vault against a real Vault Transit engine: `vault server -dev`, + # as a service (ADR-0011). The `rust` job runs the same test binary, where it + # skips itself for want of an address; this job sets the address, and the + # mise task refuses to run without one, so a skip cannot read as a pass here. + vault: + name: stack-kms-vault against vault server -dev + runs-on: blacksmith-16vcpu-ubuntu-2204 + timeout-minutes: 30 + env: + MISE_ENV: test + STACK_KMS_VAULT_ADDR: http://127.0.0.1:8200 + STACK_KMS_VAULT_TOKEN: root + services: + vault: + # Pinned by digest (the multi-platform index of 1.20, 1.20.4 when + # pinned). The image's default command is `server -dev`. + image: hashicorp/vault:1.20@sha256:268bb80aa9c6d13d65fcfa05c0c268caca068952240a8087291a6ce0b66e3a10 + ports: + - 8200:8200 + env: + VAULT_DEV_ROOT_TOKEN_ID: root + VAULT_DEV_LISTEN_ADDRESS: 0.0.0.0:8200 + SKIP_SETCAP: 'true' + options: >- + --health-cmd "wget -qO- http://127.0.0.1:8200/v1/sys/health" + --health-interval 5s + --health-timeout 5s + --health-retries 30 + steps: + - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 + with: + persist-credentials: false + + - uses: jdx/mise-action@1648a7812b9aeae629881980618f079932869151 # v4 + with: + version: 2026.4.0 + install: true + working_directory: . + cache: true + # Must name runs-on's image; see scripts/__tests__/workflow-mise-cache-image.test.mjs + cache_key_prefix: mise-v1-ubuntu-2204 + + - uses: Swatinem/rust-cache@e18b497796c12c097a38f9edb9d0641fb99eee32 # v2 + + - name: stack-kms-vault against vault server -dev + run: mise run test:vault:stack-kms-vault node-bindings: name: node bindings, napi typings, stack-auth-wasm diff --git a/.github/workflows/tests-golang.yml b/.github/workflows/tests-golang.yml index 881f4447ed..3faaf0c8d8 100644 --- a/.github/workflows/tests-golang.yml +++ b/.github/workflows/tests-golang.yml @@ -41,6 +41,12 @@ on: - conformance/stack-encrypt/** - packages/stack-encrypt-derive/** - packages/stack-guest-abi/** + # The encrypt guest links every key vendor backend (CIP-5311). + - packages/stack-kms-vendor/** + - packages/stack-kms-aws/** + - packages/stack-kms-vault/** + - packages/stack-kms-gcp/** + - packages/stack-kms-azure/** # The eql guest build links eql-bindings. - packages/eql/crates/eql-bindings/** - packages/eql/crates/eql-domains/** @@ -84,6 +90,12 @@ on: - conformance/stack-encrypt/** - packages/stack-encrypt-derive/** - packages/stack-guest-abi/** + # The encrypt guest links every key vendor backend (CIP-5311). + - packages/stack-kms-vendor/** + - packages/stack-kms-aws/** + - packages/stack-kms-vault/** + - packages/stack-kms-gcp/** + - packages/stack-kms-azure/** # The eql guest build links eql-bindings. - packages/eql/crates/eql-bindings/** - packages/eql/crates/eql-domains/** diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 7939d29e4a..17068b31eb 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -98,11 +98,10 @@ jobs: - '!docs/**' # The Go module, and the Rust crates no binding these jobs build # depends on. The auth and profile bindings compile stack-auth and - # stack-profile, which stay in. + # stack-profile, and the auth binding compiles the key vendor + # crates for `stash keyset` and, through them, stack-kms, + # stack-encrypt and stack-encrypt-derive; all of those stay in. - '!languages/golang/**' - - '!packages/stack-encrypt/**' - - '!packages/stack-encrypt-derive/**' - - '!packages/stack-kms/**' - '!packages/stack-guest-abi/**' - '!packages/stack-auth/fuzz/**' # EQL's SQL sources, test suites, codegen and tasks. What the diff --git a/.github/workflows/udeps.yml b/.github/workflows/udeps.yml index db3d33b98c..7e1cae70e4 100644 --- a/.github/workflows/udeps.yml +++ b/.github/workflows/udeps.yml @@ -14,6 +14,11 @@ on: - packages/stack-auth/** - packages/stack-profile/** - packages/stack-kms/** + - packages/stack-kms-vendor/** + - packages/stack-kms-aws/** + - packages/stack-kms-vault/** + - packages/stack-kms-gcp/** + - packages/stack-kms-azure/** - packages/stack-encrypt/** - packages/stack-encrypt-derive/** - packages/stack-guest-abi/** @@ -33,6 +38,11 @@ on: - packages/stack-auth/** - packages/stack-profile/** - packages/stack-kms/** + - packages/stack-kms-vendor/** + - packages/stack-kms-aws/** + - packages/stack-kms-vault/** + - packages/stack-kms-gcp/** + - packages/stack-kms-azure/** - packages/stack-encrypt/** - packages/stack-encrypt-derive/** - packages/stack-guest-abi/** diff --git a/AGENTS.md b/AGENTS.md index 20a056d9e9..8e493b6a63 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -91,7 +91,8 @@ Every npm package except EQL lives under `languages/typescript/`: packages in `l `cipherstash/encrypt-query-language` repository. Old upstream issue and PR links are provenance only. - `packages/stack-auth`, `packages/stack-profile`, `packages/stack-kms`, `packages/stack-encrypt`, `packages/stack-encrypt-derive`, `packages/stack-guest-abi`: The Rust crates imported from `cipherstash/cipherstash-suite` with their history — `stack-auth` and `stack-profile` (published to crates.io, one version group), `stack-kms` (published to crates.io from 0.1.0, its own version group, re-exported by `stack-encrypt` as `stack_encrypt::kms`), `stack-encrypt` and `stack-encrypt-derive` (published to crates.io from 0.1.0, one version group; `eql-bindings`' `stack-encrypt` feature depends on them from the registry), and `stack-guest-abi` (`publish = false`). They are the members of the **root Cargo workspace**, with the three node binding crates below. See "Working on the Rust crates". -- `languages/typescript/packages/auth`, `languages/typescript/packages/profile`, `languages/typescript/packages/stack-auth-wasm`: The node bindings of those crates. `@cipherstash/auth` (napi-rs v2) and its six `platforms/*` packages are published to npm from this repository by `release.yml` (`auth-artifacts`, `publish-auth`); a change to what it ships, the `stack-auth` crate included, needs an `@cipherstash/auth` changeset (`require-auth-npm-changeset.yml`). `@cipherstash/profile` and its platforms are private and never published; `@cipherstash/stack-auth-wasm` is private and builds the wasm that `@cipherstash/auth` ships. Their `build` and `test` scripts never invoke cargo; `build:native`, `build:debug` and `test:cargo` do. +- `packages/stack-kms-vendor`, `packages/stack-kms-aws`, `packages/stack-kms-vault`, `packages/stack-kms-gcp`, `packages/stack-kms-azure`: The key vendor backends (ADR-0009 to ADR-0011 in `packages/stack-encrypt/docs/adr/`). `stack-kms-vendor` is the vendor seam (`KeyVendorBackend`), keyset records, both key modes, the static key backend and the `VendorTransport` HTTP seam (feature `http`: the native reqwest transport; the WASI guest's transport is `stack-guest-abi`'s `vendor` feature). `stack-kms-aws` is the AWS KMS backend: a hand-written JSON client with SigV4 signing in-crate and the AWS credential providers; its LocalStack test runs in `tests-crates.yml`'s `localstack` job. `stack-kms-vault` is the HashiCorp Vault Transit backend: a hand-written Transit client with `batch_input` calls and token, AppRole and Kubernetes auth; its `vault server -dev` test runs in `tests-crates.yml`'s `vault` job. `stack-kms-gcp` is the Google Cloud KMS backend: a hand-written Cloud KMS REST client with CRC32C checks, service account (RS256 JWT, the pure-Rust `rsa` crate, which has the open advisory RUSTSEC-2023-0071) and metadata server credentials; the service account login is the default feature `service-account`, and the Go guest and `@cipherstash/auth` build the crate without it, so they hold no RSA code; Google has no Cloud KMS emulator, so its live test runs only in the opt-in, credentialed `integration-gcp-kms.yml`, which is inert until the `STACK_KMS_GCP_LIVE_KEY` repository variable is set. `stack-kms-azure` is the Azure Key Vault and Managed HSM backend: a hand-written Key Vault REST client (`wrapkey`/`unwrapkey`, RSA-OAEP-256 or A256KW, the associated data bound by `stack_kms_vendor::bound` because Key Vault has none) with Entra ID client secret, workload identity and managed identity credentials, and no RSA code of its own; Microsoft has no Key Vault emulator, so its live test runs only in the opt-in, credentialed `integration-azure-key-vault.yml`, which is inert until the `STACK_KMS_AZURE_LIVE_KEY` repository variable is set. All five are root workspace members, publishable in the `stack-kms-vendor` version group, and staged in `release-plz.toml` (`release = false`) until their first version is published by hand. The `stack-kms-vendor` crate docs hold the capability matrix of every key backend (key lifetime, vendor calls, what the vendor sees and can refuse, credentials, rotation, known limits); change it with the code it describes, and keep the short matrix in the Go `encrypt` package docs in step. The `@cipherstash/auth` binding compiles all five in for `stash keyset`, so a change under their `src/` needs an `@cipherstash/auth` changeset too. TypeScript (`@cipherstash/stack`) cannot use a key vendor backend yet. +- `languages/typescript/packages/auth`, `languages/typescript/packages/profile`, `languages/typescript/packages/stack-auth-wasm`: The node bindings of those crates. `@cipherstash/auth` (napi-rs v2) and its six `platforms/*` packages are published to npm from this repository by `release.yml` (`auth-artifacts`, `publish-auth`); a change to what it ships, the `stack-auth` crate and the five key vendor crates included (the binding compiles them in for `keysetCreate` and `keysetRewrap`, which `stash keyset` calls), needs an `@cipherstash/auth` changeset (`require-auth-npm-changeset.yml`). `@cipherstash/profile` and its platforms are private and never published; `@cipherstash/stack-auth-wasm` is private and builds the wasm that `@cipherstash/auth` ships. Their `build` and `test` scripts never invoke cargo; `build:native`, `build:debug` and `test:cargo` do. - `languages/golang`: The Go module — the SDK `encrypt` with its generated-code support `encrypt/gensupport` and the policy packages `encrypt/policy` and `encrypt/policy/protosource`; the credential package `auth`; the generator `stashgen` and its command `cmd/stashgen`; and `internal` (the shared guest plumbing and the `record` wire model). A wazero host with no cgo. Its two WASI guests (`encrypt/guest`, `auth/guest`) are detached Cargo workspaces built by `mise run wasm:guest:build` and `mise run wasm:auth-guest:build`; `mise run wasm:guest:build:deterministic` builds the seeded TEST build the hermetic Go tests use; the `.wasm` files they embed are gitignored. The stack-encrypt guest has a second build with the EQL types (`mise run wasm:guest:build:eql`, cargo feature `eql`, linking `eql-bindings` by path; `wasm:guest:build:eql:deterministic` is its test build), embedded by `encrypt/eql` and registered on import, so a program whose generated code names an EQL type runs it (ADR-0007, amended). `encrypt/eql`'s types and `Types` table are generated from the EQL catalog by `eql-codegen` (`eql_gen.go`, written by `mise run types:generate` in `packages/eql` and drift-gated by `types:check` and `cargo test -p eql-codegen`). Generated `*_stash.go` files are committed and CI fails when `go generate ./...` changes one. Releases are cut by hand with `.github/workflows/release-golang.yml` (`gh workflow run release-golang.yml -f version=X.Y.Z[-pre] --ref `): because the guests are gitignored on main and Go fetches a version as the files committed at its tag, the workflow builds them, commits them on a release-only commit no branch points at, tags it `languages/golang/vX.Y.Z` and pushes the tag alone, then attaches the EQL v4 SQL bundle to the GitHub release. The ref must carry the EQL v4 bundle, or the build stops at "Build the EQL SQL bundles". `scripts/go-release-version.mjs` refuses, before anything is built, every version Go would refuse (non-canonical semver, `+build` metadata, a v2+ major on a module path with no `/vN` suffix, a version shaped like a pseudo-version), a final version dispatched from anywhere but `main`, and a final version while `languages/golang/go.mod` pins a `github.com/cipherstash/vitaminc/…` module at a pseudo-version (vitaminc's Go bindings have no tags, and a TEMPORARY pin on an unreleased commit must be gone before the module is tagged); a pre-release may come from a branch and may carry such a pin. The build job also refuses a version the Go proxy already lists in `@v/list`, which catches a tag deleted after the proxy fetched it; it never asks the proxy for the version itself before the tag exists, because the proxy would cache "not found" for up to 30 minutes and `verify` would fail after the push. The build checks the `Cargo.lock` of the EQL workspace and both guest workspaces with `--locked` first, since none of those builds passes it. It runs in a job with no write token and hands the guests to a `publish` job that holds `contents: write` and runs only git and gh. `publish` fetches the commit with plain unauthenticated git rather than `actions/checkout`, whose `token` input defaults to the job's writable token, and refuses a release commit whose tree is not the one the build tested. `publish` takes the paths it commits from the workflow's fixed `GUESTS` list, never from the artifact, and refuses a `guests.sha256` that is not exactly one SHA-256 line for each of those three guests: the build job runs third-party code after writing that manifest, so a path read from it could put any file on the tag (`scripts/__tests__/go-release-artifact.test.mjs`). A version is immutable once the Go proxy has fetched it, so a tag is never moved or deleted: if `publish` or `verify` fails after the tag is pushed, use "Re-run failed jobs" on the same run, which keeps a tag that run pushed and creates or completes the release; a new dispatch of the same version is refused. Never tag a main commit by hand: `go get` would fetch a module with no guests. - `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README) - `languages/typescript/examples/*`: Working apps (basic, prisma, supabase-worker) @@ -715,7 +716,7 @@ term. `cts-common` and `zerokms-protocol` take caret requirements, because the published `stack-auth` inherits them, and an exact pin would stop the suite sharing one `cts-common` with it. -- **Three Cargo workspaces, not one.** The root workspace (the six stack-* +- **Three Cargo workspaces, not one.** The root workspace (the eleven stack-* crates and the three node binding crates), protect-ffi's and EQL's. The root `Cargo.toml` excludes the other two, and they pin their own vitaminc. The three fuzz crates, the two Go guests and the EQL v4 block ORE timing @@ -757,7 +758,7 @@ exact pin would stop the suite sharing one `cts-common` with it. `MISE_ENV=test` cargo tools — and the live tests), `crap-crates.yml`, `mutants.yml`, `fuzz.yml`, `miri.yml` and `udeps.yml`. `scripts/__tests__/crates-ci.test.mjs` fails when a mise task in the root - `mise.toml` or the five crate `tasks.toml` files loses its last CI caller. + `mise.toml` or the ten crate `tasks.toml` files loses its last CI caller. - **Auth refresh cancellation soak:** `tests-auth-refresh.yml` runs a short, seeded, virtual-time soak on every PR and the long run on auth changes and Version Packages PRs. `release-plz.yml` also requires the long run before diff --git a/Cargo.lock b/Cargo.lock index c452341fd1..d3b99f6df7 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -645,6 +645,12 @@ dependencies = [ "serde_core", ] +[[package]] +name = "const-oid" +version = "0.9.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8" + [[package]] name = "const-oid" version = "0.10.2" @@ -771,6 +777,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" dependencies = [ "generic-array", + "rand_core 0.6.4", "typenum", ] @@ -887,6 +894,17 @@ dependencies = [ "generic-array", ] +[[package]] +name = "der" +version = "0.7.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7c1832837b905bbfb5101e07cc24c8deddf52f93225eee6ead5f4d63d53ddcb" +dependencies = [ + "const-oid 0.9.6", + "pem-rfc7468", + "zeroize", +] + [[package]] name = "deranged" version = "0.5.5" @@ -932,6 +950,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "9ed9a281f7bc9b7576e61468ba615a66a5c8cfdff42420a70aa82701a3b1e292" dependencies = [ "block-buffer 0.10.4", + "const-oid 0.9.6", "crypto-common 0.1.7", "subtle", ] @@ -943,7 +962,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2" dependencies = [ "block-buffer 0.12.0", - "const-oid", + "const-oid 0.10.2", "crypto-common 0.2.1", "ctutils", "zeroize", @@ -1400,6 +1419,15 @@ dependencies = [ "tracing", ] +[[package]] +name = "hkdf" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4aaa26c720c68b866f2c96ef5c1264b3e6f473fe5d4ce61cd44bbe913e553018" +dependencies = [ + "hmac", +] + [[package]] name = "hmac" version = "0.13.0" @@ -1882,6 +1910,9 @@ name = "lazy_static" version = "1.5.1" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "20870f649af7073d53e38067b2a84312175d56ea15217e1b15bc83506ec50afb" +dependencies = [ + "spin", +] [[package]] name = "leb128fmt" @@ -2204,6 +2235,22 @@ dependencies = [ "num-traits", ] +[[package]] +name = "num-bigint-dig" +version = "0.8.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e661dda6640fad38e827a6d4a310ff4763082116fe217f279885c97f511bb0b7" +dependencies = [ + "lazy_static", + "libm", + "num-integer", + "num-iter", + "num-traits", + "rand 0.8.8", + "smallvec", + "zeroize", +] + [[package]] name = "num-complex" version = "0.4.6" @@ -2353,6 +2400,15 @@ dependencies = [ "serde_core", ] +[[package]] +name = "pem-rfc7468" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "88b39c9bfcfc231068454382784bb460aae594343fb030d46e9f50a645418412" +dependencies = [ + "base64ct", +] + [[package]] name = "percent-encoding" version = "2.3.2" @@ -2371,6 +2427,27 @@ version = "0.1.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" +[[package]] +name = "pkcs1" +version = "0.7.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8ffb9f10fa047879315e6625af03c164b16962a5368d724ed16323b68ace47f" +dependencies = [ + "der", + "pkcs8", + "spki", +] + +[[package]] +name = "pkcs8" +version = "0.10.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f950b2377845cebe5cf8b5165cb3cc1a5e0fa5cfa3e1f7f55707d8fd82e0a7b7" +dependencies = [ + "der", + "spki", +] + [[package]] name = "pkg-config" version = "0.3.32" @@ -2860,6 +2937,27 @@ dependencies = [ "serde", ] +[[package]] +name = "rsa" +version = "0.9.10" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b8573f03f5883dcaebdfcf4725caa1ecb9c15b2ef50c43a07b816e06799bb12d" +dependencies = [ + "const-oid 0.9.6", + "digest 0.10.7", + "num-bigint-dig", + "num-integer", + "num-traits", + "pkcs1", + "pkcs8", + "rand_core 0.6.4", + "sha2 0.10.9", + "signature", + "spki", + "subtle", + "zeroize", +] + [[package]] name = "rust_decimal" version = "1.43.0" @@ -3255,6 +3353,7 @@ version = "2.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "77549399552de45a898a580c1b41d445bf730df867cc44e6c0233bbc4b8329de" dependencies = [ + "digest 0.10.7", "rand_core 0.6.4", ] @@ -3324,6 +3423,22 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "spin" +version = "0.9.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3763264f6b73151db08c50ff20d7d8a0b8796e021cdea7ceedad07b80155fa0e" + +[[package]] +name = "spki" +version = "0.7.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d91ed6c858b01f942cd56b37a94b3e0a1798290327d1236e4d9cf4eaca44d29d" +dependencies = [ + "base64ct", + "der", +] + [[package]] name = "stable_deref_trait" version = "1.2.1" @@ -3372,18 +3487,27 @@ dependencies = [ name = "stack-auth-node" version = "0.35.0" dependencies = [ + "base64 0.22.1", "cts-common", "jsonwebtoken", + "miette", "mocktail", "napi", "napi-build", "napi-derive", "serde_json", "stack-auth", + "stack-kms", + "stack-kms-aws", + "stack-kms-azure", + "stack-kms-gcp", + "stack-kms-vault", + "stack-kms-vendor", "stack-profile", "tempfile", "tokio", "url", + "vitaminc 0.9.0", "vitaminc-protected 0.9.1", "zeroize", ] @@ -3467,6 +3591,7 @@ dependencies = [ "miette", "proptest", "serde_json", + "stack-kms-vendor", "thiserror 1.0.69", "vitaminc-aead 0.9.0", "vitaminc-aead-frame", @@ -3509,6 +3634,121 @@ dependencies = [ "zerokms-protocol", ] +[[package]] +name = "stack-kms-aws" +version = "0.1.0" +dependencies = [ + "aes-gcm", + "base64ct", + "futures", + "hmac", + "serde", + "serde_json", + "sha2 0.11.0", + "stack-encrypt", + "stack-kms", + "stack-kms-vendor", + "thiserror 1.0.69", + "tokio", + "url", + "uuid", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + +[[package]] +name = "stack-kms-azure" +version = "0.1.0" +dependencies = [ + "aes", + "base64ct", + "futures", + "rand_core 0.6.4", + "rsa", + "serde", + "serde_json", + "stack-encrypt", + "stack-kms", + "stack-kms-vendor", + "thiserror 1.0.69", + "tokio", + "url", + "uuid", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + +[[package]] +name = "stack-kms-gcp" +version = "0.1.0" +dependencies = [ + "aes-gcm", + "base64ct", + "futures", + "rand_core 0.6.4", + "rsa", + "serde", + "serde_json", + "stack-encrypt", + "stack-kms", + "stack-kms-vendor", + "thiserror 1.0.69", + "tokio", + "url", + "uuid", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + +[[package]] +name = "stack-kms-vault" +version = "0.1.0" +dependencies = [ + "aes-gcm", + "base64ct", + "futures", + "serde", + "serde_json", + "sha2 0.11.0", + "stack-encrypt", + "stack-kms", + "stack-kms-vendor", + "thiserror 1.0.69", + "tokio", + "url", + "uuid", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + +[[package]] +name = "stack-kms-vendor" +version = "0.1.0" +dependencies = [ + "aes-gcm", + "base64ct", + "futures", + "hkdf", + "miette", + "reqwest", + "serde_json", + "sha2 0.11.0", + "stack-auth", + "stack-encrypt", + "stack-kms", + "subtle", + "thiserror 1.0.69", + "tokio", + "uuid", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + [[package]] name = "stack-profile" version = "0.44.0" diff --git a/Cargo.toml b/Cargo.toml index 7dec619632..50cee708c3 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -9,6 +9,11 @@ members = [ "packages/stack-auth", "packages/stack-profile", "packages/stack-kms", + "packages/stack-kms-vendor", + "packages/stack-kms-aws", + "packages/stack-kms-vault", + "packages/stack-kms-gcp", + "packages/stack-kms-azure", "packages/stack-encrypt", "packages/stack-encrypt-derive", "packages/stack-guest-abi", diff --git a/SECURITY.md b/SECURITY.md index 8c44a1d228..fb312f97c7 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -19,7 +19,7 @@ This repository is the CipherStash Stack monorepo for JavaScript/TypeScript. It | `@cipherstash/wizard` | AI-powered encryption setup | | `@cipherstash/protect-ffi` | Native FFI bindings to the CipherStash Client SDK — the Rust core `@cipherstash/stack` encrypts and decrypts through | | `@cipherstash/protect-ffi-darwin-arm64`
`@cipherstash/protect-ffi-darwin-x64`
`@cipherstash/protect-ffi-linux-arm64-gnu`
`@cipherstash/protect-ffi-linux-x64-gnu`
`@cipherstash/protect-ffi-linux-x64-musl`
`@cipherstash/protect-ffi-win32-x64-msvc` | Prebuilt per-platform binaries for `@cipherstash/protect-ffi`. Installed as optional dependencies; one is selected at load time for the host platform | -| `@cipherstash/auth` | Authentication strategies for CipherStash (napi-rs native binding, with a WASM build for edge runtimes) — imported from `cipherstash/cipherstash-suite` with the `stack-auth` crate it wraps | +| `@cipherstash/auth` | Authentication strategies for CipherStash (napi-rs native binding, with a WASM build for edge runtimes) — imported from `cipherstash/cipherstash-suite` with the `stack-auth` crate it wraps. It also carries `keysetCreate` and `keysetRewrap` for `stash keyset`, which compile in the key vendor crates listed below | | `@cipherstash/auth-darwin-arm64`
`@cipherstash/auth-darwin-x64`
`@cipherstash/auth-linux-arm64-gnu`
`@cipherstash/auth-linux-x64-gnu`
`@cipherstash/auth-linux-x64-musl`
`@cipherstash/auth-win32-x64-msvc` | Prebuilt per-platform binaries for `@cipherstash/auth`. Installed as optional peer dependencies; one is selected at load time for the host platform | | `@cipherstash/eql` | Encrypt Query Language — the PostgreSQL SQL bundle (`eql_v3` schema: domains, operators, index-term extractors) that stores and queries encrypted payloads, plus its generated TypeScript types. Applied by `stash eql install` and by the Prisma Next adapter's migrations. Released in lockstep with the `eql-bindings` Rust crate, which emits the payloads this SQL reads | @@ -38,6 +38,13 @@ from the stack-* crates), released by `.github/workflows/release-golang.yml` as `languages/golang/vX.Y.Z` tags. All of these are in scope for security reports on the same terms as the npm packages above. +Five more Rust crates, **`stack-kms-vendor`**, **`stack-kms-aws`**, +**`stack-kms-vault`**, **`stack-kms-gcp`** and **`stack-kms-azure`** +(`packages/stack-kms-vendor`, `packages/stack-kms-aws`, `packages/stack-kms-vault`, +`packages/stack-kms-gcp`, `packages/stack-kms-azure`), are prepared for +crates.io but not yet published. They are in scope on the same terms, and +under the same licence as the stack-* crates below. + The five stack-* crates are licensed under the PolyForm Internal Use License 1.0.0 (each crate's `LICENSE`), not the repository's MIT licence: they may be used for internal business operations only, and not redistributed. That diff --git a/biome.json b/biome.json index 435aa70b04..f82e9c9fc9 100644 --- a/biome.json +++ b/biome.json @@ -15,6 +15,7 @@ "!packages/stack-encrypt/tests/fixtures/value_kinds.json", "!packages/stack-encrypt/tests/fixtures/stored_rows_v1.json", "!packages/stack-encrypt/tests/fixtures/json_tree_vectors.json", + "!packages/stack-kms-*/tests/fixtures", "!packages/eql/tests/encryption/fixtures/ste_vec_selectors.json", "!packages/eql/tests/encryption/fixtures/v4_terms.json", "!packages/eql/tests/encryption/fixtures/v4_rows.json", diff --git a/docs/plans/2026-11-05-key-vendor-backends-decisions.md b/docs/plans/2026-11-05-key-vendor-backends-decisions.md new file mode 100644 index 0000000000..8a4f6a49ea --- /dev/null +++ b/docs/plans/2026-11-05-key-vendor-backends-decisions.md @@ -0,0 +1,1817 @@ +# Key vendor backends: decision log (CIP-5203) + +The design is in ADR-0009, ADR-0010 and ADR-0011 +(`packages/stack-encrypt/docs/adr/`) and in the CIP-5203 sub-issues. This +file records the decisions made **during implementation without asking**, so +that they can be revised later. Each entry names the ticket, the decision, +the alternatives and the reason. + +## Stack structure + +- **Decision:** the work is one stack of pull requests on `main`. The design + PR (#1521, ADRs and `CONTEXT.md`) is the bottom layer, rebased onto the + latest `origin/main`, so every code layer can refer to the ADRs. + **Alternative:** keep #1521 outside the stack. **Reason:** the code layers + cite the ADRs and glossary terms. +- **Decision:** the layer order is CIP-5304 (trait), CIP-5305 (v2 leaf), + CIP-5306 (keyset records, modes, static), CIP-5307 (AWS), CIP-5310 (Vault), + CIP-5308 (GCP), CIP-5309 (Azure), CIP-5311 (Go), CIP-5312 (`stash keyset`), + CIP-5313 (docs). **Reason:** AWS and Vault have local emulators (LocalStack, + `vault server -dev`), so they prove the vendor transport before the two + vendors that have only recorded fixtures. + +## CIP-5304: Make DataKeySource and IndexKeySource vendor-neutral + +- **Decision:** the trait signatures of `DataKeySource` and `IndexKeySource` + do not change. The payload structs keep their ZeroKMS fields (`context`, + `decryption_policy`, the `KeyId` iv) and the `UnverifiedContext` + argument stays. New: `ZeroKmsFeature`, the `ZeroKmsExtensions` trait on + both payloads, `refuse_zerokms_extensions(vendor, payloads, + unverified_context)`, and neutral accessors (`descriptor()`, + `iv_bytes()`). A key vendor backend calls the refusal first in each + method and gets `Error::ZeroKmsOnly { vendor, feature }`. + **Alternative:** remove the ZeroKMS types from the payloads and the + trait methods. **Reason:** that breaks 22 implementations (stack-encrypt + tests, the Go guest tests, the EQL encryption tests) for no gain in + safety. A backend can ignore a public field either way; the refusal + helper is what makes "refuse, never ignore" easy to do. The ticket text + says "remove"; this is the weaker reading, chosen with the orchestrator's + guidance. +- **Decision:** an empty key context and an empty `UnverifiedContext` are + not ZeroKMS features, and a key vendor backend accepts them. + **Alternative:** refuse `Some(empty)`. **Reason:** they bind nothing. + stack-encrypt always sends an empty context and `None`. +- **Decision:** the key lifetime comes from a new provided method, + `IndexKeySource::load_index_key_with_lifetime`, which returns + `LoadedIndexKey { keyset_id, key, lifetime }`. The default + implementation calls `load_index_key` and returns + `KeyLifetime::UntilEvicted`. The cipher calls only the new method. + **Alternative:** change the return type of `load_index_key`. + **Reason:** additive; every existing source keeps the ZeroKMS behaviour + with no edit. A wrapper that forwards only `load_index_key` reports + `UntilEvicted`, which is correct for every wrapper in the tree today. +- **Decision:** `KeyLifetime` is `Settle`, `Ttl(Duration)` and + `UntilEvicted` (the default), `#[non_exhaustive]`. `Ttl(ZERO)` is the + same as `Settle`. A TTL past the range of `Instant` is `UntilEvicted`. A + lifetime the cipher does not know is never cached. **Reason:** the short + answer is the safe one. +- **Decision:** the cipher enforces the lifetime in `KeysetCache` + (`KeysetState.retention`). A `Settle` key is never inserted: each + `StackCipher::keyset` call loads it again, and the `KeysetCipher` handle + (and the pendings built from it) holds the only reference. A TTL key is + evicted on the first lookup at or after its end, and every insert sweeps + out all expired entries. Expiry raises the watermark as LRU eviction + does. **Alternative:** load the index key inside each settle. + **Reason:** term derivation is synchronous and reads `KeysetCipher::prf` + while the target is built, before the settle. Moving the load into the + settle changes the public `Term`/`Pending` shape. So in this layer "one + settle" means "one keyset selection": a caller that keeps a handle keeps + the key. The docs on `StackCipher::keyset` say so. +- **Decision:** `StackCipherBuilder::init` refuses a default keyset whose + lifetime is not `UntilEvicted`, with the new + `stack_encrypt::Error::DefaultKeysetLifetime { keyset }` + (`stack_encrypt::default_keyset_lifetime`). **Alternative:** keep the + default anyway, or make `default_keyset()` async. **Reason:** + `default_keyset()` is synchronous and holds the default for the life of + the cipher, so the cipher cannot obey a shorter lifetime. Refusing is + "refuse, never ignore". Changing `default_keyset()` is a public API break. +- **Decision:** `stack_kms::Error::Backend { vendor: &'static str, kind: + BackendErrorKind, source }` has one code, `stack_kms::backend`, and the + payload fields `vendor` and `kind`. `BackendErrorKind` is `NotFound`, + `AccessDenied`, `Throttled`, `Unavailable`, `InvalidCiphertext`, + `#[non_exhaustive]`. The message is only the vendor and the kind. + **Alternative:** one code per kind; a vendor enum. **Reason:** the miette + derive gives one code per variant, and the kind is in the payload. A + vendor enum would fix the vendor list before CIP-5305 numbers the v2 + leaf's backend byte. +- **Decision:** `Error::Backend` and `Error::ZeroKmsOnly` are new variants of + the exhaustive `stack_kms::Error`. That is a breaking change for a caller + that matches it exhaustively (the Go guest's `status_for_kms` did). The + commit is marked breaking (`!`) so release-plz bumps stack-kms 0.1 to + 0.2. **Alternative:** make `Error` `#[non_exhaustive]` now. **Reason:** + the guest relies on exhaustiveness on purpose (see `status.rs`). +- **Decision:** the Go guest maps `Backend` kinds: `NotFound` to + `STATUS_KMS_NOT_FOUND`, `AccessDenied` to `STATUS_KMS_FORBIDDEN`, + `Throttled` and `Unavailable` to `STATUS_KMS_TRANSPORT`, + `InvalidCiphertext` to `STATUS_AUTH`; `ZeroKmsOnly` to `STATUS_ENCODING`. + **Reason:** the nearest existing status for each. CIP-5311 can revisit. +- **Decision:** `IndexKey::from_key_bytes(Protected<[u8; 32]>)` is the + public constructor from raw key material. `From` stays + `test-support` only. **Reason:** `Protected` wipes the caller's copy on + drop and does not print. +- **Decision:** `is_keyset_not_found` also matches `Error::Backend { kind: + NotFound }`, so a key vendor backend's "no such keyset" forgets a name + binding as ZeroKMS's does. +- **Decision:** `Descriptor::MAX_LEN` is the literal `512`, with a test that + it equals `stack_kms::MAX_DESCRIPTOR_LEN`. The `DescriptorTooLong` message + and help say "key descriptor" and "a key backend", not "ZeroKMS". + **Reason:** the limit stays the same for every backend; only the source + of the constant moves. +- **Decision:** `CONTEXT.md` gets a glossary entry for **key lifetime**. +- **Deferred to CIP-5306:** the rule "a cipher holds ZeroKMS and at most one + key vendor backend, a second vendor is a typed error". The trait layer + has no notion of vendor yet (a cipher holds one `K`), so there is nothing + to enforce until the builder takes a key vendor backend beside ZeroKMS. +- **Deferred to CIP-5306:** how a key vendor backend's default keyset is + chosen, given that `init` refuses a default with a short lifetime. +- **Deferred (no ticket yet):** binding a `Settle` index key to exactly one + settle inside a held `KeysetCipher` handle (see above). +- **Note:** `stack-encrypt::ui derive_diagnostics` (trybuild) fails on this + machine on the base commit too: the snapshots print relative paths and + the local run prints `/…` paths. It is not caused by this change. + +## Orchestrator review of CIP-5304 + +- **Decision:** accept that, in CIP-5304, a `Settle` key lifetime lasts as + long as one keyset selection (one `KeysetCipher` handle), not one settle. + **CIP-5306 must close this gap** before it ships wrapped data keys: for a + keyset whose index key has the `Settle` lifetime, the term PRF runs inside + the settle (the deferred-backend path that `sem/mod.rs` already describes), + and the handle does not hold the index key. **Alternative:** send CIP-5304 + back. **Reason:** the gap matters only once a backend returns `Settle`, and + no backend does until CIP-5306. ADR-0009's "no key outlives the settle that + needed it" is the target, and it is unchanged. +- **Decision:** accept that the trait signatures and payload fields keep their + ZeroKMS types, and that a vendor backend refuses them through + `refuse_zerokms_extensions`. **Alternative:** remove the types now, which + breaks 22 implementations. **Reason:** the refusal is what ADR-0009 needs, + and the change is additive. Revisit when TypeScript (CIP-5314) moves onto + stack-encrypt. +- **Note:** `stack-encrypt::ui derive_diagnostics` (trybuild) fails on `main` + on a developer machine whose paths print as ``. It is not caused by + this stack. CI is the judge. + +## CIP-5305: v2 sealed leaf with key backend and key mode + +- **Decision:** the backend and mode are two `stack-kms` enums, `KeyVendor` + and `KeyMode`, with stable bytes. `KeyVendor`: `Aws = 1`, `Gcp = 2`, + `Azure = 3`, `Vault = 4`, `Static = 5`. `KeyMode`: `Wrapped = 1`, + `Derived = 2`. The value `0` is invalid for both, so a zeroed header + never decodes. A value is never reused. `KeyBackend` is `ZeroKms` or + `Vendor(KeyVendor)`, so ZeroKMS has no backend byte and a v2 leaf cannot + name it. **Alternative:** one enum with a ZeroKMS byte; the enums in + `stack-encrypt`. **Reason:** the trait method (below) lives in + `stack-kms` and must return the type. Leaving ZeroKMS out of the byte + enum makes "ZeroKMS is never v2" a type fact. +- **Decision:** `KeyVendor` is `#[non_exhaustive]`; `KeyMode` and + `KeyBackend` are exhaustive. **Reason:** a new vendor is likely; ADR-0009 + rejects a third mode, and `KeyBackend` has exactly two shapes. +- **Decision:** `KeyVendor::name()` gives stable names (`aws-kms`, + `gcp-kms`, `azure-key-vault`, `vault-transit`, `static`), and a key + vendor backend should use it as the `vendor` of `Error::Backend` and + `Error::ZeroKmsOnly`. `KeyBackend::ZeroKms` is `zerokms`. +- **Decision:** the seam is a provided method, + `DataKeySource::key_backend(&self) -> KeyBackend`, default `ZeroKms`. + **Alternative:** a method on `IndexKeySource`, or a per-keyset answer. + **Reason:** additive; every existing source keeps ZeroKMS with no edit, + and ADR-0010 routes a leaf by the backend the cipher holds, not by the + keyset. A wrapper must forward it (the doc says so); no wrapper in the + tree needs to today. +- **Decision:** `SealedValue` keeps one type and gains a private + `vendor: Option` field. `None` is v1. New API: + `VENDOR_FORMAT_VERSION = 2`, `format_version()`, `key_backend()`, + `key_mode()`, `from_vendor_parts(vendor, mode, keyset_id, iv, tag, ct)`. + `FORMAT_VERSION` stays `1` and still names v1. `into_parts` is + unchanged and drops the v2 header (documented: persist v2 with + `to_bytes`). **Alternative:** a separate v2 type, or an enum leaf. + **Reason:** every tree, visitor and binding carries `SealedValue`; one + type keeps v2 opaque to all of them. +- **Decision:** the serde form of a v1 leaf does not change. A v2 leaf + adds a last field `vendor: [backend, mode]`, skipped when `None` and + defaulted on read. **Reason:** keeps stored v1 serde data readable; no + caller in the tree uses serde for leaves. +- **Decision:** the leaf AAD's second PAE piece is the envelope header: + `[0x01]` for v1 (byte for byte the old AAD; the existing pin is + unchanged) and `[0x02, backend, mode]` for v2. **Alternative:** two + extra PAE pieces for v2. **Reason:** PAE frames each piece with its + length, so a 3-byte header can never collide with a 1-byte one, and the + v1 derivation stays identical. A new pin covers the v2 AAD. +- **Decision:** the routing refusal is + `stack_encrypt::Error::ForeignBackend { expected, found }` + (`stack_encrypt::foreign_backend`), raised in `dispatch` before any + generate or retrieve call, for any retrieve whose leaf names another + backend than `cipher.kms().key_backend()`. It is symmetric: a vendor + cipher refuses a v1 leaf too. Each retrieve `Request` now carries the + leaf's backend; the new public `Request::retrieve_leaf_key(&leaf, + context)` sets it, and the old `retrieve_data_key` names ZeroKMS. + **Alternative:** check in `check_scope` at pending construction. + **Reason:** `check_scope` has no cipher; `dispatch` is the one gate + every request passes before I/O, and the refusal is still before any + key call. A whole tree is refused if one leaf is foreign. +- **Decision:** a generate request under a key vendor backend is refused + with `stack_encrypt::Error::SealUnsupported { backend }` + (`stack_encrypt::seal_unsupported`), before any key call. **Alternative:** + write v1 (it would name ZeroKMS, wrongly) or wire v2 sealing now. + **Reason:** the v2 seal needs the keyset's key mode, which is CIP-5306's + keyset record. The private `seal_leaf_as(Some(header), ..)` already + writes v2; the tests use it. +- **Decision:** `LeafBytesError` gains `UnknownBackend(u8)` + (`stack_encrypt::leaf_backend`) and `UnknownMode(u8)` + (`stack_encrypt::leaf_mode`). A v1-only reader (an older build) refuses + v2 with `UnknownVersion(2)`, as ADR-0010 says. +- **Decision:** the Go guest maps `ForeignBackend` to `STATUS_ENCODING` + (the caller's input; no keyset option opens it, so not + `STATUS_FOREIGN_KEYSET`). `SealUnsupported` falls to `STATUS_INTERNAL` + (the guest holds ZeroKMS, so it cannot occur). CIP-5311 can revisit. +- **Decision:** a v1 known-answer test (`a_pinned_v1_leaf_still_opens` in + `tests/frozen_bytes.rs`) pins the bytes of a real v1 leaf sealed under + `DeterministicSource([7; 32])` and opens it. The existing layout pin + (`sealed_value_layout_is_pinned`) is unchanged. +- **Decision:** `eql-bindings` needs no code change. Its `c` is still + `stack-encrypt:1:` + base64 of `SealedValue::to_bytes`; the `1` there is + the EQL producer marker, not the leaf version. A new test + (`a_v2_leaf_passes_through_the_ciphertext_as_opaque_bytes`) proves a v2 + leaf round-trips through `c`. +- **Decision:** fuzzing reuses the `sealed_value_decode` target (it already + checks "no panic" and "lossless" for every input) and adds two v2 seeds + (`valid-leaf-v2-aws-wrapped`, `valid-leaf-v2-static-derived-empty-tag`). + The seed replay and a 30 s run passed. +- **Deferred to CIP-5306:** the v2 seal path (choose the leaf header from + the keyset's backend and key mode), and passing the leaf's key mode to + the key source on retrieve (`RetrieveKeyPayload` does not carry it yet; + the cipher has it in `SealedValue::key_mode`). Remove or keep + `Error::SealUnsupported` then. +- **Deferred to CIP-5306:** a cipher that holds ZeroKMS and one key vendor + backend routes each leaf to the right one. Today a cipher holds one `K`, + so `ForeignBackend` compares with that one backend. + +## CIP-5306: Keyset records, the two key modes, and the static key backend + +- **Decision:** the vendor code lives in a new crate, + `packages/stack-kms-vendor` (`stack-kms-vendor`), in the root workspace. + It depends on `stack-kms` only (plus `stack-encrypt` behind its + `test-support` feature, for the contract suite). It is `publish = false` + and listed in `release-plz.toml` with `release = false`. + **Alternative:** put it in `stack-kms`. **Reason:** `stack-kms` is the + ZeroKMS client; the vendor seam, records, HKDF and AES-GCM are a separate + concern, and each vendor crate depends on this one crate. + **Deferred to CIP-5307:** decide its release group and publish it with + the first vendor crate, which cannot publish while it depends on an + unpublished crate. +- **Decision:** the vendor seam is the trait `KeyVendorBackend`: `vendor()`, + `supports_mode(mode)` (default both), `wrap(key_ref, key, aad)`, + `unwrap(key_ref, wrapped, aad)`, `generate_wrapped(key_ref, aad)` + (default: a CSPRNG key and one `wrap`) and `sleep(duration)` (default: + return at once). Keys are `KeyBytes = Protected<[u8; 32]>`. A vendor + reports failures as `stack_kms::Error::Backend` with `KeyVendor::name()`. + The adapter `VendorKeySource` implements `DataKeySource` and + `IndexKeySource` on it, so a vendor crate never touches those traits. + **Reason:** the user named the term "key vendor backend"; the trait is + the smallest thing all four vendors can do. +- **Decision:** a vendor with no native associated data (Azure Key Vault + RSA wrap) must bind `aad` itself, for example by wrapping + `SHA-256(aad) ‖ key`. The trait docs say so. **Deferred to CIP-5309.** +- **Decision:** the keyset record text form is `stack-keyset:` + unpadded + base64url of `version(1)=1 ‖ backend(1) ‖ mode(1) ‖ keyset id(16) ‖ + name len(1, 0 = none) ‖ name ‖ key ref len(2, BE) ‖ key ref ‖ wrapped + len(2, BE) ‖ wrapped key`. Parsing refuses an unknown version, backend + or mode, a length mismatch, an empty or over-long field and invalid + UTF-8. A test pins the text of one record. **Alternative:** CBOR or + JSON. **Reason:** a fixed binary layout has one encoding per record, no + new dependency, and the backend and mode bytes are the v2 leaf's. +- **Decision:** the wrapped key in a record is bound with associated data + `frame["stack-kms-vendor/keyset-record-key/v1", purpose, keyset id, + vendor byte, mode byte]` (purpose `root-key` or `index-key`). A record + edited in its id, backend or mode does not unwrap. The name is not + bound: it is a local label, renamed by editing the record. The key ref is + not bound: the vendor key itself is the binding. +- **Decision:** keyset names resolve locally from the loaded records, with + no freshness window. Duplicate ids or names are refused at load. +- **Decision:** record creation is `KeysetRecord::create(&backend, key_ref, + mode, name)` (one vendor call; generates the keyset id and the key) and + `KeysetRecord::rewrap(&backend, new_key_ref)` (two calls; id, name and + mode stay). **Decision (open item of the ticket):** `stash` (TypeScript) + calls these through a Rust helper, not the vendor JS SDKs: a napi + function in a node binding (the way `@cipherstash/auth` ships), so the + request signing code exists once, in Rust. **Deferred to CIP-5312.** +- **Decision:** framing for every label, info and associated data is + `frame(pieces)` = piece count, then each piece's length and bytes, all + lengths `u64` little-endian. Pinned by tests. +- **Decision:** derived data keys: data key = `HKDF-SHA256(ikm = root, salt + = iv, info = frame["stack-kms-vendor/derived-data-key/v1", keyset id, + descriptor])`; index key = `HKDF-SHA256(ikm = root, salt = none, info = + frame["stack-kms-vendor/derived-index-key/v1", keyset id])`. The iv is + 16 random bytes per value; the tag is empty, and a retrieve refuses a + non-empty tag. The `hkdf` 0.13 crate (RFC 5869 test vector pinned) and + `sha2` 0.11 are new dependencies. Both derivations are pinned. +- **Decision:** wrapped data keys: each value's data key is a vendor + `generate_wrapped` under associated data `frame["stack-kms-vendor/ + wrapped-data-key/v1", keyset id, iv, descriptor]`; the wrapped key is + the leaf's tag. The iv is 16 random bytes. +- **Decision:** the default root key TTL is **5 minutes** + (`DEFAULT_ROOT_KEY_TTL`), set with `with_root_key_ttl`; `Duration::ZERO` + keeps none. The adapter, not the cipher, holds the root key, and drops + every expired root key on each lookup. A throttled record unwrap is + retried with the batch backoff. +- **Decision (closes the CIP-5304 gap):** a vendor keyset is never given to + the cipher as an index key. New `IndexKeySource::select_keyset` returns + `SelectedKeyset::Loaded(LoadedIndexKey)` (the default, ZeroKMS) or + `SelectedKeyset::PerSettle { keyset_id }` (the vendor adapter, both + modes, no vendor call). For a per-settle keyset the `KeysetState` holds + no PRF, so the handle holds no key. A term built through it pushes a new + internal request kind (`IndexKeyForSettle`); `dispatch` loads the index + key once per keyset per settle through the new provided method + `DataKeySource::load_settle_index_key`, keeps the PRF in the + `Responses`, and drops it (wiping the key) when the settle ends. Under + wrapped data keys that is one vendor unwrap per settle; under derived + data keys it is an HKDF from the cached root key. **Nothing outlives a + settle** in either mode except, under derived data keys, the root key in + the adapter for its TTL, which is the ADR-0009 trade-off. + **Alternative:** keep the CIP-5304 behaviour (the handle holds the key). + **Reason:** the orchestrator review made this binding. +- **Decision:** the term derivations run inside the settle through a + recording PRF backend (`sem/capture.rs`): a term's source may borrow, so + at build time the value is run against a backend that records every leaf + call (bytes, encoding, fully derived context) and the shape of every + sequence and map, owned; inside the settle the recording replays into the + real HMAC PRF through its own drivers. Tests show replay equals direct + derivation for every `vitaminc-prf` shape, and that a per-settle cipher + derives the same equality, match, ORE and OPE terms as a cipher that + holds the same key. The one assumption: a `PrfValue` passes its visitor + through unchanged (all `vitaminc-prf` impls do). Match terms copy the + text into a `Zeroizing`; ORE and OPE sources are already owned. +- **Decision (breaking, stack-encrypt):** `KeysetCipher::prf()` now returns + `Option<&HmacSha256Prf>`: `None` for a per-settle keyset. The commit is + marked `!`. **Alternative:** keep `&HmacSha256Prf`. **Reason:** a + per-settle handle has no PRF to return; returning a fake one would make + terms that match nothing. No caller in the tree used it outside `sem`. + An outside term type cannot defer yet (`Responses::index_prf` is + crate-private); revisit if a caller needs it. +- **Decision:** the default keyset question: `init` accepts a per-settle + default keyset (it holds no key, so there is nothing to outlive). For a + vendor-only cipher (`StackCipher::builder().kms(VendorKeySource)`), the + default is the only record when there is exactly one, else the one set + with `with_default_keyset(id)`; with neither, `init` fails `NotFound`. + Under `KeyRouter` the default keyset is ZeroKMS's default. + `Error::DefaultKeysetLifetime` stays for a source that loads a short-lived + key eagerly. +- **Decision:** new provided methods on `DataKeySource`: + `keyset_backend(keyset_id)` (default `key_backend()`), `key_mode(keyset_id)` + (default `None`) and `load_settle_index_key(keyset_id)` (default refuses + with `Error::Unexpected`). New pub field `RetrieveKeyPayload::key_mode` + and `with_key_mode`. Additive except that a struct-literal construction + of `RetrieveKeyPayload` breaks; stack-kms already takes a breaking bump in + this stack (CIP-5304), so the field rides on it. +- **Decision:** `dispatch` now asks `keyset_backend` per keyset: a leaf is + refused (`ForeignBackend`) when it names another backend than the one + that holds its keyset, and a settle seals v2 with the keyset's mode when + a key vendor backend holds the pending's keyset. The leaf's mode goes to + the key source on retrieve, and the adapter refuses a mode that is not + its keyset's (`Backend { kind: InvalidCiphertext }`). +- **Decision:** `Error::SealUnsupported` stays. It now means "a key vendor + backend holds the keyset but names no key mode for it", which is the one + case where the cipher cannot choose the leaf header. +- **Decision:** routing between ZeroKMS and one vendor is `KeyRouter` + in `stack-kms-vendor`. A keyset id or name that a vendor record has goes + to the vendor (a local record name wins over a ZeroKMS keyset of the same + name); everything else, and the default keyset, goes to ZeroKMS. The type + holds exactly one vendor; `VendorKeySource::new` refuses records of two + vendors with `RecordError::SecondVendor`, and a record of another vendor + than the backend with `RecordError::ForeignVendor`. +- **Decision:** the static key backend (`StaticKeyVendor`) wraps with + AES-256-GCM under a random 96-bit nonce: `version(1)=1 ‖ nonce(12) ‖ + ciphertext ‖ tag`. It holds several keys by reference, for rotation. + `supports_mode` is derived only: `KeysetRecord::create` refuses wrapped + (`ModeUnsupported`), and so does `VendorKeySource::new` for a record that + names it. It uses `aes-gcm` 0.10.3, already in the lockfile. +- **Decision:** batching is `run_batch(options, count, call, sleep)`: + `buffered` (order kept) with `BatchOptions { concurrency: 16, + max_attempts: 5, base_delay: 50 ms, max_delay: 2 s }`, exponential + backoff on `Throttled` only, and the first other failure (or a call still + throttled after the last attempt) fails the whole batch. It has no timer + of its own: the vendor's `sleep` is the timer, so it runs on native Rust + and in the WASI guest alike. +- **Decision:** the vendor transport is `VendorTransport { send(VendorRequest) + -> VendorResponse, sleep(Duration) }`, futures bounded by `MaybeSend`, the + shape of `ZeroKMSConnection`, plus `classify_status` (401/403 + AccessDenied, 404 NotFound, 429 Throttled, 5xx Unavailable, other 4xx + InvalidCiphertext). No vendor uses it yet. **Deferred to CIP-5307:** the + native (reqwest) implementation and the guest's `transport_send` + implementation. +- **Decision:** the contract suite is `stack_kms_vendor::contract` (feature + `test-support`): `contract::run(make_backend, key_ref)` runs every check + for each mode the vendor offers, through `VendorKeySource` and a real + `StackCipher`. `Observed` wraps any backend to count calls and inject + throttles, so lifetime, batching and backoff are tested on every vendor. + It runs here against `StaticKeyVendor` and `FakeVendor` (an in-memory + vendor, both modes, reported as AWS and Vault). Real-time TTL expiry is + tested once, here, with tokio. +- **Decision:** `mise run wasm:wasi-check` now also checks + `stack-kms-vendor` (no JS-host or native-HTTP dependency on WASI), and + `tests-crates.yml` and `udeps.yml` include its path. +- **Note:** the Go guest still compiles for wasm32-wasip1 with and without + `--features eql`. It does not use a key vendor backend yet (CIP-5311). +- **Note:** `scripts` vitest: `bench-index-expressions` and + `check-auth-npm-changeset` fail on this machine for missing + `node_modules` content, with or without this change. + +## CIP-5307: AWS KMS key vendor backend (stack-kms-aws) + +### Transports + +- **Decision:** the native `VendorTransport` is `HttpVendorTransport` in + `stack-kms-vendor`, behind a new feature `http`, with its dependencies + (reqwest, stack-auth with `http`, tokio `time`) gated to non-wasm32 + targets. It uses the workspace reqwest and `stack_auth::tls::client_config`, + the same HTTP client and TLS verifier as stack-kms's `HttpConnection`. + `sleep` is `tokio::time::sleep`. A transport failure text leaves out + reqwest's own message, which names the URL. + **Alternative:** put it in `stack-kms-aws`, or in `stack-kms`. + **Reason:** every vendor crate needs the same transport, so it belongs + beside the seam. The target gate keeps `stack-kms-vendor --features http` + clean on WASI (the `wasm:wasi-check` task now checks that). +- **Decision:** the guest `VendorTransport` is `HostVendorTransport` in + `stack-guest-abi`, module `vendor_transport`, behind a new feature + `vendor`. The type exists on wasm32 only; the conversions (headers to the + `transport_send` line format, a negative status to `TransportError`) are + pure functions with native unit tests. A header with a line break, or a + name with a colon, is refused (header injection). `sleep` is + `std::thread::sleep`, which wasm32-wasip1 implements with a `poll_oneoff` + clock subscription; the guest runs one call at a time, so blocking is + correct. **Alternative:** in `stack-kms-vendor` (a published crate cannot + depend on the unpublished `stack-guest-abi`), or in the Go guest crate (a + detached workspace with no native tests, and that wiring is CIP-5311). + **Reason:** `stack-guest-abi` already owns the import. +- **Decision (changes a CIP-5306 type):** `VendorRequest::body` and + `VendorResponse::body` are now `Zeroizing>`, and their `Debug` + shows the method, URL, status and header names only. Both types gain small + builders (`new`, `header`, `body`, `header_value`). **Reason:** an AWS + `Encrypt` request and a `Decrypt` response carry a key in the clear, and a + session token travels in a header. The crate is unpublished, so the break + costs nothing. reqwest takes the body by value, so its copy is not wiped + (as for ZeroKMS). + +### The AWS KMS client + +- **Decision:** `stack-kms-aws` is a new root workspace member with + `AwsKms`, which implements + `KeyVendorBackend` only: `wrap` is `Encrypt`, `unwrap` is `Decrypt` (with + `KeyId`, so AWS KMS also checks the key), `generate_wrapped` is one + `GenerateDataKey` (`AES_256`), and `sleep` is the transport's. Both key + modes. `key_ref` is a key ARN (a key id or alias ARN also works; the docs + recommend the key ARN, because an alias can be repointed). `send_json` is + public, so a test (and later `stash keyset`) can send another operation, + for example `CreateKey`. +- **Decision:** SigV4 is written in the crate (`stack_kms_aws::sigv4`) with + `hmac` 0.13 and `sha2` 0.11, both already in the lockfile (through `hkdf`). + No `aws-sigv4` and no AWS SDK crate. **Alternative:** the `aws-sigv4` + crate. **Reason:** it pulls the smithy runtime crates; the algorithm is + about 150 lines and is pinned by test vectors. Its build for + wasm32-wasip1 was not tried. `sign` signs every header the request has, + and adds `host`, `x-amz-date`, `x-amz-security-token` and + `authorization`. It does not add `x-amz-content-sha256` (AWS KMS does not + need it). +- **Decision:** the SigV4 vectors are, from the AWS SigV4 test suite + (`aws-sig-v4-test-suite`): `get-vanilla`, `post-vanilla`, + `get-vanilla-query-order-key-case` and `post-x-www-form-urlencoded` (a POST + with a body); and the worked IAM `ListUsers` example of the AWS SigV4 + documentation (canonical request hash and signature). All were written + from memory, and all passed on the first run, which checks the memory and + the code against each other. AWS publishes no KMS signing example that I + could reproduce reliably; the KMS request shape is pinned by the recorded + fixtures instead. +- **Decision:** the associated data is bound as the AWS KMS encryption + context: `cipherstash:aad` = unpadded base64url of all of `aad` (this is + the binding), plus, when `aad` is a keyset record key or a wrapped data + key, `cipherstash:purpose` (`root-key`, `index-key`, `data-key`), + `cipherstash:keyset-id` and `cipherstash:descriptor` (left out when + empty). New in `stack-kms-vendor::derive`: `unframe` and `describe_aad` + (`AadDescription`), so the shared crate owns the parse of its own format. + **Alternative:** the readable fields only (they leave out the iv, vendor + and mode, so they do not bind the whole aad), or `cipherstash:aad` only + (nothing readable in CloudTrail or for `kms:EncryptionContext:` policy + conditions). **Reason:** both properties. +- **Decision:** error mapping. The code comes from the body's `__type` + (namespace before `#` removed), else the `x-amzn-ErrorType` header (text + after `:` removed). The ticket's list, plus: `IncorrectKeyException` and + `InvalidKeyUsageException` to `InvalidCiphertext`; the signature and + token codes (`UnrecognizedClientException`, `InvalidSignatureException`, + `ExpiredTokenException`, ...) to `AccessDenied`; `DisabledException`, + `KMSInvalidStateException`, `KeyUnavailableException` to `Unavailable`. + A code with no fixed kind falls back to `classify_status`, but an unknown + 4xx code is `AccessDenied`, not `InvalidCiphertext` (for example a + `ValidationException` is a refused request, not a damaged value). + `LimitExceededException` is not `Throttled`: it is a resource quota, and a + retry does not help. A success that cannot be read is `Unavailable`; a + plaintext that is not 32 bytes is `InvalidCiphertext`. No transport, + credential or response failure carries key material or a secret. +- **Decision:** region from the config, else `AWS_REGION`, else + `AWS_DEFAULT_REGION`; endpoint from `with_endpoint`, else + `AWS_ENDPOINT_URL_KMS`, else `AWS_ENDPOINT_URL` (the AWS SDK names), else + `https://kms..amazonaws.com/` (`.com.cn` for `cn-` regions). A + region must be `[a-z0-9-]`, so it cannot change the endpoint host. +- **Decision:** `Clock` (`Arc SystemTime>`) is a seam on + `AwsKms` and `CachedCredentials`, for fixtures with a fixed signing time. + +### Credentials + +- **Decision:** the seam is the trait `CredentialsProvider` (future bounded + by `MaybeSend`); `Credentials` (secret and token wiped, never printed) is + the static provider, from `Credentials::new` or `Credentials::from_env`. + The guest implements the trait over a host import (CIP-5311). +- **Decision:** `CachedCredentials

` refreshes 5 minutes before expiry + (`DEFAULT_REFRESH_BEFORE_EXPIRY`), one fetch at a time (an async mutex), + and returns the cached credentials when a refresh fails but they have not + yet expired. Credentials with no expiry are kept. +- **Decision:** workload identity is native only (`cfg(not(target_arch = + "wasm32"))`): `WebIdentityCredentials` (STS `AssumeRoleWithWebIdentity`, + unsigned, form POST, the token file read on each fetch, the regional STS + endpoint, XML read by tag), `EcsCredentials` (relative or full URI; a full + URI must be HTTPS or HTTP to a loopback or the ECS/EKS link-local hosts, + as the AWS SDKs require; the token from + `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE`, read on each fetch, or + `AWS_CONTAINER_AUTHORIZATION_TOKEN`), and `ImdsCredentials` (IMDSv2 only; + `AWS_EC2_METADATA_DISABLED`, `AWS_EC2_METADATA_SERVICE_ENDPOINT`). They + send through `VendorTransport` too, so the fixtures test them. +- **Decision:** `DefaultCredentialsSource` picks one source from the + environment, in the AWS SDK order (static, IRSA, container, IMDS), by + which variables are set, not by trying each. `AwsKms::from_env` + (`DefaultAwsKms`, feature `http`, native) caches it, and gives the + credential sources their own transport with a 1 s connect limit so that + the IMDS probe fails fast off EC2. **Not supported:** the shared config + and credentials files (`~/.aws/config`), profiles, SSO and + `credential_process`. No ticket asks for them. + +### Tests + +- **Decision:** recorded fixtures (`tests/fixtures/*.json`) for + `GenerateDataKey`, `Encrypt`, `Decrypt` and `Decrypt` with a session + token, and eleven error responses. The responses are written in the AWS + response shapes; each request side (body, headers, signature) was taken + from the client once and is now pinned. Every fixture is signed at + 2026-11-05T12:00:00Z with the AWS documentation example key. +- **Decision:** the contract suite runs against `AwsKms` over an in-process + emulation of the AWS KMS JSON API (`tests/common::KmsEmulator`: AES-GCM + with the encryption context bound, the AWS error shapes, a check that each + request is signed for `kms`). **Reason:** the suite uses random keys and + ids, so recorded bytes cannot answer it. +- **Decision:** the LocalStack test (`tests/localstack.rs`) runs when + `STACK_KMS_AWS_LOCALSTACK_ENDPOINT` is set and skips itself otherwise. It + creates a key with `CreateKey`, runs the whole contract suite, and checks + three error mappings. It ran locally against `localstack/localstack:4.4` + and passed. LocalStack answers bytes that are not its blob format with + HTTP 500 (`InternalError`), not `InvalidCiphertextException`, so the test + damages a real blob instead. +- **Decision:** CI: a new `localstack` job in `tests-crates.yml` runs + LocalStack 4.4 as a service, pinned by digest, and calls the new mise task + `test:localstack:stack-kms-aws`, which refuses to run without the + endpoint (a skip must not read as a pass). New `doc:stack-kms-aws` and + `doc:stack-kms-vendor` tasks join `mise run doc`. `wasm:wasi-check` now + also checks `stack-kms-vendor --features http`, `stack-kms-aws` (default + features) and `stack-guest-abi --features vendor`. `tests-crates.yml` and + `udeps.yml` include the new path. `crates-ci.test.mjs` now expects seven + crate task files. + +### Publishing + +- **Decision:** `stack-kms-vendor` and `stack-kms-aws` are one release + group, `stack-kms-vendor`, which every later vendor crate joins. + **Alternative:** a group for each crate. **Reason:** a vendor crate + implements the seam, so a change to the seam moves every vendor. +- **Decision:** both Cargo.toml files are publishable (no `publish = + false`) and `tests-crates.yml`'s `cargo publish --dry-run` packages both + (it passed locally with all seven crates). In `release-plz.toml` both are + **staged**: `version_group = "stack-kms-vendor"`, `release = false`, + `publish = false`. `release-plz-root-config.test.mjs` and + `cargo-publish-opt-out.test.mjs` know the staged state. + **Alternative:** arm them now. **Reason:** crates.io Trusted Publishing can + only be set up on a crate that exists, so the first version is published + by hand, as it was for stack-kms. And that first publish needs the + stack-kms and stack-encrypt of this stack on crates.io first (the vendor + crates use their new API). Armed now, the first `release-crates` run after + this stack merges would fail on the vendor crates. +- **To arm them (a person, after this stack's stack-kms and stack-encrypt + are on crates.io):** `cargo publish -p stack-kms-vendor` then + `cargo publish -p stack-kms-aws`; set up Trusted Publishing for both on + crates.io (repository `cipherstash/stack`, workflow `release-plz.yml`); + then a PR that removes `release = false` and `publish = false` from both + in `release-plz.toml`, adds both to the preflight loop in + `release-plz.yml`, and moves them from `STAGED` to `GROUPS` in + `release-plz-root-config.test.mjs`. Update the publishing note in + `SECURITY.md` in the same PR. +- **Decision:** no changeset. Both crates are Rust-only and released by + release-plz. + +### Deferred + +- **Deferred to CIP-5311:** the guest wiring: enable `stack-guest-abi`'s + `vendor` feature in the guest, build `AwsKms::new(config, + HostVendorTransport, )`, and map the new errors to statuses. +- **Deferred to CIP-5312:** `stash keyset` creates records through a napi + helper; it can call `KeysetRecord::create` with `AwsKms::from_env()`. +- **Deferred (no ticket):** the AWS shared config files, profiles and SSO + (see above). +- **Note:** `scripts` vitest still has the two failures noted in CIP-5306 + (`bench-index-expressions`, `check-auth-npm-changeset`, missing + `node_modules` content); every other file passes. + +## Orchestrator review of CIP-5307 + +- **Revisit:** an unknown AWS 4xx error code maps to `AccessDenied`. That + makes a malformed request look like a permission problem. Accepted for now, + because `AccessDenied` is never retried, so the failure is loud. A later + change can add a `Rejected` kind if real cases appear. +- **Note:** the vendor crates are staged (`release = false`). Arming them is + a manual step after stack-kms and stack-encrypt from this stack are on + crates.io (see the CIP-5307 section). + +## CIP-5310: HashiCorp Vault Transit key vendor backend (stack-kms-vault) + +### Changes to the vendor seam (stack-kms-vendor) + +- **Decision:** vendor batch calls are three new provided methods on + `KeyVendorBackend`: `batch_limit() -> Option` (default + `None`), `wrap_batch(key_ref, &[WrapItem])` and `unwrap_batch(key_ref, + &[UnwrapItem])` (defaults: one `wrap` or `unwrap` per item). When + `batch_limit` is `Some(n)`, `VendorKeySource` sends the wrapped data keys + of a generate or retrieve in chunks of at most `n`, one batch call per + chunk, and generates each data key locally with the CSPRNG. The chunks run + through `run_batch`, so a throttled chunk backs off and is sent again + whole, and `concurrency` limits the chunks in flight. The keyset record key + is still one `unwrap`. **Alternative:** a Vault-only path, or a batch + `generate_wrapped`. **Reason:** additive (AWS and the static backend change + nothing); Transit has no batch data key operation, so local generation plus + a batch encrypt is the only one-call form; Google Cloud KMS and Azure have + no batch operation, so they keep the default. +- **Decision:** the contract suite forwards the batch methods in `Observed` + (each item counts as a wrap or unwrap; `Observer::batches()` counts the + calls) and has a new check, `uses_vendor_batches`: under wrapped data + keys, `2n + 1` values seal and open in exactly three batch calls. + `FakeVendor::with_batch_limit` runs the batch path in stack-kms-vendor's + own tests. +- **Decision (the "AAD bound by us" mechanism, for CIP-5309 too):** new + module `stack_kms_vendor::bound`. The vendor encrypts a **bound key** + `version(1)=1 ‖ SHA-256(frame["stack-kms-vendor/bound-key/v1", aad]) ‖ + key` (65 bytes) in place of the key; `open_bound_key` checks the digest in + constant time (`subtle`, already in the lockfile). The bytes are pinned. + **Alternative:** AEAD-wrap the vendor ciphertext under a key derived per + record, or carry a MAC. **Reason:** both need a second key that must come + from somewhere (the vendor, so one more call, or the record, which then + holds a key in the clear). The digest needs no key: it is sound when the + vendor's encryption is non-malleable, which is true of every vendor + encryption we use (AES-GCM, ChaCha20-Poly1305, RSA-OAEP). The module docs + say so. 65 bytes fits RSA-OAEP-256 with a 2048-bit key. **CIP-5309 (Azure + RSA wrap) should use it.** + +### The Vault client + +- **Decision:** `stack-kms-vault` is a new root workspace member with + `VaultTransit`, on the + `stack-kms-aws` template. Every encrypt, decrypt and rewrap uses + `batch_input`, a single key too, so there is one request shape. `wrap` and + `unwrap` are a batch of one. `generate_wrapped` is the trait default + (local key, one `wrap`): Transit's `datakey` endpoint has no batch form. + `batch_limit` is `VaultConfig::batch_size`, default 250 + (`DEFAULT_BATCH_SIZE`). Both key modes. `send_json(method, path, body)` is + public (tests use it to enable the engine and create keys). +- **Decision:** the associated data. The client **always** encrypts a bound + key. For a derived key (`derived=true`) it also sends the whole `aad` as + `context` (base64), so Vault's own derivation binds the keyset id, iv and + descriptor. Tested on Vault 1.20: a non-derived key **silently ignores** + `context`, and an unknown parameter only adds a warning, so neither can be + the binding on its own. `associated_data` (Vault 1.16 and later) is not + used: on an older Vault or another implementation it would be dropped with + only a warning. **Alternative:** context only for derived keys and the bound + key only for the others. **Reason:** one plaintext format for every key, + and a wrong derivation setting can then fail a call but never weaken the + binding (a test covers a non-derived key configured as derived). +- **Decision:** how the client knows a key is derived: `KeyDerivation` + in the config, `Detect` (the default) reads `GET /v1/{mount}/keys/{name}` + once per key and keeps the answer; `Derived` and `NotDerived` skip the + read. `Detect` also refuses a missing key (`NotFound`, because Transit's + `encrypt` would create it when the token has `create`) and a key type that + is not an AEAD (`aes256-gcm96`, `aes128-gcm96`, `chacha20-poly1305`) or + cannot encrypt and decrypt (`AccessDenied`). **Alternative:** a flag in the + keyset record. **Reason:** the record format is CIP-5306's and is shared by + every vendor; the key read costs one call per key per process. +- **Decision:** the stored wrapped key is `1 ‖ key version (u32 BE) ‖ the + decoded Vault ciphertext`, not the `vault:vN:base64` text: 98 bytes for a + bound key, about 40 fewer per value in the leaf tag. `compact_ciphertext` + and `expand_ciphertext` convert, and refuse anything that does not re-encode + to the same text. A wrapped key that does not expand is refused as + `InvalidCiphertext` before any call. +- **Decision:** `key_ref` is the Transit key name: one path segment of + `[A-Za-z0-9._-]`, not `.` or `..`. Anything else is `NotFound` before any + call (no path injection). The mount and the namespace are one or more such + segments; the address must be HTTP or HTTPS with no user, query or + fragment. +- **Decision:** error mapping (`error_kind(status, message)`): 401 and 403 + `AccessDenied`; 404 `NotFound` (a missing mount or key read); 429 + `Throttled`; 5xx `Unavailable` (503 is sealed or standby); any other 4xx + `AccessDenied`. A 400 maps by Vault's error text, lower case: + `NOT_FOUND_MARKERS` (`encryption key not found`, `key not found`) first, + then `INVALID_CIPHERTEXT_MARKERS` (`invalid ciphertext`, `message + authentication failed`, `unable to decrypt`, `could not decode`, + `ciphertext version`, `version is too new`); any other 400 is + `AccessDenied` (for example a derived key with no context), as for AWS. The + text is the first failed batch item's `error`, else `errors`. A failed item + in a 200 batch fails the whole batch. A bound key that does not open is + `InvalidCiphertext`; a success that cannot be read is `Unavailable`. +- **Decision:** a 403 calls the new `TokenProvider::invalidate` (default: + nothing), so a cached login token that Vault revoked is dropped and the + next request logs in again. The failed request is not retried. +- **Decision:** `rewrap(key_ref, wrapped, aad)` is a public method (Transit + `rewrap`, with `context` for a derived key), not a trait method. It moves a + wrapped key to the latest key version without the key leaving Vault. + `KeysetRecord::rewrap` (unwrap and wrap) still works for a move to another + key. + +### Auth + +- **Decision:** the seam is `TokenProvider` (future bounded by + `MaybeSend`, plus `invalidate`). `VaultToken` (wiped, never printed) is the + static provider (`VAULT_TOKEN`). `AppRoleLogin` (everywhere) and + `KubernetesLogin` (native only; it reads the service account token file on + each login) log in at `/v1/auth/{mount}/login` and return the token with + its lease. A refused login (HTTP 400 or 403) is `AccessDenied`. +- **Decision:** `CachedToken` logs in again before the lease ends: 5 minutes + before (`DEFAULT_REFRESH_BEFORE_EXPIRY`), or a third of the lease before + when the lease is shorter. One login at a time; a failed login returns the + cached token while its lease lasts. A token with no lease is kept until + `invalidate`. **Alternative:** `renew-self`. **Reason:** a renewal cannot + pass the token's max TTL, so a client that renews must also log in again; + one path is simpler and is tested. The docs note that each AppRole login + uses one use of a `secret_id` with a use limit. +- **Decision:** `DefaultAuth::from_env` (native) picks, in order: + `VAULT_TOKEN`; `VAULT_APPROLE_ROLE_ID` + `VAULT_APPROLE_SECRET_ID` (mount + `VAULT_APPROLE_MOUNT`, default `approle`); `VAULT_KUBERNETES_ROLE` (mount + `VAULT_KUBERNETES_MOUNT`, default `kubernetes`; token file + `VAULT_KUBERNETES_TOKEN_PATH`, default the service account path). Vault has + no standard names for the AppRole and Kubernetes variables; these are ours. + `VaultConfig::from_env` reads `VAULT_ADDR`, `VAULT_NAMESPACE` and + `VAULT_TRANSIT_MOUNT` (default `transit`). `VaultTransit::from_env` + (`DefaultVaultTransit`, feature `http`, native) builds both. +- **Not supported:** a custom CA (`VAULT_CACERT`) and mTLS client + certificates. `HttpVendorTransport` uses the platform verifier. No ticket + asks for them. + +### Tests and CI + +- **Decision:** the fixtures were recorded from `vault server -dev` 1.20.4 + by `tests/vault_dev.rs`'s `records_the_fixtures` (it runs only with + `STACK_KMS_VAULT_RECORD_FIXTURES=1`, against dev root token + `fixture-root-token`). Each file is an exchange sequence (the key read, + then the call): encrypt, decrypt, a batch of three each way, all for a + derived and a non-derived key; rewrap after a rotation; an AppRole login; + and eight real error responses. Written by hand, and named so: the + Kubernetes login (a dev server has no Kubernetes) and five errors a dev + server does not give (429, 503 sealed, 500, a policy 403, a retired key + version). +- **Decision:** the contract suite runs over an in-process Transit emulator + (`tests/common::TransitEmulator`: AES-GCM, derived keys that require + `context` and plain keys that ignore it, as Vault does; the batch and error + shapes; AppRole and Kubernetes logins; token and namespace checks), for a + derived and a non-derived key, with `Detect` and with the derivation + configured, at batch size 4 so the chunking runs. +- **Decision:** `tests/vault_dev.rs` runs when `STACK_KMS_VAULT_ADDR` is set + (token `STACK_KMS_VAULT_TOKEN`, default `root`): the whole contract suite + for a derived and a non-derived key, eight error mappings, rotation and + rewrap, and an AppRole login with a least-privilege policy. It ran locally + against `hashicorp/vault:1.20` in Docker and passed. +- **Decision:** CI: a new `vault` job in `tests-crates.yml` runs + `hashicorp/vault:1.20` (pinned by the multi-platform index digest) in dev + mode as a service and calls the new mise task + `test:vault:stack-kms-vault`, which refuses to run without the address. A + new `doc:stack-kms-vault` task joins `mise run doc`. `wasm:wasi-check` now + checks `stack-kms-vault` (default features). `tests-crates.yml` and + `udeps.yml` include the new path; `crates-ci.test.mjs` now expects eight + crate task files. +- **Decision:** `stack-kms-vault` joins the `stack-kms-vendor` release group, + staged (`release = false`, `publish = false` in `release-plz.toml`), and + the `cargo publish --dry-run` list. Arming it follows the CIP-5307 steps, + with `cargo publish -p stack-kms-vault` after `stack-kms-vendor`. No + changeset: Rust only. +- **Note:** a local `cargo publish --dry-run` can fail with "no `bound` in + `stack_kms_vendor`" after an earlier dry run packaged an older + `stack-kms-vendor` 0.1.0: cargo treats the packaged crate as an immutable + registry crate and reuses its old build. Remove + `~/.cargo/registry/src/-*/stack-kms-vendor-0.1.0`, `target/package` and the + `stack-kms-vendor` files in `target/debug/{.fingerprint,deps}`. CI starts + clean. +- **Note:** `scripts` vitest still has the two failures noted in CIP-5306; + every other file passes. + +### Deferred + +- **Deferred to CIP-5309:** Azure Key Vault RSA wrap binds its associated + data with `stack_kms_vendor::bound`. +- **Deferred to CIP-5311:** the guest wiring: `VaultTransit::new(config, + HostVendorTransport, )`, and the + status mapping of the new errors. +- **Deferred to CIP-5312:** `stash keyset` can create Vault records with + `VaultTransit::from_env()`. +- **Deferred (no ticket):** `VAULT_CACERT` and mTLS; `renew-self`. + +## Orchestrator review of CIP-5310 + +- **Revisit:** Vault with a private CA (`VAULT_CACERT`) and mTLS client + certificates is not supported. Many self-hosted Vault deployments use a + private CA, so this is likely the first customer request. It is a follow-up, + not part of CIP-5310. + +## CIP-5308: Google Cloud KMS key vendor backend (stack-kms-gcp) + +### The Cloud KMS client + +- **Decision:** `stack-kms-gcp` is a new root workspace member with + `GcpKms`, on the `stack-kms-vault` + template. `wrap` is `cryptoKeys.encrypt`, `unwrap` is `cryptoKeys.decrypt` + (REST v1, JSON, standard base64). `generate_wrapped`, `batch_limit`, + `wrap_batch` and `unwrap_batch` are the trait defaults: Cloud KMS has no + data key generation and no batch operation, so each wrapped data key is a + key from the CSPRNG and one `encrypt`, and the adapter runs the calls + through `run_batch` (concurrency 16). Both key modes. `send_json(method, + path, body)` is public, as for the other vendors. +- **Decision:** `key_ref` is the crypto key name + (`projects/{p}/locations/{l}/keyRings/{r}/cryptoKeys/{k}`). A + `…/cryptoKeyVersions/{n}` name is accepted too and pins `encrypt` to that + version; `decrypt` always names the crypto key (Cloud KMS requires that). + `parse_key_name` checks the shape: each id is `[A-Za-z0-9._-]`, not `.` or + `..`; only the project id may hold `:` (a domain-scoped project). Anything + else is `NotFound` before any call, so a key reference cannot add a method + (`:decrypt`) or change the path. **Alternative:** a version path as the + only form (the trait docs said so). **Reason:** the crypto key name keeps + rotation transparent; the trait docs now say "crypto key name". +- **Decision:** the stored wrapped key is `1 ‖ key version (u32 BE) ‖ Cloud + KMS ciphertext`, the same layout as Vault's. The version comes from the + `encrypt` response `name`, which must be a version of the requested key, + else the response is refused (`Unavailable`). `wrapped_key_version` reads + it. **Alternative:** store the ciphertext alone (Cloud KMS finds the + version from the ciphertext). **Reason:** the ticket asks to record the + version; 5 bytes per value lets an operator find which versions stored + keys still need before a version is disabled or destroyed, which makes + its ciphertexts unrecoverable. +- **Decision:** the associated data is sent whole as + `additionalAuthenticatedData`: it holds the keyset id and, for a wrapped + data key, the iv and the descriptor, as the AWS crate binds all of `aad`. + No `stack_kms_vendor::bound`: Cloud KMS AAD is native and authenticated. + There are no readable fields, because Cloud KMS does not show the AAD in + audit logs or IAM conditions. +- **Decision:** CRC32C is used on every call (in-crate, table-driven, + `stack_kms_gcp::crc32c`, RFC 3720 vectors pinned). Encrypt sends + `plaintextCrc32c` and `additionalAuthenticatedDataCrc32c` and requires + `verifiedPlaintextCrc32c`, `verifiedAdditionalAuthenticatedDataCrc32c` + and a matching `ciphertextCrc32c`. Decrypt sends `ciphertextCrc32c` and + `additionalAuthenticatedDataCrc32c` and requires a matching + `plaintextCrc32c`. A missing or wrong check is `Unavailable`. The int64 + fields are sent as strings and read as a string or a number. + **Reason:** about 40 lines, no new dependency, and it is Google's + recommended practice. No CRC32C crate was in the lockfile (`crc32fast` is + CRC-32/IEEE). +- **Decision:** error mapping (`error_kind(http status, google.rpc code, + message)`), by `error.status` first: `PERMISSION_DENIED`, `UNAUTHENTICATED` + to `AccessDenied`; `NOT_FOUND` to `NotFound`; `RESOURCE_EXHAUSTED` to + `Throttled`; `UNAVAILABLE`, `DEADLINE_EXCEEDED`, `INTERNAL`, `UNKNOWN`, + `ABORTED`, `CANCELLED` and `FAILED_PRECONDITION` (a disabled or destroyed + version, as AWS `DisabledException`) to `Unavailable`. `INVALID_ARGUMENT` + maps by its message, lower case: `CHECKSUM_MARKERS` (`checksum`, + `crc32c`) first to `Unavailable` (damaged in transit), then + `INVALID_CIPHERTEXT_MARKERS` (`decryption failed`, `ciphertext is + invalid`) to `InvalidCiphertext`, else `AccessDenied`. No code: the HTTP + status, with any other 4xx `AccessDenied` (the CIP-5307 rule). The + markers apply to any operation, not only `decrypt`: an `encrypt` never + says "Decryption failed", so the extra check bought nothing. A 401 calls + `TokenProvider::invalidate`. A plaintext that is not 32 bytes is + `InvalidCiphertext`. +- **Note:** Cloud KMS answers `PERMISSION_DENIED`, not `NOT_FOUND`, for a key + the caller may not see. So a missing key is often `AccessDenied`. The live + test accepts either. +- **Decision:** the endpoint is `GcpKmsConfig::with_endpoint` (default + `https://cloudkms.googleapis.com`; any HTTP or HTTPS origin with no path), + for regional endpoints and tests. No environment variable: the Google SDKs + have no common name for it. No `x-goog-user-project` (quota project) + header: a service account key does not need one. Add it if a customer + asks. + +### Credentials + +- **Decision:** the seam is `TokenProvider` (future bounded by `MaybeSend`, + plus `invalidate`), with `AccessToken` (wiped, never printed) as the + static provider, `CachedToken` (5 minutes before expiry, or a third of the + life when shorter; one fetch at a time; a failed fetch keeps an unexpired + token), all on the Vault template. +- **Decision:** `ServiceAccountKey::from_json(bytes)` (everywhere) and + `from_file(path)` (native) read a key file. It refuses any `type` but + `service_account` (so `authorized_user` and `external_account` files fail + with a clear message), a key that is not RSA PKCS #8 PEM, and a + `token_uri` that is not HTTPS (default `https://oauth2.googleapis.com/token`). + `ServiceAccountLogin` signs `{"alg":"RS256","kid":…,"typ":"JWT"}` and + `{"iss","scope","aud":token_uri,"iat","exp":iat+3600}` and posts the RFC + 7523 form to `token_uri`. The scope is `https://www.googleapis.com/auth/cloudkms` + (`with_scope` changes it). **Alternative:** a self-signed JWT sent + directly as the bearer token (no exchange). **Reason:** the ticket names + the exchange, and it works for every key the token endpoint accepts. +- **Decision (RSA crate):** RS256 uses the `rsa` crate 0.9.10 (RustCrypto, + pure Rust; already in `packages/eql/Cargo.lock`), with `rand_core` 0.6 + `OsRng` (getrandom 0.2, already in the root lockfile) as the blinding RNG. + It builds for wasm32-wasip1 (checked by `wasm:wasi-check`). **Alternatives:** + `ring` (in the root lockfile through jsonwebtoken, but it needs a C + toolchain with a wasm32 target, and stack-auth keeps it off wasm32 for + that reason); `aws-lc-rs` (does not build for wasm32); `jsonwebtoken` + (pulls ring or aws-lc-rs). **Known issue:** RUSTSEC-2023-0071 (the Marvin + timing attack: the private key operation is not constant time) has no + fixed `rsa` release. The risk here is low: the key signs one JWT per hour + on the client, the signature uses blinding (`try_sign_with_rng`), and an + attacker must time those signatures over the network. OSV-Scanner + (`fail-on-vuln: false`) will report it as an alert, as it already does + for the EQL lockfile. Revisit when `rsa` 0.10 is stable. +- **Decision:** `MetadataServerLogin` (native only, `cfg(not(target_arch = + "wasm32"))`) gets `http://{host}/computeMetadata/v1/instance/service-accounts/default/token` + with `Metadata-Flavor: Google`. The host is `metadata.google.internal`, or + `GCE_METADATA_HOST` (the Google SDK name), checked as `host[:port]` so it + cannot change the path. A 404 from it (no service account) is + `AccessDenied`. +- **Decision:** `DefaultCredentials::from_env` (native) picks, in order: + `GOOGLE_OAUTH_ACCESS_TOKEN` (a static token, the name Terraform's Google + provider uses; handy with `gcloud auth print-access-token`); + `GOOGLE_APPLICATION_CREDENTIALS` (a service account key file); else the + metadata server. `GcpKms::from_env` (`DefaultGcpKms`, feature `http`, + native) builds it, with a 1 s connect limit on the credentials transport + so the metadata probe fails fast off Google Cloud. **Not supported:** + `authorized_user` files (`gcloud auth application-default login`), + `external_account` files (workload identity federation) and service + account impersonation. No ticket asks for them. + +### Tests and CI + +- **Decision:** no fixture was recorded from Google Cloud (no emulator and + no credentials here). The responses are written by hand in the documented + Cloud KMS and OAuth shapes. The request sides were computed by a separate + Python script, not by the client (base64, CRC32C, the AAD frame, the form + body), and the client produced the same bytes on the first run. Fixtures: + encrypt, decrypt, an encrypt pinned to a version, a decrypt under an old + version, the service account token exchange, the metadata server token, + and 13 error responses. +- **Decision:** the JWT known-answer test signs with a TEST-ONLY 2048-bit RSA + key generated for these tests (`openssl genpkey`). The expected JWT + (`tests/fixtures/test_only_jwt_assertion.txt`) was signed by `openssl dgst + -sha256 -sign`, a second implementation, and the crate's output matches it + byte for byte. The key is stored as base64 PKCS #8 DER + (`test_only_rsa_key.pkcs8.b64`), and the service account JSON is built at + run time, so no file holds a PEM block or a key file that push protection + could take for a leaked Google key. The fixture tokens are + `fixture-*-token`, not `ya29.`. +- **Decision:** the contract suite runs over an in-process emulator + (`tests/common::KmsEmulator`: AES-256-GCM with the AAD bound, versions and + rotation, the CRC32C checks, Google's error shapes, a token endpoint that + verifies the JWT signature with the test key, and the metadata server), + with a static token and with service account credentials. +- **Decision:** `tests/live.rs` runs when `STACK_KMS_GCP_LIVE_KEY` is set + (credentials from `DefaultCredentials`): the whole contract suite, and + wrong AAD, a damaged ciphertext and a missing key. **It has not run**: + there are no Google Cloud credentials here. The new mise task + `test:live:stack-kms-gcp` refuses to run without the key and credentials. +- **Decision:** CI: a new workflow, `integration-gcp-kms.yml` (push to main + and pull requests on the crate paths, plus `workflow_dispatch`), with one + job, `live`. Its condition is `vars.STACK_KMS_GCP_LIVE_KEY != '' &&` the + canonical fork-PR clause, so it is **inert until the variable is set** and + then fails loudly if the secret is empty (a shell pre-flight first, as + `require-cs-secrets` does; that action and `ffi-binding-step-order.test.mjs` + are for `CS_*` credentials and do not apply). **Alternative:** a job in + `tests-crates.yml`, or a fork-guarded job with no variable. **Reason:** + without the variable, every PR on the crate would go red until someone + sets up Google Cloud. `workflow-dispatch-job-conditions.test.mjs` now + lists the workflow and the job and holds the variable open; + `crates-ci.test.mjs` names it as the sole caller of the live task. +- **To turn the live job on (a person):** create a symmetric + `ENCRYPT_DECRYPT` crypto key for the test only; a service account with + `roles/cloudkms.cryptoKeyEncrypterDecrypter` on that key only, and a JSON + key for it. Then set: + - repository **variable** `STACK_KMS_GCP_LIVE_KEY` = the crypto key name; + - repository **secret** `STACK_KMS_GCP_SERVICE_ACCOUNT_KEY` = the JSON key. +- **Decision:** `tests-crates.yml` and `udeps.yml` include the new path, + `cargo publish --dry-run` packages the crate (passed locally), + `wasm:wasi-check` checks it (passed), and `doc:stack-kms-gcp` joins `mise + run doc`. `crates-ci.test.mjs` now expects nine crate task files. +- **Decision:** `stack-kms-gcp` joins the `stack-kms-vendor` release group, + staged (`release = false`, `publish = false`). Arming it follows the + CIP-5307 steps, with `cargo publish -p stack-kms-gcp` after + `stack-kms-vendor`. No changeset: Rust only. +- **Note:** `scripts` vitest still has the two failures noted in CIP-5306 + (`bench-index-expressions`, `check-auth-npm-changeset`); every other file + passes. + +### Deferred + +- **Deferred to CIP-5311:** the guest wiring: `GcpKms::new(GcpKmsConfig::new(), + HostVendorTransport, )`, and the + status mapping. `ServiceAccountLogin` also works in the guest if the host + passes key bytes, but ADR-0011 says the host supplies the token. +- **Deferred to CIP-5312:** `stash keyset` can create Cloud KMS records with + `GcpKms::from_env()`. +- **Deferred (no ticket):** `authorized_user` and `external_account` + credentials, impersonation, a quota project header, and re-recording the + fixtures from real Cloud KMS once the live job has credentials. + +## Orchestrator review of CIP-5308 + +- **Revisit (security):** `stack-kms-gcp` signs its OAuth JWT with the `rsa` + crate, which has the open advisory RUSTSEC-2023-0071 (Marvin, a timing side + channel). The key it uses is the customer's service account key. Accepted + for now with the agent's reasoning, but a security reviewer must sign it off + before the crate is armed for release. One option: in the Go guest, ask the + host for an access token, not for the service account key, so that the guest + never signs (CIP-5311). +- **Not verified:** the GCP fixtures are hand-written from the documented + shapes, not recorded from Cloud KMS. The opt-in live job is the only real + check. It needs the variable `STACK_KMS_GCP_LIVE_KEY` and the secret + `STACK_KMS_GCP_SERVICE_ACCOUNT_KEY`. + +## CIP-5309: Azure Key Vault key vendor backend (stack-kms-azure) + +### The Key Vault client + +- **Decision:** `stack-kms-azure` is a new root workspace member with + `AzureKeyVault`, on the + `stack-kms-gcp` template. `wrap` is `POST {key}/{version}/wrapkey`, + `unwrap` is `POST {key}/{version}/unwrapkey` (JSON, base64url). + `generate_wrapped`, `batch_limit`, `wrap_batch` and `unwrap_batch` are the + trait defaults: Key Vault has no data key generation and no batch + operation. Both key modes. `send_json(method, key_ref, operation, body)` is + public, as for the other vendors. Key Vault and Managed HSM are one client: + the host of the key identifier selects the token scope. +- **Decision:** the API version is **7.4** (`API_VERSION`). **Alternative:** + 7.5 or 7.6. **Reason:** 7.4 is GA on Key Vault and Managed HSM and has every + operation the client uses; nothing newer is needed. +- **Decision:** `key_ref` is the key identifier, + `https://{vault}.vault.azure.net/keys/{name}` or + `https://{hsm}.managedhsm.azure.net/keys/{name}`, optionally `/{version}`. + `KeyId::parse` accepts only the six known Azure DNS suffixes (public, China + and US Government clouds, Key Vault and Managed HSM), HTTPS, no port, query, + fragment or credentials, names `[A-Za-z0-9-]` and versions `[A-Za-z0-9]{1,64}`. + Anything else is `NotFound` before any call. **Alternative:** any HTTPS host, + or a configurable endpoint. **Reason:** the client sends a bearer token to + that host, so an edited keyset record must not be able to send it elsewhere. + Private endpoints keep the public DNS names, so the list covers them. No + test needs another host: the emulator answers any URL. +- **Decision:** the algorithm follows the key type: `RSA` and `RSA-HSM` wrap + with **RSA-OAEP-256**, `oct` and `oct-HSM` with **A256KW**; `EC` and other + types are refused (`AccessDenied`). The client learns the type and, for an + unversioned key identifier, the current version from one + `GET /keys/{name}[/{version}]`, kept per key reference for the life of the + client. `AzureKeyVaultConfig::with_algorithm` fixes the algorithm; with a + versioned key identifier the client then reads no key. A key whose type + does not match the configured algorithm is refused. **Alternative:** read + the key on every wrap, or a TTL on the cached version. **Reason:** the + ticket says "resolve once"; one read per key per process is cheap. To keep + a disabled version from blocking wraps, a failed wrap drops the cached + version, so the next wrap reads the key again (tested). +- **Decision (the stored wrapped key):** `1 ‖ algorithm(1: 1 = RSA-OAEP-256, + 2 = A256KW) ‖ version length(1) ‖ version (ASCII) ‖ Azure ciphertext` + (`compact_ciphertext`, `split_wrapped_key`, `wrapped_key_version`). It is + the Vault/GCP layout with the version as a string id (Azure versions are 32 + hexadecimal characters) and with the algorithm added. An unwrap therefore + needs no key read and no configuration: the record key and every wrapped + data key name the version and algorithm that made them. The keyset record's + `key_ref` can stay unversioned (rotation needs no record change); a + versioned one pins wraps only. Size: 115 bytes for A256KW, 291 bytes for + RSA-OAEP-256 with a 2048-bit key. **Alternative:** store the full versioned + `kid` (60 to 100 bytes more per value, and it repeats the key reference). +- **Decision (no AAD):** the client **always** wraps the bound key of + `stack_kms_vendor::bound` (CIP-5310), padded with 7 zero bytes to **72 + bytes** (`WRAP_PLAINTEXT_LEN`), for both algorithms. An unwrap refuses a + plaintext that is not 72 bytes, has non-zero padding, or whose digest does + not match (`InvalidCiphertext`). **Reason:** A256KW (RFC 3394) needs a + multiple of 8 bytes and at least 16; the bound key is 65. RSA-OAEP-256 with + a 2048-bit key takes up to 190 bytes, so 72 fits too, and one plaintext + format serves both. **Alternative:** pad only for A256KW. **Reason:** one + format is simpler and has one test. Both algorithms are non-malleable + (RSA-OAEP; AES key wrap has its own integrity check), which the bound key + needs. **The Azure trade-off** (documented on `AzureKeyVault`): Azure does + not see or log the binding, so a wrapped key moved to another keyset, value + or descriptor unwraps at Azure and is then refused by the client. The + leaf AAD binds the descriptor as for every vendor. +- **Decision:** error mapping (`error_kind(status, codes, unwrap)`), where + `codes` are `error.code` and `error.innererror.code`: `KeyDisabled`, + `KeyExpired`, `KeyNotYetValid` to `Unavailable` (as AWS + `DisabledException`); 429 or `Throttled` to `Throttled`; 401, 403, + `Unauthorized`, `Forbidden` to `AccessDenied`; 404, `KeyNotFound`, + `NotFound` to `NotFound`; 5xx to `Unavailable`; 400 `BadParameter` **on an + unwrap** to `InvalidCiphertext` (Azure could not decrypt: damaged bytes, or + another key or version); any other 4xx to `AccessDenied` (the CIP-5307 + rule). A 401 calls `TokenProvider::invalidate`. A wrap response whose `kid` + is not exactly the requested key version, or with no base64url value, is + `Unavailable`. **Known limit:** a `BadParameter` on an unwrap for another + reason (for example a refused API version) also reads as + `InvalidCiphertext`; Azure gives no finer code. +- **Decision (Retry-After):** on a 429 the client sleeps the `Retry-After` + seconds (through the transport, at most `MAX_RETRY_AFTER` = 30 s; an HTTP + date is ignored), then returns `Throttled`, and the adapter's backoff + retries. **Alternative:** retry inside the client. **Reason:** the adapter + owns the retry policy; the client only honours the server's wait. A 503 + with `Retry-After` does not wait: `Unavailable` is not retried. +- **Decision:** no `rsa` dependency in the crate. Azure does every RSA + operation. The `rsa` crate is a **dev-dependency only**, for the fixture + emulator's RSA-OAEP-256, so RUSTSEC-2023-0071 (see the orchestrator review + of CIP-5308) does not reach the shipped crate. `aes` 0.8.4 (already in the + lockfile) is a dev-dependency for the emulator's RFC 3394 key wrap. + +### Credentials + +- **Decision:** the seam is `TokenProvider::token(scope)`: the scope is an + argument, because one client can reach Key Vault + (`https://vault.azure.net/.default`) and Managed HSM + (`https://managedhsm.azure.net/.default`), and national clouds have their + own. `CachedToken` keeps one token per scope (same refresh rules as GCP and + Vault); `invalidate` drops all. `AccessToken` (wiped, never printed) is the + static provider and is used for every scope; it is the guest seam + (ADR-0011). +- **Decision:** `ClientSecretLogin` (everywhere) posts the client credentials + grant to `{authority}/{tenant}/oauth2/v2.0/token`. The tenant must be + `[A-Za-z0-9.-]` (it is in the path). `AZURE_AUTHORITY_HOST` (the Azure SDK + name) changes the host and must be HTTPS with no path. +- **Decision:** `WorkloadIdentityLogin` (native only) posts the federated + token from `AZURE_FEDERATED_TOKEN_FILE` as a `client_assertion`, read on + each login. It was cheap: the same request as the client secret. +- **Decision:** `ManagedIdentityLogin` (native only): IMDS + (`GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=…`, + `Metadata: true`) or App Service (`IDENTITY_ENDPOINT` + + `X-IDENTITY-HEADER`, api-version 2019-08-01). `AZURE_CLIENT_ID` names a + user-assigned identity (`client_id`). `IDENTITY_ENDPOINT` must be HTTP to a + loopback host or HTTPS, with no query, so the identity header cannot go + elsewhere. A token's life is `expires_in` (number or string), else + `expires_on` less now. IMDS 410 (updating) is `Unavailable`; a refused + identity (400, 404) is `AccessDenied`. +- **Decision:** `DefaultCredentials::from_env` (native) picks, in the + `DefaultAzureCredential` order: `AZURE_KEY_VAULT_ACCESS_TOKEN` (a static + token; the name is ours, Azure has none); `AZURE_TENANT_ID` + + `AZURE_CLIENT_ID` + `AZURE_CLIENT_SECRET`; `AZURE_TENANT_ID` + + `AZURE_CLIENT_ID` + `AZURE_FEDERATED_TOKEN_FILE`; else managed identity + (App Service when `IDENTITY_ENDPOINT` and `IDENTITY_HEADER` are set, else + IMDS). `AzureKeyVault::from_env` (`DefaultAzureKeyVault`, feature `http`, + native) builds it with a 1 s connect limit on the credentials transport. +- **Decision (deferred, security):** **client certificate credentials are not + supported**. They sign a JWT with the service principal's RSA key, which + would put the `rsa` crate (RUSTSEC-2023-0071) back into a shipped crate. + `AZURE_CLIENT_CERTIFICATE_PATH` without a secret is refused with a clear + message. Also not supported: Azure CLI credentials, Azure Arc and Service + Fabric managed identity. + +### Tests and CI + +- **Decision:** no fixture was recorded from Azure (no emulator and no + credentials here). Responses are written by hand in the documented Key + Vault, Entra ID and managed identity shapes. The request sides were + computed by a separate Python script, not by the client (base64url, the AAD + frame, the bound key digest and padding, the form bodies); the client sent + the same bytes on the first run. RSA-OAEP-256 ciphertexts were made by + `openssl pkeyutl` under a **TEST-ONLY 2048-bit RSA key** generated for these + tests (`tests/fixtures/test_only_rsa_key.pkcs8.b64`, base64 PKCS #8 DER, no + PEM block); A256KW ciphertexts by a Python RFC 3394 key wrap over `openssl + enc -aes-256-ecb`, under the RFC 3394 example key (`TEST_ONLY_KEK`), checked + against RFC 3394 §4.6. A test opens each fixture ciphertext with the + test-only keys. Fixtures: wrap (with the key read) and unwrap for a Key + Vault RSA key and a Managed HSM `oct-HSM` key, a pinned older version, + client secret, workload identity, IMDS and App Service tokens, and 14 error + responses. Tokens and secrets are `fixture-*`; no real-shaped secret. +- **Decision:** the contract suite runs over an in-process emulator + (`tests/common::KeyVaultEmulator`: RSA-OAEP-256 with the test-only key (every + RSA version uses it), A256KW with an AES key per version, rotation, + disabled versions, throttles with `Retry-After`, Azure's error shapes, and a + token endpoint whose tokens carry a scope that each key request checks), + for an RSA key and an `oct-HSM` key, with a static token and with client + secret credentials. +- **Decision:** `tests/live.rs` runs when `STACK_KMS_AZURE_LIVE_KEY` (a Key + Vault RSA key) is set, and also tests `STACK_KMS_AZURE_LIVE_HSM_KEY` when + set: the whole contract suite, wrong AAD, a damaged ciphertext and a + missing key. **It has not run**: no Azure credentials here. The mise task + `test:live:stack-kms-azure` refuses to run without the key and credentials. +- **Decision:** CI: `integration-azure-key-vault.yml`, one job `live`, on + the GCP template: inert until the variable is set, then a pre-flight fails + the job if a secret is empty; the secrets reach only the test step. +- **To turn the live job on (a person):** create an RSA key (2048 bits or + more) in a Key Vault for the test only (optionally an `oct-HSM` AES-256 key + in a Managed HSM); an app registration with a client secret and the role + "Key Vault Crypto Service Encryption User" on that key only ("Managed HSM + Crypto Service Encryption User" on the HSM key). Then set: + - repository **variable** `STACK_KMS_AZURE_LIVE_KEY` = the key identifier; + - optional repository **variable** `STACK_KMS_AZURE_LIVE_HSM_KEY`; + - repository **secrets** `STACK_KMS_AZURE_TENANT_ID`, + `STACK_KMS_AZURE_CLIENT_ID`, `STACK_KMS_AZURE_CLIENT_SECRET`. +- **Decision:** `tests-crates.yml` and `udeps.yml` include the new path, + `cargo publish --dry-run` packages the crate (passed locally with the CI + list), `wasm:wasi-check` checks it (passed), `doc:stack-kms-azure` joins + `mise run doc`, and `crates-ci.test.mjs` expects ten crate task files. + `workflow-dispatch-job-conditions.test.mjs` lists the workflow and job. +- **Decision:** `stack-kms-azure` joins the `stack-kms-vendor` release group, + staged (`release = false`, `publish = false`). Arming it follows the + CIP-5307 steps, with `cargo publish -p stack-kms-azure` after + `stack-kms-vendor`. No changeset: Rust only. +- **Note:** `scripts` vitest still has the two failures noted in CIP-5306 + (`bench-index-expressions`, `check-auth-npm-changeset`); every other file + passes. + +### Deferred + +- **Deferred to CIP-5311:** the guest wiring: `AzureKeyVault::new(config, + HostVendorTransport, )`. The host + import must take the scope (Key Vault and Managed HSM need different + tokens). Status mapping of the new errors. +- **Deferred to CIP-5312:** `stash keyset` can create Azure records with + `AzureKeyVault::from_env()`. +- **Deferred (no ticket):** client certificate credentials (blocked on a + constant-time RSA signer), Azure CLI and Arc credentials, re-recording the + fixtures from real Azure once the live job has credentials. + +## CIP-5311: Go SDK: key vendor backends through the encrypt guest + +### Guest size (measured) + +- **Measurement** (release builds, `mise run wasm:guest:build` and + `wasm:guest:build:eql`, on this branch before and after): + + | build | before | after | growth | + |---|---|---|---| + | `stack_encrypt_guest.wasm` (no EQL types) | 1,543,167 B | 1,915,570 B | +372,403 B (+24.1%) | + | `stack_encrypt_guest_eql.wasm` | 1,598,896 B | 1,973,568 B | +374,672 B (+23.4%) | + + (The "after" column includes the vendor-only client below. Before it, + the two builds were 1,906,377 B and 1,965,401 B.) + | deterministic test build | n/a | 1,657,639 B | | + | deterministic test build, EQL | n/a | 1,726,501 B | | + + **Decision:** ship one guest with all five backends, as ADR-0011 says. + The growth is below the 30% limit the orchestrator set. **Alternative:** + one guest per vendor. **Reason:** it would multiply the embedded files, + the test builds and the release workflow. + +### The credential import + +- **Decision (binding, from the orchestrator):** the host supplies + ready-to-use short-lived credentials, never a long-lived signing key: AWS + temporary credentials, a Google OAuth access token, an Azure access token + for the scope the guest asks for, a Vault client token. The guest does no + RSA and reads no service account key. +- **Decision:** one import, `cipherstash_transport::vendor_credential_get( + vendor: u32, scope_ptr, scope_len, cred_ptr_out, cred_len_out) -> i32`, + the shape of `token_get`. `vendor` is the `KeyVendor` byte; `scope` is the + OAuth scope for Google Cloud and Azure and empty for AWS and Vault. The + credential is one value in the transport codec: an object of strings + (`access_key_id`, `secret_access_key`, optional `session_token` for AWS; + `token` for the others) and an optional `expires_at` (`i64`, Unix + seconds). **Alternative:** one import per credential kind; JSON. + **Reason:** one import keeps the surface small; the codec is the format + every other input uses, and its strings are wiped on drop. +- **Decision:** the guest asks before **each** vendor request and keeps no + credential between requests. `TokenProvider::invalidate` therefore has + nothing to drop in the guest, and the import has no "refresh" flag. The + host's credential chain (the AWS SDK cache, `oauth2.ReuseTokenSource`, + azidentity) decides when to refresh. **Alternative:** cache in the guest + with `CachedToken`/`CachedCredentials` and forward `invalidate` to the + host. **Reason:** credentials live in guest memory only for one request, + and the Go chains already cache. +- **Decision:** the guest refuses a credential whose `expires_at` has passed, + a credential with an unknown key, an empty or missing field, or a field + with a control character (`AccessDenied`, so `ErrForbidden`). A non-zero + import status is `Unavailable` (`ErrTransport`), the same status as a + failed `token_get`. The Go host attaches the source's own error to the + call's error, as it does for the token source, so `errors.Is` matches it. + **No new status codes:** the vendor failure kinds already map to the + ZeroKMS statuses (CIP-5304), and the `Diagnostic` code + `stack_kms::backend` with its `vendor` field tells them apart. +- **Decision:** `stack-kms-gcp` gets a default feature `service-account` + (the RS256 JWT login and the `rsa` and `rand_core` dependencies); `http` + implies it. The guest builds the crate with no default features, so the + `rsa` crate (RUSTSEC-2023-0071) is not in the guest's `Cargo.lock` at all. + `CLOUD_KMS_SCOPE` moved to `auth` so it stays without the feature. + `wasm:wasi-check` checks the crate without default features too. + **Alternative:** rely on LTO to drop the unused RSA code. **Reason:** the + lockfile, and so OSV, would still carry the advisory. + +### The guest + +- **Decision:** the cipher's key source is always + `KeyRouter, GuestVendor>` (see "A client with no + ZeroKMS" below) (in the deterministic build, + `KeyRouter`). `GuestVendor` is one + enum over the four vendor clients and `StaticKeyVendor`, plus `None`: with + no backend configured it holds no keyset record, so the router sends + every request to ZeroKMS and the zero-value client behaves as before. + **Alternative:** an enum over two cipher types, or an enum key source + that forwards every `DataKeySource`/`IndexKeySource` method. **Reason:** + `KeyRouter` already forwards all the provided methods correctly; a second + forwarder is where one would be forgotten. `GuestVendor::None` answers + `vendor()` with `Static`; it is never asked (it has no keyset). +- **Decision:** the vendor is configured in the `se_cipher_init` config + object: `key_vendor` (the `KeyVendor::name()`), `keyset_records` (an array + of record texts, required, not empty), and per vendor `aws_region`, + `aws_endpoint`, `vault_address`, `vault_namespace`, `vault_mount`, + `gcp_endpoint`, `static_key_ref`, `static_key` (32 bytes, a bytes value). + A key for another vendor than `key_vendor` is refused + (`ConfigError::NotForVendor`), as an unknown key is. No credential is in + the config. Azure needs no setting: the key identifier names the vault. + A refused record keeps its `RecordError` for `se_last_error` and is + `STATUS_ENCODING` (a `RecordError::Kms` keeps its key status). +- **Decision:** NewClient makes no vendor call: records are parsed and + checked at `se_cipher_init`, and a record key is unwrapped on the first + use of its keyset. **Reason:** the ZeroKMS default keyset is the only + load at init today; a vendor outage then fails the vendor keyset, not the + whole client. +- **Decision:** the deterministic test build keeps the vendor clients, so it + now imports `transport_send` and `vendor_credential_get` (not + `token_get`). Its config is the bare 32-byte seed, as before, or an object + `{"seed", key_vendor, keyset_records, ...}`. The mise import gates of all + four builds now require `vendor_credential_get`. +- **Decision:** the vendor client error types (`AwsKmsError`, + `CredentialsError`, `VaultError`, the three `AuthError`s, `GcpKmsError`, + `AzureKeyVaultError`, `TransportError`, `RecordError`) are causes whose + text the guest vouches for in `se_last_error`. **Reason:** each vendor + crate builds its messages from fixed text, the operation, the HTTP status + and the vendor's code and message, and never from a key or a secret. + +### The Go host + +- **Decision (dependency policy):** the module imports **no vendor SDK**. + The module already depends on `golang.org/x/oauth2` (for + `auth.OAuth2TokenSource`), and nothing else of a vendor. The credential + seam is two small interfaces with function adapters: + `AWSCredentialSource` (`AWSCredentialsFunc`) and `AccessTokenSource` + (`AccessTokenFunc`, `OAuth2AccessTokens(oauth2.TokenSource)`, + `StaticAccessToken`), plus `AWSCredentialsFromEnv`. The doc comments show + the three-line glue for aws-sdk-go-v2, azidentity and the Vault API + client. **Alternatives:** the vendor SDKs in the core module (four large + dependency trees for every user); a sub-package or sub-module per vendor + (`encrypt/vendor/aws`). **Reason:** sub-packages still add the SDKs to the + one `go.mod` every user downloads the requirements of, and sub-modules + need their own tags in `release-golang.yml`. The glue is short and is the + user's own choice of chain. Revisit if users ask for ready-made adapters; + then make them separate modules. +- **Decision:** the public surface is `WithKeyVendor(KeyVendor)`, + `WithKeysetRecords(...string)`, a sealed `KeyVendor` interface with five + struct types (`AWSKMS`, `GoogleCloudKMS`, `AzureKeyVault`, `VaultTransit`, + `StaticKey`), the credential types `AWSCredentials` and `AccessToken` + (both hide their secrets from `%v`, `%#v` and slog) and + `ErrNoKeyVendorCredentials`. Struct literals, not functional options, + because each backend has two to four settings. +- **Decision:** the options are checked host-side before anything is + resolved (Go principle 1, the earliest stage): records without a vendor, + a vendor without records, a missing region, address or credential + source, and a static key that is not 32 bytes are `ErrEncoding`. The + guest checks the records themselves at NewClient. +- **Decision:** `StaticKey.Key` is consumed like the client key: NewClient + wipes it whatever the outcome, a refused config and a replaced option + included. +- **Decision:** `ErrForbidden`, `ErrNotFound`, `ErrTransport` and + `ErrAuthentication` keep their names and now document their key vendor + meaning. **Alternative:** new `ErrKeyVendor*` sentinels. **Reason:** a + caller branches on the kind, not the backend; the `Diagnostic` names the + backend. + +### Tests + +- Guest (native): config parsing of every vendor and every refusal, the + credential readers (expiry, missing and unknown keys, control characters, + no secret in a message), the token providers' vendor and scope, an AWS + request signed with the host credentials, and record refusals. `mise run + wasm:guest:test`: 76 passed; clippy clean on all four feature sets, native + and wasm32. +- Go (hermetic, over the deterministic build): a static key round trip + (records built in the test by a second implementation of the record and + framing format), v2 leaves for the vendor keyset and v1 for ZeroKMS in one + client, `ErrForeignKeyset` across backends, a wrong static key; AWS KMS in + both modes through a fake `RoundTripper` that checks SigV4 with the host's + credentials and binds the encryption context, with exact vendor call + counts (derived: 1 `Decrypt`; wrapped: 1 `Decrypt` + 9 + `GenerateDataKey` for 3 rows of 3 sealed fields); an AWS refusal + (`ErrForbidden`, code `stack_kms::backend`, vendor `aws-kms`); a failed + credential source (`ErrTransport` plus the source's error, no request + sent); and the option and record checks. `mise run go:test` passes; + `golangci-lint` clean; `go generate ./...` changes nothing. +- **Not tested:** the Google Cloud, Azure and Vault clients through the Go + host (their requests are covered in their own crates; the guest test + checks their providers ask with the right vendor and scope). No live + vendor test from Go. + +### Deferred + +- **Deferred to CIP-5313:** the stack-kms side of the capability matrix. + The Go package docs carry a short one. + +### A client with no ZeroKMS (orchestrator review) + +- **Decision (binding, from the orchestrator):** a key vendor backend is an + alternative to ZeroKMS, so a Go client may have a vendor and no ZeroKMS. + The deferral recorded first is withdrawn. +- **Decision:** the Go client has no ZeroKMS when `WithKeyVendor` is given + and `WithCredentials` is not. NewClient then neither needs nor resolves + ZeroKMS credentials (no environment, no profile, no `CS_ZEROKMS_HOST`). + To hold both, give `WithCredentials` too, for example + `WithCredentials(AutoCredentials())`. **Alternative:** try + `AutoCredentials` and fall back to vendor-only when it finds nothing. + **Reason:** a client whose backend changes with the developer's profile + or the environment is not predictable; a fallback would also hide a + broken ZeroKMS setup. +- **Decision:** the guest config has no `client_id`, `client_key` or + `zerokms_url` for such a client (any one of them makes the others + required, as before). The guest key source is + `KeyRouter::new(MaybeZeroKms::Absent, vendor).with_vendor_default()`. + `MaybeZeroKms` (guest, `zerokms.rs`) forwards every `DataKeySource` + and `IndexKeySource` method to `Present(z)`, and `Absent` refuses every + request with `Error::Backend { vendor: "zerokms", kind: NotFound, source: + NoZeroKms }`. **Alternative:** a second cipher type, or the vendor source + directly. **Reason:** one cipher type for every configuration, and the + router already forwards every provided method. +- **Decision (additive, stack-kms-vendor):** `KeyRouter::with_vendor_default` + sends "no keyset named" to the vendor and reports the vendor as + `key_backend()`. A router test covers it. +- **Decision:** a new status, `STATUS_NO_ZEROKMS = 29` (`LAST_STATUS` moves), + and a new Go sentinel `ErrNoZeroKMS`, for any request that reaches the + absent ZeroKMS: a keyset no record names, or a v1 leaf. The status layer + finds the `NoZeroKms` cause by type. **Alternative:** `ErrNotFound`. + **Reason:** it is a configuration error with one fix, and a caller must be + able to tell it from a missing keyset. +- **Decision:** the default keyset option is `WithDefaultKeyset(KeysetSelector)`, + taking a `KeysetName` or a `KeysetID`, as `Client.Keyset` does. It crosses + as the config key `default_keyset`, a selector object (`{"id": 16 bytes}` + or `{"name": …}`); the guest resolves a name from the records and calls + `VendorKeySource::with_default_keyset`. It is refused (`ErrEncoding`, and + `ConfigError::OnlyWithoutZeroKms` in the guest) on a client with ZeroKMS, + whose default the ZeroKMS administrator sets. With one record it is not + needed; with several and none, NewClient fails with `ErrNotFound` + (CIP-5306's rule). `nil`, `Client.DefaultKeyset`'s selector and the + "any" selector are refused. +- **Decision:** `WithRootKeyTTL(time.Duration)` (config `root_key_ttl_ms`, + decimal milliseconds) sets `VendorKeySource::with_root_key_ttl`. Zero + keeps none; negative, or with no vendor, is `ErrEncoding`. **Measured:** + with zero, one Encrypt that derives terms unwraps the root key twice (the + data keys, then the index key in the settle), and each Decrypt once. +- **Tests:** guest config tests for the vendor-only form, the default keyset + and the TTL; `MaybeZeroKms` tests. Go tests over the **real embedded + guest** through `NewClient`, with every `CS_*` variable cleared and `HOME` + in a temporary directory: a static key client with two records (no + default is `ErrNotFound`; the default by name and by id; v2 leaves; + `ErrNoZeroKMS` for an unknown keyset and for a ZeroKMS row), and an AWS + KMS client through the fake RoundTripper (no vendor call at NewClient; 4 + `Decrypt` calls for one Encrypt and two Decrypts with `WithRootKeyTTL(0)`; + 4 credential requests); and the option refusals. + +## CIP-5312: `stash keyset create` / `rewrap` + +### How the TypeScript CLI calls Rust + +- **Decision:** the vendor calls go through two new napi functions, + `keysetCreate` and `keysetRewrap`, in the **existing `@cipherstash/auth` + binding** (`stack-auth-node`, module `src/keyset.rs`). The binding now + depends on `stack-kms`, `stack-kms-vendor` (feature `http`) and the four + vendor crates **by path**. `index.js` wraps both in the `Result` envelope, + and `index.d.ts` declares them with a new `KeysetFailure` type. + **Alternatives:** (b) a new napi package: six more platform packages, a + second trusted publishing setup and a second release path in + `release.yml`, for two functions. (c) a WASM build of the vendor crates + loaded by the CLI: the vendor crates send through `VendorTransport`, so a + JS `fetch` transport is possible, but the credential chains (IMDS, ECS, + web identity, metadata server, files) are native only, and it is a new + build and package too. (a′) `@cipherstash/protect-ffi`: it has its own + Cargo workspace, which pins its own vitaminc and does not hold the vendor + crates, and the CLI has it only as a devDependency. **Reason:** + `stack-auth-node` is in the root Cargo workspace with the vendor crates, + and `stash` already depends on `@cipherstash/auth` and its six platform + packages at the exact workspace version. No new package, platform or + release step. +- **Decision:** the vendor crates are compiled into the binary from the + tree. Their crates.io state (`release = false`, `publish = false`) does + not matter: napi builds the binding with cargo in this repository, and + the binding crate is `publish = false`. Confirmed: `pnpm run build:debug` + and `cargo build --release -p stack-auth-node` build with no registry + version of them. +- **Decision:** `stack-kms-gcp` is built **without default features**, so + the binary has no `rsa` code (RUSTSEC-2023-0071, see the orchestrator + review of CIP-5308). The binding makes its own Google credentials: + `GOOGLE_OAUTH_ACCESS_TOKEN` (for example `gcloud auth + print-access-token`), else the metadata server. A set + `GOOGLE_APPLICATION_CREDENTIALS` is refused with that remedy, not ignored. + **Alternative:** `GcpKms::from_env` (feature `http`, which implies + `service-account`). **Reason:** `@cipherstash/auth` is a published + package that every `@cipherstash/stack` user installs; it must not ship + the advisory before a security review signs it off. A person at a + terminal can run `gcloud auth print-access-token`. +- **Decision:** AWS, Vault and Azure use `DefaultAwsKms::from_env`, + `DefaultVaultTransit::from_env` and `DefaultAzureKeyVault::from_env` + unchanged. Azure does no RSA signing in the crate, so it adds no + advisory. Node's `process.env` writes reach the process environment, so + the variables from the CLI's `.env` files reach these chains too. +- **Measured:** the darwin-arm64 release binary grows from 6,144,944 B to + 6,823,632 B (+678,688 B, +11.0%). +- **Decision:** the `require-auth-npm-changeset.yml` rule now also watches + `Cargo.toml` and `src/` of the five vendor crates, and the three + integration workflows that load the auth binding filter on the same paths + (`auth-binding-step-order.test.mjs` derives that list from the rule). + **Reason:** a change to a vendor crate now changes the npm binary, and + nothing else would release it. `stack-kms` is not added, as + `stack-profile` is not: the rule lists the crates whose behaviour the + binding exposes. +- **Decision:** failures cross with the existing `__CS_FAIL__` sentinel and + a separate code set: `INVALID_OPTIONS`, `STATIC_KEY`, `VENDOR_CONFIG`, + `INVALID_RECORD` (with `diagnostic`, the `stack_kms_vendor::*` miette + code, and `help`) and `VENDOR_ERROR` (with `vendor` and `kind`, the + `BackendErrorKind`). **Alternative:** add them to `AuthFailure`. + **Reason:** they are not auth errors; the `AuthFailure` union is pinned + to `AuthError::ERROR_CODES` by a drift test. A new Rust test pins the + `KeysetFailure` codes in `index.d.ts` the same way. + +### The commands + +- **Decision:** `stash keyset create --backend + --vendor-key --mode [--name] [--key-env] + [--out] [--force] [--json]` and `stash keyset rewrap | --in + [--vendor-key] [--key-env] [--new-key-env] [--out] [--force] + [--json]`. Both are flags only, with no prompts. The CLI checks the flags + before it loads the native binding, so a usage error needs no binary. + `--mode` is required for every backend, the static backend too (ADR-0009: + the mode is chosen per keyset). +- **Decision (env names introduced):** the record prints as + `CS_KEYSET_RECORD`, or `CS_KEYSET_RECORD_` with `--name` (upper + case, other characters to `_`). A generated static key prints as + `CS_STATIC_KEY`. **No SDK reads either name today:** Go takes records + through `WithKeysetRecords(...string)` and Rust through + `KeysetRecord::parse`; neither reads an environment variable. The CLI + says so on stderr and in the help. **Alternative:** print the bare + record. **Reason:** a dotenv block can go straight into a secret store or + `--env-file`, and a name per keyset lets several records live in one + file. The TypeScript follow-up can read these names. +- **Decision (static key):** `--key-env ` names the variable that + holds the key (32 bytes as base64, base64url or 64 hex characters); the + CLI refuses a value that is not a variable name, so a key is never typed + into argv. Rust reads and decodes the variable, so the key never becomes + a JS string. Without `--key-env`, Rust generates a key with the CSPRNG and + returns it once, as padded standard base64 (Go: `base64.StdEncoding`). + Rewrap of a static record needs `--key-env` (the current key) and, when + `--vendor-key` names another reference, `--new-key-env` or a generated + key. **Alternative:** refuse to generate. **Reason:** a developer's first + static keyset should be one command. +- **Decision (rewrap):** every backend uses `KeysetRecord::rewrap` (unwrap, + then wrap), with the record's own key reference when `--vendor-key` is + not given. The wrap uses the vendor key's current version (AWS, Vault, + GCP primary version, Azure current version), so this also moves a record + after a rotation. **Alternative:** Vault's native `rewrap`, which keeps + the key inside Vault. **Reason:** one path for every vendor; the CLI + holds the unwrapped key only for one wrap. Revisit if a Vault user's + policy allows `rewrap` but not `decrypt`. +- **Decision (output):** as `stash env`: stdout carries only the dotenv + block or one NDJSON event, and every other line goes to stderr. Comment + values (name, vendor key) are JSON-quoted, so a name in an edited record + cannot inject a line. `--out` writes with mode 0600 and the `wx` flag + (an existing file is refused atomically), and `--force` overwrites, so + `rewrap --in f --out f --force` updates a file in place. `--json` with + `--out` gives `{ status: "written", … }`, which never holds the static + key. `--in` reads the one `stack-keyset:` record in a file and refuses a + file with none or with two. +- **Decision:** the help of both commands and the group summaries say that + ZeroKMS users never need them, and an unknown `keyset` subcommand prints + the same note. The help lists each backend's environment variables. +- **Tests:** Rust (`stack-auth-node`, 15 new): static create with a + generated key and with a key from the environment, rewrap to another key, + a wrong key (`VENDOR_ERROR` / `InvalidCiphertext`), the static + wrapped-mode refusal, option refusals, key decoding, and that no message + or `Debug` shows a key. JS (`@cipherstash/auth`, 6 new): both `Result` + arms through `index.js` with the real binding. CLI unit (19 new, binding + faked at the import): flag checks, the block and JSON shapes, `--out` + mode and overwrite, `--in`, failure mapping. CLI pty e2e (5 new): help, + unknown subcommand, a JSON usage error, and a static create, rewrap and + wrong-key round trip through the real binding. The vendor paths are not + run from TypeScript; their crates' fixtures and emulators cover them. +- **Note:** `pnpm run test:scripts` passes in full here (80 files) once + `pnpm install` has run; the two failures noted since CIP-5306 were + missing `node_modules` content. +- **Not done:** the CLI README has no `env` or `keyset` section; user docs + are CIP-5313. +- **Deferred to CIP-5313:** document `stash keyset` and the + `CS_KEYSET_RECORD*` names in the user docs. +- **Revisit:** if the TypeScript SDK gets key vendor backends, it should + read `CS_KEYSET_RECORD*` by itself, and then the CLI's "no SDK reads it" + line changes. + +## Orchestrator review of CIP-5312 + +- **Revisit:** the CLI suggests the env names `CS_KEYSET_RECORD[_]` and + `CS_STATIC_KEY`, but no SDK reads them. The Go SDK takes records through + `WithKeysetRecords`, and Rust through its API. Accepted as a naming + convention that the CLI states plainly. A later change can make the Go and + Rust SDKs read them by default, or the CLI can stop suggesting them. +- **Note:** the GCP path in the CLI refuses `GOOGLE_APPLICATION_CREDENTIALS`, + because the binary has no RSA code. A user runs + `GOOGLE_OAUTH_ACCESS_TOKEN=$(gcloud auth print-access-token)` instead. +- **Checked:** `pnpm run code:check` reports errors only in the ignored local + `.turbo/` cache and a `biome.json` schema-version notice from a newer local + Biome. There are no errors in tracked source. + +## CIP-5313: Document key vendor backend trade-offs (capability matrix) + +### Where the matrix lives + +- **Decision:** the capability matrix is a `# Capability matrix` section in + the crate-level rustdoc of `stack-kms-vendor`, with the subsections Key + modes, Backends, ZeroKMS-only features, Credentials, Rotation and Known + limits. It opens with the plain trade-off statement and the latency note. + **Alternative:** one table with every backend as a row and every fact as a + column. **Reason:** a table of eight columns does not read in rustdoc. The + facts split into three short tables and three bullet lists. +- **Decision:** each vendor crate links it with the intra-doc link + `stack_kms_vendor#capability-matrix` in a new section "Capabilities and + limits". `stack-kms` names the crate and the matrix in text. + **Alternative:** a docs.rs URL in `stack-kms`. **Reason:** `stack-kms` does + not depend on `stack-kms-vendor`, so an intra-doc link cannot resolve, and + docs.rs has no page for a crate that is not published yet. +- **Decision:** the Go package docs keep their short matrix, corrected (see + below), and point to the crate docs for credentials, rotation and limits. + +### Facts checked against the code (the code wins) + +- **Found (FIXED, see "The rewrap fix" below):** `KeysetRecord::rewrap` to **another** vendor + key makes every value sealed before it under **wrapped data keys** fail to + open (`InvalidCiphertext`). Each value's wrapped data key stays wrapped + under the old vendor key, and `VendorKeySource::retrieve_keys` unwraps it + with the record's current `key_ref`. A scratch test (not committed) + proved it with `FakeVendor`: derived mode opened, wrapped mode failed. + A rewrap to a new version of the same key is safe (every vendor keeps the + version in the wrapped key, or AWS behind the ARN). Derived mode is safe. + The rustdoc of `KeysetRecord::rewrap`, the `stash keyset rewrap` help and + note, the `stash-cli` skill and the unreleased CIP-5312 changeset all said + "stored values stay readable". **Decision:** correct all five texts in this + ticket; change no behaviour. **Alternative:** refuse the rewrap (code, + out of scope for a docs ticket). The orchestrator then chose the refusal, + made in this layer (below). +- **Drift fixed (ADR-0009):** "a settle of N values costs about 1 + N vendor + calls". The code makes the 1 (the index key unwrap) only when the settle + derives a search term, and a vendor with batch calls (Vault) makes ⌈N / + 250⌉. The matrix says so. +- **Drift noted (ADR-0009):** "the cipher unwraps [the root key] once and + caches it". The adapter (`VendorKeySource`) holds the root key, not the + cipher (a CIP-5306 decision). The matrix says "the adapter". +- **Drift noted (ADR-0011):** "the guest signs each vendor request itself + (SigV4, the Google OAuth JWT, the Azure AD token, the Vault token)". The + guest signs SigV4 only. The host supplies ready Google, Azure and Vault + tokens (CIP-5311 binding decision), and the guest has no RSA code. +- **Drift fixed (stack-kms-vendor):** the `KeyVendorBackend` docs named Vault + Transit `associated_data` as native associated data. The Vault client never + sends it (CIP-5310); it always wraps a bound key. The crate docs said + `bound` is for "a non-derived Vault key, an Azure RSA key"; the Vault and + Azure clients use it for every key. +- **Drift fixed (Go `doc.go`):** the matrix said every Encrypt or Decrypt + costs "1 +" one call per value under wrapped data keys; the 1 is only for + a call that derives search terms. It now also says that the guest sends + vendor calls one at a time, that the first use of a Vault or Azure key + costs one key read, that a separate `Equality` call costs one index key + unwrap, that a record key's associated data has no IV or descriptor, and + the rewrap limit above. +- **Checked, no change:** key lifetimes (`KeyLifetime::Settle` for wrapped, + `Ttl(root ttl)` for derived, `PerSettle` selection for both), the 5 minute + default TTL, `run_batch` defaults (16 at once, 5 attempts, 50 ms to 2 s), + ZeroKMS's 500 keys per request, each vendor's AAD binding and error + mappings, the credential chains and their environment variables, the six + Azure DNS suffixes, the `service-account` feature, and that the five vendor + crates carry the same licence file as `stack-kms`. + +### Other changes + +- **Decision:** `skills/stash-zerokms/SKILL.md` gets a section "Key vendor + backends (keysets outside ZeroKMS)". It names no ticket: no GitHub issue + exists for the TypeScript follow-up, and skills must not carry Linear IDs. +- **Decision:** the CLI README gets a short `stash keyset` section. This + closes CIP-5312's "Not done" item for the repository. The docs site + (cipherstash.com/docs) is maintained elsewhere and is not changed here. +- **Decision:** one `stash` patch changeset + (`stash-key-vendor-rewrap-docs.md`) for the skills, the CLI text and the + README. +- **Decision:** the rustdoc edits under `packages/stack-kms-*/src` trigger + `require-auth-npm-changeset.yml`, because the auth binding compiles those + crates in. The unreleased `auth-keyset-records.md` gets one true sentence + (a wrapped record's rewrap to another key) and so counts as modified. + **Alternative:** a separate `@cipherstash/auth` changeset that says + "documentation only". **Reason:** no empty changelog entry; the rule is + satisfied by a real fact. Checked locally with + `check-auth-npm-changeset.mjs`. +- **Decision:** `AGENTS.md`: the root workspace has eleven stack-* crates, + not six; the `stack-kms-gcp` entry names the `service-account` feature and + the advisory; the vendor crates entry points to the matrix and says that + the auth binding compiles them in and that TypeScript cannot use them; the + `@cipherstash/auth` entry names the vendor crates in its changeset rule. + `SECURITY.md` already listed the five crates; its `@cipherstash/auth` row + now says the binding compiles them in. +### The rewrap fix (orchestrator decision) + +- **Decision (binding, from the orchestrator):** **refuse** a rewrap that + would make stored values unreadable. Under wrapped data keys, + `KeysetRecord::rewrap` refuses a key reference that names another vendor + key with the new `RecordError::RewrapToAnotherKey { keyset, from, to }` + (`stack_kms_vendor::rewrap_another_key`), before any vendor call. A new + version of the same key is allowed. Under derived data keys any key is + allowed, because only the root key is rewrapped. **Alternatives:** keep a + history of old key references in the record; re-key stored data. + **Reason:** re-keying is out of scope (ADR-0009), and a key history would + grow the record with no limit. +- **Decision:** "same key" is a new provided method, + `KeyVendorBackend::same_key(a, b)`, default string equality. Google Cloud + KMS ignores a `/cryptoKeyVersions/{n}` suffix (through `parse_key_name`). + Azure compares the vault origin (lower case) and the key name, and ignores + the version (through `KeyId::parse`). A reference that does not parse is + compared as text. AWS KMS keeps the default: a key ARN, a key id and an + alias of the same key count as different keys, because the client cannot + tell offline where an alias points, and AWS rotation keeps the ARN. Vault + keeps the default: the key reference is the key name and holds no version. + The trait docs say to answer `false` when unsure (a false `false` only + refuses a rewrap). `Observed` forwards it. **Not forwarded:** the Go + guest's `GuestVendor` keeps the default, because the guest never rewraps a + record; it still compiles. +- **Decision:** the fix is on the CIP-5313 layer, in its own commit, not in + the CIP-5306 layer that introduced `rewrap`. **Reason (orchestrator):** + rebasing eight reviewed layers risks conflicts. +- **Decision:** the auth binding maps the error to a new `KeysetFailure` + code, `REWRAP_REFUSED` (`index.d.ts` and its drift test updated). The CLI + maps it to the code `rewrap_refused` with its own message and hint + (`messages.keyset.rewrapRefused`, `rewrapRefusedHint`). The help, the + rewrap note, the README, both skills, the crate matrix, the + `KeysetRecord::rewrap` docs, the Go `doc.go` and the three changesets now + state the rule. +- **Tests:** stack-kms-vendor: a wrapped rewrap to another key is refused + with no vendor call; a wrapped rewrap under its own key and a derived + rewrap to another key (static and fake vendors) open a value sealed + before it and derive the same term. stack-kms-gcp and stack-kms-azure + (their emulators): a wrapped record rewraps to a new version, unversioned + and pinned (Azure: RSA and `oct-HSM` keys), and the old value opens; a + wrapped rewrap to another key is refused with no request; a derived one + moves; `same_key` unit tests. stack-auth-node: the envelope of the new code. + CLI: the `--json` error of a refused rewrap. +- **Revisit:** an Azure client resolves an unversioned key's current version + once for its life. After a rotation it keeps wrapping under the old version + until a restart or a failed wrap. Documented; a TTL on the cached version + would close it. + +## Summary for review + +Open items first, then items that a later layer closed. Each line names the +ticket that recorded it. + +**Open** + +- **CIP-5313 (OPEN):** the docs site (cipherstash.com/docs) has nothing on + key vendor backends or `stash keyset`. +- **CIP-5313 (Revisit):** Azure caches a key's current version for the life + of the client, so a rotation reaches a running process late. +- **CIP-5314 (open ticket):** TypeScript (`@cipherstash/stack`) cannot use a + key vendor backend. +- **CIP-5304 (Revisit):** the trait payloads keep their ZeroKMS fields; + revisit when TypeScript moves onto stack-encrypt. +- **CIP-5306 (Revisit):** a term type outside stack-encrypt cannot defer its + PRF to the settle (`Responses::index_prf` is crate-private). +- **CIP-5307 (Revisit):** an unknown AWS 4xx error code maps to + `AccessDenied`, so a malformed request reads as a refusal. +- **CIP-5307 (deferred, no ticket):** AWS shared config and credentials + files, profiles, SSO and `credential_process` are not read. +- **CIP-5307/5310/5308/5309 (manual step):** the five vendor crates are + staged (`release = false`); the first publish and Trusted Publishing are a + person's job after stack-kms and stack-encrypt reach crates.io. +- **CIP-5310 (Revisit):** Vault with a private CA (`VAULT_CACERT`) or mTLS is + not supported natively; `renew-self` is not used. +- **CIP-5308 (Revisit, security):** the GCP service account login uses the + `rsa` crate (RUSTSEC-2023-0071); a security reviewer must sign it off + before the crate is armed; revisit when `rsa` 0.10 is stable. +- **CIP-5308 (Not verified):** the GCP fixtures are hand-written and the + live job has never run (needs `STACK_KMS_GCP_LIVE_KEY` and its secret). +- **CIP-5308 (deferred, no ticket):** `authorized_user` and + `external_account` credentials, impersonation, a quota project header, + re-recording the fixtures from Cloud KMS. +- **CIP-5309 (known limit):** an Azure `BadParameter` on `unwrapkey` always + reads as `InvalidCiphertext`. +- **CIP-5309 (Not verified):** the Azure fixtures are hand-written and the + live job has never run (needs `STACK_KMS_AZURE_LIVE_KEY` and its secrets). +- **CIP-5309 (deferred, security):** Azure client certificate credentials + (need a constant-time RSA signer); Azure CLI, Arc and Service Fabric + credentials; re-recording the fixtures from Azure. +- **CIP-5311 (Revisit):** ready-made Go adapters for the vendor SDKs, as + separate modules, if users ask. +- **CIP-5311 (Not tested):** the Google Cloud, Azure and Vault clients + through the Go host; no live vendor test from Go. +- **CIP-5312 (Revisit):** `stash keyset rewrap` uses unwrap and wrap for + Vault, not Transit's own `rewrap`; revisit if a policy allows `rewrap` but + not `decrypt`. +- **CIP-5312 (Revisit, orchestrator):** the CLI prints + `CS_KEYSET_RECORD[_]` and `CS_STATIC_KEY`, but no SDK reads them. +- **CIP-5304 (note):** `stack-encrypt::ui derive_diagnostics` (trybuild) + fails on a developer machine whose paths print as ``; not caused by + this stack. + +**Closed by a later layer** + +- CIP-5304 → CIP-5306: one vendor per cipher (`KeyRouter`, + `RecordError::SecondVendor`); the vendor default keyset; a `Settle` index + key bound to one settle (`SelectedKeyset::PerSettle`). +- CIP-5304/5305 → CIP-5311: the Go status mapping of the new errors (no new + codes except `STATUS_NO_ZEROKMS`). +- CIP-5305 → CIP-5306: the v2 seal path and the key mode on retrieve; routing + between ZeroKMS and one vendor. +- CIP-5306 → CIP-5307: the release group (`stack-kms-vendor`, staged) and the + native and guest transports. +- CIP-5306/5310 → CIP-5309: Azure binds its associated data with `bound`. +- CIP-5307/5310/5308/5309 → CIP-5311: the guest wiring of every vendor. +- CIP-5306/5307/5310/5308/5309 → CIP-5312: `stash keyset` through a napi + helper in `@cipherstash/auth` (GCP without the service account login). +- CIP-5311 → CIP-5313: the capability matrix on the Rust side. +- CIP-5312 → CIP-5313: the CLI README section for `stash keyset` (the docs + site is still open, above). +- **CIP-5313 (FIXED, defect):** a rewrap of a wrapped-data-key record to + another vendor key made its stored values unreadable. `KeysetRecord::rewrap` + now refuses it (`RecordError::RewrapToAnotherKey`; `REWRAP_REFUSED` in + `@cipherstash/auth`; `rewrap_refused` in `stash keyset rewrap`), and allows + a new version of the same key (`KeyVendorBackend::same_key`). diff --git a/languages/golang/encrypt/client.go b/languages/golang/encrypt/client.go index 042a7d09c5..fb195875ee 100644 --- a/languages/golang/encrypt/client.go +++ b/languages/golang/encrypt/client.go @@ -16,11 +16,12 @@ import ( ) // Client is one wasm instance holding one ZeroKMS client: its key, its -// default keyset, and the keysets it has loaded since. It is safe for +// default keyset, and the keysets it has loaded since — and, when it is +// given one, a key vendor backend with its keyset records. It is safe for // concurrent use; the guest runs one call's step at a time, because a wasm // instance is single-threaded, but never waits on the network inside one: -// a ZeroKMS request is performed between steps, with the client free for -// other callers (see step.go). Close it when done, as with any resource. +// a ZeroKMS or key vendor request is performed between steps, with the +// client free for other callers (see step.go). Close it when done, as with any resource. // // A call whose context ends fails with the context's error, and only that // call: one request's cancellation or deadline never closes the client for @@ -86,17 +87,38 @@ type Client struct { // With no options it is a working client: the credentials are // [AutoCredentials], and every other setting has a default. Each // [ClientOption] changes one. +// +// With [WithKeyVendor] and [WithKeysetRecords] the client also holds a key +// vendor backend. NewClient parses and checks the records but makes no +// vendor call: a record key is unwrapped on the first use of its keyset. func NewClient(ctx context.Context, opts ...ClientOption) (_ *Client, err error) { var cfg clientOptions for _, opt := range opts { opt(&cfg) } + // A StaticKey is consumed like the client key: wiped whatever the + // outcome, a refused config and a replaced option included. + defer func() { + wipeVendorKey(cfg.keyVendor) + for _, v := range cfg.supersededVendors { + wipeVendorKey(v) + } + }() rt := cfg.transport if rt == nil { rt = http.DefaultTransport } + // A key vendor backend is an alternative to ZeroKMS: with one and no + // WithCredentials, the client has no ZeroKMS, and no ZeroKMS + // credentials are looked for. WithCredentials(AutoCredentials()) gives + // both. + vendorOnly := cfg.keyVendor != nil && cfg.credentials == nil && len(cfg.superseded) == 0 creds := cfg.credentials - if creds == nil { + switch { + case creds != nil: + case vendorOnly: + creds = noZeroKMS{} + default: creds = AutoCredentials() } // Credentials a later WithCredentials replaced are never resolved, but @@ -114,10 +136,21 @@ func NewClient(ctx context.Context, opts ...ClientOption) (_ *Client, err error) // for. A refused config still consumes an explicit key, as // WithCredentials promises; any other Credentials has not been asked // yet, so holds nothing of this client's. - zerokmsURL, err := zerokmsEndpoint(cfg.zerokmsURL) + var zerokmsURL string + if !vendorOnly { + // A client with no ZeroKMS has no ZeroKMS endpoint to pin. + zerokmsURL, err = zerokmsEndpoint(cfg.zerokmsURL) + } if err == nil && cfg.keysetCacheSize < 0 { err = fmt.Errorf("%w: WithKeysetCacheSize must not be negative", ErrEncoding) } + var vendor *vendorSpec + if err == nil { + vendor, err = keyVendorConfig(cfg.keyVendor, cfg.keysetRecords) + } + if err == nil { + err = checkVendorOnlyOptions(&cfg, vendorOnly) + } if explicit, ok := creds.(*explicitCredentials); ok && err == nil && explicit.token == nil { // Knowable from the credentials as they were built: the one place // a missing token source is decided. @@ -173,6 +206,11 @@ func NewClient(ctx context.Context, opts ...ClientOption) (_ *Client, err error) clientKey: resolved.ClientKey, zerokmsURL: zerokmsURL, keysetCacheSize: cfg.keysetCacheSize, + vendor: vendor, + keysetRecords: cfg.keysetRecords, + vendorOnly: vendorOnly, + defaultKeyset: cfg.defaultKeyset, + rootKeyTTL: cfg.rootKeyTTL, }) if err != nil { return nil, err @@ -184,7 +222,7 @@ func NewClient(ctx context.Context, opts ...ClientOption) (_ *Client, err error) // whatever the init's outcome. resolved.ClientKey.Wipe() - t := &transport{rt: rt, token: resolved.Token} + t := &transport{rt: rt, token: resolved.Token, vendor: vendor} inst, err := newInstance(ctx, wasm, t, guest.PolicyFor(cfg.requireLockedMemory)) if err != nil { return nil, err @@ -304,11 +342,23 @@ type initConfig struct { clientKey *ClientKey zerokmsURL string keysetCacheSize int + // vendor and keysetRecords are the key vendor backend, nil and empty + // when the client has none. + vendor *vendorSpec + keysetRecords []string + // vendorOnly is a client with the key vendor backend and no ZeroKMS: + // the config then has no client id, client key or ZeroKMS endpoint. + vendorOnly bool + defaultKeyset KeysetSelector + rootKeyTTL *time.Duration } // encodeConfig renders the se_cipher_init object. The result holds the // client key; the caller wipes it, and the key it was read from. func encodeConfig(cfg initConfig) ([]byte, error) { + if cfg.vendorOnly { + return vcffi.Marshal(cfg.vendorFields()) + } if cfg.clientID == "" || cfg.clientKey.IsZero() { return nil, errors.New("encrypt: the credentials' client id and client key are required") } @@ -330,9 +380,30 @@ func encodeConfig(cfg initConfig) ([]byte, error) { if cfg.keysetCacheSize > 0 { fields = append(fields, vcvalue.Field{Key: "keyset_cache_size", Value: strconv.Itoa(cfg.keysetCacheSize)}) } + fields = append(fields, cfg.vendorFields()...) return vcffi.Marshal(fields) } +// vendorFields renders the key vendor backend's config keys: none without +// one. The result can hold a StaticKey's key; the caller wipes what it +// encodes. +func (cfg initConfig) vendorFields() vcvalue.Object { + if cfg.vendor == nil { + return nil + } + fields := cfg.vendor.configFields(cfg.keysetRecords) + if cfg.keysetCacheSize > 0 && cfg.vendorOnly { + fields = append(fields, vcvalue.Field{Key: "keyset_cache_size", Value: strconv.Itoa(cfg.keysetCacheSize)}) + } + if cfg.defaultKeyset != nil { + fields = append(fields, vcvalue.Field{Key: "default_keyset", Value: cfg.defaultKeyset.selector()}) + } + if cfg.rootKeyTTL != nil { + fields = append(fields, vcvalue.Field{Key: "root_key_ttl_ms", Value: strconv.FormatInt(cfg.rootKeyTTL.Milliseconds(), 10)}) + } + return fields +} + // Close shuts the guest down — the client key and every loaded index key // are wiped inside the instance — and releases the runtime and the // guest's memory, which is wiped on the way out. Idempotent. Every call @@ -375,7 +446,9 @@ func (c *Client) Keyset(sel KeysetSelector) *Cipher { // DefaultKeyset binds the client to its default keyset — the one a ZeroKMS // administrator set for this client, which is what naming no keyset -// resolves to: Rust's StackCipher::default_keyset. Loaded at NewClient, so +// resolves to: Rust's StackCipher::default_keyset. A client with a key +// vendor backend and no ZeroKMS has a vendor keyset as its default: its only +// keyset record, or the one [WithDefaultKeyset] names. Loaded at NewClient, so // using it never touches ZeroKMS. There is no way to redefine it from here; // which keyset is the default is the server's to say. Any other keyset is // [Client.Keyset]. @@ -466,6 +539,8 @@ func (c *Client) drive(ctx context.Context, kms bool, f stepFunc) ([]byte, error step, resumed := f, false var handle uint64 var responses []byte + var op operation + defer op.end() for { out, io, err := c.step(ctx, token, tokenErr, step) if resumed { @@ -483,6 +558,11 @@ func (c *Client) drive(ctx context.Context, kms bool, f stepFunc) ([]byte, error return nil, ctx.Err() } if io == nil { + // The guest reports a failed credential request as a transport + // failure and no more; the credential source said why. + if err != nil && op.credErr != nil { + err = fmt.Errorf("%w (key vendor credential source: %w)", err, op.credErr) + } return settle(ctx, out, err) } // A caller that gave up while the guest ran sends no request. @@ -491,7 +571,7 @@ func (c *Client) drive(ctx context.Context, kms bool, f stepFunc) ([]byte, error c.abort(io.handle) return nil, fmt.Errorf("%w: %w", err, ErrTransport) } - responses, err = c.transport.performStep(ctx, io) + responses, err = c.transport.performStep(ctx, io, &op) if err != nil { c.abort(io.handle) return nil, err diff --git a/languages/golang/encrypt/doc.go b/languages/golang/encrypt/doc.go index ea39184435..df7e10bae7 100644 --- a/languages/golang/encrypt/doc.go +++ b/languages/golang/encrypt/doc.go @@ -314,10 +314,14 @@ // // The guest imports exactly one host function, a bearer-token fetch, served // by the credentials' auth strategy. It never performs a request itself: an -// operation that needs ZeroKMS pauses and hands the request to the client, -// which sends it with any [net/http.RoundTripper] and resumes the operation -// with the response. What crosses per ZeroKMS call is what would -// cross TLS anyway; derived key material never leaves the guest. Under +// operation that needs ZeroKMS or the key vendor backend pauses and hands +// the request to the client, which sends it with any +// [net/http.RoundTripper] and resumes the operation with the response. A +// key vendor backend's short-lived credential, from the source given with +// [WithKeyVendor], and a retry's backoff are handed to the client the same +// way, so the guest never waits on the host. What crosses per ZeroKMS or +// vendor call is what would cross TLS anyway; derived and unwrapped key +// material never leaves the guest. Under // [AutoCredentials] and [OIDCFederation] the same RoundTripper also carries // the authentication requests to CTS, so one scoped to the ZeroKMS host alone // is not enough. Under [NewCredentials] those requests go through the store @@ -340,6 +344,83 @@ // resolved fail NewClient with [ErrNoCredentials]; credentials that resolve // but do not work fail it too, at the one ZeroKMS round trip it makes. // +// # Key vendor backends +// +// A key vendor backend is an alternative to ZeroKMS: [AWSKMS], +// [GoogleCloudKMS], [AzureKeyVault], [VaultTransit] or [StaticKey]. Give it +// with [WithKeyVendor], and its keyset records (the "stack-keyset:…" texts) +// with [WithKeysetRecords]: +// +// client, err := encrypt.NewClient(ctx, +// encrypt.WithKeyVendor(encrypt.AWSKMS{Region: "us-east-1", Credentials: creds}), +// encrypt.WithKeysetRecords(os.Getenv("USERS_KEYSET_RECORD"))) +// +// With no [WithCredentials] the client has no ZeroKMS and needs no +// CipherStash credentials. Its default keyset is its only keyset record, or +// the one [WithDefaultKeyset] names, and a call that needs ZeroKMS fails +// with [ErrNoZeroKMS]. With WithCredentials too, the client holds both: a +// keyset in a record goes to the vendor, and every other keyset, the default +// included, goes to ZeroKMS. Select a vendor keyset by its record's name or +// id with [Client.Keyset]. A value sealed under one backend never opens +// under the other. +// +// The guest builds and signs every vendor request. The host supplies only +// short-lived credentials, per request: temporary AWS credentials, a Google +// OAuth access token, an Azure access token for the scope the guest names, +// or a Vault token. Fill them from the vendor's own Go credential chain +// through [AWSCredentialsFunc], [AccessTokenFunc] or [OAuth2AccessTokens]. +// This package imports no vendor SDK. +// +// Each keyset record fixes one key mode. Wrapped data keys give each value +// its own vendor-wrapped data key. Derived data keys derive each data key +// from one root key per keyset (HKDF-SHA256). In wrapped data key mode, no +// key outlives the call that needed it. In derived data key mode, the root +// key is cached for the TTL. In both modes, the database holds no usable +// key material. +// +// wrapped data keys derived data keys +// key lifetime index key: one call; root key: cached in the guest +// data key: one call for 5 minutes (WithRootKeyTTL); +// the rest derived per call +// vendor calls per one per sealed or opened 0, or 1 per keyset when the +// call, per keyset value (Vault: 1 per 250), root key is not cached +// plus 1 when the call +// derives search terms +// the vendor can every value it seals or the root key unwrap: a refusal +// refuse opens, and every call that takes effect when the cached +// derives search terms root key expires +// +// The guest sends vendor calls one at a time. Under wrapped data keys each +// separate call that derives a search term, such as a field's Equality, +// costs one vendor round trip for the index key: encrypt many rows in one +// Encrypt call, not one call per row. The first use of a Vault Transit key +// or an Azure key costs one more call, a key read. +// +// The associated data (the keyset id and, for a value's data key, the IV and +// field descriptor) is bound to every wrapped key. AWS KMS binds it as the +// encryption context, which CloudTrail logs and key policies can test. +// Google Cloud KMS binds it as additional authenticated data. Vault Transit +// and Azure Key Vault wrap a key with a digest of it, which the guest checks +// after the unwrap; a derived Transit key also gets it as its derivation +// context. Azure does not see the binding. [StaticKey] offers derived data +// keys only, and nothing outside the process can refuse or audit its use. +// +// A rewrap of a keyset record (stash keyset rewrap) does not re-key stored +// values. A record under wrapped data keys may therefore move only to a new +// version of its own vendor key; a move to another key is refused. A record +// under derived data keys may move to any key. The stack-kms-vendor crate documentation has the full capability +// matrix: credentials, rotation and known limits of each backend. +// +// A vendor keyset refuses ZeroKMS-only features (a lock context, a +// decryption policy) with an error; it never ignores one. A vendor failure +// reads as the ZeroKMS one of the same kind: [ErrForbidden] for a refused +// request or bad credentials, [ErrNotFound] for a missing key, [ErrTransport] +// for an unreachable vendor, a throttle that outlasted the retries or a +// failed credential source (whose own error errors.Is also matches), and +// [ErrAuthentication] for a wrapped key that does not open. The +// [*Diagnostic]'s code is "stack_kms::backend", and its "vendor" field names +// the backend. +// // # Errors // // The generator and the compiler find a mistake in a declaration, so no call diff --git a/languages/golang/encrypt/errors.go b/languages/golang/encrypt/errors.go index 15b9dc9313..5e2090307d 100644 --- a/languages/golang/encrypt/errors.go +++ b/languages/golang/encrypt/errors.go @@ -17,7 +17,8 @@ import ( // two. var ( // ErrAuthentication is an AEAD open failure: a tampered ciphertext, a - // wrong element derivation, or a wrong AAD that reached the AEAD. Against + // wrong element derivation, or a wrong AAD that reached the AEAD. A key + // vendor backend reports a wrapped key that does not unwrap as this too. Against // ZeroKMS a wrong AAD is usually refused earlier as ErrForbidden, because // every data key is bound to its context. ErrAuthentication = guest.ErrAuthentication @@ -33,15 +34,21 @@ var ( ErrUnauthorized = guest.ErrUnauthorized // ErrForbidden is ZeroKMS refusing the request (HTTP 403): the token is // valid but not permitted, or a data key's bound context did not match - // the one presented — the production form of a wrong-AAD open. + // the one presented — the production form of a wrong-AAD open. From a + // key vendor backend: the vendor refused the request, or the guest + // refused the host's credential (expired, or the wrong shape). ErrForbidden = guest.ErrForbidden // ErrNotFound is ZeroKMS reporting a missing resource (HTTP 404): an - // unknown keyset name or id, or a data key that does not exist. + // unknown keyset name or id, or a data key that does not exist. From a + // key vendor backend: the vendor key does not exist. ErrNotFound = guest.ErrNotFound // ErrConflict is ZeroKMS reporting a resource conflict (HTTP 409). ErrConflict = guest.ErrConflict // ErrTransport is a failure to reach ZeroKMS or to read its response: // the transport returned an error, or the endpoint could not be resolved. + // From a key vendor backend also: the vendor is unavailable, it went on + // throttling past the retries, or the key vendor credential source + // failed (its error is attached, so errors.Is matches it too). ErrTransport = guest.ErrTransport // ErrKMS is any other ZeroKMS failure: an unparseable response, invalid // key material, or an unclassified server error. @@ -58,6 +65,11 @@ var ( // [Cipher.Context]. Refused before any key is retrieved; a row stored // under another tenant is a mismatch, never a decrypted value. ErrContextMismatch = guest.ErrContextMismatch + // ErrNoZeroKMS is a client that has a key vendor backend and no ZeroKMS + // asked for a ZeroKMS keyset: a keyset none of its keyset records names, + // or a value a ZeroKMS keyset sealed. Refused before any request. Give + // the keyset's record, or ZeroKMS credentials with WithCredentials. + ErrNoZeroKMS = guest.ErrNoZeroKMS // ErrMemoryLock is guest memory that could not be locked in RAM (or, // on Linux, excluded from core dumps). NewClient returns it when // WithRequireLockedMemory is given, and so does any later call under diff --git a/languages/golang/encrypt/export_test.go b/languages/golang/encrypt/export_test.go index 53dc819444..31df580177 100644 --- a/languages/golang/encrypt/export_test.go +++ b/languages/golang/encrypt/export_test.go @@ -11,6 +11,7 @@ import ( "github.com/cipherstash/stack/languages/golang/internal/guest" "github.com/cipherstash/stack/languages/golang/internal/record" "github.com/cipherstash/vitaminc/bindings/go/vcffi" + "github.com/cipherstash/vitaminc/bindings/go/vcvalue" "github.com/tetratelabs/wazero/api" ) @@ -135,6 +136,48 @@ func deterministicClient(ctx context.Context, wasm []byte, seed [32]byte) (*Clie return c, nil } +// NewDeterministicVendorClient is NewDeterministicClient with a key vendor +// backend beside the seeded ZeroKMS stand-in: the deterministic build's +// object config, {"seed", key_vendor, keyset_records, ...}. Vendor requests +// go to rt, and their credentials come from the backend's source through +// the real step loop. The StaticKey, if any, is wiped as NewClient wipes +// it. +func NewDeterministicVendorClient(ctx context.Context, seed [32]byte, rt http.RoundTripper, v KeyVendor, records ...string) (*Client, error) { + defer wipeVendorKey(v) + wasm, err := os.ReadFile(deterministicGuestPath) + if err != nil { + return nil, ErrDeterministicGuestNotBuilt + } + spec, err := keyVendorConfig(v, records) + if err != nil { + return nil, err + } + fields := vcvalue.Object{{Key: "seed", Value: seed[:]}} + if spec != nil { + fields = append(fields, spec.configFields(records)...) + } + encoded, err := vcffi.Marshal(fields) + if err != nil { + return nil, err + } + defer wipe(encoded) + tr := &transport{rt: rt, token: noToken{}, vendor: spec} + inst, err := newInstance(ctx, wasm, tr, guest.BestEffort) + if err != nil { + return nil, err + } + c := newClient(inst, tr) + out, err := c.call(ctx, func(ctx context.Context, inst *instance) ([]byte, error) { + return inst.call(ctx, inst.cipherInit, buf(encoded)) + }) + if err != nil { + _ = c.Close() + return nil, fmt.Errorf("encrypt: deterministic cipher init: %w", err) + } + copy(c.def[:], out) + return c, nil +} + // RawClient is a guest that was never given a client key: every well-formed // operation is ErrState, every malformed one ErrEncoding, and the checker // exports work. diff --git a/languages/golang/encrypt/guest/Cargo.lock b/languages/golang/encrypt/guest/Cargo.lock index 9c2cc2d59b..c9aca8ef9c 100644 --- a/languages/golang/encrypt/guest/Cargo.lock +++ b/languages/golang/encrypt/guest/Cargo.lock @@ -466,6 +466,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "78c8292055d1c1df0cce5d180393dc8cce0abec0a7102adb6c7b1eef6016d60a" dependencies = [ "generic-array", + "rand_core 0.6.4", "typenum", ] @@ -979,6 +980,15 @@ version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "7ebdb29d2ea9ed0083cd8cece49bbd968021bd99b0849edb4a9a7ee0fdf6a4e0" +[[package]] +name = "hkdf" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4aaa26c720c68b866f2c96ef5c1264b3e6f473fe5d4ce61cd44bbe913e553018" +dependencies = [ + "hmac", +] + [[package]] name = "hmac" version = "0.13.0" @@ -2172,6 +2182,11 @@ dependencies = [ "stack-encrypt", "stack-guest-abi", "stack-kms", + "stack-kms-aws", + "stack-kms-azure", + "stack-kms-gcp", + "stack-kms-vault", + "stack-kms-vendor", "uuid", "zeroize", "zerokms-protocol", @@ -2183,6 +2198,7 @@ version = "0.0.0" dependencies = [ "miette", "serde_json", + "stack-kms-vendor", "thiserror 1.0.69", "vitaminc-aead 0.9.0", "vitaminc-aead-frame", @@ -2218,6 +2234,96 @@ dependencies = [ "zerokms-protocol", ] +[[package]] +name = "stack-kms-aws" +version = "0.1.0" +dependencies = [ + "base64ct", + "futures", + "hmac", + "serde", + "serde_json", + "sha2 0.11.0", + "stack-kms", + "stack-kms-vendor", + "thiserror 1.0.69", + "url", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + +[[package]] +name = "stack-kms-azure" +version = "0.1.0" +dependencies = [ + "base64ct", + "futures", + "serde", + "serde_json", + "stack-kms", + "stack-kms-vendor", + "thiserror 1.0.69", + "url", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + +[[package]] +name = "stack-kms-gcp" +version = "0.1.0" +dependencies = [ + "base64ct", + "futures", + "serde", + "serde_json", + "stack-kms", + "stack-kms-vendor", + "thiserror 1.0.69", + "url", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + +[[package]] +name = "stack-kms-vault" +version = "0.1.0" +dependencies = [ + "base64ct", + "futures", + "serde", + "serde_json", + "stack-kms", + "stack-kms-vendor", + "thiserror 1.0.69", + "url", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + +[[package]] +name = "stack-kms-vendor" +version = "0.1.0" +dependencies = [ + "aes-gcm", + "base64ct", + "futures", + "hkdf", + "miette", + "serde_json", + "sha2 0.11.0", + "stack-kms", + "subtle", + "thiserror 1.0.69", + "uuid", + "vitaminc 0.9.0", + "web-time", + "zeroize", +] + [[package]] name = "stack-profile" version = "0.44.0" diff --git a/languages/golang/encrypt/guest/Cargo.toml b/languages/golang/encrypt/guest/Cargo.toml index 4df0b70ba7..980a2dd4dc 100644 --- a/languages/golang/encrypt/guest/Cargo.toml +++ b/languages/golang/encrypt/guest/Cargo.toml @@ -53,8 +53,20 @@ stack-encrypt = { path = "../../../../packages/stack-encrypt", default-features # buffer registry (and with them the `se_alloc`/`se_dealloc` exports), the # packed-result helpers, the status table and the step loop. # This crate defines only the exports that are its own. -stack-guest-abi = { path = "../../../../packages/stack-guest-abi" } +stack-guest-abi = { path = "../../../../packages/stack-guest-abi", features = ["vendor"] } stack-kms = { path = "../../../../packages/stack-kms", default-features = false } +# The key vendor backends (ADR-0009, ADR-0011, CIP-5311): the seam, the +# keyset records and the static key backend, and one hand-written client for +# each vendor. Default features off: no reqwest and no native credential +# chains. HTTP goes through `transport_send` (`HostVendorTransport`) and the +# short-lived credentials come from the host's `vendor_credential_get` +# import. `stack-kms-gcp` without `service-account` has no RSA code and no +# `rsa` dependency: the guest never signs with a service account key. +stack-kms-vendor = { path = "../../../../packages/stack-kms-vendor", default-features = false } +stack-kms-aws = { path = "../../../../packages/stack-kms-aws", default-features = false } +stack-kms-vault = { path = "../../../../packages/stack-kms-vault", default-features = false } +stack-kms-gcp = { path = "../../../../packages/stack-kms-gcp", default-features = false } +stack-kms-azure = { path = "../../../../packages/stack-kms-azure", default-features = false } # The EQL types, for the `eql` build only. By path, like every other # eql-bindings dependency in the repository: the SQL that stores a payload # and the Rust that emits it must agree (scripts/lint-no-eql-registry-pins.mjs). diff --git a/languages/golang/encrypt/guest/src/abi.rs b/languages/golang/encrypt/guest/src/abi.rs index 8fad3f7af5..d7ff5642c3 100644 --- a/languages/golang/encrypt/guest/src/abi.rs +++ b/languages/golang/encrypt/guest/src/abi.rs @@ -106,14 +106,27 @@ use crate::status::STATUS_INTERNAL; #[cfg(not(feature = "deterministic-kms"))] use stack_encrypt::ErrorPayload; -/// The key source the instance's cipher runs over: the host-transport -/// ZeroKMS client with host-supplied tokens — or, in the `deterministic-kms` -/// test build, the seeded source the record fixture was sealed under. +/// The key source the instance's cipher runs over: ZeroKMS and at most one +/// key vendor backend, routed by keyset ([`stack_kms_vendor::KeyRouter`]). +/// ZeroKMS is the host-transport client with host-supplied tokens — or, in +/// the `deterministic-kms` test build, the seeded source the record fixture +/// was sealed under. The vendor is [`GuestVendor`], over the step loop; with +/// none configured it holds no keyset and every request goes to ZeroKMS. #[cfg(not(feature = "deterministic-kms"))] -type GuestKms = - stack_kms::StackKms; +type GuestZeroKms = crate::zerokms::MaybeZeroKms< + stack_kms::StackKms, +>; #[cfg(feature = "deterministic-kms")] -type GuestKms = crate::deterministic::DeterministicSource; +type GuestZeroKms = crate::zerokms::MaybeZeroKms; + +/// The instance's key vendor backend: its requests, credentials and +/// backoff waits go to the host through the step loop. +type GuestVendor = crate::vendor::GuestVendor< + stack_guest_abi::vendor_transport::StepVendorTransport, + crate::host::StepCredentialImport, +>; + +type GuestKms = stack_kms_vendor::KeyRouter; /// The instance's cipher: `stack-encrypt` over [`GuestKms`]. type GuestCipher = StackCipher; @@ -205,7 +218,7 @@ pub unsafe extern "C" fn se_cipher_init(cfg_ptr: *mut u8, cfg_len: u32) -> u64 { #[cfg(not(feature = "deterministic-kms"))] use crate::config::{parse_config as parse_init, CipherConfig as InitConfig}; #[cfg(feature = "deterministic-kms")] -use crate::deterministic::Seed as InitConfig; +use crate::deterministic::SeedConfig as InitConfig; #[cfg(feature = "deterministic-kms")] fn parse_init(bytes: &[u8]) -> Result { InitConfig::parse(bytes) @@ -215,40 +228,52 @@ fn parse_init(bytes: &[u8]) -> Result { fn cipher_init(config: InitConfig) -> Result, u32> { use crate::host::{HostTokenStrategy, WasiHostConnection}; use crate::status::STATUS_KMS_TRANSPORT; + use crate::zerokms::MaybeZeroKms; use stack_kms::{ClientOpts, StackKms}; if init_refused() { return Err(init_out_of_order()); } - // One request at a time: a step hands the host one request, and the - // first version of the step loop keeps it that way. Raising it later - // changes no ABI: a step's request list can hold several. - // - // `max_keys_per_req` stays at the client default (500). That is what - // bounds "one ZeroKMS call": a batch is assembled once, then - // `Client::send_chunked` splits it into sequential requests of at most - // that many keys — so a 1200-leaf record batch is three calls, not one. - // Raising it here would trade a documented, server-friendly request size - // for a claim the server need not honour, so the bound is kept and the - // docs say 500 rather than "one". - let opts = ClientOpts::new(config.endpoint) - .with_max_concurrent_reqs(1) - .map_err(|e| last_error::record(&e, e.payload(), STATUS_INTERNAL))?; - let kms = StackKms::::connect( - opts, - HostTokenStrategy, - config.client_key, - ) - .map_err(|e| { - last_error::record_with( - &e, - e.payload(), - &crate::status::describe, - STATUS_KMS_TRANSPORT, - ) - })?; - + let vendor = build_vendor(config.vendor)?; + let kms = match config.client_key { + // A key vendor backend and no ZeroKMS (CIP-5311): no ZeroKMS client + // is built, no token is asked for, and the default keyset is the + // vendor's. + None => { + stack_kms_vendor::KeyRouter::new(MaybeZeroKms::Absent, vendor).with_vendor_default() + } + Some(client_key) => { + // One request at a time: a step hands the host one request, and the + // first version of the step loop keeps it that way. Raising it later + // changes no ABI: a step's request list can hold several. + // + // `max_keys_per_req` stays at the client default (500). That is what + // bounds "one ZeroKMS call": a batch is assembled once, then + // `Client::send_chunked` splits it into sequential requests of at most + // that many keys — so a 1200-leaf record batch is three calls, not one. + // Raising it here would trade a documented, server-friendly request size + // for a claim the server need not honour, so the bound is kept and the + // docs say 500 rather than "one". + let opts = ClientOpts::new(config.endpoint) + .with_max_concurrent_reqs(1) + .map_err(|e| last_error::record(&e, e.payload(), STATUS_INTERNAL))?; + let zerokms = StackKms::::connect( + opts, + HostTokenStrategy, + client_key, + ) + .map_err(|e| { + last_error::record_with( + &e, + e.payload(), + &crate::status::describe, + STATUS_KMS_TRANSPORT, + ) + })?; + stack_kms_vendor::KeyRouter::new(MaybeZeroKms::Present(zerokms), vendor) + } + }; let mut builder = StackCipher::builder().kms(kms); if let Some(size) = config.keyset_cache_size { builder = builder.keyset_cache_size(size); @@ -266,15 +291,31 @@ fn cipher_init(config: InitConfig) -> Result, u32> { } /// The `deterministic-kms` build's init: the config buffer is the -/// deterministic source's 32-byte seed and nothing else — no client key, no -/// endpoint, no token, no network. A test artefact, never shipped; see +/// deterministic source's 32-byte seed, or an object of the seed and the key +/// vendor backend keys — no client key, no endpoint, no token, and no +/// ZeroKMS network. A test artefact, never shipped; see /// [`crate::deterministic`]. #[cfg(feature = "deterministic-kms")] -fn cipher_init(seed: InitConfig) -> Result, u32> { +fn cipher_init(config: InitConfig) -> Result, u32> { if init_refused() { return Err(init_out_of_order()); } - let builder = StackCipher::builder().kms(seed.source()); + // The seeded source is this build's ZeroKMS, so a default keyset of the + // vendor's does not apply here. + if config + .vendor + .as_ref() + .is_some_and(|v| v.default_keyset.is_some()) + { + return Err(malformed( + "the deterministic build's default keyset is the seeded source's", + )); + } + let kms = stack_kms_vendor::KeyRouter::new( + crate::zerokms::MaybeZeroKms::Present(config.seed.source()), + build_vendor(config.vendor)?, + ); + let builder = StackCipher::builder().kms(kms); let initialising = Initialising::begin(); run(Box::pin(async move { let _initialising = initialising; @@ -312,6 +353,27 @@ impl Drop for Initialising { } } +/// Build the configured key vendor backend over the step loop: its requests, +/// credentials and backoff waits all pause the operation for the host (no +/// vendor call here: records are parsed and checked, keys are unwrapped on +/// first use). A setting the vendor client refuses is +/// `STATUS_ENCODING` naming its config key; a refused keyset record keeps +/// its error for `se_last_error` ([`crate::status::fail_record`]). +fn build_vendor( + config: Option, +) -> Result, u32> { + use crate::vendor::BuildError; + crate::vendor::build( + config, + stack_guest_abi::vendor_transport::StepVendorTransport, + crate::host::StepCredentialImport, + ) + .map_err(|e| match e { + BuildError::Setting(key) => malformed(format!("the config's {key} is not valid")), + BuildError::Record(record) => crate::status::fail_record(&record), + }) +} + /// `se_cipher_init` called a second time, while one is paused, or after /// `se_shutdown`. fn init_out_of_order() -> u32 { diff --git a/languages/golang/encrypt/guest/src/config.rs b/languages/golang/encrypt/guest/src/config.rs index 44f77c2187..db43a3f838 100644 --- a/languages/golang/encrypt/guest/src/config.rs +++ b/languages/golang/encrypt/guest/src/config.rs @@ -11,6 +11,34 @@ //! | `zerokms_url` | no | pins the ZeroKMS endpoint at init; when absent the endpoint is resolved from the access token's `services` claim on first use | //! | `keyset_cache_size` | no | how many keysets beyond the default the cipher keeps loaded (a positive decimal integer; the crate default, 1024, when absent). See `StackCipherBuilder::keyset_cache_size` | //! +//! A key vendor backend (ADR-0009, CIP-5311) is configured with the keys +//! below. They are all absent for a ZeroKMS-only client, which is then +//! unchanged. +//! +//! | key | when | meaning | +//! |-------------------|------|---------| +//! | `key_vendor` | with any key below | the backend: `aws-kms`, `vault-transit`, `gcp-kms`, `azure-key-vault` or `static` (`KeyVendor::name`) | +//! | `keyset_records` | with `key_vendor`, required | an array of one or more keyset record texts (`stack-keyset:…`), all of that backend | +//! | `aws_region` | `aws-kms`, required | the AWS region | +//! | `aws_endpoint` | `aws-kms` | overrides the regional AWS KMS endpoint | +//! | `vault_address` | `vault-transit`, required | the Vault address | +//! | `vault_namespace` | `vault-transit` | the Vault namespace | +//! | `vault_mount` | `vault-transit` | the Transit mount (default `transit`) | +//! | `gcp_endpoint` | `gcp-kms` | overrides the Cloud KMS endpoint | +//! | `static_key_ref` | `static`, required | the static key's reference, as the records name it | +//! | `static_key` | `static`, required | the static key: 32 bytes (a bytes value, not a string) | +//! | `default_keyset` | no ZeroKMS only | the default keyset, `{"id": 16 bytes}` or `{"name": …}`; else the only record | +//! | `root_key_ttl_ms` | any | how long a derived data key keyset's root key stays unwrapped, in milliseconds (`0` keeps none; default 5 minutes) | +//! +//! With a key vendor backend and none of `client_id`, `client_key` and +//! `zerokms_url`, the client has no ZeroKMS: the default keyset is the +//! vendor's, and a request for any other keyset is `STATUS_NO_ZEROKMS`. +//! +//! A vendor key for another backend than `key_vendor`, or with no +//! `key_vendor`, is refused. The vendor's credentials are never in the +//! config: the guest asks the host for them, through the step loop, before +//! each vendor request (see [`crate::vendor`]). +//! //! There is no config key for a keyset. `{"default"}` means the default a //! ZeroKMS administrator set for this client, and a client does not get to //! redefine it — the same reason `StackCipherBuilder::keyset` was removed. @@ -31,18 +59,53 @@ //! that value. use std::num::NonZeroUsize; +use std::time::Duration; use serde::{Deserialize, Deserializer}; use stack_encrypt::dynamic::frame::serde::from_slice; use stack_guest_abi::last_error::malformed; -use stack_kms::{ClientKey, ZeroKmsEndpoint}; +use stack_kms::{ClientKey, KeyVendor, ZeroKmsEndpoint}; +use stack_kms_vendor::KeyBytes; use uuid::Uuid; +use zeroize::Zeroize; + +use crate::vendor::{DefaultKeyset, VendorConfig, VendorSettings}; + +/// The [`VendorWire`] of a wire struct that declares the key vendor +/// backend keys inline (the config's, the deterministic build's seed +/// object). A macro because the two structs are distinct types. +macro_rules! vendor_wire { + ($wire:expr) => { + $crate::config::VendorWire { + key_vendor: $wire.key_vendor, + keyset_records: $wire.keyset_records, + aws_region: $wire.aws_region, + aws_endpoint: $wire.aws_endpoint, + vault_address: $wire.vault_address, + vault_namespace: $wire.vault_namespace, + vault_mount: $wire.vault_mount, + gcp_endpoint: $wire.gcp_endpoint, + static_key_ref: $wire.static_key_ref, + static_key: $wire.static_key, + default_keyset: $wire.default_keyset, + root_key_ttl_ms: $wire.root_key_ttl_ms, + } + }; +} +// The deterministic build reads the same keys beside its seed. +#[cfg(feature = "deterministic-kms")] +pub(crate) use vendor_wire; /// Everything `se_cipher_init` needs to build the cipher. pub struct CipherConfig { - pub client_key: ClientKey, + /// The ZeroKMS client key. `None` for a client with a key vendor + /// backend and no ZeroKMS: the config then has no `client_id`, + /// `client_key` or `zerokms_url`. + pub client_key: Option, pub endpoint: Option, pub keyset_cache_size: Option, + /// The key vendor backend, when one is configured. + pub vendor: Option, } /// Parse the config frame a host sent. See the [module docs](self). @@ -53,14 +116,45 @@ pub fn parse_config(bytes: &[u8]) -> Result { /// [`parse_config`], with the refusal's message. fn read(bytes: &[u8]) -> Result { let wire: Wire<'_> = from_slice(bytes).map_err(|e| format!("the config: {e}"))?; - // `from_encoded_v1` is lenient by design: hex in either case *or* the - // base64 `secretkey.json` holds, matching every native loader. - let client_key = ClientKey::from_encoded_v1(wire.client_id.0, wire.client_key) - .map_err(|_| "the config: client_key: a client key that is not valid".to_owned())?; + let vendor = vendor_wire!(wire) + .config() + .map_err(|e| format!("the config: {e}"))?; + // A key vendor backend is an alternative to ZeroKMS: with one, and no + // ZeroKMS key at all, the client has no ZeroKMS. Any ZeroKMS key given + // makes the others required, as before. + let vendor_only = vendor.is_some() + && wire.client_id.is_none() + && wire.client_key.is_none() + && wire.zerokms_url.is_none(); + let client_key = if vendor_only { + None + } else { + let client_id = wire + .client_id + .ok_or_else(|| "the config: missing field `client_id`".to_owned())?; + let encoded = wire + .client_key + .ok_or_else(|| "the config: missing field `client_key`".to_owned())?; + // `from_encoded_v1` is lenient by design: hex in either case *or* + // the base64 `secretkey.json` holds, matching every native loader. + Some( + ClientKey::from_encoded_v1(client_id.0, encoded) + .map_err(|_| "the config: client_key: a client key that is not valid".to_owned())?, + ) + }; + // The default keyset of a client with ZeroKMS is the one a ZeroKMS + // administrator set; only a vendor-only client names its own. + if !vendor_only && vendor.as_ref().is_some_and(|v| v.default_keyset.is_some()) { + return Err( + "the config: default_keyset: a key that applies only to a client with no ZeroKMS" + .to_owned(), + ); + } Ok(CipherConfig { client_key, endpoint: wire.zerokms_url.map(|url| url.0), keyset_cache_size: wire.keyset_cache_size.map(|size| size.0), + vendor, }) } @@ -72,12 +166,40 @@ fn read(bytes: &[u8]) -> Result { #[derive(Deserialize)] #[serde(deny_unknown_fields)] struct Wire<'a> { - client_id: ClientId, - client_key: &'a str, + #[serde(default, deserialize_with = "present")] + client_id: Option, + #[serde(default, borrow, deserialize_with = "present")] + client_key: Option<&'a str>, #[serde(default, deserialize_with = "present")] zerokms_url: Option, #[serde(default, deserialize_with = "present")] keyset_cache_size: Option, + // The key vendor backend keys, read into a `VendorWire`. Not a + // flattened field: serde does not deny unknown keys through a flatten. + #[serde(default, borrow, deserialize_with = "present")] + key_vendor: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + keyset_records: Option>, + #[serde(default, borrow, deserialize_with = "present")] + aws_region: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + aws_endpoint: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + vault_address: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + vault_namespace: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + vault_mount: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + gcp_endpoint: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + static_key_ref: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + static_key: Option<&'a [u8]>, + #[serde(default, borrow, deserialize_with = "present")] + default_keyset: Option>, + #[serde(default, borrow, deserialize_with = "present")] + root_key_ttl_ms: Option<&'a str>, } /// Absent is `None`. Present, a value must be a string: a null is refused. @@ -128,6 +250,168 @@ impl TryFrom<&str> for CacheSize { } } +/// The key vendor backend keys of a config, as a host writes them. Read +/// beside the ZeroKMS keys here, and beside the seed in the deterministic +/// build ([`crate::deterministic`]). +#[derive(Default)] +pub struct VendorWire<'a> { + pub(crate) key_vendor: Option<&'a str>, + pub(crate) keyset_records: Option>, + pub(crate) aws_region: Option<&'a str>, + pub(crate) aws_endpoint: Option<&'a str>, + pub(crate) vault_address: Option<&'a str>, + pub(crate) vault_namespace: Option<&'a str>, + pub(crate) vault_mount: Option<&'a str>, + pub(crate) gcp_endpoint: Option<&'a str>, + pub(crate) static_key_ref: Option<&'a str>, + pub(crate) static_key: Option<&'a [u8]>, + pub(crate) default_keyset: Option>, + pub(crate) root_key_ttl_ms: Option<&'a str>, +} + +/// `default_keyset` as a host writes it: `{"id": 16 bytes}` or +/// `{"name": string}`, one of the two. +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +pub struct KeysetSelectorWire<'a> { + #[serde(default, borrow, deserialize_with = "present")] + id: Option<&'a [u8]>, + #[serde(default, borrow, deserialize_with = "present")] + name: Option<&'a str>, +} + +impl VendorWire<'_> { + /// The key vendor backend these keys configure, or `None` when none is + /// given. A refusal names the key at fault and never its value. + pub fn config(self) -> Result, String> { + let given: Vec<&'static str> = [ + ("key_vendor", self.key_vendor.is_some()), + ("keyset_records", self.keyset_records.is_some()), + ("aws_region", self.aws_region.is_some()), + ("aws_endpoint", self.aws_endpoint.is_some()), + ("vault_address", self.vault_address.is_some()), + ("vault_namespace", self.vault_namespace.is_some()), + ("vault_mount", self.vault_mount.is_some()), + ("gcp_endpoint", self.gcp_endpoint.is_some()), + ("static_key_ref", self.static_key_ref.is_some()), + ("static_key", self.static_key.is_some()), + ("default_keyset", self.default_keyset.is_some()), + ("root_key_ttl_ms", self.root_key_ttl_ms.is_some()), + ] + .into_iter() + .filter_map(|(key, present)| present.then_some(key)) + .collect(); + if given.is_empty() { + return Ok(None); + } + let missing = |key: &str| format!("missing field `{key}`"); + let invalid = |key: &str| format!("{key}: a value its type does not accept"); + let name = self.key_vendor.ok_or_else(|| missing("key_vendor"))?; + let vendor = KeyVendor::ALL + .into_iter() + .find(|v| v.name() == name) + .ok_or_else(|| invalid("key_vendor"))?; + let records = self + .keyset_records + .ok_or_else(|| missing("keyset_records"))?; + if records.is_empty() { + return Err(invalid("keyset_records")); + } + let required = |key: &'static str, value: Option<&str>| { + let value = value.ok_or_else(|| missing(key))?; + if value.is_empty() { + return Err(invalid(key)); + } + Ok(value.to_owned()) + }; + let (settings, used): (VendorSettings, &[&str]) = match vendor { + KeyVendor::Aws => ( + VendorSettings::Aws { + region: required("aws_region", self.aws_region)?, + endpoint: self.aws_endpoint.map(str::to_owned), + }, + &["aws_region", "aws_endpoint"], + ), + KeyVendor::Vault => ( + VendorSettings::Vault { + address: required("vault_address", self.vault_address)?, + namespace: self.vault_namespace.map(str::to_owned), + mount: self.vault_mount.map(str::to_owned), + }, + &["vault_address", "vault_namespace", "vault_mount"], + ), + KeyVendor::Gcp => ( + VendorSettings::Gcp { + endpoint: self.gcp_endpoint.map(str::to_owned), + }, + &["gcp_endpoint"], + ), + KeyVendor::Azure => (VendorSettings::Azure, &[]), + KeyVendor::Static => { + let key = self.static_key.ok_or_else(|| missing("static_key"))?; + let mut bytes: [u8; 32] = key.try_into().map_err(|_| invalid("static_key"))?; + // `KeyBytes` wipes its own copy; the stack copy is wiped + // here. The config buffer is the ABI layer's to wipe. + let key = KeyBytes::new(bytes); + bytes.zeroize(); + ( + VendorSettings::Static { + key_ref: required("static_key_ref", self.static_key_ref)?, + key, + }, + &["static_key_ref", "static_key"], + ) + } + // `KeyVendor` is `#[non_exhaustive]`: a vendor this guest has no + // client for is not one it can configure. + _ => return Err(invalid("key_vendor")), + }; + let default_keyset = match self.default_keyset { + None => None, + Some(KeysetSelectorWire { + id: Some(id), + name: None, + }) => Some(DefaultKeyset::Id( + Uuid::from_slice(id).map_err(|_| invalid("default_keyset"))?, + )), + Some(KeysetSelectorWire { + id: None, + name: Some(name), + }) if !name.is_empty() => Some(DefaultKeyset::Name(name.to_owned())), + Some(_) => return Err(invalid("default_keyset")), + }; + let root_key_ttl = self + .root_key_ttl_ms + .map(|ms| { + ms.parse::() + .map(Duration::from_millis) + .map_err(|_| invalid("root_key_ttl_ms")) + }) + .transpose()?; + // Whatever else was given is for another backend. + const SHARED: [&str; 4] = [ + "key_vendor", + "keyset_records", + "default_keyset", + "root_key_ttl_ms", + ]; + if let Some(key) = given + .iter() + .find(|key| !SHARED.contains(key) && !used.contains(key)) + { + return Err(format!( + "{key}: a key that does not apply to its key_vendor" + )); + } + Ok(Some(VendorConfig { + settings, + records: records.into_iter().map(str::to_owned).collect(), + default_keyset, + root_key_ttl, + })) + } +} + #[cfg(test)] mod tests { use super::*; @@ -166,7 +450,7 @@ mod tests { ("client_key", s(&hex)), ]) .expect("minimal config parses"); - assert_eq!(cfg.client_key.key_id, id); + assert_eq!(cfg.client_key.as_ref().expect("zerokms").key_id, id); assert!(cfg.endpoint.is_none()); assert!(cfg.keyset_cache_size.is_none()); } @@ -251,9 +535,17 @@ mod tests { ("client_key", s(&encoded)), ]) .unwrap_or_else(|e| panic!("{label} must parse, got {e}")); - assert_eq!(cfg.client_key.key_id, id, "{label}"); assert_eq!( - cfg.client_key.to_hex_v1().expect("re-encode"), + cfg.client_key.as_ref().expect("zerokms").key_id, + id, + "{label}" + ); + assert_eq!( + cfg.client_key + .as_ref() + .expect("zerokms") + .to_hex_v1() + .expect("re-encode"), hex, "{label} must recover the same keyset" ); @@ -352,4 +644,237 @@ mod tests { ); assert!(message.starts_with("the config: "), "{message}"); } + + fn with_vendor(extra: Vec<(&'static str, V)>) -> Result { + let (id, hex) = client_key_hex(); + let mut entries = vec![("client_id", s(&id.to_string())), ("client_key", s(&hex))]; + entries.extend(extra); + config(entries) + } + + fn records() -> V { + V::Arr(vec![s("stack-keyset:AQ")]) + } + + #[test] + fn no_vendor_key_is_no_vendor() { + let cfg = with_vendor(vec![]).expect("config parses"); + assert!(cfg.vendor.is_none()); + } + + #[test] + fn parses_each_vendor() { + let aws = with_vendor(vec![ + ("key_vendor", s("aws-kms")), + ("keyset_records", records()), + ("aws_region", s("us-east-1")), + ("aws_endpoint", s("http://localhost:4566")), + ]) + .unwrap() + .vendor + .unwrap(); + assert!(matches!( + aws.settings, + VendorSettings::Aws { ref region, endpoint: Some(_) } if region == "us-east-1" + )); + assert_eq!(aws.records, vec!["stack-keyset:AQ".to_owned()]); + + let vault = with_vendor(vec![ + ("key_vendor", s("vault-transit")), + ("keyset_records", records()), + ("vault_address", s("https://vault.example.com")), + ("vault_mount", s("kms")), + ]) + .unwrap() + .vendor + .unwrap(); + assert!(matches!( + vault.settings, + VendorSettings::Vault { + namespace: None, + mount: Some(_), + .. + } + )); + + for (name, expected) in [ + ("gcp-kms", KeyVendor::Gcp), + ("azure-key-vault", KeyVendor::Azure), + ] { + let vendor = with_vendor(vec![("key_vendor", s(name)), ("keyset_records", records())]) + .unwrap() + .vendor + .unwrap(); + assert_eq!(vendor.settings.vendor(), expected); + } + + let static_key = with_vendor(vec![ + ("key_vendor", s("static")), + ("keyset_records", records()), + ("static_key_ref", s("local")), + ("static_key", V::Bytes(vec![7u8; 32])), + ]) + .unwrap() + .vendor + .unwrap(); + assert_eq!(static_key.settings.vendor(), KeyVendor::Static); + // The static key never prints. + assert!(!format!("{static_key:?}").contains('7')); + } + + /// Every refusal names the key at fault, never a value. + #[test] + fn refuses_bad_vendor_configs() { + let cases: Vec<(Vec<(&'static str, V)>, &str)> = vec![ + ( + vec![("keyset_records", records())], + "the config: missing field `key_vendor`", + ), + ( + vec![("key_vendor", s("zerokms")), ("keyset_records", records())], + "the config: key_vendor: a value its type does not accept", + ), + ( + vec![("key_vendor", s("azure-key-vault"))], + "the config: missing field `keyset_records`", + ), + ( + vec![ + ("key_vendor", s("azure-key-vault")), + ("keyset_records", V::Arr(vec![])), + ], + "the config: keyset_records: a value its type does not accept", + ), + ( + vec![ + ("key_vendor", s("azure-key-vault")), + ("keyset_records", s("stack-keyset:AQ")), + ], + "the config: keyset_records: ", + ), + ( + vec![("key_vendor", s("aws-kms")), ("keyset_records", records())], + "the config: missing field `aws_region`", + ), + ( + vec![ + ("key_vendor", s("aws-kms")), + ("keyset_records", records()), + ("aws_region", s("")), + ], + "the config: aws_region: a value its type does not accept", + ), + ( + vec![ + ("key_vendor", s("aws-kms")), + ("keyset_records", records()), + ("aws_region", s("us-east-1")), + ("vault_address", s("https://vault.example.com")), + ], + "the config: vault_address: a key that does not apply to its key_vendor", + ), + ( + vec![ + ("key_vendor", s("static")), + ("keyset_records", records()), + ("static_key_ref", s("local")), + ("static_key", s("not bytes")), + ], + "the config: static_key: ", + ), + ( + vec![ + ("key_vendor", s("static")), + ("keyset_records", records()), + ("static_key_ref", s("local")), + ("static_key", V::Bytes(vec![7u8; 31])), + ], + "the config: static_key: a value its type does not accept", + ), + ]; + let (_, hex) = client_key_hex(); + for (entries, expected) in cases { + let message = with_vendor(entries).err().expect("refused"); + assert!( + message.starts_with(expected), + "{message:?} is not {expected:?}" + ); + for secret in [hex.as_str(), "us-east-1", "vault.example.com", "not bytes"] { + assert!(!message.contains(secret), "{message:?} echoes {secret:?}"); + } + } + } + + #[test] + fn a_vendor_and_no_zerokms_key_is_a_vendor_only_client() { + let cfg = config(vec![ + ("key_vendor", s("gcp-kms")), + ("keyset_records", records()), + ("default_keyset", obj(vec![("name", s("payments"))])), + ("root_key_ttl_ms", s("60000")), + ]) + .expect("config parses"); + assert!(cfg.client_key.is_none()); + let vendor = cfg.vendor.expect("vendor"); + assert_eq!( + vendor.default_keyset, + Some(DefaultKeyset::Name("payments".into())) + ); + assert_eq!(vendor.root_key_ttl, Some(Duration::from_secs(60))); + + // One ZeroKMS key makes the others required. + let message = config(vec![ + ("key_vendor", s("gcp-kms")), + ("keyset_records", records()), + ("zerokms_url", s("https://zerokms.example.com")), + ]) + .err() + .expect("refused"); + assert_eq!(message, "the config: missing field `client_id`"); + // No vendor and no ZeroKMS key is still a missing client id. + assert_eq!( + config(vec![]).err().as_deref(), + Some("the config: missing field `client_id`") + ); + } + + #[test] + fn a_default_keyset_needs_a_client_with_no_zerokms() { + let message = with_vendor(vec![ + ("key_vendor", s("gcp-kms")), + ("keyset_records", records()), + ("default_keyset", obj(vec![("id", V::Bytes(vec![1; 16]))])), + ]) + .err() + .expect("refused"); + assert_eq!( + message, + "the config: default_keyset: a key that applies only to a client with no ZeroKMS" + ); + + for bad in [ + s("payments"), + obj(vec![]), + obj(vec![("name", s(""))]), + obj(vec![("id", V::Bytes(vec![1; 15]))]), + obj(vec![("name", s("a")), ("id", V::Bytes(vec![1; 16]))]), + ] { + assert!(config(vec![ + ("key_vendor", s("gcp-kms")), + ("keyset_records", records()), + ("default_keyset", bad), + ]) + .is_err()); + } + assert_eq!( + config(vec![ + ("key_vendor", s("gcp-kms")), + ("keyset_records", records()), + ("root_key_ttl_ms", s("-1")), + ]) + .err() + .as_deref(), + Some("the config: root_key_ttl_ms: a value its type does not accept") + ); + } } diff --git a/languages/golang/encrypt/guest/src/deterministic.rs b/languages/golang/encrypt/guest/src/deterministic.rs index 15567525a5..73d2770ed7 100644 --- a/languages/golang/encrypt/guest/src/deterministic.rs +++ b/languages/golang/encrypt/guest/src/deterministic.rs @@ -17,8 +17,9 @@ use stack_encrypt::dynamic::frame::serde::from_slice; use stack_guest_abi::last_error::malformed; use zeroize::Zeroize; -/// The `deterministic-kms` build's whole config: the source's 32-byte seed, -/// a frame holding bytes and nothing else. Wiped when it drops. +/// The `deterministic-kms` build's ZeroKMS stand-in config: the source's +/// 32-byte seed, a frame holding bytes and nothing else. Wiped when it +/// drops. #[derive(Deserialize)] #[serde(try_from = "&[u8]")] pub struct Seed([u8; 32]); @@ -40,13 +41,77 @@ impl Drop for Seed { } } -impl Seed { - /// Read the seed frame the host sent in place of a config. +/// The `deterministic-kms` build's whole config: the seed, and a key vendor +/// backend when one is configured. The host sends the bare seed, or, for +/// the hermetic Go tests of the vendor path, an object of the seed and the +/// key vendor backend keys ([`crate::config`] has them): +/// `{"seed": bytes, "key_vendor": .., ..}`. +pub struct SeedConfig { + pub seed: Seed, + pub vendor: Option, +} + +impl SeedConfig { + /// Read the config the host sent. pub fn parse(bytes: &[u8]) -> Result { - from_slice(bytes) - .map_err(|_| malformed("the deterministic build's config is a 32-byte seed")) + if let Ok(seed) = from_slice::(bytes) { + return Ok(Self { seed, vendor: None }); + } + let wire: SeededWire<'_> = from_slice(bytes).map_err(|_| { + malformed( + "the deterministic build's config is a 32-byte seed, or an object of a seed and the key vendor backend keys", + ) + })?; + let vendor = crate::config::vendor_wire!(wire) + .config() + .map_err(|e| malformed(format!("the config: {e}")))?; + Ok(Self { + seed: wire.seed, + vendor, + }) } +} + +/// The object form of the deterministic config. +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct SeededWire<'a> { + seed: Seed, + #[serde(default, borrow, deserialize_with = "present")] + key_vendor: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + keyset_records: Option>, + #[serde(default, borrow, deserialize_with = "present")] + aws_region: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + aws_endpoint: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + vault_address: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + vault_namespace: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + vault_mount: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + gcp_endpoint: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + static_key_ref: Option<&'a str>, + #[serde(default, borrow, deserialize_with = "present")] + static_key: Option<&'a [u8]>, + #[serde(default, borrow, deserialize_with = "present")] + default_keyset: Option>, + #[serde(default, borrow, deserialize_with = "present")] + root_key_ttl_ms: Option<&'a str>, +} +/// Absent is `None`. Present, a value must be of its type: a null is +/// refused. +fn present<'de, D: serde::Deserializer<'de>, T: Deserialize<'de>>( + d: D, +) -> Result, D::Error> { + T::deserialize(d).map(Some) +} + +impl Seed { /// The key source this seed drives. pub fn source(&self) -> DeterministicSource { DeterministicSource::new(self.0) @@ -55,8 +120,8 @@ impl Seed { #[cfg(test)] mod tests { - use super::Seed; - use crate::test_frame::{frame, V}; + use super::{Seed, SeedConfig}; + use crate::test_frame::{frame, obj, s, V}; use stack_encrypt::dynamic::frame::serde::from_slice; /// A seed is 32 bytes, and only bytes: 32 characters of text are refused. @@ -66,4 +131,30 @@ mod tests { assert!(from_slice::(&frame(&V::Bytes(vec![7; 31]))).is_err()); assert!(from_slice::(&frame(&V::Str("x".repeat(32)))).is_err()); } + + /// The object form adds a key vendor backend beside the seed; a bare + /// seed has none. + #[test] + fn a_seed_object_carries_a_key_vendor_backend() { + let bare = SeedConfig::parse(&frame(&V::Bytes(vec![7; 32]))).unwrap(); + assert!(bare.vendor.is_none()); + let with = SeedConfig::parse(&frame(&obj(vec![ + ("seed", V::Bytes(vec![7; 32])), + ("key_vendor", s("static")), + ("keyset_records", V::Arr(vec![s("stack-keyset:AQ")])), + ("static_key_ref", s("local")), + ("static_key", V::Bytes(vec![9; 32])), + ]))) + .unwrap(); + assert_eq!( + with.vendor.map(|v| v.settings.vendor()), + Some(stack_kms::KeyVendor::Static) + ); + assert!(SeedConfig::parse(&frame(&obj(vec![("seed", V::Bytes(vec![7; 31]))]))).is_err()); + assert!(SeedConfig::parse(&frame(&obj(vec![ + ("seed", V::Bytes(vec![7; 32])), + ("other", s("x")), + ]))) + .is_err()); + } } diff --git a/languages/golang/encrypt/guest/src/host.rs b/languages/golang/encrypt/guest/src/host.rs index 605bf7d42c..a29869c0e7 100644 --- a/languages/golang/encrypt/guest/src/host.rs +++ b/languages/golang/encrypt/guest/src/host.rs @@ -1,7 +1,10 @@ -//! The guest's two connections to its host: [`WasiHostConnection`] (a +//! The guest's connections to its host: [`WasiHostConnection`] (a //! [`ZeroKMSConnection`] whose requests the host performs, through the step -//! loop) and [`HostTokenStrategy`] (an [`AuthStrategy`] that asks the host -//! for the bearer token). wasm32-only — the token is an imported function. +//! loop), [`HostTokenStrategy`] (an [`AuthStrategy`] that asks the host for +//! the bearer token) and [`StepCredentialImport`] (a key vendor backend's +//! short-lived credentials, asked for through the step loop, see +//! [`crate::vendor`]). wasm32-only — the token is an imported function, and +//! a step's future is not `Send`. //! //! # Requests //! @@ -10,7 +13,10 @@ //! returns to the host with it, paused; the host performs it and resumes the //! operation with the response ([`crate::io`] has the wire form). So a host //! whose HTTP client cannot block — `fetch` on the edge — can run the guest, -//! and the guest is never running while a request is in flight. +//! and the guest is never running while a request is in flight. A key +//! vendor backend's requests, its credentials and its backoff waits pause +//! the operation the same way (`stack_guest_abi::vendor_transport`, +//! [`StepCredentialImport`]). //! //! # Import contract (module `cipherstash_transport`) //! @@ -31,6 +37,17 @@ //! bytes. Key material derived from a response never crosses back — the //! derivation runs inside the guest ([`stack_kms`]'s client, unmodified). //! +//! The same holds for a key vendor backend. A vendor request crosses as +//! the signed HTTP request the vendor would receive over TLS: a wrapped key +//! out, and, in the response, a key the vendor unwrapped. That response is +//! read only inside the guest, and the host wipes its own copy once it is +//! in guest memory (as for a ZeroKMS response). Root keys, index keys and +//! data keys are derived or unwrapped in the guest and never cross back to +//! the host. What crosses *in* for a credential step is a short-lived +//! credential (temporary AWS credentials, an access token, a Vault token), +//! never a long-lived signing key, which the guest uses to sign and then +//! drops. +//! //! Request bodies and response bodies can carry key-material contexts and //! wrapped keys, so both are wiped on drop here; the buffers the host wrote //! are reclaimed via the registry (which the ABI's `se_dealloc` also wipes). @@ -42,12 +59,14 @@ use stack_auth::{AuthError, AuthStrategy, CustomError, SecretToken, ServiceToken use stack_guest_abi::buffers; use stack_guest_abi::headers::header_value; use stack_guest_abi::step; +use stack_kms::{BackendErrorKind, KeyVendor}; use stack_kms::{BaseUrlUnresolved, ZeroKMSConnection, ZeroKMSConnectionInit, ZeroKmsEndpoint}; use zeroize::Zeroizing; use zerokms_protocol::{ViturRequest, ViturRequestError}; use crate::headers::request_headers; use crate::response::map_response; +use crate::vendor::{CredentialError, CredentialImport}; #[link(wasm_import_module = "cipherstash_transport")] extern "C" { @@ -203,3 +222,37 @@ impl AuthStrategy for &HostTokenStrategy { Ok(ServiceToken::new(SecretToken::new(text))) } } + +/// A key vendor backend's [`CredentialImport`] through the step loop: the +/// operation pauses with a credential request for the vendor and scope, and +/// the host answers it from its credential source between steps, outside +/// the client's lock, where a source may wait on the network. The guest asks +/// before each vendor request and keeps nothing, so the host decides when a +/// credential is refreshed. +pub struct StepCredentialImport; + +impl CredentialImport for StepCredentialImport { + async fn credential( + &self, + vendor: KeyVendor, + scope: &str, + ) -> Result>, CredentialError> { + let failed = |message: String| CredentialError { + kind: BackendErrorKind::Unavailable, + message, + }; + let response = step::request(step::Request::credential(vendor.byte(), scope)) + .await + .map_err(|e| failed(e.to_string()))?; + if response.status != 0 { + // The body is the host's error text: the credential source's + // reason, never a credential. + return Err(failed(format!( + "the host has no {} credential: {}", + vendor.name(), + String::from_utf8_lossy(&response.body) + ))); + } + Ok(response.body) + } +} diff --git a/languages/golang/encrypt/guest/src/io.rs b/languages/golang/encrypt/guest/src/io.rs index dad3a6e63d..520be2cf36 100644 --- a/languages/golang/encrypt/guest/src/io.rs +++ b/languages/golang/encrypt/guest/src/io.rs @@ -12,6 +12,15 @@ //! "headers": , "body": }, ... ] } //! ``` //! +//! A request is one of three objects. Besides the HTTP request above, a key +//! vendor backend's credential (`vendor` is its `KeyVendor` byte, `scope` +//! the OAuth scope or empty) and a backoff wait: +//! +//! ```text +//! { "credential": { "vendor": , "scope": } } +//! { "wait_ms": } +//! ``` +//! //! The host performs every request and calls `se_resume` with the handle and //! their responses, in the same order: //! @@ -22,7 +31,10 @@ //! `headers` is the `name: value` line format of //! [`stack_guest_abi::headers`] both ways. A `status` is the HTTP status, or //! negative for a request the host could not perform, with its error text as -//! the body: the contract `transport_send` had, field for field. A step +//! the body: the contract `transport_send` had, field for field. A +//! credential's response has status `0` and the credential, as a frame, in +//! its body ([`crate::vendor`] has its keys), or a negative status and the +//! error text; a wait's has status `0` and an empty body. A step //! carries the bearer token and a request body that can hold key-material //! contexts, and a response can carry wrapped keys, so the guest wipes its //! copies, and the host wipes the buffers with `se_dealloc`. @@ -46,23 +58,56 @@ pub fn encode_step(handle: u64, requests: &[Request]) -> Result, u32> { requests: Vec>, } #[derive(Serialize)] - struct RequestWire<'a> { - method: &'a str, - url: &'a str, - headers: Bytes<'a>, - body: Bytes<'a>, + #[serde(untagged)] + enum RequestWire<'a> { + Http { + method: &'a str, + url: &'a str, + headers: Bytes<'a>, + body: Bytes<'a>, + }, + Credential { + credential: CredentialWire<'a>, + }, + Wait { + wait_ms: u64, + }, + } + #[derive(Serialize)] + struct CredentialWire<'a> { + vendor: u32, + scope: &'a str, } let wire = Wire { handle, requests: requests .iter() - .map(|request| RequestWire { - method: request.method, - url: &request.url, - headers: Bytes(&request.headers), - body: Bytes(&request.body), + .map(|request| match request { + Request::Http(request) => Ok(RequestWire::Http { + method: request.method, + url: &request.url, + headers: Bytes(&request.headers), + body: Bytes(&request.body), + }), + Request::Credential { vendor, scope } => Ok(RequestWire::Credential { + credential: CredentialWire { + vendor: u32::from(*vendor), + scope, + }, + }), + Request::Wait(duration) => Ok(RequestWire::Wait { + // A wait longer than u64 milliseconds is not one a + // vendor client asks for; it is capped, not refused. + wait_ms: u64::try_from(duration.as_millis()).unwrap_or(u64::MAX), + }), + // `Request` is non-exhaustive: a kind this guest cannot + // encode is a bug here, refused rather than dropped. + #[allow(unreachable_patterns)] + _ => Err(stack_guest_abi::last_error::internal( + "a step request this guest cannot encode", + )), }) - .collect(), + .collect::>()?, }; // A `Buffer`, which wipes on drop and as it grows. let frame = to_vec(&wire).map_err(|_| STATUS_ENCODING)?; @@ -149,6 +194,34 @@ mod tests { assert_eq!(bytes.capacity(), bytes.len()); } + /// A credential and a wait cross as objects of their own, beside the + /// HTTP request's. + #[test] + fn a_credential_and_a_wait_are_step_requests() { + let requests = [ + Request::credential(3, "https://vault.azure.net/.default"), + Request::wait(std::time::Duration::from_millis(250)), + ]; + let bytes = encode_step(9, &requests).unwrap(); + let expected = frame(&obj(vec![ + ("handle", V::U64(9)), + ( + "requests", + V::Arr(vec![ + obj(vec![( + "credential", + obj(vec![ + ("vendor", V::U32(3)), + ("scope", V::Str("https://vault.azure.net/.default".into())), + ]), + )]), + obj(vec![("wait_ms", V::U64(250))]), + ]), + ), + ])); + assert_eq!(bytes, expected); + } + #[test] fn responses_read_in_order() { let bytes = frame(&V::Arr(vec![ diff --git a/languages/golang/encrypt/guest/src/lib.rs b/languages/golang/encrypt/guest/src/lib.rs index 2d3c7acd32..bdd52d6223 100644 --- a/languages/golang/encrypt/guest/src/lib.rs +++ b/languages/golang/encrypt/guest/src/lib.rs @@ -38,11 +38,19 @@ //! //! Control stays in Rust: request assembly, key derivation, batching, and //! AAD/PRF context binding run unmodified inside the guest. The host -//! provides exactly two imports (HTTP transport and the bearer token — see -//! [`host`]); what crosses the boundary per call is a value tree in, a -//! ciphertext/record tree out, and — inside the call — the same bytes that -//! would cross TLS anyway. The client key enters guest memory once at -//! `se_cipher_init`; derived data keys and index keys never leave. +//! provides exactly one import (the ZeroKMS bearer token — see [`host`]); +//! every HTTP request, a key vendor backend's short-lived credential and a +//! retry's backoff go to the host through the step loop instead, so the +//! guest never waits on the network. What crosses the boundary per call is +//! a value tree in, a ciphertext/record tree out, and — between its steps — +//! the same bytes that would cross TLS anyway. The client key (and a static key backend's key) enters guest +//! memory once at `se_cipher_init`; root keys, derived data keys and index +//! keys never leave. +//! +//! A client holds ZeroKMS and at most one key vendor backend (AWS KMS, +//! Google Cloud KMS, Azure Key Vault, Vault Transit or a static key), routed +//! by keyset ([`vendor`]). The vendor clients sign their requests here, with +//! credentials the host supplies, through the step loop, per request. //! //! The exports are the record path (`se_encrypt_record`, `se_decrypt_record`), //! the term of one index under a context (`se_term`, which the Go SDK asks @@ -69,7 +77,7 @@ //! Split into: //! //! - [`ops`], [`options`], [`config`], [`response`], [`headers`], -//! [`status`], [`targets`] — everything that is pure logic over +//! [`status`], [`targets`], [`vendor`] — everything that is pure logic over //! `StackCipher` / `KeysetCipher` / bytes. Compiles and unit-tests //! on the native host target (`cargo test` here, no wasm toolchain //! needed) against `stack_kms::FakeDataKeySource`. @@ -103,6 +111,8 @@ mod support; pub mod targets; #[cfg(test)] mod test_frame; +pub mod vendor; +pub mod zerokms; // The ABI's packed u64 results embed 32-bit pointers, its bounds checks // read the wasm linear-memory size, and `host` calls imported functions — diff --git a/languages/golang/encrypt/guest/src/status.rs b/languages/golang/encrypt/guest/src/status.rs index 7cfbdabd07..a3210bff48 100644 --- a/languages/golang/encrypt/guest/src/status.rs +++ b/languages/golang/encrypt/guest/src/status.rs @@ -66,7 +66,7 @@ use zerokms_protocol::{ViturRequestError, ViturRequestErrorKind}; pub use stack_guest_abi::status::{ STATUS_AUTH, STATUS_CONTEXT_MISMATCH, STATUS_ENCODING, STATUS_FOREIGN_KEYSET, STATUS_INTERNAL, STATUS_KMS_CONFLICT, STATUS_KMS_FORBIDDEN, STATUS_KMS_NOT_FOUND, STATUS_KMS_OTHER, - STATUS_KMS_TRANSPORT, STATUS_KMS_UNAUTHORIZED, STATUS_STATE, STATUS_TERM, + STATUS_KMS_TRANSPORT, STATUS_KMS_UNAUTHORIZED, STATUS_NO_ZEROKMS, STATUS_STATE, STATUS_TERM, }; /// Record a sealing/opening error as the last error @@ -109,9 +109,44 @@ pub fn describe(cause: &(dyn std::error::Error + 'static)) -> Option { { return Some(cause.to_string()); } + if describes_itself(cause) { + return Some(cause.to_string()); + } last_error::describe_std(cause) } +/// The key vendor backend errors whose text the guest vouches for: each +/// vendor crate builds its messages from the operation, the HTTP status, the +/// vendor's error code and message, and fixed text, and never puts a key, +/// a secret, a token or a value in one (CIP-5307 to CIP-5309). A host +/// credential failure is described by the guest itself ([`crate::vendor`]). +fn describes_itself(cause: &(dyn std::error::Error + 'static)) -> bool { + cause.is::() + || cause.is::() + || cause.is::() + || cause.is::() + || cause.is::() + || cause.is::() + || cause.is::() + || cause.is::() + || cause.is::() + || cause.is::() + || cause.is::() +} + +/// A keyset record or the key vendor backend refused at `se_cipher_init`: +/// record the error and return its status. A record that does not parse, +/// names another vendor or a mode the vendor does not offer, or repeats an +/// id or a name is the host's input (`STATUS_ENCODING`); a key failure +/// keeps its own status. +pub fn fail_record(error: &stack_kms_vendor::RecordError) -> u32 { + let status = match error { + stack_kms_vendor::RecordError::Kms(kms) => status_for_kms(kms), + _ => STATUS_ENCODING, + }; + record(error, status) +} + /// Map a sealing/opening error onto the ABI status word. /// /// Total over [`stack_encrypt::Error`] (which is `#[non_exhaustive]`, so the @@ -155,6 +190,11 @@ pub fn status_for_error(error: &stack_encrypt::Error) -> u32 { // children asked for as a scalar, or an altered entry. The client // cannot tell which, so neither STATUS_AUTH nor a claim of either. stack_encrypt::Error::JsonNotALeaf => STATUS_ENCODING, + // A leaf that names another key backend than this guest's (a v2 + // leaf of a key vendor backend, while this guest holds ZeroKMS) is + // refused before any key call. It is the caller's input, as a leaf + // this guest cannot decode is: no retry with other options opens it. + stack_encrypt::Error::ForeignBackend { .. } => STATUS_ENCODING, _ => STATUS_INTERNAL, } } @@ -240,6 +280,29 @@ fn status_for_kms(error: &stack_kms::Error) -> u32 { STATUS_KMS_TRANSPORT } stack_kms::Error::Unexpected(_) => STATUS_KMS_OTHER, + // A vendor-only client asked for a ZeroKMS keyset (CIP-5311). + e if crate::zerokms::is_no_zerokms(e) => STATUS_NO_ZEROKMS, + stack_kms::Error::Backend { kind, .. } => status_for_backend(*kind), + // A request for a lock context, a decryption policy or an + // unverified context against a key vendor backend: the caller's + // input, refused before any key call. + stack_kms::Error::ZeroKmsOnly { .. } => STATUS_ENCODING, + } +} + +/// A key vendor backend's failure as a status. `BackendErrorKind` is +/// `#[non_exhaustive]`, so a kind added upstream falls to +/// [`STATUS_KMS_OTHER`] until it is classified here. +fn status_for_backend(kind: stack_kms::BackendErrorKind) -> u32 { + use stack_kms::BackendErrorKind as Kind; + match kind { + Kind::NotFound => STATUS_KMS_NOT_FOUND, + Kind::AccessDenied => STATUS_KMS_FORBIDDEN, + Kind::Throttled | Kind::Unavailable => STATUS_KMS_TRANSPORT, + // The vendor could not unwrap the key: damaged, or wrapped by + // another key. Like an AEAD failure, the stored bytes do not open. + Kind::InvalidCiphertext => STATUS_AUTH, + _ => STATUS_KMS_OTHER, } } @@ -315,6 +378,30 @@ mod tests { assert_eq!(status_for_error(&load), STATUS_KMS_UNAUTHORIZED); } + #[test] + fn a_key_vendor_backend_failure_maps_by_its_kind() { + use stack_kms::BackendErrorKind as Kind; + let status = |kind| { + status_for_error(&stack_encrypt::Error::Kms(stack_kms::Error::Backend { + vendor: "test-vendor", + kind, + source: Box::new(std::io::Error::other("detail")), + })) + }; + assert_eq!(status(Kind::NotFound), STATUS_KMS_NOT_FOUND); + assert_eq!(status(Kind::AccessDenied), STATUS_KMS_FORBIDDEN); + assert_eq!(status(Kind::Throttled), STATUS_KMS_TRANSPORT); + assert_eq!(status(Kind::Unavailable), STATUS_KMS_TRANSPORT); + assert_eq!(status(Kind::InvalidCiphertext), STATUS_AUTH); + assert_eq!( + status_for_error(&stack_encrypt::Error::Kms(stack_kms::Error::ZeroKmsOnly { + vendor: "test-vendor", + feature: stack_kms::ZeroKmsFeature::DecryptionPolicy, + })), + STATUS_ENCODING + ); + } + #[test] fn aead_and_composition_errors_map_to_the_vitaminc_codes() { assert_eq!(status_for_error(&stack_encrypt::Error::Aead), STATUS_AUTH); @@ -433,6 +520,27 @@ mod tests { ); } + /// A leaf of another key backend is the caller's input, never a + /// foreign keyset (no keyset constraint would let it open). A seal under + /// a key vendor backend cannot happen here, so it is internal. + #[test] + fn a_foreign_backend_is_encoding_and_an_unsupported_seal_is_internal() { + use stack_encrypt::kms::{KeyBackend, KeyVendor}; + assert_eq!( + status_for_error(&stack_encrypt::Error::ForeignBackend { + expected: KeyBackend::ZeroKms, + found: KeyBackend::Vendor(KeyVendor::Aws), + }), + STATUS_ENCODING + ); + assert_eq!( + status_for_error(&stack_encrypt::Error::SealUnsupported { + backend: KeyBackend::Vendor(KeyVendor::Aws), + }), + STATUS_INTERNAL + ); + } + #[test] fn a_foreign_keyset_is_its_own_status_and_scope_bugs_are_internal() { let (a, b) = (uuid::Uuid::from_u128(1), uuid::Uuid::from_u128(2)); diff --git a/languages/golang/encrypt/guest/src/test_frame.rs b/languages/golang/encrypt/guest/src/test_frame.rs index 3f4693ae92..61b8bf07d8 100644 --- a/languages/golang/encrypt/guest/src/test_frame.rs +++ b/languages/golang/encrypt/guest/src/test_frame.rs @@ -13,6 +13,7 @@ pub(crate) enum V { I32(i32), U32(u32), U64(u64), + I64(i64), Bool(bool), Null, } @@ -57,6 +58,7 @@ pub(crate) fn frame(v: &V) -> Vec { V::I32(n) => scalar(to_vec(n)), V::U32(n) => scalar(to_vec(n)), V::U64(n) => scalar(to_vec(n)), + V::I64(n) => scalar(to_vec(n)), V::Bool(b) => scalar(to_vec(b)), V::Null => scalar(to_vec(&())), } diff --git a/languages/golang/encrypt/guest/src/vendor.rs b/languages/golang/encrypt/guest/src/vendor.rs new file mode 100644 index 0000000000..e057d71848 --- /dev/null +++ b/languages/golang/encrypt/guest/src/vendor.rs @@ -0,0 +1,966 @@ +//! The key vendor backend of this guest (ADR-0009, ADR-0011, CIP-5311). +//! +//! A client holds ZeroKMS and at most one key vendor backend. The host +//! names the backend and gives its keyset records in the `se_cipher_init` +//! config ([`crate::config`]); this module builds it: +//! +//! * [`GuestVendor`] is one enum over the four vendor clients and the static +//! key backend, so the cipher has one key source type whatever the host +//! chose: `KeyRouter`. With no backend configured it +//! is [`GuestVendor::None`] with no keyset records, and the router sends +//! every request to ZeroKMS, as before. +//! * The vendor clients send through the host's HTTP client +//! (`StepVendorTransport`: each request, and each backoff wait, pauses the +//! operation through the step loop) and sign each request here, in the +//! guest. +//! * The credentials they sign with come from the host through the step +//! loop too ([`CredentialImport`]; the guest's is +//! `crate::host::StepCredentialImport`): the operation pauses with a +//! credential request, and the host answers it between steps, outside the +//! client's lock. Short-lived credentials only, never a long-lived +//! signing key: AWS temporary credentials, a Google OAuth access token, an +//! Azure access token for the requested scope, a Vault client token. The +//! guest asks before each request and keeps nothing between requests, so +//! the host's own credential chain decides when to refresh. +//! +//! What this module owns is pure logic over those seams, and it is tested +//! natively with a fake transport and a fake import. The wasm32 ones are +//! `stack_guest_abi::vendor_transport`'s and `crate::host`'s. +//! +//! # The credential the host returns +//! +//! One frame, an object of string values plus an optional `expires_at` +//! (`i64`, Unix seconds), in the body of the credential step's response: +//! +//! | vendor | keys | +//! |-----------------|------| +//! | `aws-kms` | `access_key_id`, `secret_access_key`, optional `session_token` | +//! | `vault-transit` | `token` | +//! | `gcp-kms` | `token` | +//! | `azure-key-vault` | `token` (for the requested scope) | +//! +//! A credential whose `expires_at` has passed is refused before the +//! request is signed. Every secret is held in `Zeroizing` or the vendor +//! crate's own wiped type, and none is ever put in an error message. + +use std::num::NonZeroUsize; +use std::time::{Duration, SystemTime}; + +use serde::Deserialize; +use stack_encrypt::dynamic::frame::serde::from_slice; +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor, MaybeSend}; +use stack_kms_vendor::{ + KeyBytes, KeyVendorBackend, KeysetRecord, RecordError, StaticKeyVendor, UnwrapItem, + VendorKeySource, VendorTransport, WrapItem, +}; +use zeroize::Zeroizing; + +/// What the host configured: one key vendor backend and its settings. +pub enum VendorSettings { + /// AWS KMS in `region`; `endpoint` overrides the regional endpoint + /// (LocalStack, a VPC endpoint). + Aws { + region: String, + endpoint: Option, + }, + /// HashiCorp Vault Transit at `address`. + Vault { + address: String, + namespace: Option, + mount: Option, + }, + /// Google Cloud KMS; `endpoint` overrides the global endpoint. + Gcp { endpoint: Option }, + /// Azure Key Vault or Managed HSM: the key identifier in each keyset + /// record names the vault, so there is nothing to configure. + Azure, + /// The static key backend: one key, held in guest memory for the life + /// of the client. Derived data keys only. + Static { key_ref: String, key: KeyBytes }, +} + +impl VendorSettings { + /// The backend these settings build. + pub fn vendor(&self) -> KeyVendor { + match self { + Self::Aws { .. } => KeyVendor::Aws, + Self::Vault { .. } => KeyVendor::Vault, + Self::Gcp { .. } => KeyVendor::Gcp, + Self::Azure => KeyVendor::Azure, + Self::Static { .. } => KeyVendor::Static, + } + } +} + +impl std::fmt::Debug for VendorSettings { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + // The static key never prints. + f.debug_tuple("VendorSettings") + .field(&self.vendor().name()) + .finish() + } +} + +/// The key vendor backend part of the config: the backend, the keyset +/// record texts, and the adapter's settings. +#[derive(Debug)] +pub struct VendorConfig { + pub settings: VendorSettings, + pub records: Vec, + /// The default keyset of a client with no ZeroKMS. `None` is the only + /// record, when there is one. + pub default_keyset: Option, + /// How long a root key stays unwrapped. `None` is the adapter's default + /// (`DEFAULT_ROOT_KEY_TTL`). + pub root_key_ttl: Option, +} + +/// A keyset named by one of the keyset records, by its id or its name. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum DefaultKeyset { + Id(uuid::Uuid), + Name(String), +} + +/// The host could not supply a credential, or supplied one the guest +/// refuses. The message never holds a secret. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct CredentialError { + pub kind: BackendErrorKind, + pub message: String, +} + +impl CredentialError { + fn denied(message: impl Into) -> Self { + Self { + kind: BackendErrorKind::AccessDenied, + message: message.into(), + } + } +} + +/// Where the guest's vendor credentials come from, as a seam: one +/// short-lived credential for `vendor` and `scope`, as the frame bytes the +/// host wrote. `scope` is the OAuth scope for Google Cloud and Azure, and +/// empty for AWS and Vault. +/// +/// The guest asks before each vendor request and keeps nothing, so a host +/// source that refreshes needs no notice from the guest. +pub trait CredentialImport: Send + Sync { + /// The current credential, or why the host has none. + fn credential( + &self, + vendor: KeyVendor, + scope: &str, + ) -> impl std::future::Future>, CredentialError>> + MaybeSend; +} + +/// A credential as the host writes it. Every key is optional here; which +/// ones a vendor needs, and which it refuses, is [`CredentialFields`]'s. +#[derive(Deserialize)] +#[serde(deny_unknown_fields)] +struct CredentialWire<'a> { + #[serde(default, borrow)] + access_key_id: Option<&'a str>, + #[serde(default, borrow)] + secret_access_key: Option<&'a str>, + #[serde(default, borrow)] + session_token: Option<&'a str>, + #[serde(default, borrow)] + token: Option<&'a str>, + #[serde(default)] + expires_at: Option, +} + +/// A credential's fields, read out of the host's value. +struct CredentialFields { + strings: Vec<(String, Zeroizing)>, + /// What is left of its life. `None` when the host gave no expiry. + remaining: Option, +} + +impl CredentialFields { + /// Read `bytes`, the host's credential for `vendor`, at `now`. + fn read(vendor: KeyVendor, bytes: &[u8], now: SystemTime) -> Result { + let not_valid = || { + CredentialError::denied(format!( + "the host's {} credential is not an object of string values", + vendor.name() + )) + }; + let wire: CredentialWire<'_> = from_slice(bytes).map_err(|_| not_valid())?; + let mut remaining = None; + if let Some(seconds) = wire.expires_at { + let expires = u64::try_from(seconds) + .ok() + .and_then(|s| SystemTime::UNIX_EPOCH.checked_add(Duration::from_secs(s))) + .ok_or_else(not_valid)?; + let left = expires.duration_since(now).ok().filter(|d| !d.is_zero()); + remaining = Some(left.ok_or_else(|| { + CredentialError::denied(format!( + "the host's {} credential has expired", + vendor.name() + )) + })?); + } + let mut strings = Vec::with_capacity(4); + for (key, text) in [ + ("access_key_id", wire.access_key_id), + ("secret_access_key", wire.secret_access_key), + ("session_token", wire.session_token), + ("token", wire.token), + ] { + let Some(text) = text else { continue }; + // A value read from a file keeps its newline; no secret here has + // meaningful surrounding whitespace. An interior control + // character is refused: it would split a header. + let text = Zeroizing::new(text.trim().to_owned()); + if text.chars().any(char::is_control) { + return Err(CredentialError::denied(format!( + "the host's {} credential has a control character in `{key}`", + vendor.name() + ))); + } + strings.push((key.to_owned(), text)); + } + Ok(Self { strings, remaining }) + } + + /// Take the non-empty string `key`, or `None`. + fn take(&mut self, key: &str) -> Option> { + let at = self.strings.iter().position(|(k, _)| k == key)?; + let (_, value) = self.strings.swap_remove(at); + (!value.is_empty()).then_some(value) + } + + /// Take the required string `key`. + fn require( + &mut self, + vendor: KeyVendor, + key: &str, + ) -> Result, CredentialError> { + self.take(key).ok_or_else(|| { + CredentialError::denied(format!( + "the host's {} credential has no `{key}`", + vendor.name() + )) + }) + } + + /// Refuse a key this vendor's credential does not have: a host that + /// sends `token` for AWS has the vendor wrong, and must be told. + fn finish(self, vendor: KeyVendor) -> Result, CredentialError> { + match self.strings.first() { + Some((key, _)) => Err(CredentialError::denied(format!( + "the host's {} credential has a key it does not use: `{key}`", + vendor.name() + ))), + None => Ok(self.remaining), + } + } +} + +/// Read a token credential (Vault, Google Cloud, Azure): the token and what +/// is left of its life. +fn read_token( + vendor: KeyVendor, + bytes: &[u8], + now: SystemTime, +) -> Result<(Zeroizing, Option), CredentialError> { + let mut fields = CredentialFields::read(vendor, bytes, now)?; + let token = fields.require(vendor, "token")?; + let remaining = fields.finish(vendor)?; + Ok((token, remaining)) +} + +/// The clock the credential expiry is checked against. A seam for tests. +pub type Now = fn() -> SystemTime; + +/// The credential and token providers of the vendor clients, over the +/// host's credentials. One type serves AWS (`CredentialsProvider`) and the +/// three token providers. +pub struct HostCredentials { + import: C, + now: Now, +} + +impl HostCredentials { + /// Providers over `import`, checking expiry against the system clock. + pub fn new(import: C) -> Self { + Self { + import, + now: SystemTime::now, + } + } + + /// The same, with `now` as the clock. + pub fn with_now(mut self, now: Now) -> Self { + self.now = now; + self + } +} + +impl HostCredentials { + async fn fetch( + &self, + vendor: KeyVendor, + scope: &str, + ) -> Result>, CredentialError> { + self.import.credential(vendor, scope).await + } + + async fn aws(&self) -> Result { + let vendor = KeyVendor::Aws; + let bytes = self.fetch(vendor, "").await?; + let mut fields = CredentialFields::read(vendor, &bytes, (self.now)())?; + let id = fields.require(vendor, "access_key_id")?; + let secret = fields.require(vendor, "secret_access_key")?; + let session = fields.take("session_token"); + let remaining = fields.finish(vendor)?; + // The vendor type takes `String`s and wipes its own copies of the + // secret and the session token; ours wipe on drop here. + let credentials = stack_kms_aws::Credentials::new( + id.as_str(), + secret.as_str(), + session.as_ref().map(|token| token.as_str().to_owned()), + ); + Ok(match remaining { + Some(left) => credentials.with_expiry((self.now)() + left), + None => credentials, + }) + } + + async fn token( + &self, + vendor: KeyVendor, + scope: &str, + ) -> Result<(Zeroizing, Option), CredentialError> { + let bytes = self.fetch(vendor, scope).await?; + read_token(vendor, &bytes, (self.now)()) + } +} + +impl stack_kms_aws::CredentialsProvider for HostCredentials { + async fn credentials( + &self, + ) -> Result { + self.aws() + .await + .map_err(|e| stack_kms_aws::CredentialsError::new(e.kind, e.message)) + } +} + +impl stack_kms_vault::TokenProvider for HostCredentials { + async fn token(&self) -> Result { + let (token, remaining) = self + .token(KeyVendor::Vault, "") + .await + .map_err(|e| stack_kms_vault::AuthError::new(e.kind, e.message))?; + let token = stack_kms_vault::VaultToken::new(token.as_str()); + Ok(match remaining { + Some(left) => token.with_lease(left), + None => token, + }) + } +} + +impl stack_kms_gcp::TokenProvider for HostCredentials { + async fn token(&self) -> Result { + let (token, remaining) = self + .token(KeyVendor::Gcp, stack_kms_gcp::CLOUD_KMS_SCOPE) + .await + .map_err(|e| stack_kms_gcp::AuthError::new(e.kind, e.message))?; + let token = stack_kms_gcp::AccessToken::new(token.as_str()); + Ok(match remaining { + Some(left) => token.with_expires_in(left), + None => token, + }) + } +} + +impl stack_kms_azure::TokenProvider for HostCredentials { + async fn token( + &self, + scope: &str, + ) -> Result { + let (token, remaining) = self + .token(KeyVendor::Azure, scope) + .await + .map_err(|e| stack_kms_azure::AuthError::new(e.kind, e.message))?; + let token = stack_kms_azure::AccessToken::new(token.as_str()); + Ok(match remaining { + Some(left) => token.with_expires_in(left), + None => token, + }) + } +} + +/// The guest's one key vendor backend: whichever the host configured. +/// +/// One enum, not a type parameter, so that the instance's cipher has one +/// type whatever the host chose at run time. +pub enum GuestVendor { + /// No key vendor backend. It holds no keyset records, so the router + /// never calls it, and every method refuses. + None, + Aws(stack_kms_aws::AwsKms>), + Vault(stack_kms_vault::VaultTransit>), + Gcp(stack_kms_gcp::GcpKms>), + Azure(stack_kms_azure::AzureKeyVault>), + Static(StaticKeyVendor), +} + +impl std::fmt::Debug for GuestVendor { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let name = match self { + Self::None => "none", + Self::Aws(_) => KeyVendor::Aws.name(), + Self::Vault(_) => KeyVendor::Vault.name(), + Self::Gcp(_) => KeyVendor::Gcp.name(), + Self::Azure(_) => KeyVendor::Azure.name(), + Self::Static(_) => KeyVendor::Static.name(), + }; + f.debug_tuple("GuestVendor").field(&name).finish() + } +} + +/// Every method of [`GuestVendor::None`]: it holds no keyset, so a call is +/// a routing bug, refused rather than answered. +fn no_vendor() -> Error { + Error::Backend { + vendor: "none", + kind: BackendErrorKind::NotFound, + source: "no key vendor backend is configured".into(), + } +} + +/// Dispatch one `KeyVendorBackend` method to the configured client. +macro_rules! dispatch { + ($self:expr, $backend:ident => $call:expr, none => $none:expr) => { + match $self { + GuestVendor::None => $none, + GuestVendor::Aws($backend) => $call, + GuestVendor::Vault($backend) => $call, + GuestVendor::Gcp($backend) => $call, + GuestVendor::Azure($backend) => $call, + GuestVendor::Static($backend) => $call, + } + }; +} + +impl KeyVendorBackend for GuestVendor +where + T: VendorTransport + Send + Sync, + C: CredentialImport, +{ + /// The configured vendor. [`GuestVendor::None`] answers `Static`: it is + /// never asked, because it holds no keyset (the router asks a backend + /// only for a keyset one of its records names). + fn vendor(&self) -> KeyVendor { + dispatch!(self, b => b.vendor(), none => KeyVendor::Static) + } + + fn supports_mode(&self, mode: KeyMode) -> bool { + dispatch!(self, b => b.supports_mode(mode), none => false) + } + + async fn wrap(&self, key_ref: &str, key: &KeyBytes, aad: &[u8]) -> Result, Error> { + dispatch!(self, b => b.wrap(key_ref, key, aad).await, none => Err(no_vendor())) + } + + async fn unwrap(&self, key_ref: &str, wrapped: &[u8], aad: &[u8]) -> Result { + dispatch!(self, b => b.unwrap(key_ref, wrapped, aad).await, none => Err(no_vendor())) + } + + async fn generate_wrapped( + &self, + key_ref: &str, + aad: &[u8], + ) -> Result<(KeyBytes, Vec), Error> { + dispatch!(self, b => b.generate_wrapped(key_ref, aad).await, none => Err(no_vendor())) + } + + fn batch_limit(&self) -> Option { + dispatch!(self, b => b.batch_limit(), none => None) + } + + async fn wrap_batch( + &self, + key_ref: &str, + items: &[WrapItem<'_>], + ) -> Result>, Error> { + dispatch!(self, b => b.wrap_batch(key_ref, items).await, none => Err(no_vendor())) + } + + async fn unwrap_batch( + &self, + key_ref: &str, + items: &[UnwrapItem<'_>], + ) -> Result, Error> { + dispatch!(self, b => b.unwrap_batch(key_ref, items).await, none => Err(no_vendor())) + } + + async fn sleep(&self, duration: Duration) { + dispatch!(self, b => b.sleep(duration).await, none => ()) + } +} + +/// Why the configured backend could not be built: a setting the vendor +/// client refused (`STATUS_ENCODING`, naming the config key), or a keyset +/// record ([`RecordError`]). +#[derive(Debug)] +pub enum BuildError { + /// The config key whose value the vendor client refused. + Setting(&'static str), + /// A keyset record was refused. + Record(RecordError), +} + +impl From for BuildError { + fn from(error: RecordError) -> Self { + Self::Record(error) + } +} + +/// Build the key source of the configured backend: the client over +/// `transport` and `import`, and the adapter over the keyset records. With +/// no config, [`GuestVendor::None`] and no records. +pub fn build( + config: Option, + transport: T, + import: C, +) -> Result>, BuildError> +where + T: VendorTransport + Send + Sync, + C: CredentialImport, +{ + let Some(VendorConfig { + settings, + records, + default_keyset, + root_key_ttl, + }) = config + else { + return Ok(VendorKeySource::new(GuestVendor::None, [])?); + }; + let credentials = HostCredentials::new(import); + let backend = match settings { + VendorSettings::Aws { region, endpoint } => { + let mut config = stack_kms_aws::AwsKmsConfig::new(region) + .map_err(|_| BuildError::Setting("aws_region"))?; + if let Some(endpoint) = endpoint { + config = config + .with_endpoint(endpoint) + .map_err(|_| BuildError::Setting("aws_endpoint"))?; + } + GuestVendor::Aws(stack_kms_aws::AwsKms::new(config, transport, credentials)) + } + VendorSettings::Vault { + address, + namespace, + mount, + } => { + let mut config = stack_kms_vault::VaultConfig::new(address) + .map_err(|_| BuildError::Setting("vault_address"))?; + if let Some(namespace) = namespace { + config = config + .with_namespace(namespace) + .map_err(|_| BuildError::Setting("vault_namespace"))?; + } + if let Some(mount) = mount { + config = config + .with_mount(mount) + .map_err(|_| BuildError::Setting("vault_mount"))?; + } + GuestVendor::Vault(stack_kms_vault::VaultTransit::new( + config, + transport, + credentials, + )) + } + VendorSettings::Gcp { endpoint } => { + let mut config = stack_kms_gcp::GcpKmsConfig::new(); + if let Some(endpoint) = endpoint { + config = config + .with_endpoint(endpoint) + .map_err(|_| BuildError::Setting("gcp_endpoint"))?; + } + GuestVendor::Gcp(stack_kms_gcp::GcpKms::new(config, transport, credentials)) + } + VendorSettings::Azure => GuestVendor::Azure(stack_kms_azure::AzureKeyVault::new( + stack_kms_azure::AzureKeyVaultConfig::new(), + transport, + credentials, + )), + VendorSettings::Static { key_ref, key } => { + GuestVendor::Static(StaticKeyVendor::new(key_ref, key)) + } + }; + let records = records + .iter() + .map(|text| KeysetRecord::parse(text)) + .collect::, _>>()?; + let default = match default_keyset { + None => None, + Some(DefaultKeyset::Id(id)) => Some(id), + Some(DefaultKeyset::Name(name)) => Some( + records + .iter() + .find(|r| r.name() == Some(name.as_str())) + .map(KeysetRecord::id) + .ok_or(RecordError::UnknownDefault)?, + ), + }; + let mut source = VendorKeySource::new(backend, records)?; + if let Some(id) = default { + source = source.with_default_keyset(id)?; + } + if let Some(ttl) = root_key_ttl { + source = source.with_root_key_ttl(ttl); + } + Ok(source) +} + +#[cfg(test)] +mod tests { + use std::sync::Mutex; + + use super::*; + use crate::test_frame::{frame, obj, V}; + use stack_encrypt::Controlled as _; + use stack_kms_aws::CredentialsProvider; + use stack_kms_vendor::{TransportError, VendorRequest, VendorResponse}; + + /// A fixed clock: 2026-11-06T00:00:00Z. + fn now() -> SystemTime { + SystemTime::UNIX_EPOCH + Duration::from_secs(1_793_923_200) + } + + fn object(entries: Vec<(&'static str, V)>) -> V { + obj(entries) + } + + fn string(text: &str) -> V { + V::Str(text.into()) + } + + type Answer = Box Result + Send + Sync>; + + /// A fake import: answers each call with a new value from `answer` and + /// records what it was asked. + struct FakeImport { + answer: Answer, + asked: Mutex>, + } + + impl FakeImport { + fn new(answer: impl Fn() -> Result + Send + Sync + 'static) -> Self { + Self { + answer: Box::new(answer), + asked: Mutex::new(Vec::new()), + } + } + + fn unused() -> Self { + Self::new(|| Err(CredentialError::denied("unused"))) + } + } + + impl CredentialImport for FakeImport { + async fn credential( + &self, + vendor: KeyVendor, + scope: &str, + ) -> Result>, CredentialError> { + self.asked.lock().unwrap().push((vendor, scope.to_owned())); + (self.answer)().map(|value| Zeroizing::new(frame(&value))) + } + } + + fn aws_value() -> V { + object(vec![ + ("access_key_id", string("AKIDEXAMPLE")), + ("secret_access_key", string("fixture-secret")), + ]) + } + + fn block(f: F) -> F::Output { + futures::executor::block_on(f) + } + + #[test] + fn aws_credentials_are_read_from_the_host_value() { + let expires = now() + Duration::from_secs(900); + let seconds = expires + .duration_since(SystemTime::UNIX_EPOCH) + .unwrap() + .as_secs(); + let provider = HostCredentials::new(FakeImport::new(move || { + Ok(object(vec![ + ("access_key_id", string("AKIDEXAMPLE")), + ("secret_access_key", string("fixture-secret\n")), + ("session_token", string("fixture-session-token")), + ("expires_at", V::I64(seconds as i64)), + ])) + })) + .with_now(now); + let credentials = block(provider.credentials()).unwrap(); + assert_eq!(credentials.access_key_id(), "AKIDEXAMPLE"); + assert_eq!(credentials.secret_access_key(), "fixture-secret"); + assert_eq!(credentials.session_token(), Some("fixture-session-token")); + assert_eq!(credentials.expires_at(), Some(expires)); + assert_eq!( + provider.import.asked.lock().unwrap().as_slice(), + &[(KeyVendor::Aws, String::new())] + ); + } + + #[test] + fn a_missing_session_token_and_expiry_are_allowed() { + let provider = HostCredentials::new(FakeImport::new(|| Ok(aws_value()))); + let credentials = block(provider.credentials()).unwrap(); + assert_eq!(credentials.session_token(), None); + assert_eq!(credentials.expires_at(), None); + } + + #[test] + fn bad_credentials_are_refused_without_their_secret_in_the_message() { + fn with(extra: Vec<(&'static str, V)>) -> V { + let mut entries = vec![("access_key_id", string("AKIDEXAMPLE"))]; + entries.extend(extra); + object(entries) + } + fn expired() -> i64 { + now() + .duration_since(SystemTime::UNIX_EPOCH) + .unwrap() + .as_secs() as i64 + } + type Case = (&'static str, fn() -> V); + let cases: Vec = vec![ + ("not an object", || string("fixture-secret")), + ("no secret", || with(vec![])), + ("empty secret", || { + with(vec![("secret_access_key", string(""))]) + }), + ("expired", || { + with(vec![ + ("secret_access_key", string("fixture-secret")), + ("expires_at", V::I64(expired())), + ]) + }), + ("negative expiry", || { + with(vec![ + ("secret_access_key", string("fixture-secret")), + ("expires_at", V::I64(-1)), + ]) + }), + ("control character", || { + with(vec![("secret_access_key", string("fixture\nsecret"))]) + }), + ("unknown key", || { + with(vec![ + ("secret_access_key", string("fixture-secret")), + ("token", string("fixture-secret")), + ]) + }), + ("not a string", || { + with(vec![("secret_access_key", V::I64(1))]) + }), + ]; + for (label, value) in cases { + let provider = HostCredentials::new(FakeImport::new(move || Ok(value()))).with_now(now); + let error = block(provider.aws()).expect_err(label); + assert_eq!(error.kind, BackendErrorKind::AccessDenied, "{label}"); + assert!( + !error.message.contains("fixture"), + "{label}: {}", + error.message + ); + } + } + + #[test] + fn a_host_failure_keeps_its_kind() { + let failure = CredentialError { + kind: BackendErrorKind::Unavailable, + message: "the host credential source failed".into(), + }; + let provider = HostCredentials::new(FakeImport::new(move || Err(failure.clone()))); + let error = block(provider.credentials()).unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); + } + + #[test] + fn token_providers_ask_with_their_vendor_and_scope() { + let provider = HostCredentials::new(FakeImport::new(|| { + Ok(object(vec![("token", string("fixture-token"))])) + })); + let token = block(stack_kms_vault::TokenProvider::token(&provider)).unwrap(); + assert_eq!(token.token(), "fixture-token"); + + let token = block(stack_kms_gcp::TokenProvider::token(&provider)).unwrap(); + assert_eq!(token.token(), "fixture-token"); + + let token = block(stack_kms_azure::TokenProvider::token( + &provider, + stack_kms_azure::MANAGED_HSM_SCOPE, + )) + .unwrap(); + assert_eq!(token.token(), "fixture-token"); + + assert_eq!( + provider.import.asked.lock().unwrap().as_slice(), + &[ + (KeyVendor::Vault, String::new()), + (KeyVendor::Gcp, stack_kms_gcp::CLOUD_KMS_SCOPE.to_owned()), + ( + KeyVendor::Azure, + stack_kms_azure::MANAGED_HSM_SCOPE.to_owned() + ), + ] + ); + + // An AWS credential where a token is wanted is refused. + let provider = HostCredentials::new(FakeImport::new(|| Ok(aws_value()))); + assert!(block(stack_kms_vault::TokenProvider::token(&provider)).is_err()); + } + + /// A transport that answers every request with one response and keeps + /// the requests. + #[derive(Default)] + struct Recording { + requests: Mutex>, + response: Mutex>, + } + + impl VendorTransport for &Recording { + async fn send(&self, request: VendorRequest) -> Result { + self.requests.lock().unwrap().push(request); + self.response + .lock() + .unwrap() + .take() + .ok_or_else(|| TransportError("no response".into())) + } + + async fn sleep(&self, _: Duration) {} + } + + fn aws_settings() -> VendorConfig { + VendorConfig { + settings: VendorSettings::Aws { + region: "us-east-1".into(), + endpoint: None, + }, + records: Vec::new(), + default_keyset: None, + root_key_ttl: None, + } + } + + #[test] + fn an_aws_request_is_signed_with_the_host_credentials() { + let transport = Recording::default(); + *transport.response.lock().unwrap() = Some(VendorResponse::new( + 200, + br#"{"KeyId":"arn:aws:kms:us-east-1:111122223333:key/k","Plaintext":"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="}"#.to_vec(), + )); + let import = FakeImport::new(|| { + Ok(object(vec![ + ("access_key_id", string("AKIDEXAMPLE")), + ("secret_access_key", string("fixture-secret")), + ("session_token", string("fixture-session-token")), + ])) + }); + let source = build(Some(aws_settings()), &transport, import).unwrap(); + let key = block(source.backend().unwrap( + "arn:aws:kms:us-east-1:111122223333:key/k", + b"wrapped", + b"aad", + )) + .unwrap(); + assert_eq!(key.risky_ref(), &[0u8; 32]); + + let requests = transport.requests.lock().unwrap(); + let request = &requests[0]; + assert_eq!(request.url, "https://kms.us-east-1.amazonaws.com/"); + let authorization = request.header_value("authorization").unwrap(); + assert!(authorization.starts_with("AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/")); + assert_eq!( + request.header_value("x-amz-security-token"), + Some("fixture-session-token") + ); + assert_eq!( + request.header_value("x-amz-target"), + Some("TrentService.Decrypt") + ); + } + + #[test] + fn no_config_builds_no_vendor_and_no_keyset() { + let transport = Recording::default(); + let source = + build::<&Recording, FakeImport>(None, &transport, FakeImport::unused()).unwrap(); + assert!(matches!(source.backend(), GuestVendor::None)); + assert!(block(source.backend().unwrap("k", b"w", b"a")).is_err()); + assert!(!source.backend().supports_mode(KeyMode::Derived)); + } + + #[test] + fn a_bad_setting_names_its_config_key() { + let config = VendorConfig { + settings: VendorSettings::Aws { + region: "us east 1".into(), + endpoint: None, + }, + records: Vec::new(), + default_keyset: None, + root_key_ttl: None, + }; + let error = build(Some(config), &Recording::default(), FakeImport::unused()).unwrap_err(); + assert!(matches!(error, BuildError::Setting("aws_region"))); + + let config = VendorConfig { + settings: VendorSettings::Vault { + address: "not a url".into(), + namespace: None, + mount: None, + }, + records: Vec::new(), + default_keyset: None, + root_key_ttl: None, + }; + let error = build(Some(config), &Recording::default(), FakeImport::unused()).unwrap_err(); + assert!(matches!(error, BuildError::Setting("vault_address"))); + } + + #[test] + fn a_record_of_another_vendor_is_refused() { + // The pinned AWS record of stack-kms-vendor's own tests. + let aws = "stack-keyset:AQEBAQIDBAUGBwgJCgsMDQ4PEARhY21lAChhcm46YXdzOmttczp1cy1lYXN0LTE6MTExMTIyMjIzMzMzOmtleS9rAAMBAgM"; + let config = VendorConfig { + settings: VendorSettings::Static { + key_ref: "local".into(), + key: KeyBytes::new([7; 32]), + }, + records: vec![aws.to_owned()], + default_keyset: None, + root_key_ttl: None, + }; + let error = build(Some(config), &Recording::default(), FakeImport::unused()).unwrap_err(); + assert!(matches!( + error, + BuildError::Record(RecordError::ForeignVendor { .. }) + )); + + let config = VendorConfig { + records: vec!["stack-keyset:***".to_owned()], + ..aws_settings() + }; + assert!(matches!( + build(Some(config), &Recording::default(), FakeImport::unused(),), + Err(BuildError::Record(_)) + )); + } +} diff --git a/languages/golang/encrypt/guest/src/zerokms.rs b/languages/golang/encrypt/guest/src/zerokms.rs new file mode 100644 index 0000000000..1487bdad6d --- /dev/null +++ b/languages/golang/encrypt/guest/src/zerokms.rs @@ -0,0 +1,190 @@ +//! The ZeroKMS side of the instance's key router, which a client with a key +//! vendor backend may leave out (CIP-5311). +//! +//! A client is ZeroKMS and at most one key vendor backend +//! (`stack_kms_vendor::KeyRouter`). A key vendor backend is an alternative to +//! ZeroKMS, so a client may also have the vendor alone: the host gives no +//! client id and no client key, and the router's ZeroKMS side is +//! [`MaybeZeroKms::Absent`]. The router then sends the default keyset to the +//! vendor (`KeyRouter::with_vendor_default`), and anything it still sends +//! here (a keyset no record names, a v1 leaf) is refused with +//! [`no_zerokms`], which the status layer reports as `STATUS_NO_ZEROKMS`. + +use std::borrow::Cow; + +use stack_kms::{ + BackendErrorKind, DataKey, DataKeySource, DataKeyWithTag, Error, GenerateKeyPayload, + IdentifiedBy, IndexKey, IndexKeySource, KeyBackend, KeyMode, LoadedIndexKey, + RetrieveKeyPayload, SelectedKeyset, UnverifiedContext, +}; +use uuid::Uuid; + +/// The ZeroKMS client, or none. +pub enum MaybeZeroKms { + /// The client has ZeroKMS. + Present(Z), + /// The client has a key vendor backend and no ZeroKMS. + Absent, +} + +/// The cause of [`no_zerokms`]: its type is what the status layer finds to +/// report `STATUS_NO_ZEROKMS`. +#[derive(Debug, Clone, Copy)] +pub struct NoZeroKms; + +impl std::fmt::Display for NoZeroKms { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str( + "this client has no ZeroKMS, only a key vendor backend, and no keyset record names this keyset", + ) + } +} + +impl std::error::Error for NoZeroKms {} + +/// The refusal of every request that reaches [`MaybeZeroKms::Absent`]. +pub fn no_zerokms() -> Error { + Error::Backend { + vendor: KeyBackend::ZeroKms.name(), + kind: BackendErrorKind::NotFound, + source: Box::new(NoZeroKms), + } +} + +/// Whether `error` is [`no_zerokms`]. +pub fn is_no_zerokms(error: &Error) -> bool { + matches!(error, Error::Backend { source, .. } if source.is::()) +} + +impl DataKeySource for MaybeZeroKms { + async fn generate_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option>, + ) -> Result, Error> { + match self { + Self::Present(z) => { + z.generate_keys(payloads, keyset_id, unverified_context) + .await + } + Self::Absent => Err(no_zerokms()), + } + } + + async fn retrieve_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option<&UnverifiedContext>, + ) -> Result, Error> { + match self { + Self::Present(z) => { + z.retrieve_keys(payloads, keyset_id, unverified_context) + .await + } + Self::Absent => Err(no_zerokms()), + } + } + + fn key_backend(&self) -> KeyBackend { + match self { + Self::Present(z) => z.key_backend(), + Self::Absent => KeyBackend::ZeroKms, + } + } + + fn keyset_backend(&self, keyset_id: Uuid) -> KeyBackend { + match self { + Self::Present(z) => z.keyset_backend(keyset_id), + Self::Absent => KeyBackend::ZeroKms, + } + } + + fn key_mode(&self, keyset_id: Uuid) -> Option { + match self { + Self::Present(z) => z.key_mode(keyset_id), + Self::Absent => None, + } + } + + async fn load_settle_index_key(&self, keyset_id: Uuid) -> Result { + match self { + Self::Present(z) => z.load_settle_index_key(keyset_id).await, + Self::Absent => Err(no_zerokms()), + } + } +} + +impl IndexKeySource for MaybeZeroKms { + async fn load_index_key( + &self, + keyset_id: Option, + ) -> Result<(Uuid, IndexKey), Error> { + match self { + Self::Present(z) => z.load_index_key(keyset_id).await, + Self::Absent => Err(no_zerokms()), + } + } + + async fn load_index_key_with_lifetime( + &self, + keyset_id: Option, + ) -> Result { + match self { + Self::Present(z) => z.load_index_key_with_lifetime(keyset_id).await, + Self::Absent => Err(no_zerokms()), + } + } + + async fn select_keyset( + &self, + keyset_id: Option, + ) -> Result { + match self { + Self::Present(z) => z.select_keyset(keyset_id).await, + Self::Absent => Err(no_zerokms()), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use stack_kms::FakeDataKeySource; + + fn block(f: F) -> F::Output { + futures::executor::block_on(f) + } + + #[test] + fn absent_refuses_every_request_with_the_typed_error() { + let absent = MaybeZeroKms::::Absent; + let id = Uuid::from_u128(1); + for error in [ + block(absent.generate_keys(vec![], Some(id), None)).unwrap_err(), + block(absent.retrieve_keys(vec![], Some(id), None)).unwrap_err(), + block(absent.load_settle_index_key(id)).unwrap_err(), + block(absent.load_index_key(None)).unwrap_err(), + block(absent.load_index_key_with_lifetime(None)).unwrap_err(), + block(absent.select_keyset(None)).unwrap_err(), + ] { + assert!(is_no_zerokms(&error), "{error:?}"); + } + assert_eq!(absent.key_backend(), KeyBackend::ZeroKms); + assert_eq!(absent.key_mode(id), None); + // Another backend error is not this one. + assert!(!is_no_zerokms(&Error::Backend { + vendor: KeyBackend::ZeroKms.name(), + kind: BackendErrorKind::NotFound, + source: "other".into(), + })); + } + + #[test] + fn present_forwards() { + let present = MaybeZeroKms::Present(FakeDataKeySource::new()); + let (id, _) = block(present.load_index_key(None)).unwrap(); + assert_eq!(present.keyset_backend(id), KeyBackend::ZeroKms); + } +} diff --git a/languages/golang/encrypt/keyvendor.go b/languages/golang/encrypt/keyvendor.go new file mode 100644 index 0000000000..9e62ce642d --- /dev/null +++ b/languages/golang/encrypt/keyvendor.go @@ -0,0 +1,552 @@ +package encrypt + +import ( + "context" + "errors" + "fmt" + "log/slog" + "os" + "time" + + "github.com/cipherstash/vitaminc/bindings/go/vcffi" + "github.com/cipherstash/vitaminc/bindings/go/vcvalue" + "golang.org/x/oauth2" +) + +// KeyVendor is the one key vendor backend a [Client] can hold beside +// ZeroKMS: [AWSKMS], [GoogleCloudKMS], [AzureKeyVault], [VaultTransit] or +// [StaticKey]. Give it with [WithKeyVendor], and its keyset records with +// [WithKeysetRecords]. +// +// A keyset whose record the client holds is sealed and opened through the +// key vendor backend. Every other keyset, and the default keyset, is +// ZeroKMS's. Select a vendor keyset by the name or the id in its record: +// client.Keyset(encrypt.KeysetName("payments")). +// +// The guest makes the vendor requests and signs them. The client's +// RoundTripper ([WithTransport]) sends them. Keys stay in the guest: the +// host supplies only short-lived credentials, never key material. See the +// package documentation for what each backend and key mode can and cannot +// do. +// +// The interface is sealed: the five types above are its only +// implementations. +type KeyVendor interface { + keyVendor() (*vendorSpec, error) +} + +// vendorSpec is a KeyVendor as the client uses it: the config keys the +// guest reads, and where the per-request credential comes from. +type vendorSpec struct { + // name is the guest's key_vendor value. + name string + // id is the KeyVendor byte the guest names in a credential request. + id uint32 + // fields are the backend's settings in the guest config. + fields vcvalue.Object + // staticKey is StaticKey's key, consumed by NewClient. + staticKey []byte + aws AWSCredentialSource + token AccessTokenSource +} + +// The KeyVendor bytes of the v2 leaf header and the keyset record. They +// cross in the guest's credential requests (step.go). +const ( + vendorAWS = 1 + vendorGCP = 2 + vendorAzure = 3 + vendorVault = 4 + vendorStatic = 5 +) + +// AWSKMS is AWS KMS as the key vendor backend. The key in each keyset +// record is a KMS key ARN. Both key modes. +type AWSKMS struct { + // Region is the AWS region of the keys, such as "us-east-1". Required. + Region string + // Endpoint overrides the regional AWS KMS endpoint, for a VPC endpoint + // or LocalStack. Empty means https://kms..amazonaws.com/. + Endpoint string + // Credentials supplies temporary AWS credentials, asked before each + // request. Required. Adapt the AWS SDK's chain with + // [AWSCredentialsFunc]; [AWSCredentialsFromEnv] reads the environment. + Credentials AWSCredentialSource +} + +func (v AWSKMS) keyVendor() (*vendorSpec, error) { + if v.Region == "" { + return nil, fmt.Errorf("%w: AWSKMS needs a Region", ErrEncoding) + } + if v.Credentials == nil { + return nil, fmt.Errorf("%w: AWSKMS needs Credentials", ErrEncoding) + } + fields := vcvalue.Object{{Key: "aws_region", Value: v.Region}} + if v.Endpoint != "" { + fields = append(fields, vcvalue.Field{Key: "aws_endpoint", Value: v.Endpoint}) + } + return &vendorSpec{name: "aws-kms", id: vendorAWS, fields: fields, aws: v.Credentials}, nil +} + +// GoogleCloudKMS is Google Cloud KMS as the key vendor backend. The key in +// each keyset record is a crypto key name +// (projects/…/locations/…/keyRings/…/cryptoKeys/…). Both key modes. +type GoogleCloudKMS struct { + // Endpoint overrides https://cloudkms.googleapis.com, for a regional + // endpoint. Empty means the global one. + Endpoint string + // Token supplies an OAuth 2.0 access token with the Cloud KMS scope, + // asked before each request. Required. [OAuth2AccessTokens] adapts + // golang.org/x/oauth2/google's default credentials. + Token AccessTokenSource +} + +func (v GoogleCloudKMS) keyVendor() (*vendorSpec, error) { + if v.Token == nil { + return nil, fmt.Errorf("%w: GoogleCloudKMS needs a Token source", ErrEncoding) + } + var fields vcvalue.Object + if v.Endpoint != "" { + fields = append(fields, vcvalue.Field{Key: "gcp_endpoint", Value: v.Endpoint}) + } + return &vendorSpec{name: "gcp-kms", id: vendorGCP, fields: fields, token: v.Token}, nil +} + +// AzureKeyVault is Azure Key Vault or Managed HSM as the key vendor +// backend. The key in each keyset record is a key identifier +// (https://.vault.azure.net/keys/), which names the vault. +// Both key modes. +type AzureKeyVault struct { + // Token supplies a Microsoft Entra ID access token for the scope the + // client asks for: https://vault.azure.net/.default for Key Vault, + // https://managedhsm.azure.net/.default for Managed HSM. Required. + // Adapt azidentity with [AccessTokenFunc]. + Token AccessTokenSource +} + +func (v AzureKeyVault) keyVendor() (*vendorSpec, error) { + if v.Token == nil { + return nil, fmt.Errorf("%w: AzureKeyVault needs a Token source", ErrEncoding) + } + return &vendorSpec{name: "azure-key-vault", id: vendorAzure, token: v.Token}, nil +} + +// VaultTransit is HashiCorp Vault's Transit secrets engine as the key +// vendor backend. The key in each keyset record is a Transit key name. +// Both key modes. +type VaultTransit struct { + // Address is the Vault address, such as "https://vault.example.com:8200". + // Required. + Address string + // Namespace is the Vault Enterprise namespace. Empty means none. + Namespace string + // Mount is the Transit mount path. Empty means "transit". + Mount string + // Token supplies a Vault client token, asked before each request; the + // scope it is given is empty. Required. Adapt the Vault API client's + // token with [AccessTokenFunc]. + Token AccessTokenSource +} + +func (v VaultTransit) keyVendor() (*vendorSpec, error) { + if v.Address == "" { + return nil, fmt.Errorf("%w: VaultTransit needs an Address", ErrEncoding) + } + if v.Token == nil { + return nil, fmt.Errorf("%w: VaultTransit needs a Token source", ErrEncoding) + } + fields := vcvalue.Object{{Key: "vault_address", Value: v.Address}} + if v.Namespace != "" { + fields = append(fields, vcvalue.Field{Key: "vault_namespace", Value: v.Namespace}) + } + if v.Mount != "" { + fields = append(fields, vcvalue.Field{Key: "vault_mount", Value: v.Mount}) + } + return &vendorSpec{name: "vault-transit", id: vendorVault, fields: fields, token: v.Token}, nil +} + +// StaticKey is a 32-byte key the application holds, as the key vendor +// backend: for development, tests, and deployments that keep a key in their +// own secret store. Derived data keys only, because the key is always in +// memory. Nothing outside the process can refuse or audit its use. +type StaticKey struct { + // KeyRef is the key's reference, as the keyset records name it. + // Required. + KeyRef string + // Key is the key: 32 bytes. NewClient wipes it whatever the outcome, + // as it consumes the client key: after NewClient returns, the slice + // holds zeros. + Key []byte +} + +// String hides the key. +func (v StaticKey) String() string { + return fmt.Sprintf("encrypt.StaticKey{KeyRef: %s, Key: }", v.KeyRef) +} + +// GoString hides it from %#v too. +func (v StaticKey) GoString() string { return v.String() } + +// LogValue hides it from slog. +func (v StaticKey) LogValue() slog.Value { + return slog.GroupValue(slog.String("key_ref", v.KeyRef)) +} + +func (v StaticKey) keyVendor() (*vendorSpec, error) { + if v.KeyRef == "" { + return nil, fmt.Errorf("%w: StaticKey needs a KeyRef", ErrEncoding) + } + if len(v.Key) != 32 { + return nil, fmt.Errorf("%w: StaticKey's Key is %d bytes, not 32", ErrEncoding, len(v.Key)) + } + return &vendorSpec{ + name: "static", + id: vendorStatic, + fields: vcvalue.Object{{Key: "static_key_ref", Value: v.KeyRef}}, + staticKey: v.Key, + }, nil +} + +// wipeVendorKey zeroes a StaticKey's key, the one key material a KeyVendor +// value holds. Any other KeyVendor holds none. +func wipeVendorKey(v KeyVendor) { + switch k := v.(type) { + case StaticKey: + wipe(k.Key) + case *StaticKey: + if k != nil { + wipe(k.Key) + } + } +} + +// WithKeyVendor gives the client one key vendor backend. It needs +// [WithKeysetRecords]; one without the other is refused. A later +// WithKeyVendor replaces an earlier one. The default is none: every keyset +// is ZeroKMS's, as before. +// +// A key vendor backend is an alternative to ZeroKMS. With no +// [WithCredentials] the client has no ZeroKMS: NewClient neither needs nor +// looks for ZeroKMS credentials, the default keyset is a vendor keyset (see +// [WithDefaultKeyset]), and a call that needs a ZeroKMS keyset fails with +// [ErrNoZeroKMS]. To hold both, also give WithCredentials, for example +// WithCredentials(AutoCredentials()). +func WithKeyVendor(v KeyVendor) ClientOption { + return func(o *clientOptions) { + if o.keyVendor != nil { + o.supersededVendors = append(o.supersededVendors, o.keyVendor) + } + o.keyVendor = v + } +} + +// WithKeysetRecords gives the keyset records of the client's key vendor +// backend: the "stack-keyset:…" texts that `stash keyset` writes, one per +// keyset. Each record names its keyset id, its name, the vendor key, its key +// mode and its wrapped key; none holds a key in the clear. Every record must +// be of the backend [WithKeyVendor] gives, and two records may not share an +// id or a name. Records add up across calls. +func WithKeysetRecords(records ...string) ClientOption { + return func(o *clientOptions) { o.keysetRecords = append(o.keysetRecords, records...) } +} + +// WithDefaultKeyset names the default keyset of a client that has a key +// vendor backend and no ZeroKMS: the keyset [Client.DefaultKeyset] and every +// call that names no keyset use. Name it by the name or the id in its keyset +// record, as [Client.Keyset] does: [KeysetName] or [KeysetID]. With one +// keyset record it is that record's keyset and this option is not needed; +// with several and no default, NewClient fails with [ErrNotFound]. +// +// A client with ZeroKMS has the default keyset its ZeroKMS administrator +// set, and refuses this option with [ErrEncoding]. +func WithDefaultKeyset(sel KeysetSelector) ClientOption { + return func(o *clientOptions) { o.defaultKeyset, o.defaultKeysetSet = sel, true } +} + +// WithRootKeyTTL sets how long the guest keeps a keyset's root key +// unwrapped under derived data keys, after which the next use unwraps it +// again: one vendor call, which the vendor can refuse. Zero keeps none: +// each use of a root key unwraps it, so an Encrypt that also derives search +// terms unwraps it twice. The default is 5 minutes. It has +// no effect on wrapped data keys, which keep nothing beyond one call. +// Negative, or with no [WithKeyVendor], is refused with [ErrEncoding]. +func WithRootKeyTTL(d time.Duration) ClientOption { + return func(o *clientOptions) { o.rootKeyTTL = &d } +} + +// checkVendorOnlyOptions refuses the options that need a key vendor +// backend, or a client with no ZeroKMS, when the client has not got one. +func checkVendorOnlyOptions(cfg *clientOptions, vendorOnly bool) error { + if cfg.rootKeyTTL != nil { + if cfg.keyVendor == nil { + return fmt.Errorf("%w: WithRootKeyTTL needs WithKeyVendor", ErrEncoding) + } + if *cfg.rootKeyTTL < 0 { + return fmt.Errorf("%w: WithRootKeyTTL must not be negative", ErrEncoding) + } + } + if !cfg.defaultKeysetSet { + return nil + } + if !vendorOnly { + return fmt.Errorf("%w: WithDefaultKeyset needs WithKeyVendor and no WithCredentials: a client with ZeroKMS has the default keyset its administrator set", ErrEncoding) + } + switch cfg.defaultKeyset.(type) { + case KeysetName, KeysetID: + return nil + default: + return fmt.Errorf("%w: WithDefaultKeyset takes a KeysetName or a KeysetID", ErrEncoding) + } +} + +// noZeroKMS is the Credentials of a client that has a key vendor backend +// and no ZeroKMS: nothing to resolve, no client key and no token. +type noZeroKMS struct{} + +func (noZeroKMS) resolve(context.Context, resolveOptions) (*resolvedCredentials, error) { + return &resolvedCredentials{}, nil +} + +// keyVendorConfig checks the key vendor options host-side, before anything +// is resolved, and renders them. Nil when no backend is configured. +func keyVendorConfig(v KeyVendor, records []string) (*vendorSpec, error) { + if v == nil { + if len(records) > 0 { + return nil, fmt.Errorf("%w: WithKeysetRecords needs WithKeyVendor", ErrEncoding) + } + return nil, nil + } + spec, err := v.keyVendor() + if err != nil { + return nil, err + } + if len(records) == 0 { + return nil, fmt.Errorf("%w: WithKeyVendor needs WithKeysetRecords", ErrEncoding) + } + return spec, nil +} + +// configFields renders the backend and its records as guest config keys. +// The result holds a StaticKey's key: the caller wipes what it encodes. +func (s *vendorSpec) configFields(records []string) vcvalue.Object { + fields := vcvalue.Object{ + {Key: "key_vendor", Value: s.name}, + {Key: "keyset_records", Value: append([]string(nil), records...)}, + } + fields = append(fields, s.fields...) + if s.staticKey != nil { + fields = append(fields, vcvalue.Field{Key: "static_key", Value: s.staticKey}) + } + return fields +} + +// ErrNoKeyVendorCredentials is a credential source with nothing to give, +// such as [AWSCredentialsFromEnv] with the variables unset. The wrapped +// error says what to set. +var ErrNoKeyVendorCredentials = errors.New("encrypt: no key vendor credentials") + +// AWSCredentials are temporary AWS credentials: what the guest signs an +// AWS KMS request with (SigV4). Long-term access keys work too, with no +// SessionToken, but prefer the temporary credentials of a role. +type AWSCredentials struct { + AccessKeyID string + SecretAccessKey string + // SessionToken is the session token of temporary credentials. + SessionToken string + // Expires is when the credentials expire. Zero means unknown; the guest + // refuses credentials whose expiry has passed. + Expires time.Time +} + +// String hides the secret and the session token. +func (c AWSCredentials) String() string { + return fmt.Sprintf("encrypt.AWSCredentials{AccessKeyID: %s, SecretAccessKey: , SessionToken: , Expires: %s}", c.AccessKeyID, c.Expires) +} + +// GoString hides them from %#v too. +func (c AWSCredentials) GoString() string { return c.String() } + +// LogValue hides them from slog. +func (c AWSCredentials) LogValue() slog.Value { + return slog.GroupValue(slog.String("access_key_id", c.AccessKeyID), slog.Time("expires", c.Expires)) +} + +// AWSCredentialSource supplies [AWSCredentials]. The client asks before +// each AWS KMS request, so a source that fetches should cache, as the AWS +// SDK's chain does. It is called with the client's lock held; it must not +// call the client. +type AWSCredentialSource interface { + AWSCredentials(ctx context.Context) (AWSCredentials, error) +} + +// AWSCredentialsFunc adapts a function to an [AWSCredentialSource]. With +// the AWS SDK for Go v2 (github.com/aws/aws-sdk-go-v2/config), whose +// credentials cache refreshes before expiry: +// +// cfg, err := config.LoadDefaultConfig(ctx) +// // ... +// creds := encrypt.AWSCredentialsFunc(func(ctx context.Context) (encrypt.AWSCredentials, error) { +// c, err := cfg.Credentials.Retrieve(ctx) +// return encrypt.AWSCredentials{ +// AccessKeyID: c.AccessKeyID, SecretAccessKey: c.SecretAccessKey, +// SessionToken: c.SessionToken, Expires: c.Expires, +// }, err +// }) +type AWSCredentialsFunc func(ctx context.Context) (AWSCredentials, error) + +// AWSCredentials calls f. +func (f AWSCredentialsFunc) AWSCredentials(ctx context.Context) (AWSCredentials, error) { + return f(ctx) +} + +// AWSCredentialsFromEnv reads AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and +// AWS_SESSION_TOKEN on each request. Without the first two it fails with +// [ErrNoKeyVendorCredentials]. For anything else (a role, IRSA, IMDS, SSO) +// use the AWS SDK's chain through [AWSCredentialsFunc]. +func AWSCredentialsFromEnv() AWSCredentialSource { + return AWSCredentialsFunc(func(context.Context) (AWSCredentials, error) { + c := AWSCredentials{ + AccessKeyID: os.Getenv("AWS_ACCESS_KEY_ID"), + SecretAccessKey: os.Getenv("AWS_SECRET_ACCESS_KEY"), + SessionToken: os.Getenv("AWS_SESSION_TOKEN"), + } + if c.AccessKeyID == "" || c.SecretAccessKey == "" { + return AWSCredentials{}, fmt.Errorf("%w: set AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY", ErrNoKeyVendorCredentials) + } + return c, nil + }) +} + +// AccessToken is a short-lived bearer token for a key vendor backend: a +// Google OAuth 2.0 access token, a Microsoft Entra ID access token, or a +// Vault client token. +type AccessToken struct { + Token string + // Expires is when the token expires. Zero means unknown; the guest + // refuses a token whose expiry has passed. + Expires time.Time +} + +// String hides the token. +func (t AccessToken) String() string { + return fmt.Sprintf("encrypt.AccessToken{Token: , Expires: %s}", t.Expires) +} + +// GoString hides it from %#v too. +func (t AccessToken) GoString() string { return t.String() } + +// LogValue hides it from slog. +func (t AccessToken) LogValue() slog.Value { + return slog.GroupValue(slog.Time("expires", t.Expires)) +} + +// AccessTokenSource supplies an [AccessToken] for scope. The client asks +// before each vendor request, so a source that fetches should cache. Azure +// asks for the scope of the vault the key is in; Google Cloud asks for +// https://www.googleapis.com/auth/cloudkms; Vault asks with an empty scope. +// It is called with the client's lock held; it must not call the client. +type AccessTokenSource interface { + AccessToken(ctx context.Context, scope string) (AccessToken, error) +} + +// AccessTokenFunc adapts a function to an [AccessTokenSource]. With azidentity +// (github.com/Azure/azure-sdk-for-go/sdk/azidentity), whose credentials +// cache tokens: +// +// cred, err := azidentity.NewDefaultAzureCredential(nil) +// // ... +// tokens := encrypt.AccessTokenFunc(func(ctx context.Context, scope string) (encrypt.AccessToken, error) { +// t, err := cred.GetToken(ctx, policy.TokenRequestOptions{Scopes: []string{scope}}) +// return encrypt.AccessToken{Token: t.Token, Expires: t.ExpiresOn}, err +// }) +// +// With the Vault API client (github.com/hashicorp/vault/api), after a +// login method has set its token: +// +// tokens := encrypt.AccessTokenFunc(func(context.Context, string) (encrypt.AccessToken, error) { +// return encrypt.AccessToken{Token: vault.Token()}, nil +// }) +type AccessTokenFunc func(ctx context.Context, scope string) (AccessToken, error) + +// AccessToken calls f. +func (f AccessTokenFunc) AccessToken(ctx context.Context, scope string) (AccessToken, error) { + return f(ctx, scope) +} + +// OAuth2AccessTokens adapts a golang.org/x/oauth2 token source, such as +// Google's default credentials, to an [AccessTokenSource]. The scope is +// the source's own, set when it was made, so the scope asked for is not +// passed on: +// +// creds, err := google.FindDefaultCredentials(ctx, "https://www.googleapis.com/auth/cloudkms") +// // ... +// tokens := encrypt.OAuth2AccessTokens(creds.TokenSource) +// +// Wrap a source that fetches in oauth2.ReuseTokenSource, as +// google.FindDefaultCredentials does, so that each request does not fetch. +func OAuth2AccessTokens(source oauth2.TokenSource) AccessTokenSource { + return AccessTokenFunc(func(context.Context, string) (AccessToken, error) { + if source == nil { + return AccessToken{}, errors.New("encrypt: nil oauth2 token source") + } + t, err := source.Token() + if err != nil { + return AccessToken{}, err + } + return AccessToken{Token: t.AccessToken, Expires: t.Expiry}, nil + }) +} + +// StaticAccessToken is a source that always gives token, with no expiry: +// for a token from `gcloud auth print-access-token`, a Vault token in +// VAULT_TOKEN, or a test. It is used until the vendor refuses it. +func StaticAccessToken(token string) AccessTokenSource { + return AccessTokenFunc(func(context.Context, string) (AccessToken, error) { + return AccessToken{Token: token}, nil + }) +} + +// vendorCredential asks the backend's source for the credential the guest +// asked for, and renders it as the guest reads it. The result holds a +// secret; the caller wipes it. +func (s *vendorSpec) vendorCredential(ctx context.Context, vendor uint32, scope string) ([]byte, error) { + if vendor != s.id { + return nil, fmt.Errorf("encrypt: the guest asked for a credential of key vendor %d, but the client's is %s", vendor, s.name) + } + var fields vcvalue.Object + var expires time.Time + switch { + case s.aws != nil: + c, err := s.aws.AWSCredentials(ctx) + if err != nil { + return nil, err + } + if c.AccessKeyID == "" || c.SecretAccessKey == "" { + return nil, fmt.Errorf("%w: the AWS credential source returned no access key id or secret", ErrNoKeyVendorCredentials) + } + fields = vcvalue.Object{ + {Key: "access_key_id", Value: c.AccessKeyID}, + {Key: "secret_access_key", Value: c.SecretAccessKey}, + } + if c.SessionToken != "" { + fields = append(fields, vcvalue.Field{Key: "session_token", Value: c.SessionToken}) + } + expires = c.Expires + case s.token != nil: + t, err := s.token.AccessToken(ctx, scope) + if err != nil { + return nil, err + } + if t.Token == "" { + return nil, fmt.Errorf("%w: the %s token source returned an empty token", ErrNoKeyVendorCredentials, s.name) + } + fields = vcvalue.Object{{Key: "token", Value: t.Token}} + expires = t.Expires + default: + return nil, fmt.Errorf("encrypt: the %s key backend takes no credentials", s.name) + } + if !expires.IsZero() { + fields = append(fields, vcvalue.Field{Key: "expires_at", Value: expires.Unix()}) + } + return vcffi.Marshal(fields) +} diff --git a/languages/golang/encrypt/keyvendor_test.go b/languages/golang/encrypt/keyvendor_test.go new file mode 100644 index 0000000000..457a2cf40a --- /dev/null +++ b/languages/golang/encrypt/keyvendor_test.go @@ -0,0 +1,644 @@ +package encrypt_test + +import ( + "bytes" + "context" + "crypto/aes" + "crypto/cipher" + "crypto/rand" + "crypto/sha256" + "encoding/base64" + "encoding/binary" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "io" + "log/slog" + "net/http" + "reflect" + "strings" + "sync" + "testing" + "time" + + "github.com/cipherstash/stack/languages/golang/encrypt" + "github.com/cipherstash/stack/languages/golang/encrypt/internal/testusers" +) + +// Hermetic tests of a client with a key vendor backend (CIP-5311): the +// deterministic-kms test build stands in for ZeroKMS, the backend is the +// static key or AWS KMS behind a fake RoundTripper, and its credentials +// come back through the real step loop. + +// The key vendor and key mode bytes of the keyset record (CIP-5306). +const ( + recordAWS = 1 + recordStatic = 5 + modeWrapped = 1 + modeDerived = 2 +) + +// frame is stack-kms-vendor's framing: the piece count, then each piece's +// length and bytes, all lengths u64 little-endian. Written again here, so a +// record this test builds is a second implementation's check of the format. +func frame(pieces ...[]byte) []byte { + out := binary.LittleEndian.AppendUint64(nil, uint64(len(pieces))) + for _, p := range pieces { + out = binary.LittleEndian.AppendUint64(out, uint64(len(p))) + out = append(out, p...) + } + return out +} + +// recordKeyAAD is the associated data a keyset record's key is wrapped +// under. +func recordKeyAAD(id [16]byte, vendor, mode byte) []byte { + purpose := "root-key" + if mode == modeWrapped { + purpose = "index-key" + } + return frame([]byte("stack-kms-vendor/keyset-record-key/v1"), []byte(purpose), id[:], []byte{vendor}, []byte{mode}) +} + +// keysetRecord renders a keyset record's text form. +func keysetRecord(id [16]byte, vendor, mode byte, name, keyRef string, wrapped []byte) string { + b := []byte{1, vendor, mode} + b = append(b, id[:]...) + b = append(b, byte(len(name))) + b = append(b, name...) + b = binary.BigEndian.AppendUint16(b, uint16(len(keyRef))) + b = append(b, keyRef...) + b = binary.BigEndian.AppendUint16(b, uint16(len(wrapped))) + b = append(b, wrapped...) + return "stack-keyset:" + base64.RawURLEncoding.EncodeToString(b) +} + +// staticWrap wraps key as the static key backend does: version 1, a 12-byte +// nonce, then AES-256-GCM under staticKey with aad. +func staticWrap(t *testing.T, staticKey, key, aad []byte) []byte { + t.Helper() + block, err := aes.NewCipher(staticKey) + if err != nil { + t.Fatal(err) + } + gcm, err := cipher.NewGCM(block) + if err != nil { + t.Fatal(err) + } + nonce := make([]byte, gcm.NonceSize()) + if _, err := rand.Read(nonce); err != nil { + t.Fatal(err) + } + return append(append([]byte{1}, nonce...), gcm.Seal(nil, nonce, key, aad)...) +} + +func randomID(t *testing.T) [16]byte { + t.Helper() + var id [16]byte + if _, err := rand.Read(id[:]); err != nil { + t.Fatal(err) + } + return id +} + +// vendorClient is a deterministic client with v, or a skip when the test +// build is absent. +func vendorClient(t *testing.T, rt http.RoundTripper, v encrypt.KeyVendor, records ...string) *encrypt.Client { + t.Helper() + c, err := encrypt.NewDeterministicVendorClient(context.Background(), testSeed, rt, v, records...) + if errors.Is(err, encrypt.ErrDeterministicGuestNotBuilt) { + t.Skip(err) + } + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = c.Close() }) + return c +} + +// roundTrip seals people under cipher and opens them through cipher and the +// client, and returns what was sealed. +func roundTrip(t *testing.T, c *encrypt.Client, cipher *encrypt.Cipher) []testusers.EncryptedUser { + t.Helper() + ctx := context.Background() + encrypted, err := testusers.Encrypt(ctx, cipher, people) + if err != nil { + t.Fatal(err) + } + want := append([]testusers.User(nil), people...) + want[0].Internal = "" + for _, d := range []encrypt.Decrypter{cipher, c} { + back, err := testusers.Decrypt(ctx, d, encrypted) + if err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(back, want) { + t.Fatalf("round trip: got %+v", back) + } + } + return encrypted +} + +func TestStaticKeyVendorRoundTrip(t *testing.T) { + staticKey := bytes.Repeat([]byte{0x42}, 32) + rootKey := bytes.Repeat([]byte{0x07}, 32) + id := randomID(t) + record := keysetRecord(id, recordStatic, modeDerived, "local", "dev-key", staticWrap(t, staticKey, rootKey, recordKeyAAD(id, recordStatic, modeDerived))) + + key := append([]byte(nil), staticKey...) + c := vendorClient(t, refusing{}, encrypt.StaticKey{KeyRef: "dev-key", Key: key}, record) + if !bytes.Equal(key, make([]byte, 32)) { + t.Error("the static key was not wiped") + } + ctx := context.Background() + + vendor := c.Keyset(encrypt.KeysetName("local")) + got, err := vendor.KeysetID(ctx) + if err != nil { + t.Fatal(err) + } + if got != encrypt.KeysetID(id) { + t.Fatalf("keyset id %s, want %s", got, encrypt.KeysetID(id)) + } + sealed := roundTrip(t, c, vendor) + // A vendor keyset seals v2 leaves; the router keeps the default keyset + // on the ZeroKMS stand-in, which seals v1. + if v := sealed[0].Email.Ciphertext[0]; v != 2 { + t.Errorf("a vendor leaf has format version %d, want 2", v) + } + zerokms := roundTrip(t, c, c.DefaultKeyset()) + if v := zerokms[0].Email.Ciphertext[0]; v != 1 { + t.Errorf("a ZeroKMS leaf has format version %d, want 1", v) + } + + // A keyset-bound cipher refuses the other backend's rows before any key + // is asked for. + if _, err := testusers.Decrypt(ctx, c.DefaultKeyset(), sealed); !errors.Is(err, encrypt.ErrForeignKeyset) { + t.Errorf("default keyset opening a vendor row: %v, want ErrForeignKeyset", err) + } + + // Equal plaintexts give equal terms under the vendor keyset too, and + // they differ from the ZeroKMS keyset's. + if !sealed[1].Age.Equality.Equal(sealed[2].Age.Equality) || sealed[1].Age.Equality.Equal(zerokms[1].Age.Equality) { + t.Error("vendor equality terms do not follow the plaintext and the keyset") + } + + // The same record under another static key does not open. + other := vendorClient(t, refusing{}, encrypt.StaticKey{KeyRef: "dev-key", Key: bytes.Repeat([]byte{0x43}, 32)}, record) + if _, err := testusers.Encrypt(ctx, other.Keyset(encrypt.KeysetName("local")), people); !errors.Is(err, encrypt.ErrAuthentication) { + t.Errorf("a wrong static key: %v, want ErrAuthentication", err) + } +} + +// refusing is a RoundTripper for a backend that must make no request. +type refusing struct{} + +func (refusing) RoundTrip(*http.Request) (*http.Response, error) { + return nil, errors.New("this test makes no HTTP request") +} + +// fakeKMS answers the AWS KMS JSON API for the three operations the client +// uses. A "ciphertext blob" is the key and the SHA-256 of the encryption +// context's cipherstash:aad, so a decrypt under another context fails as +// AWS KMS fails it. It checks that every request is SigV4-signed with the +// host's credentials. +type fakeKMS struct { + mu sync.Mutex + calls map[string]int + refuse bool + failures []string +} + +func (f *fakeKMS) fail(format string, args ...any) { + f.failures = append(f.failures, fmt.Sprintf(format, args...)) +} + +func (f *fakeKMS) RoundTrip(req *http.Request) (*http.Response, error) { + f.mu.Lock() + defer f.mu.Unlock() + body, err := io.ReadAll(req.Body) + _ = req.Body.Close() + if err != nil { + return nil, err + } + op := strings.TrimPrefix(req.Header.Get("X-Amz-Target"), "TrentService.") + if f.calls == nil { + f.calls = map[string]int{} + } + f.calls[op]++ + if req.URL.String() != "https://kms.ap-southeast-2.amazonaws.com/" { + f.fail("URL %s", req.URL) + } + if auth := req.Header.Get("Authorization"); !strings.HasPrefix(auth, "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/") || !strings.Contains(auth, "/ap-southeast-2/kms/aws4_request") { + f.fail("authorization %q", auth) + } + if token := req.Header.Get("X-Amz-Security-Token"); token != "fixture-session-token" { + f.fail("session token %q", token) + } + if f.refuse { + return response(400, `{"__type":"AccessDeniedException","message":"not allowed"}`), nil + } + var in struct { + KeyID string `json:"KeyId"` + CiphertextBlob []byte `json:"CiphertextBlob"` + EncryptionContext map[string]string `json:"EncryptionContext"` + } + if err := json.Unmarshal(body, &in); err != nil { + f.fail("body: %v", err) + return response(400, `{"__type":"ValidationException","message":"bad body"}`), nil + } + if in.KeyID != awsKeyARN { + f.fail("key %q", in.KeyID) + } + bound := sha256.Sum256([]byte(in.EncryptionContext["cipherstash:aad"])) + switch op { + case "GenerateDataKey": + key := make([]byte, 32) + _, _ = rand.Read(key) + blob := append(append([]byte(nil), key...), bound[:]...) + return response(200, fmt.Sprintf(`{"KeyId":%q,"Plaintext":%q,"CiphertextBlob":%q}`, awsKeyARN, base64.StdEncoding.EncodeToString(key), base64.StdEncoding.EncodeToString(blob))), nil + case "Decrypt": + if len(in.CiphertextBlob) != 64 || !bytes.Equal(in.CiphertextBlob[32:], bound[:]) { + return response(400, `{"__type":"InvalidCiphertextException","message":""}`), nil + } + return response(200, fmt.Sprintf(`{"KeyId":%q,"Plaintext":%q}`, awsKeyARN, base64.StdEncoding.EncodeToString(in.CiphertextBlob[:32]))), nil + } + f.fail("operation %q", op) + return response(400, `{"__type":"UnsupportedOperationException","message":""}`), nil +} + +func (f *fakeKMS) count(op string) int { + f.mu.Lock() + defer f.mu.Unlock() + return f.calls[op] +} + +func response(status int, body string) *http.Response { + return &http.Response{ + StatusCode: status, + Header: http.Header{"Content-Type": {"application/x-amz-json-1.1"}}, + Body: io.NopCloser(strings.NewReader(body)), + ContentLength: int64(len(body)), + } +} + +const awsKeyARN = "arn:aws:kms:ap-southeast-2:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab" + +// awsRecord is a keyset record whose key the fake KMS unwraps: the key and +// the digest of the encryption context the client sends for it. +func awsRecord(t *testing.T, mode byte, name string) string { + t.Helper() + id := randomID(t) + aad := base64.RawURLEncoding.EncodeToString(recordKeyAAD(id, recordAWS, mode)) + bound := sha256.Sum256([]byte(aad)) + key := make([]byte, 32) + _, _ = rand.Read(key) + return keysetRecord(id, recordAWS, mode, name, awsKeyARN, append(key, bound[:]...)) +} + +// countingCredentials is the host's credential source: fixed temporary +// credentials, counted. +type countingCredentials struct { + mu sync.Mutex + asked int + err error +} + +func (c *countingCredentials) AWSCredentials(context.Context) (encrypt.AWSCredentials, error) { + c.mu.Lock() + defer c.mu.Unlock() + c.asked++ + if c.err != nil { + return encrypt.AWSCredentials{}, c.err + } + return encrypt.AWSCredentials{ + AccessKeyID: "AKIDEXAMPLE", + SecretAccessKey: "fixture-secret-access-key", + SessionToken: "fixture-session-token", + Expires: time.Now().Add(time.Hour), + }, nil +} + +func TestAWSKMSVendorThroughTheHostTransport(t *testing.T) { + ctx := context.Background() + for _, tc := range []struct { + name string + mode byte + // The vendor calls one Encrypt of len(people) rows makes, by + // operation. + sealCalls map[string]int + }{ + // Derived data keys: one unwrap of the root key, cached for its TTL. + {"derived", modeDerived, map[string]int{"Decrypt": 1}}, + // Wrapped data keys: the index key once per settle, and one + // GenerateDataKey per sealed field of every row (3 per row). + {"wrapped", modeWrapped, map[string]int{"Decrypt": 1, "GenerateDataKey": 3 * len(people)}}, + } { + t.Run(tc.name, func(t *testing.T) { + kms := &fakeKMS{} + creds := &countingCredentials{} + c := vendorClient(t, kms, encrypt.AWSKMS{Region: "ap-southeast-2", Credentials: creds}, awsRecord(t, tc.mode, "payments")) + cipher := c.Keyset(encrypt.KeysetName("payments")) + if _, err := testusers.Encrypt(ctx, cipher, people); err != nil { + t.Fatal(err) + } + for op, want := range tc.sealCalls { + if got := kms.count(op); got != want { + t.Errorf("%s calls: %d, want %d", op, got, want) + } + } + roundTrip(t, c, cipher) + if len(kms.failures) > 0 { + t.Fatalf("the fake KMS saw: %s", strings.Join(kms.failures, "; ")) + } + calls := 0 + for _, n := range kms.calls { + calls += n + } + // The guest asks for a credential before each vendor request, and + // the client answers it from the source once per operation: an + // operation of many requests (a batch of wrapped keys) asks once. + if creds.asked < 1 || creds.asked > calls { + t.Errorf("the credential source was asked %d times for %d requests", creds.asked, calls) + } + if tc.mode == modeWrapped && creds.asked >= calls { + t.Errorf("the credential source was asked %d times for %d requests, not once per operation", creds.asked, calls) + } + }) + } +} + +func TestAWSKMSVendorFailures(t *testing.T) { + ctx := context.Background() + + // AWS KMS refuses: ErrForbidden, with the vendor in the diagnostic. + kms := &fakeKMS{refuse: true} + c := vendorClient(t, kms, encrypt.AWSKMS{Region: "ap-southeast-2", Credentials: &countingCredentials{}}, awsRecord(t, modeDerived, "payments")) + _, err := testusers.Encrypt(ctx, c.Keyset(encrypt.KeysetName("payments")), people) + if !errors.Is(err, encrypt.ErrForbidden) { + t.Fatalf("refused: %v, want ErrForbidden", err) + } + d := encrypt.WantDiagnostic(t, err, "stack_kms::backend") + if d.Fields["vendor"] != "aws-kms" { + t.Errorf("vendor field %v", d.Fields["vendor"]) + } + + // The host's credential source fails: the guest reports a transport + // failure, and the source's own error is attached for errors.Is. No + // request is sent. + errSource := errors.New("fixture credential source is down") + kms = &fakeKMS{} + c = vendorClient(t, kms, encrypt.AWSKMS{Region: "ap-southeast-2", Credentials: &countingCredentials{err: errSource}}, awsRecord(t, modeDerived, "payments")) + _, err = testusers.Encrypt(ctx, c.Keyset(encrypt.KeysetName("payments")), people) + if !errors.Is(err, errSource) || !errors.Is(err, encrypt.ErrTransport) { + t.Fatalf("source failure: %v, want the source's error and ErrTransport", err) + } + if kms.count("Decrypt") != 0 { + t.Error("a request was sent without credentials") + } + // No secret in the error text. + if strings.Contains(err.Error(), "fixture-secret") { + t.Errorf("the error holds a secret: %v", err) + } +} + +func TestKeyVendorOptionsAreCheckedBeforeAnythingResolves(t *testing.T) { + ctx := context.Background() + aws := encrypt.AWSKMS{Region: "us-east-1", Credentials: encrypt.AWSCredentialsFromEnv()} + for _, tc := range []struct { + name string + opts []encrypt.ClientOption + }{ + {"records with no vendor", []encrypt.ClientOption{encrypt.WithKeysetRecords("stack-keyset:AQ")}}, + {"a vendor with no records", []encrypt.ClientOption{encrypt.WithKeyVendor(aws)}}, + {"no region", []encrypt.ClientOption{encrypt.WithKeyVendor(encrypt.AWSKMS{Credentials: aws.Credentials}), encrypt.WithKeysetRecords("r")}}, + {"no credentials", []encrypt.ClientOption{encrypt.WithKeyVendor(encrypt.AWSKMS{Region: "us-east-1"}), encrypt.WithKeysetRecords("r")}}, + {"no Vault address", []encrypt.ClientOption{encrypt.WithKeyVendor(encrypt.VaultTransit{Token: encrypt.StaticAccessToken("t")}), encrypt.WithKeysetRecords("r")}}, + {"no Google token", []encrypt.ClientOption{encrypt.WithKeyVendor(encrypt.GoogleCloudKMS{}), encrypt.WithKeysetRecords("r")}}, + {"no Azure token", []encrypt.ClientOption{encrypt.WithKeyVendor(encrypt.AzureKeyVault{}), encrypt.WithKeysetRecords("r")}}, + {"a short static key", []encrypt.ClientOption{encrypt.WithKeyVendor(encrypt.StaticKey{KeyRef: "k", Key: make([]byte, 16)}), encrypt.WithKeysetRecords("r")}}, + } { + t.Run(tc.name, func(t *testing.T) { + // No credentials are given: a check that ran after resolving + // would fail with ErrNoCredentials or a guest error instead. + _, err := encrypt.NewClient(ctx, tc.opts...) + if !errors.Is(err, encrypt.ErrEncoding) { + t.Fatalf("%v, want ErrEncoding", err) + } + }) + } + + // A refused config still consumes a static key, and so does one a + // later WithKeyVendor replaced. + first, second := bytes.Repeat([]byte{1}, 32), bytes.Repeat([]byte{2}, 32) + _, err := encrypt.NewClient(ctx, + encrypt.WithKeyVendor(encrypt.StaticKey{KeyRef: "a", Key: first}), + encrypt.WithKeyVendor(encrypt.StaticKey{KeyRef: "b", Key: second})) + if !errors.Is(err, encrypt.ErrEncoding) { + t.Fatalf("%v, want ErrEncoding", err) + } + if !bytes.Equal(first, make([]byte, 32)) || !bytes.Equal(second, make([]byte, 32)) { + t.Error("a static key was not wiped") + } +} + +func TestKeyVendorRecordsAreCheckedAtNewClient(t *testing.T) { + staticKey := bytes.Repeat([]byte{0x42}, 32) + for _, tc := range []struct { + name, code string + record string + }{ + {"another vendor's record", "stack_kms_vendor::foreign_vendor", awsRecord(t, modeDerived, "payments")}, + {"wrapped data keys under the static key", "stack_kms_vendor::mode_unsupported", keysetRecord(randomID(t), recordStatic, modeWrapped, "", "dev-key", []byte{1})}, + {"not a record", "stack_kms_vendor::record_prefix", "payments"}, + } { + t.Run(tc.name, func(t *testing.T) { + _, err := encrypt.NewDeterministicVendorClient(context.Background(), testSeed, refusing{}, + encrypt.StaticKey{KeyRef: "dev-key", Key: append([]byte(nil), staticKey...)}, tc.record) + if errors.Is(err, encrypt.ErrDeterministicGuestNotBuilt) { + t.Skip(err) + } + if !errors.Is(err, encrypt.ErrEncoding) { + t.Fatalf("%v, want ErrEncoding", err) + } + encrypt.WantCode(t, err, tc.code) + }) + } +} + +func TestVendorCredentialsDoNotPrintTheirSecrets(t *testing.T) { + c := encrypt.AWSCredentials{AccessKeyID: "AKIDEXAMPLE", SecretAccessKey: "fixture-secret", SessionToken: "fixture-session"} + tok := encrypt.AccessToken{Token: "fixture-token"} + for _, s := range []string{fmt.Sprint(c), fmt.Sprintf("%+v", c), fmt.Sprintf("%#v", c), fmt.Sprint(tok), fmt.Sprintf("%#v", tok)} { + if strings.Contains(s, "fixture") { + t.Errorf("printed a secret: %s", s) + } + } +} + +func TestStaticKeyDoesNotPrintItsKey(t *testing.T) { + // 0xAB prints as 171 in %v and as ab in %x. + k := encrypt.StaticKey{KeyRef: "dev-key", Key: bytes.Repeat([]byte{0xAB}, 32)} + var logged bytes.Buffer + slog.New(slog.NewTextHandler(&logged, nil)).Info("vendor", "key", k) + for _, s := range []string{fmt.Sprint(k), fmt.Sprintf("%+v", k), fmt.Sprintf("%#v", k), fmt.Sprintf("%x", k), logged.String()} { + if strings.Contains(s, "171") || strings.Contains(strings.ToLower(s), "abab") { + t.Errorf("printed the key: %s", s) + } + if !strings.Contains(s, "dev-key") && !strings.Contains(s, hex.EncodeToString([]byte("dev-key"))) { + t.Errorf("lost the key ref: %s", s) + } + } +} + +// noZeroKMSEnvironment clears every place AutoCredentials would look, so a +// test that passes proves NewClient did not look. +func noZeroKMSEnvironment(t *testing.T) { + t.Helper() + for _, name := range []string{"CS_CLIENT_ID", "CS_CLIENT_KEY", "CS_CLIENT_ACCESS_KEY", "CS_WORKSPACE_CRN", "CS_ZEROKMS_HOST", "CS_VITUR_HOST", "CS_CONFIG_PATH"} { + t.Setenv(name, "") + } + t.Setenv("HOME", t.TempDir()) +} + +// vendorOnlyClient is NewClient with a key vendor backend and no +// credentials, over the embedded guest, or a skip when it is absent. +func vendorOnlyClient(t *testing.T, opts ...encrypt.ClientOption) (*encrypt.Client, error) { + t.Helper() + if _, err := encrypt.EmbeddedGuest(); err != nil { + t.Skip(err) + } + c, err := encrypt.NewClient(context.Background(), opts...) + if err == nil { + t.Cleanup(func() { _ = c.Close() }) + } + return c, err +} + +func TestVendorOnlyClientWithAStaticKey(t *testing.T) { + noZeroKMSEnvironment(t) + ctx := context.Background() + staticKey := bytes.Repeat([]byte{0x42}, 32) + record := func(name string) (string, [16]byte) { + id := randomID(t) + root := make([]byte, 32) + _, _ = rand.Read(root) + return keysetRecord(id, recordStatic, modeDerived, name, "dev-key", staticWrap(t, staticKey, root, recordKeyAAD(id, recordStatic, modeDerived))), id + } + first, _ := record("first") + second, secondID := record("second") + vendor := func() encrypt.KeyVendor { + return encrypt.StaticKey{KeyRef: "dev-key", Key: append([]byte(nil), staticKey...)} + } + + // Two records and no default: there is no default keyset. + _, err := vendorOnlyClient(t, encrypt.WithKeyVendor(vendor()), encrypt.WithKeysetRecords(first, second)) + if !errors.Is(err, encrypt.ErrNotFound) { + t.Fatalf("two records and no default: %v, want ErrNotFound", err) + } + + c, err := vendorOnlyClient(t, encrypt.WithKeyVendor(vendor()), encrypt.WithKeysetRecords(first, second), + encrypt.WithDefaultKeyset(encrypt.KeysetName("second"))) + if err != nil { + t.Fatal(err) + } + def, err := c.DefaultKeyset().KeysetID(ctx) + if err != nil { + t.Fatal(err) + } + if def != encrypt.KeysetID(secondID) { + t.Fatalf("default keyset %s, want %s", def, encrypt.KeysetID(secondID)) + } + sealed := roundTrip(t, c, c.DefaultKeyset()) + if v := sealed[0].Email.Ciphertext[0]; v != 2 { + t.Errorf("leaf format version %d, want 2", v) + } + roundTrip(t, c, c.Keyset(encrypt.KeysetName("first"))) + + // By id works as well as by name. + byID, err := vendorOnlyClient(t, encrypt.WithKeyVendor(vendor()), encrypt.WithKeysetRecords(first, second), + encrypt.WithDefaultKeyset(encrypt.KeysetID(secondID))) + if err != nil { + t.Fatal(err) + } + if _, err := testusers.Decrypt(ctx, byID.DefaultKeyset(), sealed); err != nil { + t.Fatalf("the default keyset by id does not open its rows: %v", err) + } + + // Anything that needs ZeroKMS fails with ErrNoZeroKMS: a keyset no + // record names, and a row a ZeroKMS keyset sealed. + if _, err := c.Keyset(encrypt.KeysetName("tenant-a")).KeysetID(ctx); !errors.Is(err, encrypt.ErrNoZeroKMS) { + t.Errorf("a keyset with no record: %v, want ErrNoZeroKMS", err) + } + zerokms, err := testusers.Encrypt(ctx, deterministicClient(t).DefaultKeyset(), people) + if err != nil { + t.Fatal(err) + } + if _, err := testusers.Decrypt(ctx, c, zerokms); !errors.Is(err, encrypt.ErrNoZeroKMS) { + t.Errorf("a ZeroKMS row: %v, want ErrNoZeroKMS", err) + } +} + +func TestVendorOnlyClientWithAWSKMS(t *testing.T) { + noZeroKMSEnvironment(t) + kms := &fakeKMS{} + creds := &countingCredentials{} + c, err := vendorOnlyClient(t, + encrypt.WithKeyVendor(encrypt.AWSKMS{Region: "ap-southeast-2", Credentials: creds}), + encrypt.WithKeysetRecords(awsRecord(t, modeDerived, "payments")), + encrypt.WithTransport(kms), + // Keep no root key, so each use of it unwraps it again. + encrypt.WithRootKeyTTL(0)) + if err != nil { + t.Fatal(err) + } + if n := kms.count("Decrypt"); n != 0 { + t.Errorf("NewClient made %d vendor calls, want 0", n) + } + roundTrip(t, c, c.DefaultKeyset()) + // The Encrypt uses the root key twice (the data keys, then the index + // key for the terms in the settle); each Decrypt uses it once. + if n := kms.count("Decrypt"); n != 4 { + t.Errorf("Decrypt calls: %d, want 4", n) + } + if len(kms.failures) > 0 { + t.Fatalf("the fake KMS saw: %s", strings.Join(kms.failures, "; ")) + } + // One credential per operation: the Encrypt's two vendor calls share + // one, and each Decrypt asks once. + if creds.asked != 3 { + t.Errorf("the credential source was asked %d times, want 3", creds.asked) + } +} + +func TestVendorOnlyOptionsAreChecked(t *testing.T) { + noZeroKMSEnvironment(t) + ctx := context.Background() + static := func() encrypt.KeyVendor { + return encrypt.StaticKey{KeyRef: "k", Key: make([]byte, 32)} + } + for _, tc := range []struct { + name string + opts []encrypt.ClientOption + }{ + {"a default keyset with ZeroKMS credentials", []encrypt.ClientOption{ + encrypt.WithKeyVendor(static()), encrypt.WithKeysetRecords("r"), + encrypt.WithCredentials(encrypt.AutoCredentials()), encrypt.WithDefaultKeyset(encrypt.KeysetName("a")), + }}, + {"a default keyset with no vendor", []encrypt.ClientOption{encrypt.WithDefaultKeyset(encrypt.KeysetName("a"))}}, + {"the default selector as the default", []encrypt.ClientOption{ + encrypt.WithKeyVendor(static()), encrypt.WithKeysetRecords("r"), encrypt.WithDefaultKeyset(nil), + }}, + {"a root key TTL with no vendor", []encrypt.ClientOption{encrypt.WithRootKeyTTL(time.Minute)}}, + {"a negative root key TTL", []encrypt.ClientOption{ + encrypt.WithKeyVendor(static()), encrypt.WithKeysetRecords("r"), encrypt.WithRootKeyTTL(-time.Second), + }}, + } { + t.Run(tc.name, func(t *testing.T) { + if _, err := encrypt.NewClient(ctx, tc.opts...); !errors.Is(err, encrypt.ErrEncoding) { + t.Fatalf("%v, want ErrEncoding", err) + } + }) + } +} diff --git a/languages/golang/encrypt/options.go b/languages/golang/encrypt/options.go index 7a6bd19c5b..8b0916d199 100644 --- a/languages/golang/encrypt/options.go +++ b/languages/golang/encrypt/options.go @@ -1,6 +1,9 @@ package encrypt -import "net/http" +import ( + "net/http" + "time" +) // ClientOption configures [NewClient]. Each one sets one thing; a later // option setting the same thing wins. NewClient with none is a working @@ -22,6 +25,17 @@ type clientOptions struct { transport http.RoundTripper guest []byte requireLockedMemory bool + // keyVendor is WithKeyVendor's backend; supersededVendors every earlier + // one a later WithKeyVendor replaced, whose StaticKey NewClient still + // wipes. + keyVendor KeyVendor + supersededVendors []KeyVendor + keysetRecords []string + // defaultKeyset is WithDefaultKeyset's keyset; rootKeyTTL is + // WithRootKeyTTL's, nil when not set. + defaultKeyset KeysetSelector + defaultKeysetSet bool + rootKeyTTL *time.Duration } // WithCredentials supplies the client id, the client key and the auth diff --git a/languages/golang/encrypt/step.go b/languages/golang/encrypt/step.go index 34f93350d1..7a53f81174 100644 --- a/languages/golang/encrypt/step.go +++ b/languages/golang/encrypt/step.go @@ -4,6 +4,7 @@ import ( "context" "errors" "fmt" + "time" "github.com/cipherstash/stack/languages/golang/internal/guest" "github.com/cipherstash/vitaminc/bindings/go/vcffi" @@ -18,6 +19,11 @@ import ( // runs, never across a network round trip, and a caller that gives up // between two steps aborts the operation (se_abort) while the guest is not // running at all. +// +// A key vendor backend pauses the same way, for three things: its HTTP +// requests, the short-lived credential it signs them with (answered from +// the backend's credential source, which may itself wait on the network), +// and the backoff before a retry. The guest blocks on none of them. // stepFunc runs one step of an operation in the guest, under ctx. type stepFunc func(ctx context.Context, inst *instance) ([]byte, error) @@ -29,14 +35,35 @@ type ioStep struct { requests []ioRequest } -// ioRequest is one HTTP request the guest needs performed. The headers -// carry the bearer token and the body key-material contexts, so both are -// wiped once performed. +// ioRequest is one thing the guest needs done: an HTTP request performed +// (method and url set), a key vendor credential fetched (credential set), +// or a wait sat out (wait set). An HTTP request's headers carry the bearer +// token and its body key-material contexts, so both are wiped once +// performed. type ioRequest struct { method, url string headers, body []byte + // credential is the key vendor credential the guest asks for. + credential *credentialRequest + // wait is how long the guest asks to wait, when it asks. + wait *time.Duration } +// credentialRequest is a key vendor credential the guest asks for: the +// vendor's KeyVendor byte and the OAuth scope, empty when it has none. +type credentialRequest struct { + vendor uint32 + scope string +} + +// maxScopeBytes bounds the scope of a credential request. A real OAuth +// scope is well under it. +const maxScopeBytes = 1024 + +// maxWait bounds a wait the guest asks for. A vendor client's backoff is +// far under it; a longer one is cut to it rather than hold a caller. +const maxWait = 30 * time.Second + func (s *ioStep) wipe() { for _, r := range s.requests { wipe(r.headers) @@ -123,6 +150,23 @@ func decodeRequest(item any) (ioRequest, bool) { if !ok { return ioRequest{}, false } + if len(obj) == 1 { + switch obj[0].Key { + case "credential": + c, ok := decodeCredentialRequest(obj[0].Value) + return ioRequest{credential: c}, ok + case "wait_ms": + ms, ok := obj[0].Value.(uint64) + if !ok { + return ioRequest{}, false + } + wait := maxWait + if ms < uint64(maxWait/time.Millisecond) { + wait = time.Duration(ms) * time.Millisecond //nolint:gosec // bounded by maxWait above + } + return ioRequest{wait: &wait}, true + } + } var r ioRequest for _, f := range obj { switch f.Key { @@ -151,11 +195,59 @@ func decodeRequest(item any) (ioRequest, bool) { return r, true } +// decodeCredentialRequest reads `{"vendor": u32, "scope": string}`. +func decodeCredentialRequest(value any) (*credentialRequest, bool) { + obj, ok := value.(vcvalue.Object) + if !ok { + return nil, false + } + var c credentialRequest + var sawVendor, sawScope bool + for _, f := range obj { + switch f.Key { + case "vendor": + c.vendor, ok = f.Value.(uint32) + sawVendor = true + case "scope": + c.scope, ok = f.Value.(string) + sawScope = true + default: + ok = false + } + if !ok { + return nil, false + } + } + if !sawVendor || !sawScope || len(c.scope) > maxScopeBytes { + return nil, false + } + return &c, true +} + +// operation is what the host keeps for one operation across its steps: the +// key vendor credentials it has fetched for it, so that an operation that +// makes several vendor requests asks the source once per vendor and scope, +// and why the source last failed, which the guest sees only as a failed +// request. The credentials are wiped when the operation ends. +type operation struct { + credentials map[credentialRequest][]byte + credErr error +} + +// end wipes the credentials the operation fetched. +func (op *operation) end() { + for _, c := range op.credentials { + wipe(c) + } + op.credentials = nil +} + // performStep performs every request of a step, in order, and encodes the // responses for se_resume: `[{"status": i32, "headers": bytes, "body": -// bytes}, ...]`. The responses can carry wrapped keys: the caller wipes the -// encoding once the guest has it, and every other copy is wiped here. -func (t *transport) performStep(ctx context.Context, step *ioStep) ([]byte, error) { +// bytes}, ...]`. The responses can carry wrapped keys or a credential: the +// caller wipes the encoding once the guest has it, and every other copy is +// wiped here. +func (t *transport) performStep(ctx context.Context, step *ioStep, op *operation) ([]byte, error) { defer step.wipe() responses := make([]any, 0, len(step.requests)) var bodies [][]byte @@ -165,7 +257,16 @@ func (t *transport) performStep(ctx context.Context, step *ioStep) ([]byte, erro } }() for _, r := range step.requests { - status, headers, body := t.perform(ctx, r.method, r.url, r.headers, r.body) + var status int32 + var headers, body []byte + switch { + case r.credential != nil: + status, body = t.credential(ctx, *r.credential, op) + case r.wait != nil: + status = t.sit(ctx, *r.wait) + default: + status, headers, body = t.perform(ctx, r.method, r.url, r.headers, r.body) + } if headers == nil { headers = []byte{} } @@ -181,3 +282,44 @@ func (t *transport) performStep(ctx context.Context, step *ioStep) ([]byte, erro } return vcffi.Marshal(responses) } + +// credential answers a credential request: status 0 and the credential, +// encoded for the guest, or transportFailed and the error text. The +// credential is fetched once per operation, vendor and scope, outside the +// client's lock, under the caller's context; the body returned is a copy +// the caller wipes. +func (t *transport) credential(ctx context.Context, req credentialRequest, op *operation) (int32, []byte) { + if c, ok := op.credentials[req]; ok { + return 0, append([]byte(nil), c...) + } + if t.vendor == nil { + op.credErr = errors.New("encrypt: the guest asked for a key vendor credential, but the client has no key vendor backend") + return transportFailed, []byte(op.credErr.Error()) + } + cred, err := t.vendor.vendorCredential(ctx, req.vendor, req.scope) + if err != nil { + op.credErr = err + // The guest reads the text; it names the failure, never a secret, + // since the source's errors carry none. + return transportFailed, []byte(err.Error()) + } + if op.credentials == nil { + op.credentials = make(map[credentialRequest][]byte) + } + op.credentials[req] = cred + return 0, append([]byte(nil), cred...) +} + +// sit waits d, or until ctx ends: a retry's backoff, sat out with the +// client free for other callers. Status 0 once it has passed; a caller +// that gives up meanwhile ends the operation in drive, between steps. +func (t *transport) sit(ctx context.Context, d time.Duration) int32 { + timer := time.NewTimer(d) + defer timer.Stop() + select { + case <-timer.C: + return 0 + case <-ctx.Done(): + return transportFailed + } +} diff --git a/languages/golang/encrypt/step_test.go b/languages/golang/encrypt/step_test.go index 6abd5bf1ed..ce828f94fd 100644 --- a/languages/golang/encrypt/step_test.go +++ b/languages/golang/encrypt/step_test.go @@ -337,3 +337,65 @@ func marshalStep(t *testing.T, v any) []byte { } return b } + +// A key vendor credential request and a wait decode as their own kinds; a +// malformed one is refused. A wait past maxWait is cut to it. +func TestCredentialAndWaitRequestsDecode(t *testing.T) { + cred := vcvalue.Object{{Key: "credential", Value: vcvalue.Object{ + {Key: "vendor", Value: uint32(3)}, + {Key: "scope", Value: "https://vault.azure.net/.default"}, + }}} + r, ok := decodeRequest(cred) + if !ok || r.credential == nil || *r.credential != (credentialRequest{vendor: 3, scope: "https://vault.azure.net/.default"}) { + t.Fatalf("credential request decoded as %+v, %v", r, ok) + } + for _, tc := range []struct { + ms uint64 + want time.Duration + }{{250, 250 * time.Millisecond}, {0, 0}, {1 << 62, maxWait}} { + r, ok := decodeRequest(vcvalue.Object{{Key: "wait_ms", Value: tc.ms}}) + if !ok || r.wait == nil || *r.wait != tc.want { + t.Errorf("wait of %dms decoded as %+v, %v, want %v", tc.ms, r, ok, tc.want) + } + } + for name, bad := range map[string]vcvalue.Object{ + "no scope": {{Key: "credential", Value: vcvalue.Object{{Key: "vendor", Value: uint32(1)}}}}, + "string vendor": {{Key: "credential", Value: vcvalue.Object{{Key: "vendor", Value: "aws"}, {Key: "scope", Value: ""}}}}, + "extra key": {{Key: "credential", Value: vcvalue.Object{{Key: "vendor", Value: uint32(1)}, {Key: "scope", Value: ""}, {Key: "x", Value: ""}}}}, + "long scope": {{Key: "credential", Value: vcvalue.Object{{Key: "vendor", Value: uint32(1)}, {Key: "scope", Value: string(make([]byte, maxScopeBytes+1))}}}}, + "string wait": {{Key: "wait_ms", Value: "250"}}, + } { + if _, ok := decodeRequest(bad); ok { + t.Errorf("%s: accepted", name) + } + } +} + +// A wait ends early when its caller gives up, so a backoff never holds a +// caller past its context. +func TestAWaitEndsWithItsContext(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + cancel() + var tr transport + start := time.Now() + if status := tr.sit(ctx, time.Hour); status != transportFailed { + t.Errorf("status %d, want transportFailed", status) + } + if time.Since(start) > time.Second { + t.Error("the wait outlived its context") + } + if status := tr.sit(context.Background(), time.Millisecond); status != 0 { + t.Errorf("status %d, want 0", status) + } +} + +// A credential request on a client with no key vendor backend fails, and +// the operation keeps why. +func TestACredentialRequestWithNoKeyVendorFails(t *testing.T) { + var tr transport + var op operation + status, body := tr.credential(context.Background(), credentialRequest{vendor: 1}, &op) + if status != transportFailed || op.credErr == nil || len(body) == 0 { + t.Errorf("status %d, body %q, credErr %v", status, body, op.credErr) + } +} diff --git a/languages/golang/encrypt/transport.go b/languages/golang/encrypt/transport.go index d463a6b229..a43fa0034f 100644 --- a/languages/golang/encrypt/transport.go +++ b/languages/golang/encrypt/transport.go @@ -33,8 +33,10 @@ type tokenSource interface { Token(ctx context.Context) (string, error) } -// transport performs the guest's ZeroKMS requests over a RoundTripper and -// implements its token_get import over a tokenSource. One per Client. +// transport performs the guest's ZeroKMS and key vendor requests over a +// RoundTripper, answers its key vendor credential requests from that +// backend's credential source, and implements its token_get import over a +// tokenSource. One per Client. type transport struct { rt http.RoundTripper token tokenSource @@ -51,6 +53,9 @@ type transport struct { held string fetchErr error tokenErr error + // vendor is the client's key vendor backend, nil when it has none. Its + // credential source answers the guest's credential requests (step.go). + vendor *vendorSpec } // transportFailed is the status of a request that could not be performed @@ -203,6 +208,12 @@ func (b *requestBody) Close() error { // tokenGet is token_get: hand the guest the token fetched for this step. // It never waits: the fetch happened before the step (fetchToken). func (t *transport) tokenGet(ctx context.Context, m api.Module, tokenPtrOut, tokenLenOut uint32) int32 { + if t.token == nil { + // A client with a key vendor backend and no ZeroKMS has no token + // source; its guest builds no ZeroKMS client, so it never asks. + t.tokenErr = ErrNoZeroKMS + return hostFailed + } token := t.held if token == "" { t.tokenErr = t.fetchErr diff --git a/languages/golang/internal/guest/errors.go b/languages/golang/internal/guest/errors.go index 9020d2a2ad..5a53318114 100644 --- a/languages/golang/internal/guest/errors.go +++ b/languages/golang/internal/guest/errors.go @@ -91,6 +91,9 @@ var ( // A guest package's call loop takes the step, performs it and resumes; // it never reaches a caller. ErrNeedsIO = errors.New("cipherstash: the guest is waiting on the host for I/O") + // ErrNoZeroKMS is a ZeroKMS keyset asked of a client that has a key + // vendor backend and no ZeroKMS, refused before any request. + ErrNoZeroKMS = errors.New("cipherstash: this client has no ZeroKMS, only a key vendor backend") // ErrMemoryLock is guest memory that could not be locked in RAM (or, // on Linux, excluded from core dumps). A constructor returns it when // asked for locked memory and refused, and so does any later call under diff --git a/languages/golang/internal/guest/status.go b/languages/golang/internal/guest/status.go index f93e6a1d98..e9f7287535 100644 --- a/languages/golang/internal/guest/status.go +++ b/languages/golang/internal/guest/status.go @@ -39,7 +39,8 @@ const ( StatusContextMismatch = 28 // Not a failure: the operation is paused, waiting on HTTP requests the // host performs for it. See ErrNeedsIO. - StatusNeedsIO = 29 + StatusNeedsIO = 29 + StatusNoZeroKMS = 30 ) // StatusError is the sentinel a guest status decodes to. A status this host @@ -105,6 +106,8 @@ func StatusError(status uint32) error { return ErrContextMismatch case StatusNeedsIO: return ErrNeedsIO + case StatusNoZeroKMS: + return ErrNoZeroKMS default: return unrecognizedStatus(status) } diff --git a/languages/typescript/packages/auth/Cargo.toml b/languages/typescript/packages/auth/Cargo.toml index 5f14a11a72..e105cfe1a6 100644 --- a/languages/typescript/packages/auth/Cargo.toml +++ b/languages/typescript/packages/auth/Cargo.toml @@ -16,6 +16,21 @@ napi = { version = "2", features = ["async", "tokio_rt"] } napi-derive = "2" serde_json = "1" zeroize = { workspace = true } +# `stash keyset create` / `rewrap` (src/keyset.rs, CIP-5312): keyset records +# for the key vendor backends. These crates are compiled into this binary from +# the tree, by path, so their crates.io release state does not matter here. +# stack-kms-gcp has no default features: without `service-account` it has no +# `rsa` code (RUSTSEC-2023-0071), and the binding builds its own credentials +# from an access token or the metadata server. +stack-kms = { path = "../../../../packages/stack-kms", default-features = false } +stack-kms-vendor = { path = "../../../../packages/stack-kms-vendor", features = ["http"] } +stack-kms-aws = { path = "../../../../packages/stack-kms-aws" } +stack-kms-vault = { path = "../../../../packages/stack-kms-vault" } +stack-kms-gcp = { path = "../../../../packages/stack-kms-gcp", default-features = false } +stack-kms-azure = { path = "../../../../packages/stack-kms-azure" } +vitaminc = { workspace = true } +base64 = { workspace = true } +miette = { workspace = true } [dev-dependencies] # `stack-auth/test-utils` gives the Rust #[cfg(test)] unit tests the no-timeout diff --git a/languages/typescript/packages/auth/__tests__/keyset.test.ts b/languages/typescript/packages/auth/__tests__/keyset.test.ts new file mode 100644 index 0000000000..5282c07715 --- /dev/null +++ b/languages/typescript/packages/auth/__tests__/keyset.test.ts @@ -0,0 +1,120 @@ +import { afterEach, describe, expect, it } from 'vitest' + +// Runtime coverage for `keysetCreate` / `keysetRewrap` through index.js, on +// both Result arms. The static key backend needs no network, so these run +// with no vendor. The vendor paths are covered by the vendor crates' own +// fixtures; the Rust unit tests in src/keyset.rs cover the option checks. +const { keysetCreate, keysetRewrap } = + require('../index.js') as typeof import('../index') + +const KEY_A = 'AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8=' +const KEY_B = 'ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3ODk6Ozw9Pj8=' +const VARS = ['CS_TEST_KEYSET_KEY_A', 'CS_TEST_KEYSET_KEY_B'] as const + +afterEach(() => { + for (const name of VARS) delete process.env[name] +}) + +describe('keysetCreate', () => { + it('creates a static keyset record with a generated key', async () => { + const result = await keysetCreate({ + backend: 'static', + vendorKey: 'key-a', + mode: 'derived', + name: 'users', + }) + if (result.failure) throw result.failure.error + expect(result.data.record).toMatch(/^stack-keyset:[A-Za-z0-9_-]+$/) + expect(result.data).toMatchObject({ + backend: 'static', + mode: 'derived', + name: 'users', + vendorKey: 'key-a', + }) + expect(result.data.keysetId).toMatch(/^[0-9a-f-]{36}$/) + expect( + Buffer.from(result.data.generatedStaticKey ?? '', 'base64'), + ).toHaveLength(32) + }) + + it('reads a static key from the named environment variable', async () => { + process.env.CS_TEST_KEYSET_KEY_A = KEY_A + const result = await keysetCreate({ + backend: 'static', + vendorKey: 'key-a', + mode: 'derived', + staticKeyEnv: 'CS_TEST_KEYSET_KEY_A', + }) + if (result.failure) throw result.failure.error + expect(result.data.generatedStaticKey).toBeUndefined() + expect(result.data.name).toBeUndefined() + }) + + it('returns a failure, not a throw, for wrapped data keys on static', async () => { + const result = await keysetCreate({ + backend: 'static', + vendorKey: 'key-a', + mode: 'wrapped', + }) + expect(result.data).toBeUndefined() + expect(result.failure?.type).toBe('INVALID_RECORD') + expect(result.failure?.diagnostic).toBe( + 'stack_kms_vendor::mode_unsupported', + ) + expect(result.failure?.error).toBeInstanceOf(Error) + }) + + it('returns INVALID_OPTIONS for an unknown backend', async () => { + const result = await keysetCreate({ + backend: 'zerokms', + vendorKey: 'k', + mode: 'derived', + }) + expect(result.failure?.type).toBe('INVALID_OPTIONS') + expect(result.failure?.error.message).toContain( + 'aws, gcp, azure, vault, static', + ) + }) +}) + +describe('keysetRewrap', () => { + it('moves a static record to another key, keeping its id and name', async () => { + process.env.CS_TEST_KEYSET_KEY_A = KEY_A + process.env.CS_TEST_KEYSET_KEY_B = KEY_B + const created = await keysetCreate({ + backend: 'static', + vendorKey: 'key-a', + mode: 'derived', + name: 'users', + staticKeyEnv: 'CS_TEST_KEYSET_KEY_A', + }) + if (created.failure) throw created.failure.error + + const rewrapped = await keysetRewrap({ + record: created.data.record, + vendorKey: 'key-b', + staticKeyEnv: 'CS_TEST_KEYSET_KEY_A', + newStaticKeyEnv: 'CS_TEST_KEYSET_KEY_B', + }) + if (rewrapped.failure) throw rewrapped.failure.error + expect(rewrapped.data).toMatchObject({ + keysetId: created.data.keysetId, + name: 'users', + vendorKey: 'key-b', + }) + + // The old key no longer opens it. + const wrong = await keysetRewrap({ + record: rewrapped.data.record, + staticKeyEnv: 'CS_TEST_KEYSET_KEY_A', + }) + expect(wrong.failure?.type).toBe('VENDOR_ERROR') + expect(wrong.failure?.kind).toBe('InvalidCiphertext') + }) + + it('returns INVALID_RECORD for text that is not a record', async () => { + const result = await keysetRewrap({ record: 'nope' }) + expect(result.failure?.type).toBe('INVALID_RECORD') + expect(result.failure?.help).toContain('stash keyset create') + }) +}) diff --git a/languages/typescript/packages/auth/index.d.ts b/languages/typescript/packages/auth/index.d.ts index cd36baf3b0..21b2c20606 100644 --- a/languages/typescript/packages/auth/index.d.ts +++ b/languages/typescript/packages/auth/index.d.ts @@ -20,9 +20,23 @@ */ import type { Result } from "@byteslice/result"; -import type { TokenResult, AuthResult, AutoStrategyOptions } from "./native"; +import type { + TokenResult, + AuthResult, + AutoStrategyOptions, + KeysetCreateOptions, + KeysetRewrapOptions, + KeysetResult, +} from "./native"; -export type { TokenResult, AuthResult, AutoStrategyOptions }; +export type { + TokenResult, + AuthResult, + AutoStrategyOptions, + KeysetCreateOptions, + KeysetRewrapOptions, + KeysetResult, +}; /** Fields present on every `AuthFailure`. */ interface FailureBase { @@ -183,6 +197,60 @@ export declare function beginDeviceCodeFlow( /** Provision a device client in ZeroKMS after login. */ export declare function bindClientDevice(): Promise>; +/** + * A failure of {@link keysetCreate} or {@link keysetRewrap}. Discriminated by + * `type`. No message carries key material or a credential. + * + * - `INVALID_OPTIONS`: an option is missing or not accepted. + * - `STATIC_KEY`: a static key could not be read from its variable. + * - `VENDOR_CONFIG`: the vendor client could not be configured from the + * environment (for example no region, address or credential). + * - `INVALID_RECORD`: the record could not be parsed or made (`diagnostic` + * names the `stack_kms_vendor::*` code). + * - `VENDOR_ERROR`: the vendor call failed; `kind` is `NotFound`, + * `AccessDenied`, `Throttled`, `Unavailable` or `InvalidCiphertext`. + * - `REWRAP_REFUSED`: {@link keysetRewrap} named another vendor key for a + * record in the wrapped key mode. Each stored value's data key stays + * wrapped under the key that sealed it, so such a record may move only to + * a new version of its own key. No vendor call was made. + */ +export type KeysetFailure = FailureBase & { + type: + | "INVALID_OPTIONS" + | "STATIC_KEY" + | "VENDOR_CONFIG" + | "INVALID_RECORD" + | "VENDOR_ERROR" + | "REWRAP_REFUSED"; + /** The key backend, as the SDKs name it (`aws-kms`, …), when known. */ + vendor?: string; + /** The vendor failure kind, for `VENDOR_ERROR`. */ + kind?: string; + /** The `stack_kms_vendor::*` diagnostic code, for `INVALID_RECORD`. */ + diagnostic?: string; +}; + +/** + * Create a keyset in a key vendor backend (AWS KMS, Google Cloud KMS, Azure + * Key Vault, Vault Transit or the static key backend) and return its keyset + * record. ZeroKMS users never need this. The vendor's credentials come from + * its environment chain, as in the Rust and Go SDKs. Node only. + */ +export declare function keysetCreate( + options: KeysetCreateOptions, +): Promise>; + +/** + * Wrap a keyset record's key under a vendor key again, for example after a + * key rotation in the vendor. The keyset id, name and mode stay the same. + * Stored data is not re-keyed: a record in the wrapped key mode may move only + * to a new version of its own vendor key (else `REWRAP_REFUSED`); a record + * in the derived key mode may move to any key of its backend. Node only. + */ +export declare function keysetRewrap( + options: KeysetRewrapOptions, +): Promise>; + /** * Deprecated alias for {@link DeviceSessionStrategy}, exported at runtime as * `module.exports.OAuthStrategy = DeviceSessionStrategy`. diff --git a/languages/typescript/packages/auth/index.js b/languages/typescript/packages/auth/index.js index a5f4fe8ff5..6181689bc1 100644 --- a/languages/typescript/packages/auth/index.js +++ b/languages/typescript/packages/auth/index.js @@ -242,6 +242,10 @@ module.exports = { OAuthStrategy: DeviceSessionStrategy, beginDeviceCodeFlow: wrapAsync(native.beginDeviceCodeFlow), bindClientDevice: wrapAsync(native.bindClientDevice), + // Keyset records for the key vendor backends (`stash keyset`). Their + // failures use the same sentinel, with `KeysetFailure` codes. + keysetCreate: wrapAsync(native.keysetCreate), + keysetRewrap: wrapAsync(native.keysetRewrap), }; if (native.beginDeviceCodeFlowWithBaseUrl) { diff --git a/languages/typescript/packages/auth/native.d.ts b/languages/typescript/packages/auth/native.d.ts index d38d002410..0c163a481b 100644 --- a/languages/typescript/packages/auth/native.d.ts +++ b/languages/typescript/packages/auth/native.d.ts @@ -3,6 +3,93 @@ /* auto-generated by NAPI-RS */ +/** The options of [`keyset_create`]. */ +export interface KeysetCreateOptions { + /** The key backend: `aws`, `gcp`, `azure`, `vault` or `static`. */ + backend: string + /** + * The vendor key that wraps the keyset's key: an AWS KMS key ARN, a + * Cloud KMS crypto key name, an Azure Key Vault key identifier, a Vault + * Transit key name, or the reference of a static key. + */ + vendorKey: string + /** + * The key mode: `wrapped` or `derived`. The static key backend offers + * `derived` only. + */ + mode: string + /** An optional local name for the keyset. */ + name?: string + /** + * Static key backend only: the name of the environment variable that + * holds the static key (32 bytes, as base64, base64url or hex). When it + * is not given, a new static key is generated and returned. + */ + staticKeyEnv?: string +} +/** The options of [`keyset_rewrap`]. */ +export interface KeysetRewrapOptions { + /** The keyset record text (`stack-keyset:…`). */ + record: string + /** + * The vendor key to wrap under. When it is not given, the record's own + * vendor key is used, which moves the record to that key's current + * version. + */ + vendorKey?: string + /** + * Static key backend only: the environment variable that holds the + * static key the record names now. + */ + staticKeyEnv?: string + /** + * Static key backend only: the environment variable that holds the + * static key of `vendor_key`, when that is another reference. When it is + * not given, a new static key is generated and returned. + */ + newStaticKeyEnv?: string +} +/** A keyset record that [`keyset_create`] or [`keyset_rewrap`] made. */ +export interface KeysetResult { + /** The keyset record text (`stack-keyset:…`). It is not secret. */ + record: string + /** The keyset id (a UUID). */ + keysetId: string + /** The keyset name, if it has one. */ + name?: string + /** + * The key backend, as the SDKs name it (`aws-kms`, `gcp-kms`, + * `azure-key-vault`, `vault-transit` or `static`). + */ + backend: string + /** The key mode: `wrapped` or `derived`. */ + mode: string + /** The vendor key that wraps the keyset's key. */ + vendorKey: string + /** + * Static key backend only: a static key this call generated, as padded + * standard base64. **This is a secret.** It is the only copy. + */ + generatedStaticKey?: string +} +/** + * Create a keyset in a key vendor backend and return its keyset record. + * + * One vendor call. The vendor's credentials come from its environment + * chain, as in the SDKs. + */ +export declare function keysetCreate(options: KeysetCreateOptions): Promise +/** + * Wrap a keyset record's key under a vendor key again (after a rotation in + * the vendor, or to move it to another vendor key of the same backend). The + * keyset id, name and mode stay the same. Stored data is not re-keyed, so a + * record in the wrapped key mode may move only to a new version of its own + * vendor key; a move to another key fails with `REWRAP_REFUSED`, before any + * vendor call. + * + * Two vendor calls: an unwrap and a wrap. + */ +export declare function keysetRewrap(options: KeysetRewrapOptions): Promise /** * The result of a successful `getToken()` call. * diff --git a/languages/typescript/packages/auth/src/keyset.rs b/languages/typescript/packages/auth/src/keyset.rs new file mode 100644 index 0000000000..017f942f9b --- /dev/null +++ b/languages/typescript/packages/auth/src/keyset.rs @@ -0,0 +1,896 @@ +//! Keyset records for the key vendor backends: the native half of +//! `stash keyset create` and `stash keyset rewrap` (CIP-5312). +//! +//! A ZeroKMS user never needs these functions. They exist for a keyset held +//! by a key vendor backend (AWS KMS, Google Cloud KMS, Azure Key Vault, +//! HashiCorp Vault Transit, or the static key backend), which the Rust and Go +//! SDKs load from a keyset record. +//! +//! Each vendor client is the Rust one, configured from the environment the +//! way the SDKs configure it (`AwsKms::from_env` and its peers), so the +//! request signing code exists once, in Rust. No credential crosses this +//! boundary as an argument. The one secret that can come back is a static +//! key that this module generated, because the caller must store it. + +use base64::engine::general_purpose::{STANDARD, STANDARD_NO_PAD, URL_SAFE, URL_SAFE_NO_PAD}; +use base64::Engine as _; +use miette::Diagnostic; +use napi::Status; +use napi_derive::napi; +use stack_kms::{KeyMode, KeyVendor}; +use stack_kms_vendor::{ + HttpVendorTransport, HttpVendorTransportOptions, KeyBytes, KeyVendorBackend, KeysetRecord, + RecordError, StaticKeyVendor, +}; +use vitaminc::protected::{Controlled, Protected}; +use vitaminc::random::{Generatable, SafeRand}; +use vitaminc_protected::OpaqueDebug; +use zeroize::Zeroizing; + +use crate::FAILURE_SENTINEL; + +/// The length of a static key, in bytes. +const STATIC_KEY_LEN: usize = 32; + +// --------------------------------------------------------------------------- +// The napi surface +// --------------------------------------------------------------------------- + +/// The options of [`keyset_create`]. +#[napi(object)] +#[derive(Debug, Clone)] +pub struct KeysetCreateOptions { + /// The key backend: `aws`, `gcp`, `azure`, `vault` or `static`. + pub backend: String, + /// The vendor key that wraps the keyset's key: an AWS KMS key ARN, a + /// Cloud KMS crypto key name, an Azure Key Vault key identifier, a Vault + /// Transit key name, or the reference of a static key. + pub vendor_key: String, + /// The key mode: `wrapped` or `derived`. The static key backend offers + /// `derived` only. + pub mode: String, + /// An optional local name for the keyset. + pub name: Option, + /// Static key backend only: the name of the environment variable that + /// holds the static key (32 bytes, as base64, base64url or hex). When it + /// is not given, a new static key is generated and returned. + pub static_key_env: Option, +} + +/// The options of [`keyset_rewrap`]. +#[napi(object)] +#[derive(Debug, Clone)] +pub struct KeysetRewrapOptions { + /// The keyset record text (`stack-keyset:…`). + pub record: String, + /// The vendor key to wrap under. When it is not given, the record's own + /// vendor key is used, which moves the record to that key's current + /// version. + pub vendor_key: Option, + /// Static key backend only: the environment variable that holds the + /// static key the record names now. + pub static_key_env: Option, + /// Static key backend only: the environment variable that holds the + /// static key of `vendor_key`, when that is another reference. When it is + /// not given, a new static key is generated and returned. + pub new_static_key_env: Option, +} + +/// A keyset record that [`keyset_create`] or [`keyset_rewrap`] made. +#[derive(OpaqueDebug)] +#[napi(object)] +pub struct KeysetResult { + /// The keyset record text (`stack-keyset:…`). It is not secret. + pub record: String, + /// The keyset id (a UUID). + pub keyset_id: String, + /// The keyset name, if it has one. + pub name: Option, + /// The key backend, as the SDKs name it (`aws-kms`, `gcp-kms`, + /// `azure-key-vault`, `vault-transit` or `static`). + pub backend: String, + /// The key mode: `wrapped` or `derived`. + pub mode: String, + /// The vendor key that wraps the keyset's key. + pub vendor_key: String, + /// Static key backend only: a static key this call generated, as padded + /// standard base64. **This is a secret.** It is the only copy. + pub generated_static_key: Option, +} + +/// Create a keyset in a key vendor backend and return its keyset record. +/// +/// One vendor call. The vendor's credentials come from its environment +/// chain, as in the SDKs. +#[napi] +pub async fn keyset_create(options: KeysetCreateOptions) -> napi::Result { + create(options).await.map_err(KeysetError::into_napi) +} + +/// Wrap a keyset record's key under a vendor key again (after a rotation in +/// the vendor, or to move it to another vendor key of the same backend). The +/// keyset id, name and mode stay the same. Stored data is not re-keyed, so a +/// record in the wrapped key mode may move only to a new version of its own +/// vendor key; a move to another key fails with `REWRAP_REFUSED`, before any +/// vendor call. +/// +/// Two vendor calls: an unwrap and a wrap. +#[napi] +pub async fn keyset_rewrap(options: KeysetRewrapOptions) -> napi::Result { + rewrap(options).await.map_err(KeysetError::into_napi) +} + +// --------------------------------------------------------------------------- +// Errors +// --------------------------------------------------------------------------- + +/// A failure of a keyset operation. No message carries key material or a +/// credential: the vendor crates build their messages from fixed text, the +/// HTTP status and the vendor's error code (see CIP-5311 in the decision +/// log), and this module never puts a key in one. +#[derive(Debug)] +pub(crate) enum KeysetError { + /// An option is missing or has a value this module does not accept. + Options(String), + /// A static key could not be read or generated. + StaticKey(String), + /// The vendor client could not be configured from the environment. + VendorConfig { vendor: KeyVendor, message: String }, + /// The record could not be parsed, created or rewrapped. + Record(RecordError), +} + +impl KeysetError { + /// The stable code the TypeScript side narrows on. + fn code(&self) -> &'static str { + match self { + Self::Options(_) => "INVALID_OPTIONS", + Self::StaticKey(_) => "STATIC_KEY", + Self::VendorConfig { .. } => "VENDOR_CONFIG", + Self::Record(RecordError::Kms(_)) => "VENDOR_ERROR", + Self::Record(RecordError::RewrapToAnotherKey { .. }) => "REWRAP_REFUSED", + Self::Record(_) => "INVALID_RECORD", + } + } + + /// The failure envelope: `{ type, message, help?, vendor?, kind?, diagnostic? }`. + pub(crate) fn envelope(&self) -> serde_json::Value { + let mut fields = serde_json::Map::new(); + let _ = fields.insert("type".into(), self.code().into()); + let _ = fields.insert("message".into(), self.message().into()); + match self { + Self::VendorConfig { vendor, .. } => { + let _ = fields.insert("vendor".into(), vendor.name().into()); + } + Self::Record(error) => { + if let Some(code) = error.code() { + let _ = fields.insert("diagnostic".into(), code.to_string().into()); + } + if let Some(help) = error.help() { + let _ = fields.insert("help".into(), help.to_string().into()); + } + if let RecordError::Kms(stack_kms::Error::Backend { vendor, kind, .. }) = error { + let _ = fields.insert("vendor".into(), (*vendor).into()); + let _ = fields.insert("kind".into(), kind.as_str().into()); + } + } + Self::Options(_) | Self::StaticKey(_) => {} + } + serde_json::Value::Object(fields) + } + + fn message(&self) -> String { + match self { + Self::Options(message) | Self::StaticKey(message) => message.clone(), + Self::VendorConfig { vendor, message } => { + format!("Could not configure the {vendor} client from the environment: {message}") + } + Self::Record(error) => error_chain(error), + } + } + + fn into_napi(self) -> napi::Error { + napi::Error::new( + Status::GenericFailure, + format!("{FAILURE_SENTINEL}{}", self.envelope()), + ) + } +} + +impl From for KeysetError { + fn from(error: RecordError) -> Self { + Self::Record(error) + } +} + +/// An error and its sources, joined with `: `, without repeats (a +/// `#[error(transparent)]` wrapper prints its source's text). +fn error_chain(error: &(dyn std::error::Error + 'static)) -> String { + let mut text = error.to_string(); + let mut source = error.source(); + while let Some(cause) = source { + let next = cause.to_string(); + if !text.ends_with(&next) { + text.push_str(": "); + text.push_str(&next); + } + source = cause.source(); + } + text +} + +fn vendor_config(vendor: KeyVendor, error: impl std::error::Error + 'static) -> KeysetError { + KeysetError::VendorConfig { + vendor, + message: error_chain(&error), + } +} + +// --------------------------------------------------------------------------- +// Parsing the options +// --------------------------------------------------------------------------- + +/// The `--backend` values, in the order the help text lists them. +pub(crate) const BACKENDS: [(&str, KeyVendor); 5] = [ + ("aws", KeyVendor::Aws), + ("gcp", KeyVendor::Gcp), + ("azure", KeyVendor::Azure), + ("vault", KeyVendor::Vault), + ("static", KeyVendor::Static), +]; + +fn parse_backend(text: &str) -> Result { + BACKENDS + .iter() + .find(|(name, _)| *name == text) + .map(|(_, vendor)| *vendor) + .ok_or_else(|| { + KeysetError::Options(format!( + "Unknown key backend {text:?}. Use one of: aws, gcp, azure, vault, static." + )) + }) +} + +fn parse_mode(text: &str) -> Result { + KeyMode::ALL + .into_iter() + .find(|mode| mode.name() == text) + .ok_or_else(|| { + KeysetError::Options(format!( + "Unknown key mode {text:?}. Use wrapped or derived." + )) + }) +} + +fn non_empty(field: &str, value: Option) -> Result, KeysetError> { + match value { + Some(value) if value.is_empty() => Err(KeysetError::Options(format!( + "The {field} must not be empty." + ))), + other => Ok(other), + } +} + +/// Refuse a static key option on a vendor that has no static key. +fn refuse_static_options( + vendor: KeyVendor, + options: &[&Option], +) -> Result<(), KeysetError> { + if vendor != KeyVendor::Static && options.iter().any(|option| option.is_some()) { + return Err(KeysetError::Options(format!( + "A static key option is only for the static key backend, not {vendor}." + ))); + } + Ok(()) +} + +// --------------------------------------------------------------------------- +// Static keys +// --------------------------------------------------------------------------- + +/// Read a static key from the environment variable `var`. The variable holds +/// 32 bytes as base64 (standard or URL-safe, padded or not) or as 64 +/// hexadecimal characters. +pub(crate) fn read_static_key(var: &str) -> Result { + let text = Zeroizing::new(std::env::var(var).map_err(|_| { + KeysetError::StaticKey(format!( + "The environment variable {var} is not set (it must hold the static key)." + )) + })?); + decode_static_key(text.trim()).ok_or_else(|| { + KeysetError::StaticKey(format!( + "The environment variable {var} does not hold a static key: \ + use 32 bytes as base64 or as 64 hexadecimal characters." + )) + }) +} + +/// Decode a static key, or `None` when `text` is not 32 bytes in one of the +/// accepted encodings. Never returns the text in an error. +pub(crate) fn decode_static_key(text: &str) -> Option { + let bytes: Zeroizing> = + if text.len() == 2 * STATIC_KEY_LEN && text.bytes().all(|byte| byte.is_ascii_hexdigit()) { + Zeroizing::new( + (0..STATIC_KEY_LEN) + .map(|i| u8::from_str_radix(&text[2 * i..2 * i + 2], 16)) + .collect::, _>>() + .ok()?, + ) + } else { + Zeroizing::new( + [STANDARD, STANDARD_NO_PAD, URL_SAFE, URL_SAFE_NO_PAD] + .iter() + .find_map(|engine| engine.decode(text).ok())?, + ) + }; + let array: [u8; STATIC_KEY_LEN] = bytes.as_slice().try_into().ok()?; + Some(Protected::new(array)) +} + +/// A new static key from the CSPRNG, and its padded standard base64. +fn generate_static_key() -> Result<(KeyBytes, String), KeysetError> { + let failed = |_| KeysetError::StaticKey("Could not generate a static key.".into()); + let mut rng = SafeRand::from_entropy().map_err(failed)?; + let key = KeyBytes::random(&mut rng).map_err(failed)?; + let text = STANDARD.encode(key.risky_ref()); + Ok((key, text)) +} + +// --------------------------------------------------------------------------- +// The operations +// --------------------------------------------------------------------------- + +/// What to do with the backend, once it is built. +enum Operation { + Create { + vendor_key: String, + mode: KeyMode, + name: Option, + }, + Rewrap { + record: KeysetRecord, + vendor_key: String, + }, +} + +impl Operation { + async fn run(self, backend: &B) -> Result { + Ok(match self { + Self::Create { + vendor_key, + mode, + name, + } => KeysetRecord::create(backend, vendor_key, mode, name).await?, + Self::Rewrap { record, vendor_key } => record.rewrap(backend, vendor_key).await?, + }) + } +} + +pub(crate) async fn create(options: KeysetCreateOptions) -> Result { + let vendor = parse_backend(&options.backend)?; + let mode = parse_mode(&options.mode)?; + let name = non_empty("keyset name", options.name)?; + let static_key_env = non_empty("static key variable", options.static_key_env)?; + refuse_static_options(vendor, &[&static_key_env])?; + if options.vendor_key.is_empty() { + return Err(KeysetError::Options( + "The vendor key must not be empty.".into(), + )); + } + // Refused here too, so that a static key is not generated for nothing. + if vendor == KeyVendor::Static && mode != KeyMode::Derived { + return Err(RecordError::ModeUnsupported { vendor, mode }.into()); + } + let operation = Operation::Create { + vendor_key: options.vendor_key, + mode, + name, + }; + let (record, generated) = if vendor == KeyVendor::Static { + let key_ref = options_vendor_key_of(&operation); + let (key, generated) = match &static_key_env { + Some(var) => (read_static_key(var)?, None), + None => { + let (key, text) = generate_static_key()?; + (key, Some(text)) + } + }; + let backend = StaticKeyVendor::new(key_ref, key); + (operation.run(&backend).await?, generated) + } else { + (run_vendor(vendor, operation).await?, None) + }; + Ok(result(&record, generated)) +} + +pub(crate) async fn rewrap(options: KeysetRewrapOptions) -> Result { + let record = KeysetRecord::parse(options.record.trim())?; + let vendor = record.vendor(); + let vendor_key = + non_empty("vendor key", options.vendor_key)?.unwrap_or_else(|| record.key_ref().to_owned()); + let static_key_env = non_empty("static key variable", options.static_key_env)?; + let new_static_key_env = non_empty("new static key variable", options.new_static_key_env)?; + refuse_static_options(vendor, &[&static_key_env, &new_static_key_env])?; + + if vendor != KeyVendor::Static { + let record = run_vendor(vendor, Operation::Rewrap { record, vendor_key }).await?; + return Ok(result(&record, None)); + } + + let var = static_key_env.ok_or_else(|| { + KeysetError::Options( + "The record is in the static key backend: name the variable that holds its \ + static key with --key-env." + .into(), + ) + })?; + let current = read_static_key(&var)?; + let mut backend = StaticKeyVendor::new(record.key_ref(), current); + let mut generated = None; + if vendor_key != record.key_ref() { + let key = match &new_static_key_env { + Some(var) => read_static_key(var)?, + None => { + let (key, text) = generate_static_key()?; + generated = Some(text); + key + } + }; + backend = backend.with_key(vendor_key.clone(), key); + } else if new_static_key_env.is_some() { + return Err(KeysetError::Options( + "--new-key-env needs --vendor-key with another static key reference.".into(), + )); + } + let record = Operation::Rewrap { record, vendor_key } + .run(&backend) + .await?; + Ok(result(&record, generated)) +} + +fn options_vendor_key_of(operation: &Operation) -> String { + match operation { + Operation::Create { vendor_key, .. } | Operation::Rewrap { vendor_key, .. } => { + vendor_key.clone() + } + } +} + +fn result(record: &KeysetRecord, generated_static_key: Option) -> KeysetResult { + KeysetResult { + record: record.to_string(), + keyset_id: record.id().to_string(), + name: record.name().map(str::to_owned), + backend: record.vendor().name().to_owned(), + mode: record.mode().name().to_owned(), + vendor_key: record.key_ref().to_owned(), + generated_static_key, + } +} + +/// Build the vendor's client from the environment and run `operation` on it. +async fn run_vendor(vendor: KeyVendor, operation: Operation) -> Result { + match vendor { + KeyVendor::Aws => { + let kms = stack_kms_aws::DefaultAwsKms::from_env() + .map_err(|error| vendor_config(vendor, error))?; + operation.run(&kms).await + } + KeyVendor::Vault => { + let vault = stack_kms_vault::DefaultVaultTransit::from_env() + .map_err(|error| vendor_config(vendor, error))?; + operation.run(&vault).await + } + KeyVendor::Azure => { + let azure = stack_kms_azure::DefaultAzureKeyVault::from_env() + .map_err(|error| vendor_config(vendor, error))?; + operation.run(&azure).await + } + KeyVendor::Gcp => run_gcp(operation).await, + // The callers handle the static key backend before this point, and + // `KeyVendor` is non-exhaustive. + _ => Err(KeysetError::Options(format!( + "stash keyset cannot reach the {vendor} key backend." + ))), + } +} + +/// Google Cloud KMS, with the credentials this binding supports. +/// +/// The binding builds `stack-kms-gcp` without its `service-account` feature, +/// so it has no RSA code (RUSTSEC-2023-0071). It therefore takes a token from +/// `GOOGLE_OAUTH_ACCESS_TOKEN` (for example from `gcloud auth +/// print-access-token`), else the metadata server on Google Cloud. A service +/// account key file (`GOOGLE_APPLICATION_CREDENTIALS`) is refused with that +/// remedy, so it is never ignored. +async fn run_gcp(operation: Operation) -> Result { + use stack_kms_gcp::{AccessToken, CachedToken, GcpKms, GcpKmsConfig, MetadataServerLogin}; + + let vendor = KeyVendor::Gcp; + let env = stack_kms_gcp::process_env(); + let transport = HttpVendorTransport::new().map_err(|error| vendor_config(vendor, error))?; + if let Some(token) = AccessToken::from_env_lookup(&env) { + let kms = GcpKms::new(GcpKmsConfig::new(), transport, token); + return operation.run(&kms).await; + } + if env("GOOGLE_APPLICATION_CREDENTIALS").is_some() { + return Err(KeysetError::VendorConfig { + vendor, + message: "stash keyset does not read service account key files \ + (GOOGLE_APPLICATION_CREDENTIALS). Set GOOGLE_OAUTH_ACCESS_TOKEN, \ + for example to the output of `gcloud auth print-access-token`, \ + or run on Google Cloud with an attached service account." + .into(), + }); + } + // A 1 second connect limit, as in `GcpKms::from_env`, so that off Google + // Cloud the metadata server probe fails fast. + let credentials_transport = HttpVendorTransport::with_options(HttpVendorTransportOptions { + timeout: std::time::Duration::from_secs(10), + connect_timeout: Some(std::time::Duration::from_secs(1)), + }) + .map_err(|error| vendor_config(vendor, error))?; + let login = MetadataServerLogin::from_env_lookup(credentials_transport, &env) + .map_err(|error| vendor_config(vendor, error))?; + let kms = GcpKms::new(GcpKmsConfig::new(), transport, CachedToken::new(login)); + operation.run(&kms).await +} + +// --------------------------------------------------------------------------- +// Tests +// --------------------------------------------------------------------------- + +#[cfg(test)] +#[allow(clippy::unwrap_used, clippy::expect_used, clippy::panic)] +mod tests { + use super::*; + + const KEY_A: &str = "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="; + const KEY_A_HEX: &str = "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"; + const KEY_B: &str = "ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3ODk6Ozw9Pj8="; + + fn create_options(backend: &str, mode: &str) -> KeysetCreateOptions { + KeysetCreateOptions { + backend: backend.into(), + vendor_key: "key-a".into(), + mode: mode.into(), + name: Some("users".into()), + static_key_env: None, + } + } + + /// Each test uses its own variable names, because tests run in parallel + /// threads of one process. + fn set(var: &str, value: &str) { + std::env::set_var(var, value); + } + + fn envelope(error: KeysetError) -> serde_json::Value { + error.envelope() + } + + #[test] + fn decodes_every_accepted_static_key_encoding() { + let expected: Vec = (0u8..32).collect(); + for text in [ + KEY_A, + KEY_A.trim_end_matches('='), + KEY_A_HEX, + &KEY_A_HEX.to_uppercase(), + // base64url of the same bytes (no `+` or `/` in it, so it is + // also the standard form; check a key with them below). + ] { + let key = decode_static_key(text).unwrap(); + assert_eq!(key.risky_ref().as_slice(), expected.as_slice(), "{text}"); + } + let all_ones = [0xfbu8; 32]; + let standard = STANDARD.encode(all_ones); + let url = URL_SAFE_NO_PAD.encode(all_ones); + assert!(standard.contains('+') || standard.contains('/')); + for text in [standard.as_str(), url.as_str()] { + assert_eq!(decode_static_key(text).unwrap().risky_ref(), &all_ones); + } + } + + #[test] + fn refuses_a_static_key_that_is_not_32_bytes() { + for text in [ + "", + "abcd", + &STANDARD.encode([0u8; 31]), + &STANDARD.encode([0u8; 33]), + &"0".repeat(63), + "not base64 at all!", + ] { + assert!(decode_static_key(text).is_none(), "{text:?}"); + } + } + + #[test] + fn a_static_key_error_does_not_name_the_value() { + set("CIP5312_TEST_BAD_KEY", "secret-looking-value"); + let error = read_static_key("CIP5312_TEST_BAD_KEY").err().unwrap(); + let text = envelope(error).to_string(); + assert!(text.contains("CIP5312_TEST_BAD_KEY")); + assert!(!text.contains("secret-looking-value")); + assert!(text.contains("\"type\":\"STATIC_KEY\"")); + } + + #[test] + fn a_missing_static_key_variable_is_refused() { + let error = read_static_key("CIP5312_TEST_UNSET_KEY").err().unwrap(); + assert!(envelope(error).to_string().contains("is not set")); + } + + #[tokio::test] + async fn creates_a_static_record_with_a_generated_key() { + let created = create(create_options("static", "derived")).await.unwrap(); + assert!(created.record.starts_with("stack-keyset:")); + assert_eq!(created.backend, "static"); + assert_eq!(created.mode, "derived"); + assert_eq!(created.name.as_deref(), Some("users")); + assert_eq!(created.vendor_key, "key-a"); + let record = KeysetRecord::parse(&created.record).unwrap(); + assert_eq!(record.id().to_string(), created.keyset_id); + + // The generated key opens the record: rewrap under it. + let key = created.generated_static_key.unwrap(); + assert!(decode_static_key(&key).is_some()); + set("CIP5312_TEST_GENERATED", &key); + let rewrapped = rewrap(KeysetRewrapOptions { + record: created.record.clone(), + vendor_key: None, + static_key_env: Some("CIP5312_TEST_GENERATED".into()), + new_static_key_env: None, + }) + .await + .unwrap(); + assert_eq!(rewrapped.keyset_id, created.keyset_id); + assert!(rewrapped.generated_static_key.is_none()); + // A new nonce, so the record text changes. + assert_ne!(rewrapped.record, created.record); + } + + #[tokio::test] + async fn creates_a_static_record_with_a_key_from_the_environment() { + set("CIP5312_TEST_KEY_ENV", KEY_A); + let created = create(KeysetCreateOptions { + static_key_env: Some("CIP5312_TEST_KEY_ENV".into()), + ..create_options("static", "derived") + }) + .await + .unwrap(); + assert!(created.generated_static_key.is_none()); + + // The record opens with that key and not with another one. + let record = KeysetRecord::parse(&created.record).unwrap(); + let right = StaticKeyVendor::new("key-a", decode_static_key(KEY_A).unwrap()); + let wrong = StaticKeyVendor::new("key-a", decode_static_key(KEY_B).unwrap()); + assert!(record.rewrap(&right, "key-a").await.is_ok()); + assert!(record.rewrap(&wrong, "key-a").await.is_err()); + } + + #[tokio::test] + async fn rewraps_a_static_record_to_another_key() { + set("CIP5312_TEST_OLD", KEY_A); + set("CIP5312_TEST_NEW", KEY_B); + let created = create(KeysetCreateOptions { + static_key_env: Some("CIP5312_TEST_OLD".into()), + ..create_options("static", "derived") + }) + .await + .unwrap(); + let rewrapped = rewrap(KeysetRewrapOptions { + record: format!(" {}\n", created.record), + vendor_key: Some("key-b".into()), + static_key_env: Some("CIP5312_TEST_OLD".into()), + new_static_key_env: Some("CIP5312_TEST_NEW".into()), + }) + .await + .unwrap(); + assert_eq!(rewrapped.keyset_id, created.keyset_id); + assert_eq!(rewrapped.name, created.name); + assert_eq!(rewrapped.mode, "derived"); + assert_eq!(rewrapped.vendor_key, "key-b"); + + // Only the new key opens it now. + let record = KeysetRecord::parse(&rewrapped.record).unwrap(); + let new = StaticKeyVendor::new("key-b", decode_static_key(KEY_B).unwrap()); + let old = StaticKeyVendor::new("key-b", decode_static_key(KEY_A).unwrap()); + assert!(record.rewrap(&new, "key-b").await.is_ok()); + assert!(record.rewrap(&old, "key-b").await.is_err()); + } + + #[tokio::test] + async fn rewrap_with_a_wrong_static_key_is_a_vendor_error() { + set("CIP5312_TEST_RIGHT", KEY_A); + set("CIP5312_TEST_WRONG", KEY_B); + let created = create(KeysetCreateOptions { + static_key_env: Some("CIP5312_TEST_RIGHT".into()), + ..create_options("static", "derived") + }) + .await + .unwrap(); + let error = rewrap(KeysetRewrapOptions { + record: created.record, + vendor_key: None, + static_key_env: Some("CIP5312_TEST_WRONG".into()), + new_static_key_env: None, + }) + .await + .err() + .unwrap(); + let envelope = envelope(error); + assert_eq!(envelope["type"], "VENDOR_ERROR"); + assert_eq!(envelope["vendor"], "static"); + assert_eq!(envelope["kind"], "InvalidCiphertext"); + } + + #[tokio::test] + async fn refuses_wrapped_data_keys_on_the_static_backend() { + let error = create(create_options("static", "wrapped")) + .await + .err() + .unwrap(); + let envelope = envelope(error); + assert_eq!(envelope["type"], "INVALID_RECORD"); + assert_eq!(envelope["diagnostic"], "stack_kms_vendor::mode_unsupported"); + assert!(envelope["help"].as_str().unwrap().contains("derived")); + } + + #[tokio::test] + async fn refuses_bad_options() { + let cases = [ + create_options("zerokms", "derived"), + create_options("aws", "both"), + KeysetCreateOptions { + vendor_key: String::new(), + ..create_options("aws", "wrapped") + }, + KeysetCreateOptions { + name: Some(String::new()), + ..create_options("aws", "wrapped") + }, + KeysetCreateOptions { + static_key_env: Some("CIP5312_TEST_ANY".into()), + ..create_options("aws", "wrapped") + }, + ]; + for options in cases { + let error = create(options.clone()).await.err().unwrap(); + assert_eq!(envelope(error)["type"], "INVALID_OPTIONS", "{options:?}"); + } + } + + #[tokio::test] + async fn rewrap_refuses_a_bad_record_and_missing_static_options() { + let error = rewrap(KeysetRewrapOptions { + record: "not-a-record".into(), + vendor_key: None, + static_key_env: None, + new_static_key_env: None, + }) + .await + .err() + .unwrap(); + let envelope_value = envelope(error); + assert_eq!(envelope_value["type"], "INVALID_RECORD"); + assert_eq!( + envelope_value["diagnostic"], + "stack_kms_vendor::record_prefix" + ); + + let created = create(create_options("static", "derived")).await.unwrap(); + let error = rewrap(KeysetRewrapOptions { + record: created.record.clone(), + vendor_key: None, + static_key_env: None, + new_static_key_env: None, + }) + .await + .err() + .unwrap(); + assert!(envelope(error)["message"] + .as_str() + .unwrap() + .contains("--key-env")); + + set( + "CIP5312_TEST_SAME", + created.generated_static_key.as_deref().unwrap(), + ); + let error = rewrap(KeysetRewrapOptions { + record: created.record, + vendor_key: None, + static_key_env: Some("CIP5312_TEST_SAME".into()), + new_static_key_env: Some("CIP5312_TEST_SAME".into()), + }) + .await + .err() + .unwrap(); + assert_eq!(envelope(error)["type"], "INVALID_OPTIONS"); + } + + /// The hand-written `KeysetFailure` union in `index.d.ts` lists every + /// code `KeysetError` emits. + #[test] + fn index_dts_lists_every_keyset_failure_code() { + let dts = include_str!("../index.d.ts"); + let errors = [ + KeysetError::Options(String::new()), + KeysetError::StaticKey(String::new()), + KeysetError::VendorConfig { + vendor: KeyVendor::Aws, + message: String::new(), + }, + KeysetError::Record(RecordError::NotARecord), + KeysetError::Record(RecordError::RewrapToAnotherKey { + keyset: Default::default(), + from: "a".into(), + to: "b".into(), + }), + KeysetError::Record(RecordError::Kms(stack_kms::Error::Backend { + vendor: "static", + kind: stack_kms::BackendErrorKind::NotFound, + source: "none".into(), + })), + ]; + for error in errors { + let code = error.code(); + assert!( + dts.contains(&format!("\"{code}\"")), + "index.d.ts KeysetFailure lacks {code}" + ); + } + assert!(dts.contains("export type KeysetFailure")); + } + + #[test] + fn a_rewrap_to_another_key_has_its_own_code() { + let error = KeysetError::Record(RecordError::RewrapToAnotherKey { + keyset: Default::default(), + from: "arn:aws:kms:us-east-1:111122223333:key/a".into(), + to: "arn:aws:kms:us-east-1:111122223333:key/b".into(), + }); + let value = error.envelope(); + assert_eq!(value["type"], "REWRAP_REFUSED"); + assert_eq!(value["diagnostic"], "stack_kms_vendor::rewrap_another_key"); + assert!(value["help"] + .as_str() + .is_some_and(|help| help.contains("new version of the same key"))); + } + + #[test] + fn the_backend_names_are_the_cli_names() { + let names: Vec<_> = BACKENDS.iter().map(|(name, _)| *name).collect(); + assert_eq!(names, ["aws", "gcp", "azure", "vault", "static"]); + for (name, vendor) in BACKENDS { + assert_eq!(parse_backend(name).unwrap(), vendor); + } + } + + #[test] + fn a_vendor_config_error_names_the_vendor() { + let envelope = envelope(KeysetError::VendorConfig { + vendor: KeyVendor::Aws, + message: "no region".into(), + }); + assert_eq!(envelope["type"], "VENDOR_CONFIG"); + assert_eq!(envelope["vendor"], "aws-kms"); + assert!(envelope["message"].as_str().unwrap().contains("no region")); + } + + #[test] + fn the_result_debug_hides_the_generated_key() { + let result = KeysetResult { + record: "stack-keyset:x".into(), + keyset_id: "id".into(), + name: None, + backend: "static".into(), + mode: "derived".into(), + vendor_key: "key-a".into(), + generated_static_key: Some("the-secret-key".into()), + }; + assert!(!format!("{result:?}").contains("the-secret-key")); + } +} diff --git a/languages/typescript/packages/auth/src/lib.rs b/languages/typescript/packages/auth/src/lib.rs index 67bd19b9c2..5c8977fb4e 100644 --- a/languages/typescript/packages/auth/src/lib.rs +++ b/languages/typescript/packages/auth/src/lib.rs @@ -13,6 +13,11 @@ use stack_auth::{ use vitaminc_protected::OpaqueDebug; use zeroize::Zeroizing; +mod keyset; +pub use keyset::{ + keyset_create, keyset_rewrap, KeysetCreateOptions, KeysetResult, KeysetRewrapOptions, +}; + // --------------------------------------------------------------------------- // Error helpers // --------------------------------------------------------------------------- diff --git a/languages/typescript/packages/cli/README.md b/languages/typescript/packages/cli/README.md index c051e26253..e71a9ff97e 100644 --- a/languages/typescript/packages/cli/README.md +++ b/languages/typescript/packages/cli/README.md @@ -342,6 +342,19 @@ npx stash decrypt status The backfill loads your encryption client exactly as `encrypt backfill` does (client file or Prisma Next contract), so export the `CS_*` credentials of the environment that wrote the data. +### `npx stash keyset` — keysets in a key vendor backend + +**ZeroKMS users never need these commands.** They are for a keyset kept in a key vendor backend instead of ZeroKMS: AWS KMS, Google Cloud KMS, Azure Key Vault, HashiCorp Vault Transit, or a static key. Only the Rust crates and the Go SDK can use such a keyset today; `@cipherstash/stack` cannot yet. + +```bash +npx stash keyset create --backend aws --vendor-key arn:aws:kms:us-east-1:111122223333:key/ --mode wrapped --name users +npx stash keyset rewrap --in keyset.env --out keyset.env --force +``` + +`keyset create` makes a **keyset record** and prints it as a dotenv line, `CS_KEYSET_RECORD_=stack-keyset:…` (or `CS_KEYSET_RECORD` with no `--name`). The record is not secret. No SDK reads that variable by itself: pass its value to `encrypt.WithKeysetRecords` (Go) or `KeysetRecord::parse` (Rust). `--mode wrapped` gives each value its own vendor-wrapped data key; `--mode derived` derives the data keys from one root key, which the SDK caches for a TTL. The static backend offers `derived` only, and prints a generated key as `CS_STATIC_KEY` unless `--key-env` names the variable that holds one. Credentials come from each vendor's environment chain; `npx stash keyset create --help` lists the variables. + +`keyset rewrap` wraps the record's key again, under the same vendor key after a rotation or under another one. It does not re-key stored data, so a `derived` record may move to any key, but **a `wrapped` record may move only to a new version of its own vendor key**: another key is refused (`rewrap_refused`) before any vendor call. The `stash-cli` agent skill has the full flag reference. + --- ## Migration mode diff --git a/languages/typescript/packages/cli/src/bin/main.ts b/languages/typescript/packages/cli/src/bin/main.ts index 47485afc59..ea3f195444 100644 --- a/languages/typescript/packages/cli/src/bin/main.ts +++ b/languages/typescript/packages/cli/src/bin/main.ts @@ -135,6 +135,9 @@ Commands: env Mint deployment credentials and print them as env vars + keyset create Create a keyset record for a key vendor backend (not needed with ZeroKMS) + keyset rewrap Rewrap a keyset record under the current or another vendor key (not needed with ZeroKMS) + Options: --help, -h Show help --version, -v Show version @@ -590,6 +593,70 @@ async function runSchemaCommand( } } +/** + * `stash keyset create|rewrap`. The module loads `@cipherstash/auth` only + * after its own argv checks, so a usage error never needs the native binary. + */ +async function runKeysetCommand( + sub: string | undefined, + commandArgs: string[], + flags: Record, + values: Record, +) { + const keyset = await import('../commands/keyset/index.js') + // parseArgs booleanises a value flag with no value (`--out --force`). + const missing = (names: string[]) => + names.filter((name) => flags[name] === true) + const output = { + out: values.out, + force: flags.force, + json: flags.json, + } + switch (sub) { + case 'create': + if (commandArgs.length > 0) { + p.log.error( + `Unexpected argument '${commandArgs[0]}'. stash keyset create takes flags only.`, + { output: process.stderr }, + ) + throw new CliExit(1) + } + await keyset.keysetCreateCommand( + { + ...output, + backend: values.backend, + vendorKey: values['vendor-key'], + mode: values.mode, + name: values.name, + keyEnv: values['key-env'], + }, + missing(['backend', 'vendor-key', 'mode', 'name', 'key-env', 'out']), + ) + break + case 'rewrap': + if (commandArgs.length > 1) { + p.log.error('stash keyset rewrap takes one keyset record.', { + output: process.stderr, + }) + throw new CliExit(1) + } + await keyset.keysetRewrapCommand( + { + ...output, + record: commandArgs[0], + in: values.in, + vendorKey: values['vendor-key'], + keyEnv: values['key-env'], + newKeyEnv: values['new-key-env'], + }, + missing(['in', 'vendor-key', 'key-env', 'new-key-env', 'out']), + ) + break + default: + keyset.keysetUnknownSubcommand(sub) + } +} + // The CLI body. Loaded by the thin launcher in stash.ts via dynamic import so // that a missing native binary (evaluated when this module's command graph // loads) surfaces as friendly guidance rather than a raw stack trace. @@ -722,6 +789,9 @@ async function dispatch( unexpectedArg: subcommand ?? commandArgs[0], }) break + case 'keyset': + await runKeysetCommand(subcommand, commandArgs, flags, values) + break case 'manifest': // Pure metadata (no native code) — safe to run anywhere, including when // the native binary is missing. diff --git a/languages/typescript/packages/cli/src/cli/registry.ts b/languages/typescript/packages/cli/src/cli/registry.ts index e052aa7122..46e681aa38 100644 --- a/languages/typescript/packages/cli/src/cli/registry.ts +++ b/languages/typescript/packages/cli/src/cli/registry.ts @@ -833,4 +833,168 @@ export const registry: CommandGroup[] = [ }, ], }, + { + title: 'Key vendor backends', + commands: [ + { + name: 'keyset create', + summary: + 'Create a keyset record for a key vendor backend (not needed with ZeroKMS)', + long: [ + 'ZeroKMS users never need this command. It is only for a keyset held', + 'by a key vendor backend: AWS KMS, Google Cloud KMS, Azure Key Vault,', + 'HashiCorp Vault Transit, or the static key backend.', + '', + 'Generates a keyset id and a key, wraps the key under --vendor-key', + '(one vendor call), and prints the keyset record as a dotenv line,', + 'CS_KEYSET_RECORD (or CS_KEYSET_RECORD_ with --name). The', + 'record is not secret: only the vendor key can unwrap its key. No SDK', + 'reads that variable by itself: pass its value to', + 'encrypt.WithKeysetRecords (Go) or KeysetRecord::parse (Rust).', + '', + 'Credentials come from the vendor environment, never from a flag:', + ' aws AWS_REGION; AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY', + ' (+ AWS_SESSION_TOKEN), web identity, ECS or IMDS.', + ' No ~/.aws profiles: export them first, for example with', + ' `aws configure export-credentials --format env`.', + ' gcp GOOGLE_OAUTH_ACCESS_TOKEN (e.g. `gcloud auth', + ' print-access-token`), else the metadata server. Service', + ' account key files are not read.', + ' azure AZURE_KEY_VAULT_ACCESS_TOKEN; AZURE_TENANT_ID +', + ' AZURE_CLIENT_ID + AZURE_CLIENT_SECRET or', + ' AZURE_FEDERATED_TOKEN_FILE; else managed identity.', + ' vault VAULT_ADDR (+ VAULT_NAMESPACE, VAULT_TRANSIT_MOUNT);', + ' VAULT_TOKEN, AppRole or Kubernetes auth.', + ' static --key-env names the variable that holds the 32-byte key', + ' (base64 or hex). Without it a new key is generated and', + ' printed as CS_STATIC_KEY, which is a secret. Derived only.', + '', + 'Stdout carries only the dotenv block (or the --json event); progress', + 'goes to stderr, so `stash keyset create … > keyset.env` is safe.', + ].join('\n'), + examples: [ + 'keyset create --backend aws --vendor-key arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab --mode wrapped --name users', + 'keyset create --backend vault --vendor-key users --mode derived --out keyset.env', + 'keyset create --backend static --vendor-key dev-key --mode derived --out .env.keyset.local', + ], + flags: [ + { + name: '--backend', + value: '', + description: 'The key vendor backend. Required.', + }, + { + name: '--vendor-key', + value: '', + description: + 'The vendor key that wraps the keyset key: an AWS KMS key ARN, a Cloud KMS crypto key name, an Azure Key Vault key identifier, a Vault Transit key name, or a label for a static key. Required.', + }, + { + name: '--mode', + value: '', + description: + 'The key mode. wrapped: each value has its own vendor-wrapped data key. derived: data keys come from a root key cached for a short time. The static backend offers derived only. Required.', + }, + { + name: '--name', + value: '', + description: + 'A local name for the keyset (1 to 255 bytes). Also names the variable: CS_KEYSET_RECORD_.', + }, + { + name: '--key-env', + value: '', + description: + 'Static backend only: the environment variable that holds the static key. Omit it to generate a new key.', + }, + { + name: '--out', + value: '', + description: + 'Write the block to this file (mode 0600) instead of stdout. An existing file is refused unless --force.', + }, + { + name: '--force', + description: 'Overwrite an existing --out file.', + }, + { + name: '--json', + description: + 'Emit one NDJSON object: { status: "created", record, keysetId, envVar, … } (with a generated static key in staticKey), or { status: "written", path, … } with --out, which never carries the static key. Failures are { status: "error" }.', + }, + ], + }, + { + name: 'keyset rewrap', + summary: + 'Rewrap a keyset record under the current or another vendor key (not needed with ZeroKMS)', + long: [ + 'ZeroKMS users never need this command.', + '', + 'Unwraps the record key and wraps it again (two vendor calls). With', + 'no --vendor-key it uses the record’s own vendor key, which moves the', + 'record to that key’s current version after a rotation. With', + '--vendor-key it moves the record to another key of the same backend.', + 'The keyset id, name and mode stay the same. Stored data is not', + 're-keyed, so the rule depends on the mode. A derived-mode record may', + 'move to any key of its backend. A wrapped-mode record may move only', + 'to a new version of its own vendor key: each stored value keeps its', + 'data key wrapped under the key that sealed it, so a move to another', + 'key is refused (code rewrap_refused) before any vendor call.', + '', + 'The record is the argument, or --in (a file that --out wrote,', + 'or the bare record). Credentials come from the vendor environment, as', + 'for `keyset create`. For a static record, --key-env names the', + 'variable that holds its current key, and --new-key-env the new', + 'key (omit it to generate one) when --vendor-key names another key.', + ].join('\n'), + examples: [ + 'keyset rewrap --in keyset.env --out keyset.env --force', + 'keyset rewrap stack-keyset:AQEC… --vendor-key arn:aws:kms:us-east-1:111122223333:key/new-key-id', + 'keyset rewrap --in .env.keyset.local --key-env CS_STATIC_KEY --vendor-key dev-key-2 --new-key-env CS_STATIC_KEY_2', + ], + flags: [ + { + name: '--in', + value: '', + description: + 'Read the record from this file instead of the argument. The file must hold exactly one record.', + }, + { + name: '--vendor-key', + value: '', + description: + 'The vendor key to wrap under. Default: the record’s own vendor key.', + }, + { + name: '--key-env', + value: '', + description: + 'Static backend only: the environment variable that holds the record’s current static key. Required for a static record.', + }, + { + name: '--new-key-env', + value: '', + description: + 'Static backend only: the environment variable that holds the static key of --vendor-key. Omit it to generate a new key.', + }, + { + name: '--out', + value: '', + description: + 'Write the block to this file (mode 0600) instead of stdout. An existing file is refused unless --force.', + }, + { + name: '--force', + description: 'Overwrite an existing --out file.', + }, + { + name: '--json', + description: + 'Emit one NDJSON object: { status: "rewrapped", … }, or { status: "written", … } with --out. Failures are { status: "error" }.', + }, + ], + }, + ], + }, ] diff --git a/languages/typescript/packages/cli/src/commands/keyset/__tests__/keyset.test.ts b/languages/typescript/packages/cli/src/commands/keyset/__tests__/keyset.test.ts new file mode 100644 index 0000000000..bcf1853c5f --- /dev/null +++ b/languages/typescript/packages/cli/src/commands/keyset/__tests__/keyset.test.ts @@ -0,0 +1,427 @@ +import { + mkdtempSync, + readFileSync, + rmSync, + statSync, + writeFileSync, +} from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { + afterEach, + beforeEach, + describe, + expect, + it, + type MockInstance, + vi, +} from 'vitest' +import { CliExit } from '../../../cli/exit.js' +import { messages } from '../../../messages.js' + +// The vendor calls happen in the native binding. Fake it at that boundary, so +// the unit suite is native-free (the binding's own tests drive the static key +// backend for real). +const authMock = vi.hoisted(() => ({ + keysetCreate: vi.fn(), + keysetRewrap: vi.fn(), +})) +vi.mock('@cipherstash/auth', () => ({ default: authMock })) + +const clack = vi.hoisted(() => { + const spinnerInstance = { start: vi.fn(), stop: vi.fn() } + return { + spinnerInstance, + intro: vi.fn(), + outro: vi.fn(), + spinner: vi.fn(() => spinnerInstance), + log: { info: vi.fn(), warn: vi.fn(), error: vi.fn(), success: vi.fn() }, + } +}) +vi.mock('@clack/prompts', () => clack) + +const { + formatBlock, + keysetCreateCommand, + keysetRewrapCommand, + readRecordFile, + recordVar, +} = await import('../index.js') + +const RECORD = 'stack-keyset:AQECAAAAAAAAAAAAAAAAAAAAAAA' + +function result(over: Record = {}) { + return { + record: RECORD, + keysetId: '0f0e0d0c-0b0a-4908-8706-050403020100', + name: 'users', + backend: 'aws-kms', + mode: 'wrapped', + vendorKey: 'arn:aws:kms:us-east-1:111122223333:key/k', + ...over, + } +} + +async function expectExit(promise: Promise, code: number) { + try { + await promise + } catch (err) { + expect(err).toBeInstanceOf(CliExit) + expect((err as CliExit).code).toBe(code) + return + } + throw new Error(`expected CliExit(${code}), but the command returned`) +} + +let stdout: MockInstance +let dir: string + +beforeEach(() => { + vi.clearAllMocks() + stdout = vi.spyOn(console, 'log').mockImplementation(() => {}) + dir = mkdtempSync(join(tmpdir(), 'stash-keyset-')) +}) + +afterEach(() => { + stdout.mockRestore() + rmSync(dir, { recursive: true, force: true }) +}) + +const printed = () => + stdout.mock.calls.map((call: unknown[]) => String(call[0])).join('\n') +const errors = () => + clack.log.error.mock.calls.map((call) => String(call[0])).join('\n') + +describe('stash keyset create', () => { + it('passes the flags to the native binding and prints a dotenv block', async () => { + authMock.keysetCreate.mockResolvedValue({ data: result() }) + await keysetCreateCommand({ + backend: 'aws', + vendorKey: 'arn:aws:kms:us-east-1:111122223333:key/k', + mode: 'wrapped', + name: 'users', + }) + expect(authMock.keysetCreate).toHaveBeenCalledWith({ + backend: 'aws', + vendorKey: 'arn:aws:kms:us-east-1:111122223333:key/k', + mode: 'wrapped', + name: 'users', + }) + const out = printed() + expect(out).toContain(`CS_KEYSET_RECORD_USERS=${RECORD}`) + expect(out).toContain('# backend: aws-kms, mode: wrapped') + // Stdout carries only the block: every line is a comment, blank or VAR=. + for (const line of out.split('\n')) { + expect( + line === '' || line.startsWith('#') || /^[A-Z_]+=/.test(line), + ).toBe(true) + } + }) + + it('refuses missing flags before loading the binding', async () => { + await expectExit(keysetCreateCommand({ backend: 'aws' }), 1) + expect(errors()).toContain(messages.keyset.missingCreateFlags) + expect(errors()).toContain('--vendor-key, --mode') + expect(authMock.keysetCreate).not.toHaveBeenCalled() + }) + + it('refuses an unknown backend and says ZeroKMS needs no keyset', async () => { + await expectExit( + keysetCreateCommand({ + backend: 'zerokms', + vendorKey: 'k', + mode: 'derived', + }), + 1, + ) + expect(errors()).toContain(messages.keyset.unknownBackend) + const hints = clack.log.info.mock.calls.map((call) => String(call[0])) + expect(hints).toContain(messages.keyset.zeroKmsNote) + }) + + it('refuses wrapped data keys on the static backend', async () => { + await expectExit( + keysetCreateCommand({ + backend: 'static', + vendorKey: 'k', + mode: 'wrapped', + }), + 1, + ) + expect(errors()).toContain(messages.keyset.staticDerivedOnly) + expect(authMock.keysetCreate).not.toHaveBeenCalled() + }) + + it('takes a variable NAME in --key-env, never a key', async () => { + await expectExit( + keysetCreateCommand({ + backend: 'static', + vendorKey: 'k', + mode: 'derived', + keyEnv: 'AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8=', + }), + 1, + ) + expect(errors()).toContain('NAME of an environment variable') + // The refused value is not echoed. + expect(errors()).not.toContain('AAECAwQF') + + await expectExit( + keysetCreateCommand({ + backend: 'aws', + vendorKey: 'k', + mode: 'wrapped', + keyEnv: 'CS_STATIC_KEY', + }), + 1, + ) + expect(errors()).toContain('--key-env is only for --backend static') + }) + + it('refuses a name with a control character', async () => { + await expectExit( + keysetCreateCommand({ + backend: 'aws', + vendorKey: 'k', + mode: 'wrapped', + name: 'a\nCS_X=1', + }), + 1, + ) + expect(authMock.keysetCreate).not.toHaveBeenCalled() + }) + + it('reports a flag that lost its value', async () => { + await expectExit(keysetCreateCommand({}, ['out']), 1) + expect(errors()).toContain('--out requires a value') + }) + + it('maps a native failure to an error and a hint', async () => { + const error = new Error( + 'Could not configure the aws-kms client from the environment: no region', + ) + authMock.keysetCreate.mockResolvedValue({ + failure: { type: 'VENDOR_CONFIG', error, vendor: 'aws-kms' }, + }) + await expectExit( + keysetCreateCommand({ backend: 'aws', vendorKey: 'k', mode: 'wrapped' }), + 1, + ) + expect(errors()).toContain('no region') + const hints = clack.log.info.mock.calls.map((call) => String(call[0])) + expect(hints).toContain(messages.keyset.vendorConfigHint) + }) + + it('emits the --json error envelope with the native code', async () => { + authMock.keysetCreate.mockResolvedValue({ + failure: { + type: 'VENDOR_ERROR', + kind: 'AccessDenied', + error: new Error('denied'), + }, + }) + await expectExit( + keysetCreateCommand({ + backend: 'aws', + vendorKey: 'k', + mode: 'wrapped', + json: true, + }), + 1, + ) + const event = JSON.parse(printed()) + expect(event).toMatchObject({ + status: 'error', + code: 'vendor_error', + message: 'denied', + }) + expect(event.hint).toContain('credentials') + }) + + it('prints a generated static key and warns that it is a secret', async () => { + authMock.keysetCreate.mockResolvedValue({ + data: result({ + backend: 'static', + mode: 'derived', + name: undefined, + vendorKey: 'dev-key', + generatedStaticKey: 'c2VjcmV0LXN0YXRpYy1rZXktZm9yLXRlc3Rpbmc=', + }), + }) + await keysetCreateCommand({ + backend: 'static', + vendorKey: 'dev-key', + mode: 'derived', + }) + expect(authMock.keysetCreate).toHaveBeenCalledWith({ + backend: 'static', + vendorKey: 'dev-key', + mode: 'derived', + }) + const out = printed() + expect(out).toContain(`CS_KEYSET_RECORD=${RECORD}`) + expect(out).toContain( + 'CS_STATIC_KEY=c2VjcmV0LXN0YXRpYy1rZXktZm9yLXRlc3Rpbmc=', + ) + expect(clack.log.warn).toHaveBeenCalledWith( + messages.keyset.generatedStaticKeyWarning, + expect.anything(), + ) + }) + + it('writes --out with mode 0600, refuses to overwrite without --force', async () => { + authMock.keysetCreate.mockResolvedValue({ + data: result({ generatedStaticKey: 'c2VjcmV0' }), + }) + const out = join(dir, 'keyset.env') + const options = { + backend: 'aws', + vendorKey: 'k', + mode: 'wrapped', + out, + json: true, + } + await keysetCreateCommand(options) + expect(statSync(out).mode & 0o777).toBe(0o600) + expect(readFileSync(out, 'utf-8')).toContain( + `CS_KEYSET_RECORD_USERS=${RECORD}`, + ) + // The --json confirmation never carries the static key. + const event = JSON.parse(printed()) + expect(event).toMatchObject({ + status: 'written', + path: out, + staticKeyVar: 'CS_STATIC_KEY', + }) + expect(JSON.stringify(event)).not.toContain('c2VjcmV0') + + stdout.mockClear() + await expectExit(keysetCreateCommand(options), 1) + expect(JSON.parse(printed())).toMatchObject({ code: 'write_conflict' }) + + await keysetCreateCommand({ ...options, force: true }) + }) + + it('tells the user when the installed binding is too old', async () => { + const { keysetCreate } = authMock + // biome-ignore lint/suspicious/noExplicitAny: removing a mocked method. + ;(authMock as any).keysetCreate = undefined + try { + await expectExit( + keysetCreateCommand({ + backend: 'aws', + vendorKey: 'k', + mode: 'wrapped', + }), + 1, + ) + expect(errors()).toContain('has no keyset functions') + } finally { + authMock.keysetCreate = keysetCreate + } + }) +}) + +describe('stash keyset rewrap', () => { + it('reads the record from --in and passes the static key variables', async () => { + const file = join(dir, 'keyset.env') + writeFileSync(file, formatBlock(result(), 'CS_KEYSET_RECORD_USERS')) + authMock.keysetRewrap.mockResolvedValue({ + data: result({ vendorKey: 'b' }), + }) + await keysetRewrapCommand({ + in: file, + vendorKey: 'b', + keyEnv: 'OLD_KEY', + newKeyEnv: 'NEW_KEY', + }) + expect(authMock.keysetRewrap).toHaveBeenCalledWith({ + record: RECORD, + vendorKey: 'b', + staticKeyEnv: 'OLD_KEY', + newStaticKeyEnv: 'NEW_KEY', + }) + const hints = clack.log.info.mock.calls.map((call) => String(call[0])) + expect(hints).toContain(messages.keyset.rewrapNote) + }) + + it('maps a refused rewrap to its own code, message and hint', async () => { + authMock.keysetRewrap.mockResolvedValue({ + failure: { + type: 'REWRAP_REFUSED', + error: new Error('native text'), + help: 'native help', + }, + }) + await expectExit( + keysetRewrapCommand({ record: RECORD, vendorKey: 'other', json: true }), + 1, + ) + const event = JSON.parse(printed()) + expect(event).toMatchObject({ + status: 'error', + code: 'rewrap_refused', + message: messages.keyset.rewrapRefused, + hint: messages.keyset.rewrapRefusedHint, + }) + }) + + it('takes the record as an argument, trimmed', async () => { + authMock.keysetRewrap.mockResolvedValue({ data: result() }) + await keysetRewrapCommand({ record: ` ${RECORD}\n`, json: true }) + expect(authMock.keysetRewrap).toHaveBeenCalledWith({ record: RECORD }) + expect(JSON.parse(printed())).toMatchObject({ + status: 'rewrapped', + record: RECORD, + envVar: 'CS_KEYSET_RECORD_USERS', + }) + }) + + it('refuses no record, two sources, and text that is not a record', async () => { + await expectExit(keysetRewrapCommand({}), 1) + expect(errors()).toContain(messages.keyset.missingRecord) + await expectExit(keysetRewrapCommand({ record: RECORD, in: 'x' }), 1) + await expectExit(keysetRewrapCommand({ record: 'hello' }), 1) + expect(authMock.keysetRewrap).not.toHaveBeenCalled() + }) + + it('rewrites the --in file in place with --force', async () => { + const file = join(dir, 'keyset.env') + writeFileSync(file, `${RECORD}\n`) + const next = 'stack-keyset:AQECAQEBAQEBAQEBAQEBAQEBAQE' + authMock.keysetRewrap.mockResolvedValue({ data: result({ record: next }) }) + await keysetRewrapCommand({ in: file, out: file, force: true }) + expect(readFileSync(file, 'utf-8')).toContain(next) + expect(statSync(file).mode & 0o777).toBe(0o600) + }) +}) + +describe('readRecordFile', () => { + it('refuses a file with no record or with two', () => { + const none = join(dir, 'none') + writeFileSync(none, 'FOO=bar\n') + expect(() => readRecordFile(none)).toThrow('holds no keyset record') + const two = join(dir, 'two') + writeFileSync(two, `A=${RECORD}\nB=stack-keyset:BBBB\n`) + expect(() => readRecordFile(two)).toThrow('holds 2 keyset records') + }) +}) + +describe('recordVar and formatBlock', () => { + it('derives the variable from the name', () => { + expect(recordVar(undefined)).toBe('CS_KEYSET_RECORD') + expect(recordVar('users')).toBe('CS_KEYSET_RECORD_USERS') + expect(recordVar('billing-eu 2')).toBe('CS_KEYSET_RECORD_BILLING_EU_2') + expect(recordVar('---')).toBe('CS_KEYSET_RECORD') + }) + + it('quotes a name from a record so it cannot leave its comment line', () => { + const block = formatBlock( + result({ name: 'x\nEVIL=1' }), + 'CS_KEYSET_RECORD_X', + ) + expect(block.split('\n').some((line) => line.startsWith('EVIL='))).toBe( + false, + ) + }) +}) diff --git a/languages/typescript/packages/cli/src/commands/keyset/index.ts b/languages/typescript/packages/cli/src/commands/keyset/index.ts new file mode 100644 index 0000000000..329b667d26 --- /dev/null +++ b/languages/typescript/packages/cli/src/commands/keyset/index.ts @@ -0,0 +1,553 @@ +import { chmodSync, readFileSync, writeFileSync } from 'node:fs' +import { resolve } from 'node:path' +import * as p from '@clack/prompts' +import { CliExit } from '../../cli/exit.js' +import { messages } from '../../messages.js' +import { emitJsonError, emitJsonEvent } from '../auth/events.js' + +/** + * `stash keyset create` / `stash keyset rewrap` (CIP-5312): keyset records for + * the key vendor backends (AWS KMS, Google Cloud KMS, Azure Key Vault, Vault + * Transit, the static key backend). ZeroKMS users never need them. + * + * The vendor calls are made by Rust, through `keysetCreate` / `keysetRewrap` + * in the `@cipherstash/auth` native binding, with the same vendor clients the + * Rust and Go SDKs use. Credentials come from each vendor's environment chain; + * no flag carries a secret. A static key is named by the environment variable + * that holds it (`--key-env`), never passed as a value. + * + * As with `stash env`, every human-facing line goes to STDERR, and stdout + * carries only the dotenv block (or, with `--json`, one NDJSON event), so + * `stash keyset create … > keyset.env` stays clean. + */ +const CHROME = { output: process.stderr } as const + +/** The `--backend` values. The Rust side holds the same list. */ +export const KEYSET_BACKENDS = [ + 'aws', + 'gcp', + 'azure', + 'vault', + 'static', +] as const + +/** The `--mode` values. */ +export const KEYSET_MODES = ['wrapped', 'derived'] as const + +/** + * The prefix of the dotenv variable the record is printed as. No SDK reads it + * by itself today: the app passes its value to `encrypt.WithKeysetRecords` + * (Go) or `KeysetRecord::parse` (Rust). + */ +export const RECORD_VAR = 'CS_KEYSET_RECORD' + +/** The dotenv variable a generated static key is printed as. */ +export const STATIC_KEY_VAR = 'CS_STATIC_KEY' + +/** Every keyset record starts with this. */ +const RECORD_PREFIX = 'stack-keyset:' +const RECORD_PATTERN = /stack-keyset:[A-Za-z0-9_-]+/g + +// biome-ignore lint/suspicious/noControlCharactersInRegex: matching control characters is the point. +const CONTROL_CHARS = /[\x00-\x1f\x7f]/ + +/** The flags shared by both subcommands. */ +interface OutputOptions { + /** Write the block to this file (mode 0600) instead of stdout. */ + out?: string + /** Overwrite an existing `--out` file. */ + force?: boolean + /** Emit one NDJSON event instead of a dotenv block. */ + json?: boolean +} + +export interface KeysetCreateOptions extends OutputOptions { + backend?: string + vendorKey?: string + mode?: string + name?: string + keyEnv?: string +} + +export interface KeysetRewrapOptions extends OutputOptions { + /** The record, as the positional argument. */ + record?: string + /** A file holding the record (a dotenv block from `--out`, or the bare record). */ + in?: string + vendorKey?: string + keyEnv?: string + newKeyEnv?: string +} + +/** A value-taking flag that argv parsing turned into a boolean (no value). */ +export type MissingValues = string[] + +/** What the native binding returns (`KeysetResult` in `@cipherstash/auth`). */ +interface KeysetResult { + record: string + keysetId: string + name?: string + backend: string + mode: string + vendorKey: string + generatedStaticKey?: string +} + +interface KeysetFailure { + type: string + error: Error + help?: string + vendor?: string + kind?: string +} + +type NativeResult = + | { data: KeysetResult; failure?: undefined } + | { data?: undefined; failure: KeysetFailure } + +/** An error with a code for the `--json` stream and an optional hint. */ +export class KeysetError extends Error { + constructor( + readonly code: string, + message: string, + readonly hint?: string, + ) { + super(message) + this.name = 'KeysetError' + } +} + +// --------------------------------------------------------------------------- +// Entry points +// --------------------------------------------------------------------------- + +export async function keysetCreateCommand( + options: KeysetCreateOptions, + missingValues: MissingValues = [], +): Promise { + await run('create', options, async () => { + rejectMissingValues(missingValues) + const input = validateCreate(options) + const outPath = resolveOut(options) + const auth = await loadNative() + const result = (await auth.keysetCreate(input)) as NativeResult + return { result: unwrap(result), outPath } + }) +} + +export async function keysetRewrapCommand( + options: KeysetRewrapOptions, + missingValues: MissingValues = [], +): Promise { + await run('rewrap', options, async () => { + rejectMissingValues(missingValues) + const input = validateRewrap(options) + const outPath = resolveOut(options) + const auth = await loadNative() + const result = (await auth.keysetRewrap(input)) as NativeResult + return { result: unwrap(result), outPath } + }) +} + +/** Unknown `stash keyset `. */ +export function keysetUnknownSubcommand(sub: string | undefined): never { + p.log.error( + `${messages.keyset.unknownSubcommand}: ${sub ?? '(none)'}`, + CHROME, + ) + p.log.info(messages.keyset.zeroKmsNote, CHROME) + throw new CliExit(1) +} + +// --------------------------------------------------------------------------- +// The run loop: one place renders success and failure for both subcommands +// --------------------------------------------------------------------------- + +async function run( + verb: 'create' | 'rewrap', + options: OutputOptions, + body: () => Promise<{ result: KeysetResult; outPath: string | undefined }>, +): Promise { + const json = options.json ?? false + if (!json) p.intro(`stash keyset ${verb}`, CHROME) + try { + const s = json ? null : p.spinner(CHROME) + let outcome: { result: KeysetResult; outPath: string | undefined } + s?.start( + verb === 'create' ? 'Creating the keyset...' : 'Rewrapping the keyset...', + ) + try { + outcome = await body() + } catch (err) { + s?.stop( + verb === 'create' + ? 'Could not create the keyset.' + : 'Could not rewrap the keyset.', + ) + throw err + } + s?.stop(verb === 'create' ? 'Keyset created.' : 'Keyset rewrapped.') + render(verb, outcome.result, outcome.outPath, options) + } catch (err) { + if (err instanceof CliExit) throw err + const failure = + err instanceof KeysetError + ? err + : new KeysetError( + 'keyset_failed', + err instanceof Error ? err.message : String(err), + ) + if (json) { + emitJsonError(failure.code, failure.message, failure.hint) + } else { + p.log.error(failure.message, CHROME) + if (failure.hint) p.log.info(failure.hint, CHROME) + } + throw new CliExit(1) + } +} + +function render( + verb: 'create' | 'rewrap', + result: KeysetResult, + outPath: string | undefined, + options: OutputOptions, +): void { + const envVar = recordVar(result.name) + const block = formatBlock(result, envVar) + const event = { + keysetId: result.keysetId, + ...(result.name !== undefined ? { name: result.name } : {}), + backend: result.backend, + mode: result.mode, + vendorKey: result.vendorKey, + envVar, + record: result.record, + } + const status = verb === 'create' ? 'created' : 'rewrapped' + + if (outPath) { + writeOut(outPath, block, options.force ?? false) + if (options.json) { + // Secret-free, as `stash env --write --json` is: a generated static key + // is in the 0600 file only. + emitJsonEvent({ + status: 'written', + path: outPath, + ...event, + ...(result.generatedStaticKey ? { staticKeyVar: STATIC_KEY_VAR } : {}), + }) + return + } + p.log.success(`Wrote ${outPath}`, CHROME) + } else if (options.json) { + emitJsonEvent({ + status, + ...event, + ...(result.generatedStaticKey + ? { + staticKeyVar: STATIC_KEY_VAR, + staticKey: result.generatedStaticKey, + } + : {}), + }) + return + } else { + console.log(block) + } + + if (result.generatedStaticKey) { + p.log.warn(messages.keyset.generatedStaticKeyWarning, CHROME) + } + p.log.info( + `No SDK reads ${envVar} by itself: pass its value to encrypt.WithKeysetRecords (Go) or KeysetRecord::parse (Rust).`, + CHROME, + ) + if (verb === 'rewrap') { + p.log.info(messages.keyset.rewrapNote, CHROME) + } + p.outro('Done!', CHROME) +} + +// --------------------------------------------------------------------------- +// Validation (no native code: these run before the binding loads) +// --------------------------------------------------------------------------- + +function rejectMissingValues(missing: MissingValues): void { + if (missing.length > 0) { + throw new KeysetError( + 'missing_flag_value', + `${missing.map((flag) => `--${flag}`).join(', ')} ${missing.length === 1 ? 'requires' : 'require'} a value.`, + ) + } +} + +function validateCreate(options: KeysetCreateOptions) { + const missing = [ + !options.backend && '--backend', + !options.vendorKey && '--vendor-key', + !options.mode && '--mode', + ].filter(Boolean) + if (missing.length > 0) { + throw new KeysetError( + 'missing_flag', + `${messages.keyset.missingCreateFlags}: ${missing.join(', ')}.`, + 'For example: stash keyset create --backend aws --vendor-key --mode wrapped', + ) + } + const backend = options.backend as string + if (!(KEYSET_BACKENDS as readonly string[]).includes(backend)) { + throw new KeysetError( + 'invalid_backend', + `${messages.keyset.unknownBackend} '${backend}'. Use one of: ${KEYSET_BACKENDS.join(', ')}.`, + messages.keyset.zeroKmsNote, + ) + } + const mode = options.mode as string + if (!(KEYSET_MODES as readonly string[]).includes(mode)) { + throw new KeysetError( + 'invalid_mode', + `Unknown key mode '${mode}'. Use wrapped or derived.`, + ) + } + if (backend === 'static' && mode !== 'derived') { + throw new KeysetError('mode_unsupported', messages.keyset.staticDerivedOnly) + } + if (backend !== 'static' && options.keyEnv !== undefined) { + throw new KeysetError( + 'invalid_option', + '--key-env is only for --backend static.', + ) + } + rejectControlChars('--vendor-key', options.vendorKey) + rejectControlChars('--name', options.name) + rejectEnvName('--key-env', options.keyEnv) + return { + backend, + vendorKey: options.vendorKey as string, + mode, + ...(options.name !== undefined ? { name: options.name } : {}), + ...(options.keyEnv !== undefined ? { staticKeyEnv: options.keyEnv } : {}), + } +} + +function validateRewrap(options: KeysetRewrapOptions) { + if (options.record !== undefined && options.in !== undefined) { + throw new KeysetError( + 'invalid_option', + 'Pass the record as an argument or with --in, not both.', + ) + } + let record: string + if (options.in !== undefined) { + record = readRecordFile(options.in) + } else if (options.record !== undefined) { + record = options.record.trim() + } else { + throw new KeysetError( + 'missing_record', + messages.keyset.missingRecord, + 'For example: stash keyset rewrap --in keyset.env --out keyset.env --force', + ) + } + if (!record.startsWith(RECORD_PREFIX)) { + throw new KeysetError( + 'invalid_record', + `This is not a keyset record: it does not start with ${RECORD_PREFIX}`, + ) + } + rejectControlChars('--vendor-key', options.vendorKey) + rejectEnvName('--key-env', options.keyEnv) + rejectEnvName('--new-key-env', options.newKeyEnv) + return { + record, + ...(options.vendorKey !== undefined + ? { vendorKey: options.vendorKey } + : {}), + ...(options.keyEnv !== undefined ? { staticKeyEnv: options.keyEnv } : {}), + ...(options.newKeyEnv !== undefined + ? { newStaticKeyEnv: options.newKeyEnv } + : {}), + } +} + +/** + * Read the one keyset record in `path`: a dotenv block that `--out` wrote, or + * a file holding the bare record. + */ +export function readRecordFile(path: string): string { + let text: string + try { + text = readFileSync(resolve(process.cwd(), path), 'utf-8') + } catch (err) { + throw new KeysetError( + 'read_failed', + `Could not read ${path}: ${err instanceof Error ? err.message : String(err)}`, + ) + } + const records = [...new Set(text.match(RECORD_PATTERN) ?? [])] + if (records.length === 0) { + throw new KeysetError( + 'invalid_record', + `${path} holds no keyset record (no ${RECORD_PREFIX} text).`, + ) + } + if (records.length > 1) { + throw new KeysetError( + 'invalid_record', + `${path} holds ${records.length} keyset records. Rewrap them one at a time: pass each record as an argument.`, + ) + } + return records[0] as string +} + +function rejectControlChars(flag: string, value: string | undefined): void { + if (value !== undefined && CONTROL_CHARS.test(value)) { + throw new KeysetError( + 'invalid_option', + `${flag} must not contain newlines or control characters.`, + ) + } +} + +/** `--key-env` names a variable; it never holds the key itself. */ +function rejectEnvName(flag: string, value: string | undefined): void { + if (value !== undefined && !/^[A-Za-z_][A-Za-z0-9_]*$/.test(value)) { + throw new KeysetError( + 'invalid_option', + `${flag} takes the NAME of an environment variable (for example ${flag} ${STATIC_KEY_VAR}), not the key itself.`, + ) + } +} + +function resolveOut(options: OutputOptions): string | undefined { + return options.out === undefined + ? undefined + : resolve(process.cwd(), options.out) +} + +// --------------------------------------------------------------------------- +// Output +// --------------------------------------------------------------------------- + +/** + * The dotenv variable for a record: `CS_KEYSET_RECORD`, or + * `CS_KEYSET_RECORD_` for a named keyset. + */ +export function recordVar(name: string | undefined): string { + const suffix = (name ?? '') + .toUpperCase() + .replace(/[^A-Z0-9]+/g, '_') + .replace(/^_+|_+$/g, '') + return suffix ? `${RECORD_VAR}_${suffix}` : RECORD_VAR +} + +/** + * The dotenv block. Values in comments are JSON-quoted, so a name or key + * reference from a record cannot break out of its comment line. + */ +export function formatBlock(result: KeysetResult, envVar: string): string { + const lines = [ + '# CipherStash keyset record for a key vendor backend.', + `# backend: ${result.backend}, mode: ${result.mode}`, + `# keyset id: ${result.keysetId}${result.name !== undefined ? `, name: ${JSON.stringify(result.name)}` : ''}`, + `# vendor key: ${JSON.stringify(result.vendorKey)}`, + '# The record is not secret. Only the vendor key can unwrap its key.', + `${envVar}=${result.record}`, + ] + if (result.generatedStaticKey) { + lines.push( + '', + `# The static key for vendor key ${JSON.stringify(result.vendorKey)}. THIS IS A SECRET:`, + '# keep it in a secret store. It is the only copy, and without it the keyset cannot be read.', + `${STATIC_KEY_VAR}=${result.generatedStaticKey}`, + ) + } + return lines.join('\n') +} + +function writeOut(path: string, block: string, force: boolean): void { + try { + // `wx` refuses an existing file atomically; `--force` overwrites it. + writeFileSync(path, `${block}\n`, { + encoding: 'utf-8', + mode: 0o600, + flag: force ? 'w' : 'wx', + }) + chmodSync(path, 0o600) + } catch (err) { + if ((err as NodeJS.ErrnoException).code === 'EEXIST') { + throw new KeysetError( + 'write_conflict', + `${path} already exists. Pass --force to overwrite it, or choose another path.`, + ) + } + throw new KeysetError( + 'write_failed', + `Could not write ${path}: ${err instanceof Error ? err.message : String(err)}`, + ) + } +} + +// --------------------------------------------------------------------------- +// The native binding +// --------------------------------------------------------------------------- + +interface NativeKeyset { + keysetCreate(options: unknown): Promise + keysetRewrap(options: unknown): Promise +} + +async function loadNative(): Promise { + const mod: { default?: unknown } = await import('@cipherstash/auth') + const auth = mod.default ?? mod + if (!hasKeysetFunctions(auth)) { + throw new KeysetError( + 'native_outdated', + 'The installed @cipherstash/auth has no keyset functions. Install the @cipherstash/auth version this stash release pins.', + ) + } + return auth +} + +function hasKeysetFunctions(value: unknown): value is NativeKeyset { + if (typeof value !== 'object' || value === null) return false + const candidate = value as Record + return ( + typeof candidate.keysetCreate === 'function' && + typeof candidate.keysetRewrap === 'function' + ) +} + +function unwrap(result: NativeResult): KeysetResult { + if (result.failure) { + const failure = result.failure + if (failure.type === 'REWRAP_REFUSED') { + throw new KeysetError( + 'rewrap_refused', + messages.keyset.rewrapRefused, + messages.keyset.rewrapRefusedHint, + ) + } + throw new KeysetError( + failure.type.toLowerCase(), + failure.error.message, + failure.help ?? hintFor(failure), + ) + } + return result.data +} + +/** A remedy for the failures that have an obvious one. */ +function hintFor(failure: KeysetFailure): string | undefined { + switch (failure.type) { + case 'VENDOR_CONFIG': + return messages.keyset.vendorConfigHint + case 'VENDOR_ERROR': + return failure.kind === 'AccessDenied' + ? 'Check that the credentials in this environment may use the vendor key (encrypt and decrypt).' + : undefined + default: + return undefined + } +} diff --git a/languages/typescript/packages/cli/src/messages.ts b/languages/typescript/packages/cli/src/messages.ts index a6a5776c03..b011ac072b 100644 --- a/languages/typescript/packages/cli/src/messages.ts +++ b/languages/typescript/packages/cli/src/messages.ts @@ -451,6 +451,31 @@ export const messages = { nameRequiresValue: '--name requires a value', unexpectedArgument: 'Unexpected argument', }, + keyset: { + /** + * `stash keyset` copy that the unit and e2e suites assert on. Every one + * of these fires before the native binding loads or a vendor is called. + */ + zeroKmsNote: + 'ZeroKMS users never need stash keyset: it is only for keysets in a key vendor backend (AWS KMS, Google Cloud KMS, Azure Key Vault, Vault Transit, or a static key).', + unknownSubcommand: 'Unknown keyset command', + missingCreateFlags: 'stash keyset create needs', + unknownBackend: 'Unknown key backend', + staticDerivedOnly: + 'The static key backend offers the derived key mode only. Pass --mode derived.', + missingRecord: + 'Pass the keyset record to rewrap, as an argument or with --in ', + generatedStaticKeyWarning: + 'A new static key was generated (CS_STATIC_KEY). It is a secret and the only copy: put it in a secret store now. Without it, nothing sealed under this keyset can be read.', + rewrapNote: + 'Replace the old record with the new one in every app. Stored values are not re-keyed, and they stay readable: a wrapped-mode record moves only to a new version of its own vendor key, and a derived-mode record may move to any key.', + rewrapRefused: + 'This keyset uses the wrapped key mode, so its record cannot move to another vendor key: the values stored under it would no longer open.', + rewrapRefusedHint: + 'Each stored value keeps its data key wrapped under the vendor key that sealed it. Rewrap under a new version of the same key (omit --vendor-key after a rotation), or create a new keyset under the other key and re-encrypt the data into it.', + vendorConfigHint: + 'The vendor client reads its settings and credentials from the environment, as the SDKs do. Run `stash keyset create --help` for the variables of each backend.', + }, plan: { /** * Stable leader of the refusal shown when `plan --complete-rollout` runs diff --git a/languages/typescript/packages/cli/tests/e2e/keyset.e2e.test.ts b/languages/typescript/packages/cli/tests/e2e/keyset.e2e.test.ts new file mode 100644 index 0000000000..b087987e38 --- /dev/null +++ b/languages/typescript/packages/cli/tests/e2e/keyset.e2e.test.ts @@ -0,0 +1,159 @@ +import { mkdtempSync, readFileSync, rmSync, statSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterAll, describe, expect, it } from 'vitest' +import { messages } from '../../src/messages.js' +import { run } from '../helpers/run.js' +import { runPiped } from '../helpers/spawn-piped.js' + +/** + * `stash keyset create` / `rewrap` end to end. The vendor backends need cloud + * credentials, so the round trip uses the static key backend, which runs the + * real native binding (`keysetCreate` / `keysetRewrap` in @cipherstash/auth) + * with no network. The vendor clients are tested in their own crates. + */ + +/** A fixed, test-only static key (bytes 0..31), and a second one. */ +const TEST_ONLY_KEY_A = 'AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8=' +const TEST_ONLY_KEY_B = 'ICEiIyQlJicoKSorLC0uLzAxMjM0NTY3ODk6Ozw9Pj8=' + +const dir = mkdtempSync(join(tmpdir(), 'stash-keyset-e2e-')) +afterAll(() => rmSync(dir, { recursive: true, force: true })) + +function jsonLine(stdout: string): Record { + const line = stdout + .split('\n') + .map((l) => l.trim()) + .find((l) => l.startsWith('{')) + if (!line) throw new Error(`no JSON line in stdout: ${stdout}`) + return JSON.parse(line) as Record +} + +describe('stash keyset', () => { + it('help says ZeroKMS users never need it', async () => { + const r = await run(['keyset', 'create', '--help'], { + env: { npm_config_user_agent: '' }, + }) + expect(r.exitCode).toBe(0) + expect(r.output).toContain('Usage: npx stash keyset create [options]') + expect(r.output).toContain('ZeroKMS users never need this command') + for (const flag of ['--backend', '--vendor-key', '--mode', '--key-env']) { + expect(r.output).toContain(flag) + } + }) + + it('lists both subcommands in the group help and the banner', async () => { + const group = await run(['keyset', '--help'], { + env: { npm_config_user_agent: '' }, + }) + expect(group.output).toContain('keyset create') + expect(group.output).toContain('keyset rewrap') + const banner = await run(['--help']) + expect(banner.output).toContain('keyset create') + }) + + it('an unknown subcommand exits 1 with the ZeroKMS note', async () => { + const r = await runPiped(['keyset', 'bogus'], { timeoutMs: 8000 }) + expect(r.exitCode).toBe(1) + expect(r.stderr).toContain(messages.keyset.unknownSubcommand) + expect(r.stderr).toContain('ZeroKMS users never need stash keyset') + }) + + it('missing flags fail with a JSON error before any vendor call', async () => { + const r = await runPiped( + ['keyset', 'create', '--backend', 'aws', '--json'], + { + timeoutMs: 8000, + }, + ) + expect(r.exitCode).toBe(1) + expect(jsonLine(r.stdout)).toMatchObject({ + status: 'error', + code: 'missing_flag', + }) + }) + + it('creates a static keyset record and rewraps it to another key', async () => { + const env = { + CS_TEST_ONLY_KEY_A: TEST_ONLY_KEY_A, + CS_TEST_ONLY_KEY_B: TEST_ONLY_KEY_B, + } + const out = join(dir, 'keyset.env') + const created = await runPiped( + [ + 'keyset', + 'create', + '--backend', + 'static', + '--vendor-key', + 'key-a', + '--mode', + 'derived', + '--name', + 'users', + '--key-env', + 'CS_TEST_ONLY_KEY_A', + '--out', + out, + ], + { env, timeoutMs: 15000 }, + ) + expect(created.stderr).not.toContain('Error') + expect(created.exitCode).toBe(0) + expect(statSync(out).mode & 0o777).toBe(0o600) + const block = readFileSync(out, 'utf-8') + const record = /^CS_KEYSET_RECORD_USERS=(stack-keyset:\S+)$/m.exec( + block, + )?.[1] + expect(record).toBeDefined() + // The key came from the environment, so none is written. + expect(block).not.toContain('CS_STATIC_KEY=') + expect(block).not.toContain(TEST_ONLY_KEY_A) + + const rewrapped = await runPiped( + [ + 'keyset', + 'rewrap', + '--in', + out, + '--key-env', + 'CS_TEST_ONLY_KEY_A', + '--vendor-key', + 'key-b', + '--new-key-env', + 'CS_TEST_ONLY_KEY_B', + '--json', + ], + { env, timeoutMs: 15000 }, + ) + expect(rewrapped.exitCode).toBe(0) + const event = jsonLine(rewrapped.stdout) + expect(event).toMatchObject({ + status: 'rewrapped', + name: 'users', + backend: 'static', + mode: 'derived', + vendorKey: 'key-b', + envVar: 'CS_KEYSET_RECORD_USERS', + }) + expect(event.record).not.toBe(record) + + // The old key no longer opens the rewrapped record. + const wrong = await runPiped( + [ + 'keyset', + 'rewrap', + String(event.record), + '--key-env', + 'CS_TEST_ONLY_KEY_A', + '--json', + ], + { env, timeoutMs: 15000 }, + ) + expect(wrong.exitCode).toBe(1) + expect(jsonLine(wrong.stdout)).toMatchObject({ + status: 'error', + code: 'vendor_error', + }) + }) +}) diff --git a/mise.toml b/mise.toml index c0a664fa8c..131082f089 100644 --- a/mise.toml +++ b/mise.toml @@ -10,6 +10,11 @@ includes = [ "packages/stack-auth/tasks.toml", "packages/stack-kms/tasks.toml", + "packages/stack-kms-vendor/tasks.toml", + "packages/stack-kms-aws/tasks.toml", + "packages/stack-kms-vault/tasks.toml", + "packages/stack-kms-gcp/tasks.toml", + "packages/stack-kms-azure/tasks.toml", "packages/stack-profile/tasks.toml", "packages/stack-encrypt/tasks.toml", "packages/stack-guest-abi/tasks.toml", @@ -119,6 +124,28 @@ check() { check stack-auth --no-default-features check stack-kms --no-default-features check stack-encrypt --no-default-features +# The vendor seam the guest will carry key vendor backends through +# (ADR-0011): no transport of its own, so it must build for WASI as is. Its +# `http` feature is the native transport, gated to non-wasm32 targets, so it +# must add nothing here either. +check stack-kms-vendor +check stack-kms-vendor --features http +# The AWS KMS key vendor backend, default features on: its native transport +# and workload identity credential sources are gated off on wasm32. +check stack-kms-aws +# The Vault Transit key vendor backend, default features on: its native +# transport and Kubernetes auth are gated off on wasm32. +check stack-kms-vault +# The Google Cloud KMS key vendor backend, default features on: its native +# transport, key files and metadata server are gated off on wasm32. Its RS256 +# signer (the `rsa` crate) is pure Rust. +check stack-kms-gcp +# Without `service-account`, the form the Go guest builds (CIP-5311): no RSA. +check stack-kms-gcp --no-default-features +# The Azure Key Vault key vendor backend, default features on: its native +# transport, workload identity and managed identity are gated off on wasm32. +# It does no RSA itself (Azure does). +check stack-kms-azure # The Go binding's credential guest (ADR-0005) compiles stack-profile for # WASI: the hostname and the process id are native-only there. check stack-profile @@ -126,6 +153,8 @@ check stack-profile # status table, transport import); its export and import modules exist only # here. check stack-guest-abi +# The guest's vendor transport over the step loop. +check stack-guest-abi --features vendor echo "all WASI crates compile with no JS-host or native-HTTP deps" """ @@ -198,8 +227,9 @@ echo "embedded into languages/golang/encrypt/wasm/stack_encrypt_guest.wasm" # under encrypt/testdata, which `go build` and the package's `//go:embed # wasm` both ignore, so no binary ever carries it; the tests read it from # disk and skip when it is absent. The import-surface gate denies what the real -# build's does; it requires no transport import, because a build with no -# ZeroKMS client has none. +# build's does; it requires no token_get, because a build with no ZeroKMS +# client has none, and the key vendor clients need no import: they reach the +# host through the step loop. [tasks."wasm:guest:build:deterministic"] description = "Build the deterministic-kms TEST build of the stack-encrypt WASI guest (wasm32-wasip1, release), for the Go fixture and round-trip tests" shell = "bash -c" @@ -217,7 +247,9 @@ python3 "$root/scripts/check-wasm-imports.py" "$module" \\ --deny-prefix wasi_snapshot_preview1:fd_prestat \\ --require wasi_snapshot_preview1:random_get # No transport imports to require: with no ZeroKMS client in the build the -# linker drops the host module altogether, which is the point of the build. +# linker drops the host module altogether. The key vendor clients stay +# (CIP-5311) and reach the host through the step loop, so the hermetic Go +# tests drive a vendor through a fake RoundTripper with no import at all. cp "$module" ../testdata/stack_encrypt_guest_deterministic.wasm echo "test build at languages/golang/encrypt/testdata/stack_encrypt_guest_deterministic.wasm" """ diff --git a/packages/eql/crates/eql-bindings/src/encryption.rs b/packages/eql/crates/eql-bindings/src/encryption.rs index 4baa22fcf6..1b0ed661ed 100644 --- a/packages/eql/crates/eql-bindings/src/encryption.rs +++ b/packages/eql/crates/eql-bindings/src/encryption.rs @@ -571,6 +571,37 @@ mod tests { ); } + /// EQL carries a leaf as its opaque `SealedValue` bytes, so a v2 leaf (a + /// key vendor backend's) passes through `c` unchanged: same prefix, same + /// payload shape, and the backend and mode read back as written. + #[test] + fn a_v2_leaf_passes_through_the_ciphertext_as_opaque_bytes() { + use stack_encrypt::kms::{KeyBackend, KeyMode, KeyVendor}; + + let leaf = SealedValue::from_vendor_parts( + KeyVendor::Aws, + KeyMode::Wrapped, + Default::default(), + [3; 16], + vec![9; 8], + vec![0xDE, 0xAD], + ) + .unwrap(); + let bytes = leaf.to_bytes(); + let ciphertext = CiphertextVisitor.sealed(leaf).unwrap(); + assert_eq!( + ciphertext.0, + format!("{CIPHERTEXT_PREFIX}{}", STANDARD.encode(&bytes)) + ); + + let CipherText::Single(read) = ciphertext.read(NativeCiphertextVisitor).unwrap() else { + panic!("a ciphertext reads back as one leaf"); + }; + assert_eq!(read.to_bytes(), bytes); + assert_eq!(read.key_backend(), KeyBackend::Vendor(KeyVendor::Aws)); + assert_eq!(read.key_mode(), Some(KeyMode::Wrapped)); + } + #[test] fn a_separator_in_a_name_is_escaped_never_a_third_part() { let odd = id("users/email", "x"); diff --git a/packages/stack-encrypt/CONTEXT.md b/packages/stack-encrypt/CONTEXT.md index ee3130de66..40beb9df7d 100644 --- a/packages/stack-encrypt/CONTEXT.md +++ b/packages/stack-encrypt/CONTEXT.md @@ -261,13 +261,72 @@ and whose ZeroKMS key requests are queued but not sent. Pendings compose (`zip`, `map`, `all`) so a whole struct or `Vec` settles in one batched call. _Avoid_: future, promise +**Settle**: +A pending's key requests going out and coming back as one batched round +trip to the key backend; composed pendings settle together. The unit a key +lifetime is measured in: under wrapped data keys, no key outlives one +settle. +_Avoid_: operation (that is what produces a target), request, flush + +**Key lifetime**: +How long the cipher may keep an index key after its key backend returned +it: one settle only, for a TTL, or until the bounded cache evicts it (the +ZeroKMS behaviour). The key backend states it (`KeyLifetime`); the cipher +enforces it. +_Avoid_: key TTL (a TTL is one of the lifetimes), cache policy + **Keyset**: -The ZeroKMS key domain a data key is minted under and an index key belongs -to — one per tenant is the common shape. A client may use any number; -`StackCipher` is scoped to the client, not to a keyset. Its **id** (a UUID) -is its identity: globally unique, carried in every sealed leaf, never -re-checked. -_Avoid_: dataset, key ring, tenant (a tenant *has* a keyset) +The key domain a data key is minted under and an index key belongs to — +one per tenant is the common shape. A keyset lives in exactly one key +backend: in ZeroKMS it is a ZeroKMS keyset; in a third-party vendor's KMS +it is a keyset record the customer creates, naming the vendor key that +protects it. A client may use any number; `StackCipher` is scoped to the +client, not to a keyset. Its **id** (a UUID) is its identity: globally +unique, carried in every sealed leaf, never re-checked. +_Avoid_: dataset, key ring, tenant (a tenant *has* a keyset), vendor key +(that is what protects a keyset, not the keyset) + +**Key backend**: +The service a keyset's data keys and index key come from: ZeroKMS, or one +key vendor backend. Every keyset lives in exactly one. +_Avoid_: KMS (ambiguous between ZeroKMS and a vendor's), key provider (that +is the source of ZeroKMS client credentials), key source + +**Key vendor backend**: +A key backend that is not ZeroKMS: AWS KMS, Azure Key Vault, HashiCorp +Vault Transit, Google Cloud KMS, or a static key. It works at less than +ZeroKMS parity, and each gap is a stated trade-off: what it cannot do it +refuses, never ignores. +_Avoid_: vendor KMS, third-party KMS, BYOK (that is ZeroKMS rooted in a +customer's KMS, not ZeroKMS replaced by it) + +**Keyset record**: +The customer-held description of a keyset in a key vendor backend: its id, +its backend, its key mode, the vendor key that protects it, and the wrapped +key material. ZeroKMS keeps its own keysets, so a ZeroKMS user never holds +or persists one. +_Avoid_: keyset config, key file + +**Wrapped data key**: +The key mode where no key outlives the settle that needed it. Each value +has its own data key, generated and wrapped by the vendor key and stored, +wrapped, with the value; the index key is wrapped in the keyset record and +unwrapped once per settle. Search terms are still built in the client. The +vendor sees, and can refuse, every settle and every value. +_Avoid_: envelope mode, DEK, no-key mode (keys exist, only never beyond one +settle) + +**Derived data key**: +The key mode where a keyset's root key is wrapped by the vendor key, and +each value's data key is derived from it locally. Opening a value costs no +vendor call once the root key is unwrapped; the vendor sees and can refuse +the keyset, not the value. +_Avoid_: local mode, cached mode + +**Root key**: +Under derived data keys, the keyset secret every data key is derived from. It is held unwrapped in the application's memory, for a +bounded time. +_Avoid_: master key, KEK (the vendor key is the one that wraps) **Keyset cipher**: `KeysetCipher`, the cipher bound to one keyset, and what every operation diff --git a/packages/stack-encrypt/docs/adr/0009-two-key-modes-for-key-vendor-backends.md b/packages/stack-encrypt/docs/adr/0009-two-key-modes-for-key-vendor-backends.md new file mode 100644 index 0000000000..eac845b12e --- /dev/null +++ b/packages/stack-encrypt/docs/adr/0009-two-key-modes-for-key-vendor-backends.md @@ -0,0 +1,67 @@ +--- +status: proposed +date: 2026-11-04 +--- + +# Two key modes for key vendor backends + +CIP-5203 makes Stack Encrypt usable with AWS KMS, Azure Key Vault, HashiCorp +Vault Transit and Google Cloud KMS as alternatives to ZeroKMS. An earlier +project tried to make KMS support fully general in the vitaminc layer. That +was too complex, so this work adds key vendor backends at the +`DataKeySource` / `IndexKeySource` seam in `stack-kms` and accepts less than +ZeroKMS parity. Each gap is a stated trade-off. + +ZeroKMS gives each value its own data key and settles a batch of N values in +one call. The vendors cannot do this. AWS KMS, Google Cloud KMS and Azure Key +Vault unwrap one key per call and have no batch operation. Only Vault Transit +batches. There is no single vendor design that is both "per-value keys" and +"few vendor calls". We therefore give each vendor backend two key modes, and +the customer chooses one per keyset: + +- **Wrapped data key.** No key outlives the settle that needed it. Each value + has its own data key, generated and wrapped by the vendor key, and stored + wrapped in the leaf. The index key is wrapped in the keyset record and + unwrapped once per settle. A settle of N values costs about 1 + N vendor + calls. The vendor sees, and can refuse, every settle and every value. +- **Derived data key.** The keyset record holds a root key wrapped by the + vendor key. The cipher unwraps it once and caches it for a TTL. Each data + key is HKDF-SHA256(root key, salt = iv, info = keyset id ‖ descriptor), and + the index key is derived from the same root key under a separate label. + Vendor calls are O(keysets), not O(values). The vendor sees, and can refuse, + the keyset unwrap, not the value. + +In both modes, search terms are built in the client, as they are today, and +Postgres holds no usable key material. + +## Considered options + +- **One mode per vendor.** Rejected. A customer who needs per-value audit and + refusal and a customer who needs throughput would each have no option on + some vendors. +- **Vendor-side PRF for search terms** (AWS `GenerateMac`, GCP MAC keys, + Vault `hmac`), so that the index key is never in the client. Rejected. The + vendor would receive the search input, or a hash of it that it can + brute-force for low-entropy values. One call per term would make a match + index cost about 20 calls per value. Azure has no MAC operation. "No key + outlives one settle" gives the same refusal property without these costs. +- **A hybrid mode** (per-value data keys and a cached index key). Rejected. + Its trade-off is not clear enough to justify a third mode in the test + matrix. +- **Vendor-side encryption of plaintext.** Rejected. It moves the trust + boundary to the vendor and has size limits (4 KiB on AWS). + +## Consequences + +- The cipher enforces key lifetime, not each backend. `load_index_key` + returns a lifetime (*this settle only*, *cache for TTL*), and the per-keyset + cache obeys it. One test then covers every backend. +- Under wrapped data keys, each separate `encryptQuery` costs one vendor + round trip. A caller who wants one round trip composes query terms into one + settle. +- The static key backend offers derived data keys only, because a static key + is always in memory. +- A vendor backend refuses ZeroKMS-only features (decryption policy, lock + context, `UnverifiedContext`) with a typed error. It never ignores them. +- A cipher holds ZeroKMS and at most one key vendor backend. v1 has no + migration between backends in either direction. diff --git a/packages/stack-encrypt/docs/adr/0010-a-v2-leaf-names-its-key-backend-and-mode.md b/packages/stack-encrypt/docs/adr/0010-a-v2-leaf-names-its-key-backend-and-mode.md new file mode 100644 index 0000000000..9892e7cd69 --- /dev/null +++ b/packages/stack-encrypt/docs/adr/0010-a-v2-leaf-names-its-key-backend-and-mode.md @@ -0,0 +1,41 @@ +--- +status: proposed +date: 2026-11-04 +--- + +# A v2 leaf names its key backend and key mode + +A v1 leaf (`SealedValue`, `cipher.rs`) is +`0x01 ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ ct`. It has no field that names a +key backend, because ZeroKMS was the only one. ADR-0009 lets one cipher hold +ZeroKMS and one key vendor backend, with two key modes. A reader must +therefore know which backend and which mode sealed a leaf before it asks for +a key. + +We keep v1 unchanged, byte for byte, and ZeroKMS writers continue to write +it. Every value sealed under ZeroKMS stays decryptable with no migration. +Leaves sealed by a key vendor backend use v2: + +``` +0x02 ‖ backend ‖ mode ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ ct +``` + +Under wrapped data keys the tag holds the wrapped data key. Under derived +data keys the tag is empty, and the iv is the derivation salt. `leaf_aad` +already binds the version and the header into the AAD, so the new bytes are +authenticated with no new mechanism. + +A v1-only reader refuses a v2 leaf with `UnknownVersion`. A cipher that does +not hold the backend a v2 leaf names refuses it before any key call, as it +refuses a foreign keyset today. EQL carries the leaf as opaque `SealedValue` +bytes, so the EQL payload shape does not change. The TypeScript FFI does not +use this leaf format (protect-ffi goes through `cipherstash-client`), so it is +not affected. + +## Considered options + +- **A v2 for every leaf, ZeroKMS included.** Rejected. It changes stored data + and every writer for no gain, and it puts existing data at risk. +- **Infer the backend from the keyset id.** Rejected. A reader would need + every keyset record before it could even route a leaf, and a wrong record + would produce a failed unwrap rather than a clear refusal. diff --git a/packages/stack-encrypt/docs/adr/0011-hand-written-vendor-clients-signed-in-the-guest.md b/packages/stack-encrypt/docs/adr/0011-hand-written-vendor-clients-signed-in-the-guest.md new file mode 100644 index 0000000000..2520d1586e --- /dev/null +++ b/packages/stack-encrypt/docs/adr/0011-hand-written-vendor-clients-signed-in-the-guest.md @@ -0,0 +1,47 @@ +--- +status: proposed +date: 2026-11-04 +--- + +# Hand-written vendor clients, signed in the guest + +The Go SDK reaches Stack Encrypt through a WASI guest. The guest builds the +ZeroKMS requests and derives the keys. The Go host is only an HTTP pipe +(`transport_send`) and a token source, and no key material crosses back to the +host (ADR-0005). Key vendor backends (ADR-0009) must keep that property. + +The official Rust SDKs (`aws-sdk-kms`, the Google Cloud and Azure crates) +depend on tokio and hyper and do not build for wasm32-wasip1. We therefore +write a small REST client for each vendor, over one transport trait of the +same shape as `ZeroKMSConnection`. Native Rust and the guest use the same +code. Each vendor is a separate crate (`stack-kms-aws` and so on), so that no +user compiles all four. + +The guest signs each vendor request itself (SigV4, the Google OAuth JWT, the +Azure AD token, the Vault token). It gets short-lived signing credentials from +the Go host through a new credential import, of the same shape as `token_get`. +The host fills the import from the vendor's own Go credential chain, so we do +not implement IMDS, the GCP metadata server or Azure managed identity in the +guest. Credentials cross the boundary. Key material does not. + +The Go module ships one guest that contains all vendor clients, not one guest +for each vendor. This decision depends on a measurement of the guest size. + +## Considered options + +- **Official SDKs on native Rust and hand-written clients in the guest.** + Rejected. There would be two code paths for each vendor, and the guest path + would be the less tested one. +- **The Go host calls the vendor's Go SDK and passes the unwrapped key to the + guest.** Rejected. It breaks the rule that key material never crosses to the + host. +- **One guest for each vendor.** Rejected for now. It multiplies the embedded + `.wasm` files, the deterministic test builds and the release workflow by up + to five. + +## Consequences + +We own the request signing code for four vendors, and its tests. Recorded +HTTP fixtures cover every vendor. LocalStack and `vault server -dev` run on +every PR. Live Google Cloud and Azure tests run in an opt-in, credentialed +workflow. diff --git a/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-aws-wrapped b/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-aws-wrapped new file mode 100644 index 0000000000..e9778634e5 Binary files /dev/null and b/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-aws-wrapped differ diff --git a/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-static-derived-empty-tag b/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-static-derived-empty-tag new file mode 100644 index 0000000000..e0d3f3010c Binary files /dev/null and b/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-static-derived-empty-tag differ diff --git a/packages/stack-encrypt/fuzz/fuzz_targets/sealed_value_decode.rs b/packages/stack-encrypt/fuzz/fuzz_targets/sealed_value_decode.rs index a38487378f..5c8ae321d0 100644 --- a/packages/stack-encrypt/fuzz/fuzz_targets/sealed_value_decode.rs +++ b/packages/stack-encrypt/fuzz/fuzz_targets/sealed_value_decode.rs @@ -3,9 +3,10 @@ use libfuzzer_sys::fuzz_target; use stack_encrypt::SealedValue; -// Fuzz the frozen v1 leaf byte decoder. A stored ciphertext column comes back -// through `SealedValue::from_bytes` before anything is authenticated, so the -// bytes are attacker-controlled: decoding must never panic, only return `Err`. +// Fuzz the frozen leaf byte decoder, v1 (ZeroKMS) and v2 (a key vendor +// backend: `0x02 ‖ backend ‖ mode ‖ …`); the seed corpus holds both. A +// stored ciphertext column comes back through `SealedValue::from_bytes` +// before anything is authenticated, so the bytes are attacker-controlled: decoding must never panic, only return `Err`. // // The format is documented as lossless, so a second invariant is checked on // every accepted input: re-encoding gives back exactly the bytes decoded. diff --git a/packages/stack-encrypt/src/cipher.rs b/packages/stack-encrypt/src/cipher.rs index 0efa88ef09..4ab70362e1 100644 --- a/packages/stack-encrypt/src/cipher.rs +++ b/packages/stack-encrypt/src/cipher.rs @@ -68,10 +68,12 @@ //! `aws-lc-rs` on native, RustCrypto on wasm32; vitaminc's own random nonce //! and versioned leaf layout). The leaf AAD is the labelled derivation //! `leaf_aad` (private): -//! `PAE("stack-encrypt/leaf", version, keyset_id, derived_aad, tag)`, with -//! [`SealedValue::FORMAT_VERSION`] — the version byte that prefixes the -//! leaf's frozen byte encoding ([`SealedValue::to_bytes`]) — and the keyset -//! id bound under the tag, so a stored leaf relabelled with a different +//! `PAE("stack-encrypt/leaf", header, keyset_id, derived_aad, tag)`, with +//! the leaf's envelope header — the [`SealedValue::FORMAT_VERSION`] byte +//! of a v1 (ZeroKMS) leaf, or `version ‖ backend ‖ mode` of a v2 leaf of a +//! key vendor backend, the bytes that prefix the leaf's frozen byte +//! encoding ([`SealedValue::to_bytes`]) — and the keyset id bound under +//! the tag, so a stored leaf relabelled with a different //! version byte fails verification instead of selecting different parsing //! rules, and one re-pointed at another keyset fails instead of asking that //! keyset for a key it never minted. The `tag` is always bound, so the @@ -97,7 +99,10 @@ use std::sync::{Arc, Mutex, PoisonError}; use std::time::Duration; use serde::{Deserialize, Serialize}; -use stack_kms::{DataKey, DataKeySource, DataKeyWithTag, IdentifiedBy, IndexKeySource}; +use stack_kms::{ + DataKey, DataKeySource, DataKeyWithTag, IdentifiedBy, IndexKeySource, KeyBackend, KeyMode, + KeyVendor, +}; #[cfg(feature = "http")] use stack_kms::{EnvKeyProvider, StackKms, StackKmsBuilder}; #[cfg(all(feature = "http", not(target_arch = "wasm32")))] @@ -111,7 +116,7 @@ use vitaminc_aead::{ use vitaminc_encrypt::{Aes256Cipher, Key as AesKey}; use vitaminc_protected::Protected; -use crate::keyset::{KeysetCache, KeysetCipher, KeysetState, Lookup, DEFAULT_NAME_TTL}; +use crate::keyset::{KeysetCache, KeysetCipher, KeysetState, Lookup, Retention, DEFAULT_NAME_TTL}; use crate::Descriptor; /// The recursive ciphertext container produced by [`StackCipher`]: vitaminc's @@ -135,7 +140,7 @@ pub type StackCipherText = Node; #[derive(Debug, thiserror::Error, miette::Diagnostic)] #[non_exhaustive] pub enum Error { - /// A ZeroKMS data-key or keyset operation failed. Transparent: the + /// A key backend's data-key or keyset operation failed. Transparent: the /// message, code and help are the [`stack_kms::Error`]'s, so a caller /// across a binding sees `stack_kms::keyset_not_found` itself. #[error(transparent)] @@ -236,21 +241,34 @@ pub enum Error { #[error("expected {expected} data keys from ZeroKMS but received {received}")] #[diagnostic(code(stack_encrypt::key_count_mismatch))] KeyCountMismatch { expected: usize, received: usize }, - /// A context rendered to a descriptor longer than ZeroKMS can bind - /// ([`Descriptor::MAX_LEN`]). Raised before any request is sent, so no - /// key is minted or retrieved for the batch. + /// A context rendered to a descriptor longer than a key backend can + /// bind ([`Descriptor::MAX_LEN`]). Raised before any request is sent, so + /// no key is minted or retrieved for the batch. #[error( - "context renders to a {len}-byte ZeroKMS descriptor; the limit is {} bytes", + "context renders to a {len}-byte key descriptor; the limit is {} bytes", Descriptor::MAX_LEN )] #[diagnostic( code(stack_encrypt::descriptor_too_long), help( - "ZeroKMS binds a descriptor of at most {} bytes. Shorten the context: fewer parts, shorter labels, or an id in place of a long name.", + "A key backend binds a descriptor of at most {} bytes. Shorten the context: fewer parts, shorter labels, or an id in place of a long name.", Descriptor::MAX_LEN ) )] DescriptorTooLong { len: usize }, + /// The key backend gave the default keyset's index key a lifetime + /// shorter than the cipher's own ([`stack_kms::KeyLifetime::Settle`] or + /// [`stack_kms::KeyLifetime::Ttl`]). The cipher holds the default keyset + /// for its whole life, so it refuses to start rather than keep a key + /// longer than its backend allows. + #[error( + "the key backend does not let the cipher keep the index key of the default keyset {keyset}" + )] + #[diagnostic( + code(stack_encrypt::default_keyset_lifetime), + help("Use a key backend whose default keyset the cipher may keep, and select keysets with a shorter key lifetime with `StackCipher::keyset`.") + )] + DefaultKeysetLifetime { keyset: Uuid }, /// Building a ZeroKMS client from the environment failed: credentials or /// client key missing or malformed. /// @@ -371,6 +389,33 @@ pub enum Error { ) )] ForeignKeyset { expected: Uuid, found: Uuid }, + /// A leaf names another key backend than the cipher's. A v1 leaf names + /// ZeroKMS; a v2 leaf names its key vendor backend + /// ([`SealedValue::key_backend`]). The cipher refuses the leaf before + /// any key call, as it refuses a [`ForeignKeyset`](Self::ForeignKeyset), + /// so the leaf never reaches a backend that did not seal it. The leaf's + /// backend byte is read before the leaf is authenticated: read this as + /// "not this cipher's leaf", never as "an untampered leaf". + #[error("leaf names key backend {found}, but this cipher's key backend is {expected}")] + #[diagnostic( + code(stack_encrypt::foreign_backend), + help("Open the value with a cipher that holds the key backend that sealed it.") + )] + ForeignBackend { + expected: KeyBackend, + found: KeyBackend, + }, + /// A data key was requested from a key vendor backend. This build seals + /// v1 leaves only, and a v1 leaf names ZeroKMS, so it cannot seal under + /// another key backend. Refused before any key call. + #[error("this build cannot seal a value under key backend {backend}")] + #[diagnostic( + code(stack_encrypt::seal_unsupported), + help( + "Seal values under ZeroKMS. Sealing under a key vendor backend is not available yet." + ) + )] + SealUnsupported { backend: KeyBackend }, /// A data key was requested through a [`StackCipher`] rather than a /// [`KeysetCipher`]: a [`Request::generate_data_key`] needs a keyset /// to mint under, and only a keyset-scoped pending has one. Always a @@ -450,10 +495,18 @@ impl crate::ErrorPayload for Error { fields.extend(source.payload()); fields } + Self::DefaultKeysetLifetime { keyset } => { + payload([("keyset", keyset.to_string().into())]) + } Self::ForeignKeyset { expected, found } => payload([ ("expected", expected.to_string().into()), ("found", found.to_string().into()), ]), + Self::ForeignBackend { expected, found } => payload([ + ("expected", expected.name().into()), + ("found", found.name().into()), + ]), + Self::SealUnsupported { backend } => payload([("backend", backend.name().into())]), #[cfg(feature = "http")] Self::Config(error) => error .downcast_ref::() @@ -692,6 +745,14 @@ impl StackCipher { /// ([`keyset_name_ttl`](StackCipherBuilder::keyset_name_ttl)) before it /// is asked again — ZeroKMS allows renames, and a running process /// notices one within that window. Selecting by id never asks twice. + /// + /// The key backend says how long the cipher may keep a keyset's index + /// key ([`KeyLifetime`](crate::kms::KeyLifetime)), and the cache obeys + /// it. ZeroKMS keys stay until evicted. A key with a TTL is loaded again + /// once the TTL ends. A key for one settle only is never cached: each + /// selection loads it again, and the returned handle (with what is built + /// from it) holds the only reference to it, so it drops with them. Hold + /// such a handle for one unit of work, not for the life of the process. pub async fn keyset( &self, keyset: impl Into, @@ -727,6 +788,11 @@ impl StackCipher { return Err(error); } }; + // An index key the cache may not keep goes to the caller alone: the + // handle, and what is built from it, hold the only references. + if state.retention == Retention::Never { + return Ok(KeysetCipher::new(self, state)); + } let state = self.keysets().insert(state, resolution); Ok(KeysetCipher::new(self, state)) } @@ -765,14 +831,19 @@ impl StackCipher { /// costs, shared by eager loading at /// [`init`](StackCipherBuilder::init) and lazy loading in /// [`StackCipher::keyset`] so both hold a keyset in exactly the same shape. -/// ZeroKMS answered a load with "no such keyset" — as opposed to not -/// answering at all. +/// The key backend answered a load with "no such keyset" — as opposed to +/// not answering at all. ZeroKMS says so as a keyset-not-found; a key vendor +/// backend as a [`NotFound`](stack_kms::BackendErrorKind::NotFound) failure. fn is_keyset_not_found(error: &Error) -> bool { matches!( error, - Error::Kms(stack_kms::Error::LoadKeyset( - stack_kms::LoadKeysetError::KeysetNotFound(_) - )) + Error::Kms( + stack_kms::Error::LoadKeyset(stack_kms::LoadKeysetError::KeysetNotFound(_)) + | stack_kms::Error::Backend { + kind: stack_kms::BackendErrorKind::NotFound, + .. + } + ) ) } @@ -784,32 +855,65 @@ async fn load_keyset( IdentifiedBy::Name(name) => Some(name.to_string()), IdentifiedBy::Uuid(_) => None, }; - let (id, index_key) = kms.load_index_key(Some(keyset)).await?; - Ok(keyset_state_of(id, name, &index_key)) + let selected = kms.select_keyset(Some(keyset)).await?; + keyset_state(selected, name) } -/// The state the cipher holds for a loaded keyset: its id, the name it was -/// selected by, and the PRF keyed by its index key. One construction for -/// every way a keyset is loaded. -fn keyset_state_of( - id: Uuid, +/// The state the cipher holds for a selected keyset. +/// +/// A loaded index key becomes its PRF, kept as long as the lifetime the key +/// backend gave it. A keyset whose index key is loaded inside each settle +/// ([`SelectedKeyset::PerSettle`](stack_kms::SelectedKeyset)) holds no key +/// material at all, so its state is kept until evicted, and every term +/// built through it is derived inside the settle. +fn keyset_state( + selected: stack_kms::SelectedKeyset, name: Option, - index_key: &stack_kms::IndexKey, -) -> Arc { - Arc::new(KeysetState { - id, - name, - prf: hmac_prf_from_index_key(index_key), - }) +) -> Result, Error> { + let state = match selected { + stack_kms::SelectedKeyset::Loaded(loaded) => KeysetState { + id: loaded.keyset_id, + name, + prf: Some(hmac_prf_from_index_key(&loaded.key)), + retention: Retention::of(loaded.lifetime, std::time::Instant::now()), + }, + stack_kms::SelectedKeyset::PerSettle { keyset_id } => KeysetState { + id: keyset_id, + name, + prf: None, + retention: Retention::UntilEvicted, + }, + // A selection this build does not know: refused, never guessed at. + _ => { + return Err(Error::Other( + "the key backend selected the keyset in a way this build does not know".into(), + )) + } + }; + Ok(Arc::new(state)) } /// The client's own default keyset — the one a ZeroKMS administrator set for /// this client — asked for by naming nothing. Loaded once, by /// [`init`](StackCipherBuilder::init); it is not the caller's to choose, so /// there is no id or name to carry. +/// +/// The cipher holds the default keyset for its whole life +/// ([`StackCipher::default_keyset`] is synchronous), so a key backend that +/// gives the cipher an index key it may not keep is refused here +/// ([`Error::DefaultKeysetLifetime`]), not obeyed in part. A key backend +/// that loads the index key inside each settle gives the cipher no key to +/// keep, so its default keyset is accepted. async fn load_default_keyset(kms: &K) -> Result, Error> { - let (id, index_key) = kms.load_index_key(None).await?; - Ok(keyset_state_of(id, None, &index_key)) + let selected = kms.select_keyset(None).await?; + if let stack_kms::SelectedKeyset::Loaded(loaded) = &selected { + if loaded.lifetime != stack_kms::KeyLifetime::UntilEvicted { + return Err(Error::DefaultKeysetLifetime { + keyset: loaded.keyset_id, + }); + } + } + keyset_state(selected, None) } /// The state of a [`StackCipherBuilder`] that has not been given a data-key @@ -1009,7 +1113,9 @@ impl StackCipherBuilder { /// Build the local HMAC-SHA256 PRF from a per-keyset /// [`IndexKey`](crate::kms::IndexKey), wiping the intermediate stack copy of the /// raw key bytes. -fn hmac_prf_from_index_key(index_key: &stack_kms::IndexKey) -> vitaminc_hmac::HmacSha256Prf { +pub(crate) fn hmac_prf_from_index_key( + index_key: &stack_kms::IndexKey, +) -> vitaminc_hmac::HmacSha256Prf { use zeroize::Zeroize; // `[u8; 32]` is `Copy`: the move into `Protected` leaves this stack copy @@ -1122,8 +1228,9 @@ impl StackCipher { } } -/// A single sealed leaf: the ZeroKMS metadata needed to retrieve its data key -/// (`iv`, `tag`) plus the vitaminc `LocalCipherText` sealed under that key. +/// A single sealed leaf: the key backend metadata needed to retrieve its +/// data key (`iv`, `tag`) plus the vitaminc `LocalCipherText` sealed under +/// that key. /// /// This is the only byte-format commitment the crate makes for *ciphertext* — /// the container tree ([`StackCipherText`]) has no canonical encoding, so @@ -1137,7 +1244,8 @@ impl StackCipher { /// [`to_bytes`](Self::to_bytes) / [`from_bytes`](Self::from_bytes) are the /// canonical encoding — the one storage format every consumer (this crate, /// the language bindings, anything reading a database column) agrees on. -/// The v1 layout: +/// There are two versions. ZeroKMS writes v1, and only v1. A key vendor +/// backend writes v2 (ADR-0010). The v1 layout: /// /// | offset | field | size | value | /// |-----------------|--------------------|-----------------|-------| @@ -1148,10 +1256,30 @@ impl StackCipher { /// | 35 | ZeroKMS key `tag` | `tag_len` | required to retrieve the key | /// | 35 + `tag_len` | local ciphertext | rest of buffer | the vitaminc `LocalCipherText` | /// +/// The v2 layout adds two bytes after the version, and is otherwise the +/// same: +/// +/// | offset | field | size | value | +/// |-----------------|--------------------|-----------------|-------| +/// | 0 | envelope version | 1 | [`VENDOR_FORMAT_VERSION`](Self::VENDOR_FORMAT_VERSION) (`0x02`) | +/// | 1 | key backend | 1 | [`KeyVendor::byte`](stack_kms::KeyVendor::byte): AWS KMS `1`, Google Cloud KMS `2`, Azure Key Vault `3`, Vault `4`, static `5` | +/// | 2 | key mode | 1 | [`KeyMode::byte`](stack_kms::KeyMode::byte): wrapped `1`, derived `2` | +/// | 3 | keyset id | 16 | the keyset the data key belongs to, raw UUID bytes | +/// | 19 | `iv` | 16 | identifies the data key; under derived data keys, the derivation salt | +/// | 35 | `tag_len` | 2 | length of `tag`, `u16` little-endian | +/// | 37 | `tag` | `tag_len` | under wrapped data keys, the wrapped data key; under derived data keys, empty | +/// | 37 + `tag_len` | local ciphertext | rest of buffer | the vitaminc `LocalCipherText` | +/// +/// The value `0` of the backend byte and of the mode byte is invalid, and +/// [`from_bytes`](Self::from_bytes) refuses a byte that names no backend or +/// no mode. ZeroKMS has no v2 backend byte, so a v2 leaf never names it. +/// /// The keyset id is what lets a leaf be opened without the caller saying /// which keyset it belongs to: retrieving the data key needs the keyset, /// and the leaf is self-describing so that a leaf lifted from a tree — what -/// a database column holds — is too. +/// a database column holds — is too. The backend byte does the same for +/// the key backend: a cipher refuses a leaf that names another key backend +/// than its own ([`Error::ForeignBackend`]) before any key call. /// /// The local ciphertext is itself a framed value — vitaminc's leaf wire /// format, versioned and owned by vitaminc — so the full stored byte string @@ -1165,22 +1293,26 @@ impl StackCipher { /// └─────────────────────────────────────────────────────────────────────────────────────┘ /// ``` /// -/// Both version bytes and the keyset id are authenticated under the one GCM -/// tag, each bound by the layer that owns its framing: the envelope version -/// and the keyset id through this crate's leaf-AAD derivation, -/// `PAE("stack-encrypt/leaf", version, keyset_id, derived_aad, tag)`, and -/// the inner version through vitaminc's `Context::for_leaf`, applied inside -/// `Aes256Cipher` to the AAD this crate hands it. Relabel either version -/// byte, or re-point the leaf at another keyset, in storage and the leaf -/// fails authentication rather than parsing under the wrong rules. Parsing -/// is structural only — nothing about a decoded leaf is trusted until it -/// decrypts. +/// Both version bytes, the v2 backend and mode bytes, and the keyset id are +/// authenticated under the one GCM tag, each bound by the layer that owns +/// its framing: the envelope header and the keyset id through this crate's +/// leaf-AAD derivation, +/// `PAE("stack-encrypt/leaf", header, keyset_id, derived_aad, tag)`, where +/// `header` is the version byte of a v1 leaf and `version ‖ backend ‖ mode` +/// of a v2 leaf, and the inner version through vitaminc's +/// `Context::for_leaf`, applied inside `Aes256Cipher` to the AAD this crate +/// hands it. Relabel a header byte, or re-point the leaf at another keyset, +/// in storage and the leaf fails authentication rather than parsing under +/// the wrong rules. Parsing is structural only — nothing about a decoded +/// leaf is trusted until it decrypts. /// /// The `serde` `Serialize`/`Deserialize` derives and /// [`into_parts`](Self::into_parts) / [`from_parts`](Self::from_parts) /// remain for callers that manage their own storage format; they carry the /// same fields, but their wire form is the serialiser's, not a commitment -/// of this crate. +/// of this crate. The serde form of a v1 leaf has no field for the backend, +/// and a v2 leaf adds one (`vendor`). `into_parts` drops the backend and +/// the mode, so persist a v2 leaf with [`to_bytes`](Self::to_bytes). #[derive(Debug, Serialize, Deserialize)] #[serde(try_from = "SealedValueRepr")] pub struct SealedValue { @@ -1194,12 +1326,79 @@ pub struct SealedValue { /// The leaf sealed by [`vitaminc_encrypt::Aes256Cipher`] under the data /// key: `version ‖ nonce ‖ ciphertext ‖ gcm_tag`. ciphertext: LocalCipherText, + /// The key vendor backend and key mode of a v2 leaf; `None` for a v1 + /// (ZeroKMS) leaf. Last, and absent from a v1 leaf's serde form, so + /// that form does not change. + #[serde(default, skip_serializing_if = "Option::is_none")] + vendor: Option, +} + +/// The two v2 header bytes: the key vendor backend and the key mode. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +#[serde(into = "[u8; 2]", try_from = "[u8; 2]")] +pub(crate) struct VendorHeader { + pub(crate) vendor: KeyVendor, + pub(crate) mode: KeyMode, +} + +impl VendorHeader { + /// Decode the backend byte and the mode byte. + fn from_bytes(vendor: u8, mode: u8) -> Result { + Ok(Self { + vendor: KeyVendor::from_byte(vendor).ok_or(LeafBytesError::UnknownBackend(vendor))?, + mode: KeyMode::from_byte(mode).ok_or(LeafBytesError::UnknownMode(mode))?, + }) + } +} + +impl From for [u8; 2] { + fn from(header: VendorHeader) -> Self { + [header.vendor.byte(), header.mode.byte()] + } +} + +impl TryFrom<[u8; 2]> for VendorHeader { + type Error = LeafBytesError; + + fn try_from([vendor, mode]: [u8; 2]) -> Result { + Self::from_bytes(vendor, mode) + } +} + +/// The envelope header bytes of a leaf, as written and as bound into the +/// leaf AAD: `version` for v1, `version ‖ backend ‖ mode` for v2. +struct LeafHeader { + bytes: [u8; 3], + len: usize, +} + +impl LeafHeader { + fn of(vendor: Option) -> Self { + match vendor { + None => Self { + bytes: [SealedValue::FORMAT_VERSION, 0, 0], + len: 1, + }, + Some(header) => Self { + bytes: [ + SealedValue::VENDOR_FORMAT_VERSION, + header.vendor.byte(), + header.mode.byte(), + ], + len: 3, + }, + } + } + + fn as_bytes(&self) -> &[u8] { + &self.bytes[..self.len] + } } /// A [`SealedValue`] byte encoding failed to encode or decode. Purely /// structural — a leaf that *decodes* has proven nothing about integrity /// (that is the AEAD open's job); a leaf that fails here was never a valid -/// v1 encoding at all. +/// v1 or v2 encoding at all. #[derive(Debug, PartialEq, Eq, thiserror::Error, miette::Diagnostic)] #[non_exhaustive] pub enum LeafBytesError { @@ -1227,6 +1426,21 @@ pub enum LeafBytesError { #[error("key tag of {0} bytes exceeds the format's u16 length field")] #[diagnostic(code(stack_encrypt::leaf_tag_too_long))] TagTooLong(usize), + /// The backend byte of a v2 leaf names no key vendor backend this build + /// knows. `0` is never valid. + #[error("unknown sealed-leaf key backend {0}")] + #[diagnostic( + code(stack_encrypt::leaf_backend), + help("The value was sealed by a newer build of stack-encrypt, or the bytes are not a sealed value.") + )] + UnknownBackend(u8), + /// The mode byte of a v2 leaf names no key mode. `0` is never valid. + #[error("unknown sealed-leaf key mode {0}")] + #[diagnostic( + code(stack_encrypt::leaf_mode), + help("The value was sealed by a newer build of stack-encrypt, or the bytes are not a sealed value.") + )] + UnknownMode(u8), } impl crate::ErrorPayload for LeafBytesError { @@ -1235,21 +1449,40 @@ impl crate::ErrorPayload for LeafBytesError { match self { Self::UnknownVersion(version) => payload([("version", (*version).into())]), Self::TagTooLong(len) => payload([("len", (*len).into())]), + Self::UnknownBackend(byte) => payload([("backend", (*byte).into())]), + Self::UnknownMode(byte) => payload([("mode", (*byte).into())]), Self::Truncated => serde_json::Map::new(), } } } impl SealedValue { - /// The version byte prefixing the frozen byte encoding - /// ([`to_bytes`](Self::to_bytes)). Also bound into every leaf's AAD (the + /// The version byte of the v1 encoding ([`to_bytes`](Self::to_bytes)), + /// the one ZeroKMS writes. Also bound into every leaf's AAD (the /// private `leaf_aad` derivation): bumping it re-keys authentication, so /// old leaves can never be relabelled as the new version (nor new as /// old). pub const FORMAT_VERSION: u8 = 1; - /// Encode into the frozen v1 byte layout — see the type-level docs for - /// the format. The inverse of [`from_bytes`](Self::from_bytes). + /// The version byte of the v2 encoding, the one a key vendor backend + /// writes. It is followed by the backend byte and the mode byte, and + /// all three are bound into the leaf AAD. + pub const VENDOR_FORMAT_VERSION: u8 = 2; + + /// The version byte this leaf encodes with: + /// [`FORMAT_VERSION`](Self::FORMAT_VERSION) for a ZeroKMS leaf, + /// [`VENDOR_FORMAT_VERSION`](Self::VENDOR_FORMAT_VERSION) for a leaf of a + /// key vendor backend. + pub fn format_version(&self) -> u8 { + match self.vendor { + None => Self::FORMAT_VERSION, + Some(_) => Self::VENDOR_FORMAT_VERSION, + } + } + + /// Encode into the frozen byte layout — v1 for a ZeroKMS leaf, v2 for a + /// leaf of a key vendor backend; see the type-level docs for the + /// format. The inverse of [`from_bytes`](Self::from_bytes). /// /// Infallible: every way of building a `SealedValue` rejects a tag too /// long for the `u16` length field ([`LeafBytesError::TagTooLong`]), so a @@ -1261,10 +1494,13 @@ impl SealedValue { // The saturating fallback is unreachable, and asserted so in tests. let tag_len = u16::try_from(self.tag.len()).unwrap_or(u16::MAX); debug_assert_eq!(usize::from(tag_len), self.tag.len()); + let header = LeafHeader::of(self.vendor); + let header = header.as_bytes(); let ciphertext = self.ciphertext.as_ref(); - let mut out = - Vec::with_capacity(1 + 16 + self.iv.len() + 2 + self.tag.len() + ciphertext.len()); - out.push(Self::FORMAT_VERSION); + let mut out = Vec::with_capacity( + header.len() + 16 + self.iv.len() + 2 + self.tag.len() + ciphertext.len(), + ); + out.extend_from_slice(header); out.extend_from_slice(self.keyset_id.as_bytes()); out.extend_from_slice(&self.iv); out.extend_from_slice(&tag_len.to_le_bytes()); @@ -1273,16 +1509,23 @@ impl SealedValue { out } - /// Decode the frozen v1 byte layout — the inverse of + /// Decode the frozen v1 or v2 byte layout — the inverse of /// [`to_bytes`](Self::to_bytes). Structural only: a decoded leaf is /// untrusted bytes until it decrypts. pub fn from_bytes(bytes: &[u8]) -> Result { const IV_LEN: usize = 16; let (&version, rest) = bytes.split_first().ok_or(LeafBytesError::Truncated)?; - if version != Self::FORMAT_VERSION { - return Err(LeafBytesError::UnknownVersion(version)); - } + let (vendor, rest) = match version { + Self::FORMAT_VERSION => (None, rest), + Self::VENDOR_FORMAT_VERSION => match rest { + [vendor, mode, rest @ ..] => { + (Some(VendorHeader::from_bytes(*vendor, *mode)?), rest) + } + _ => return Err(LeafBytesError::Truncated), + }, + _ => return Err(LeafBytesError::UnknownVersion(version)), + }; if rest.len() < 16 + IV_LEN + 2 { return Err(LeafBytesError::Truncated); } @@ -1302,6 +1545,7 @@ impl SealedValue { iv, tag: tag.to_vec(), ciphertext: LocalCipherText::from(ciphertext.to_vec()), + vendor, }) } @@ -1314,8 +1558,8 @@ impl SealedValue { Ok(()) } - /// Rebuild a leaf from its persisted parts — the inverse of - /// [`into_parts`](Self::into_parts). + /// Rebuild a v1 (ZeroKMS) leaf from its persisted parts — the inverse + /// of [`into_parts`](Self::into_parts). /// /// Fails with [`LeafBytesError::TagTooLong`] if `tag` does not fit the /// byte format's `u16` length field. Structural only: nothing about the @@ -1332,10 +1576,31 @@ impl SealedValue { iv, tag, ciphertext: LocalCipherText::from(ciphertext), + vendor: None, }) } + /// Rebuild a v2 leaf of the key vendor backend `vendor`, in key mode + /// `mode`, from its parts. As [`from_parts`](Self::from_parts) in every + /// other way. + pub fn from_vendor_parts( + vendor: KeyVendor, + mode: KeyMode, + keyset_id: Uuid, + iv: stack_kms::Iv, + tag: Vec, + ciphertext: Vec, + ) -> Result { + let mut leaf = Self::from_parts(keyset_id, iv, tag, ciphertext)?; + leaf.vendor = Some(VendorHeader { vendor, mode }); + Ok(leaf) + } + /// Decompose into `(keyset_id, iv, tag, ciphertext)` for persistence. + /// + /// The parts do not include the key backend or the key mode. A v2 leaf + /// rebuilt with [`from_parts`](Self::from_parts) is a v1 leaf, which + /// fails to open; persist a v2 leaf with [`to_bytes`](Self::to_bytes). pub fn into_parts(self) -> (Uuid, stack_kms::Iv, Vec, Vec) { ( self.keyset_id, @@ -1352,12 +1617,33 @@ impl SealedValue { self.keyset_id } - /// The ZeroKMS IV identifying this leaf's data key. + /// The key backend this leaf names: ZeroKMS for a v1 leaf, the key + /// vendor backend for a v2 leaf. Authenticated: a leaf relabelled with + /// another backend fails to open. A cipher refuses a leaf that names + /// another backend than its own before any key call + /// ([`Error::ForeignBackend`]). + pub fn key_backend(&self) -> KeyBackend { + match self.vendor { + None => KeyBackend::ZeroKms, + Some(header) => KeyBackend::Vendor(header.vendor), + } + } + + /// The key mode of a v2 leaf; `None` for a v1 (ZeroKMS) leaf, since + /// ZeroKMS has no key mode. Authenticated as + /// [`key_backend`](Self::key_backend) is. + pub fn key_mode(&self) -> Option { + self.vendor.map(|header| header.mode) + } + + /// The IV identifying this leaf's data key. Under derived data keys, it + /// is the derivation salt. pub fn iv(&self) -> &stack_kms::Iv { &self.iv } - /// The ZeroKMS key tag. + /// The key tag. Under wrapped data keys, it is the wrapped data key; + /// under derived data keys, it is empty. pub fn tag(&self) -> &[u8] { &self.tag } @@ -1375,6 +1661,7 @@ impl Clone for SealedValue { iv: self.iv, tag: self.tag.clone(), ciphertext: LocalCipherText::from(self.ciphertext.as_ref().to_vec()), + vendor: self.vendor, } } } @@ -1400,6 +1687,8 @@ struct SealedValueRepr { iv: stack_kms::Iv, tag: Vec, ciphertext: LocalCipherText, + #[serde(default)] + vendor: Option, } impl TryFrom for SealedValue { @@ -1411,6 +1700,7 @@ impl TryFrom for SealedValue { iv, tag, ciphertext, + vendor, } = repr; Self::tag_fits_length_field(&tag)?; Ok(Self { @@ -1418,6 +1708,7 @@ impl TryFrom for SealedValue { iv, tag, ciphertext, + vendor, }) } } @@ -1568,14 +1859,18 @@ impl PendingStackCipherText { } /// Seal under `keyset_id`, drawing one key per leaf from `keys` in - /// vitaminc's walk order. A leaf with no key left is refused. + /// vitaminc's walk order. `header` is the v2 header of a key vendor + /// backend's keyset, or `None` for a v1 (ZeroKMS) leaf. A leaf with no + /// key left is refused. pub(crate) fn seal_with( self, keyset_id: Uuid, + header: Option, mut keys: impl Iterator, ) -> Result { let minted = MintedKeys { keyset_id, + header, keys: RefCell::new(&mut keys), }; seal_all(&minted, self.0).map_err(|_| Unspecified) @@ -1624,11 +1919,24 @@ fn leaf_cipher(key: &DataKey) -> Result { /// dev-persisted data existed. A leaf sealed under an earlier form fails /// authentication in `open_leaf` with a plain AEAD error, indistinguishable /// from tampering; re-encrypt anything that matters. -fn leaf_aad(aad: &Context<'_>, keyset_id: Uuid, tag: &[u8]) -> Context<'static> { +/// +/// # The v2 header +/// +/// The second piece is the leaf's envelope header: the version byte alone +/// for a v1 leaf, so a v1 leaf's AAD is byte for byte what it always was, +/// and `version ‖ backend ‖ mode` for a v2 leaf. PAE frames each piece with +/// its length, so a v2 header can never encode as a v1 one, and a v2 leaf +/// relabelled with another backend or mode fails verification. +fn leaf_aad( + aad: &Context<'_>, + vendor: Option, + keyset_id: Uuid, + tag: &[u8], +) -> Context<'static> { const LEAF_AAD_DOMAIN: &[u8] = b"stack-encrypt/leaf"; Context::pae(&[ LEAF_AAD_DOMAIN, - &[SealedValue::FORMAT_VERSION], + LeafHeader::of(vendor).as_bytes(), keyset_id.as_bytes(), aad.as_bytes(), tag, @@ -1641,11 +1949,25 @@ fn leaf_aad(aad: &Context<'_>, keyset_id: Uuid, tag: &[u8]) -> Context<'static> /// the leaf's place, the keyset the key was minted under, and the key /// `tag` — `tag` is always bound, so the leaf is cryptographically tied to /// its ZeroKMS data key. +#[cfg(test)] fn seal_leaf( plaintext: Protected>, aad: &Context<'_>, keyset_id: Uuid, key: DataKeyWithTag, +) -> Result { + // A ZeroKMS leaf is v1. + seal_leaf_as(None, plaintext, aad, keyset_id, key) +} + +/// [`seal_leaf`] in the v1 format (`vendor` is `None`) or the v2 format of +/// a key vendor backend. +fn seal_leaf_as( + vendor: Option, + plaintext: Protected>, + aad: &Context<'_>, + keyset_id: Uuid, + key: DataKeyWithTag, ) -> Result { // The `DataKeySource` is caller-supplied, so the key tag is not trusted // to fit the frozen byte format's `u16` length field: an oversized tag @@ -1655,12 +1977,13 @@ fn seal_leaf( SealedValue::tag_fits_length_field(&key.tag).map_err(|_| Unspecified)?; let iv = key.key.iv; let cipher = leaf_cipher(&key.key)?; - let ciphertext = cipher.seal(plaintext, &leaf_aad(aad, keyset_id, &key.tag))?; + let ciphertext = cipher.seal(plaintext, &leaf_aad(aad, vendor, keyset_id, &key.tag))?; Ok(SealedValue { keyset_id, iv, tag: key.tag, ciphertext, + vendor, }) } @@ -1668,7 +1991,10 @@ fn seal_leaf( fn open_leaf(keyed: KeyedLeaf, aad: &Context<'_>) -> Result>, Unspecified> { let KeyedLeaf { leaf, key } = keyed; let cipher = leaf_cipher(&key)?; - cipher.open(leaf.ciphertext, &leaf_aad(aad, leaf.keyset_id, &leaf.tag)) + cipher.open( + leaf.ciphertext, + &leaf_aad(aad, leaf.vendor, leaf.keyset_id, &leaf.tag), + ) } // ============================================================================= @@ -1684,6 +2010,9 @@ fn open_leaf(keyed: KeyedLeaf, aad: &Context<'_>) -> Result>, /// synchronous. A leaf for which no key is left is refused. struct MintedKeys<'k, I> { keyset_id: Uuid, + /// The v2 header of a key vendor backend's keyset, or `None` for a v1 + /// (ZeroKMS) leaf. + header: Option, keys: RefCell<&'k mut I>, } @@ -1697,7 +2026,7 @@ impl> Cipher for MintedKeys<'_, I> { aad: &Context<'_>, ) -> Result { let key = self.keys.borrow_mut().next().ok_or(Unspecified)?; - seal_leaf(plaintext, aad, self.keyset_id, key) + seal_leaf_as(self.header, plaintext, aad, self.keyset_id, key) } /// Opening goes through [`BoundKeys`], whose leaves carry their @@ -1859,6 +2188,7 @@ mod tests { let mut keys = std::iter::empty::(); let minted = MintedKeys { keyset_id: Uuid::nil(), + header: None, keys: RefCell::new(&mut keys), }; let aad = Context::from_encoded(b"aad"); @@ -1873,6 +2203,43 @@ mod tests { ); } + /// "No such keyset" is an answer about the name from either kind of key + /// backend: ZeroKMS's keyset-not-found, or a key vendor backend's + /// `NotFound`. Every other failure is no answer. + #[test] + fn a_keyset_is_not_found_by_zerokms_or_by_a_key_vendor_backend() { + use stack_kms::{BackendErrorKind, LoadKeysetError}; + use zerokms_protocol::{ViturRequestError, ViturRequestErrorKind}; + let vitur = |kind| ViturRequestError::new(kind, "stubbed", std::io::Error::other("detail")); + let backend = |kind| { + Error::Kms(stack_kms::Error::Backend { + vendor: "test-vendor", + kind, + source: Box::new(std::io::Error::other("detail")), + }) + }; + assert!(is_keyset_not_found(&Error::Kms( + stack_kms::Error::LoadKeyset(LoadKeysetError::KeysetNotFound(vitur( + ViturRequestErrorKind::NotFound + ))) + ))); + assert!(is_keyset_not_found(&backend(BackendErrorKind::NotFound))); + for kind in [ + BackendErrorKind::AccessDenied, + BackendErrorKind::Throttled, + BackendErrorKind::Unavailable, + BackendErrorKind::InvalidCiphertext, + ] { + assert!(!is_keyset_not_found(&backend(kind)), "{kind:?}"); + } + assert!(!is_keyset_not_found(&Error::Kms( + stack_kms::Error::LoadKeyset(LoadKeysetError::Forbidden(vitur( + ViturRequestErrorKind::Forbidden + ))) + ))); + assert!(!is_keyset_not_found(&Error::Aead)); + } + /// Byte-level pin for the [`leaf_aad`] derivation. This is part of the /// frozen leaf format: a change to the domain label, the version byte, /// the keyset id's place, the piece order, or the PAE framing makes @@ -1882,7 +2249,12 @@ mod tests { #[test] fn leaf_aad_bytes_are_pinned() { let keyset = Uuid::from_bytes(*b"keyset-fixture16"); - let aad = leaf_aad(&Context::from_encoded(b"caller-aad"), keyset, b"key-tag"); + let aad = leaf_aad( + &Context::from_encoded(b"caller-aad"), + None, + keyset, + b"key-tag", + ); let hex: String = aad.as_bytes().iter().map(|b| format!("{b:02x}")).collect(); // PAE: LE64 count (5) ‖ per piece LE64 length ‖ piece, the pieces // being "stack-encrypt/leaf", [FORMAT_VERSION], the keyset id's 16 @@ -1892,4 +2264,182 @@ mod tests { "05000000000000001200000000000000737461636b2d656e63727970742f6c65616601000000000000000110000000000000006b65797365742d6669787475726531360a0000000000000063616c6c65722d61616407000000000000006b65792d746167" ); } + + /// The leaf-AAD pin for a v2 leaf: the header piece is + /// `version ‖ backend ‖ mode`, three bytes, where a v1 leaf's is one. + #[test] + fn v2_leaf_aad_bytes_are_pinned() { + let keyset = Uuid::from_bytes(*b"keyset-fixture16"); + let header = VendorHeader { + vendor: KeyVendor::Aws, + mode: KeyMode::Wrapped, + }; + let aad = leaf_aad( + &Context::from_encoded(b"caller-aad"), + Some(header), + keyset, + b"key-tag", + ); + let hex: String = aad.as_bytes().iter().map(|b| format!("{b:02x}")).collect(); + assert_eq!( + hex, + "05000000000000001200000000000000737461636b2d656e63727970742f6c656166030000000000000002010110000000000000006b65797365742d6669787475726531360a0000000000000063616c6c65722d61616407000000000000006b65792d746167" + ); + } + + const V2_HEADER: VendorHeader = VendorHeader { + vendor: KeyVendor::Static, + mode: KeyMode::Derived, + }; + + fn test_key() -> DataKeyWithTag { + DataKeyWithTag { + key: DataKey { + iv: *b"v2-leaf-iv-salt!", + key: [0x42; 32], + }, + tag: Vec::new(), + decryption_policy: None, + } + } + + fn open_with_test_key(leaf: SealedValue, aad: &Context<'_>) -> Result, Unspecified> { + use vitaminc_protected::Controlled; + let key = test_key().key; + open_leaf(KeyedLeaf { leaf, key }, aad).map(|plaintext| plaintext.risky_unwrap()) + } + + fn sealed_v2_bytes(aad: &Context<'_>) -> Vec { + let keyset = Uuid::from_bytes(*b"keyset-fixture16"); + let leaf = seal_leaf_as( + Some(V2_HEADER), + Protected::new(b"v2 plaintext".to_vec()), + aad, + keyset, + test_key(), + ) + .expect("seal a v2 leaf"); + assert_eq!(leaf.format_version(), SealedValue::VENDOR_FORMAT_VERSION); + assert_eq!( + leaf.key_backend(), + KeyBackend::Vendor(KeyVendor::Static), + "the leaf names its key vendor backend" + ); + assert_eq!(leaf.key_mode(), Some(KeyMode::Derived)); + leaf.to_bytes() + } + + /// A v2 leaf seals, survives its byte encoding, and opens under the + /// same key and AAD. + #[test] + fn a_v2_leaf_round_trips_through_its_bytes() { + let aad = Context::from_encoded(b"ctx"); + let bytes = sealed_v2_bytes(&aad); + assert_eq!( + &bytes[..3], + &[2, KeyVendor::Static.byte(), KeyMode::Derived.byte()] + ); + let leaf = SealedValue::from_bytes(&bytes).expect("decode v2 leaf"); + assert_eq!(leaf.to_bytes(), bytes, "the v2 encoding is lossless"); + assert_eq!( + open_with_test_key(leaf, &aad).expect("open v2 leaf"), + b"v2 plaintext" + ); + } + + /// The leaf AAD binds each v2 header byte: every other value of the + /// version, the backend or the mode byte is refused, by the decoder or + /// by the open. None of them opens. + #[test] + fn changing_any_v2_header_byte_makes_the_open_fail() { + let aad = Context::from_encoded(b"ctx"); + let bytes = sealed_v2_bytes(&aad); + let mut opened_by_decoder = 0; + for position in 0..3 { + for value in 0..=u8::MAX { + if value == bytes[position] { + continue; + } + let mut tampered = bytes.clone(); + tampered[position] = value; + match SealedValue::from_bytes(&tampered) { + Err(_) => {} + Ok(leaf) => { + opened_by_decoder += 1; + assert!( + open_with_test_key(leaf, &aad).is_err(), + "byte {position} changed to {value} must not open" + ); + } + } + } + } + // Every other valid backend and mode decodes, so the AEAD is what + // refuses them; the count proves the AEAD was reached. (Version 1 + // reads the following bytes at v1 offsets, and this fixture's + // `tag_len` then overruns the buffer, so the decoder refuses it. + // `a_v2_leaf_relabelled_as_v1_does_not_open` covers the AEAD case.) + assert_eq!( + opened_by_decoder, + (KeyVendor::ALL.len() - 1) + (KeyMode::ALL.len() - 1) + ); + } + + /// The same header refusal holds when a v2 leaf is relabelled as v1 + /// without a shift in the following bytes: a v1 leaf built from the v2 + /// leaf's parts does not open. + #[test] + fn a_v2_leaf_relabelled_as_v1_does_not_open() { + let aad = Context::from_encoded(b"ctx"); + let leaf = SealedValue::from_bytes(&sealed_v2_bytes(&aad)).expect("decode v2 leaf"); + let (keyset, iv, tag, ciphertext) = leaf.into_parts(); + let relabelled = SealedValue::from_parts(keyset, iv, tag, ciphertext).expect("v1 parts"); + assert_eq!(relabelled.key_backend(), KeyBackend::ZeroKms); + assert!(open_with_test_key(relabelled, &aad).is_err()); + } + + /// A v1 leaf opens as it always did: the v1 AAD header is the version + /// byte alone. + #[test] + fn a_v1_leaf_still_opens_under_the_v1_aad() { + let aad = Context::from_encoded(b"ctx"); + let keyset = Uuid::from_bytes(*b"keyset-fixture16"); + let leaf = seal_leaf( + Protected::new(b"v1 plaintext".to_vec()), + &aad, + keyset, + test_key(), + ) + .expect("seal a v1 leaf"); + let bytes = leaf.to_bytes(); + assert_eq!(bytes[0], SealedValue::FORMAT_VERSION); + assert_eq!(leaf.key_mode(), None); + let leaf = SealedValue::from_bytes(&bytes).expect("decode v1 leaf"); + assert_eq!( + open_with_test_key(leaf, &aad).expect("open v1 leaf"), + b"v1 plaintext" + ); + } + + /// The serde form keeps the backend and the mode of a v2 leaf, and a v1 + /// leaf's serde form has no field for them. + #[test] + fn serde_keeps_the_v2_header_and_leaves_v1_unchanged() { + let aad = Context::from_encoded(b"ctx"); + let v2 = SealedValue::from_bytes(&sealed_v2_bytes(&aad)).expect("decode v2 leaf"); + let json = serde_json::to_value(&v2).expect("serialise v2"); + assert_eq!(json["vendor"], serde_json::json!([5, 2])); + let back: SealedValue = serde_json::from_value(json).expect("deserialise v2"); + assert_eq!(back.to_bytes(), v2.to_bytes()); + + let v1 = SealedValue::from_parts(Uuid::nil(), [0; 16], vec![1], vec![2]).expect("v1"); + let json = serde_json::to_value(&v1).expect("serialise v1"); + assert!(json.get("vendor").is_none(), "{json}"); + let back: SealedValue = serde_json::from_value(json).expect("deserialise v1"); + assert_eq!(back.to_bytes(), v1.to_bytes()); + + let mut bad = serde_json::to_value(&v2).expect("serialise v2"); + bad["vendor"] = serde_json::json!([0, 2]); + assert!(serde_json::from_value::(bad).is_err()); + } } diff --git a/packages/stack-encrypt/src/codes.rs b/packages/stack-encrypt/src/codes.rs index f2af81fd84..0b6e21386d 100644 --- a/packages/stack-encrypt/src/codes.rs +++ b/packages/stack-encrypt/src/codes.rs @@ -82,9 +82,17 @@ mod tests { expected: a, found: b, }, + Error::ForeignBackend { .. } => Error::ForeignBackend { + expected: crate::kms::KeyBackend::ZeroKms, + found: crate::kms::KeyBackend::Vendor(crate::kms::KeyVendor::Aws), + }, + Error::SealUnsupported { .. } => Error::SealUnsupported { + backend: crate::kms::KeyBackend::Vendor(crate::kms::KeyVendor::Static), + }, Error::NoKeyset => Error::NoKeyset, Error::NotOpened => Error::NotOpened, Error::Plan(_) => Error::Plan(PlanError::NoContext), + Error::DefaultKeysetLifetime { .. } => Error::DefaultKeysetLifetime { keyset: a }, Error::Blob(_) => Error::Blob(BlobError::NotABlob), ])); errors.extend(boxed(variants![ @@ -98,6 +106,8 @@ mod tests { LeafBytesError::UnknownVersion(_) => LeafBytesError::UnknownVersion(9), LeafBytesError::Truncated => LeafBytesError::Truncated, LeafBytesError::TagTooLong(_) => LeafBytesError::TagTooLong(70_000), + LeafBytesError::UnknownBackend(_) => LeafBytesError::UnknownBackend(0), + LeafBytesError::UnknownMode(_) => LeafBytesError::UnknownMode(0), ])); errors.extend(boxed(variants![ TermError::Prf(_) => TermError::Prf(cause()), @@ -369,6 +379,40 @@ mod tests { assert_eq!(error.to_string(), "Unexpected error: kms"); } + /// A key vendor backend's failure shows through the encryption error as + /// itself: its code, and its vendor and kind as fields. Its refusal of a + /// ZeroKMS-only feature does the same. + #[test] + fn a_key_backend_error_shows_through() { + use crate::ErrorPayload; + use serde_json::json; + let failure = Error::Kms(crate::kms::Error::Backend { + vendor: "test-vendor", + kind: crate::kms::BackendErrorKind::AccessDenied, + source: Box::new(std::io::Error::other("vendor detail")), + }); + assert_eq!( + failure.code().map(|code| code.to_string()).as_deref(), + Some("stack_kms::backend") + ); + assert_eq!( + serde_json::Value::Object(failure.payload()), + json!({ "vendor": "test-vendor", "kind": "AccessDenied" }) + ); + let refusal = Error::Kms(crate::kms::Error::ZeroKmsOnly { + vendor: "test-vendor", + feature: crate::kms::ZeroKmsFeature::UnverifiedContext, + }); + assert_eq!( + refusal.code().map(|code| code.to_string()).as_deref(), + Some("stack_kms::zerokms_only") + ); + assert_eq!( + serde_json::Value::Object(refusal.payload()), + json!({ "vendor": "test-vendor", "feature": "unverified_context" }) + ); + } + /// Every error type's structured fields, one variant per arm. A binding /// hands these over as they are, so a field that goes missing or changes /// name is a break for every caller that reads it. @@ -426,6 +470,10 @@ mod tests { Box::new(Error::KeysetMismatch { left: a, right: b }), json!({ "left": a.to_string(), "right": b.to_string() }), ), + ( + Box::new(Error::DefaultKeysetLifetime { keyset: a }), + json!({ "keyset": a.to_string() }), + ), (Box::new(Error::Aead), json!({})), ( Box::new(Error::OpenedType { expected: "i32" }), @@ -450,6 +498,27 @@ mod tests { json!({ "len": 70_000 }), ), (Box::new(LeafBytesError::Truncated), json!({})), + ( + Box::new(LeafBytesError::UnknownBackend(9)), + json!({ "backend": 9 }), + ), + ( + Box::new(LeafBytesError::UnknownMode(7)), + json!({ "mode": 7 }), + ), + ( + Box::new(Error::ForeignBackend { + expected: crate::kms::KeyBackend::ZeroKms, + found: crate::kms::KeyBackend::Vendor(crate::kms::KeyVendor::Vault), + }), + json!({ "expected": "zerokms", "found": "vault-transit" }), + ), + ( + Box::new(Error::SealUnsupported { + backend: crate::kms::KeyBackend::Vendor(crate::kms::KeyVendor::Gcp), + }), + json!({ "backend": "gcp-kms" }), + ), ( Box::new(TermError::Bytes(TermBytesError::OddMatchTermsLength(3))), json!({ "len": 3 }), diff --git a/packages/stack-encrypt/src/descriptor.rs b/packages/stack-encrypt/src/descriptor.rs index bf61d81abc..2aed78b936 100644 --- a/packages/stack-encrypt/src/descriptor.rs +++ b/packages/stack-encrypt/src/descriptor.rs @@ -84,16 +84,18 @@ impl Descriptor { /// The separator between the parts of a list. pub const SEPARATOR: char = '/'; - /// The longest descriptor ZeroKMS accepts, in bytes of the rendered - /// string: the protocol's [`MAX_DESCRIPTOR_LEN`](crate::kms::MAX_DESCRIPTOR_LEN). - /// ZeroKMS derives key material over a fixed block of that size holding - /// the descriptor, so a longer one cannot be bound. Every data-key + /// The longest descriptor a key backend accepts, in bytes of the + /// rendered string. The limit is the same for every key backend, so + /// that a context which fits one fits all. It is ZeroKMS's + /// [`MAX_DESCRIPTOR_LEN`](crate::kms::MAX_DESCRIPTOR_LEN): ZeroKMS + /// derives key material over a fixed block of that size holding the + /// descriptor, so a longer one cannot be bound. Every data-key /// request checks its descriptors against this before anything is sent /// ([`Error::DescriptorTooLong`](crate::Error::DescriptorTooLong)); a /// context is free to be long, but what it renders to must fit — and /// the base64 escape grows a part by a third, so an escaped part fits /// less than a plain one. - pub const MAX_LEN: usize = stack_kms::MAX_DESCRIPTOR_LEN; + pub const MAX_LEN: usize = 512; /// Render `context` — any [`IntoContext`] type — from its parts. /// @@ -924,6 +926,16 @@ mod tests { assert!(!escaped.fits()); } + /// The limit is stated here, not taken from ZeroKMS, so that a key + /// vendor backend does not reach a ZeroKMS constant. It must stay equal + /// to ZeroKMS's, or a context that fits one key backend would not fit + /// another. + #[test] + fn the_limit_is_zerokms_limit_for_every_key_backend() { + assert_eq!(Descriptor::MAX_LEN, 512); + assert_eq!(Descriptor::MAX_LEN, crate::kms::MAX_DESCRIPTOR_LEN); + } + #[test] fn a_clone_shares_the_rendering() { let descriptor = Descriptor::of("a".repeat(Descriptor::MAX_LEN)); diff --git a/packages/stack-encrypt/src/dynamic/json_query.rs b/packages/stack-encrypt/src/dynamic/json_query.rs index 09ffaa1f74..181256bb74 100644 --- a/packages/stack-encrypt/src/dynamic/json_query.rs +++ b/packages/stack-encrypt/src/dynamic/json_query.rs @@ -762,26 +762,24 @@ pub(super) fn derive_input<'a, K: 'static>( }) }) } - Input::Path(path) if op == OpSpec::Selector => { - let selector = keyset.json_selector_now(&path, column())?; - Pending::ready(keyset, Ok(Operand::Text(selector.to_string()))) - } + Input::Path(path) if op == OpSpec::Selector => keyset + .json_selector_planned(&path, column())? + .map(|selector| Operand::Text(selector.to_string())), Input::Path(path) => { // A path with no wildcard names one node: its key holds the // one token, and nothing that covers a node above it. - let key = keyset.json_subtree_key_now( - &path, - options, - JsonSubtreeKey::DEFAULT_BOUND, - column(), - )?; - let [token]: [String; 1] = key.tokens().try_into().map_err(|_| Error::Internal)?; - Pending::ready(keyset, Ok(Operand::Text(token))) - } - Input::Read(path, bound) => { - let key = keyset.json_subtree_key_now(&path, options, bound, column())?; - Pending::ready(keyset, Ok(Operand::TextList(key.tokens()))) + keyset + .json_subtree_key_planned(&path, options, JsonSubtreeKey::DEFAULT_BOUND, column())? + .try_map(|key| { + let [token]: [String; 1] = key.tokens().try_into().map_err(|_| { + crate::Error::Other("a path with no wildcard reads one token".into()) + })?; + Ok(Operand::Text(token)) + }) } + Input::Read(path, bound) => keyset + .json_subtree_key_planned(&path, options, bound, column())? + .map(|key| Operand::TextList(key.tokens())), Input::Elements(bound, predicates) => elements(keyset, label, options, bound, &predicates)?, Input::Predicate(node) => super::json_predicate::derive(keyset, label, options, *node)?, }) @@ -857,13 +855,13 @@ fn elements<'a, K: 'static>( let mut members = Vec::with_capacity(predicates.len()); for predicate in predicates { let tokens = keyset - .json_subtree_key_now( + .json_subtree_key_planned( &predicate.path, options, bound, NonEmpty::from(label.clone()), )? - .tokens(); + .map(|key| key.tokens()); let leaf = |value: &Parsed| { terms( keyset, @@ -878,8 +876,9 @@ fn elements<'a, K: 'static>( Bounds::Between(low, high) => (None, vec![leaf(low), leaf(high)]), }; members.push( - Pending::all(keyset, leaves) - .map(move |leaves| MemberOperand::new(tokens, compare, leaves)), + tokens + .zip(Pending::all(keyset, leaves)) + .map(move |(tokens, leaves)| MemberOperand::new(tokens, compare, leaves)), ); } Ok(Pending::all(keyset, members).try_map(|m| eql(&ElementsOperand { m }).map(Operand::One))) @@ -993,13 +992,31 @@ pub(super) fn open<'a, K: 'static>( ) -> Result, Error> { let (read, groups) = read_open(request, rows)?; let column = || NonEmpty::from(label.clone()); + let context = || AeadContext::from(NonEmpty::from(label.clone())); + // A keyset whose index key lasts one settle holds no PRF to derive the + // read's selector or key with before the settle: the read opens in one + // settle that loads the index key and derives them inside it. + let Some(prf) = keyset.prf() else { + let by = match read { + Read::Selector(path) => crate::json::ReadBy::Selector(path), + Read::Subtree(path, bound) => crate::json::ReadBy::Subtree(path, bound), + }; + return Ok(crate::json::open_read_in_settle( + keyset, + by, + options, + column(), + context, + groups, + )? + .try_map(opened_frame)); + }; let opener = match read { - Read::Selector(path) => Opener::Selector(keyset.json_selector_now(&path, column())?), + Read::Selector(path) => Opener::Selector(keyset.json_selector_now(prf, &path, column())?), Read::Subtree(path, bound) => { - Opener::Subtree(keyset.json_subtree_key_now(&path, options, bound, column())?) + Opener::Subtree(keyset.json_subtree_key_now(prf, &path, options, bound, column())?) } }; - let context = || AeadContext::from(NonEmpty::from(label.clone())); // Opened through the client, confined to the deriving keyset, as a // record's leaves are under `Scope::Keyset`: the read's key is that // keyset's, so a document sealed under another cannot answer it. diff --git a/packages/stack-encrypt/src/json/derive.rs b/packages/stack-encrypt/src/json/derive.rs index 73bf163f0f..ddf8767f07 100644 --- a/packages/stack-encrypt/src/json/derive.rs +++ b/packages/stack-encrypt/src/json/derive.rs @@ -32,7 +32,7 @@ use super::query::{group_of, JsonSelector, JsonSelectorSet, JsonSubtreeKey, Read use super::shuffle; use super::{JsonDocument, JsonEntry, JsonExtracted, JsonHeader, JsonOptions, JsonSubtree, Tag}; use crate::keyset::KeysetState; -use crate::sem::TermError; +use crate::sem::{with_prf, TermError}; use crate::target::{AeadContext, CallerContext, Decryption, Pending, Request}; use crate::{DecryptInto, Descriptor, Error, KeysetCipher, NonEmpty, StackCipher}; @@ -322,20 +322,61 @@ fn prf16_of( /// [`JsonSubtreeKey`] holds one, so that opening a walk's rows can derive /// the selector each row must carry from its parent's path and its own /// decrypted label. Never printed; compared by keyset and context. +/// +/// A keyset whose index key lasts one settle holds no PRF +/// ([`KeysetState::prf`] is `None`). A fulfilment that opens rows under such +/// a keyset takes the PRF its settle loaded ([`in_settle`](Self::in_settle)), +/// and the selectors it builds that way drop with the fulfilment, so the +/// index key does not outlive the settle. A [`JsonSubtreeKey`] never holds +/// that PRF. #[derive(Clone)] pub(crate) struct Selectors { keyset: Arc, context: Context<'static>, + /// The PRF the settle loaded for a keyset that holds none. Set only + /// inside a fulfilment ([`in_settle`](Self::in_settle)). + settle: Option>, } impl Selectors { pub(crate) fn new(keyset: Arc, context: Context<'static>) -> Self { - Self { keyset, context } + Self { + keyset, + context, + settle: None, + } + } + + /// The keyset whose index key a settle must load before these selectors + /// derive anything: the keyset, when it holds no PRF of its own. + pub(crate) fn settle_keyset(&self) -> Option { + self.keyset.prf.is_none().then_some(self.keyset.id) + } + + /// These selectors over the PRF the settle loaded, for a keyset that + /// holds none. The pending that runs the fulfilment must have asked for + /// that index key ([`Request::index_key`] or [`Request::load_keyset`]), + /// or this is [`Error::ResponseShape`]. + pub(crate) fn in_settle(mut self, responses: &crate::target::Responses) -> Result { + if self.keyset.prf.is_none() { + self.settle = Some(responses.index_prf_shared(self.keyset.id)?); + } + Ok(self) + } + + /// The PRF the selectors derive under: the keyset's, or the one the + /// settle loaded ([`in_settle`](Self::in_settle)). + fn prf(&self) -> Result<&HmacSha256Prf, Error> { + self.keyset + .prf + .as_ref() + .or(self.settle.as_deref()) + .ok_or(Error::ResponseShape) } /// The selector of the path whose selector bytes are `bytes`. fn of(&self, bytes: Protected>) -> Result { - prf16_of(&self.keyset.prf, SELECTOR_DOMAIN, &self.context, bytes).map_err(Error::from) + prf16_of(self.prf()?, SELECTOR_DOMAIN, &self.context, bytes).map_err(Error::from) } } @@ -349,7 +390,8 @@ impl Selectors { Arc::new(KeysetState { id: Uuid::from_u128(1), name: None, - prf: HmacSha256Prf::new(Protected::new([7; 32])), + prf: Some(HmacSha256Prf::new(Protected::new([7; 32]))), + retention: crate::keyset::Retention::UntilEvicted, }), Context::from_encoded(b"users/profile").into_owned(), ) @@ -756,20 +798,39 @@ pub(crate) fn terms_pending<'a, K>( context: NonEmpty, ) -> Pending<'a, JsonTerms, K> { let context = context.into_prf_context().into_owned(); - let terms = cipher - .cipher() - .json_randomness() - .generator() - .and_then(|mut rng| { - let walked = walk(value, options, cipher.prf(), &context, &mut rng)?; - let mut selectors = group_of(&walked.terms.nodes, None); - selectors.shuffle(&mut rng); - Ok(JsonTerms { - query_selectors: Some(selectors), - ..walked.terms + let query_terms = |value: &Value, + options: &JsonOptions, + prf: &HmacSha256Prf, + context: &Context<'_>, + mut rng: SafeRand| + -> Result { + let walked = walk(value, options, prf, context, &mut rng)?; + let mut selectors = group_of(&walked.terms.nodes, None); + selectors.shuffle(&mut rng); + Ok(JsonTerms { + query_selectors: Some(selectors), + ..walked.terms + }) + }; + let rng = match cipher.cipher().json_randomness().generator() { + Ok(rng) => rng, + Err(error) => return Pending::failed(cipher, error), + }; + match cipher.prf() { + Some(prf) => Pending::ready(cipher, query_terms(value, options, prf, &context, rng)), + // The keyset's index key lasts one settle: the terms are derived + // inside it, over a copy of the query value, wiped when it drops. + None => { + if let Err(error) = options.check() { + return Pending::failed(cipher, error.into()); + } + let value = PlainJson::protect(value.clone()); + let options = options.clone(); + with_prf(cipher, move |prf| { + query_terms(value.risky_ref().value(), &options, prf, &context, rng) }) - }); - Pending::ready(cipher, terms) + } + } } /// Seal `value` into a [`JsonDocument`] under `context`: one data key for @@ -798,16 +859,6 @@ pub(crate) fn seal_pending<'a, K: 'static>( Ok(rng) => rng, Err(error) => return Pending::failed(cipher, error), }; - let walked = match walk( - value.risky_ref().value(), - options, - cipher.prf(), - &prf_context, - &mut rng, - ) { - Ok(walked) => walked, - Err(error) => return Pending::failed(cipher, error.into()), - }; let whole = if options.sealed_root { match sealed_root_plaintext(value.risky_ref().value()) { Ok(bytes) => Some(bytes), @@ -818,6 +869,44 @@ pub(crate) fn seal_pending<'a, K: 'static>( }; let keyset_id = cipher.keyset_id(); let sealed_root = options.sealed_root; + let Some(prf) = cipher.prf() else { + // The keyset's index key lasts one settle: the document is walked + // inside it, under the index key the settle loads beside the data + // key. The options are checked now, as the walk below checks them. + if let Err(error) = options.check() { + return Pending::failed(cipher, error.into()); + } + let options = options.clone(); + return Pending::request( + cipher, + vec![ + Request::generate_under(descriptor), + Request::index_key(keyset_id), + ], + move |responses| { + let key = responses.next_generated_key()?; + let prf = responses.index_prf(keyset_id)?; + let walked = walk( + value.risky_ref().value(), + &options, + prf, + &prf_context, + &mut rng, + )?; + seal(walked, whole, keyset_id, key, &aad, sealed_root, rng) + }, + ); + }; + let walked = match walk( + value.risky_ref().value(), + options, + prf, + &prf_context, + &mut rng, + ) { + Ok(walked) => walked, + Err(error) => return Pending::failed(cipher, error.into()), + }; Pending::request( cipher, vec![Request::generate_under(descriptor)], @@ -1224,7 +1313,8 @@ fn open_document<'a, K>( let aes = leaf_cipher(&key)?; if open_root { let keyset = responses.next_keyset()?; - open_whole_chain(doc, &aes, &aad, Selectors::new(keyset, prf_context)) + let selectors = Selectors::new(keyset, prf_context).in_settle(responses)?; + open_whole_chain(doc, &aes, &aad, selectors) } else { open_sealed_root(&doc, &aes, &aad) } @@ -1258,7 +1348,7 @@ fn open_whole_chain( ) -> Result { let root = JsonPath::root(); let token = prf16( - &selectors.keyset.prf, + selectors.prf()?, TOKEN_DOMAIN, &selectors.context, &root.token_bytes().unwrap_or_default(), @@ -1359,30 +1449,40 @@ impl DecryptInto for JsonExtracted { Err(error) => return Pending::failed(cipher, error), }; Pending::request(cipher, vec![request], move |responses| { - let key = responses.next_retrieved_key()?; - let aes = leaf_cipher(&key)?; - let opened = open_entry( - &aes, - &node.header, - &aad, - &node.entry, - node.slot, - &cascade::no_children(), - ) - .map_err(|error| match error { - Error::Aead => Error::JsonNotALeaf, - other => other, - })?; - opened - .whole - .or(opened.scalar) - .map(hand_over) - .ok_or_else(|| shape("a JSON scalar, or a sealed root")) + open_extracted(&node, &aad, responses) }) }) } } +/// Open the entry `->` returned, its selector already checked, under the +/// data key its request retrieved: the fulfilment of a [`JsonExtracted`]. +fn open_extracted( + node: &super::JsonNode, + aad: &Context<'_>, + responses: &mut crate::target::Responses, +) -> Result { + let key = responses.next_retrieved_key()?; + let aes = leaf_cipher(&key)?; + let opened = open_entry( + &aes, + &node.header, + aad, + &node.entry, + node.slot, + &cascade::no_children(), + ) + .map_err(|error| match error { + Error::Aead => Error::JsonNotALeaf, + other => other, + })?; + opened + .whole + .or(opened.scalar) + .map(hand_over) + .ok_or_else(|| shape("a JSON scalar, or a sealed root")) +} + /// Open the rows of a read into the values of the nodes its tokens named, /// in the key's order, each still under its `Protected`. Shared by the /// three openings below, so they agree on every row and on the fallback: @@ -1394,11 +1494,145 @@ fn open_reads<'a, K>( cipher: &'a StackCipher, context: AeadContext, ) -> Pending<'a, Vec>, K> { - if subtree.is_sealed_root() { - open_fallback(subtree, cipher, context) - .try_map(|(key, mut whole)| select_reads(&key, &mut whole)) - } else { - open_subtree(subtree, cipher, context) + let sealed_root = subtree.is_sealed_root(); + let (nodes, key) = subtree.into_parts(); + let planned = match plan_rows(nodes, sealed_root, context) { + Ok(planned) => planned, + Err(error) => return Pending::failed(cipher, error), + }; + // A key derived under a keyset whose index key lasts one settle holds + // no PRF: the settle loads that index key beside the data key, and the + // rows' selectors are derived under it. The sealed root needs none. + let mut requests: Vec = planned.request().into_iter().collect(); + if let (PlannedRows::Walk { .. }, Some(keyset)) = (&planned, key.selectors.settle_keyset()) { + requests.push(Request::index_key(keyset)); + } + Pending::request(cipher, requests, move |responses| { + let selectors = match &planned { + PlannedRows::Walk { .. } => key.selectors.clone().in_settle(responses)?, + _ => key.selectors.clone(), + }; + open_rows(planned, &key, &selectors, responses) + }) +} + +/// A read's rows, checked and ready to open: what [`plan_rows`] decides +/// before any key is retrieved. +enum PlannedRows { + /// No rows: the read reached nothing, and nothing is retrieved. + Empty, + /// The sealed root a read fell back to, opened whole. + Fallback { + root: super::JsonNode, + request: Request, + aad: Context<'static>, + }, + /// The rows of one document, in slot order, each placed by the chain. + Walk { + nodes: Vec, + header: JsonHeader, + request: Request, + aad: Context<'static>, + }, +} + +impl PlannedRows { + /// The data-key request the rows need, if any. + fn request(&self) -> Option { + match self { + Self::Empty => None, + Self::Fallback { request, .. } | Self::Walk { request, .. } => Some(request.clone()), + } + } +} + +/// Check a read's rows and plan their key retrieval: everything opening +/// them refuses before any key is retrieved. +fn plan_rows( + mut nodes: Vec, + sealed_root: bool, + context: AeadContext, +) -> Result { + if sealed_root { + let Some(root) = nodes.into_iter().next() else { + return Err(shape("the sealed root")); + }; + let (request, aad) = key_retrieval(&root.header, context.into_context())?; + return Ok(PlannedRows::Fallback { root, request, aad }); + } + let Some(first) = nodes.first() else { + return Ok(PlannedRows::Empty); + }; + let header = first.header.clone(); + // A read is of one row, so its rows carry one header. Rows of two + // documents are not the answer to it. Checked first: two documents' + // entries can share a slot. + if nodes.iter().any(|n| n.header != header) { + return Err(wrong("rows of one document")); + } + // A read returns each entry once. Two rows at one slot are the server + // repeating an entry, which would let it line up one container as many + // times as it likes: an honest database never returns one, so it is the + // wrong entry, not a shape the target refuses. + nodes.sort_by_key(|n| n.slot); + if nodes.windows(2).any(|pair| pair[0].slot == pair[1].slot) { + return Err(wrong("rows of distinct entries")); + } + let (request, aad) = key_retrieval(&header, context.into_context())?; + Ok(PlannedRows::Walk { + nodes, + header, + request, + aad, + }) +} + +/// Open planned rows into the values of the nodes `key`'s tokens named, in +/// the key's order, under the data key their request retrieved and, for +/// walked rows, `selectors` (the key's own, over the settle's PRF where the +/// keyset holds none). +fn open_rows( + planned: PlannedRows, + key: &JsonSubtreeKey, + selectors: &Selectors, + responses: &mut crate::target::Responses, +) -> Result>, Error> { + match planned { + PlannedRows::Empty => Ok(Vec::new()), + PlannedRows::Fallback { root, aad, .. } => { + let retrieved = responses.next_retrieved_key()?; + let aes = leaf_cipher(&retrieved)?; + let opened = open_entry( + &aes, + &root.header, + &aad, + &root.entry, + root.slot, + &cascade::no_children(), + )?; + let mut whole = opened.whole.ok_or_else(|| shape("a sealed root"))?; + select_reads(key, &mut whole) + } + PlannedRows::Walk { + nodes, header, aad, .. + } => { + let key_bytes = responses.next_retrieved_key()?; + let aes = leaf_cipher(&key_bytes)?; + let rows = nodes + .into_iter() + .map(|n| ReturnedRow { + entry: n.entry, + slot: n.slot, + depth: n.depth, + }) + .collect(); + let opening = Opening { + header: &header, + aad: &aad, + aes: &aes, + }; + place(rows, &key.reads, selectors, key.bound, &opening) + } } } @@ -1477,56 +1711,6 @@ struct ReturnedRow { depth: Option, } -/// Open every row and place each node a token named, checking every row -/// against the key as it goes ([`place`]). -fn open_subtree<'a, K>( - subtree: JsonSubtree, - cipher: &'a StackCipher, - context: AeadContext, -) -> Pending<'a, Vec>, K> { - let (mut nodes, key) = subtree.into_parts(); - let Some(first) = nodes.first() else { - return Pending::ready(cipher, Ok(Vec::new())); - }; - let header = first.header.clone(); - // A read is of one row, so its rows carry one header. Rows of two - // documents are not the answer to it. Checked first: two documents' - // entries can share a slot. - if nodes.iter().any(|n| n.header != header) { - return Pending::failed(cipher, wrong("rows of one document")); - } - // A read returns each entry once. Two rows at one slot are the server - // repeating an entry, which would let it line up one container as many - // times as it likes: an honest database never returns one, so it is the - // wrong entry, not a shape the target refuses. - nodes.sort_by_key(|n| n.slot); - if nodes.windows(2).any(|pair| pair[0].slot == pair[1].slot) { - return Pending::failed(cipher, wrong("rows of distinct entries")); - } - let (request, aad) = match key_retrieval(&header, context.into_context()) { - Ok(parts) => parts, - Err(error) => return Pending::failed(cipher, error), - }; - Pending::request(cipher, vec![request], move |responses| { - let key_bytes = responses.next_retrieved_key()?; - let aes = leaf_cipher(&key_bytes)?; - let rows = nodes - .into_iter() - .map(|n| ReturnedRow { - entry: n.entry, - slot: n.slot, - depth: n.depth, - }) - .collect(); - let opening = Opening { - header: &header, - aad: &aad, - aes: &aes, - }; - place(rows, &key.reads, &key.selectors, key.bound, &opening) - }) -} - /// The cipher side of opening a document's rows: its header, the AAD its /// entries open under, and the cipher of its data key. struct Opening<'o> { @@ -1876,36 +2060,115 @@ fn select_reads( .collect()) } -/// Open the sealed root a read fell back to: the whole document, under its -/// `Protected`, with the key that keyed the read, whose reads the caller -/// evaluates over it. -fn open_fallback<'a, K>( - subtree: JsonSubtree, - cipher: &'a StackCipher, - context: AeadContext, -) -> Pending<'a, (JsonSubtreeKey, Protected), K> { - let (nodes, key) = subtree.into_parts(); - let Some(root) = nodes.into_iter().next() else { - return Pending::failed(cipher, shape("the sealed root")); - }; - let (request, aad) = match key_retrieval(&root.header, context.into_context()) { - Ok(parts) => parts, - Err(error) => return Pending::failed(cipher, error), - }; - Pending::request(cipher, vec![request], move |responses| { - let retrieved = responses.next_retrieved_key()?; - let aes = leaf_cipher(&retrieved)?; - let opened = open_entry( - &aes, - &root.header, - &aad, - &root.entry, - root.slot, - &cascade::no_children(), - )?; - let whole = opened.whole.ok_or_else(|| shape("a sealed root"))?; - Ok((key, whole)) - }) +/// What a read of a JSON column opens its rows by: the path of a +/// selector, or the path and the wildcard bound of a subtree read. +#[cfg(feature = "dynamic")] +pub(crate) enum ReadBy { + Selector(JsonPath), + Subtree(JsonPath, u32), +} + +/// Open the groups of rows a read returned, one value list per group, +/// under a keyset whose index key lasts one settle ([`Selectors`]). +/// +/// Under such a keyset the selector or the read key that checks the rows +/// cannot be derived before the settle, so the read is opened in one +/// settle that loads the index key beside the groups' data keys: the +/// selector or key is derived inside it, the rows are checked against it, +/// and they are opened. Every group opens as [`JsonExtracted`] or +/// [`JsonSubtree`] opens it, with one difference: a selector's entry that +/// carries another selector is refused after its data key is retrieved, +/// not before, since the selector is not known before. The pending is +/// confined to the keyset, as a read's openings are. +#[cfg(feature = "dynamic")] +pub(crate) fn open_read_in_settle<'a, 'c, K>( + keyset: &'a KeysetCipher<'_, K>, + read: ReadBy, + options: &JsonOptions, + descriptor: NonEmpty>, + context: impl Fn() -> AeadContext, + groups: Vec>, +) -> Result>, K>, Error> { + if let ReadBy::Subtree(path, bound) = &read { + let _ = read_count(path, *bound)?; + } + let keyset_id = keyset.keyset_id(); + let state = Arc::clone(keyset.state()); + let prf_context = descriptor.into_prf_context().into_owned(); + let mut requests = vec![Request::index_key(keyset_id)]; + let mut plans = Vec::with_capacity(groups.len()); + for nodes in groups { + let plan = match &read { + ReadBy::Selector(_) => match <[super::JsonNode; 1]>::try_from(nodes) { + Ok([node]) => { + let (request, aad) = key_retrieval(&node.header, context().into_context())?; + requests.push(request); + GroupPlan::Extracted(Some((node, aad))) + } + // A selector's group holds one entry or none. + Err(_) => GroupPlan::Extracted(None), + }, + ReadBy::Subtree(..) => { + let sealed_root = matches!( + nodes.as_slice(), + [node] if node.slot == 0 && node.depth == Some(0) && node.header.sealed_root() + ); + let planned = plan_rows(nodes, sealed_root, context())?; + requests.extend(planned.request()); + GroupPlan::Rows(planned) + } + }; + plans.push(plan); + } + let options = options.clone(); + let pending = Pending::request(keyset.cipher(), requests, move |responses| { + let prf = responses.index_prf_shared(keyset_id)?; + match read { + ReadBy::Selector(path) => { + let JsonSelector(selector) = selector_under(&prf, &path, &prf_context)?; + drop(prf); + plans + .into_iter() + .map(|plan| match plan { + GroupPlan::Extracted(Some((node, aad))) => { + if node.entry.s != selector { + return Err(wrong("the entry at the path asked for")); + } + Ok(vec![open_extracted(&node, &aad, responses)?]) + } + _ => Ok(Vec::new()), + }) + .collect() + } + ReadBy::Subtree(path, bound) => { + let selectors = Selectors::new(state, prf_context).in_settle(responses)?; + let key = subtree_key_for(selectors.clone(), &prf, &options, &path, bound)?; + drop(prf); + plans + .into_iter() + .map(|plan| match plan { + GroupPlan::Rows(planned) => { + Ok(open_rows(planned, &key, &selectors, responses)? + .into_iter() + .map(hand_over) + .collect()) + } + GroupPlan::Extracted(_) => Err(Error::ResponseShape), + }) + .collect() + } + } + }); + Ok(pending.scoped_to(keyset_id)) +} + +/// One group of a read's rows, planned by [`open_read_in_settle`]. +#[cfg(feature = "dynamic")] +enum GroupPlan { + /// A selector's entry and the AAD it opens under, or no entry. + Extracted(Option<(super::JsonNode, Context<'static>)>), + /// A subtree read's rows. + Rows(PlannedRows), } // ============================================================================= @@ -1998,6 +2261,31 @@ fn read_tokens(wildcards: usize, bound: u32) -> Option { Some(count) } +/// How many tokens a read of `path` at `bound` sends, or the refusal of a +/// path no read can take. Needs no key, so a read is refused before its +/// index key is loaded. +fn read_count(path: &JsonPath, bound: u32) -> Result { + // The bound is not checked against PostgreSQL's integer: a path with no + // wildcard never uses it, and with one the token cap refuses any bound + // of MAX_READ_TOKENS or more first, naming the bound in its help. + let segments = path.segments(); + // No sealed document has a node deeper than this, and the expansion + // recurses once per step. + if segments.len() > MAX_DEPTH { + return Err(TermError::JsonTooDeep); + } + for segment in segments { + if let Segment::Index(index) = segment { + let _ = fit_ordinal(*index)?; + } + } + let wildcards = segments + .iter() + .filter(|segment| matches!(segment, Segment::Wildcard)) + .count(); + read_tokens(wildcards, bound).ok_or(TermError::JsonWildcardTooWide) +} + /// The key that reads `path`: one token per concrete path the path names, /// each derived from the path alone, in document order, with a probe for /// the element at `bound` of each array a wildcard expands. @@ -2012,30 +2300,13 @@ fn read_tokens(wildcards: usize, bound: u32) -> Option { /// an element past the bound need not have what the path names. fn subtree_key_for( selectors: Selectors, + prf: &HmacSha256Prf, options: &JsonOptions, path: &JsonPath, bound: u32, ) -> Result { - // The bound is not checked against PostgreSQL's integer: a path with no - // wildcard never uses it, and with one the token cap refuses any bound - // of MAX_READ_TOKENS or more first, naming the bound in its help. + let count = read_count(path, bound)?; let segments = path.segments(); - // No sealed document has a node deeper than this, and the expansion - // recurses once per step. - if segments.len() > MAX_DEPTH { - return Err(TermError::JsonTooDeep); - } - for segment in segments { - if let Segment::Index(index) = segment { - let _ = fit_ordinal(*index)?; - } - } - let wildcards = segments - .iter() - .filter(|segment| matches!(segment, Segment::Wildcard)) - .count(); - let count = read_tokens(wildcards, bound).ok_or(TermError::JsonWildcardTooWide)?; - let prf = &selectors.keyset.prf; let context = &selectors.context; let mut steps = Vec::with_capacity(segments.len()); let mut prefix = JsonPath::root(); @@ -2107,32 +2378,51 @@ impl KeysetCipher<'_, K> { /// The selector of `path`, under the field `descriptor`: what `->` and /// the path functions take. Every element of an array shares its /// parent's selector, so a position in `path` reads as the wildcard. - /// Derived locally; the pending carries no request. + /// Derived locally; the pending carries no request, except under a + /// keyset whose index key lasts one settle, where it asks for that key + /// and the selector is derived inside the settle. pub fn json_selector<'c>( &self, path: &JsonPath, descriptor: NonEmpty>, ) -> Pending<'_, JsonSelector, K> { - Pending::ready(self, self.json_selector_now(path, descriptor)) + self.json_selector_planned(path, descriptor) + .unwrap_or_else(|error| Pending::failed(self, error)) + } + + /// [`json_selector`](Self::json_selector), with any refusal that can be + /// decided now returned as a plain error: for the engine's own callers. + /// Under a keyset that holds its PRF the selector is derived now, and + /// the pending is ready. + pub(crate) fn json_selector_planned<'c>( + &self, + path: &JsonPath, + descriptor: NonEmpty>, + ) -> Result, Error> { + match self.prf() { + Some(prf) => Ok(Pending::ready( + self, + Ok(self.json_selector_now(prf, path, descriptor)?), + )), + None => { + let context = descriptor.into_prf_context().into_owned(); + let path = path.clone(); + Ok(with_prf(self, move |prf| { + selector_under(prf, &path, &context) + })) + } + } } - /// [`json_selector`](Self::json_selector), derived and returned now: - /// for the engine's own callers, which derive locally and want a - /// refusal as a plain error. + /// The selector of `path` under `prf`, the PRF of this keyset that the + /// caller holds, derived and returned now. pub(crate) fn json_selector_now<'c>( &self, + prf: &HmacSha256Prf, path: &JsonPath, descriptor: NonEmpty>, ) -> Result { - let context = descriptor.into_prf_context().into_owned(); - prf16( - self.prf(), - SELECTOR_DOMAIN, - &context, - &path.selector_bytes(), - ) - .map(JsonSelector) - .map_err(Error::from) + selector_under(prf, path, &descriptor.into_prf_context().into_owned()) } /// The key that reads the node at `path`, under the field `descriptor` @@ -2183,16 +2473,45 @@ impl KeysetCipher<'_, K> { bound: u32, descriptor: NonEmpty>, ) -> Pending<'_, JsonSubtreeKey, K> { - Pending::ready( - self, - self.json_subtree_key_now(path, options, bound, descriptor), - ) + self.json_subtree_key_planned(path, options, bound, descriptor) + .unwrap_or_else(|error| Pending::failed(self, error)) } /// [`json_subtree_key_with_bound`](Self::json_subtree_key_with_bound), - /// derived and returned now, for the engine's own callers. + /// with any refusal that can be decided now returned as a plain error: + /// for the engine's own callers. Under a keyset that holds its PRF the + /// key is derived now, and the pending is ready; under one whose index + /// key lasts one settle, the path is checked now and its tokens are + /// derived inside the settle. The key holds no PRF either way. + pub(crate) fn json_subtree_key_planned<'c>( + &self, + path: &JsonPath, + options: &JsonOptions, + bound: u32, + descriptor: NonEmpty>, + ) -> Result, Error> { + match self.prf() { + Some(prf) => Ok(Pending::ready( + self, + Ok(self.json_subtree_key_now(prf, path, options, bound, descriptor)?), + )), + None => { + let _ = read_count(path, bound)?; + let context = descriptor.into_prf_context().into_owned(); + let selectors = Selectors::new(Arc::clone(self.state()), context); + let (options, path) = (options.clone(), path.clone()); + Ok(with_prf(self, move |prf| { + subtree_key_for(selectors, prf, &options, &path, bound).map_err(Error::from) + })) + } + } + } + + /// The read key of `path` under `prf`, the PRF of this keyset that the + /// caller holds, derived and returned now. The key holds no PRF. pub(crate) fn json_subtree_key_now<'c>( &self, + prf: &HmacSha256Prf, path: &JsonPath, options: &JsonOptions, bound: u32, @@ -2200,10 +2519,21 @@ impl KeysetCipher<'_, K> { ) -> Result { let context = descriptor.into_prf_context().into_owned(); let selectors = Selectors::new(Arc::clone(self.state()), context); - subtree_key_for(selectors, options, path, bound).map_err(Error::from) + subtree_key_for(selectors, prf, options, path, bound).map_err(Error::from) } } +/// The selector of `path` under `prf` and the column's PRF `context`. +fn selector_under( + prf: &HmacSha256Prf, + path: &JsonPath, + context: &Context<'_>, +) -> Result { + prf16(prf, SELECTOR_DOMAIN, context, &path.selector_bytes()) + .map(JsonSelector) + .map_err(Error::from) +} + #[cfg(test)] mod tests { use super::*; @@ -2360,6 +2690,7 @@ mod tests { fn key_for(options: &JsonOptions, path: &str, bound: u32) -> Result { subtree_key_for( Selectors::for_tests(), + &prf(), options, &JsonPath::parse(path).unwrap(), bound, @@ -2697,7 +3028,7 @@ mod tests { let options = JsonOptions { sealed_root: sealed, ..JsonOptions::new() }; let t = terms(&value, &options); for n in t.nodes() { - let key = subtree_key_for(Selectors::for_tests(), &options, &n.positional, 4).unwrap(); + let key = subtree_key_for(Selectors::for_tests(), &prf(), &options, &n.positional, 4).unwrap(); prop_assert_eq!(key.token_bytes(), vec![n.token], "{}", n.positional); } } @@ -3050,7 +3381,8 @@ mod tests { Arc::new(KeysetState { id: Uuid::from_u128(2), name: None, - prf: prf(), + prf: Some(prf()), + retention: crate::keyset::Retention::UntilEvicted, }), selectors.context.clone(), ); diff --git a/packages/stack-encrypt/src/json/mod.rs b/packages/stack-encrypt/src/json/mod.rs index 34857bca5c..b7e7a48af8 100644 --- a/packages/stack-encrypt/src/json/mod.rs +++ b/packages/stack-encrypt/src/json/mod.rs @@ -154,6 +154,8 @@ pub fn wipe_value(value: &mut serde_json::Value) { /// steps, which names no node. #[cfg(feature = "dynamic")] pub(crate) use derive::MAX_DEPTH; +#[cfg(feature = "dynamic")] +pub(crate) use derive::{open_read_in_settle, ReadBy}; pub(crate) use derive::{seal_pending, terms_pending}; pub(crate) use plain::PlainJson; pub(crate) use shuffle::Randomness; diff --git a/packages/stack-encrypt/src/keyset.rs b/packages/stack-encrypt/src/keyset.rs index a4473d13b7..0486ae8305 100644 --- a/packages/stack-encrypt/src/keyset.rs +++ b/packages/stack-encrypt/src/keyset.rs @@ -20,6 +20,12 @@ //! [`init`](crate::StackCipherBuilder::init), so a misconfigured client //! fails at startup, and never evicts. //! +//! A key backend says how long the cipher may keep a keyset's index key +//! ([`KeyLifetime`]): until evicted (ZeroKMS), for a TTL, or for one settle +//! only. The cache obeys it, so each backend does not have to. The default +//! keyset must be one the cipher may keep until evicted, because the cipher +//! holds it for its whole life. +//! //! # Ids are identity; names are looked up //! //! A keyset's id is its identity: a sealed leaf carries it, and an id @@ -51,7 +57,7 @@ use std::num::NonZeroUsize; use std::sync::Arc; use std::time::{Duration, Instant}; -use stack_kms::IdentifiedBy; +use stack_kms::{IdentifiedBy, KeyLifetime}; use uuid::Uuid; use vitaminc_hmac::HmacSha256Prf; @@ -64,11 +70,63 @@ use crate::StackCipher; pub const DEFAULT_NAME_TTL: Duration = Duration::from_secs(5 * 60); /// What the cipher holds per loaded keyset: its resolved id, the name it was -/// loaded under if any, and the PRF keyed by its index key. +/// loaded under if any, the PRF keyed by its index key, and how long the +/// cache may keep that PRF. +/// +/// `prf` is `None` for a keyset whose key backend loads the index key inside +/// each settle ([`SelectedKeyset::PerSettle`](stack_kms::SelectedKeyset)). +/// Such a state holds no key material, so the cache may keep it until it is +/// evicted, and a term built through it is derived inside the settle. pub(crate) struct KeysetState { pub(crate) id: Uuid, pub(crate) name: Option, - pub(crate) prf: HmacSha256Prf, + pub(crate) prf: Option, + pub(crate) retention: Retention, +} + +/// How long the [`KeysetCache`] may keep a keyset's state: the +/// [`KeyLifetime`] its key backend gave the index key, fixed to a point in +/// time when it was loaded. +/// +/// The cipher enforces this, not each backend. A state the cache may not +/// keep at all is handed to the caller and never stored, so the only +/// references to it are the [`KeysetCipher`] handles and the pendings +/// built from them. When those drop, the PRF drops with them. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) enum Retention { + /// Never cached: the index key is for the settle that needed it. + Never, + /// Cached until this instant, then loaded again. + Until(Instant), + /// Cached until the bound evicts it (the ZeroKMS behaviour). + UntilEvicted, +} + +impl Retention { + /// The retention of an index key with `lifetime`, loaded at `now`. + /// + /// A zero TTL is never cached, the same as a settle. A TTL too long for + /// the clock to represent is kept until evicted. A lifetime this crate + /// does not know yet is never cached: the short answer is the safe one. + pub(crate) fn of(lifetime: KeyLifetime, now: Instant) -> Self { + match lifetime { + KeyLifetime::UntilEvicted => Self::UntilEvicted, + KeyLifetime::Ttl(ttl) if !ttl.is_zero() => { + now.checked_add(ttl).map_or(Self::UntilEvicted, Self::Until) + } + _ => Self::Never, + } + } + + /// Whether the cache may still hand this state out at `now`. The end of + /// a TTL is exclusive: at the instant it ends, the key is expired. + pub(crate) fn is_live_at(self, now: Instant) -> bool { + match self { + Self::Never => false, + Self::Until(end) => now < end, + Self::UntilEvicted => true, + } + } } /// A loaded keyset in the cache, with the name it is currently bound under @@ -288,13 +346,28 @@ impl KeysetCache { /// Look a keyset up by id or name, marking it most recently used. pub(crate) fn get(&mut self, by: &IdentifiedBy) -> Lookup { + self.get_at(by, Instant::now()) + } + + /// [`get`](Self::get), at `now`. A keyset whose index key has passed the + /// end of its lifetime is evicted here, as if the bound had evicted it, + /// and the lookup is a miss: the caller loads it again. + fn get_at(&mut self, by: &IdentifiedBy, now: Instant) -> Lookup { let (id, fresh) = match by { IdentifiedBy::Uuid(id) => (*id, true), IdentifiedBy::Name(name) => match self.by_name.get::(name) { - Some(alias) => (alias.id, alias.is_fresh_at(Instant::now(), self.name_ttl)), + Some(alias) => (alias.id, alias.is_fresh_at(now, self.name_ttl)), None => return Lookup::Miss(self.resolution()), }, }; + if self + .by_id + .get(&id) + .is_some_and(|entry| !entry.state.retention.is_live_at(now)) + { + self.evict(id); + return Lookup::Miss(self.resolution()); + } let Some(state) = self.touch(id) else { return Lookup::Miss(self.resolution()); }; @@ -341,6 +414,19 @@ impl KeysetCache { state: Arc, resolution: Resolution, ) -> Arc { + self.insert_at(state, resolution, Instant::now()) + } + + /// [`insert`](Self::insert), at `now`. Every entry whose index key has + /// passed the end of its lifetime is evicted first, so expired key + /// material does not wait in memory for a lookup that may never come. + fn insert_at( + &mut self, + state: Arc, + resolution: Resolution, + now: Instant, + ) -> Arc { + self.evict_expired(now); // A name lookup is answered with whatever the name means now: when // a later lookup has already bound it — to this keyset or another — // the caller gets that keyset (held, since no binding outlives its @@ -492,7 +578,26 @@ impl KeysetCache { else { return; }; - if let Some(entry) = self.by_id.remove(&oldest) { + self.evict(oldest); + } + + /// Evict every entry whose index key is past the end of its lifetime at + /// `now`. + fn evict_expired(&mut self, now: Instant) { + let expired: Vec = self + .by_id + .iter() + .filter(|(_, entry)| !entry.state.retention.is_live_at(now)) + .map(|(id, _)| *id) + .collect(); + for id in expired { + self.evict(id); + } + } + + /// Evict the bounded entry for `oldest`, with the names bound to it. + fn evict(&mut self, id: Uuid) { + if let Some(entry) = self.by_id.remove(&id) { // The entry's place in the order outlives it as a watermark: // once it is gone there is nothing left to order an older answer // for this keyset against. It is taken whether or not the entry @@ -504,11 +609,7 @@ impl KeysetCache { if let Some(name) = entry.name { // A name that has since moved to another id keeps its // binding: only this id's binding goes with it. - if self - .by_name - .get(&name) - .is_some_and(|alias| alias.id == oldest) - { + if self.by_name.get(&name).is_some_and(|alias| alias.id == id) { let _ = self.by_name.remove(&name); } } @@ -607,12 +708,17 @@ impl<'k, K> KeysetCipher<'k, K> { } /// The PRF index terms are derived from, keyed by this keyset's index - /// key. + /// key, or `None` when this handle holds no index key. + /// + /// A keyset in a key vendor backend holds no index key between settles + /// (ADR-0009: no index key outlives the settle that needed it), so its + /// handle returns `None`, and this crate's term types derive their terms + /// inside the settle instead. A ZeroKMS keyset always returns `Some`. /// /// Public so that other crates can implement their own term types /// against this cipher (see [`crate::sem`]). - pub fn prf(&self) -> &HmacSha256Prf { - &self.state.prf + pub fn prf(&self) -> Option<&HmacSha256Prf> { + self.state.prf.as_ref() } /// The keyset's loaded state, shared: for a value that must derive @@ -640,7 +746,8 @@ mod tests { Arc::new(KeysetState { id: Uuid::from_u128(id), name: name.map(str::to_owned), - prf: HmacSha256Prf::new(Protected::new([id as u8; 32])), + prf: Some(HmacSha256Prf::new(Protected::new([id as u8; 32]))), + retention: Retention::UntilEvicted, }) } @@ -1444,4 +1551,286 @@ mod tests { "a wide window is fresh" ); } + + // ========================================================================= + // Key lifetime + // ========================================================================= + + fn expiring(id: u128, retention: Retention) -> Arc { + Arc::new(KeysetState { + id: Uuid::from_u128(id), + name: None, + prf: Some(HmacSha256Prf::new(Protected::new([id as u8; 32]))), + retention, + }) + } + + #[test] + fn a_key_lifetime_is_fixed_to_a_retention_when_it_is_loaded() { + let now = Instant::now(); + let hour = Duration::from_secs(3600); + assert_eq!( + Retention::of(KeyLifetime::UntilEvicted, now), + Retention::UntilEvicted + ); + assert_eq!(Retention::of(KeyLifetime::Settle, now), Retention::Never); + assert_eq!( + Retention::of(KeyLifetime::Ttl(Duration::ZERO), now), + Retention::Never, + "a zero TTL is a settle" + ); + assert_eq!( + Retention::of(KeyLifetime::Ttl(hour), now), + Retention::Until(now + hour) + ); + assert_eq!( + Retention::of(KeyLifetime::Ttl(Duration::MAX), now), + Retention::UntilEvicted, + "a TTL past the clock's range is kept until evicted" + ); + } + + #[test] + fn a_ttl_ends_exclusively() { + let now = Instant::now(); + let end = now + Duration::from_secs(1); + assert!(Retention::Until(end).is_live_at(now)); + assert!(!Retention::Until(end).is_live_at(end), "expired at its end"); + assert!(!Retention::Never.is_live_at(now)); + assert!(Retention::UntilEvicted.is_live_at(end)); + } + + /// A keyset whose TTL has ended is a miss, and is gone from the cache: + /// its PRF does not wait there for the next load. + #[test] + fn a_keyset_past_its_ttl_is_evicted_on_lookup() { + let mut cache = cache(4); + let now = Instant::now(); + let end = now + Duration::from_secs(60); + let loaded = expiring(1, Retention::Until(end)); + let watch = Arc::downgrade(&loaded); + let resolution = cache.resolution(); + let _ = cache.insert_at(loaded, resolution, now); + + assert!( + matches!( + cache.get_at(&id(1), end - Duration::from_nanos(1)), + Lookup::Hit(_) + ), + "within its TTL the keyset is cached" + ); + assert!( + matches!(cache.get_at(&id(1), end), Lookup::Miss(_)), + "at the end of its TTL the keyset is loaded again" + ); + assert_eq!(cache.len(), 0, "the expired entry is evicted"); + assert!(watch.upgrade().is_none(), "and its PRF is dropped"); + } + + /// An insert sweeps out every expired entry, not only the one it is + /// about: an entry nobody asks for again is not kept past its TTL. + #[test] + fn an_insert_evicts_every_expired_keyset() { + let mut cache = cache(4); + let now = Instant::now(); + let end = now + Duration::from_secs(60); + for (n, retention) in [(1, Retention::Until(end)), (2, Retention::UntilEvicted)] { + let resolution = cache.resolution(); + let _ = cache.insert_at(expiring(n, retention), resolution, now); + } + assert_eq!(cache.len(), 2); + + let resolution = cache.resolution(); + let _ = cache.insert_at(expiring(3, Retention::UntilEvicted), resolution, end); + + assert_eq!(cache.len(), 2, "keyset 1 expired, 2 and 3 remain"); + assert!(matches!(cache.get_at(&id(1), end), Lookup::Miss(_))); + assert!(matches!(cache.get_at(&id(2), end), Lookup::Hit(_))); + assert!(matches!(cache.get_at(&id(3), end), Lookup::Hit(_))); + } + + /// An expired keyset's name goes with it, as on eviction. + #[test] + fn an_expired_keyset_takes_its_name_with_it() { + let mut cache = cache(4); + cache.name_ttl = Duration::MAX; + let now = Instant::now(); + let end = now + Duration::from_secs(60); + let named = Arc::new(KeysetState { + id: Uuid::from_u128(1), + name: Some("acme".into()), + prf: Some(HmacSha256Prf::new(Protected::new([1u8; 32]))), + retention: Retention::Until(end), + }); + let resolution = cache.resolution(); + let _ = cache.insert_at(named, resolution, now); + assert!(matches!(cache.get_at(&name("acme"), now), Lookup::Hit(_))); + + assert!(matches!(cache.get_at(&name("acme"), end), Lookup::Miss(_))); + assert_eq!(cache.names(), 0); + assert!(matches!(cache.get_at(&name("acme"), end), Lookup::Miss(_))); + } + + mod through_the_cipher { + use super::*; + use std::borrow::Cow; + use std::sync::atomic::{AtomicUsize, Ordering}; + + use stack_kms::{ + DataKey, DataKeySource, DataKeyWithTag, FakeDataKeySource, GenerateKeyPayload, + IndexKey, IndexKeySource, LoadedIndexKey, RetrieveKeyPayload, UnverifiedContext, + }; + + use crate::{nonempty, Error}; + + /// A key backend that gives each keyset it loads a fixed lifetime, + /// and the default keyset its own. Keys are the fake's. + struct Lifetimed { + inner: FakeDataKeySource, + default: KeyLifetime, + selected: KeyLifetime, + loads: AtomicUsize, + } + + impl Lifetimed { + fn new(default: KeyLifetime, selected: KeyLifetime) -> Self { + Self { + inner: FakeDataKeySource::new(), + default, + selected, + loads: AtomicUsize::new(0), + } + } + + fn loads(&self) -> usize { + self.loads.load(Ordering::SeqCst) + } + } + + impl DataKeySource for Lifetimed { + async fn generate_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option>, + ) -> Result, stack_kms::Error> { + self.inner + .generate_keys(payloads, keyset_id, unverified_context) + .await + } + + async fn retrieve_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option<&UnverifiedContext>, + ) -> Result, stack_kms::Error> { + self.inner + .retrieve_keys(payloads, keyset_id, unverified_context) + .await + } + } + + impl IndexKeySource for Lifetimed { + async fn load_index_key( + &self, + keyset_id: Option, + ) -> Result<(Uuid, IndexKey), stack_kms::Error> { + self.inner.load_index_key(keyset_id).await + } + + async fn load_index_key_with_lifetime( + &self, + keyset_id: Option, + ) -> Result { + let _ = self.loads.fetch_add(1, Ordering::SeqCst); + let lifetime = match keyset_id { + None => self.default, + Some(_) => self.selected, + }; + let (id, key) = self.inner.load_index_key(keyset_id).await?; + Ok(LoadedIndexKey::new(id, key, lifetime)) + } + } + + /// The one test that proves key lifetime for every key backend: the + /// cipher, not the backend, obeys the lifetime. Two selections of + /// one keyset cost one load when the key may be cached, and two when + /// it may not; a key that may not be cached is gone once the handle + /// that used it drops. Under every lifetime the keyset still seals + /// and opens. + #[tokio::test] + async fn the_cipher_obeys_every_key_lifetime() { + let hour = Duration::from_secs(3600); + for (lifetime, cached) in [ + (KeyLifetime::UntilEvicted, true), + (KeyLifetime::Ttl(hour), true), + (KeyLifetime::Ttl(Duration::ZERO), false), + (KeyLifetime::Settle, false), + ] { + let cipher = StackCipher::builder() + .kms(Lifetimed::new(KeyLifetime::UntilEvicted, lifetime)) + .init() + .await + .unwrap(); + assert_eq!( + cipher.kms().loads(), + 1, + "{lifetime:?}: init loads the default" + ); + let tenant = Uuid::from_u128(42); + + let first = cipher.keyset(tenant).await.unwrap(); + let watch = Arc::downgrade(&first.state); + let sealed = first + .encrypt_tree("hello".to_string(), nonempty!("greeting")) + .await + .unwrap(); + drop(first); + + assert_eq!( + watch.upgrade().is_some(), + cached, + "{lifetime:?}: whether the cipher kept the index key" + ); + + let second = cipher.keyset(tenant).await.unwrap(); + let expected = if cached { 2 } else { 3 }; + assert_eq!( + cipher.kms().loads(), + expected, + "{lifetime:?}: a second selection reloads only an uncached key" + ); + let opened: String = second + .decrypt_tree(sealed, nonempty!("greeting")) + .await + .unwrap(); + assert_eq!(opened, "hello", "{lifetime:?}"); + } + } + + /// The cipher holds its default keyset for its whole life, so a + /// backend that will not let it keep that index key is refused at + /// init, not obeyed in part. + #[tokio::test] + async fn a_default_keyset_the_cipher_may_not_keep_is_refused_at_init() { + for lifetime in [ + KeyLifetime::Settle, + KeyLifetime::Ttl(Duration::from_secs(60)), + ] { + let refused = StackCipher::builder() + .kms(Lifetimed::new(lifetime, KeyLifetime::UntilEvicted)) + .init() + .await + .err(); + assert!( + matches!( + refused, + Some(Error::DefaultKeysetLifetime { keyset }) if keyset == Uuid::nil() + ), + "{lifetime:?}: {refused:?}" + ); + } + } + } } diff --git a/packages/stack-encrypt/src/sem/mod.rs b/packages/stack-encrypt/src/sem/mod.rs index 954260a4fc..68c92294d9 100644 --- a/packages/stack-encrypt/src/sem/mod.rs +++ b/packages/stack-encrypt/src/sem/mod.rs @@ -187,9 +187,69 @@ use zeroize::{Zeroize, Zeroizing}; use stack_kms::MaybeSend; use crate::target::core::Term; -use crate::target::{DecryptField, Decryptable, Decryption}; +use crate::target::{DecryptField, Decryptable, Decryption, Request}; use crate::{Error, KeysetCipher, Pending}; +/// Derive a term through `cipher`'s PRF. +/// +/// When the handle holds the PRF (a ZeroKMS keyset), the term is derived +/// now and the pending carries no requests. When it holds none (a keyset in +/// a key vendor backend, whose index key lasts one settle), the pending asks +/// the settle for the keyset's index key and `derive` runs inside the +/// settle, over the PRF the settle loaded. Either way `derive` sees the same +/// PRF, so the term is the same. +fn derive_term<'a, K, T, F>(cipher: &'a KeysetCipher<'_, K>, derive: F) -> Pending<'a, T, K> +where + T: MaybeSend + 'a, + F: FnOnce(&HmacSha256Prf) -> Result + MaybeSend + 'a, +{ + with_prf(cipher, move |prf| derive(prf).map_err(Error::from)) +} + +/// [`derive_term`] for any derivation through the keyset's PRF, failing +/// with any [`Error`]: the JSON terms, selectors and read keys use it. +pub(crate) fn with_prf<'a, K, T, F>(cipher: &'a KeysetCipher<'_, K>, derive: F) -> Pending<'a, T, K> +where + T: MaybeSend + 'a, + F: FnOnce(&HmacSha256Prf) -> Result + MaybeSend + 'a, +{ + match cipher.prf() { + Some(prf) => Pending::ready(cipher, derive(prf)), + None => { + let keyset = cipher.keyset_id(); + Pending::request(cipher, vec![Request::index_key(keyset)], move |responses| { + derive(responses.index_prf(keyset)?) + }) + } + } +} + +/// An equality term of `value` under `context`, through [`derive_term`]. +/// A value derived inside the settle is normalized now, because it may +/// borrow; the normalized form is owned. +fn equality_pending<'a, K, T>( + cipher: &'a KeysetCipher<'_, K>, + value: T, + context: Context<'_>, +) -> Pending<'a, EqualityTerm, K> +where + T: Encrypt, +{ + match cipher.prf() { + Some(prf) => Pending::ready(cipher, equality(prf, value, context).map_err(Error::from)), + None => { + let normalized = match normalized(value) { + Ok(normalized) => normalized, + Err(error) => return Pending::ready(cipher, Err(Error::from(error))), + }; + let context = context.into_owned(); + derive_term(cipher, move |prf| { + equality_of_normalized(prf, normalized, context) + }) + } + } +} + // The version suffix (`/v1`, `/v2`) versions the *derivation* (domain + // input framing), not the crate. Any change to the bytes a term derives from must bump it: a changed // derivation under an unchanged domain makes every existing term silently @@ -648,8 +708,7 @@ where Self: 'a, { let context = context.into_prf_context().into_owned(); - let term = equality(cipher.prf(), source, context).map_err(Error::from); - Pending::ready(cipher, term) + equality_pending(cipher, source, context) } } @@ -1333,9 +1392,18 @@ where Self: 'a, { let context = context.into_prf_context().into_owned(); - let term = - match_term(cipher.prf(), source.as_ref(), context, O::options()).map_err(Error::from); - Pending::ready(cipher, term) + match cipher.prf() { + Some(prf) => Pending::ready( + cipher, + match_term(prf, source.as_ref(), context, O::options()).map_err(Error::from), + ), + None => { + let text = zeroize::Zeroizing::new(source.as_ref().to_owned()); + derive_term(cipher, move |prf| { + match_term(prf, &text, context, O::options()) + }) + } + } } } @@ -2002,8 +2070,7 @@ where Self: 'a, { let context = context.into_prf_context().into_owned(); - let term = ore(cipher.prf(), source, context).map_err(Error::from); - Pending::ready(cipher, term) + derive_term(cipher, move |prf| ore(prf, source, context)) } } @@ -2026,8 +2093,7 @@ where Self: 'a, { let context = context.into_prf_context().into_owned(); - let term = ope(cipher.prf(), source, context).map_err(Error::from); - Pending::ready(cipher, term) + derive_term(cipher, move |prf| ope(prf, source, context)) } } @@ -2050,8 +2116,7 @@ where Self: 'a, { let context = context.into_prf_context().into_owned(); - let term = ore_block(cipher.prf(), source, context).map_err(Error::from); - Pending::ready(cipher, term) + derive_term(cipher, move |prf| ore_block(prf, source, context)) } } @@ -2121,8 +2186,7 @@ impl KeysetCipher<'_, K> { where T: Encrypt, { - let term = equality(self.prf(), value, descriptor.into_prf_context()); - Pending::ready(self, term.map_err(Error::from)) + equality_pending(self, value, descriptor.into_prf_context()) } /// Generate a match (full-text) term for `text` under the field @@ -2166,8 +2230,10 @@ impl KeysetCipher<'_, K> { normalized: Normalized, descriptor: NonEmpty>, ) -> Pending<'_, EqualityTerm, K> { - let term = equality_of_normalized(self.prf(), normalized, descriptor.into_prf_context()); - Pending::ready(self, term.map_err(Error::from)) + let context = descriptor.into_prf_context().into_owned(); + derive_term(self, move |prf| { + equality_of_normalized(prf, normalized, context) + }) } /// [`match_terms`](Self::match_terms) under runtime options, for the @@ -2186,8 +2252,17 @@ impl KeysetCipher<'_, K> { where O: MatchConfig + MaybeSend, { - let term = match_term(self.prf(), text, descriptor.into_prf_context(), options); - Pending::ready(self, term.map_err(Error::from)) + let context = descriptor.into_prf_context().into_owned(); + match self.prf() { + Some(prf) => Pending::ready( + self, + match_term(prf, text, context, options).map_err(Error::from), + ), + None => { + let text = zeroize::Zeroizing::new(text.to_owned()); + derive_term(self, move |prf| match_term(prf, &text, context, options)) + } + } } /// Generate an order-revealing (CLLW ORE) term for a range-queryable value @@ -2201,26 +2276,30 @@ impl KeysetCipher<'_, K> { /// wipes it; pass a `String` for borrowed text. Returns the raw CLLW /// ciphertext; the target-directed path wraps the same bytes in /// [`OreTerm`]. - pub fn ore_term<'c, T: Orderable>( - &self, + pub fn ore_term<'s, 'c, T: Orderable + 's>( + &'s self, value: T, descriptor: NonEmpty>, - ) -> Pending<'_, OreClww8VariableV1, K> { - let term = ore(self.prf(), value, descriptor.into_prf_context()).map(OreTerm::into_inner); - Pending::ready(self, term.map_err(Error::from)) + ) -> Pending<'s, OreClww8VariableV1, K> { + let context = descriptor.into_prf_context().into_owned(); + derive_term(self, move |prf| { + ore(prf, value, context).map(OreTerm::into_inner) + }) } /// Generate an order-preserving (CLLW OPE) term: ciphertexts compare with /// plain lexicographic byte order, no custom comparator required. /// Encrypt-only — pair with the record ciphertext for round-trips. Key /// handling and input bounds as for [`ore_term`](Self::ore_term). - pub fn ope_term<'c, T: Orderable>( - &self, + pub fn ope_term<'s, 'c, T: Orderable + 's>( + &'s self, value: T, descriptor: NonEmpty>, - ) -> Pending<'_, OpeClww8VariableV1, K> { - let term = ope(self.prf(), value, descriptor.into_prf_context()).map(OpeTerm::into_inner); - Pending::ready(self, term.map_err(Error::from)) + ) -> Pending<'s, OpeClww8VariableV1, K> { + let context = descriptor.into_prf_context().into_owned(); + derive_term(self, move |prf| { + ope(prf, value, context).map(OpeTerm::into_inner) + }) } /// Generate a block ORE term: ore-rs's 6-bit block ORE, the scheme @@ -2229,13 +2308,13 @@ impl KeysetCipher<'_, K> { /// [`OreBlockTerm`] for which scheme a type is encrypted with and what /// is refused. Byte-identical to the target-directed path's left half /// for the same descriptor; the right half carries a random nonce. - pub fn ore_block_term<'c, T: Orderable>( - &self, + pub fn ore_block_term<'s, 'c, T: Orderable + 's>( + &'s self, value: T, descriptor: NonEmpty>, - ) -> Pending<'_, OreBlockTerm, K> { - let term = ore_block(self.prf(), value, descriptor.into_prf_context()); - Pending::ready(self, term.map_err(Error::from)) + ) -> Pending<'s, OreBlockTerm, K> { + let context = descriptor.into_prf_context().into_owned(); + derive_term(self, move |prf| ore_block(prf, value, context)) } } diff --git a/packages/stack-encrypt/src/target/core.rs b/packages/stack-encrypt/src/target/core.rs index cbc024d876..0faf607096 100644 --- a/packages/stack-encrypt/src/target/core.rs +++ b/packages/stack-encrypt/src/target/core.rs @@ -99,8 +99,12 @@ pub(crate) fn seal_pending<'a, K>( .take(tree.key_count()) .collect(); Pending::request(cipher, requests, move |responses| { - tree.seal_with(keyset_id, responses.drain_generated()) - .map_err(Error::from) + tree.seal_with( + keyset_id, + responses.seal_header(), + responses.drain_generated(), + ) + .map_err(Error::from) }) } @@ -150,12 +154,7 @@ fn collect_retrieve_requests( ) { match ciphertext { CipherText::Single(leaf) | CipherText::EmptySequence(leaf) | CipherText::EmptyMap(leaf) => { - out.push(Request::retrieve_under( - *leaf.iv(), - leaf.tag().to_vec(), - descriptor.clone(), - leaf.keyset_id(), - )); + out.push(Request::retrieve_leaf(leaf, descriptor.clone())); } CipherText::Sequence(items) => { for item in items { diff --git a/packages/stack-encrypt/src/target/pending.rs b/packages/stack-encrypt/src/target/pending.rs index 6b2867b457..f178e54681 100644 --- a/packages/stack-encrypt/src/target/pending.rs +++ b/packages/stack-encrypt/src/target/pending.rs @@ -17,11 +17,13 @@ use std::pin::Pin; use std::sync::Arc; use stack_kms::{ - DataKey, DataKeySource, GenerateKeyPayload, IndexKeySource, Iv, MaybeSend, RetrieveKeyPayload, + DataKey, DataKeySource, GenerateKeyPayload, IndexKeySource, Iv, KeyBackend, KeyMode, MaybeSend, + RetrieveKeyPayload, }; use uuid::Uuid; use super::request::{tally, KeysetAnswer, Request, RequestKind, Responses}; +use crate::cipher::VendorHeader; use crate::keyset::KeysetState; use crate::{Descriptor, Error, KeysetCipher, StackCipher}; @@ -590,6 +592,8 @@ struct Retrieve { tag: Vec, descriptor: Descriptor, keyset_id: Uuid, + backend: KeyBackend, + mode: Option, } /// Issue the batched ZeroKMS calls for `requests`: at most one @@ -607,6 +611,7 @@ async fn dispatch( let mut generates: Vec = Vec::new(); let mut retrieves: Vec = Vec::new(); let mut loads: Vec = Vec::new(); + let mut index_keysets: Vec = Vec::new(); for request in requests { match request.into_kind() { RequestKind::GenerateDataKey { descriptor } => generates.push(descriptor), @@ -616,12 +621,21 @@ async fn dispatch( tag, descriptor, keyset_id, + backend, + mode, } => retrieves.push(Retrieve { iv, tag, descriptor, keyset_id, + backend, + mode, }), + RequestKind::IndexKeyForSettle { keyset_id } => { + if !index_keysets.contains(&keyset_id) { + index_keysets.push(keyset_id); + } + } } } @@ -636,6 +650,34 @@ async fn dispatch( .chain(retrieves.iter().map(|retrieve| &retrieve.descriptor)) .try_for_each(Descriptor::check)?; + // A leaf names the key backend that sealed it. One that names another + // backend than the one that holds its keyset is refused here, before any + // key call, as a foreign keyset is refused at construction: the leaf + // never reaches a backend that did not seal it. + if let Some((expected, found)) = retrieves.iter().find_map(|retrieve| { + let expected = cipher.kms().keyset_backend(retrieve.keyset_id); + (retrieve.backend != expected).then_some((expected, retrieve.backend)) + }) { + return Err(Error::ForeignBackend { expected, found }); + } + // The leaves this settle seals take the format of their keyset's + // backend: v1 under ZeroKMS, v2 (with the key mode) under a key vendor + // backend. A key vendor backend that names no mode for the keyset + // cannot say how to seal, so it is refused before any key call. + let seal_header = match (generates.is_empty(), keyset) { + (false, Some(keyset)) => match cipher.kms().keyset_backend(keyset) { + KeyBackend::ZeroKms => None, + backend @ KeyBackend::Vendor(vendor) => { + let mode = cipher + .kms() + .key_mode(keyset) + .ok_or(Error::SealUnsupported { backend })?; + Some(VendorHeader { vendor, mode }) + } + }, + _ => None, + }; + // The retrieves and the keyset loads run beside each other, after the // generates. Both sides are built here, before any key has been through // the stack, and boxed: a future is built on the stack and moved with @@ -678,7 +720,34 @@ async fn dispatch( let (retrieved, keysets) = futures::future::join(retrieve_keys, load_keysets).await; let retrieved = retrieved.map_err(|error| *error)?; - Ok(Responses::new(generated, retrieved).with_keysets(keysets)) + + // A keyset request answered with a keyset whose index key lasts one + // settle carries no PRF, so its index key is loaded below with the + // others: the fulfilment that asked for the keyset derives under it + // (`Selectors::in_settle` in the JSON module). + for answer in &keysets { + if let Ok(state) = &answer.outcome { + if state.prf.is_none() && !index_keysets.contains(&state.id) { + index_keysets.push(state.id); + } + } + } + + // The index keys of the keysets whose handles hold none: one load per + // keyset, inside this settle. The PRFs live in the responses and drop, + // with the keys wiped, when the settle ends. + let mut prfs = HashMap::with_capacity(index_keysets.len()); + for keyset_id in index_keysets { + let key = cipher.kms().load_settle_index_key(keyset_id).await?; + let _ = prfs.insert( + keyset_id, + Arc::new(crate::cipher::hmac_prf_from_index_key(&key)), + ); + } + + Ok(Responses::new(generated, retrieved) + .with_keysets(keysets) + .with_settle(prfs, seal_header)) } /// The data keys of `retrieves`, in request order: one `retrieve_keys` per @@ -706,7 +775,15 @@ async fn retrieve_data_keys( .iter() .map(|&index| { let retrieve = &retrieves[index]; - RetrieveKeyPayload::new(retrieve.iv, retrieve.descriptor.as_str(), &retrieve.tag) + let payload = RetrieveKeyPayload::new( + retrieve.iv, + retrieve.descriptor.as_str(), + &retrieve.tag, + ); + match retrieve.mode { + Some(mode) => payload.with_key_mode(mode), + None => payload, + } }) .collect(); let expected = payloads.len(); diff --git a/packages/stack-encrypt/src/target/request.rs b/packages/stack-encrypt/src/target/request.rs index 792929c8b4..e6d4bbecc3 100644 --- a/packages/stack-encrypt/src/target/request.rs +++ b/packages/stack-encrypt/src/target/request.rs @@ -12,10 +12,14 @@ //! sibling's key material are unit-testable on their own. use super::context::AeadContext; +use std::collections::HashMap; use std::sync::Arc; -use stack_kms::{DataKey, DataKeyWithTag, Iv}; +use stack_kms::{DataKey, DataKeyWithTag, Iv, KeyBackend, KeyMode}; use uuid::Uuid; +use vitaminc_hmac::HmacSha256Prf; + +use crate::cipher::VendorHeader; use crate::keyset::KeysetState; use crate::{Descriptor, Error}; @@ -40,6 +44,13 @@ pub(super) enum RequestKind { tag: Vec, descriptor: Descriptor, keyset_id: Uuid, + /// The key backend the leaf names. The settle refuses a request + /// whose backend is not the cipher's, before any key call. + backend: KeyBackend, + /// The key mode the leaf names (a v2 leaf); `None` for a v1 leaf. + /// Passed to the key backend, which refuses a mode that is not its + /// keyset's. + mode: Option, }, /// The loaded state of the keyset `keyset_id`, its index-key PRF: from /// the cipher's keyset cache, or loaded from ZeroKMS and cached on a @@ -53,6 +64,11 @@ pub(super) enum RequestKind { /// place a pending reaches ZeroKMS, so the keyset is asked for as a /// request and answered beside the document's data key. LoadKeyset { keyset_id: Uuid }, + /// Load the index key of `keyset_id` inside the settle, for a term + /// derived through a keyset whose handle holds no index key. Every such + /// request of one settle shares one load per keyset, and the key drops + /// when the settle ends. + IndexKeyForSettle { keyset_id: Uuid }, } impl Request { @@ -91,6 +107,44 @@ impl Request { Self::retrieve_under(iv, tag, Descriptor::of(context.into()), keyset_id) } + /// Request re-derivation of the data key of `leaf`, under `context` — + /// which must be the one the key was generated under. + /// + /// The same request as [`retrieve_data_key`](Self::retrieve_data_key) + /// with the leaf's iv, tag and keyset id, and it also carries the key + /// backend the leaf names ([`SealedValue::key_backend`]). The settle + /// refuses the request with [`Error::ForeignBackend`] before any key + /// call when that is not the cipher's key backend. + /// [`retrieve_data_key`](Self::retrieve_data_key) always names ZeroKMS, + /// so prefer this constructor when a [`SealedValue`] is in hand. + /// + /// [`SealedValue`]: crate::SealedValue + /// [`SealedValue::key_backend`]: crate::SealedValue::key_backend + pub fn retrieve_leaf_key(leaf: &crate::SealedValue, context: impl Into) -> Self { + Self::retrieve_leaf(leaf, Descriptor::of(context.into())) + } + + /// [`retrieve_leaf_key`](Self::retrieve_leaf_key) over an already + /// rendered descriptor. Crate-internal, as + /// [`generate_under`](Self::generate_under). + pub(crate) fn retrieve_leaf(leaf: &crate::SealedValue, descriptor: Descriptor) -> Self { + Self(RequestKind::RetrieveDataKey { + iv: *leaf.iv(), + tag: leaf.tag().to_vec(), + descriptor, + keyset_id: leaf.keyset_id(), + backend: leaf.key_backend(), + mode: leaf.key_mode(), + }) + } + + /// Ask the settle for the index key of `keyset_id`. Crate-internal: a + /// term derived through a keyset whose handle holds no index key pushes + /// it, and reads the PRF back with [`Responses::index_prf`]. + pub(crate) fn index_key(keyset_id: Uuid) -> Self { + Self(RequestKind::IndexKeyForSettle { keyset_id }) + } + /// [`generate_data_key`](Self::generate_data_key) over a descriptor that /// has already been rendered. /// @@ -117,6 +171,8 @@ impl Request { tag, descriptor, keyset_id, + backend: KeyBackend::ZeroKms, + mode: None, }) } @@ -132,13 +188,14 @@ impl Request { self.0 } - /// The keyset a retrieve or keyset request names; `None` for a generate - /// request, which mints under the pending's keyset. + /// The keyset a retrieve, keyset or index-key request names; `None` for + /// a generate request, which mints under the pending's keyset. pub(super) fn retrieve_keyset(&self) -> Option { match &self.0 { RequestKind::GenerateDataKey { .. } => None, RequestKind::RetrieveDataKey { keyset_id, .. } - | RequestKind::LoadKeyset { keyset_id } => Some(*keyset_id), + | RequestKind::LoadKeyset { keyset_id } + | RequestKind::IndexKeyForSettle { keyset_id } => Some(*keyset_id), } } } @@ -152,6 +209,8 @@ pub(super) fn tally(requests: &[Request]) -> (usize, usize, usize) { RequestKind::GenerateDataKey { .. } => generate += 1, RequestKind::RetrieveDataKey { .. } => retrieve += 1, RequestKind::LoadKeyset { .. } => keyset += 1, + // An index key is shared by every term of the settle, not drawn. + RequestKind::IndexKeyForSettle { .. } => {} } } (generate, retrieve, keyset) @@ -175,6 +234,14 @@ pub struct Responses { generated: Queue, retrieved: Queue, keysets: Queue, + /// The PRF of each keyset whose index key this settle loaded. Shared by + /// every fulfilment of the settle, and dropped (the key wiped) when the + /// last view of the responses drops, at the end of the settle. + prfs: Arc>>, + /// The v2 header the generated keys are sealed under: the key vendor + /// backend and the key mode of the pending's keyset. `None` seals v1 + /// (ZeroKMS). + seal_header: Option, } /// One kind's responses, drawn front first. A response is handed out as a @@ -227,6 +294,8 @@ impl Responses { generated: Queue::new(generated), retrieved: Queue::new(retrieved), keysets: Queue::new(Vec::new()), + prfs: Arc::default(), + seal_header: None, } } @@ -237,6 +306,43 @@ impl Responses { self } + /// Add what a settle loaded beside the data keys: the index-key PRFs and + /// the header of the leaves it seals. + pub(super) fn with_settle( + mut self, + prfs: HashMap>, + seal_header: Option, + ) -> Self { + self.prfs = Arc::new(prfs); + self.seal_header = seal_header; + self + } + + /// The PRF of `keyset`'s index key, loaded for this settle. A + /// fulfilment reads it only if its pending pushed + /// [`Request::index_key`] for that keyset; otherwise it is + /// [`Error::ResponseShape`]. + pub(crate) fn index_prf(&self, keyset: Uuid) -> Result<&HmacSha256Prf, Error> { + self.prfs + .get(&keyset) + .map(|prf| &**prf) + .ok_or(Error::ResponseShape) + } + + /// [`index_prf`](Self::index_prf), shared: for a fulfilment that hands + /// the PRF to a value it builds and drops inside the settle (a JSON + /// read's selectors). Nothing built with it may outlive the fulfilment, + /// or the index key outlives the settle. + pub(crate) fn index_prf_shared(&self, keyset: Uuid) -> Result, Error> { + self.prfs.get(&keyset).cloned().ok_or(Error::ResponseShape) + } + + /// The v2 header to seal the generated keys' leaves under, or `None` + /// for a v1 leaf. + pub(crate) fn seal_header(&self) -> Option { + self.seal_header + } + /// The next generated data key, in [`Request::generate_data_key`] order. pub fn next_generated_key(&mut self) -> Result { self.generated.pop_front().ok_or(Error::ResponseShape) @@ -277,6 +383,8 @@ impl Responses { generated: self.generated.split_front(generated), retrieved: self.retrieved.split_front(retrieved), keysets: self.keysets.split_front(keysets), + prfs: Arc::clone(&self.prfs), + seal_header: self.seal_header, }) } @@ -437,7 +545,11 @@ mod tests { tag, descriptor, keyset_id, + backend, + mode, } => { + assert_eq!(backend, stack_kms::KeyBackend::ZeroKms); + assert_eq!(mode, None); assert_eq!(keyset_id, ks()); assert_eq!(iv, Iv::default()); assert_eq!(tag, vec![7, 8, 9]); @@ -603,7 +715,10 @@ mod tests { outcome: Ok(Arc::new(KeysetState { id: Uuid::from_u128(id), name: None, - prf: vitaminc_hmac::HmacSha256Prf::new(vitaminc_protected::Protected::new([1; 32])), + prf: Some(vitaminc_hmac::HmacSha256Prf::new( + vitaminc_protected::Protected::new([1; 32]), + )), + retention: crate::keyset::Retention::UntilEvicted, })), } } diff --git a/packages/stack-encrypt/tests/common/mod.rs b/packages/stack-encrypt/tests/common/mod.rs index 9fe1842226..79e01f3825 100644 --- a/packages/stack-encrypt/tests/common/mod.rs +++ b/packages/stack-encrypt/tests/common/mod.rs @@ -15,7 +15,8 @@ use std::sync::{Arc, Mutex}; use stack_encrypt::StackCipher; use stack_kms::{ DataKey, DataKeySource, DataKeyWithTag, FakeDataKeySource, GenerateKeyPayload, IdentifiedBy, - IndexKey, IndexKeySource, RetrieveKeyPayload, UnverifiedContext, + IndexKey, IndexKeySource, KeyBackend, KeyMode, RetrieveKeyPayload, SelectedKeyset, + UnverifiedContext, }; use uuid::Uuid; @@ -92,12 +93,39 @@ pub struct CountingSource { generate_calls: Arc, retrieve_calls: Arc, load_calls: Arc, + backend: KeyBackend, + mode: Option, + /// The key mode of every retrieve payload, in order. + retrieved_modes: Arc>>>, } impl CountingSource { pub fn new() -> Self { Self::over(FakeDataKeySource::new()) } + + /// The same source, reporting `mode` as the key mode of every keyset. + pub fn with_mode(self, mode: KeyMode) -> Self { + Self { + mode: Some(mode), + ..self + } + } + + /// The key mode of every retrieve payload the cipher sent, in order. + pub fn retrieved_modes(&self) -> Arc>>> { + self.retrieved_modes.clone() + } + + /// The same source, reporting `backend` as its key backend. The keys + /// still come from the fake, so this stands in for a key vendor backend + /// only in what the cipher routes on. + pub fn with_backend(backend: KeyBackend) -> Self { + Self { + backend, + ..Self::new() + } + } } impl CountingSource { @@ -107,6 +135,9 @@ impl CountingSource { generate_calls: Arc::new(AtomicUsize::new(0)), retrieve_calls: Arc::new(AtomicUsize::new(0)), load_calls: Arc::new(AtomicUsize::new(0)), + backend: KeyBackend::ZeroKms, + mode: None, + retrieved_modes: Arc::default(), } } @@ -141,10 +172,22 @@ impl DataKeySource for CountingSource { unverified_context: Option<&UnverifiedContext>, ) -> Result, stack_kms::Error> { self.retrieve_calls.fetch_add(1, AtomicOrdering::SeqCst); + self.retrieved_modes + .lock() + .unwrap() + .extend(payloads.iter().map(|payload| payload.key_mode)); self.inner .retrieve_keys(payloads, keyset_id, unverified_context) .await } + + fn key_backend(&self) -> KeyBackend { + self.backend + } + + fn key_mode(&self, _keyset_id: Uuid) -> Option { + self.mode + } } impl IndexKeySource for CountingSource { @@ -157,6 +200,99 @@ impl IndexKeySource for CountingSource { } } +/// A key source whose keysets load their index key inside each settle, as +/// a key vendor backend's do ([`SelectedKeyset::PerSettle`]), over the keys +/// of a [`DeterministicSource`] seeded with [`PER_SETTLE_SEED`]. A cipher +/// over it holds no index key between settles, so it derives every term +/// inside the settle that needs it, under the index key that source gives +/// a cipher that holds it ([`deterministic_cipher`]): the two derive the +/// same terms and open each other's ciphertexts. +pub struct PerSettleSource { + inner: DeterministicSource, + settle_loads: Arc, +} + +/// The seed of the [`DeterministicSource`] under a [`PerSettleSource`]. +pub const PER_SETTLE_SEED: [u8; 32] = [11; 32]; + +impl PerSettleSource { + pub fn new() -> Self { + Self { + inner: DeterministicSource::new(PER_SETTLE_SEED), + settle_loads: Arc::new(AtomicUsize::new(0)), + } + } + + /// The counter of index keys loaded for one settle. + pub fn settle_loads(&self) -> Arc { + self.settle_loads.clone() + } +} + +impl DataKeySource for PerSettleSource { + async fn generate_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option>, + ) -> Result, stack_kms::Error> { + self.inner + .generate_keys(payloads, keyset_id, unverified_context) + .await + } + + async fn retrieve_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option<&UnverifiedContext>, + ) -> Result, stack_kms::Error> { + self.inner + .retrieve_keys(payloads, keyset_id, unverified_context) + .await + } + + async fn load_settle_index_key(&self, keyset_id: Uuid) -> Result { + self.settle_loads.fetch_add(1, AtomicOrdering::SeqCst); + let (_, key) = self + .inner + .load_index_key(Some(IdentifiedBy::Uuid(keyset_id))) + .await?; + Ok(key) + } +} + +impl IndexKeySource for PerSettleSource { + async fn load_index_key( + &self, + keyset_id: Option, + ) -> Result<(Uuid, IndexKey), stack_kms::Error> { + self.inner.load_index_key(keyset_id).await + } + + async fn select_keyset( + &self, + keyset_id: Option, + ) -> Result { + let (keyset_id, _) = self.inner.load_index_key(keyset_id).await?; + Ok(SelectedKeyset::PerSettle { keyset_id }) + } +} + +/// A cipher over [`PerSettleSource`], with its counter of index keys +/// loaded for one settle, zeroed once the cipher is built. +pub async fn per_settle_cipher() -> (StackCipher, Arc) { + let source = PerSettleSource::new(); + let settle_loads = source.settle_loads(); + let cipher = StackCipher::builder() + .kms(source) + .init() + .await + .expect("build cipher"); + settle_loads.store(0, AtomicOrdering::SeqCst); + (cipher, settle_loads) +} + /// A cipher over [`CountingSource`], with its counter of keyset lookups, /// zeroed once the cipher has loaded its default keyset. pub async fn loads_counting_cipher() -> (StackCipher, Arc) { diff --git a/packages/stack-encrypt/tests/frozen_bytes.rs b/packages/stack-encrypt/tests/frozen_bytes.rs index 9cbcec7e71..8b682a8454 100644 --- a/packages/stack-encrypt/tests/frozen_bytes.rs +++ b/packages/stack-encrypt/tests/frozen_bytes.rs @@ -1,9 +1,11 @@ //! Byte-level pins for the frozen encodings stack-encrypt commits to across //! languages: //! -//! * the [`SealedValue`] leaf layout -//! (`version ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ local_ciphertext`) — the -//! storage +//! * the [`SealedValue`] leaf layouts — v1 +//! (`version ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ local_ciphertext`, which +//! ZeroKMS writes) and v2 +//! (`version ‖ backend ‖ mode ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ +//! local_ciphertext`, which a key vendor backend writes) — the storage //! format a database column holds, and //! * the index-term encodings (equality: raw 32 bytes; match: LE `u16` //! positions; ORE/OPE: raw CLLW ciphertext bytes). @@ -29,8 +31,9 @@ use stack_encrypt::sem::{ use stack_encrypt::target::legacy::EncryptInto; use stack_encrypt::{CipherText, Error, LeafBytesError, SealedValue, StackCipher}; use stack_kms::{ - DataKey, DataKeySource, DataKeyWithTag, FakeDataKeySource, GenerateKeyPayload, IdentifiedBy, - IndexKey, IndexKeySource, RetrieveKeyPayload, UnverifiedContext, + DataKey, DataKeySource, DataKeyWithTag, DeterministicSource, FakeDataKeySource, + GenerateKeyPayload, IdentifiedBy, IndexKey, IndexKeySource, KeyBackend, KeyMode, KeyVendor, + RetrieveKeyPayload, UnverifiedContext, }; use uuid::Uuid; @@ -46,6 +49,13 @@ fn hex(bytes: &[u8]) -> String { bytes.iter().map(|b| format!("{b:02x}")).collect() } +fn unhex(hex: &str) -> Vec { + (0..hex.len()) + .step_by(2) + .map(|i| u8::from_str_radix(&hex[i..i + 2], 16).expect("hex")) + .collect() +} + // ============================================================================= // SealedValue leaf // ============================================================================= @@ -98,11 +108,168 @@ fn sealed_value_from_bytes_inverts_to_bytes() { #[test] fn sealed_value_rejects_unknown_version() { let mut bytes = fixture_leaf().to_bytes(); - bytes[0] = 2; - assert!(matches!( - SealedValue::from_bytes(&bytes), - Err(LeafBytesError::UnknownVersion(2)) - )); + for version in [0, 3, u8::MAX] { + bytes[0] = version; + assert_eq!( + SealedValue::from_bytes(&bytes).map(|_| ()), + Err(LeafBytesError::UnknownVersion(version)) + ); + } +} + +/// A known-answer test for a real v1 leaf: these bytes were sealed once by +/// [`DeterministicSource`] (seed `[7; 32]`) under the context +/// `users/email`, and must open, unchanged, for as long as v1 is read. A +/// change to the v1 framing, the v1 leaf AAD or the v1 decoder fails here. +#[tokio::test] +async fn a_pinned_v1_leaf_still_opens() { + const V1_LEAF: &str = "010000000000000000000000000000000045608626b2b6c6de1bfc90dd87c420ac2000d702c5b01099cca8d52801ec7a9ee26928d0f8c71e120a2a62cc9b7f2e96579a0217b7edda95f2d12a73f9735ffa3d8c85c6d15ae926a3e179d6d05cefd80edc848475f7d5ce7dc54a4c"; + let bytes = unhex(V1_LEAF); + let leaf = SealedValue::from_bytes(&bytes).expect("decode the pinned v1 leaf"); + assert_eq!(leaf.format_version(), SealedValue::FORMAT_VERSION); + assert_eq!(leaf.key_backend(), KeyBackend::ZeroKms); + assert_eq!(leaf.key_mode(), None); + assert_eq!( + hex(&leaf.to_bytes()), + V1_LEAF, + "v1 re-encodes byte for byte" + ); + + let cipher = StackCipher::builder() + .kms(DeterministicSource::new([7; 32])) + .init() + .await + .expect("build cipher"); + let plaintext: String = cipher + .decrypt(CipherText::Single(leaf), b"users/email".as_slice()) + .await + .expect("the pinned v1 leaf opens"); + assert_eq!(plaintext, "known answer"); +} + +// ============================================================================= +// SealedValue v2 leaf +// ============================================================================= + +/// [`fixture_leaf`]'s parts as a v2 leaf of `vendor` in `mode`. +fn fixture_v2_leaf(vendor: KeyVendor, mode: KeyMode) -> SealedValue { + let (keyset_id, iv, tag, ciphertext) = fixture_leaf().into_parts(); + SealedValue::from_vendor_parts(vendor, mode, keyset_id, iv, tag, ciphertext) + .expect("fixture tag fits the length field") +} + +#[test] +fn v2_sealed_value_layout_is_pinned() { + let bytes = fixture_v2_leaf(KeyVendor::Aws, KeyMode::Wrapped).to_bytes(); + + // version(02) ‖ backend(01 — AWS KMS) ‖ mode(01 — wrapped) ‖ then the + // v1 fields: keyset_id ‖ iv ‖ tag_len ‖ tag ‖ local_ciphertext. + assert_eq!( + hex(&bytes), + "0201016b65797365742d666978747572653136303132333435363738396162636465660300aabbccdeadbeef" + ); + // Past the two extra header bytes, v2 is the v1 layout. + assert_eq!(&bytes[3..], &fixture_leaf().to_bytes()[1..]); +} + +#[test] +fn v2_sealed_value_from_bytes_inverts_to_bytes_for_every_backend_and_mode() { + for vendor in KeyVendor::ALL { + for mode in KeyMode::ALL { + let original = fixture_v2_leaf(vendor, mode); + let bytes = original.to_bytes(); + assert_eq!(&bytes[..3], &[2, vendor.byte(), mode.byte()]); + let decoded = SealedValue::from_bytes(&bytes).expect("decode v2 leaf"); + assert_eq!(decoded.format_version(), SealedValue::VENDOR_FORMAT_VERSION); + assert_eq!(decoded.key_backend(), KeyBackend::Vendor(vendor)); + assert_eq!(decoded.key_mode(), Some(mode)); + assert_eq!(decoded.keyset_id(), original.keyset_id()); + assert_eq!(decoded.iv(), original.iv()); + assert_eq!(decoded.tag(), original.tag()); + assert_eq!(decoded.ciphertext(), original.ciphertext()); + assert_eq!(decoded.to_bytes(), bytes); + assert_eq!( + decoded.clone().to_bytes(), + bytes, + "a clone keeps the v2 header" + ); + } + } +} + +#[test] +fn v2_sealed_value_rejects_an_unknown_backend_or_mode() { + let bytes = fixture_v2_leaf(KeyVendor::Static, KeyMode::Derived).to_bytes(); + for byte in [0, 6, u8::MAX] { + let mut backend = bytes.clone(); + backend[1] = byte; + assert_eq!( + SealedValue::from_bytes(&backend).map(|_| ()), + Err(LeafBytesError::UnknownBackend(byte)) + ); + } + for byte in [0, 3, u8::MAX] { + let mut mode = bytes.clone(); + mode[2] = byte; + assert_eq!( + SealedValue::from_bytes(&mode).map(|_| ()), + Err(LeafBytesError::UnknownMode(byte)) + ); + } +} + +#[test] +fn v2_sealed_value_rejects_truncation() { + let bytes = fixture_v2_leaf(KeyVendor::Vault, KeyMode::Wrapped).to_bytes(); + let tag_end = 3 + 16 + 16 + 2 + 3; + for len in 0..tag_end { + assert_eq!( + SealedValue::from_bytes(&bytes[..len]).map(|_| ()), + Err(LeafBytesError::Truncated), + "prefix of {len} bytes must be rejected" + ); + } + assert!(SealedValue::from_bytes(&bytes[..tag_end]).is_ok()); +} + +/// Under derived data keys the tag is empty: the shortest v2 leaf is the +/// header and the fixed-width fields. +#[test] +fn v2_sealed_value_decodes_the_shortest_leaf_the_layout_allows() { + let leaf = SealedValue::from_vendor_parts( + KeyVendor::Static, + KeyMode::Derived, + Uuid::nil(), + [7; 16], + Vec::new(), + Vec::new(), + ) + .expect("empty tag fits"); + let bytes = leaf.to_bytes(); + assert_eq!(bytes.len(), 3 + 16 + 16 + 2); + let decoded = SealedValue::from_bytes(&bytes).expect("the fixed fields alone are a leaf"); + assert!(decoded.tag().is_empty()); + assert!(decoded.ciphertext().is_empty()); + assert_eq!( + SealedValue::from_bytes(&bytes[..bytes.len() - 1]).map(|_| ()), + Err(LeafBytesError::Truncated) + ); +} + +#[test] +fn v2_sealed_value_rejects_oversized_tag_on_construction() { + let result = SealedValue::from_vendor_parts( + KeyVendor::Aws, + KeyMode::Wrapped, + Uuid::nil(), + [0; 16], + vec![0; usize::from(u16::MAX) + 1], + Vec::new(), + ); + assert_eq!( + result.map(|_| ()), + Err(LeafBytesError::TagTooLong(usize::from(u16::MAX) + 1)) + ); } #[test] diff --git a/packages/stack-encrypt/tests/json_operators.rs b/packages/stack-encrypt/tests/json_operators.rs index 902628dacd..5788385de0 100644 --- a/packages/stack-encrypt/tests/json_operators.rs +++ b/packages/stack-encrypt/tests/json_operators.rs @@ -284,7 +284,22 @@ mod bytes { /// and a text array for a subtree read. #[tokio::test] async fn every_operator_derives_its_tagged_operand() { - let cipher = common::stack_cipher().await; + every_operator_derives_its_tagged_operand_under(common::stack_cipher().await).await; +} + +/// As above, under a keyset whose index key lasts one settle: every +/// operand is derived inside the settle, in the same encoding. +#[tokio::test] +async fn every_operator_derives_its_tagged_operand_per_settle() { + let (cipher, settle_loads) = common::per_settle_cipher().await; + every_operator_derives_its_tagged_operand_under(cipher).await; + assert!( + settle_loads.load(std::sync::atomic::Ordering::SeqCst) > 0, + "the operands were derived under index keys loaded per settle" + ); +} + +async fn every_operator_derives_its_tagged_operand_under(cipher: StackCipher) { let keyset = cipher.default_keyset(); let plan = plan(); let needle = |term: &QueryTerm| match term.operand() { @@ -572,7 +587,25 @@ fn malformed_inputs_are_refused_with_their_reason() { /// read's request or past the bound, each group opening on its own. #[tokio::test] async fn reads_open_against_the_request_that_prepared_them() { - let cipher = common::stack_cipher().await; + reads_open_against_the_request_that_prepared_them_under(common::stack_cipher().await).await; +} + +/// As above, under a keyset whose index key lasts one settle: the read's +/// selector or key is derived, and its rows checked and opened, inside the +/// one settle that loads the index key. +#[tokio::test] +async fn reads_open_against_the_request_that_prepared_them_per_settle() { + let (cipher, settle_loads) = common::per_settle_cipher().await; + reads_open_against_the_request_that_prepared_them_under(cipher).await; + assert!( + settle_loads.load(std::sync::atomic::Ordering::SeqCst) > 0, + "the reads were derived under index keys loaded per settle" + ); +} + +async fn reads_open_against_the_request_that_prepared_them_under( + cipher: StackCipher, +) { let keyset = cipher.default_keyset(); let plan = plan(); let docs = [ diff --git a/packages/stack-encrypt/tests/json_per_settle.rs b/packages/stack-encrypt/tests/json_per_settle.rs new file mode 100644 index 0000000000..aa146d1c80 --- /dev/null +++ b/packages/stack-encrypt/tests/json_per_settle.rs @@ -0,0 +1,179 @@ +//! The JSON index under a keyset whose index key lasts one settle, as a key +//! vendor backend's does: the cipher holds no index key, so a document's +//! terms, a read's selector and key, and the selectors that check a read's +//! rows are all derived inside the settle that loads the key. They are the +//! ones a cipher that holds the same key derives, so a document sealed +//! through either opens through the other. + +mod common; + +use std::sync::atomic::Ordering; + +use common::PerSettleSource; +use serde_json::{json, Value}; +use stack_encrypt::json::{ + JsonDocument, JsonNode, JsonOptions, JsonPath, JsonSelector, JsonSubtree, JsonSubtreeKey, +}; +use stack_encrypt::target::{AeadContext, Borrowed, CallerContext}; +use stack_encrypt::{nonempty, StackCipher}; +use stack_kms::{DataKeySource, DeterministicSource, IndexKeySource}; + +fn fixture() -> Value { + json!({"hello": "world-1", "nested": {"deep": "constant", "list": [{"x": 1}, {"x": 2}]}, "number": 1}) +} + +fn aead() -> AeadContext { + AeadContext::from(nonempty!("users").with("profile")) +} + +fn open_root() -> JsonOptions { + JsonOptions { + sealed_root: false, + ..JsonOptions::new() + } +} + +async fn seal(cipher: &StackCipher, value: &Value, options: &JsonOptions) -> JsonDocument +where + K: DataKeySource + IndexKeySource + Send + Sync + 'static, +{ + cipher + .default_keyset() + .run( + stack_encrypt::target::json_document::<_, Borrowed>(options.clone()), + value, + CallerContext::from(nonempty!("users").with("profile")), + ) + .await + .expect("seals") +} + +async fn open(cipher: &StackCipher, doc: JsonDocument) -> Value +where + K: DataKeySource + IndexKeySource + Send + Sync + 'static, +{ + cipher + .decrypt_as::(doc, aead()) + .await + .expect("opens") +} + +async fn selector(cipher: &StackCipher, path: &str) -> JsonSelector +where + K: DataKeySource + IndexKeySource + Send + Sync + 'static, +{ + cipher + .default_keyset() + .json_selector( + &JsonPath::parse(path).unwrap(), + nonempty!("users").with("profile"), + ) + .await + .expect("derives a selector") +} + +async fn key(cipher: &StackCipher, path: &str, options: &JsonOptions) -> JsonSubtreeKey +where + K: DataKeySource + IndexKeySource + Send + Sync + 'static, +{ + cipher + .default_keyset() + .json_subtree_key( + &JsonPath::parse(path).unwrap(), + options, + nonempty!("users").with("profile"), + ) + .await + .expect("derives a read key") +} + +/// The rows `json_tree_subtree(doc, key.tokens(), 0)` returns for `key`. +fn walk(doc: &JsonDocument, key: &JsonSubtreeKey) -> Vec { + doc.subtree(&key.token_bytes()) + .iter() + .map(|(j, _, d)| { + let mut wire = serde_json::to_value(doc.node(*j).unwrap()).unwrap(); + wire["d"] = json!(d); + serde_json::from_value(wire).unwrap() + }) + .collect() +} + +/// A cipher that holds its index key, one whose index key lasts one +/// settle, over the same keys, and the latter's counter of settle loads. +async fn ciphers() -> ( + StackCipher, + StackCipher, + std::sync::Arc, +) { + let (per_settle, loads) = common::per_settle_cipher().await; + let held = common::deterministic_cipher(common::PER_SETTLE_SEED).await; + (held, per_settle, loads) +} + +#[tokio::test] +async fn a_per_settle_keyset_holds_no_index_key() { + let (_, per_settle, loads) = ciphers().await; + assert!(per_settle.default_keyset().prf().is_none()); + assert_eq!(loads.load(Ordering::SeqCst), 0, "building loads none"); +} + +/// A document sealed under either cipher opens under both, with the root +/// open (each read derives the root's token under the keyset the document +/// names) and sealed. +#[tokio::test] +async fn a_document_opens_under_either_cipher() { + let (held, per_settle, loads) = ciphers().await; + for options in [open_root(), JsonOptions::default()] { + let mine = seal(&per_settle, &fixture(), &options).await; + assert_eq!(open(&per_settle, mine.clone()).await, fixture()); + assert_eq!(open(&held, mine).await, fixture()); + let theirs = seal(&held, &fixture(), &options).await; + assert_eq!(open(&per_settle, theirs).await, fixture()); + } + assert!(loads.load(Ordering::SeqCst) > 0); +} + +/// A selector and a read key derived inside the settle are the ones a +/// cipher holding the index key derives. +#[tokio::test] +async fn selectors_and_read_keys_match_a_cipher_that_holds_the_key() { + let (held, per_settle, loads) = ciphers().await; + for path in ["$", "$.hello", "$.nested.list[*].x"] { + assert_eq!( + selector(&per_settle, path).await.to_string(), + selector(&held, path).await.to_string(), + "{path}" + ); + for options in [open_root(), JsonOptions::default()] { + assert_eq!( + key(&per_settle, path, &options).await.tokens(), + key(&held, path, &options).await.tokens(), + "{path}" + ); + } + } + assert_eq!(loads.load(Ordering::SeqCst), 9, "one load per settle"); +} + +/// A read keyed under a per-settle keyset opens its rows: the selectors +/// that check them are derived under the index key the open's settle +/// loads, since the key holds none. +#[tokio::test] +async fn a_subtree_read_opens_under_a_per_settle_keyset() { + let (held, per_settle, loads) = ciphers().await; + let options = open_root(); + let doc = seal(&held, &fixture(), &options).await; + let read = key(&per_settle, "$.nested.list[*].x", &options).await; + let before = loads.load(Ordering::SeqCst); + let values = per_settle + .decrypt_as::, JsonSubtree>(read.open(walk(&doc, &read)), aead()) + .await + .expect("opens"); + assert_eq!(values, [json!(1), json!(2)]); + assert_eq!( + loads.load(Ordering::SeqCst), + before + 1, + "the open loads one" + ); +} diff --git a/packages/stack-encrypt/tests/key_backend.rs b/packages/stack-encrypt/tests/key_backend.rs new file mode 100644 index 0000000000..d9986dde57 --- /dev/null +++ b/packages/stack-encrypt/tests/key_backend.rs @@ -0,0 +1,480 @@ +//! Routing by key backend: a sealed leaf names the key backend that sealed +//! it (a v1 leaf names ZeroKMS, a v2 leaf its key vendor backend), and a +//! cipher refuses a leaf that names another backend than its own before +//! any key call, as it refuses a leaf from a foreign keyset. +//! +//! The "vendor" cipher here is the fake source reporting a vendor backend: +//! what the cipher routes and seals on, and nothing else. The real adapter +//! and its contract suite are in `stack-kms-vendor`. + +mod common; + +use std::sync::atomic::Ordering; + +use common::CountingSource; +use stack_encrypt::kms::{KeyBackend, KeyMode, KeyVendor}; +use stack_encrypt::{nonempty, CipherText, Error, SealedValue, StackCipher, StackCipherText}; +use uuid::Uuid; + +const AAD: &[u8] = b"ctx"; + +async fn cipher_over( + source: CountingSource, +) -> ( + StackCipher, + std::sync::Arc, + std::sync::Arc, +) { + let (generates, retrieves) = source.counters(); + let cipher = StackCipher::builder() + .kms(source) + .init() + .await + .expect("build cipher"); + (cipher, generates, retrieves) +} + +fn v2_leaf(vendor: KeyVendor, keyset: Uuid) -> SealedValue { + SealedValue::from_vendor_parts( + vendor, + KeyMode::Wrapped, + keyset, + [3; 16], + vec![9; 8], + vec![0xDE, 0xAD], + ) + .expect("v2 leaf") +} + +fn v1_leaf(keyset: Uuid) -> SealedValue { + SealedValue::from_parts(keyset, [3; 16], vec![9; 8], vec![0xDE, 0xAD]).expect("v1 leaf") +} + +fn assert_foreign(result: Result, expected: KeyBackend, found: KeyBackend) { + match result { + Err(Error::ForeignBackend { + expected: e, + found: f, + }) => { + assert_eq!((e, f), (expected, found)); + } + other => panic!("expected ForeignBackend, got {other:?}"), + } +} + +/// A ZeroKMS cipher refuses a v2 leaf before any retrieve, through the +/// client and through a keyset cipher. +#[tokio::test] +async fn a_zerokms_cipher_refuses_a_v2_leaf_before_any_key_call() { + let (cipher, _, retrieves) = cipher_over(CountingSource::new()).await; + let keyset = cipher.default_keyset(); + let vendor = KeyBackend::Vendor(KeyVendor::Aws); + + let result = cipher + .decrypt::( + CipherText::Single(v2_leaf(KeyVendor::Aws, keyset.keyset_id())), + AAD, + ) + .await; + assert_foreign(result, KeyBackend::ZeroKms, vendor); + + let result = keyset + .decrypt_tree::( + CipherText::Single(v2_leaf(KeyVendor::Aws, keyset.keyset_id())), + AAD, + ) + .await; + assert_foreign(result, KeyBackend::ZeroKms, vendor); + + assert_eq!(retrieves.load(Ordering::SeqCst), 0, "no key call"); +} + +/// One v2 leaf in a tree of v1 leaves refuses the whole tree, before any +/// retrieve: the v1 leaves are not retrieved first. +#[tokio::test] +async fn one_v2_leaf_refuses_a_whole_tree_before_any_key_call() { + let (cipher, _, retrieves) = cipher_over(CountingSource::new()).await; + let keyset = cipher.default_keyset(); + let v1 = keyset + .encrypt_tree("first".to_string(), AAD) + .await + .expect("encrypt"); + let tree: StackCipherText = CipherText::Sequence(vec![ + v1, + CipherText::Single(v2_leaf(KeyVendor::Vault, keyset.keyset_id())), + ]); + + let result = cipher.decrypt::, _>(tree, AAD).await; + assert!( + matches!( + result, + Err(Error::ForeignBackend { + found: KeyBackend::Vendor(KeyVendor::Vault), + .. + }) + ), + "{result:?}" + ); + assert_eq!(retrieves.load(Ordering::SeqCst), 0, "no key call"); +} + +/// A cipher over a key vendor backend refuses a v1 leaf (which names +/// ZeroKMS) and a v2 leaf of another vendor, before any key call. A v2 leaf +/// of its own vendor passes the routing check and reaches the key source. +#[tokio::test] +async fn a_vendor_cipher_routes_only_its_own_leaves_to_its_key_source() { + let own = KeyBackend::Vendor(KeyVendor::Static); + let (cipher, _, retrieves) = cipher_over(CountingSource::with_backend(own)).await; + let keyset = cipher.default_keyset().keyset_id(); + + let result = cipher + .decrypt::(CipherText::Single(v1_leaf(keyset)), AAD) + .await; + assert_foreign(result, own, KeyBackend::ZeroKms); + + let result = cipher + .decrypt::(CipherText::Single(v2_leaf(KeyVendor::Gcp, keyset)), AAD) + .await; + assert_foreign(result, own, KeyBackend::Vendor(KeyVendor::Gcp)); + assert_eq!(retrieves.load(Ordering::SeqCst), 0, "no key call"); + + // The fake holds no key for this made-up leaf, so the retrieve fails; + // what matters is that the leaf reached the key source. + let result = cipher + .decrypt::(CipherText::Single(v2_leaf(KeyVendor::Static, keyset)), AAD) + .await; + assert!(matches!(result, Err(Error::Kms(_))), "{result:?}"); + assert_eq!(retrieves.load(Ordering::SeqCst), 1, "one key call"); +} + +/// A cipher over a key vendor backend that names no key mode for the keyset +/// cannot say how to seal, so it refuses, before any key call, rather than +/// write a leaf that names the wrong backend or mode. +#[tokio::test] +async fn a_vendor_cipher_refuses_to_seal_before_any_key_call() { + let own = KeyBackend::Vendor(KeyVendor::Azure); + let (cipher, generates, _) = cipher_over(CountingSource::with_backend(own)).await; + + let result = cipher + .default_keyset() + .encrypt_tree("value".to_string(), AAD) + .await; + assert!( + matches!(result, Err(Error::SealUnsupported { backend }) if backend == own), + "{result:?}" + ); + assert_eq!(generates.load(Ordering::SeqCst), 0, "no key call"); +} + +/// A ZeroKMS cipher still seals v1 leaves, and they still open. +#[tokio::test] +async fn a_zerokms_cipher_still_writes_and_opens_v1() { + let (cipher, _, _) = cipher_over(CountingSource::new()).await; + let ct = cipher + .default_keyset() + .encrypt_tree("value".to_string(), AAD) + .await + .expect("encrypt"); + let CipherText::Single(leaf) = &ct else { + panic!("expected a Single leaf, got {ct:?}"); + }; + assert_eq!(leaf.format_version(), SealedValue::FORMAT_VERSION); + assert_eq!(leaf.key_backend(), KeyBackend::ZeroKms); + assert_eq!(leaf.key_mode(), None); + let pt: String = cipher.decrypt(ct, AAD).await.expect("decrypt"); + assert_eq!(pt, "value"); +} + +/// `Request::retrieve_leaf_key` carries the leaf's backend, so a +/// hand-built request for a v2 leaf is refused as a tree's would be. +#[tokio::test] +async fn a_hand_built_request_for_a_v2_leaf_is_refused() { + use stack_encrypt::target::{Pending, Request}; + + let (cipher, _, retrieves) = cipher_over(CountingSource::new()).await; + let keyset = cipher.default_keyset(); + let leaf = v2_leaf(KeyVendor::Aws, keyset.keyset_id()); + let pending: Pending<'_, (), _> = Pending::request( + &keyset, + vec![Request::retrieve_leaf_key(&leaf, nonempty!("ctx"))], + |_| Ok(()), + ); + let result = pending.await; + assert!( + matches!(result, Err(Error::ForeignBackend { .. })), + "{result:?}" + ); + assert_eq!(retrieves.load(Ordering::SeqCst), 0, "no key call"); +} + +/// A source whose keysets are selected per settle: the cipher holds no +/// index key, and asks for it inside each settle that builds a term. The +/// key is the fake source's, so its terms must equal a ZeroKMS-shaped +/// cipher's over the same key, byte for byte. +struct PerSettle { + inner: stack_encrypt::kms::FakeDataKeySource, + loads: std::sync::Arc, +} + +impl stack_encrypt::kms::DataKeySource for PerSettle { + async fn generate_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option>, + ) -> Result, stack_encrypt::kms::Error> { + self.inner + .generate_keys(payloads, keyset_id, unverified_context) + .await + } + + async fn retrieve_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option<&stack_encrypt::kms::UnverifiedContext>, + ) -> Result, stack_encrypt::kms::Error> { + self.inner + .retrieve_keys(payloads, keyset_id, unverified_context) + .await + } + + async fn load_settle_index_key( + &self, + keyset_id: Uuid, + ) -> Result { + use stack_encrypt::kms::IndexKeySource; + let _ = self.loads.fetch_add(1, Ordering::SeqCst); + let (_, key) = self + .inner + .load_index_key(Some(stack_encrypt::kms::IdentifiedBy::Uuid(keyset_id))) + .await?; + Ok(key) + } +} + +impl stack_encrypt::kms::IndexKeySource for PerSettle { + async fn load_index_key( + &self, + keyset_id: Option, + ) -> Result<(Uuid, stack_encrypt::kms::IndexKey), stack_encrypt::kms::Error> { + self.inner.load_index_key(keyset_id).await + } + + async fn select_keyset( + &self, + keyset_id: Option, + ) -> Result { + let (keyset_id, _) = self.inner.load_index_key(keyset_id).await?; + Ok(stack_encrypt::kms::SelectedKeyset::PerSettle { keyset_id }) + } +} + +#[tokio::test] +async fn a_term_derived_inside_the_settle_equals_one_derived_now() { + use stack_encrypt::sem::{DefaultMatch, EqualityTerm, MatchTerms, OreTerm}; + use stack_encrypt::target::legacy::EncryptInto; + + let loads = std::sync::Arc::new(std::sync::atomic::AtomicUsize::new(0)); + let deferred = StackCipher::builder() + .kms(PerSettle { + inner: stack_encrypt::kms::FakeDataKeySource::new(), + loads: std::sync::Arc::clone(&loads), + }) + .init() + .await + .expect("a per-settle default keyset is accepted"); + let eager = StackCipher::builder() + .kms(stack_encrypt::kms::FakeDataKeySource::new()) + .init() + .await + .unwrap(); + let (later, now) = (deferred.default_keyset(), eager.default_keyset()); + assert_eq!(later.keyset_id(), now.keyset_id()); + assert!(later.prf().is_none(), "the handle holds no index key"); + assert!(now.prf().is_some()); + assert_eq!(loads.load(Ordering::SeqCst), 0, "selection loads no key"); + + let ctx = nonempty!("users").with("email"); + let borrowed = String::from("alice"); + let a = later.equality_term(borrowed.as_str(), ctx).await.unwrap(); + let b = now.equality_term(borrowed.as_str(), ctx).await.unwrap(); + assert_eq!(a.as_bytes(), b.as_bytes()); + assert_eq!(loads.load(Ordering::SeqCst), 1, "one load for the settle"); + + let a = later + .match_terms::("alice wonderland", ctx) + .await + .unwrap(); + let b = now + .match_terms::("alice wonderland", ctx) + .await + .unwrap(); + assert_eq!(a.positions(), b.positions()); + assert_eq!( + later.ore_term(42u64, ctx).await.unwrap(), + now.ore_term(42u64, ctx).await.unwrap() + ); + assert_eq!( + later.ope_term(42u64, ctx).await.unwrap(), + now.ope_term(42u64, ctx).await.unwrap() + ); + + // The target-directed path, and several terms in one settle: one load. + let before = loads.load(Ordering::SeqCst); + let eq = "bob".encrypt_into_with_context::(&later, ctx); + let mt = "bob builder" + .to_string() + .encrypt_into_with_context::, _>(&later, ctx); + let ore = 7u64.encrypt_into_with_context::, _>(&later, ctx); + let pair = eq.zip(mt).zip(ore); + let ((eq, mt), ore) = pair.await.unwrap(); + assert_eq!( + loads.load(Ordering::SeqCst), + before + 1, + "one load per settle" + ); + assert_eq!( + eq.as_bytes(), + now.equality_term("bob", ctx).await.unwrap().as_bytes() + ); + assert_eq!( + mt.positions(), + now.match_terms::("bob builder", ctx) + .await + .unwrap() + .positions() + ); + assert_eq!(ore.into_inner(), now.ore_term(7u64, ctx).await.unwrap()); + + // A value still seals and opens through the per-settle keyset. + let sealed = later.encrypt_tree("v".to_string(), AAD).await.unwrap(); + let opened: String = deferred.decrypt(sealed, AAD).await.unwrap(); + assert_eq!(opened, "v"); +} + +/// A key vendor backend's keyset seals v2 leaves that name the backend and +/// its key mode, and a retrieve passes that mode to the key source. +#[tokio::test] +async fn a_vendor_keyset_seals_v2_and_passes_its_mode_on_retrieve() { + for mode in KeyMode::ALL { + let own = KeyBackend::Vendor(KeyVendor::Vault); + let source = CountingSource::with_backend(own).with_mode(mode); + let modes = source.retrieved_modes(); + let (cipher, _, _) = cipher_over(source).await; + let ct = cipher + .default_keyset() + .encrypt_tree(vec!["a".to_string(), "b".to_string()], AAD) + .await + .expect("encrypt"); + let CipherText::Sequence(items) = &ct else { + panic!("expected a sequence, got {ct:?}"); + }; + for item in items { + let CipherText::Single(leaf) = item else { + panic!("expected a leaf"); + }; + assert_eq!(leaf.format_version(), SealedValue::VENDOR_FORMAT_VERSION); + assert_eq!(leaf.key_backend(), own); + assert_eq!(leaf.key_mode(), Some(mode)); + } + let pt: Vec = cipher.decrypt(ct, AAD).await.expect("decrypt"); + assert_eq!(pt, ["a", "b"]); + assert_eq!(*modes.lock().unwrap(), [Some(mode), Some(mode)]); + } +} + +/// A source that answers per keyset: a leaf is refused when it names +/// another backend than the one holding its keyset, whatever the source's +/// own backend is. +#[tokio::test] +async fn the_backend_that_holds_the_keyset_decides() { + struct Split { + inner: CountingSource, + vendor_keyset: Uuid, + } + impl stack_encrypt::kms::DataKeySource for Split { + async fn generate_keys( + &self, + payloads: Vec>, + keyset_id: Option, + uc: Option>, + ) -> Result, stack_encrypt::kms::Error> { + self.inner.generate_keys(payloads, keyset_id, uc).await + } + async fn retrieve_keys( + &self, + payloads: Vec>, + keyset_id: Option, + uc: Option<&stack_encrypt::kms::UnverifiedContext>, + ) -> Result, stack_encrypt::kms::Error> { + self.inner.retrieve_keys(payloads, keyset_id, uc).await + } + fn keyset_backend(&self, keyset_id: Uuid) -> KeyBackend { + if keyset_id == self.vendor_keyset { + KeyBackend::Vendor(KeyVendor::Aws) + } else { + KeyBackend::ZeroKms + } + } + fn key_mode(&self, keyset_id: Uuid) -> Option { + (keyset_id == self.vendor_keyset).then_some(KeyMode::Derived) + } + } + impl stack_encrypt::kms::IndexKeySource for Split { + async fn load_index_key( + &self, + keyset_id: Option, + ) -> Result<(Uuid, stack_encrypt::kms::IndexKey), stack_encrypt::kms::Error> { + self.inner.load_index_key(keyset_id).await + } + } + + let vendor_keyset = Uuid::from_u128(77); + let cipher = StackCipher::builder() + .kms(Split { + inner: CountingSource::new(), + vendor_keyset, + }) + .init() + .await + .expect("cipher"); + let zerokms = cipher.default_keyset(); + let vendor = cipher.keyset(vendor_keyset).await.expect("vendor keyset"); + + // Each keyset seals in its own backend's format, and both open. + let v1 = zerokms.encrypt_tree("z".to_string(), AAD).await.unwrap(); + let v2 = vendor.encrypt_tree("v".to_string(), AAD).await.unwrap(); + let (CipherText::Single(l1), CipherText::Single(l2)) = (&v1, &v2) else { + panic!("leaves"); + }; + assert_eq!(l1.key_backend(), KeyBackend::ZeroKms); + assert_eq!(l2.key_backend(), KeyBackend::Vendor(KeyVendor::Aws)); + assert_eq!(l2.key_mode(), Some(KeyMode::Derived)); + let opened: String = cipher.decrypt(v2, AAD).await.unwrap(); + assert_eq!(opened, "v"); + let opened: String = cipher.decrypt(v1, AAD).await.unwrap(); + assert_eq!(opened, "z"); + + // A ZeroKMS-shaped leaf for the vendor keyset is refused. + let result = cipher + .decrypt::(CipherText::Single(v1_leaf(vendor_keyset)), AAD) + .await; + assert_foreign( + result, + KeyBackend::Vendor(KeyVendor::Aws), + KeyBackend::ZeroKms, + ); + // A vendor leaf for the ZeroKMS keyset is refused. + let result = cipher + .decrypt::( + CipherText::Single(v2_leaf(KeyVendor::Aws, zerokms.keyset_id())), + AAD, + ) + .await; + assert_foreign( + result, + KeyBackend::ZeroKms, + KeyBackend::Vendor(KeyVendor::Aws), + ); +} diff --git a/packages/stack-guest-abi/Cargo.toml b/packages/stack-guest-abi/Cargo.toml index d125c2d489..ff4883b7ef 100644 --- a/packages/stack-guest-abi/Cargo.toml +++ b/packages/stack-guest-abi/Cargo.toml @@ -13,7 +13,15 @@ license-file = "LICENSE" # own; also keeps release-plz from picking it up. publish = false +[features] +# `vendor_transport`: the guest's `VendorTransport` for key vendor backends +# (ADR-0011), over the same `transport_send` import. Off by default, so the +# credential guest does not carry the vendor seam. +vendor = ["dep:stack-kms-vendor"] + [dependencies] +# Only `vendor_transport`. No default features: the guest has no native HTTP. +stack-kms-vendor = { path = "../stack-kms-vendor", default-features = false, optional = true } zeroize = { workspace = true } # `last_error`: an error is any miette diagnostic plus its structured fields # (a serde_json map, as `ErrorPayload::payload` gives them), encoded as a diff --git a/packages/stack-guest-abi/src/lib.rs b/packages/stack-guest-abi/src/lib.rs index 37a246a52e..b3e08849dc 100644 --- a/packages/stack-guest-abi/src/lib.rs +++ b/packages/stack-guest-abi/src/lib.rs @@ -111,3 +111,5 @@ pub mod value; pub mod abi; #[cfg(target_arch = "wasm32")] pub mod transport; +#[cfg(feature = "vendor")] +pub mod vendor_transport; diff --git a/packages/stack-guest-abi/src/status.rs b/packages/stack-guest-abi/src/status.rs index 7e32fdc4d2..974314ba3d 100644 --- a/packages/stack-guest-abi/src/status.rs +++ b/packages/stack-guest-abi/src/status.rs @@ -128,10 +128,16 @@ pub const STATUS_CONTEXT_MISMATCH: u32 = 28; /// them, and resumes it with the responses. No error is recorded. pub const STATUS_NEEDS_IO: u32 = 29; +/// A client with a key vendor backend and no ZeroKMS was asked for a +/// ZeroKMS keyset: a keyset none of its keyset records names, or a value a +/// ZeroKMS keyset sealed. Refused before any request. A configuration +/// error: the fix is ZeroKMS credentials, or a keyset record. +pub const STATUS_NO_ZEROKMS: u32 = 30; + /// The last code in the table. A guest appending a code of its own starts /// at `LAST_STATUS + 1` and moves this constant with it, so two guests can /// never claim one number. -pub const LAST_STATUS: u32 = STATUS_NEEDS_IO; +pub const LAST_STATUS: u32 = STATUS_NO_ZEROKMS; #[cfg(test)] mod tests { @@ -171,6 +177,7 @@ mod tests { STATUS_AUTH_REFRESH_REQUIRED, STATUS_CONTEXT_MISMATCH, STATUS_NEEDS_IO, + STATUS_NO_ZEROKMS, ]; for (i, code) in codes.iter().enumerate() { assert_eq!(*code, i as u32 + 1, "code {i} is out of sequence"); diff --git a/packages/stack-guest-abi/src/step.rs b/packages/stack-guest-abi/src/step.rs index e99047dbc4..a3fdb2ef6c 100644 --- a/packages/stack-guest-abi/src/step.rs +++ b/packages/stack-guest-abi/src/step.rs @@ -15,6 +15,12 @@ //! cancellation never interrupts it: an aborted operation is a future //! dropped between two polls, and everything it held is wiped on drop. //! +//! A request is one of three things the host does for the guest +//! ([`Request`]): an HTTP request, a key vendor backend's short-lived +//! credential, or a wait (a retry's backoff). The guest blocks on none of +//! them, so the host can fetch a credential over the network, or sit out a +//! backoff, with the instance free for other calls. +//! //! # Handles //! //! - A handle names one paused operation. It is issued when an operation @@ -50,6 +56,7 @@ use std::future::Future; use std::pin::Pin; use std::rc::Rc; use std::task::{Context, Poll, Waker}; +use std::time::Duration; use vitaminc_protected::OpaqueDebug; use zeroize::Zeroizing; @@ -64,41 +71,105 @@ pub type Operation = Pin, u32>>>>; /// The most operations that may be paused at once. pub const MAX_OPEN: usize = 64; -/// One HTTP request an operation needs the host to perform. +/// One thing an operation needs the host to do before it can go on: an +/// HTTP request to perform, a key vendor backend's credential to fetch, or +/// a wait to sit out. The host answers each with a [`Response`]. /// -/// The headers (in the [`crate::headers`] line format, a bearer token -/// among them) and the body (which can carry key-material contexts) are -/// wiped on drop. -#[derive(OpaqueDebug)] +/// All three pause the operation the same way, so the guest never blocks +/// on the host: the host fetches a credential and waits out a backoff +/// between steps, as it performs a request, with the instance free for +/// other calls. #[non_exhaustive] -pub struct Request { - /// The HTTP method. - #[non_sensitive] - pub method: &'static str, - /// The URL. - #[non_sensitive] - pub url: String, - /// The request headers. - pub headers: Zeroizing>, - /// The request body. - pub body: Zeroizing>, +pub enum Request { + /// An HTTP request. Answered with its HTTP response. + Http(HttpRequest), + /// A short-lived credential of the key vendor backend `vendor` (its + /// `KeyVendor` byte), for the OAuth `scope` (empty when the vendor has + /// none). Answered with status `0` and the credential, as a frame, in + /// the body; or a negative status and the error text, when the host has + /// none. + Credential { + /// The `KeyVendor` byte of the backend. + vendor: u8, + /// The OAuth scope, or empty. + scope: String, + }, + /// A wait of `duration`: the backoff before a retry. Answered with + /// status `0` and an empty body once it has passed. + Wait(Duration), } impl Request { - /// A request. + /// An HTTP request. pub fn new( method: &'static str, url: String, headers: Zeroizing>, body: Zeroizing>, ) -> Self { - Self { + Self::Http(HttpRequest { method, url, headers, body, + }) + } + + /// A request for a key vendor backend's credential. + pub fn credential(vendor: u8, scope: impl Into) -> Self { + Self::Credential { + vendor, + scope: scope.into(), } } + + /// A request to wait `duration`. + pub fn wait(duration: Duration) -> Self { + Self::Wait(duration) + } + + /// The HTTP request, when this is one. + pub fn http(&self) -> Option<&HttpRequest> { + match self { + Self::Http(request) => Some(request), + _ => None, + } + } +} + +/// `Debug` never shows an HTTP request's headers or body. +impl std::fmt::Debug for Request { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Http(request) => request.fmt(f), + Self::Credential { vendor, scope } => f + .debug_struct("Credential") + .field("vendor", vendor) + .field("scope", scope) + .finish(), + Self::Wait(duration) => f.debug_tuple("Wait").field(duration).finish(), + } + } +} + +/// One HTTP request an operation needs the host to perform. +/// +/// The headers (in the [`crate::headers`] line format, a bearer token +/// among them) and the body (which can carry key-material contexts) are +/// wiped on drop. +#[derive(OpaqueDebug)] +#[non_exhaustive] +pub struct HttpRequest { + /// The HTTP method. + #[non_sensitive] + pub method: &'static str, + /// The URL. + #[non_sensitive] + pub url: String, + /// The request headers. + pub headers: Zeroizing>, + /// The request body. + pub body: Zeroizing>, } /// The host's answer to one [`Request`]. @@ -498,10 +569,10 @@ mod tests { let _reset = reset(); let (handle, requests) = needs_io(start(sequential(2))); assert_eq!(requests.len(), 1); - assert_eq!(requests[0].url, "https://kms/0"); + assert_eq!(requests[0].http().unwrap().url, "https://kms/0"); let (again, requests) = needs_io(resume(handle, vec![ok(b"a")])); assert_eq!(again, handle, "a paused operation keeps its handle"); - assert_eq!(requests[0].url, "https://kms/1"); + assert_eq!(requests[0].http().unwrap().url, "https://kms/1"); let done = resume(handle, vec![ok(b"b")]).unwrap(); assert!(matches!(done, Step::Done(out) if out == b"ab")); assert_eq!(open_count(), 0); @@ -549,7 +620,10 @@ mod tests { Ok(out) }); let (handle, requests) = needs_io(start(work)); - let urls: Vec<_> = requests.iter().map(|r| r.url.as_str()).collect(); + let urls: Vec<_> = requests + .iter() + .map(|r| r.http().unwrap().url.as_str()) + .collect(); assert_eq!(urls, ["https://kms/a", "https://kms/b"]); let done = resume(handle, vec![ok(b"1"), ok(b"2")]).unwrap(); assert!(matches!(done, Step::Done(out) if out == b"12")); diff --git a/packages/stack-guest-abi/src/vendor_transport.rs b/packages/stack-guest-abi/src/vendor_transport.rs new file mode 100644 index 0000000000..74814a4baa --- /dev/null +++ b/packages/stack-guest-abi/src/vendor_transport.rs @@ -0,0 +1,209 @@ +//! The guest's [`VendorTransport`]: a key vendor backend's HTTP requests, +//! and its backoff waits, handed to the host through the step loop +//! (ADR-0011, [`crate::step`]). +//! +//! The vendor client signs each request inside the guest and the host only +//! carries the bytes, as for ZeroKMS. The conversions between the step's +//! form (a status that is negative for a transport failure, headers in the +//! [`crate::headers`] line format) and [`VendorResponse`] are pure, and are +//! tested on the native target. [`StepVendorTransport`] itself pauses the +//! operation, whose future is not `Send`, and so exists on wasm32 only. +//! +//! Only the transport is here. Which vendor the guest builds, and where its +//! credentials come from, is the guest's own wiring. + +use stack_kms_vendor::{TransportError, VendorResponse}; +use zeroize::Zeroizing; + +/// Encode request headers in the step's line format. +/// +/// A header name or value with a line break cannot be carried: it would +/// split into two lines (or inject a header). It is refused. +pub fn encode_request_headers( + headers: &[(String, String)], +) -> Result>, TransportError> { + let has_break = |text: &str| text.contains(['\n', '\r']); + if let Some((name, _)) = headers + .iter() + .find(|(name, value)| has_break(name) || has_break(value) || name.contains(':')) + { + return Err(TransportError(format!( + "the header {name:?} cannot be sent: a name or value has a line break or a name has a colon" + ))); + } + let pairs: Vec<(&str, &str)> = headers + .iter() + .map(|(name, value)| (name.as_str(), value.as_str())) + .collect(); + // The headers can hold a credential (a session token): wipe on drop. + Ok(Zeroizing::new(crate::headers::encode_headers(&pairs))) +} + +/// Decode response headers from the step's line format. Malformed lines +/// are skipped, as [`crate::headers::header_value`] does. +pub fn decode_response_headers(buffer: &[u8]) -> Vec<(String, String)> { + buffer + .split(|&byte| byte == b'\n') + .filter_map(|line| { + let line = std::str::from_utf8(line).ok()?; + let (name, value) = line.split_once(':')?; + Some((name.trim().to_owned(), value.trim().to_owned())) + }) + .collect() +} + +/// Turn the host's response to a step request into a [`VendorResponse`]. +/// +/// A negative status is a transport failure, and the body is then the +/// host's error text. A status above 999 cannot be an HTTP status and is a +/// transport failure too. +pub fn response_from_host( + status: i32, + headers: &[u8], + body: Zeroizing>, +) -> Result { + if status < 0 { + return Err(TransportError(format!( + "the host could not send the request: {}", + String::from_utf8_lossy(&body) + ))); + } + let status = u16::try_from(status) + .ok() + .filter(|status| (100..=999).contains(status)) + .ok_or_else(|| TransportError(format!("the host returned the status {status}")))?; + Ok(VendorResponse { + status, + headers: decode_response_headers(headers), + body, + }) +} + +/// The method of `request` as the step loop carries it: one of the HTTP +/// methods a vendor client sends. Any other is refused, not sent. +pub fn step_method(method: &str) -> Result<&'static str, TransportError> { + const METHODS: [&str; 6] = ["GET", "POST", "PUT", "PATCH", "DELETE", "HEAD"]; + METHODS + .into_iter() + .find(|known| *known == method) + .ok_or_else(|| TransportError(format!("the HTTP method {method:?} cannot be sent"))) +} + +#[cfg(target_arch = "wasm32")] +pub use step_transport::StepVendorTransport; + +#[cfg(target_arch = "wasm32")] +mod step_transport { + use std::time::Duration; + + use stack_kms_vendor::{TransportError, VendorRequest, VendorResponse, VendorTransport}; + + use super::{encode_request_headers, response_from_host, step_method}; + use crate::step; + + /// A [`VendorTransport`] whose HTTP client is the host's, through the + /// step loop: `send` pauses the operation with the request, and `sleep` + /// pauses it with a wait, so the guest blocks on neither and the host + /// sits out a backoff with the instance free for other calls. + #[derive(Clone, Copy, Debug, Default)] + pub struct StepVendorTransport; + + impl VendorTransport for StepVendorTransport { + async fn send(&self, request: VendorRequest) -> Result { + let method = step_method(&request.method)?; + let headers = encode_request_headers(&request.headers)?; + let VendorRequest { url, body, .. } = request; + let response = step::request(step::Request::new(method, url, headers, body)) + .await + .map_err(|error| TransportError(error.to_string()))?; + response_from_host(response.status, &response.headers, response.body) + } + + async fn sleep(&self, duration: Duration) { + // The host answers a wait once it has passed. An operation that + // is not in a step (never the case for a vendor call) or a host + // that could not wait goes on at once: the retry still happens, + // only sooner. + let _ = step::request(step::Request::wait(duration)).await; + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn request_headers_encode_in_the_line_format() { + let headers = vec![ + ("host".to_owned(), "kms.us-east-1.amazonaws.com".to_owned()), + ("x-amz-target".to_owned(), "TrentService.Decrypt".to_owned()), + ]; + assert_eq!( + encode_request_headers(&headers).unwrap().as_slice(), + b"host: kms.us-east-1.amazonaws.com\nx-amz-target: TrentService.Decrypt" + ); + } + + #[test] + fn a_header_with_a_line_break_or_a_colon_in_its_name_is_refused() { + for (name, value) in [ + ("x-a", "1\nx-b: 2"), + ("x-a", "1\r"), + ("x-a\n", "1"), + ("x:a", "1"), + ] { + let headers = vec![(name.to_owned(), value.to_owned())]; + assert!(encode_request_headers(&headers).is_err(), "{name:?}"); + } + let headers = vec![("x-a".to_owned(), "a:b".to_owned())]; + assert!(encode_request_headers(&headers).is_ok()); + } + + #[test] + fn response_headers_decode_and_skip_malformed_lines() { + assert_eq!( + decode_response_headers(b"Content-Type: application/json\nbad line\nx-a: b:c "), + vec![ + ("Content-Type".to_owned(), "application/json".to_owned()), + ("x-a".to_owned(), "b:c".to_owned()), + ] + ); + assert!(decode_response_headers(b"").is_empty()); + assert!(decode_response_headers(&[0xff, b':', b'a']).is_empty()); + } + + #[test] + fn a_host_status_becomes_a_response_or_a_transport_error() { + let response = response_from_host( + 400, + b"x-amzn-errortype: ThrottlingException", + Zeroizing::new(b"{}".to_vec()), + ) + .unwrap(); + assert_eq!(response.status, 400); + assert_eq!( + response.header_value("X-Amzn-ErrorType"), + Some("ThrottlingException") + ); + assert_eq!(response.body.as_slice(), b"{}"); + + let error = + response_from_host(-1, b"", Zeroizing::new(b"dial tcp: refused".to_vec())).unwrap_err(); + assert!(error.0.contains("dial tcp: refused")); + assert!(response_from_host(99, b"", Zeroizing::new(vec![])).is_err()); + assert!(response_from_host(1000, b"", Zeroizing::new(vec![])).is_err()); + assert!(response_from_host(100, b"", Zeroizing::new(vec![])).is_ok()); + assert!(response_from_host(999, b"", Zeroizing::new(vec![])).is_ok()); + } + + #[test] + fn only_the_http_methods_a_vendor_client_sends_are_carried() { + for method in ["GET", "POST", "PUT", "PATCH", "DELETE", "HEAD"] { + assert_eq!(step_method(method).unwrap(), method); + } + for method in ["post", "CONNECT", "", "POST\n"] { + assert!(step_method(method).is_err(), "{method:?}"); + } + } +} diff --git a/packages/stack-kms-aws/Cargo.toml b/packages/stack-kms-aws/Cargo.toml new file mode 100644 index 0000000000..540cd4d0e1 --- /dev/null +++ b/packages/stack-kms-aws/Cargo.toml @@ -0,0 +1,47 @@ +[package] +name = "stack-kms-aws" +description = "The AWS KMS key vendor backend for Stack Encrypt: a hand-written AWS KMS client with SigV4 signing, for native Rust and the WASI guest" +version = "0.1.0" +edition.workspace = true +authors.workspace = true +repository.workspace = true +homepage.workspace = true +keywords.workspace = true +categories.workspace = true +license-file = "LICENSE" +# Publishable, in the `stack-kms-vendor` version group with the vendor seam it +# implements. release-plz.toml keeps `release = false` until the first version +# is on crates.io (published by hand; see stack-kms-vendor's Cargo.toml). + +[features] +default = ["http"] +# `AwsKms::from_env`: the native transport (`HttpVendorTransport`) and the +# default credential chain over it. Native only; on wasm32 it adds nothing. +http = ["stack-kms-vendor/http"] + +[dependencies] +stack-kms = { path = "../stack-kms", version = "0.2.0", default-features = false } +stack-kms-vendor = { path = "../stack-kms-vendor", version = "0.1.0" } + +vitaminc = { workspace = true, features = ["protected"] } +base64ct = { version = "1.7", features = ["alloc"] } +futures = "0.3.25" +hmac = "0.13" +sha2 = "0.11" +serde = { workspace = true } +serde_json = { workspace = true } +thiserror = { workspace = true } +url = { workspace = true } +web-time = { workspace = true } +zeroize = { workspace = true } + +[dev-dependencies] +stack-kms-vendor = { path = "../stack-kms-vendor", features = ["test-support", "http"] } +stack-encrypt = { path = "../stack-encrypt", default-features = false } +aes-gcm = { version = "0.10.3", features = ["zeroize"] } +tokio = { workspace = true, features = ["rt", "macros", "time"] } +uuid = { workspace = true } +vitaminc = { workspace = true, features = ["protected", "random"] } + +[package.metadata.docs.rs] +all-features = true diff --git a/packages/stack-kms-aws/LICENSE b/packages/stack-kms-aws/LICENSE new file mode 100644 index 0000000000..2cbd67a66d --- /dev/null +++ b/packages/stack-kms-aws/LICENSE @@ -0,0 +1,96 @@ +# PolyForm Internal Use License 1.0.0 + + + +## Acceptance + +In order to get any license under these terms, you must agree +to them as both strict obligations and conditions to all +your licenses. + +## Copyright License + +The licensor grants you a copyright license for the software +to do everything you might do with the software that would +otherwise infringe the licensor's copyright in it for any +permitted purpose. However, you may only make changes or +new works based on the software according to [Changes and New +Works License](#changes-and-new-works-license), and you may +not distribute the software. + +## Changes and New Works License + +The licensor grants you an additional copyright license to +make changes and new works based on the software for any +permitted purpose. + +## Patent License + +The licensor grants you a patent license for the software that +covers patent claims the licensor can license, or becomes able +to license, that you would infringe by using the software. + +## Fair Use + +You may have "fair use" rights for the software under the +law. These terms do not limit them. + +## Internal Business Use + +Use of the software for the internal business operations of +you and your company is use for a permitted purpose. + +## No Other Rights + +These terms do not allow you to sublicense or transfer any of +your licenses to anyone else, or prevent the licensor from +granting licenses to anyone else. These terms do not imply +any other licenses. + +## Patent Defense + +If you make any written claim that the software infringes or +contributes to infringement of any patent, your patent license +for the software granted under these terms ends immediately. If +your company makes such a claim, your patent license ends +immediately for work on behalf of your company. + +## Violations + +The first time you are notified in writing that you have +violated any of these terms, or done anything with the software +not covered by your licenses, your licenses can nonetheless +continue if you come into full compliance with these terms, +and take practical steps to correct past violations, within +32 days of receiving notice. Otherwise, all your licenses +end immediately. + +## No Liability + +***As far as the law allows, the software comes as is, without +any warranty or condition, and the licensor will not be liable +to you for any damages arising out of these terms or the use +or nature of the software, under any kind of legal claim.*** + +## Definitions + +The **licensor** is the individual or entity offering these +terms, and the **software** is the software the licensor makes +available under these terms. + +**You** refers to the individual or entity agreeing to these +terms. + +**Your company** is any legal entity, sole proprietorship, +or other kind of organization that you work for, plus all +organizations that have control over, are under the control of, +or are under common control with that organization. **Control** +means ownership of substantially all the assets of an entity, +or the power to direct its management and policies by vote, +contract, or otherwise. Control can be direct or indirect. + +**Your licenses** are all the licenses granted to you for the +software under these terms. + +**Use** means anything you do with the software requiring one +of your licenses. diff --git a/packages/stack-kms-aws/src/client.rs b/packages/stack-kms-aws/src/client.rs new file mode 100644 index 0000000000..9060e4178a --- /dev/null +++ b/packages/stack-kms-aws/src/client.rs @@ -0,0 +1,543 @@ +//! The AWS KMS client and its [`KeyVendorBackend`] implementation. + +use std::collections::BTreeMap; +use std::time::Duration; + +use base64ct::{Base64, Base64UrlUnpadded, Encoding}; +use serde::de::DeserializeOwned; +use serde::{Deserialize, Serialize}; +use stack_kms::{BackendErrorKind, Error, KeyVendor}; +use stack_kms_vendor::derive::describe_aad; +use stack_kms_vendor::{ + classify_status, KeyBytes, KeyVendorBackend, VendorRequest, VendorResponse, VendorTransport, +}; +use vitaminc::protected::Protected; +use zeroize::Zeroizing; + +use crate::config::AwsKmsConfig; +use crate::credentials::{CredentialsError, CredentialsProvider}; +use crate::sigv4::{sign, SigningParams}; +use crate::time::{system_clock, Clock}; + +/// The vendor name in errors. +const VENDOR: &str = "aws-kms"; +/// The AWS KMS JSON protocol content type. +const CONTENT_TYPE: &str = "application/x-amz-json-1.1"; +/// The SigV4 signing name of AWS KMS. +const SERVICE: &str = "kms"; + +/// The encryption context key that binds the whole associated data. +pub const CONTEXT_AAD: &str = "cipherstash:aad"; +/// The encryption context key of the keyset id. +pub const CONTEXT_KEYSET_ID: &str = "cipherstash:keyset-id"; +/// The encryption context key of the value's descriptor (wrapped data keys +/// only). +pub const CONTEXT_DESCRIPTOR: &str = "cipherstash:descriptor"; +/// The encryption context key of what the key is (`root-key`, `index-key` +/// or `data-key`). +pub const CONTEXT_PURPOSE: &str = "cipherstash:purpose"; + +/// A failure of an AWS KMS call. It is the `source` of the +/// [`Error::Backend`] the client returns; the kind is on the +/// `Error::Backend`. +#[derive(Debug, thiserror::Error)] +pub enum AwsKmsError { + /// AWS KMS answered with an error. + #[error("AWS KMS {operation} failed with HTTP {status}: {code}: {message}")] + Service { + /// The operation (`Decrypt`). + operation: &'static str, + /// The HTTP status. + status: u16, + /// The AWS error code (`ThrottlingException`), or `unknown`. + code: String, + /// The AWS error message. + message: String, + }, + /// The request got no response. + #[error("AWS KMS {operation} got no response: {source}")] + Transport { + /// The operation. + operation: &'static str, + /// The transport failure. + source: stack_kms_vendor::TransportError, + }, + /// The credentials could not be got. + #[error(transparent)] + Credentials(#[from] CredentialsError), + /// The request could not be signed. + #[error(transparent)] + Signing(#[from] crate::sigv4::SigningError), + /// AWS KMS answered with a success that this client cannot read. + #[error("AWS KMS {operation} returned a response this client cannot read: {reason}")] + InvalidResponse { + /// The operation. + operation: &'static str, + /// What is wrong with it. + reason: &'static str, + }, +} + +fn backend(kind: BackendErrorKind, error: AwsKmsError) -> Error { + Error::Backend { + vendor: VENDOR, + kind, + source: Box::new(error), + } +} + +/// The failure kind of an AWS KMS error code. `None` for a code with no +/// fixed kind: the HTTP status decides. +pub fn error_kind(code: &str) -> Option { + Some(match code { + "ThrottlingException" => BackendErrorKind::Throttled, + "AccessDeniedException" + | "UnrecognizedClientException" + | "InvalidSignatureException" + | "IncompleteSignature" + | "ExpiredTokenException" + | "MissingAuthenticationToken" + | "InvalidClientTokenId" => BackendErrorKind::AccessDenied, + "NotFoundException" => BackendErrorKind::NotFound, + "InvalidCiphertextException" | "IncorrectKeyException" | "InvalidKeyUsageException" => { + BackendErrorKind::InvalidCiphertext + } + "KMSInternalException" + | "DependencyTimeoutException" + | "KeyUnavailableException" + | "DisabledException" + | "KMSInvalidStateException" + | "ServiceUnavailable" + | "InternalFailure" => BackendErrorKind::Unavailable, + _ => return None, + }) +} + +#[derive(Deserialize)] +struct ErrorBody { + #[serde(rename = "__type")] + kind: Option, + #[serde(alias = "Message")] + message: Option, +} + +/// The error of a failed AWS KMS response. +fn service_error(operation: &'static str, response: &VendorResponse) -> Error { + let body: Option = serde_json::from_slice(&response.body).ok(); + // `__type` may carry a namespace (`com.amazonaws.kms#NotFoundException`); + // the `x-amzn-ErrorType` header may carry a suffix after a colon. + let code = body + .as_ref() + .and_then(|body| body.kind.as_deref()) + .or_else(|| response.header_value("x-amzn-errortype")) + .map(|code| { + let code = code.rsplit('#').next().unwrap_or(code); + code.split(':').next().unwrap_or(code).to_owned() + }) + .unwrap_or_else(|| "unknown".to_owned()); + let message = body.and_then(|body| body.message).unwrap_or_default(); + let kind = error_kind(&code) + .or_else(|| classify_status(response.status)) + .unwrap_or(BackendErrorKind::Unavailable); + // A 4xx with an unknown code is a request AWS KMS refused, not a + // damaged ciphertext, unless the code says so. + let kind = match (kind, error_kind(&code)) { + (BackendErrorKind::InvalidCiphertext, None) => BackendErrorKind::AccessDenied, + (kind, _) => kind, + }; + backend( + kind, + AwsKmsError::Service { + operation, + status: response.status, + code, + message, + }, + ) +} + +/// The encryption context that binds `aad`: the whole associated data, and +/// (when it is the associated data of a keyset record key or a wrapped +/// data key) the readable keyset id, descriptor and purpose, for CloudTrail +/// and for `kms:EncryptionContext:` key policy conditions. +pub fn encryption_context(aad: &[u8]) -> BTreeMap<&'static str, String> { + let mut context = BTreeMap::new(); + let _ = context.insert(CONTEXT_AAD, Base64UrlUnpadded::encode_string(aad)); + if let Some(description) = describe_aad(aad) { + let _ = context.insert(CONTEXT_PURPOSE, description.purpose.to_owned()); + let _ = context.insert(CONTEXT_KEYSET_ID, description.keyset_id.to_string()); + if let Some(descriptor) = description.descriptor.filter(|d| !d.is_empty()) { + let _ = context.insert(CONTEXT_DESCRIPTOR, descriptor.to_owned()); + } + } + context +} + +#[derive(Serialize)] +#[serde(rename_all = "PascalCase")] +struct GenerateDataKeyRequest<'a> { + key_id: &'a str, + key_spec: &'static str, + encryption_context: &'a BTreeMap<&'static str, String>, +} + +#[derive(Serialize)] +#[serde(rename_all = "PascalCase")] +struct EncryptRequest<'a> { + key_id: &'a str, + plaintext: &'a str, + encryption_context: &'a BTreeMap<&'static str, String>, +} + +#[derive(Serialize)] +#[serde(rename_all = "PascalCase")] +struct DecryptRequest<'a> { + key_id: &'a str, + ciphertext_blob: &'a str, + encryption_context: &'a BTreeMap<&'static str, String>, +} + +#[derive(Deserialize)] +#[serde(rename_all = "PascalCase")] +struct KeyResponse { + ciphertext_blob: Option, + plaintext: Option, +} + +/// An AWS KMS client, and the AWS KMS key vendor backend +/// ([`KeyVendor::Aws`]). +/// +/// It speaks the AWS KMS JSON API (`TrentService.GenerateDataKey`, +/// `Encrypt` and `Decrypt`), signs each request with SigV4 in-crate, and +/// sends it through a [`VendorTransport`], so the same code runs on native +/// Rust and in the WASI guest. +/// +/// * `key_ref` (from the keyset record) is the AWS KMS key: a key ARN, a +/// key id, or an alias ARN. AWS KMS keys rotate behind one ARN, so the +/// ARN is the versioned reference. Prefer the key ARN: an alias can be +/// repointed. +/// * [`same_key`](KeyVendorBackend::same_key) is string equality. A key +/// ARN, a key id and an alias of one key count as **different** keys: the +/// client cannot tell, offline, where an alias points. So a record in the +/// wrapped data key mode rewraps only to the exact reference it holds, +/// which is enough because AWS KMS rotation keeps the ARN. +/// * The associated data is bound as the encryption context +/// ([`encryption_context`]). AWS KMS refuses a `Decrypt` whose context is +/// not the one the key was encrypted under. +/// * Both key modes. Under wrapped data keys each data key is one +/// `GenerateDataKey` call (AES-256). +/// * Errors map to [`BackendErrorKind`] by the AWS error code +/// ([`error_kind`]), then by HTTP status. +pub struct AwsKms { + config: AwsKmsConfig, + transport: T, + credentials: C, + clock: Clock, +} + +impl std::fmt::Debug for AwsKms { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("AwsKms") + .field("config", &self.config) + .finish_non_exhaustive() + } +} + +impl AwsKms { + /// A client for `config`, sending through `transport`, signing with the + /// credentials of `credentials`. + pub fn new(config: AwsKmsConfig, transport: T, credentials: C) -> Self { + Self { + config, + transport, + credentials, + clock: system_clock(), + } + } + + /// Read the signing time from `clock`. For tests with recorded + /// requests. + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = clock; + self + } + + /// The configuration. + pub fn config(&self) -> &AwsKmsConfig { + &self.config + } + + /// Sign and send one AWS KMS JSON request. + /// + /// `operation` is the `TrentService` operation (`Decrypt`). The body + /// is wiped after it is sent. Any AWS KMS operation can be sent this + /// way (for example `CreateKey` in a test); [`KeyVendorBackend`] uses + /// `GenerateDataKey`, `Encrypt` and `Decrypt`. + pub async fn send_json( + &self, + operation: &'static str, + body: Zeroizing>, + ) -> Result { + let credentials = self.credentials.credentials().await.map_err(|error| { + let kind = error.kind(); + backend(kind, AwsKmsError::Credentials(error)) + })?; + let mut request = VendorRequest { + method: "POST".to_owned(), + url: self.config.endpoint().to_owned(), + headers: vec![ + ("content-type".to_owned(), CONTENT_TYPE.to_owned()), + ( + "x-amz-target".to_owned(), + format!("TrentService.{operation}"), + ), + ], + body, + }; + sign( + &mut request, + &SigningParams { + credentials: &credentials, + region: self.config.region(), + service: SERVICE, + time: (self.clock)(), + }, + ) + .map_err(|error| backend(BackendErrorKind::Unavailable, error.into()))?; + let response = self.transport.send(request).await.map_err(|source| { + backend( + BackendErrorKind::Unavailable, + AwsKmsError::Transport { operation, source }, + ) + })?; + if classify_status(response.status).is_some() { + return Err(service_error(operation, &response)); + } + serde_json::from_slice(&response.body).map_err(|_| invalid(operation, "invalid JSON")) + } + + async fn call( + &self, + operation: &'static str, + request: &Q, + ) -> Result { + let body = Zeroizing::new( + serde_json::to_vec(request) + .map_err(|_| invalid(operation, "unserializable request"))?, + ); + self.send_json(operation, body).await + } +} + +fn invalid(operation: &'static str, reason: &'static str) -> Error { + backend( + BackendErrorKind::Unavailable, + AwsKmsError::InvalidResponse { operation, reason }, + ) +} + +/// The ciphertext blob of a response, decoded. +fn ciphertext(operation: &'static str, response: &KeyResponse) -> Result, Error> { + let blob = response + .ciphertext_blob + .as_deref() + .ok_or_else(|| invalid(operation, "no CiphertextBlob"))?; + Base64::decode_vec(blob).map_err(|_| invalid(operation, "CiphertextBlob is not base64")) +} + +/// The 256-bit plaintext key of a response, decoded. Every copy is wiped. +fn plaintext_key(operation: &'static str, response: KeyResponse) -> Result { + let encoded = Zeroizing::new( + response + .plaintext + .ok_or_else(|| invalid(operation, "no Plaintext"))?, + ); + let decoded = Zeroizing::new( + Base64::decode_vec(&encoded).map_err(|_| invalid(operation, "Plaintext is not base64"))?, + ); + let mut key = Zeroizing::new([0u8; 32]); + if decoded.len() != key.len() { + // A key of another size is not a key this crate wrapped. + return Err(backend( + BackendErrorKind::InvalidCiphertext, + AwsKmsError::InvalidResponse { + operation, + reason: "the plaintext is not a 256-bit key", + }, + )); + } + key.copy_from_slice(&decoded); + Ok(Protected::new(*key)) +} + +impl KeyVendorBackend for AwsKms { + fn vendor(&self) -> KeyVendor { + KeyVendor::Aws + } + + async fn wrap(&self, key_ref: &str, key: &KeyBytes, aad: &[u8]) -> Result, Error> { + use vitaminc::protected::Controlled; + let context = encryption_context(aad); + let plaintext = Zeroizing::new(Base64::encode_string(key.risky_ref())); + let response = self + .call( + "Encrypt", + &EncryptRequest { + key_id: key_ref, + plaintext: &plaintext, + encryption_context: &context, + }, + ) + .await?; + ciphertext("Encrypt", &response) + } + + async fn unwrap(&self, key_ref: &str, wrapped: &[u8], aad: &[u8]) -> Result { + let context = encryption_context(aad); + let blob = Base64::encode_string(wrapped); + let response = self + .call( + "Decrypt", + &DecryptRequest { + key_id: key_ref, + ciphertext_blob: &blob, + encryption_context: &context, + }, + ) + .await?; + plaintext_key("Decrypt", response) + } + + async fn generate_wrapped( + &self, + key_ref: &str, + aad: &[u8], + ) -> Result<(KeyBytes, Vec), Error> { + let context = encryption_context(aad); + let response = self + .call( + "GenerateDataKey", + &GenerateDataKeyRequest { + key_id: key_ref, + key_spec: "AES_256", + encryption_context: &context, + }, + ) + .await?; + let wrapped = ciphertext("GenerateDataKey", &response)?; + let key = plaintext_key("GenerateDataKey", response)?; + Ok((key, wrapped)) + } + + async fn sleep(&self, duration: Duration) { + self.transport.sleep(duration).await; + } +} + +#[cfg(test)] +mod tests { + use stack_kms::KeyMode; + use stack_kms_vendor::derive::{record_key_aad, wrapped_data_key_aad}; + use uuid::Uuid; + + use super::*; + + #[test] + fn the_context_binds_the_aad_and_names_its_fields() { + let id = Uuid::from_u128(1); + let aad = wrapped_data_key_aad(id, &[2u8; 16], "users/email"); + let context = encryption_context(&aad); + assert_eq!( + context.get(CONTEXT_AAD).unwrap(), + &Base64UrlUnpadded::encode_string(&aad) + ); + assert_eq!(context.get(CONTEXT_PURPOSE).unwrap(), "data-key"); + assert_eq!(context.get(CONTEXT_KEYSET_ID).unwrap(), &id.to_string()); + assert_eq!(context.get(CONTEXT_DESCRIPTOR).unwrap(), "users/email"); + + let context = encryption_context(&record_key_aad(id, KeyVendor::Aws, KeyMode::Derived)); + assert_eq!(context.get(CONTEXT_PURPOSE).unwrap(), "root-key"); + assert!(!context.contains_key(CONTEXT_DESCRIPTOR)); + + let context = encryption_context(b"other"); + assert_eq!(context.len(), 1); + let context = encryption_context(&wrapped_data_key_aad(id, &[2u8; 16], "")); + assert!(!context.contains_key(CONTEXT_DESCRIPTOR)); + } + + #[test] + fn error_codes_map_to_kinds() { + for (code, kind) in [ + ("ThrottlingException", BackendErrorKind::Throttled), + ("AccessDeniedException", BackendErrorKind::AccessDenied), + ("NotFoundException", BackendErrorKind::NotFound), + ( + "InvalidCiphertextException", + BackendErrorKind::InvalidCiphertext, + ), + ("IncorrectKeyException", BackendErrorKind::InvalidCiphertext), + ("KMSInternalException", BackendErrorKind::Unavailable), + ("DependencyTimeoutException", BackendErrorKind::Unavailable), + ("DisabledException", BackendErrorKind::Unavailable), + ] { + assert_eq!(error_kind(code), Some(kind), "{code}"); + } + assert_eq!(error_kind("ValidationException"), None); + } + + fn kind_of(response: VendorResponse) -> BackendErrorKind { + match service_error("Decrypt", &response) { + Error::Backend { kind, .. } => kind, + other => panic!("{other:?}"), + } + } + + #[test] + fn the_code_comes_from_the_body_or_the_header() { + let namespaced = VendorResponse::new( + 400, + br#"{"__type":"com.amazonaws.kms#NotFoundException","message":"m"}"#.to_vec(), + ); + assert_eq!(kind_of(namespaced), BackendErrorKind::NotFound); + let header = VendorResponse::new(400, vec![]).header( + "x-amzn-ErrorType", + "ThrottlingException:http://internal.amazon.com/coral/", + ); + assert_eq!(kind_of(header), BackendErrorKind::Throttled); + // No code: the status decides, and an unknown 4xx is a refused + // request. + assert_eq!( + kind_of(VendorResponse::new(503, b"busy".to_vec())), + BackendErrorKind::Unavailable + ); + assert_eq!( + kind_of(VendorResponse::new( + 400, + br#"{"__type":"ValidationException"}"#.to_vec() + )), + BackendErrorKind::AccessDenied + ); + assert_eq!( + kind_of(VendorResponse::new(403, vec![])), + BackendErrorKind::AccessDenied + ); + } + + #[test] + fn the_error_message_names_the_operation_and_code_only() { + let error = service_error( + "Decrypt", + &VendorResponse::new( + 400, + br#"{"__type":"InvalidCiphertextException","Message":"bad"}"#.to_vec(), + ), + ); + let Error::Backend { source, .. } = error else { + panic!() + }; + assert_eq!( + source.to_string(), + "AWS KMS Decrypt failed with HTTP 400: InvalidCiphertextException: bad" + ); + } +} diff --git a/packages/stack-kms-aws/src/config.rs b/packages/stack-kms-aws/src/config.rs new file mode 100644 index 0000000000..39653488ae --- /dev/null +++ b/packages/stack-kms-aws/src/config.rs @@ -0,0 +1,167 @@ +//! The region and endpoint of an AWS KMS client. + +use crate::credentials::{process_env, EnvLookup}; + +/// The configuration is not usable. +#[derive(Debug, thiserror::Error, PartialEq, Eq)] +pub enum ConfigError { + /// No region was given, and neither `AWS_REGION` nor + /// `AWS_DEFAULT_REGION` is set. + #[error("no AWS region: set AWS_REGION or give one in the config")] + NoRegion, + /// The region is not a region name. + #[error("{0:?} is not an AWS region name")] + InvalidRegion(String), + /// The endpoint is not an absolute HTTP or HTTPS URL. + #[error("{0:?} is not an HTTP or HTTPS endpoint URL")] + InvalidEndpoint(String), +} + +/// The DNS suffix of the AWS partition of `region`. +pub(crate) fn dns_suffix(region: &str) -> &'static str { + if region.starts_with("cn-") { + "amazonaws.com.cn" + } else { + "amazonaws.com" + } +} + +/// The region and endpoint of an [`AwsKms`](crate::AwsKms) client. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct AwsKmsConfig { + region: String, + endpoint: String, +} + +impl AwsKmsConfig { + /// The AWS KMS endpoint of `region` (`https://kms..amazonaws.com/`). + pub fn new(region: impl Into) -> Result { + let region = region.into(); + let valid = !region.is_empty() + && region.len() <= 64 + && region + .bytes() + .all(|b| b.is_ascii_lowercase() || b.is_ascii_digit() || b == b'-'); + if !valid { + return Err(ConfigError::InvalidRegion(region)); + } + let endpoint = format!("https://kms.{region}.{}/", dns_suffix(®ion)); + Ok(Self { region, endpoint }) + } + + /// The region from `AWS_REGION`, else `AWS_DEFAULT_REGION`, and the + /// endpoint from `AWS_ENDPOINT_URL_KMS`, else `AWS_ENDPOINT_URL`, else + /// the region's. + pub fn from_env() -> Result { + Self::from_env_lookup(&process_env()) + } + + /// As [`from_env`](Self::from_env), reading the variables with `env`. + pub fn from_env_lookup(env: &EnvLookup) -> Result { + let region = env("AWS_REGION") + .or_else(|| env("AWS_DEFAULT_REGION")) + .ok_or(ConfigError::NoRegion)?; + let config = Self::new(region)?; + match env("AWS_ENDPOINT_URL_KMS").or_else(|| env("AWS_ENDPOINT_URL")) { + Some(endpoint) => config.with_endpoint(endpoint), + None => Ok(config), + } + } + + /// Send to `endpoint` instead of the region's AWS KMS endpoint: a VPC + /// endpoint, a FIPS endpoint, or a local emulator such as LocalStack + /// (`http://localhost:4566`). Requests are still signed for the region. + pub fn with_endpoint(mut self, endpoint: impl Into) -> Result { + let endpoint = endpoint.into(); + let parsed = url::Url::parse(&endpoint) + .map_err(|_| ConfigError::InvalidEndpoint(endpoint.clone()))?; + if !matches!(parsed.scheme(), "http" | "https") || parsed.host().is_none() { + return Err(ConfigError::InvalidEndpoint(endpoint)); + } + self.endpoint = parsed.to_string(); + Ok(self) + } + + /// The region. + pub fn region(&self) -> &str { + &self.region + } + + /// The endpoint URL. + pub fn endpoint(&self) -> &str { + &self.endpoint + } +} + +#[cfg(test)] +mod tests { + use std::collections::HashMap; + use std::sync::Arc; + + use super::*; + + fn env(pairs: &[(&str, &str)]) -> EnvLookup { + let map: HashMap = pairs + .iter() + .map(|(k, v)| ((*k).to_owned(), (*v).to_owned())) + .collect(); + Arc::new(move |name| map.get(name).cloned()) + } + + #[test] + fn the_endpoint_follows_the_region_and_partition() { + let config = AwsKmsConfig::new("ap-southeast-2").unwrap(); + assert_eq!(config.region(), "ap-southeast-2"); + assert_eq!( + config.endpoint(), + "https://kms.ap-southeast-2.amazonaws.com/" + ); + assert_eq!( + AwsKmsConfig::new("cn-north-1").unwrap().endpoint(), + "https://kms.cn-north-1.amazonaws.com.cn/" + ); + for bad in ["", "US-EAST-1", "us east 1", "us-east-1.evil.com/"] { + assert_eq!( + AwsKmsConfig::new(bad), + Err(ConfigError::InvalidRegion(bad.to_owned())) + ); + } + } + + #[test] + fn the_region_and_endpoint_come_from_the_environment() { + assert_eq!( + AwsKmsConfig::from_env_lookup(&env(&[])), + Err(ConfigError::NoRegion) + ); + let config = + AwsKmsConfig::from_env_lookup(&env(&[("AWS_DEFAULT_REGION", "eu-west-1")])).unwrap(); + assert_eq!(config.region(), "eu-west-1"); + let config = AwsKmsConfig::from_env_lookup(&env(&[ + ("AWS_REGION", "us-west-2"), + ("AWS_DEFAULT_REGION", "eu-west-1"), + ("AWS_ENDPOINT_URL", "http://localhost:4566"), + ])) + .unwrap(); + assert_eq!(config.region(), "us-west-2"); + assert_eq!(config.endpoint(), "http://localhost:4566/"); + let config = AwsKmsConfig::from_env_lookup(&env(&[ + ("AWS_REGION", "us-west-2"), + ("AWS_ENDPOINT_URL", "http://localhost:4566"), + ("AWS_ENDPOINT_URL_KMS", "http://localhost:4567"), + ])) + .unwrap(); + assert_eq!(config.endpoint(), "http://localhost:4567/"); + } + + #[test] + fn an_endpoint_must_be_http_or_https() { + let config = AwsKmsConfig::new("us-east-1").unwrap(); + for bad in ["localhost:4566", "ftp://localhost/", "not a url"] { + assert_eq!( + config.clone().with_endpoint(bad), + Err(ConfigError::InvalidEndpoint(bad.to_owned())) + ); + } + } +} diff --git a/packages/stack-kms-aws/src/credentials/chain.rs b/packages/stack-kms-aws/src/credentials/chain.rs new file mode 100644 index 0000000000..8258690157 --- /dev/null +++ b/packages/stack-kms-aws/src/credentials/chain.rs @@ -0,0 +1,90 @@ +//! The default credential chain. + +use stack_kms_vendor::VendorTransport; + +use super::{ + CachedCredentials, Credentials, CredentialsError, CredentialsProvider, EcsCredentials, + EnvLookup, ImdsCredentials, WebIdentityCredentials, +}; + +/// The credentials source the environment names, chosen once, in the order +/// the AWS SDKs use: +/// +/// 1. static credentials: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`; +/// 2. IRSA: `AWS_WEB_IDENTITY_TOKEN_FILE` and `AWS_ROLE_ARN`; +/// 3. the container endpoint: `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI` or +/// `AWS_CONTAINER_CREDENTIALS_FULL_URI`; +/// 4. IMDSv2, unless `AWS_EC2_METADATA_DISABLED=true`. +/// +/// The shared config and credentials files (`~/.aws/config`) and SSO are +/// not read. Native only. +#[derive(Debug)] +pub enum DefaultCredentialsSource { + /// Static credentials from the environment. + Static(Credentials), + /// IRSA. + WebIdentity(WebIdentityCredentials), + /// The container endpoint. + Ecs(EcsCredentials), + /// IMDSv2. + Imds(ImdsCredentials), +} + +/// The default credential chain, cached. +pub type DefaultCredentials = CachedCredentials>; + +impl DefaultCredentialsSource { + /// Choose the source from the variables `env` reads. `region` is the + /// STS region for IRSA. Fails when no source is configured (IMDS + /// disabled and nothing else set) or a container URI is refused. + pub fn from_env_lookup( + env: &EnvLookup, + transport: T, + region: Option<&str>, + ) -> Result { + if let Some(credentials) = Credentials::from_env_lookup(env) { + return Ok(Self::Static(credentials)); + } + if env("AWS_WEB_IDENTITY_TOKEN_FILE").is_some() && env("AWS_ROLE_ARN").is_some() { + return WebIdentityCredentials::from_env_lookup(env, transport, region) + .map(Self::WebIdentity) + .ok_or_else(|| CredentialsError::access_denied("IRSA is not configured")); + } + if env("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI").is_some() + || env("AWS_CONTAINER_CREDENTIALS_FULL_URI").is_some() + { + return EcsCredentials::from_env_lookup(env, transport)? + .map(Self::Ecs) + .ok_or_else(|| { + CredentialsError::access_denied("the container endpoint is not configured") + }); + } + ImdsCredentials::from_env_lookup(env, transport) + .map(Self::Imds) + .ok_or_else(|| { + CredentialsError::access_denied( + "no AWS credentials are configured: set AWS_ACCESS_KEY_ID and \ + AWS_SECRET_ACCESS_KEY, IRSA, a container endpoint, or enable IMDS", + ) + }) + } + + /// Choose the source from the process environment, and cache it. + pub fn cached_from_env( + transport: T, + region: Option<&str>, + ) -> Result, CredentialsError> { + Self::from_env_lookup(&super::process_env(), transport, region).map(CachedCredentials::new) + } +} + +impl CredentialsProvider for DefaultCredentialsSource { + async fn credentials(&self) -> Result { + match self { + Self::Static(credentials) => credentials.credentials().await, + Self::WebIdentity(provider) => provider.credentials().await, + Self::Ecs(provider) => provider.credentials().await, + Self::Imds(provider) => provider.credentials().await, + } + } +} diff --git a/packages/stack-kms-aws/src/credentials/ecs.rs b/packages/stack-kms-aws/src/credentials/ecs.rs new file mode 100644 index 0000000000..c1d9bdfc76 --- /dev/null +++ b/packages/stack-kms-aws/src/credentials/ecs.rs @@ -0,0 +1,166 @@ +//! Credentials from the ECS container endpoint (and EKS Pod Identity). + +use stack_kms_vendor::{VendorRequest, VendorTransport}; +use zeroize::Zeroizing; + +use super::response::json_credentials; +use super::{Credentials, CredentialsError, CredentialsProvider, EnvLookup}; + +/// The ECS task credentials host, for `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`. +const ECS_HOST: &str = "http://169.254.170.2"; + +/// Where the authorization token of the container endpoint comes from. +/// +/// No `Debug`: the `Debug` of `Zeroizing` prints the token. +#[derive(Clone)] +enum AuthToken { + None, + /// `AWS_CONTAINER_AUTHORIZATION_TOKEN`. + Value(Zeroizing), + /// `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE`, read on each fetch (EKS + /// Pod Identity rotates it). + File(std::path::PathBuf), +} + +/// Credentials from the container credentials endpoint: ECS task roles +/// (`AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`) and EKS Pod Identity or any +/// other `AWS_CONTAINER_CREDENTIALS_FULL_URI`. +/// +/// Wrap it in [`CachedCredentials`](super::CachedCredentials). Native only. +pub struct EcsCredentials { + transport: T, + uri: String, + token: AuthToken, +} + +impl std::fmt::Debug for EcsCredentials { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("EcsCredentials") + .field("uri", &self.uri) + .finish_non_exhaustive() + } +} + +impl EcsCredentials { + /// From `AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`, else + /// `AWS_CONTAINER_CREDENTIALS_FULL_URI`, with the token from + /// `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` or + /// `AWS_CONTAINER_AUTHORIZATION_TOKEN`. `Ok(None)` when neither URI is + /// set. + /// + /// A full URI must be HTTPS, or plain HTTP to a loopback address or to + /// one of the ECS and EKS link-local hosts, as the AWS SDKs require: + /// the credentials must not cross a network in the clear. + pub fn from_env_lookup( + env: &EnvLookup, + transport: T, + ) -> Result, CredentialsError> { + let uri = if let Some(relative) = env("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI") { + format!("{ECS_HOST}{relative}") + } else if let Some(full) = env("AWS_CONTAINER_CREDENTIALS_FULL_URI") { + check_full_uri(&full)?; + full + } else { + return Ok(None); + }; + let token = if let Some(file) = env("AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE") { + AuthToken::File(file.into()) + } else if let Some(value) = env("AWS_CONTAINER_AUTHORIZATION_TOKEN") { + AuthToken::Value(Zeroizing::new(value)) + } else { + AuthToken::None + }; + Ok(Some(Self { + transport, + uri, + token, + })) + } +} + +/// Refuse a full URI that would send credentials in the clear over a +/// network. +fn check_full_uri(uri: &str) -> Result<(), CredentialsError> { + let parsed = url::Url::parse(uri).map_err(|_| { + CredentialsError::access_denied("AWS_CONTAINER_CREDENTIALS_FULL_URI is not a valid URL") + })?; + let allowed = match parsed.scheme() { + "https" => true, + "http" => match parsed.host() { + Some(url::Host::Ipv4(ip)) => { + ip.is_loopback() + || ip.octets() == [169, 254, 170, 2] + || ip.octets() == [169, 254, 170, 23] + } + Some(url::Host::Ipv6(ip)) => { + ip.is_loopback() || ip.segments() == [0xfd00, 0x0ec2, 0, 0, 0, 0, 0, 0x23] + } + Some(url::Host::Domain(domain)) => domain == "localhost", + None => false, + }, + _ => false, + }; + if allowed { + Ok(()) + } else { + Err(CredentialsError::access_denied( + "AWS_CONTAINER_CREDENTIALS_FULL_URI must be HTTPS, or HTTP to a loopback or ECS/EKS link-local host", + )) + } +} + +impl CredentialsProvider for EcsCredentials { + async fn credentials(&self) -> Result { + let mut request = VendorRequest::new("GET", self.uri.as_str()); + let token = match &self.token { + AuthToken::None => None, + AuthToken::Value(value) => Some(value.clone()), + AuthToken::File(path) => Some(Zeroizing::new( + std::fs::read_to_string(path) + .map_err(|error| { + CredentialsError::access_denied(format!( + "the container authorization token file {} could not be read: {error}", + path.display() + )) + })? + .trim() + .to_owned(), + )), + }; + if let Some(token) = token { + request = request.header("authorization", token.as_str()); + } + let response = self.transport.send(request).await.map_err(|error| { + CredentialsError::unavailable(format!("the container endpoint did not answer: {error}")) + })?; + json_credentials("the container endpoint", &response) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn full_uris_must_not_cross_a_network_in_the_clear() { + for allowed in [ + "https://example.com/creds", + "http://127.0.0.1:8080/creds", + "http://localhost/creds", + "http://169.254.170.2/v2/credentials", + "http://169.254.170.23/v1/credentials", + "http://[fd00:ec2::23]/v1/credentials", + "http://[::1]/creds", + ] { + assert!(check_full_uri(allowed).is_ok(), "{allowed}"); + } + for refused in [ + "http://example.com/creds", + "http://10.0.0.1/creds", + "ftp://127.0.0.1/creds", + "not a url", + ] { + assert!(check_full_uri(refused).is_err(), "{refused}"); + } + } +} diff --git a/packages/stack-kms-aws/src/credentials/imds.rs b/packages/stack-kms-aws/src/credentials/imds.rs new file mode 100644 index 0000000000..745d821ab6 --- /dev/null +++ b/packages/stack-kms-aws/src/credentials/imds.rs @@ -0,0 +1,109 @@ +//! Credentials from the EC2 instance metadata service, IMDSv2. + +use stack_kms_vendor::{classify_status, VendorRequest, VendorTransport}; +use zeroize::Zeroizing; + +use super::response::{json_credentials, status_failure}; +use super::{Credentials, CredentialsError, CredentialsProvider, EnvLookup}; + +/// The IMDS endpoint when `AWS_EC2_METADATA_SERVICE_ENDPOINT` is not set. +pub const DEFAULT_IMDS_ENDPOINT: &str = "http://169.254.169.254"; +/// The lifetime asked for a session token, in seconds (six hours, the +/// maximum). +const TOKEN_TTL_SECONDS: &str = "21600"; +const CREDENTIALS_PATH: &str = "/latest/meta-data/iam/security-credentials/"; + +/// Credentials of the EC2 instance profile, from the instance metadata +/// service with IMDSv2 session tokens. IMDSv1 is not used. +/// +/// Each fetch gets a session token (`PUT /latest/api/token`), the role +/// name, and the role's credentials. Wrap it in +/// [`CachedCredentials`](super::CachedCredentials). Off EC2 the endpoint +/// does not answer, so give the transport a short connect timeout. Native +/// only. +pub struct ImdsCredentials { + transport: T, + endpoint: String, +} + +impl std::fmt::Debug for ImdsCredentials { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("ImdsCredentials") + .field("endpoint", &self.endpoint) + .finish_non_exhaustive() + } +} + +impl ImdsCredentials { + /// IMDS at `endpoint` (`http://169.254.169.254`, no trailing slash + /// needed). + pub fn new(transport: T, endpoint: impl Into) -> Self { + Self { + transport, + endpoint: endpoint.into().trim_end_matches('/').to_owned(), + } + } + + /// IMDS at `AWS_EC2_METADATA_SERVICE_ENDPOINT`, else the default + /// endpoint. `None` when `AWS_EC2_METADATA_DISABLED` is `true`. + pub fn from_env_lookup(env: &EnvLookup, transport: T) -> Option { + if env("AWS_EC2_METADATA_DISABLED").is_some_and(|value| value.eq_ignore_ascii_case("true")) + { + return None; + } + let endpoint = env("AWS_EC2_METADATA_SERVICE_ENDPOINT") + .unwrap_or_else(|| DEFAULT_IMDS_ENDPOINT.to_owned()); + Some(Self::new(transport, endpoint)) + } + + async fn send(&self, request: VendorRequest) -> Result>, CredentialsError> { + let response = self.transport.send(request).await.map_err(|error| { + CredentialsError::unavailable(format!("IMDS did not answer: {error}")) + })?; + if classify_status(response.status).is_some() { + return Err(status_failure("IMDS", &response)); + } + Ok(response.body) + } +} + +impl CredentialsProvider for ImdsCredentials { + async fn credentials(&self) -> Result { + let token = self + .send( + VendorRequest::new("PUT", format!("{}/latest/api/token", self.endpoint)) + .header("x-aws-ec2-metadata-token-ttl-seconds", TOKEN_TTL_SECONDS), + ) + .await?; + let token = Zeroizing::new( + String::from_utf8(token.to_vec()) + .map_err(|_| CredentialsError::unavailable("IMDS returned an invalid token"))?, + ); + let roles = self + .send( + VendorRequest::new("GET", format!("{}{CREDENTIALS_PATH}", self.endpoint)) + .header("x-aws-ec2-metadata-token", token.as_str()), + ) + .await?; + let role = std::str::from_utf8(&roles) + .ok() + .and_then(|roles| roles.lines().map(str::trim).find(|line| !line.is_empty())) + .map(str::to_owned) + .ok_or_else(|| { + CredentialsError::access_denied("the instance has no IAM instance profile") + })?; + let request = VendorRequest::new( + "GET", + format!( + "{}{CREDENTIALS_PATH}{}", + self.endpoint, + crate::sigv4::uri_encode(role.as_bytes()) + ), + ) + .header("x-aws-ec2-metadata-token", token.as_str()); + let response = self.transport.send(request).await.map_err(|error| { + CredentialsError::unavailable(format!("IMDS did not answer: {error}")) + })?; + json_credentials("IMDS", &response) + } +} diff --git a/packages/stack-kms-aws/src/credentials/mod.rs b/packages/stack-kms-aws/src/credentials/mod.rs new file mode 100644 index 0000000000..62e77ec2fb --- /dev/null +++ b/packages/stack-kms-aws/src/credentials/mod.rs @@ -0,0 +1,418 @@ +//! AWS credentials and where they come from. +//! +//! [`CredentialsProvider`] is the seam: the client asks it for +//! [`Credentials`] before it signs each request. This crate provides: +//! +//! * [`Credentials`] itself: static credentials, from explicit values or +//! from `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and +//! `AWS_SESSION_TOKEN` ([`Credentials::from_env`]). Everywhere. +//! * [`CachedCredentials`]: keeps the credentials another provider returns +//! and fetches new ones before they expire. Everywhere. +//! * Workload identity, **native only**: [`WebIdentityCredentials`] (IRSA +//! on EKS: STS `AssumeRoleWithWebIdentity`), [`EcsCredentials`] (the ECS +//! and EKS Pod Identity container endpoint) and [`ImdsCredentials`] (the +//! EC2 instance metadata service, IMDSv2). [`DefaultCredentials`] picks +//! one from the environment, as the AWS SDKs do. +//! +//! In the WASI guest the host supplies the credentials (ADR-0011): the +//! guest implements [`CredentialsProvider`] over a host import. + +use std::fmt; +use std::future::Future; +use std::time::Duration; + +use futures::lock::Mutex; +use stack_kms::{BackendErrorKind, MaybeSend}; +use web_time::SystemTime; +use zeroize::Zeroizing; + +use crate::time::{system_clock, Clock}; + +#[cfg(not(target_arch = "wasm32"))] +mod chain; +#[cfg(not(target_arch = "wasm32"))] +mod ecs; +#[cfg(not(target_arch = "wasm32"))] +mod imds; +#[cfg(not(target_arch = "wasm32"))] +mod response; +#[cfg(not(target_arch = "wasm32"))] +mod web_identity; + +#[cfg(not(target_arch = "wasm32"))] +pub use chain::{DefaultCredentials, DefaultCredentialsSource}; +#[cfg(not(target_arch = "wasm32"))] +pub use ecs::EcsCredentials; +#[cfg(not(target_arch = "wasm32"))] +pub use imds::{ImdsCredentials, DEFAULT_IMDS_ENDPOINT}; +#[cfg(not(target_arch = "wasm32"))] +pub use web_identity::WebIdentityCredentials; + +/// How an environment variable is read: the process environment unless a +/// test gives another. +pub type EnvLookup = std::sync::Arc Option + Send + Sync>; + +/// The process environment. An empty variable counts as unset. +pub fn process_env() -> EnvLookup { + std::sync::Arc::new(|name| std::env::var(name).ok().filter(|value| !value.is_empty())) +} + +/// AWS credentials: an access key id, a secret access key and, for +/// temporary credentials, a session token and an expiry time. +/// +/// The secret and the session token are wiped on drop and never printed. +#[derive(Clone)] +pub struct Credentials { + access_key_id: String, + secret_access_key: Zeroizing, + session_token: Option>, + expires_at: Option, +} + +impl fmt::Debug for Credentials { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("Credentials") + .field("access_key_id", &self.access_key_id) + .field("secret_access_key", &"") + .field( + "session_token", + &self.session_token.as_ref().map(|_| ""), + ) + .field("expires_at", &self.expires_at) + .finish() + } +} + +impl Credentials { + /// Credentials from explicit values. They do not expire. + pub fn new( + access_key_id: impl Into, + secret_access_key: impl Into, + session_token: Option, + ) -> Self { + Self { + access_key_id: access_key_id.into(), + secret_access_key: Zeroizing::new(secret_access_key.into()), + session_token: session_token.map(Zeroizing::new), + expires_at: None, + } + } + + /// The same credentials, expiring at `expires_at`. + pub fn with_expiry(mut self, expires_at: SystemTime) -> Self { + self.expires_at = Some(expires_at); + self + } + + /// Credentials from `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and + /// (optional) `AWS_SESSION_TOKEN`. `None` unless both of the first two + /// are set. + pub fn from_env() -> Option { + Self::from_env_lookup(&process_env()) + } + + /// As [`from_env`](Self::from_env), reading the variables with `env`. + pub fn from_env_lookup(env: &EnvLookup) -> Option { + let access_key_id = env("AWS_ACCESS_KEY_ID")?; + let secret = Zeroizing::new(env("AWS_SECRET_ACCESS_KEY")?); + Some(Self { + access_key_id, + secret_access_key: secret, + session_token: env("AWS_SESSION_TOKEN").map(Zeroizing::new), + expires_at: None, + }) + } + + /// The access key id. + pub fn access_key_id(&self) -> &str { + &self.access_key_id + } + + /// The secret access key. + pub fn secret_access_key(&self) -> &str { + &self.secret_access_key + } + + /// The session token, for temporary credentials. + pub fn session_token(&self) -> Option<&str> { + self.session_token.as_ref().map(|token| token.as_str()) + } + + /// When the credentials expire. `None` for credentials that do not. + pub fn expires_at(&self) -> Option { + self.expires_at + } +} + +/// The credentials could not be got. +#[derive(Debug, thiserror::Error)] +#[error("AWS credentials: {message}")] +pub struct CredentialsError { + kind: BackendErrorKind, + message: String, +} + +impl CredentialsError { + /// A failure of `kind`, described by `message`. Never put a secret in + /// the message. + pub fn new(kind: BackendErrorKind, message: impl Into) -> Self { + Self { + kind, + message: message.into(), + } + } + + /// No credentials are configured, or they were refused. + pub fn access_denied(message: impl Into) -> Self { + Self::new(BackendErrorKind::AccessDenied, message) + } + + /// The credentials source did not answer. + pub fn unavailable(message: impl Into) -> Self { + Self::new(BackendErrorKind::Unavailable, message) + } + + /// The failure kind the client reports. + pub fn kind(&self) -> BackendErrorKind { + self.kind + } +} + +/// Where a client gets its credentials. The client asks before it signs +/// each request, so a provider that fetches should cache +/// ([`CachedCredentials`]). +/// +/// The returned future is bounded by [`MaybeSend`]: `Send` on native +/// targets, unbounded on wasm32. +pub trait CredentialsProvider: Send + Sync { + /// The current credentials. + fn credentials( + &self, + ) -> impl Future> + MaybeSend; +} + +impl CredentialsProvider for Credentials { + fn credentials( + &self, + ) -> impl Future> + MaybeSend { + std::future::ready(Ok(self.clone())) + } +} + +/// The default time before expiry at which [`CachedCredentials`] fetches +/// new credentials. +pub const DEFAULT_REFRESH_BEFORE_EXPIRY: Duration = Duration::from_secs(5 * 60); + +/// Keeps the credentials `inner` returns, and fetches new ones when they +/// are within the refresh window of their expiry +/// ([`DEFAULT_REFRESH_BEFORE_EXPIRY`]). +/// +/// One fetch runs at a time: callers that arrive during a fetch wait for +/// it and share its result. If a fetch fails while the cached credentials +/// have not yet expired, they are still returned. Credentials with no +/// expiry are kept for good. +pub struct CachedCredentials

{ + inner: P, + cached: Mutex>, + refresh_before: Duration, + clock: Clock, +} + +impl fmt::Debug for CachedCredentials

{ + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("CachedCredentials") + .field("inner", &self.inner) + .field("refresh_before", &self.refresh_before) + .finish_non_exhaustive() + } +} + +impl CachedCredentials

{ + /// Cache the credentials of `inner`. + pub fn new(inner: P) -> Self { + Self { + inner, + cached: Mutex::new(None), + refresh_before: DEFAULT_REFRESH_BEFORE_EXPIRY, + clock: system_clock(), + } + } + + /// Fetch new credentials `refresh_before` before expiry. + pub fn with_refresh_before(mut self, refresh_before: Duration) -> Self { + self.refresh_before = refresh_before; + self + } + + /// Read the time from `clock`. + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = clock; + self + } + + /// The provider whose credentials are cached. + pub fn inner(&self) -> &P { + &self.inner + } +} + +impl CredentialsProvider for CachedCredentials

{ + async fn credentials(&self) -> Result { + let mut cached = self.cached.lock().await; + let now = (self.clock)(); + if let Some(credentials) = cached.as_ref() { + let fresh = credentials + .expires_at + .is_none_or(|expiry| now + self.refresh_before < expiry); + if fresh { + return Ok(credentials.clone()); + } + } + match self.inner.credentials().await { + Ok(credentials) => { + *cached = Some(credentials.clone()); + Ok(credentials) + } + Err(error) => match cached.as_ref() { + Some(credentials) if credentials.expires_at.is_some_and(|e| now < e) => { + Ok(credentials.clone()) + } + _ => Err(error), + }, + } + } +} + +#[cfg(test)] +mod tests { + use std::collections::HashMap; + use std::sync::atomic::{AtomicU64, AtomicUsize, Ordering}; + use std::sync::Arc; + + use web_time::UNIX_EPOCH; + + use super::*; + + fn env(pairs: &[(&str, &str)]) -> EnvLookup { + let map: HashMap = pairs + .iter() + .map(|(k, v)| ((*k).to_owned(), (*v).to_owned())) + .collect(); + Arc::new(move |name| map.get(name).cloned()) + } + + #[test] + fn static_credentials_come_from_the_environment() { + assert!(Credentials::from_env_lookup(&env(&[])).is_none()); + assert!(Credentials::from_env_lookup(&env(&[("AWS_ACCESS_KEY_ID", "AKID")])).is_none()); + let credentials = Credentials::from_env_lookup(&env(&[ + ("AWS_ACCESS_KEY_ID", "AKID"), + ("AWS_SECRET_ACCESS_KEY", "secret"), + ])) + .unwrap(); + assert_eq!(credentials.access_key_id(), "AKID"); + assert_eq!(credentials.secret_access_key(), "secret"); + assert_eq!(credentials.session_token(), None); + assert_eq!(credentials.expires_at(), None); + let credentials = Credentials::from_env_lookup(&env(&[ + ("AWS_ACCESS_KEY_ID", "AKID"), + ("AWS_SECRET_ACCESS_KEY", "secret"), + ("AWS_SESSION_TOKEN", "token"), + ])) + .unwrap(); + assert_eq!(credentials.session_token(), Some("token")); + } + + #[test] + fn debug_hides_the_secret_and_the_token() { + let text = format!( + "{:?}", + Credentials::new("AKID", "the-secret", Some("the-token".into())) + ); + assert!(text.contains("AKID")); + assert!(!text.contains("the-secret")); + assert!(!text.contains("the-token")); + } + + /// A provider that hands out credentials expiring `lifetime` after the + /// test clock, or fails when told to. + struct Counting { + calls: AtomicUsize, + fail: std::sync::atomic::AtomicBool, + now: Arc, + lifetime: Option, + } + + impl CredentialsProvider for Counting { + async fn credentials(&self) -> Result { + let call = self.calls.fetch_add(1, Ordering::SeqCst); + if self.fail.load(Ordering::SeqCst) { + return Err(CredentialsError::unavailable("down")); + } + let credentials = Credentials::new(format!("AKID{call}"), "secret", None); + Ok(match self.lifetime { + Some(lifetime) => credentials.with_expiry( + UNIX_EPOCH + Duration::from_secs(self.now.load(Ordering::SeqCst)) + lifetime, + ), + None => credentials, + }) + } + } + + fn cached(lifetime: Option) -> (CachedCredentials, Arc) { + let now = Arc::new(AtomicU64::new(1_000_000)); + let clock_now = Arc::clone(&now); + let provider = Counting { + calls: AtomicUsize::new(0), + fail: false.into(), + now: Arc::clone(&now), + lifetime, + }; + let clock: Clock = + Arc::new(move || UNIX_EPOCH + Duration::from_secs(clock_now.load(Ordering::SeqCst))); + (CachedCredentials::new(provider).with_clock(clock), now) + } + + #[tokio::test] + async fn credentials_are_fetched_again_before_they_expire() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + assert_eq!(cache.credentials().await.unwrap().access_key_id(), "AKID0"); + now.fetch_add(3600 - 301, Ordering::SeqCst); + assert_eq!(cache.credentials().await.unwrap().access_key_id(), "AKID0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(cache.credentials().await.unwrap().access_key_id(), "AKID1"); + assert_eq!(cache.inner().calls.load(Ordering::SeqCst), 2); + } + + #[tokio::test] + async fn credentials_without_expiry_are_kept() { + let (cache, now) = cached(None); + cache.credentials().await.unwrap(); + now.fetch_add(10 * 365 * 86_400, Ordering::SeqCst); + assert_eq!(cache.credentials().await.unwrap().access_key_id(), "AKID0"); + assert_eq!(cache.inner().calls.load(Ordering::SeqCst), 1); + } + + #[tokio::test] + async fn a_failed_refresh_keeps_unexpired_credentials_only() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + cache.credentials().await.unwrap(); + cache.inner().fail.store(true, Ordering::SeqCst); + now.fetch_add(3500, Ordering::SeqCst); + assert_eq!(cache.credentials().await.unwrap().access_key_id(), "AKID0"); + now.fetch_add(100, Ordering::SeqCst); + let error = cache.credentials().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); + } + + #[tokio::test] + async fn the_refresh_window_is_configurable() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + let cache = cache.with_refresh_before(Duration::ZERO); + cache.credentials().await.unwrap(); + now.fetch_add(3599, Ordering::SeqCst); + assert_eq!(cache.credentials().await.unwrap().access_key_id(), "AKID0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(cache.credentials().await.unwrap().access_key_id(), "AKID1"); + } +} diff --git a/packages/stack-kms-aws/src/credentials/response.rs b/packages/stack-kms-aws/src/credentials/response.rs new file mode 100644 index 0000000000..7c02820589 --- /dev/null +++ b/packages/stack-kms-aws/src/credentials/response.rs @@ -0,0 +1,207 @@ +//! Parsing the credential responses of STS, the container endpoint and +//! IMDS. + +use serde::Deserialize; +use stack_kms::BackendErrorKind; +use stack_kms_vendor::{classify_status, VendorResponse}; +use zeroize::Zeroizing; + +use super::{Credentials, CredentialsError}; +use crate::time::parse_rfc3339; + +/// The JSON credentials of the container endpoint and IMDS. +#[derive(Deserialize)] +#[serde(rename_all = "PascalCase")] +struct JsonCredentials { + code: Option, + access_key_id: String, + secret_access_key: String, + token: Option, + expiration: Option, +} + +/// The failure for a response that is not a success, from `source`. +pub(super) fn status_failure(source: &str, response: &VendorResponse) -> CredentialsError { + let kind = classify_status(response.status).unwrap_or(BackendErrorKind::Unavailable); + // A 400 from a credential source is a refused request, not a damaged + // ciphertext. + let kind = match kind { + BackendErrorKind::InvalidCiphertext => BackendErrorKind::AccessDenied, + kind => kind, + }; + CredentialsError::new(kind, format!("{source} returned HTTP {}", response.status)) +} + +/// Parse the JSON credentials the container endpoint and IMDS return. +pub(super) fn json_credentials( + source: &str, + response: &VendorResponse, +) -> Result { + if classify_status(response.status).is_some() { + return Err(status_failure(source, response)); + } + let parsed: JsonCredentials = serde_json::from_slice(&response.body) + .map_err(|_| CredentialsError::unavailable(format!("{source} returned invalid JSON")))?; + // Move the secrets into wiping containers at once. + let secret = Zeroizing::new(parsed.secret_access_key); + let token = parsed.token.map(Zeroizing::new); + if let Some(code) = parsed.code.as_deref() { + if code != "Success" { + return Err(CredentialsError::access_denied(format!( + "{source} returned the code {code:?}" + ))); + } + } + let mut credentials = Credentials { + access_key_id: parsed.access_key_id, + secret_access_key: secret, + session_token: token, + expires_at: None, + }; + if let Some(expiration) = parsed.expiration.as_deref() { + credentials.expires_at = Some(parse_rfc3339(expiration).ok_or_else(|| { + CredentialsError::unavailable(format!("{source} returned an invalid expiry time")) + })?); + } + Ok(credentials) +} + +/// The text of the first `…` element in `xml`, with the five +/// XML entities decoded. +pub(super) fn xml_text(xml: &str, tag: &str) -> Option> { + let open = format!("<{tag}>"); + let close = format!(""); + let start = xml.find(&open)? + open.len(); + let end = start + xml[start..].find(&close)?; + let raw = &xml[start..end]; + Some(Zeroizing::new( + raw.replace("<", "<") + .replace(">", ">") + .replace(""", "\"") + .replace("'", "'") + .replace("&", "&"), + )) +} + +/// Parse an STS `AssumeRole*` response: the credentials, or the STS error. +pub(super) fn sts_credentials(response: &VendorResponse) -> Result { + let xml = std::str::from_utf8(&response.body) + .map_err(|_| CredentialsError::unavailable("STS returned a body that is not UTF-8"))?; + if classify_status(response.status).is_some() { + let code = xml_text(xml, "Code"); + let code = code.as_deref().map(String::as_str).unwrap_or(""); + let kind = match code { + "Throttling" | "ThrottlingException" | "RequestLimitExceeded" => { + BackendErrorKind::Throttled + } + "IDPCommunicationError" | "ServiceUnavailable" | "InternalFailure" => { + BackendErrorKind::Unavailable + } + _ => return Err(status_failure(&format!("STS ({code})"), response)), + }; + return Err(CredentialsError::new( + kind, + format!("STS returned HTTP {} ({code})", response.status), + )); + } + let field = |tag: &str| { + xml_text(xml, tag) + .ok_or_else(|| CredentialsError::unavailable(format!("the STS response has no {tag}"))) + }; + let access_key_id = field("AccessKeyId")?; + let expiration = field("Expiration")?; + Ok(Credentials { + access_key_id: access_key_id.to_string(), + secret_access_key: field("SecretAccessKey")?, + session_token: Some(field("SessionToken")?), + expires_at: Some( + parse_rfc3339(&expiration).ok_or_else(|| { + CredentialsError::unavailable("STS returned an invalid expiry time") + })?, + ), + }) +} + +#[cfg(test)] +mod tests { + use std::time::Duration; + + use web_time::UNIX_EPOCH; + + use super::*; + + #[test] + fn json_credentials_parse() { + let response = VendorResponse::new( + 200, + br#"{"Code":"Success","Type":"AWS-HMAC","AccessKeyId":"ASIA1","SecretAccessKey":"s","Token":"t","Expiration":"2015-08-30T12:36:00Z"}"#.to_vec(), + ); + let credentials = json_credentials("IMDS", &response).unwrap(); + assert_eq!(credentials.access_key_id(), "ASIA1"); + assert_eq!(credentials.secret_access_key(), "s"); + assert_eq!(credentials.session_token(), Some("t")); + assert_eq!( + credentials.expires_at(), + Some(UNIX_EPOCH + Duration::from_secs(1_440_938_160)) + ); + } + + #[test] + fn json_credentials_refuse_failures() { + let failed = br#"{"Code":"Failed","AccessKeyId":"","SecretAccessKey":""}"#.to_vec(); + let error = json_credentials("IMDS", &VendorResponse::new(200, failed)).unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + let error = + json_credentials("IMDS", &VendorResponse::new(200, b"not json".to_vec())).unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); + let bad_time = + br#"{"AccessKeyId":"a","SecretAccessKey":"s","Expiration":"tomorrow"}"#.to_vec(); + let error = json_credentials("IMDS", &VendorResponse::new(200, bad_time)).unwrap_err(); + assert!(error.to_string().contains("expiry")); + for (status, kind) in [ + (404, BackendErrorKind::NotFound), + (401, BackendErrorKind::AccessDenied), + (400, BackendErrorKind::AccessDenied), + (500, BackendErrorKind::Unavailable), + (429, BackendErrorKind::Throttled), + ] { + let error = json_credentials("ECS", &VendorResponse::new(status, vec![])).unwrap_err(); + assert_eq!(error.kind(), kind, "{status}"); + } + } + + #[test] + fn xml_text_decodes_entities() { + assert_eq!( + xml_text("x&y<>"'", "B") + .unwrap() + .as_str(), + "x&y<>\"'" + ); + assert!(xml_text("x", "B").is_none()); + assert!(xml_text("", "B").is_none()); + } + + #[test] + fn sts_errors_map_to_kinds() { + for (status, code, kind) in [ + (400, "Throttling", BackendErrorKind::Throttled), + (400, "IDPCommunicationError", BackendErrorKind::Unavailable), + (403, "AccessDenied", BackendErrorKind::AccessDenied), + (400, "InvalidIdentityToken", BackendErrorKind::AccessDenied), + (400, "ExpiredTokenException", BackendErrorKind::AccessDenied), + (503, "Other", BackendErrorKind::Unavailable), + ] { + let body = format!( + "Sender{code}m" + ); + let error = + sts_credentials(&VendorResponse::new(status, body.into_bytes())).unwrap_err(); + assert_eq!(error.kind(), kind, "{code}"); + } + let error = sts_credentials(&VendorResponse::new(200, b"".to_vec())).unwrap_err(); + assert!(error.to_string().contains("AccessKeyId")); + let error = sts_credentials(&VendorResponse::new(200, vec![0xff])).unwrap_err(); + assert!(error.to_string().contains("UTF-8")); + } +} diff --git a/packages/stack-kms-aws/src/credentials/web_identity.rs b/packages/stack-kms-aws/src/credentials/web_identity.rs new file mode 100644 index 0000000000..b4051e28dd --- /dev/null +++ b/packages/stack-kms-aws/src/credentials/web_identity.rs @@ -0,0 +1,122 @@ +//! IRSA: credentials from STS `AssumeRoleWithWebIdentity`. + +use std::path::PathBuf; + +use stack_kms_vendor::{VendorRequest, VendorTransport}; + +use super::response::sts_credentials; +use super::{Credentials, CredentialsError, CredentialsProvider, EnvLookup}; +use crate::sigv4::uri_encode; + +/// The STS API version. +const STS_VERSION: &str = "2011-06-15"; +/// The session name when `AWS_ROLE_SESSION_NAME` is not set. +const DEFAULT_SESSION_NAME: &str = "stack-kms-aws"; + +/// Credentials for an IAM role, assumed with a web identity token: IAM +/// Roles for Service Accounts (IRSA) on EKS, or any OIDC token file. +/// +/// Each fetch reads the token file again (the platform rotates it) and +/// calls STS `AssumeRoleWithWebIdentity`, which needs no AWS credentials. +/// Wrap it in [`CachedCredentials`](super::CachedCredentials). Native only. +pub struct WebIdentityCredentials { + transport: T, + role_arn: String, + token_file: PathBuf, + session_name: String, + sts_endpoint: String, +} + +impl std::fmt::Debug for WebIdentityCredentials { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("WebIdentityCredentials") + .field("role_arn", &self.role_arn) + .field("token_file", &self.token_file) + .field("session_name", &self.session_name) + .field("sts_endpoint", &self.sts_endpoint) + .finish_non_exhaustive() + } +} + +impl WebIdentityCredentials { + /// Assume `role_arn` with the token in `token_file`, through the STS + /// endpoint of `region` (the global endpoint for `None`). + pub fn new( + transport: T, + role_arn: impl Into, + token_file: impl Into, + region: Option<&str>, + ) -> Self { + Self { + transport, + role_arn: role_arn.into(), + token_file: token_file.into(), + session_name: DEFAULT_SESSION_NAME.to_owned(), + sts_endpoint: sts_endpoint(region), + } + } + + /// From `AWS_WEB_IDENTITY_TOKEN_FILE`, `AWS_ROLE_ARN` and (optional) + /// `AWS_ROLE_SESSION_NAME`. `None` unless the first two are set. + pub fn from_env_lookup(env: &EnvLookup, transport: T, region: Option<&str>) -> Option { + let token_file = env("AWS_WEB_IDENTITY_TOKEN_FILE")?; + let role_arn = env("AWS_ROLE_ARN")?; + let mut provider = Self::new(transport, role_arn, token_file, region); + if let Some(name) = env("AWS_ROLE_SESSION_NAME") { + provider.session_name = name; + } + Some(provider) + } + + /// Use `endpoint` for STS (a full URL). + pub fn with_sts_endpoint(mut self, endpoint: impl Into) -> Self { + self.sts_endpoint = endpoint.into(); + self + } + + /// Use `name` as the role session name. + pub fn with_session_name(mut self, name: impl Into) -> Self { + self.session_name = name.into(); + self + } +} + +/// The STS endpoint of `region`. +pub(super) fn sts_endpoint(region: Option<&str>) -> String { + match region { + Some(region) => format!( + "https://sts.{region}.{}/", + crate::config::dns_suffix(region) + ), + None => "https://sts.amazonaws.com/".to_owned(), + } +} + +impl CredentialsProvider for WebIdentityCredentials { + async fn credentials(&self) -> Result { + let token = zeroize::Zeroizing::new(std::fs::read_to_string(&self.token_file).map_err( + |error| { + CredentialsError::access_denied(format!( + "the web identity token file {} could not be read: {error}", + self.token_file.display() + )) + }, + )?); + let body = zeroize::Zeroizing::new(format!( + "Action=AssumeRoleWithWebIdentity&RoleArn={}&RoleSessionName={}&Version={STS_VERSION}&WebIdentityToken={}", + uri_encode(self.role_arn.as_bytes()), + uri_encode(self.session_name.as_bytes()), + uri_encode(token.trim().as_bytes()), + )); + let request = VendorRequest::new("POST", self.sts_endpoint.as_str()) + .header( + "content-type", + "application/x-www-form-urlencoded; charset=utf-8", + ) + .body(body.as_bytes().to_vec()); + let response = self.transport.send(request).await.map_err(|error| { + CredentialsError::unavailable(format!("STS did not answer: {error}")) + })?; + sts_credentials(&response) + } +} diff --git a/packages/stack-kms-aws/src/lib.rs b/packages/stack-kms-aws/src/lib.rs new file mode 100644 index 0000000000..2c5f0d2ec7 --- /dev/null +++ b/packages/stack-kms-aws/src/lib.rs @@ -0,0 +1,132 @@ +#![doc(html_favicon_url = "https://cipherstash.com/favicon.ico")] +//! The AWS KMS key vendor backend for Stack Encrypt (ADR-0009, ADR-0011). +//! +//! [`AwsKms`] is a small, hand-written AWS KMS client: the JSON API +//! (`GenerateDataKey`, `Encrypt`, `Decrypt`), SigV4 signing in-crate +//! ([`sigv4`]), and a [`VendorTransport`](stack_kms_vendor::VendorTransport) +//! for the HTTP. It implements +//! [`KeyVendorBackend`](stack_kms_vendor::KeyVendorBackend), so +//! [`VendorKeySource`](stack_kms_vendor::VendorKeySource) builds both key +//! modes on it. No AWS SDK crate is used: they need tokio and hyper, and +//! this crate also builds for wasm32-wasip1, the Go module's WASI guest. +//! +//! # Native Rust +//! +//! ```no_run +//! # async fn example() -> Result<(), Box> { +//! use stack_kms_aws::AwsKms; +//! use stack_kms_vendor::{KeysetRecord, VendorKeySource}; +//! use stack_kms::KeyMode; +//! +//! // AWS_REGION, and credentials from the default chain: environment +//! // variables, IRSA, the container endpoint or IMDSv2. +//! let kms = AwsKms::from_env()?; +//! let record = KeysetRecord::create( +//! &kms, +//! "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab", +//! KeyMode::Wrapped, +//! Some("users".to_owned()), +//! ) +//! .await?; +//! let source = VendorKeySource::new(kms, [record])?; +//! # let _ = source; +//! # Ok(()) +//! # } +//! ``` +//! +//! # Capabilities and limits +//! +//! The [capability matrix](stack_kms_vendor#capability-matrix) of +//! `stack-kms-vendor` compares this backend with ZeroKMS and the other +//! key vendor backends: key lifetime, vendor calls, what the vendor sees +//! and can refuse, credentials, rotation and known limits. +//! +//! # The WASI guest +//! +//! The guest builds [`AwsKms::new`] with its host transport and a +//! [`CredentialsProvider`] over a host import. The workload identity +//! providers (IRSA, the container endpoint, IMDS) are native only: in the +//! guest, the host's own credential chain supplies the credentials. + +#![deny(unsafe_code)] +#![warn(clippy::unwrap_used)] +#![warn(clippy::expect_used)] +#![warn(clippy::panic)] +#![warn(clippy::mem_forget)] +#![warn(clippy::print_stdout)] +#![warn(clippy::print_stderr)] +#![warn(clippy::dbg_macro)] +#![warn(unreachable_pub)] +#![warn(unused_results)] +#![cfg_attr(test, allow(clippy::unwrap_used))] +#![cfg_attr(test, allow(clippy::expect_used))] +#![cfg_attr(test, allow(clippy::panic))] +#![cfg_attr(test, allow(unused_results))] + +mod client; +mod config; +pub mod credentials; +pub mod sigv4; +mod time; + +pub use client::{ + encryption_context, error_kind, AwsKms, AwsKmsError, CONTEXT_AAD, CONTEXT_DESCRIPTOR, + CONTEXT_KEYSET_ID, CONTEXT_PURPOSE, +}; +pub use config::{AwsKmsConfig, ConfigError}; +pub use credentials::{CachedCredentials, Credentials, CredentialsError, CredentialsProvider}; +pub use time::{system_clock, Clock}; + +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +mod native { + use std::time::Duration; + + use stack_kms_vendor::{HttpVendorTransport, HttpVendorTransportOptions}; + + use crate::credentials::{DefaultCredentials, DefaultCredentialsSource}; + use crate::{AwsKms, AwsKmsConfig}; + + /// [`AwsKms::from_env`] failed. + #[derive(Debug, thiserror::Error)] + pub enum FromEnvError { + /// The region or endpoint. + #[error(transparent)] + Config(#[from] crate::ConfigError), + /// The credentials. + #[error(transparent)] + Credentials(#[from] crate::CredentialsError), + /// The HTTP client. + #[error(transparent)] + Transport(#[from] stack_kms_vendor::TransportError), + } + + /// The client [`AwsKms::from_env`] builds. + pub type DefaultAwsKms = AwsKms>; + + impl DefaultAwsKms { + /// A client configured from the environment, as the AWS SDKs are: + /// the region and endpoint from [`AwsKmsConfig::from_env`], and the + /// credentials from the default chain + /// ([`DefaultCredentialsSource`]), cached. Native only. + /// + /// The credentials transport opens connections with a 1 second + /// limit, so that off EC2 the IMDS probe fails fast. + pub fn from_env() -> Result { + let config = AwsKmsConfig::from_env()?; + let transport = HttpVendorTransport::new()?; + let credentials_transport = + HttpVendorTransport::with_options(HttpVendorTransportOptions { + timeout: Duration::from_secs(10), + connect_timeout: Some(Duration::from_secs(1)), + })?; + let credentials = DefaultCredentialsSource::cached_from_env( + credentials_transport, + Some(config.region()), + )?; + Ok(AwsKms::new(config, transport, credentials)) + } + } +} + +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +pub use native::{DefaultAwsKms, FromEnvError}; diff --git a/packages/stack-kms-aws/src/sigv4.rs b/packages/stack-kms-aws/src/sigv4.rs new file mode 100644 index 0000000000..1b767931be --- /dev/null +++ b/packages/stack-kms-aws/src/sigv4.rs @@ -0,0 +1,440 @@ +//! AWS Signature Version 4, in-crate. +//! +//! The AWS SDKs and the `aws-sigv4` crate are not used: the SDKs need tokio +//! and hyper, and this crate must build for wasm32-wasip1 (ADR-0011). +//! Signing is a few HMAC-SHA256 calls, so it is written here and tested +//! against the AWS SigV4 test suite. +//! +//! [`sign`] adds `host` (when the request has none), `x-amz-date`, +//! `x-amz-security-token` (for temporary credentials) and `authorization` +//! to a [`VendorRequest`]. It signs every header the request carries. +//! +//! See . + +use std::fmt::Write as _; + +use hmac::{Hmac, KeyInit, Mac}; +use sha2::{Digest, Sha256}; +use stack_kms_vendor::VendorRequest; +use web_time::SystemTime; +use zeroize::Zeroizing; + +use crate::credentials::Credentials; +use crate::time::amz_date; + +/// The signing algorithm name. +pub const ALGORITHM: &str = "AWS4-HMAC-SHA256"; + +/// A request could not be signed. +#[derive(Debug, thiserror::Error, PartialEq, Eq)] +pub enum SigningError { + /// The request URL does not parse or has no host. + #[error("the request URL is not a valid absolute URL with a host")] + InvalidUrl, +} + +/// What a signature is made with, apart from the request. +#[derive(Clone, Copy, Debug)] +pub struct SigningParams<'a> { + /// The credentials. + pub credentials: &'a Credentials, + /// The AWS region (`us-east-1`). + pub region: &'a str, + /// The service signing name (`kms`, `sts`). + pub service: &'a str, + /// The signing time. + pub time: SystemTime, +} + +/// Sign `request` in place (see the module docs). +pub fn sign(request: &mut VendorRequest, params: &SigningParams<'_>) -> Result<(), SigningError> { + let url = url::Url::parse(&request.url).map_err(|_| SigningError::InvalidUrl)?; + let host = url.host_str().ok_or(SigningError::InvalidUrl)?; + let host = match url.port() { + Some(port) => format!("{host}:{port}"), + None => host.to_owned(), + }; + let (date, timestamp) = amz_date(params.time); + if request.header_value("host").is_none() { + request.headers.push(("host".to_owned(), host)); + } + request + .headers + .push(("x-amz-date".to_owned(), timestamp.clone())); + if let Some(token) = params.credentials.session_token() { + request + .headers + .push(("x-amz-security-token".to_owned(), token.to_owned())); + } + + let (canonical, signed_headers) = canonical_request(request, &url); + let scope = format!("{date}/{}/{}/aws4_request", params.region, params.service); + let string_to_sign = format!( + "{ALGORITHM}\n{timestamp}\n{scope}\n{}", + hex(&Sha256::digest(canonical.as_bytes())) + ); + let key = signing_key( + params.credentials.secret_access_key(), + &date, + params.region, + params.service, + ); + let signature = hex(&hmac(key.as_slice(), string_to_sign.as_bytes())); + request.headers.push(( + "authorization".to_owned(), + format!( + "{ALGORITHM} Credential={}/{scope}, SignedHeaders={signed_headers}, Signature={signature}", + params.credentials.access_key_id() + ), + )); + Ok(()) +} + +/// The canonical request of `request` and its signed header list. +fn canonical_request(request: &VendorRequest, url: &url::Url) -> (String, String) { + let mut headers: Vec<(String, String)> = Vec::new(); + for (name, value) in &request.headers { + let name = name.to_ascii_lowercase(); + let value = value.split_whitespace().collect::>().join(" "); + match headers.iter_mut().find(|(n, _)| *n == name) { + Some((_, joined)) => { + joined.push(','); + joined.push_str(&value); + } + None => headers.push((name, value)), + } + } + headers.sort_by(|a, b| a.0.cmp(&b.0)); + let signed_headers = headers + .iter() + .map(|(name, _)| name.as_str()) + .collect::>() + .join(";"); + let mut canonical = String::new(); + canonical.push_str(&request.method); + canonical.push('\n'); + canonical.push_str(&canonical_uri(url.path())); + canonical.push('\n'); + canonical.push_str(&canonical_query(url.query().unwrap_or(""))); + canonical.push('\n'); + for (name, value) in &headers { + let _ = writeln!(canonical, "{name}:{value}"); + } + canonical.push('\n'); + canonical.push_str(&signed_headers); + canonical.push('\n'); + canonical.push_str(&hex(&Sha256::digest(request.body.as_slice()))); + (canonical, signed_headers) +} + +/// Each path segment decoded and encoded again with the SigV4 rules. +fn canonical_uri(path: &str) -> String { + if path.is_empty() { + return "/".to_owned(); + } + path.split('/') + .map(|segment| uri_encode(&percent_decode(segment))) + .collect::>() + .join("/") +} + +/// The query pairs decoded, encoded again with the SigV4 rules, and sorted +/// by name, then value. +fn canonical_query(query: &str) -> String { + let mut pairs: Vec<(String, String)> = query + .split('&') + .filter(|pair| !pair.is_empty()) + .map(|pair| { + let (name, value) = pair.split_once('=').unwrap_or((pair, "")); + ( + uri_encode(&percent_decode(name)), + uri_encode(&percent_decode(value)), + ) + }) + .collect(); + pairs.sort(); + pairs + .iter() + .map(|(name, value)| format!("{name}={value}")) + .collect::>() + .join("&") +} + +/// Percent-encode every byte except the RFC 3986 unreserved characters. +pub(crate) fn uri_encode(bytes: &[u8]) -> String { + let mut out = String::with_capacity(bytes.len()); + for &byte in bytes { + if byte.is_ascii_alphanumeric() || matches!(byte, b'-' | b'_' | b'.' | b'~') { + out.push(byte as char); + } else { + let _ = write!(out, "%{byte:02X}"); + } + } + out +} + +/// Decode `%XX` escapes. An escape that is not two hex digits is kept as +/// it is. +fn percent_decode(text: &str) -> Vec { + let bytes = text.as_bytes(); + let mut out = Vec::with_capacity(bytes.len()); + let mut i = 0; + while i < bytes.len() { + let decoded = (bytes[i] == b'%') + .then(|| bytes.get(i + 1..i + 3)) + .flatten() + .and_then(|hex| std::str::from_utf8(hex).ok()) + .and_then(|hex| u8::from_str_radix(hex, 16).ok()); + match decoded { + Some(byte) => { + out.push(byte); + i += 3; + } + None => { + out.push(bytes[i]); + i += 1; + } + } + } + out +} + +/// The SigV4 signing key: `HMAC` chained over the date, region, service and +/// `aws4_request`, from `AWS4` + the secret. +fn signing_key(secret: &str, date: &str, region: &str, service: &str) -> Zeroizing> { + let initial = Zeroizing::new(format!("AWS4{secret}")); + let k_date = Zeroizing::new(hmac(initial.as_bytes(), date.as_bytes())); + let k_region = Zeroizing::new(hmac(&k_date, region.as_bytes())); + let k_service = Zeroizing::new(hmac(&k_region, service.as_bytes())); + Zeroizing::new(hmac(&k_service, b"aws4_request")) +} + +fn hmac(key: &[u8], data: &[u8]) -> Vec { + // HMAC takes a key of any length. + #[allow(clippy::expect_used)] + let mut mac = Hmac::::new_from_slice(key).expect("HMAC accepts any key length"); + mac.update(data); + mac.finalize().into_bytes().to_vec() +} + +pub(crate) fn hex(bytes: &[u8]) -> String { + bytes.iter().fold(String::new(), |mut out, byte| { + let _ = write!(out, "{byte:02x}"); + out + }) +} + +#[cfg(test)] +mod tests { + //! Vectors from the AWS SigV4 test suite (`aws-sig-v4-test-suite`, as + //! shipped in botocore and the AWS SDKs). Every case uses the access key + //! `AKIDEXAMPLE`, the secret `wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY`, + //! region `us-east-1`, service `service` and the time + //! `20150830T123600Z`, except the IAM example from the AWS SigV4 + //! documentation (service `iam`). + + use std::time::Duration; + + use web_time::UNIX_EPOCH; + + use super::*; + + const ACCESS_KEY: &str = "AKIDEXAMPLE"; + const SECRET: &str = "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY"; + + fn suite_time() -> SystemTime { + UNIX_EPOCH + Duration::from_secs(1_440_938_160) + } + + fn signed(mut request: VendorRequest, service: &str) -> VendorRequest { + let credentials = Credentials::new(ACCESS_KEY, SECRET, None); + sign( + &mut request, + &SigningParams { + credentials: &credentials, + region: "us-east-1", + service, + time: suite_time(), + }, + ) + .unwrap(); + request + } + + fn canonical(request: &VendorRequest) -> String { + canonical_request(request, &url::Url::parse(&request.url).unwrap()).0 + } + + fn authorization(request: &VendorRequest) -> &str { + request.header_value("authorization").unwrap() + } + + #[test] + fn get_vanilla() { + let request = signed( + VendorRequest::new("GET", "https://example.amazonaws.com/"), + "service", + ); + assert_eq!( + canonical(&request.clone().without_authorization()), + "GET\n/\n\nhost:example.amazonaws.com\nx-amz-date:20150830T123600Z\n\nhost;x-amz-date\n\ + e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + ); + assert_eq!( + authorization(&request), + "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/service/aws4_request, \ + SignedHeaders=host;x-amz-date, \ + Signature=5fa00fa31553b73ebf1942676e86291e8372ff2a2260956d9b8aae1d763fbf31" + ); + } + + #[test] + fn post_vanilla() { + let request = signed( + VendorRequest::new("POST", "https://example.amazonaws.com/"), + "service", + ); + assert_eq!( + authorization(&request), + "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/service/aws4_request, \ + SignedHeaders=host;x-amz-date, \ + Signature=5da7c1a2acd57cee7505fc6676e4e544621c30862966e37dddb68e92efbe5d6b" + ); + } + + #[test] + fn get_vanilla_query_order_key_case() { + let request = signed( + VendorRequest::new( + "GET", + "https://example.amazonaws.com/?Param2=value2&Param1=value1", + ), + "service", + ); + assert_eq!( + authorization(&request), + "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/service/aws4_request, \ + SignedHeaders=host;x-amz-date, \ + Signature=b97d918cfa904a5beff61c982a1b6f458b799221646efd99d3219ec94cdf2500" + ); + } + + #[test] + fn post_x_www_form_urlencoded() { + let request = signed( + VendorRequest::new("POST", "https://example.amazonaws.com/") + .header("Content-Type", "application/x-www-form-urlencoded") + .body(b"Param1=value1".to_vec()), + "service", + ); + assert_eq!( + authorization(&request), + "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/service/aws4_request, \ + SignedHeaders=content-type;host;x-amz-date, \ + Signature=ff11897932ad3f4e8b18135d722051e5ac45fc38421b1da7b9d196a0fe09473a" + ); + } + + /// The worked example in the AWS SigV4 documentation: IAM `ListUsers`. + #[test] + fn the_documented_iam_example() { + let request = signed( + VendorRequest::new( + "GET", + "https://iam.amazonaws.com/?Action=ListUsers&Version=2010-05-08", + ) + .header( + "Content-Type", + "application/x-www-form-urlencoded; charset=utf-8", + ), + "iam", + ); + let canonical = canonical(&request.clone().without_authorization()); + assert_eq!( + hex(&Sha256::digest(canonical.as_bytes())), + "f536975d06c0309214f805bb90ccff089219ecd68b2577efef23edd43b7e1a59" + ); + assert_eq!( + authorization(&request), + "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20150830/us-east-1/iam/aws4_request, \ + SignedHeaders=content-type;host;x-amz-date, \ + Signature=5d672d79c15b13162d9279b0855cfba6789a8edb4c82c400e06b5924a6f2b5d7" + ); + } + + #[test] + fn a_session_token_is_sent_and_signed() { + let credentials = Credentials::new(ACCESS_KEY, SECRET, Some("session-token".to_owned())); + let mut request = VendorRequest::new("POST", "https://kms.us-east-1.amazonaws.com/"); + sign( + &mut request, + &SigningParams { + credentials: &credentials, + region: "us-east-1", + service: "kms", + time: suite_time(), + }, + ) + .unwrap(); + assert_eq!( + request.header_value("x-amz-security-token"), + Some("session-token") + ); + assert!( + authorization(&request).contains("SignedHeaders=host;x-amz-date;x-amz-security-token,") + ); + } + + #[test] + fn a_port_is_part_of_the_host() { + let request = signed(VendorRequest::new("POST", "http://localhost:4566/"), "kms"); + assert_eq!(request.header_value("host"), Some("localhost:4566")); + let request = signed( + VendorRequest::new("POST", "https://kms.us-east-1.amazonaws.com:443/"), + "kms", + ); + assert_eq!( + request.header_value("host"), + Some("kms.us-east-1.amazonaws.com") + ); + } + + #[test] + fn an_invalid_url_is_refused() { + let credentials = Credentials::new(ACCESS_KEY, SECRET, None); + let params = SigningParams { + credentials: &credentials, + region: "us-east-1", + service: "kms", + time: suite_time(), + }; + for url in ["not a url", "data:text/plain,hello"] { + assert_eq!( + sign(&mut VendorRequest::new("GET", url), ¶ms), + Err(SigningError::InvalidUrl) + ); + } + } + + #[test] + fn paths_and_queries_canonicalise() { + assert_eq!(canonical_uri(""), "/"); + assert_eq!(canonical_uri("/example%20space/"), "/example%20space/"); + assert_eq!(canonical_uri("/a~b/%7e"), "/a~b/~"); + assert_eq!(canonical_query("b=2&a=1&a=0"), "a=0&a=1&b=2"); + assert_eq!(canonical_query("Param=a%20b&x"), "Param=a%20b&x="); + assert_eq!(canonical_query("k=%zz"), "k=%25zz"); + assert_eq!(canonical_query(""), ""); + } + + trait WithoutAuthorization { + fn without_authorization(self) -> Self; + } + + impl WithoutAuthorization for VendorRequest { + fn without_authorization(mut self) -> Self { + self.headers.retain(|(name, _)| name != "authorization"); + self + } + } +} diff --git a/packages/stack-kms-aws/src/time.rs b/packages/stack-kms-aws/src/time.rs new file mode 100644 index 0000000000..409fef14bd --- /dev/null +++ b/packages/stack-kms-aws/src/time.rs @@ -0,0 +1,164 @@ +//! UTC dates without a date crate: the SigV4 timestamp and the expiry +//! times in AWS credential responses. + +use std::time::Duration; + +use web_time::{SystemTime, UNIX_EPOCH}; + +/// The time source of a client: `SystemTime::now` unless a test sets +/// another. +pub type Clock = std::sync::Arc SystemTime + Send + Sync>; + +/// The system clock. +pub fn system_clock() -> Clock { + std::sync::Arc::new(SystemTime::now) +} + +/// The civil date of a day count since 1970-01-01 (Howard Hinnant's +/// `civil_from_days`). +fn civil_from_days(days: i64) -> (i64, u32, u32) { + let z = days + 719_468; + let era = z.div_euclid(146_097); + let doe = z.rem_euclid(146_097); + let yoe = (doe - doe / 1460 + doe / 36_524 - doe / 146_096) / 365; + let doy = doe - (365 * yoe + yoe / 4 - yoe / 100); + let mp = (5 * doy + 2) / 153; + let day = (doy - (153 * mp + 2) / 5 + 1) as u32; + let month = if mp < 10 { mp + 3 } else { mp - 9 } as u32; + let year = yoe + era * 400 + i64::from(month <= 2); + (year, month, day) +} + +/// The day count since 1970-01-01 of a civil date (Howard Hinnant's +/// `days_from_civil`). +// Only the native credential providers parse expiry times. +#[cfg_attr(target_arch = "wasm32", allow(dead_code))] +fn days_from_civil(year: i64, month: u32, day: u32) -> i64 { + let year = if month <= 2 { year - 1 } else { year }; + let era = year.div_euclid(400); + let yoe = year.rem_euclid(400); + let month = i64::from(month); + let mp = if month > 2 { month - 3 } else { month + 9 }; + let doy = (153 * mp + 2) / 5 + i64::from(day) - 1; + let doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; + era * 146_097 + doe - 719_468 +} + +/// The SigV4 date (`YYYYMMDD`) and timestamp (`YYYYMMDD'T'HHMMSS'Z'`) of +/// `time`, in UTC. A time before 1970 counts as 1970-01-01. +pub(crate) fn amz_date(time: SystemTime) -> (String, String) { + let seconds = time + .duration_since(UNIX_EPOCH) + .map_or(0, |since| since.as_secs()); + let days = i64::try_from(seconds / 86_400).unwrap_or(i64::MAX / 2); + let rest = seconds % 86_400; + let (year, month, day) = civil_from_days(days); + let date = format!("{year:04}{month:02}{day:02}"); + let timestamp = format!( + "{date}T{:02}{:02}{:02}Z", + rest / 3600, + (rest / 60) % 60, + rest % 60 + ); + (date, timestamp) +} + +/// Parse an RFC 3339 UTC time as AWS returns it: `2026-11-05T12:00:00Z`, +/// with optional fractional seconds, and `Z` or `+00:00`. `None` for any +/// other form. +#[cfg_attr(target_arch = "wasm32", allow(dead_code))] +pub(crate) fn parse_rfc3339(text: &str) -> Option { + let text = text.trim(); + let text = text + .strip_suffix('Z') + .or_else(|| text.strip_suffix("+00:00"))?; + let (date, time) = text.split_once('T')?; + let mut date_parts = date.split('-'); + let year: i64 = date_parts.next()?.parse().ok()?; + let month: u32 = date_parts.next()?.parse().ok()?; + let day: u32 = date_parts.next()?.parse().ok()?; + if date_parts.next().is_some() || !(1..=12).contains(&month) || !(1..=31).contains(&day) { + return None; + } + let (time, fraction) = match time.split_once('.') { + Some((time, fraction)) => (time, Some(fraction)), + None => (time, None), + }; + let mut time_parts = time.split(':'); + let hour: u64 = time_parts.next()?.parse().ok()?; + let minute: u64 = time_parts.next()?.parse().ok()?; + let second: u64 = time_parts.next()?.parse().ok()?; + if time_parts.next().is_some() || hour > 23 || minute > 59 || second > 60 { + return None; + } + let nanos = match fraction { + Some(digits) if !digits.is_empty() && digits.bytes().all(|b| b.is_ascii_digit()) => { + let digits = &digits[..digits.len().min(9)]; + let value: u32 = digits.parse().ok()?; + value * 10u32.pow(9 - digits.len() as u32) + } + Some(_) => return None, + None => 0, + }; + let days = u64::try_from(days_from_civil(year, month, day)).ok()?; + let seconds = days * 86_400 + hour * 3600 + minute * 60 + second; + UNIX_EPOCH.checked_add(Duration::new(seconds, nanos)) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn at(seconds: u64) -> SystemTime { + UNIX_EPOCH + Duration::from_secs(seconds) + } + + #[test] + fn amz_date_formats_utc() { + // 2015-08-30T12:36:00Z, the time of the AWS SigV4 test suite. + let (date, timestamp) = amz_date(at(1_440_938_160)); + assert_eq!(date, "20150830"); + assert_eq!(timestamp, "20150830T123600Z"); + assert_eq!(amz_date(at(0)).1, "19700101T000000Z"); + // A leap day, and the last second of a year. + assert_eq!(amz_date(at(951_782_400)).1, "20000229T000000Z"); + assert_eq!(amz_date(at(1_798_761_599)).1, "20261231T235959Z"); + } + + #[test] + fn rfc3339_parses_the_forms_aws_returns() { + assert_eq!( + parse_rfc3339("2015-08-30T12:36:00Z"), + Some(at(1_440_938_160)) + ); + assert_eq!( + parse_rfc3339("2015-08-30T12:36:00+00:00"), + Some(at(1_440_938_160)) + ); + assert_eq!( + parse_rfc3339("2015-08-30T12:36:00.250Z"), + Some(at(1_440_938_160) + Duration::from_millis(250)) + ); + assert_eq!(parse_rfc3339("2000-02-29T00:00:00Z"), Some(at(951_782_400))); + for bad in [ + "2015-08-30T12:36:00", + "2015-08-30 12:36:00Z", + "2015-13-30T12:36:00Z", + "2015-08-30T24:00:00Z", + "2015-08-30T12:36:00.Z", + "2015-08-30T12:36:00.1xZ", + "1969-12-31T23:59:59Z", + "", + ] { + assert_eq!(parse_rfc3339(bad), None, "{bad}"); + } + } + + #[test] + fn the_two_conversions_agree() { + for days in [0, 1, 59, 60, 365, 10_957, 11_016, 20_000, 2_932_896] { + let (year, month, day) = civil_from_days(days); + assert_eq!(days_from_civil(year, month, day), days); + } + } +} diff --git a/packages/stack-kms-aws/tasks.toml b/packages/stack-kms-aws/tasks.toml new file mode 100644 index 0000000000..c8b26e30e6 --- /dev/null +++ b/packages/stack-kms-aws/tasks.toml @@ -0,0 +1,25 @@ +# Rustdoc with warnings as errors: a broken intra-doc link or a rustdoc +# warning fails the build. The root `doc` task fans out over every +# `doc:`; the root `test:doc` runs the doc examples. +["doc:stack-kms-aws"] +description = "Build docs for stack-kms-aws with all features (warnings are errors)" +env = { RUSTDOCFLAGS = "-D warnings" } +run = "cargo doc -p stack-kms-aws --no-deps --all-features" + +# The AWS KMS backend against a running LocalStack (tests/localstack.rs). The +# test skips itself when STACK_KMS_AWS_LOCALSTACK_ENDPOINT is unset, which is +# right for `cargo nextest run --workspace` and wrong here: a skip would read +# as a pass. So this task refuses to run without it. tests-crates.yml's +# `localstack` job runs LocalStack as a service and calls this task. +# +# docker run -d --rm -p 4566:4566 -e SERVICES=kms localstack/localstack:4.4 +# STACK_KMS_AWS_LOCALSTACK_ENDPOINT=http://localhost:4566 mise run test:localstack:stack-kms-aws +["test:localstack:stack-kms-aws"] +description = "Run the stack-kms-aws tests against LocalStack (needs STACK_KMS_AWS_LOCALSTACK_ENDPOINT)" +run = """ +if [ -z "${STACK_KMS_AWS_LOCALSTACK_ENDPOINT:-}" ]; then + echo "error: set STACK_KMS_AWS_LOCALSTACK_ENDPOINT (for example http://localhost:4566)" >&2 + exit 1 +fi +mise x --env test -- cargo nextest run --locked -p stack-kms-aws --all-features --test localstack --no-tests=fail +""" diff --git a/packages/stack-kms-aws/tests/common/mod.rs b/packages/stack-kms-aws/tests/common/mod.rs new file mode 100644 index 0000000000..b9b2ab6726 --- /dev/null +++ b/packages/stack-kms-aws/tests/common/mod.rs @@ -0,0 +1,331 @@ +//! Shared test support: recorded fixtures, a replaying transport, and an +//! in-process emulation of the AWS KMS JSON API. + +#![allow(dead_code, clippy::unwrap_used, clippy::expect_used, clippy::panic)] + +use std::collections::{BTreeMap, HashMap, VecDeque}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use aes_gcm::aead::{Aead, KeyInit, Payload}; +use aes_gcm::{Aes256Gcm, Nonce}; +use base64ct::{Base64, Encoding}; +use serde::Deserialize; +use serde_json::{json, Value}; +use stack_kms_aws::{AwsKms, AwsKmsConfig, Clock, Credentials}; +use stack_kms_vendor::{TransportError, VendorRequest, VendorResponse, VendorTransport}; +use web_time::{SystemTime, UNIX_EPOCH}; + +/// The key the fixtures use. +pub const KEY_ARN: &str = + "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab"; +/// 2026-11-05T12:00:00Z, the signing time of every fixture. +pub const FIXTURE_TIME: u64 = 1_793_880_000; + +pub fn fixture_clock() -> Clock { + Arc::new(|| UNIX_EPOCH + Duration::from_secs(FIXTURE_TIME)) +} + +/// The credentials of every fixture: the AWS documentation example key. +pub fn fixture_credentials() -> Credentials { + Credentials::new( + "AKIDEXAMPLE", + "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY", + None, + ) +} + +#[derive(Deserialize)] +pub struct FixtureRequest { + pub method: String, + pub url: String, + pub headers: Vec<(String, String)>, + pub body: Value, +} + +#[derive(Clone, Deserialize)] +pub struct FixtureResponse { + pub status: u16, + #[serde(default)] + pub headers: Vec<(String, String)>, + pub body: Value, +} + +impl FixtureResponse { + pub fn to_response(&self) -> VendorResponse { + let body = match &self.body { + Value::String(text) => text.clone().into_bytes(), + other => serde_json::to_vec(other).unwrap(), + }; + let mut response = VendorResponse::new(self.status, body); + for (name, value) in &self.headers { + response = response.header(name, value); + } + response + } +} + +#[derive(Deserialize)] +pub struct Fixture { + pub request: FixtureRequest, + pub response: FixtureResponse, +} + +pub fn load Deserialize<'de>>(name: &str) -> T { + let path = format!("{}/tests/fixtures/{name}", env!("CARGO_MANIFEST_DIR")); + let text = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}")); + serde_json::from_str(&text).unwrap_or_else(|e| panic!("{path}: {e}")) +} + +/// A transport that checks each request against a recorded one and +/// answers with the recorded response. +#[derive(Default)] +pub struct Replay { + exchanges: Mutex, VendorResponse)>>, +} + +impl Replay { + pub fn new() -> Self { + Self::default() + } + + /// Expect `fixture`'s request and answer with its response. + pub fn expect(self, fixture: Fixture) -> Self { + let response = fixture.response.to_response(); + self.exchanges + .lock() + .unwrap() + .push_back((Some(fixture.request), response)); + self + } + + /// Answer the next request, whatever it is, with `response`. + pub fn answer(self, response: VendorResponse) -> Self { + self.exchanges.lock().unwrap().push_back((None, response)); + self + } + + pub fn is_done(&self) -> bool { + self.exchanges.lock().unwrap().is_empty() + } +} + +impl VendorTransport for Replay { + async fn send(&self, request: VendorRequest) -> Result { + let (expected, response) = self + .exchanges + .lock() + .unwrap() + .pop_front() + .expect("a request the fixtures do not expect"); + if let Some(expected) = expected { + let body: Value = serde_json::from_slice(&request.body).unwrap(); + let actual = json!({ + "method": request.method, + "url": request.url, + "headers": request.headers, + "body": body, + }); + let wanted = json!({ + "method": expected.method, + "url": expected.url, + "headers": expected.headers, + "body": expected.body, + }); + assert_eq!( + actual, + wanted, + "the request differs from the fixture; actual:\n{}", + serde_json::to_string_pretty(&actual).unwrap() + ); + } + Ok(response) + } + + async fn sleep(&self, _duration: Duration) {} +} + +/// A client over `transport` with the fixture time and credentials. +pub fn fixture_client(transport: T) -> AwsKms { + AwsKms::new( + AwsKmsConfig::new("us-east-1").unwrap(), + transport, + fixture_credentials(), + ) + .with_clock(fixture_clock()) +} + +/// An in-process emulation of the AWS KMS JSON API: `CreateKey`, +/// `GenerateDataKey`, `Encrypt` and `Decrypt`, with the encryption context +/// bound by AES-256-GCM, and the AWS error shapes. It checks that each +/// request is signed for the fixture access key and the `kms` service. +/// +/// The contract suite uses random keys and ids, so recorded bytes cannot +/// serve it: this answers it instead. Clones share one key store. +#[derive(Clone, Default)] +pub struct KmsEmulator { + keys: Arc>>, + sleeps: Arc>>, +} + +impl KmsEmulator { + pub fn with_key(key_id: &str) -> Self { + let emulator = Self::default(); + emulator + .keys + .lock() + .unwrap() + .insert(key_id.to_owned(), rand_key()); + emulator + } + + pub fn sleeps(&self) -> Vec { + self.sleeps.lock().unwrap().clone() + } + + fn error(code: &str, message: &str) -> VendorResponse { + VendorResponse::new( + 400, + serde_json::to_vec(&json!({"__type": code, "message": message})).unwrap(), + ) + .header("content-type", "application/x-amz-json-1.1") + } + + fn ok(body: Value) -> VendorResponse { + VendorResponse::new(200, serde_json::to_vec(&body).unwrap()) + .header("content-type", "application/x-amz-json-1.1") + } + + fn seal(key_id: &str, key: &[u8; 32], context: &Value, plaintext: &[u8]) -> Vec { + let cipher = Aes256Gcm::new_from_slice(key).unwrap(); + let nonce = rand_key(); + let aad = serde_json::to_vec(context).unwrap(); + let sealed = cipher + .encrypt( + Nonce::from_slice(&nonce[..12]), + Payload { + msg: plaintext, + aad: &aad, + }, + ) + .unwrap(); + let mut blob = vec![key_id.len() as u8]; + blob.extend_from_slice(key_id.as_bytes()); + blob.extend_from_slice(&nonce[..12]); + blob.extend_from_slice(&sealed); + blob + } + + fn handle(&self, operation: &str, body: &Value) -> VendorResponse { + let context = body.get("EncryptionContext").cloned().unwrap_or(json!({})); + // A BTreeMap gives one byte form per context. + let context: BTreeMap = serde_json::from_value(context).unwrap(); + let context = serde_json::to_value(context).unwrap(); + let key_id = body.get("KeyId").and_then(Value::as_str).unwrap_or(""); + let keys = self.keys.lock().unwrap(); + match operation { + "CreateKey" => { + drop(keys); + let id = format!("emulated-{}", self.keys.lock().unwrap().len()); + self.keys.lock().unwrap().insert(id.clone(), rand_key()); + Self::ok(json!({"KeyMetadata": {"KeyId": id, "Arn": id}})) + } + "GenerateDataKey" | "Encrypt" => { + let Some(key) = keys.get(key_id) else { + return Self::error("NotFoundException", "no such key"); + }; + let (plaintext, give_back) = if operation == "Encrypt" { + let encoded = body.get("Plaintext").and_then(Value::as_str).unwrap(); + (Base64::decode_vec(encoded).unwrap(), false) + } else { + assert_eq!(body.get("KeySpec"), Some(&json!("AES_256"))); + (rand_key().to_vec(), true) + }; + let blob = Self::seal(key_id, key, &context, &plaintext); + let mut answer = json!({ + "KeyId": key_id, + "CiphertextBlob": Base64::encode_string(&blob), + }); + if give_back { + answer["Plaintext"] = json!(Base64::encode_string(&plaintext)); + } + Self::ok(answer) + } + "Decrypt" => { + let blob = body + .get("CiphertextBlob") + .and_then(Value::as_str) + .and_then(|blob| Base64::decode_vec(blob).ok()) + .unwrap_or_default(); + let Some((&len, rest)) = blob.split_first() else { + return Self::error("InvalidCiphertextException", "empty"); + }; + let len = usize::from(len); + if rest.len() < len + 12 { + return Self::error("InvalidCiphertextException", "short"); + } + let (blob_key, rest) = rest.split_at(len); + if blob_key != key_id.as_bytes() { + return Self::error("IncorrectKeyException", "another key"); + } + let Some(key) = keys.get(key_id) else { + return Self::error("NotFoundException", "no such key"); + }; + let (nonce, sealed) = rest.split_at(12); + let aad = serde_json::to_vec(&context).unwrap(); + match Aes256Gcm::new_from_slice(key).unwrap().decrypt( + Nonce::from_slice(nonce), + Payload { + msg: sealed, + aad: &aad, + }, + ) { + Ok(plaintext) => Self::ok(json!({ + "KeyId": key_id, + "Plaintext": Base64::encode_string(&plaintext), + })), + Err(_) => Self::error("InvalidCiphertextException", "does not open"), + } + } + other => Self::error("UnknownOperationException", other), + } + } +} + +fn rand_key() -> [u8; 32] { + use vitaminc::random::{Generatable, SafeRand}; + let mut rng = SafeRand::from_entropy().unwrap(); + <[u8; 32]>::random(&mut rng).unwrap() +} + +impl VendorTransport for KmsEmulator { + async fn send(&self, request: VendorRequest) -> Result { + assert_eq!(request.method, "POST"); + assert_eq!( + request.header_value("content-type"), + Some("application/x-amz-json-1.1") + ); + let authorization = request.header_value("authorization").unwrap_or(""); + assert!( + authorization.starts_with("AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/") + && authorization.contains("/us-east-1/kms/aws4_request, SignedHeaders="), + "unsigned or wrongly scoped request: {authorization}" + ); + let operation = request + .header_value("x-amz-target") + .and_then(|target| target.strip_prefix("TrentService.")) + .expect("an X-Amz-Target of TrentService") + .to_owned(); + let body: Value = serde_json::from_slice(&request.body).unwrap(); + Ok(self.handle(&operation, &body)) + } + + async fn sleep(&self, duration: Duration) { + self.sleeps.lock().unwrap().push(duration); + } +} + +/// The time now, for tests that do not pin it. +pub fn now() -> SystemTime { + SystemTime::now() +} diff --git a/packages/stack-kms-aws/tests/contract.rs b/packages/stack-kms-aws/tests/contract.rs new file mode 100644 index 0000000000..23083dc54f --- /dev/null +++ b/packages/stack-kms-aws/tests/contract.rs @@ -0,0 +1,53 @@ +//! The shared contract suite, run against the AWS KMS backend over an +//! in-process emulation of the AWS KMS JSON API, in both key modes. +//! +//! The suite generates random keys and keyset ids, so recorded exchanges +//! cannot answer it; the emulator does (see `common::KmsEmulator`). The +//! request shapes are pinned by `tests/fixtures.rs`, and the real service +//! is exercised by `tests/localstack.rs`. + +mod common; + +use common::{fixture_client, KmsEmulator, KEY_ARN}; +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor}; +use stack_kms_aws::encryption_context; +use stack_kms_vendor::derive::wrapped_data_key_aad; +use stack_kms_vendor::{contract, KeyVendorBackend}; +use uuid::Uuid; + +#[tokio::test] +async fn aws_kms_meets_the_contract() { + let emulator = KmsEmulator::with_key(KEY_ARN); + contract::run(|| fixture_client(emulator.clone()), KEY_ARN).await; +} + +#[tokio::test] +async fn a_wrapped_key_opens_only_under_its_own_context_and_key() { + let emulator = KmsEmulator::with_key(KEY_ARN); + let kms = fixture_client(emulator.clone()); + let id = Uuid::from_u128(1); + let aad = wrapped_data_key_aad(id, &[1u8; 16], "users/email"); + let (_, wrapped) = kms.generate_wrapped(KEY_ARN, &aad).await.unwrap(); + + let other = wrapped_data_key_aad(id, &[1u8; 16], "users/name"); + assert_ne!(encryption_context(&aad), encryption_context(&other)); + let error = kms.unwrap(KEY_ARN, &wrapped, &other).await.unwrap_err(); + assert!(matches!( + error, + Error::Backend { + kind: BackendErrorKind::InvalidCiphertext, + .. + } + )); + + let error = kms.unwrap("no-such-key", &wrapped, &aad).await.unwrap_err(); + assert!(matches!( + error, + Error::Backend { + kind: BackendErrorKind::InvalidCiphertext, + .. + } + )); + assert_eq!(kms.vendor(), KeyVendor::Aws); + assert!(kms.supports_mode(KeyMode::Derived)); +} diff --git a/packages/stack-kms-aws/tests/credentials.rs b/packages/stack-kms-aws/tests/credentials.rs new file mode 100644 index 0000000000..c30369a281 --- /dev/null +++ b/packages/stack-kms-aws/tests/credentials.rs @@ -0,0 +1,425 @@ +//! The workload identity credential providers and the default chain, +//! against recorded responses of STS, the container endpoint and IMDS. + +#![cfg(not(target_arch = "wasm32"))] + +use std::collections::{HashMap, VecDeque}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use stack_kms::BackendErrorKind; +use stack_kms_aws::credentials::{ + CachedCredentials, CredentialsProvider, DefaultCredentialsSource, EcsCredentials, EnvLookup, + ImdsCredentials, WebIdentityCredentials, +}; +use stack_kms_vendor::{TransportError, VendorRequest, VendorResponse, VendorTransport}; +use web_time::UNIX_EPOCH; + +/// A transport that answers from a script and keeps every request. +#[derive(Clone, Debug, Default)] +struct Scripted { + responses: Arc>>, + sent: Arc>>, +} + +impl Scripted { + fn answering(responses: impl IntoIterator) -> Self { + let scripted = Self::default(); + scripted.responses.lock().unwrap().extend(responses); + scripted + } + + fn sent(&self) -> Vec { + self.sent.lock().unwrap().clone() + } +} + +impl VendorTransport for Scripted { + async fn send(&self, request: VendorRequest) -> Result { + self.sent.lock().unwrap().push(request); + self.responses + .lock() + .unwrap() + .pop_front() + .ok_or_else(|| TransportError("connection refused".into())) + } + + async fn sleep(&self, _duration: Duration) {} +} + +fn env(pairs: &[(&str, &str)]) -> EnvLookup { + let map: HashMap = pairs + .iter() + .map(|(k, v)| ((*k).to_owned(), (*v).to_owned())) + .collect(); + Arc::new(move |name| map.get(name).cloned()) +} + +/// A file holding `contents`, removed on drop. +struct TempFile(std::path::PathBuf); + +impl TempFile { + fn new(name: &str, contents: &str) -> Self { + let path = + std::env::temp_dir().join(format!("stack-kms-aws-{}-{name}", std::process::id())); + std::fs::write(&path, contents).unwrap(); + Self(path) + } + + fn path(&self) -> &str { + self.0.to_str().unwrap() + } +} + +impl Drop for TempFile { + fn drop(&mut self) { + let _ = std::fs::remove_file(&self.0); + } +} + +/// An STS `AssumeRoleWithWebIdentity` response, as STS sends it. +const STS_RESPONSE: &str = r#" + + system:serviceaccount:default:app + sts.amazonaws.com + + arn:aws:sts::111122223333:assumed-role/app/stack-kms-aws + AROACLKWSDQRAOEXAMPLE:stack-kms-aws + + + FwoGZXIvYXdzEXAMPLE&TOKEN + wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY + 2026-11-05T13:00:00Z + ASIAIOSFODNN7EXAMPLE + + app + arn:aws:iam::111122223333:oidc-provider/oidc.eks.us-east-1.amazonaws.com/id/EXAMPLE + + + ad4156e9-bce1-11e2-82e6-6b6efEXAMPLE + +"#; + +/// The JSON credentials the container endpoint and IMDS return. +const JSON_CREDENTIALS: &str = r#"{ + "Code": "Success", + "LastUpdated": "2026-11-05T11:55:00Z", + "Type": "AWS-HMAC", + "AccessKeyId": "ASIAJSONEXAMPLE", + "SecretAccessKey": "json-secret", + "Token": "json-token", + "Expiration": "2026-11-05T18:00:00Z" +}"#; + +fn ok(body: &str) -> VendorResponse { + VendorResponse::new(200, body.as_bytes().to_vec()) +} + +#[tokio::test] +async fn irsa_assumes_the_role_with_the_token_file() { + let token = TempFile::new("irsa-token", "eyJhbGciOiJSUzI1NiJ9.token\n"); + let transport = Scripted::answering([ok(STS_RESPONSE)]); + let provider = WebIdentityCredentials::from_env_lookup( + &env(&[ + ("AWS_WEB_IDENTITY_TOKEN_FILE", token.path()), + ("AWS_ROLE_ARN", "arn:aws:iam::111122223333:role/app"), + ("AWS_ROLE_SESSION_NAME", "my session"), + ]), + transport.clone(), + Some("ap-southeast-2"), + ) + .unwrap(); + let credentials = provider.credentials().await.unwrap(); + assert_eq!(credentials.access_key_id(), "ASIAIOSFODNN7EXAMPLE"); + assert_eq!( + credentials.secret_access_key(), + "wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY" + ); + assert_eq!( + credentials.session_token(), + Some("FwoGZXIvYXdzEXAMPLE&TOKEN") + ); + assert_eq!( + credentials.expires_at(), + Some(UNIX_EPOCH + Duration::from_secs(1_793_883_600)) + ); + + let sent = transport.sent(); + assert_eq!(sent.len(), 1); + assert_eq!(sent[0].method, "POST"); + assert_eq!(sent[0].url, "https://sts.ap-southeast-2.amazonaws.com/"); + assert_eq!( + sent[0].header_value("content-type"), + Some("application/x-www-form-urlencoded; charset=utf-8") + ); + assert!( + sent[0].header_value("authorization").is_none(), + "AssumeRoleWithWebIdentity is not signed" + ); + assert_eq!( + std::str::from_utf8(&sent[0].body).unwrap(), + "Action=AssumeRoleWithWebIdentity\ + &RoleArn=arn%3Aaws%3Aiam%3A%3A111122223333%3Arole%2Fapp\ + &RoleSessionName=my%20session&Version=2011-06-15\ + &WebIdentityToken=eyJhbGciOiJSUzI1NiJ9.token" + ); +} + +#[tokio::test] +async fn irsa_fails_cleanly() { + assert!(WebIdentityCredentials::from_env_lookup( + &env(&[("AWS_ROLE_ARN", "arn")]), + Scripted::default(), + None + ) + .is_none()); + + let provider = WebIdentityCredentials::new( + Scripted::default(), + "arn:aws:iam::111122223333:role/app", + "/no/such/token/file", + None, + ); + let error = provider.credentials().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + + let token = TempFile::new("irsa-denied", "token"); + let denied = VendorResponse::new( + 403, + b"SenderAccessDenied\ + Not authorized to perform sts:AssumeRoleWithWebIdentity" + .to_vec(), + ); + let transport = Scripted::answering([denied]); + let provider = WebIdentityCredentials::new(transport.clone(), "arn", token.path(), None) + .with_sts_endpoint("https://sts.example/"); + let error = provider.credentials().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert_eq!(transport.sent()[0].url, "https://sts.example/"); + + let provider = WebIdentityCredentials::new(Scripted::default(), "arn", token.path(), None); + let error = provider.credentials().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); +} + +#[tokio::test] +async fn the_container_endpoint_gives_credentials() { + let transport = Scripted::answering([ok(JSON_CREDENTIALS)]); + let provider = EcsCredentials::from_env_lookup( + &env(&[( + "AWS_CONTAINER_CREDENTIALS_RELATIVE_URI", + "/v2/credentials/5c4f4e3a", + )]), + transport.clone(), + ) + .unwrap() + .unwrap(); + let credentials = provider.credentials().await.unwrap(); + assert_eq!(credentials.access_key_id(), "ASIAJSONEXAMPLE"); + assert_eq!(credentials.session_token(), Some("json-token")); + let sent = transport.sent(); + assert_eq!(sent[0].method, "GET"); + assert_eq!(sent[0].url, "http://169.254.170.2/v2/credentials/5c4f4e3a"); + assert!(sent[0].header_value("authorization").is_none()); +} + +#[tokio::test] +async fn eks_pod_identity_sends_the_token_from_its_file() { + let token = TempFile::new("pod-identity", "pod-identity-token\n"); + let transport = Scripted::answering([ok(JSON_CREDENTIALS)]); + let provider = EcsCredentials::from_env_lookup( + &env(&[ + ( + "AWS_CONTAINER_CREDENTIALS_FULL_URI", + "http://169.254.170.23/v1/credentials", + ), + ("AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE", token.path()), + ("AWS_CONTAINER_AUTHORIZATION_TOKEN", "ignored"), + ]), + transport.clone(), + ) + .unwrap() + .unwrap(); + provider.credentials().await.unwrap(); + let sent = transport.sent(); + assert_eq!(sent[0].url, "http://169.254.170.23/v1/credentials"); + assert_eq!( + sent[0].header_value("authorization"), + Some("pod-identity-token") + ); + + let transport = Scripted::answering([ok(JSON_CREDENTIALS)]); + let provider = EcsCredentials::from_env_lookup( + &env(&[ + ( + "AWS_CONTAINER_CREDENTIALS_FULL_URI", + "https://creds.example/", + ), + ("AWS_CONTAINER_AUTHORIZATION_TOKEN", "static-token"), + ]), + transport.clone(), + ) + .unwrap() + .unwrap(); + provider.credentials().await.unwrap(); + assert_eq!( + transport.sent()[0].header_value("authorization"), + Some("static-token") + ); + + assert!( + EcsCredentials::from_env_lookup(&env(&[]), Scripted::default()) + .unwrap() + .is_none() + ); + assert!(EcsCredentials::from_env_lookup( + &env(&[( + "AWS_CONTAINER_CREDENTIALS_FULL_URI", + "http://evil.example/creds" + )]), + Scripted::default() + ) + .is_err()); +} + +#[tokio::test] +async fn imds_uses_a_session_token() { + let transport = Scripted::answering([ + ok("imds-session-token"), + ok("app-instance-role\n"), + ok(JSON_CREDENTIALS), + ]); + let provider = ImdsCredentials::from_env_lookup(&env(&[]), transport.clone()).unwrap(); + let credentials = provider.credentials().await.unwrap(); + assert_eq!(credentials.access_key_id(), "ASIAJSONEXAMPLE"); + assert_eq!( + credentials.expires_at(), + Some(UNIX_EPOCH + Duration::from_secs(1_793_901_600)) + ); + let sent = transport.sent(); + assert_eq!(sent.len(), 3); + assert_eq!( + (sent[0].method.as_str(), sent[0].url.as_str()), + ("PUT", "http://169.254.169.254/latest/api/token") + ); + assert_eq!( + sent[0].header_value("x-aws-ec2-metadata-token-ttl-seconds"), + Some("21600") + ); + assert_eq!( + sent[1].url, + "http://169.254.169.254/latest/meta-data/iam/security-credentials/" + ); + assert_eq!( + sent[2].url, + "http://169.254.169.254/latest/meta-data/iam/security-credentials/app-instance-role" + ); + for request in &sent[1..] { + assert_eq!(request.method, "GET"); + assert_eq!( + request.header_value("x-aws-ec2-metadata-token"), + Some("imds-session-token") + ); + } +} + +#[tokio::test] +async fn imds_can_be_moved_or_disabled() { + assert!(ImdsCredentials::from_env_lookup( + &env(&[("AWS_EC2_METADATA_DISABLED", "TRUE")]), + Scripted::default() + ) + .is_none()); + let transport = Scripted::answering([ok("t"), ok("role"), ok(JSON_CREDENTIALS)]); + let provider = ImdsCredentials::from_env_lookup( + &env(&[( + "AWS_EC2_METADATA_SERVICE_ENDPOINT", + "http://[fd00:ec2::254]/", + )]), + transport.clone(), + ) + .unwrap(); + provider.credentials().await.unwrap(); + assert_eq!( + transport.sent()[0].url, + "http://[fd00:ec2::254]/latest/api/token" + ); +} + +#[tokio::test] +async fn imds_failures_map_to_kinds() { + // Off EC2: no answer. + let provider = ImdsCredentials::new(Scripted::default(), "http://169.254.169.254"); + let error = provider.credentials().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); + // No instance profile: an empty role list, or a 404. + let provider = ImdsCredentials::new(Scripted::answering([ok("t"), ok("")]), "http://imds"); + let error = provider.credentials().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + let provider = ImdsCredentials::new( + Scripted::answering([ok("t"), VendorResponse::new(404, vec![])]), + "http://imds", + ); + let error = provider.credentials().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::NotFound); + // IMDSv1 only (token refused). + let provider = ImdsCredentials::new( + Scripted::answering([VendorResponse::new(403, vec![])]), + "http://imds", + ); + let error = provider.credentials().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); +} + +#[test] +fn the_chain_picks_the_source_the_environment_names() { + let pick = |pairs: &[(&str, &str)]| { + DefaultCredentialsSource::from_env_lookup(&env(pairs), Scripted::default(), None) + }; + assert!(matches!( + pick(&[ + ("AWS_ACCESS_KEY_ID", "AKID"), + ("AWS_SECRET_ACCESS_KEY", "s"), + ("AWS_ROLE_ARN", "arn"), + ("AWS_WEB_IDENTITY_TOKEN_FILE", "/t"), + ]), + Ok(DefaultCredentialsSource::Static(_)) + )); + assert!(matches!( + pick(&[ + ("AWS_ROLE_ARN", "arn"), + ("AWS_WEB_IDENTITY_TOKEN_FILE", "/t"), + ("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI", "/v2"), + ]), + Ok(DefaultCredentialsSource::WebIdentity(_)) + )); + assert!(matches!( + pick(&[("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI", "/v2")]), + Ok(DefaultCredentialsSource::Ecs(_)) + )); + assert!(matches!(pick(&[]), Ok(DefaultCredentialsSource::Imds(_)))); + let error = pick(&[("AWS_EC2_METADATA_DISABLED", "true")]).unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert!(pick(&[("AWS_CONTAINER_CREDENTIALS_FULL_URI", "http://evil.example/")]).is_err()); +} + +#[tokio::test] +async fn the_cached_chain_fetches_once_until_near_expiry() { + let transport = Scripted::answering([ok("t"), ok("role"), ok(JSON_CREDENTIALS)]); + let source = + DefaultCredentialsSource::from_env_lookup(&env(&[]), transport.clone(), None).unwrap(); + // 2026-11-05T12:00:00Z; the credentials expire at 18:00. + let cache = CachedCredentials::new(source) + .with_clock(Arc::new(|| UNIX_EPOCH + Duration::from_secs(1_793_880_000))); + for _ in 0..3 { + assert_eq!( + cache.credentials().await.unwrap().access_key_id(), + "ASIAJSONEXAMPLE" + ); + } + assert_eq!( + transport.sent().len(), + 3, + "one IMDS fetch: token, role, credentials" + ); +} diff --git a/packages/stack-kms-aws/tests/fixtures.rs b/packages/stack-kms-aws/tests/fixtures.rs new file mode 100644 index 0000000000..81fc1b0be1 --- /dev/null +++ b/packages/stack-kms-aws/tests/fixtures.rs @@ -0,0 +1,212 @@ +//! Each AWS KMS call and each error mapping, against recorded HTTP +//! exchanges (`tests/fixtures/*.json`). +//! +//! Every fixture request is signed at 2026-11-05T12:00:00Z with the AWS +//! documentation example key, so its bytes, signature included, are fixed. +//! The SigV4 algorithm itself is tested against the AWS test suite in +//! `src/sigv4.rs`; these fixtures pin the KMS request shape on top of it. + +mod common; + +use common::{fixture_client, load, Fixture, FixtureResponse, Replay, KEY_ARN}; +use serde::Deserialize; +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor}; +use stack_kms_aws::{AwsKms, AwsKmsConfig, Credentials, CredentialsError, CredentialsProvider}; +use stack_kms_vendor::derive::{record_key_aad, wrapped_data_key_aad}; +use stack_kms_vendor::{ + KeyVendorBackend, TransportError, VendorRequest, VendorResponse, VendorTransport, +}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; + +const KEYSET: Uuid = Uuid::from_u128(0x0f1e2d3c_4b5a_6978_8796_a5b4c3d2e1f0); + +fn record_aad() -> Vec { + record_key_aad(KEYSET, KeyVendor::Aws, KeyMode::Wrapped) +} + +fn kind_of(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "aws-kms"); + kind + } + other => panic!("not a backend error: {other:?}"), + } +} + +/// The ciphertext blob of a fixture's response. +fn blob_of(fixture: &str) -> Vec { + let fixture: Fixture = load(fixture); + let blob = fixture.response.body["CiphertextBlob"].as_str().unwrap(); + ::decode_vec(blob).unwrap() +} + +#[tokio::test] +async fn generate_data_key() { + let transport = Replay::new().expect(load("generate_data_key.json")); + let kms = fixture_client(transport); + let aad = wrapped_data_key_aad(KEYSET, &[7u8; 16], "users/email"); + let (key, wrapped) = kms.generate_wrapped(KEY_ARN, &aad).await.unwrap(); + let expected: Vec = (0u8..32).collect(); + assert_eq!(key.risky_ref().as_slice(), expected.as_slice()); + assert_eq!(wrapped, blob_of("generate_data_key.json")); +} + +#[tokio::test] +async fn encrypt() { + let transport = Replay::new().expect(load("encrypt.json")); + let kms = fixture_client(transport); + let wrapped = kms + .wrap(KEY_ARN, &Protected::new([0x42u8; 32]), &record_aad()) + .await + .unwrap(); + assert_eq!(wrapped, blob_of("encrypt.json")); +} + +#[tokio::test] +async fn decrypt() { + let transport = Replay::new().expect(load("decrypt.json")); + let kms = fixture_client(transport); + let key = kms + .unwrap(KEY_ARN, &blob_of("encrypt.json"), &record_aad()) + .await + .unwrap(); + assert_eq!(key.risky_ref(), &[0x42u8; 32]); +} + +#[tokio::test] +async fn decrypt_with_a_session_token() { + let transport = Replay::new().expect(load("decrypt_with_session_token.json")); + let kms = AwsKms::new( + AwsKmsConfig::new("us-east-1").unwrap(), + transport, + Credentials::new( + "ASIAEXAMPLE", + "wJalrXUtnFEMI/K7MDENG+bPxRfiCYEXAMPLEKEY", + Some("IQoJb3JpZ2luX2VjEXAMPLESESSIONTOKEN".to_owned()), + ), + ) + .with_clock(common::fixture_clock()); + let key = kms + .unwrap(KEY_ARN, &blob_of("encrypt.json"), &record_aad()) + .await + .unwrap(); + assert_eq!(key.risky_ref(), &[0x42u8; 32]); +} + +#[derive(Deserialize)] +struct ErrorCase { + name: String, + response: FixtureResponse, + kind: String, +} + +#[tokio::test] +async fn every_recorded_error_maps_to_its_kind() { + let cases: Vec = load("errors.json"); + assert!(cases.len() >= 10); + for case in cases { + let kms = fixture_client(Replay::new().answer(case.response.to_response())); + let error = kms + .unwrap(KEY_ARN, b"blob", &record_aad()) + .await + .unwrap_err(); + let message = format!("{error:?}"); + assert_eq!( + format!("{:?}", kind_of(error)), + case.kind, + "{}: {message}", + case.name + ); + } +} + +/// A transport that never gets a response. +struct Down; + +impl VendorTransport for Down { + async fn send(&self, _request: VendorRequest) -> Result { + Err(TransportError("connection refused".into())) + } + + async fn sleep(&self, _duration: std::time::Duration) {} +} + +#[tokio::test] +async fn no_response_is_unavailable() { + let error = fixture_client(Down) + .unwrap(KEY_ARN, b"blob", &record_aad()) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); +} + +struct NoCredentials; + +impl CredentialsProvider for NoCredentials { + async fn credentials(&self) -> Result { + Err(CredentialsError::access_denied("none configured")) + } +} + +#[tokio::test] +async fn missing_credentials_are_access_denied_and_send_nothing() { + let kms = AwsKms::new( + AwsKmsConfig::new("us-east-1").unwrap(), + Replay::new(), + NoCredentials, + ); + let error = kms + .unwrap(KEY_ARN, b"blob", &record_aad()) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::AccessDenied); +} + +#[tokio::test] +async fn an_unreadable_success_is_unavailable() { + for body in [ + b"not json".to_vec(), + br#"{"KeyId":"k"}"#.to_vec(), + br#"{"Plaintext":"not base64!"}"#.to_vec(), + ] { + let kms = fixture_client(Replay::new().answer(VendorResponse::new(200, body))); + let error = kms + .unwrap(KEY_ARN, b"blob", &record_aad()) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); + } + let kms = fixture_client( + Replay::new().answer(VendorResponse::new(200, br#"{"KeyId":"k"}"#.to_vec())), + ); + let error = kms + .wrap(KEY_ARN, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); +} + +#[tokio::test] +async fn a_plaintext_that_is_not_a_256_bit_key_is_refused() { + // 16 bytes: an AES_128 data key, not one this crate asked for. + let body = br#"{"KeyId":"k","Plaintext":"AAECAwQFBgcICQoLDA0ODw=="}"#.to_vec(); + let kms = fixture_client(Replay::new().answer(VendorResponse::new(200, body))); + let error = kms + .unwrap(KEY_ARN, b"blob", &record_aad()) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); +} + +#[tokio::test] +async fn the_backend_reports_aws_and_sleeps_on_its_transport() { + let emulator = common::KmsEmulator::default(); + let kms = fixture_client(emulator.clone()); + assert_eq!(kms.vendor(), KeyVendor::Aws); + assert!(kms.supports_mode(KeyMode::Wrapped)); + assert!(kms.supports_mode(KeyMode::Derived)); + kms.sleep(std::time::Duration::from_millis(50)).await; + assert_eq!(emulator.sleeps(), [std::time::Duration::from_millis(50)]); +} diff --git a/packages/stack-kms-aws/tests/fixtures/decrypt.json b/packages/stack-kms-aws/tests/fixtures/decrypt.json new file mode 100644 index 0000000000..366a5889ef --- /dev/null +++ b/packages/stack-kms-aws/tests/fixtures/decrypt.json @@ -0,0 +1,55 @@ +{ + "request": { + "body": { + "CiphertextBlob": "AQICAAoLDA0ODxAREhMUFRYXGBkaGxwdHh8gISIjJCUmJygpKissLS4vMDEyMzQ1Njc4OTo7PD0+P0BBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWltcXV5fYGFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6e3x9fn+AgYKDhIWGh4iJiouMjY6PkJGSk5SVlpeYmZqbnJ2enw==", + "EncryptionContext": { + "cipherstash:aad": "BQAAAAAAAAAlAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Iva2V5c2V0LXJlY29yZC1rZXkvdjEJAAAAAAAAAGluZGV4LWtleRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8AEAAAAAAAAAAQEAAAAAAAAAAQ", + "cipherstash:keyset-id": "0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0", + "cipherstash:purpose": "index-key" + }, + "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab" + }, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amz-target", + "TrentService.Decrypt" + ], + [ + "host", + "kms.us-east-1.amazonaws.com" + ], + [ + "x-amz-date", + "20261105T120000Z" + ], + [ + "authorization", + "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20261105/us-east-1/kms/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=910c41000bf1f6858f446685c53520113b4bc50962606b4edc4c74172d98b438" + ] + ], + "method": "POST", + "url": "https://kms.us-east-1.amazonaws.com/" + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "8a3e7b2c-5f1d-4c9a-9e0b-1d2c3b4a5f60" + ] + ], + "body": { + "EncryptionAlgorithm": "SYMMETRIC_DEFAULT", + "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab", + "Plaintext": "QkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=" + } + } +} diff --git a/packages/stack-kms-aws/tests/fixtures/decrypt_with_session_token.json b/packages/stack-kms-aws/tests/fixtures/decrypt_with_session_token.json new file mode 100644 index 0000000000..15a5165cb0 --- /dev/null +++ b/packages/stack-kms-aws/tests/fixtures/decrypt_with_session_token.json @@ -0,0 +1,59 @@ +{ + "request": { + "body": { + "CiphertextBlob": "AQICAAoLDA0ODxAREhMUFRYXGBkaGxwdHh8gISIjJCUmJygpKissLS4vMDEyMzQ1Njc4OTo7PD0+P0BBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWltcXV5fYGFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6e3x9fn+AgYKDhIWGh4iJiouMjY6PkJGSk5SVlpeYmZqbnJ2enw==", + "EncryptionContext": { + "cipherstash:aad": "BQAAAAAAAAAlAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Iva2V5c2V0LXJlY29yZC1rZXkvdjEJAAAAAAAAAGluZGV4LWtleRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8AEAAAAAAAAAAQEAAAAAAAAAAQ", + "cipherstash:keyset-id": "0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0", + "cipherstash:purpose": "index-key" + }, + "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab" + }, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amz-target", + "TrentService.Decrypt" + ], + [ + "host", + "kms.us-east-1.amazonaws.com" + ], + [ + "x-amz-date", + "20261105T120000Z" + ], + [ + "x-amz-security-token", + "IQoJb3JpZ2luX2VjEXAMPLESESSIONTOKEN" + ], + [ + "authorization", + "AWS4-HMAC-SHA256 Credential=ASIAEXAMPLE/20261105/us-east-1/kms/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token;x-amz-target, Signature=87529610061f62243d2fc0e7ed2c5694c5a79d6ca31823d4757f9f3acb61f059" + ] + ], + "method": "POST", + "url": "https://kms.us-east-1.amazonaws.com/" + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "8a3e7b2c-5f1d-4c9a-9e0b-1d2c3b4a5f60" + ] + ], + "body": { + "EncryptionAlgorithm": "SYMMETRIC_DEFAULT", + "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab", + "Plaintext": "QkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=" + } + } +} diff --git a/packages/stack-kms-aws/tests/fixtures/encrypt.json b/packages/stack-kms-aws/tests/fixtures/encrypt.json new file mode 100644 index 0000000000..947f0b84cb --- /dev/null +++ b/packages/stack-kms-aws/tests/fixtures/encrypt.json @@ -0,0 +1,55 @@ +{ + "request": { + "body": { + "EncryptionContext": { + "cipherstash:aad": "BQAAAAAAAAAlAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Iva2V5c2V0LXJlY29yZC1rZXkvdjEJAAAAAAAAAGluZGV4LWtleRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8AEAAAAAAAAAAQEAAAAAAAAAAQ", + "cipherstash:keyset-id": "0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0", + "cipherstash:purpose": "index-key" + }, + "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab", + "Plaintext": "QkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=" + }, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amz-target", + "TrentService.Encrypt" + ], + [ + "host", + "kms.us-east-1.amazonaws.com" + ], + [ + "x-amz-date", + "20261105T120000Z" + ], + [ + "authorization", + "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20261105/us-east-1/kms/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=46054a72a4c907b68f5c7eca6952215b242082b8e5ec1821a434c1db81bc78ad" + ] + ], + "method": "POST", + "url": "https://kms.us-east-1.amazonaws.com/" + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "8a3e7b2c-5f1d-4c9a-9e0b-1d2c3b4a5f60" + ] + ], + "body": { + "CiphertextBlob": "AQICAAoLDA0ODxAREhMUFRYXGBkaGxwdHh8gISIjJCUmJygpKissLS4vMDEyMzQ1Njc4OTo7PD0+P0BBQkNERUZHSElKS0xNTk9QUVJTVFVWV1hZWltcXV5fYGFiY2RlZmdoaWprbG1ub3BxcnN0dXZ3eHl6e3x9fn+AgYKDhIWGh4iJiouMjY6PkJGSk5SVlpeYmZqbnJ2enw==", + "EncryptionAlgorithm": "SYMMETRIC_DEFAULT", + "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab" + } + } +} diff --git a/packages/stack-kms-aws/tests/fixtures/errors.json b/packages/stack-kms-aws/tests/fixtures/errors.json new file mode 100644 index 0000000000..d9c1a2aa16 --- /dev/null +++ b/packages/stack-kms-aws/tests/fixtures/errors.json @@ -0,0 +1,222 @@ +[ + { + "name": "ThrottlingException", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": { + "__type": "ThrottlingException", + "message": "You have exceeded the rate at which you may call KMS. Reduce the frequency of your calls." + } + }, + "kind": "Throttled" + }, + { + "name": "AccessDeniedException", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": { + "__type": "AccessDeniedException", + "Message": "User: arn:aws:iam::111122223333:user/app is not authorized to perform: kms:Decrypt on resource: arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab because no identity-based policy allows the kms:Decrypt action" + } + }, + "kind": "AccessDenied" + }, + { + "name": "NotFoundException", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": { + "__type": "NotFoundException", + "message": "Key 'arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab' does not exist" + } + }, + "kind": "NotFound" + }, + { + "name": "InvalidCiphertextException", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": { + "__type": "InvalidCiphertextException" + } + }, + "kind": "InvalidCiphertext" + }, + { + "name": "IncorrectKeyException", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": { + "__type": "IncorrectKeyException", + "message": "The key ID in the request does not identify a CMK that can perform this operation." + } + }, + "kind": "InvalidCiphertext" + }, + { + "name": "KMSInternalException", + "response": { + "status": 500, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": { + "__type": "KMSInternalException", + "message": "An internal error occurred." + } + }, + "kind": "Unavailable" + }, + { + "name": "DependencyTimeoutException", + "response": { + "status": 503, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": { + "__type": "DependencyTimeoutException", + "message": "The system timed out while trying to fulfill the request." + } + }, + "kind": "Unavailable" + }, + { + "name": "a bare 503 from a load balancer", + "response": { + "status": 503, + "headers": [ + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": "Service Unavailable" + }, + "kind": "Unavailable" + }, + { + "name": "UnrecognizedClientException", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": { + "__type": "UnrecognizedClientException", + "message": "The security token included in the request is invalid." + } + }, + "kind": "AccessDenied" + }, + { + "name": "the code in the x-amzn-ErrorType header only", + "response": { + "status": 400, + "headers": [ + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ], + [ + "x-amzn-ErrorType", + "ThrottlingException:http://internal.amazon.com/coral/com.amazon.coral.availability/" + ] + ], + "body": "" + }, + "kind": "Throttled" + }, + { + "name": "a namespaced __type", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "0f9c1b2a-3d4e-4f50-8a6b-7c8d9e0f1a2b" + ] + ], + "body": { + "__type": "com.amazonaws.kms#NotFoundException", + "message": "Alias 'alias/missing' is not found." + } + }, + "kind": "NotFound" + } +] diff --git a/packages/stack-kms-aws/tests/fixtures/generate_data_key.json b/packages/stack-kms-aws/tests/fixtures/generate_data_key.json new file mode 100644 index 0000000000..a52b0b1822 --- /dev/null +++ b/packages/stack-kms-aws/tests/fixtures/generate_data_key.json @@ -0,0 +1,57 @@ +{ + "request": { + "body": { + "EncryptionContext": { + "cipherstash:aad": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAABwcHBwcHBwcHBwcHBwcHBwsAAAAAAAAAdXNlcnMvZW1haWw", + "cipherstash:descriptor": "users/email", + "cipherstash:keyset-id": "0f1e2d3c-4b5a-6978-8796-a5b4c3d2e1f0", + "cipherstash:purpose": "data-key" + }, + "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab", + "KeySpec": "AES_256" + }, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amz-target", + "TrentService.GenerateDataKey" + ], + [ + "host", + "kms.us-east-1.amazonaws.com" + ], + [ + "x-amz-date", + "20261105T120000Z" + ], + [ + "authorization", + "AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/20261105/us-east-1/kms/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=53c2c392adc7c80c76e0a0b77b8061735637915c738d82d7c40a3d9bf9874de1" + ] + ], + "method": "POST", + "url": "https://kms.us-east-1.amazonaws.com/" + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/x-amz-json-1.1" + ], + [ + "x-amzn-requestid", + "8a3e7b2c-5f1d-4c9a-9e0b-1d2c3b4a5f60" + ] + ], + "body": { + "CiphertextBlob": "AQIDAGRlZmdoaWprbG1ub3BxcnN0dXZ3eHl6e3x9fn+AgYKDhIWGh4iJiouMjY6PkJGSk5SVlpeYmZqbnJ2en6ChoqOkpaanqKmqq6ytrq+wsbKztLW2t7i5uru8vb6/wMHCw8TFxsfIycrLzM3Oz9DR0tPU1dbX2Nna29zd3t/g4eLj5OXm5+jp6uvs7e7v8PHy8/T19vf4+Q==", + "KeyId": "arn:aws:kms:us-east-1:111122223333:key/1234abcd-12ab-34cd-56ef-1234567890ab", + "KeyMaterialId": "0d4a6b1c2e3f40516273849506a7b8c9d0e1f2a3b4c5d6e7f8091a2b3c4d5e6f", + "Plaintext": "AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8=" + } + } +} diff --git a/packages/stack-kms-aws/tests/localstack.rs b/packages/stack-kms-aws/tests/localstack.rs new file mode 100644 index 0000000000..ad043b2059 --- /dev/null +++ b/packages/stack-kms-aws/tests/localstack.rs @@ -0,0 +1,98 @@ +//! The AWS KMS backend against LocalStack, over the native transport. +//! +//! Runs only when `STACK_KMS_AWS_LOCALSTACK_ENDPOINT` is set (for example +//! `http://localhost:4566`); otherwise each test passes at once and says it +//! was skipped. CI runs LocalStack as a service and sets the variable. +//! +//! ```sh +//! docker run -d --rm -p 4566:4566 -e SERVICES=kms localstack/localstack:4.4 +//! STACK_KMS_AWS_LOCALSTACK_ENDPOINT=http://localhost:4566 \ +//! cargo nextest run -p stack-kms-aws --all-features --test localstack +//! ``` + +#![cfg(not(target_arch = "wasm32"))] + +use serde_json::Value; +use stack_kms::{BackendErrorKind, Error}; +use stack_kms_aws::{AwsKms, AwsKmsConfig, Credentials}; +use stack_kms_vendor::derive::wrapped_data_key_aad; +use stack_kms_vendor::{contract, HttpVendorTransport, KeyVendorBackend}; +use uuid::Uuid; +use zeroize::Zeroizing; + +const ENDPOINT_VAR: &str = "STACK_KMS_AWS_LOCALSTACK_ENDPOINT"; + +type Client = AwsKms; + +fn client(endpoint: &str) -> Client { + let config = AwsKmsConfig::new("us-east-1") + .unwrap() + .with_endpoint(endpoint) + .unwrap(); + // LocalStack accepts any credentials; these are its documented ones. + AwsKms::new( + config, + HttpVendorTransport::new().unwrap(), + Credentials::new("test", "test", None), + ) +} + +fn endpoint() -> Option { + let endpoint = std::env::var(ENDPOINT_VAR).ok().filter(|e| !e.is_empty()); + if endpoint.is_none() { + eprintln!("skipped: set {ENDPOINT_VAR} to run against LocalStack"); + } + endpoint +} + +/// A new symmetric AWS KMS key in LocalStack; its ARN. +async fn create_key(kms: &Client) -> String { + let body = br#"{"KeySpec":"SYMMETRIC_DEFAULT","KeyUsage":"ENCRYPT_DECRYPT","Description":"stack-kms-aws test"}"#; + let created: Value = kms + .send_json("CreateKey", Zeroizing::new(body.to_vec())) + .await + .unwrap(); + created["KeyMetadata"]["Arn"].as_str().unwrap().to_owned() +} + +#[tokio::test] +async fn aws_kms_meets_the_contract_on_localstack() { + let Some(endpoint) = endpoint() else { return }; + let key_arn = create_key(&client(&endpoint)).await; + contract::run(|| client(&endpoint), &key_arn).await; +} + +#[tokio::test] +async fn localstack_errors_map_to_kinds() { + let Some(endpoint) = endpoint() else { return }; + let kms = client(&endpoint); + let key_arn = create_key(&kms).await; + let aad = wrapped_data_key_aad(Uuid::new_v4(), &[3u8; 16], "users/email"); + let (_, wrapped) = kms.generate_wrapped(&key_arn, &aad).await.unwrap(); + + let kind = |error: Error| match error { + Error::Backend { kind, .. } => kind, + other => panic!("{other:?}"), + }; + + // Another encryption context. + let other = wrapped_data_key_aad(Uuid::new_v4(), &[3u8; 16], "users/email"); + let error = kms.unwrap(&key_arn, &wrapped, &other).await.unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::InvalidCiphertext); + + // A damaged ciphertext. (LocalStack answers bytes that are not its + // blob format at all with HTTP 500, not InvalidCiphertextException, so + // the last byte of a real blob is changed instead. The fixtures cover + // the AWS answer.) + let mut damaged = wrapped.clone(); + if let Some(last) = damaged.last_mut() { + *last ^= 1; + } + let error = kms.unwrap(&key_arn, &damaged, &aad).await.unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::InvalidCiphertext); + + // A key that does not exist. + let missing = "arn:aws:kms:us-east-1:000000000000:key/00000000-0000-0000-0000-000000000000"; + let error = kms.generate_wrapped(missing, &aad).await.unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::NotFound); +} diff --git a/packages/stack-kms-azure/Cargo.toml b/packages/stack-kms-azure/Cargo.toml new file mode 100644 index 0000000000..ae7260eef5 --- /dev/null +++ b/packages/stack-kms-azure/Cargo.toml @@ -0,0 +1,54 @@ +[package] +name = "stack-kms-azure" +description = "The Azure Key Vault and Managed HSM key vendor backend for Stack Encrypt: a hand-written Key Vault client with Entra ID client secret, workload identity and managed identity credentials, for native Rust and the WASI guest" +version = "0.1.0" +edition.workspace = true +authors.workspace = true +repository.workspace = true +homepage.workspace = true +keywords.workspace = true +categories.workspace = true +license-file = "LICENSE" +# Publishable, in the `stack-kms-vendor` version group with the vendor seam it +# implements. release-plz.toml keeps `release = false` until the first version +# is on crates.io (published by hand; see stack-kms-vendor's Cargo.toml). + +[features] +default = ["http"] +# `AzureKeyVault::from_env`: the native transport (`HttpVendorTransport`) and +# the default credentials over it. Native only; on wasm32 it adds nothing. +http = ["stack-kms-vendor/http"] + +[dependencies] +stack-kms = { path = "../stack-kms", version = "0.2.0", default-features = false } +stack-kms-vendor = { path = "../stack-kms-vendor", version = "0.1.0" } + +vitaminc = { workspace = true, features = ["protected"] } +base64ct = { version = "1.7", features = ["alloc"] } +futures = "0.3.25" +serde = { workspace = true } +serde_json = { workspace = true } +thiserror = { workspace = true } +url = { workspace = true } +web-time = { workspace = true } +zeroize = { workspace = true } + +[dev-dependencies] +stack-kms-vendor = { path = "../stack-kms-vendor", features = ["test-support", "http"] } +stack-encrypt = { path = "../stack-encrypt", default-features = false } +# The fixture emulator only: RSA-OAEP-256 unwrap with a TEST-ONLY key, and +# AES for an RFC 3394 key wrap. The crate itself does no RSA (Azure does). +rsa = { version = "0.9.10", default-features = false, features = ["std", "sha2"] } +rand_core = { version = "0.6.4", features = ["getrandom"] } +aes = "0.8.4" +tokio = { workspace = true, features = ["rt", "macros", "time"] } +uuid = { workspace = true } +vitaminc = { workspace = true, features = ["protected", "random"] } + +# Builds `AzureKeyVault::from_env`, which needs the native transport. +[[test]] +name = "live" +required-features = ["http"] + +[package.metadata.docs.rs] +all-features = true diff --git a/packages/stack-kms-azure/LICENSE b/packages/stack-kms-azure/LICENSE new file mode 100644 index 0000000000..2cbd67a66d --- /dev/null +++ b/packages/stack-kms-azure/LICENSE @@ -0,0 +1,96 @@ +# PolyForm Internal Use License 1.0.0 + + + +## Acceptance + +In order to get any license under these terms, you must agree +to them as both strict obligations and conditions to all +your licenses. + +## Copyright License + +The licensor grants you a copyright license for the software +to do everything you might do with the software that would +otherwise infringe the licensor's copyright in it for any +permitted purpose. However, you may only make changes or +new works based on the software according to [Changes and New +Works License](#changes-and-new-works-license), and you may +not distribute the software. + +## Changes and New Works License + +The licensor grants you an additional copyright license to +make changes and new works based on the software for any +permitted purpose. + +## Patent License + +The licensor grants you a patent license for the software that +covers patent claims the licensor can license, or becomes able +to license, that you would infringe by using the software. + +## Fair Use + +You may have "fair use" rights for the software under the +law. These terms do not limit them. + +## Internal Business Use + +Use of the software for the internal business operations of +you and your company is use for a permitted purpose. + +## No Other Rights + +These terms do not allow you to sublicense or transfer any of +your licenses to anyone else, or prevent the licensor from +granting licenses to anyone else. These terms do not imply +any other licenses. + +## Patent Defense + +If you make any written claim that the software infringes or +contributes to infringement of any patent, your patent license +for the software granted under these terms ends immediately. If +your company makes such a claim, your patent license ends +immediately for work on behalf of your company. + +## Violations + +The first time you are notified in writing that you have +violated any of these terms, or done anything with the software +not covered by your licenses, your licenses can nonetheless +continue if you come into full compliance with these terms, +and take practical steps to correct past violations, within +32 days of receiving notice. Otherwise, all your licenses +end immediately. + +## No Liability + +***As far as the law allows, the software comes as is, without +any warranty or condition, and the licensor will not be liable +to you for any damages arising out of these terms or the use +or nature of the software, under any kind of legal claim.*** + +## Definitions + +The **licensor** is the individual or entity offering these +terms, and the **software** is the software the licensor makes +available under these terms. + +**You** refers to the individual or entity agreeing to these +terms. + +**Your company** is any legal entity, sole proprietorship, +or other kind of organization that you work for, plus all +organizations that have control over, are under the control of, +or are under common control with that organization. **Control** +means ownership of substantially all the assets of an entity, +or the power to direct its management and policies by vote, +contract, or otherwise. Control can be direct or indirect. + +**Your licenses** are all the licenses granted to you for the +software under these terms. + +**Use** means anything you do with the software requiring one +of your licenses. diff --git a/packages/stack-kms-azure/src/auth/client_secret.rs b/packages/stack-kms-azure/src/auth/client_secret.rs new file mode 100644 index 0000000000..de28889510 --- /dev/null +++ b/packages/stack-kms-azure/src/auth/client_secret.rs @@ -0,0 +1,257 @@ +//! The Entra ID token endpoint: a service principal's client secret +//! (OAuth 2.0 client credentials), and the shared request of every login +//! that posts to it. + +use std::fmt; + +use stack_kms_vendor::{VendorRequest, VendorTransport}; +use zeroize::Zeroizing; + +use super::{read_token_response, AccessToken, AuthError, TokenProvider}; +use crate::time::{system_clock, Clock}; + +/// The Entra ID host of the public cloud. `AZURE_AUTHORITY_HOST` (the Azure +/// SDK name) changes it, for example to `https://login.chinacloudapi.cn`. +pub const DEFAULT_AUTHORITY_HOST: &str = "https://login.microsoftonline.com"; + +/// The `client_assertion_type` of a federated token (RFC 7523). +pub const JWT_BEARER_ASSERTION_TYPE: &str = + "urn:ietf:params:oauth:client-assertion-type:jwt-bearer"; + +/// A tenant id: a GUID or a domain name, `[A-Za-z0-9.-]`, so that it cannot +/// change the token endpoint path. +pub fn is_tenant_id(text: &str) -> bool { + !text.is_empty() + && text.len() <= 256 + && text + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-')) + && !text.starts_with('.') +} + +/// An Entra ID application: the tenant, the client id and the authority +/// host. Shared by the logins that post to the token endpoint. +#[derive(Clone, Debug)] +pub(crate) struct EntraApp { + tenant_id: String, + client_id: String, + authority_host: String, +} + +impl EntraApp { + pub(crate) fn new(tenant_id: &str, client_id: &str) -> Result { + if !is_tenant_id(tenant_id) { + return Err(AuthError::access_denied(format!( + "{tenant_id:?} is not an Entra ID tenant id" + ))); + } + if client_id.is_empty() { + return Err(AuthError::access_denied("the client id is empty")); + } + Ok(Self { + tenant_id: tenant_id.to_owned(), + client_id: client_id.to_owned(), + authority_host: DEFAULT_AUTHORITY_HOST.to_owned(), + }) + } + + pub(crate) fn with_authority_host(mut self, host: String) -> Result { + let parsed = url::Url::parse(&host).map_err(|_| { + AuthError::access_denied(format!("{host:?} is not an authority host URL")) + })?; + if parsed.scheme() != "https" + || parsed.host().is_none() + || parsed.path() != "/" + || parsed.query().is_some() + || parsed.fragment().is_some() + || !parsed.username().is_empty() + || parsed.password().is_some() + { + return Err(AuthError::access_denied(format!( + "{host:?} is not an HTTPS authority host with no path" + ))); + } + self.authority_host = parsed.as_str().trim_end_matches('/').to_owned(); + Ok(self) + } + + pub(crate) fn client_id(&self) -> &str { + &self.client_id + } + + pub(crate) fn tenant_id(&self) -> &str { + &self.tenant_id + } + + pub(crate) fn token_url(&self) -> String { + format!( + "{}/{}/oauth2/v2.0/token", + self.authority_host, self.tenant_id + ) + } + + /// Post the client credentials form `extra` (after `grant_type`, + /// `client_id` and `scope`) and read the token. `source` names the login + /// in errors. + pub(crate) async fn post( + &self, + transport: &T, + clock: &Clock, + source: &str, + scope: &str, + extra: &[(&str, &str)], + ) -> Result { + let body = { + let mut form = url::form_urlencoded::Serializer::new(String::new()); + let _ = form + .append_pair("grant_type", "client_credentials") + .append_pair("client_id", &self.client_id) + .append_pair("scope", scope); + for (name, value) in extra { + let _ = form.append_pair(name, value); + } + Zeroizing::new(form.finish().into_bytes()) + }; + let mut request = VendorRequest::new("POST", self.token_url()) + .header("content-type", "application/x-www-form-urlencoded"); + request.body = body; + let response = transport.send(request).await.map_err(|error| { + AuthError::unavailable(format!( + "the Entra ID token endpoint got no response: {error}" + )) + })?; + read_token_response(source, &response, clock()) + } +} + +/// A service principal's client secret, exchanged at the Entra ID token +/// endpoint for an access token (OAuth 2.0 client credentials). Entra ID +/// issues the token for about an hour. Wrap it in a +/// [`CachedToken`](super::CachedToken). +/// +/// The secret is wiped on drop and never printed. Everywhere: the WASI guest +/// can use it if the host passes the secret, but ADR-0011 has the host +/// supply the token. +pub struct ClientSecretLogin { + app: EntraApp, + secret: Zeroizing, + transport: T, + clock: Clock, +} + +impl fmt::Debug for ClientSecretLogin { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("ClientSecretLogin") + .field("tenant_id", &self.app.tenant_id()) + .field("client_id", &self.app.client_id()) + .field("client_secret", &"") + .finish_non_exhaustive() + } +} + +impl ClientSecretLogin { + /// Log in as `client_id` of `tenant_id` with `secret`, through + /// `transport`. Refuses a tenant id that is not a GUID or domain name. + pub fn new( + tenant_id: &str, + client_id: &str, + secret: impl Into, + transport: T, + ) -> Result { + let secret = Zeroizing::new(secret.into()); + if secret.is_empty() { + return Err(AuthError::access_denied("the client secret is empty")); + } + Ok(Self { + app: EntraApp::new(tenant_id, client_id)?, + secret, + transport, + clock: system_clock(), + }) + } + + /// Post to the Entra ID host `host` (`https://login.microsoftonline.us`) + /// instead of [`DEFAULT_AUTHORITY_HOST`]. + pub fn with_authority_host(mut self, host: impl Into) -> Result { + self.app = self.app.with_authority_host(host.into())?; + Ok(self) + } + + /// Read the time from `clock` (for an `expires_on` response). + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = clock; + self + } + + /// The token endpoint URL. + pub fn token_url(&self) -> String { + self.app.token_url() + } + + /// The transport. + pub fn transport(&self) -> &T { + &self.transport + } +} + +impl TokenProvider for ClientSecretLogin { + async fn token(&self, scope: &str) -> Result { + self.app + .post( + &self.transport, + &self.clock, + "the client secret login", + scope, + &[("client_secret", self.secret.as_str())], + ) + .await + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn tenant_ids_are_checked() { + for good in [ + "00000000-0000-0000-0000-000000000000", + "contoso.onmicrosoft.com", + "organizations", + ] { + assert!(is_tenant_id(good), "{good}"); + } + for bad in ["", "a/b", "a?b", "a b", ".", "..", "a%2fb", "a#b"] { + assert!(!is_tenant_id(bad), "{bad:?}"); + } + } + + #[test] + fn the_login_is_checked_and_hides_the_secret() { + let login = ClientSecretLogin::new("tenant", "client", "the-secret", ()).unwrap(); + assert_eq!( + login.token_url(), + "https://login.microsoftonline.com/tenant/oauth2/v2.0/token" + ); + assert!(!format!("{login:?}").contains("the-secret")); + let login = login + .with_authority_host("https://login.microsoftonline.us/") + .unwrap(); + assert_eq!( + login.token_url(), + "https://login.microsoftonline.us/tenant/oauth2/v2.0/token" + ); + for bad in [ + "http://login.microsoftonline.com", + "https://login.microsoftonline.com/x", + "https://login.microsoftonline.com/?a=b", + "not a url", + ] { + let login = ClientSecretLogin::new("tenant", "client", "s", ()).unwrap(); + assert!(login.with_authority_host(bad).is_err(), "{bad}"); + } + assert!(ClientSecretLogin::new("a/b", "client", "s", ()).is_err()); + assert!(ClientSecretLogin::new("tenant", "", "s", ()).is_err()); + assert!(ClientSecretLogin::new("tenant", "client", "", ()).is_err()); + } +} diff --git a/packages/stack-kms-azure/src/auth/managed_identity.rs b/packages/stack-kms-azure/src/auth/managed_identity.rs new file mode 100644 index 0000000000..581259e088 --- /dev/null +++ b/packages/stack-kms-azure/src/auth/managed_identity.rs @@ -0,0 +1,227 @@ +//! The managed identity of an Azure resource: IMDS, or the App Service +//! identity endpoint. Native only. + +use std::fmt; + +use stack_kms_vendor::{VendorRequest, VendorTransport}; +use zeroize::Zeroizing; + +use super::{read_token_response, resource_of, AccessToken, AuthError, TokenProvider}; +use crate::config::EnvLookup; +use crate::time::{system_clock, Clock}; + +/// The token URL of the Azure Instance Metadata Service. +pub const IMDS_TOKEN_URL: &str = "http://169.254.169.254/metadata/identity/oauth2/token"; +/// The IMDS API version of a token request. +pub const IMDS_API_VERSION: &str = "2018-02-01"; +/// The API version of an App Service (and Functions) identity endpoint +/// request. +pub const APP_SERVICE_API_VERSION: &str = "2019-08-01"; + +/// Where the managed identity's token comes from. +enum Source { + /// IMDS: Azure VMs, VM scale sets, AKS nodes and container instances. + Imds, + /// App Service and Functions: `IDENTITY_ENDPOINT`, with + /// `IDENTITY_HEADER` in `X-IDENTITY-HEADER`. + AppService { + endpoint: String, + header: Zeroizing, + }, +} + +/// The token of the managed identity of the Azure resource this process +/// runs on. Native only. Wrap it in a [`CachedToken`](super::CachedToken). +/// +/// With a client id it asks for that user-assigned identity; without one, +/// for the system-assigned identity. +pub struct ManagedIdentityLogin { + source: Source, + client_id: Option, + transport: T, + clock: Clock, +} + +impl fmt::Debug for ManagedIdentityLogin { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + let source = match &self.source { + Source::Imds => "imds".to_owned(), + Source::AppService { endpoint, .. } => format!("app service at {endpoint}"), + }; + f.debug_struct("ManagedIdentityLogin") + .field("source", &source) + .field("client_id", &self.client_id) + .finish_non_exhaustive() + } +} + +/// An App Service identity endpoint: HTTP to a loopback host (App Service +/// serves it on `127.0.0.1`), or HTTPS, with no query, fragment or +/// credentials. The identity header is sent there, so it must not be +/// anywhere else. +fn is_identity_endpoint(endpoint: &str) -> bool { + let Ok(parsed) = url::Url::parse(endpoint) else { + return false; + }; + let loopback = match parsed.host() { + Some(url::Host::Domain(domain)) => domain == "localhost", + Some(url::Host::Ipv4(ip)) => ip.is_loopback(), + Some(url::Host::Ipv6(ip)) => ip.is_loopback(), + None => return false, + }; + (parsed.scheme() == "https" || (parsed.scheme() == "http" && loopback)) + && parsed.query().is_none() + && parsed.fragment().is_none() + && parsed.username().is_empty() + && parsed.password().is_none() +} + +impl ManagedIdentityLogin { + /// The system-assigned identity, from IMDS, through `transport`. + pub fn imds(transport: T) -> Self { + Self { + source: Source::Imds, + client_id: None, + transport, + clock: system_clock(), + } + } + + /// The identity of an App Service or Functions app, from `endpoint` + /// (`IDENTITY_ENDPOINT`) with `header` (`IDENTITY_HEADER`). + pub fn app_service( + endpoint: impl Into, + header: impl Into, + transport: T, + ) -> Result { + let endpoint = endpoint.into(); + if !is_identity_endpoint(&endpoint) { + return Err(AuthError::access_denied(format!( + "{endpoint:?} is not an App Service identity endpoint (HTTP to a loopback \ + host, or HTTPS, with no query)" + ))); + } + let header = Zeroizing::new(header.into()); + if header.is_empty() || header.contains(['\r', '\n']) { + return Err(AuthError::access_denied( + "IDENTITY_HEADER is empty or holds a line break", + )); + } + Ok(Self { + source: Source::AppService { endpoint, header }, + client_id: None, + transport, + clock: system_clock(), + }) + } + + /// App Service when `IDENTITY_ENDPOINT` and `IDENTITY_HEADER` are set, + /// else IMDS. `AZURE_CLIENT_ID`, if set, names a user-assigned identity. + pub fn from_env_lookup(transport: T, env: &EnvLookup) -> Result { + let login = match (env("IDENTITY_ENDPOINT"), env("IDENTITY_HEADER")) { + (Some(endpoint), Some(header)) => Self::app_service(endpoint, header, transport)?, + _ => Self::imds(transport), + }; + Ok(match env("AZURE_CLIENT_ID") { + Some(client_id) => login.with_client_id(client_id), + None => login, + }) + } + + /// Ask for the user-assigned identity with `client_id`. + pub fn with_client_id(mut self, client_id: impl Into) -> Self { + self.client_id = Some(client_id.into()); + self + } + + /// Read the time from `clock` (App Service answers with `expires_on`). + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = clock; + self + } + + /// The token URL for `scope`. + pub fn url(&self, scope: &str) -> String { + let (base, api_version) = match &self.source { + Source::Imds => (IMDS_TOKEN_URL, IMDS_API_VERSION), + Source::AppService { endpoint, .. } => (endpoint.as_str(), APP_SERVICE_API_VERSION), + }; + let mut query = url::form_urlencoded::Serializer::new(String::new()); + let _ = query + .append_pair("api-version", api_version) + .append_pair("resource", resource_of(scope)); + if let Some(client_id) = &self.client_id { + let _ = query.append_pair("client_id", client_id); + } + format!("{base}?{}", query.finish()) + } + + /// The transport. + pub fn transport(&self) -> &T { + &self.transport + } +} + +impl TokenProvider for ManagedIdentityLogin { + async fn token(&self, scope: &str) -> Result { + let request = VendorRequest::new("GET", self.url(scope)); + let (request, source) = match &self.source { + Source::Imds => (request.header("metadata", "true"), "IMDS"), + Source::AppService { header, .. } => ( + request.header("x-identity-header", header.as_str()), + "the App Service identity endpoint", + ), + }; + let response = self.transport.send(request).await.map_err(|error| { + AuthError::unavailable(format!("{source} got no response (not on Azure?): {error}")) + })?; + read_token_response(source, &response, (self.clock)()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn identity_endpoints_are_checked() { + for good in [ + "http://127.0.0.1:41056/msi/token", + "http://localhost:8081/msi/token/", + "http://[::1]:80/token", + "https://identity.example/token", + ] { + assert!(is_identity_endpoint(good), "{good}"); + } + for bad in [ + "http://169.254.169.254/token", + "http://evil.example/token", + "http://127.0.0.1/token?x=1", + "http://user@127.0.0.1/token", + "file:///etc/passwd", + "not a url", + ] { + assert!(!is_identity_endpoint(bad), "{bad}"); + } + } + + #[test] + fn urls_name_the_resource_and_the_identity() { + let login = ManagedIdentityLogin::imds(()); + assert_eq!( + login.url("https://vault.azure.net/.default"), + "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01\ + &resource=https%3A%2F%2Fvault.azure.net" + ); + let login = ManagedIdentityLogin::app_service("http://127.0.0.1:41056/msi/token", "h", ()) + .unwrap() + .with_client_id("c-1"); + assert_eq!( + login.url("https://managedhsm.azure.net/.default"), + "http://127.0.0.1:41056/msi/token?api-version=2019-08-01\ + &resource=https%3A%2F%2Fmanagedhsm.azure.net&client_id=c-1" + ); + assert!(!format!("{login:?}").contains("\"h\"")); + assert!(ManagedIdentityLogin::app_service("http://127.0.0.1/t", "a\nb", ()).is_err()); + } +} diff --git a/packages/stack-kms-azure/src/auth/mod.rs b/packages/stack-kms-azure/src/auth/mod.rs new file mode 100644 index 0000000000..eb593a036d --- /dev/null +++ b/packages/stack-kms-azure/src/auth/mod.rs @@ -0,0 +1,711 @@ +//! Microsoft Entra ID (Azure AD) access tokens for Key Vault and Managed +//! HSM, and the credentials that get them. +//! +//! [`TokenProvider`] is the seam: the client asks it for an [`AccessToken`] +//! for a scope (`https://vault.azure.net/.default` or +//! `https://managedhsm.azure.net/.default`) before each request. This crate +//! provides: +//! +//! * [`AccessToken`] itself: a static token, for example from +//! `az account get-access-token --resource https://vault.azure.net`. +//! Everywhere. It is used for every scope until Azure refuses it. +//! * [`ClientSecretLogin`]: a service principal's client secret, exchanged +//! at the Entra ID token endpoint (OAuth 2.0 client credentials). +//! Everywhere. +//! * [`WorkloadIdentityLogin`]: a federated token from a file (AKS +//! workload identity, `AZURE_FEDERATED_TOKEN_FILE`), exchanged as a +//! client assertion. **Native only.** +//! * [`ManagedIdentityLogin`]: the managed identity of an Azure VM, VM scale +//! set, AKS node or container instance (IMDS), or of an App Service or +//! Functions app (`IDENTITY_ENDPOINT`). **Native only.** +//! * [`CachedToken`]: keeps the token of each scope and gets a new one +//! before it expires. +//! * [`DefaultCredentials`] (native): picks one from the environment. +//! +//! In the WASI guest the host supplies the token (ADR-0011): the guest +//! implements [`TokenProvider`] over a host import. +//! +//! **Not supported:** client certificate credentials +//! (`AZURE_CLIENT_CERTIFICATE_PATH`). They sign a JWT with the service +//! principal's RSA key, and the only pure-Rust RSA crate has an open timing +//! advisory (RUSTSEC-2023-0071). Use a client secret, workload identity or a +//! managed identity. Also not supported: Azure CLI credentials, Azure Arc +//! and Service Fabric managed identity. + +use std::collections::HashMap; +use std::fmt; +use std::future::Future; +use std::sync::{Mutex as SyncMutex, PoisonError}; +use std::time::Duration; + +use futures::lock::Mutex; +use serde::Deserialize; +use stack_kms::{BackendErrorKind, MaybeSend}; +use stack_kms_vendor::{classify_status, VendorResponse}; +use web_time::SystemTime; +use zeroize::Zeroizing; + +use crate::config::EnvLookup; +use crate::time::{system_clock, unix_seconds, Clock}; + +mod client_secret; +#[cfg(not(target_arch = "wasm32"))] +mod managed_identity; +#[cfg(not(target_arch = "wasm32"))] +mod workload_identity; + +pub use client_secret::{ + is_tenant_id, ClientSecretLogin, DEFAULT_AUTHORITY_HOST, JWT_BEARER_ASSERTION_TYPE, +}; +#[cfg(not(target_arch = "wasm32"))] +pub use managed_identity::{ + ManagedIdentityLogin, APP_SERVICE_API_VERSION, IMDS_API_VERSION, IMDS_TOKEN_URL, +}; +#[cfg(not(target_arch = "wasm32"))] +pub use workload_identity::WorkloadIdentityLogin; + +/// The environment variable of a static access token. Azure has no +/// standard name for one; this one is ours. +pub const ACCESS_TOKEN_VARIABLE: &str = "AZURE_KEY_VAULT_ACCESS_TOKEN"; + +/// An Entra ID access token, and how long it lasts from when it was +/// issued. +/// +/// The token is wiped on drop and never printed. +#[derive(Clone)] +pub struct AccessToken { + token: Zeroizing, + expires_in: Option, +} + +impl fmt::Debug for AccessToken { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("AccessToken") + .field("token", &"") + .field("expires_in", &self.expires_in) + .finish() + } +} + +impl AccessToken { + /// A token with no known expiry: it is used until Azure refuses it. + pub fn new(token: impl Into) -> Self { + Self { + token: Zeroizing::new(token.into()), + expires_in: None, + } + } + + /// The same token, which expires `expires_in` from now. + pub fn with_expires_in(mut self, expires_in: Duration) -> Self { + self.expires_in = Some(expires_in); + self + } + + /// The token from [`ACCESS_TOKEN_VARIABLE`], or `None` when it is not + /// set. + pub fn from_env_lookup(env: &EnvLookup) -> Option { + env(ACCESS_TOKEN_VARIABLE).map(Self::new) + } + + /// The token. + pub fn token(&self) -> &str { + &self.token + } + + /// How long the token lasts from when it was issued. `None` for a token + /// with no known expiry. + pub fn expires_in(&self) -> Option { + self.expires_in + } +} + +/// A token could not be got. +#[derive(Debug, thiserror::Error)] +#[error("Azure credentials: {message}")] +pub struct AuthError { + kind: BackendErrorKind, + message: String, +} + +impl AuthError { + /// A failure of `kind`, described by `message`. Never put a secret in + /// the message. + pub fn new(kind: BackendErrorKind, message: impl Into) -> Self { + Self { + kind, + message: message.into(), + } + } + + /// No credentials are configured, or they were refused. + pub fn access_denied(message: impl Into) -> Self { + Self::new(BackendErrorKind::AccessDenied, message) + } + + /// The token endpoint or the managed identity endpoint did not answer. + pub fn unavailable(message: impl Into) -> Self { + Self::new(BackendErrorKind::Unavailable, message) + } + + /// The failure kind the client reports. + pub fn kind(&self) -> BackendErrorKind { + self.kind + } +} + +/// Where a client gets its access tokens. The client asks before each +/// request, so a provider that logs in should cache ([`CachedToken`]). +/// +/// The returned future is bounded by [`MaybeSend`]: `Send` on native +/// targets, unbounded on wasm32. +pub trait TokenProvider: Send + Sync { + /// A token for `scope` (`https://vault.azure.net/.default`). A managed + /// identity asks for the resource, the scope without `/.default`. + fn token( + &self, + scope: &str, + ) -> impl Future> + MaybeSend; + + /// Azure refused a token (HTTP 401). A caching provider drops its + /// tokens, so that the next request gets a new one. The default does + /// nothing. + fn invalidate(&self) {} +} + +impl TokenProvider for AccessToken { + fn token( + &self, + _scope: &str, + ) -> impl Future> + MaybeSend { + std::future::ready(Ok(self.clone())) + } +} + +/// The resource of a scope: the scope without `/.default`. +pub fn resource_of(scope: &str) -> &str { + scope.strip_suffix("/.default").unwrap_or(scope) +} + +/// An integer of a JSON response that may be a number or a string (IMDS and +/// App Service send strings). +fn read_u64(value: Option<&serde_json::Value>) -> Option { + match value? { + serde_json::Value::String(text) => text.trim().parse().ok(), + serde_json::Value::Number(number) => number.as_u64(), + _ => None, + } +} + +#[derive(Deserialize)] +struct TokenResponse { + access_token: Option, + expires_in: Option, + expires_on: Option, +} + +#[derive(Deserialize)] +struct TokenErrorBody { + error: Option, + error_description: Option, +} + +/// The access token of a token endpoint or managed identity response, read +/// at `now`. `source` names it in errors. +/// +/// The life is `expires_in` (seconds, a number or a string), else +/// `expires_on` (Unix seconds) less `now`. +pub(crate) fn read_token_response( + source: &str, + response: &VendorResponse, + now: SystemTime, +) -> Result { + if let Some(kind) = classify_status(response.status) { + // A refused grant is HTTP 400 or 401 (`invalid_client`); IMDS + // answers 400 or 404 for an identity it does not have. None of these + // is a missing key or a damaged ciphertext. IMDS answers 410 while + // it updates, which a later try fixes. + let kind = match (response.status, kind) { + (410, _) => BackendErrorKind::Unavailable, + (_, BackendErrorKind::InvalidCiphertext | BackendErrorKind::NotFound) => { + BackendErrorKind::AccessDenied + } + (_, kind) => kind, + }; + let detail = serde_json::from_slice::(&response.body) + .ok() + .map(|body| { + let error = match body.error { + Some(serde_json::Value::String(text)) => text, + Some(serde_json::Value::Object(object)) => object + .get("message") + .and_then(|m| m.as_str()) + .unwrap_or_default() + .to_owned(), + _ => String::new(), + }; + match body.error_description { + Some(description) => format!("{error}: {description}"), + None => error, + } + }) + .unwrap_or_default(); + return Err(AuthError::new( + kind, + format!("{source} failed with HTTP {}: {detail}", response.status), + )); + } + let parsed: TokenResponse = serde_json::from_slice(&response.body).map_err(|_| { + AuthError::unavailable(format!("{source} returned a response that is not JSON")) + })?; + let token = AccessToken { + token: Zeroizing::new( + parsed + .access_token + .filter(|token| !token.is_empty()) + .ok_or_else(|| { + AuthError::unavailable(format!("{source} returned no access token")) + })?, + ), + expires_in: None, + }; + let life = match read_u64(parsed.expires_in.as_ref()) { + Some(seconds) => Some(seconds), + None => { + read_u64(parsed.expires_on.as_ref()).map(|end| end.saturating_sub(unix_seconds(now))) + } + }; + Ok(match life { + None | Some(0) => token, + Some(seconds) => token.with_expires_in(Duration::from_secs(seconds)), + }) +} + +/// The default time before a token expires at which [`CachedToken`] gets a +/// new one. For a short-lived token the window is a third of its life. +pub const DEFAULT_REFRESH_BEFORE_EXPIRY: Duration = Duration::from_secs(5 * 60); + +/// A token in the cache, and when to get the next one. +struct Cached { + token: AccessToken, + /// When to get a new token. `None`: never. + refresh_at: Option, + /// When the token expires. `None`: never. + expires_at: Option, +} + +/// Keeps the token `inner` returns for each scope, and gets a new one when +/// it is near its expiry: [`DEFAULT_REFRESH_BEFORE_EXPIRY`] before, or a +/// third of its life before when it lives less than three times that. +/// +/// One fetch runs at a time: callers that arrive during a fetch wait for it. +/// If a fetch fails while the cached token has not expired, the cached +/// token is still returned. A token with no expiry is kept until Azure +/// refuses it ([`TokenProvider::invalidate`], which drops every scope's). +pub struct CachedToken

{ + inner: P, + cached: SyncMutex>, + fetch: Mutex<()>, + refresh_before: Duration, + clock: Clock, +} + +impl fmt::Debug for CachedToken

{ + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("CachedToken") + .field("inner", &self.inner) + .field("refresh_before", &self.refresh_before) + .finish_non_exhaustive() + } +} + +impl CachedToken

{ + /// Cache the tokens of `inner`. + pub fn new(inner: P) -> Self { + Self { + inner, + cached: SyncMutex::new(HashMap::new()), + fetch: Mutex::new(()), + refresh_before: DEFAULT_REFRESH_BEFORE_EXPIRY, + clock: system_clock(), + } + } + + /// Get a new token at most `refresh_before` before the token expires. + pub fn with_refresh_before(mut self, refresh_before: Duration) -> Self { + self.refresh_before = refresh_before; + self + } + + /// Read the time from `clock`. + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = clock; + self + } + + /// The provider whose tokens are cached. + pub fn inner(&self) -> &P { + &self.inner + } + + /// The cached token of `scope`, if it does not need a refresh yet. + fn fresh(&self, scope: &str, now: SystemTime) -> Option { + let cached = self.cached.lock().unwrap_or_else(PoisonError::into_inner); + cached + .get(scope) + .filter(|cached| cached.refresh_at.is_none_or(|at| now < at)) + .map(|cached| cached.token.clone()) + } +} + +impl TokenProvider for CachedToken

{ + async fn token(&self, scope: &str) -> Result { + if let Some(token) = self.fresh(scope, (self.clock)()) { + return Ok(token); + } + let _fetch = self.fetch.lock().await; + // Another caller may have got a token while this one waited. + let now = (self.clock)(); + if let Some(token) = self.fresh(scope, now) { + return Ok(token); + } + match self.inner.token(scope).await { + Ok(token) => { + let (refresh_at, expires_at) = match token.expires_in { + Some(life) => { + let window = self.refresh_before.min(life / 3); + (now.checked_add(life - window), now.checked_add(life)) + } + None => (None, None), + }; + let _ = self + .cached + .lock() + .unwrap_or_else(PoisonError::into_inner) + .insert( + scope.to_owned(), + Cached { + token: token.clone(), + refresh_at, + expires_at, + }, + ); + Ok(token) + } + Err(error) => { + let cached = self.cached.lock().unwrap_or_else(PoisonError::into_inner); + match cached.get(scope) { + Some(cached) if cached.expires_at.is_some_and(|end| now < end) => { + Ok(cached.token.clone()) + } + _ => Err(error), + } + } + } + } + + fn invalidate(&self) { + self.cached + .lock() + .unwrap_or_else(PoisonError::into_inner) + .clear(); + self.inner.invalidate(); + } +} + +#[cfg(not(target_arch = "wasm32"))] +mod default { + use stack_kms_vendor::VendorTransport; + + use super::*; + use crate::config::process_env; + + /// The credentials [`DefaultCredentials::from_env`] picks. + #[derive(Debug)] + pub enum DefaultCredentials { + /// [`ACCESS_TOKEN_VARIABLE`]. + Token(AccessToken), + /// `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and `AZURE_CLIENT_SECRET`. + ClientSecret(CachedToken>), + /// `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and + /// `AZURE_FEDERATED_TOKEN_FILE`. + WorkloadIdentity(CachedToken>), + /// A managed identity: App Service (`IDENTITY_ENDPOINT`) or IMDS. + ManagedIdentity(CachedToken>), + } + + impl DefaultCredentials { + /// Pick credentials from the environment, in this order (the order + /// of the Azure SDKs' `DefaultAzureCredential`, less the ones this + /// crate does not support): + /// + /// 1. [`ACCESS_TOKEN_VARIABLE`]: a static access token. + /// 2. `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and `AZURE_CLIENT_SECRET`: + /// a client secret. `AZURE_CLIENT_CERTIFICATE_PATH` with no + /// secret is refused (not supported). + /// 3. `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and + /// `AZURE_FEDERATED_TOKEN_FILE`: workload identity. + /// 4. Otherwise a managed identity: App Service when + /// `IDENTITY_ENDPOINT` and `IDENTITY_HEADER` are set, else IMDS. + /// `AZURE_CLIENT_ID`, if set, names a user-assigned identity. + /// + /// `AZURE_AUTHORITY_HOST` changes the Entra ID host (for a national + /// cloud). The logins send through `transport` and are cached. + pub fn from_env(transport: T) -> Result { + Self::from_env_lookup(transport, &process_env()) + } + + /// As [`from_env`](Self::from_env), reading the variables with `env`. + pub fn from_env_lookup(transport: T, env: &EnvLookup) -> Result { + if let Some(token) = AccessToken::from_env_lookup(env) { + return Ok(Self::Token(token)); + } + let tenant = env("AZURE_TENANT_ID"); + let client = env("AZURE_CLIENT_ID"); + if let (Some(tenant), Some(client)) = (&tenant, &client) { + if let Some(secret) = env("AZURE_CLIENT_SECRET") { + let login = ClientSecretLogin::new(tenant, client, secret, transport)?; + let login = match env("AZURE_AUTHORITY_HOST") { + Some(host) => login.with_authority_host(host)?, + None => login, + }; + return Ok(Self::ClientSecret(CachedToken::new(login))); + } + if env("AZURE_CLIENT_CERTIFICATE_PATH").is_some() { + return Err(AuthError::access_denied( + "client certificate credentials (AZURE_CLIENT_CERTIFICATE_PATH) are \ + not supported; use a client secret, workload identity or a managed \ + identity", + )); + } + if let Some(file) = env("AZURE_FEDERATED_TOKEN_FILE") { + let login = WorkloadIdentityLogin::new(tenant, client, file, transport)?; + let login = match env("AZURE_AUTHORITY_HOST") { + Some(host) => login.with_authority_host(host)?, + None => login, + }; + return Ok(Self::WorkloadIdentity(CachedToken::new(login))); + } + } + Ok(Self::ManagedIdentity(CachedToken::new( + ManagedIdentityLogin::from_env_lookup(transport, env)?, + ))) + } + } + + impl TokenProvider for DefaultCredentials { + async fn token(&self, scope: &str) -> Result { + match self { + Self::Token(token) => Ok(token.clone()), + Self::ClientSecret(login) => login.token(scope).await, + Self::WorkloadIdentity(login) => login.token(scope).await, + Self::ManagedIdentity(login) => login.token(scope).await, + } + } + + fn invalidate(&self) { + match self { + Self::Token(_) => {} + Self::ClientSecret(login) => login.invalidate(), + Self::WorkloadIdentity(login) => login.invalidate(), + Self::ManagedIdentity(login) => login.invalidate(), + } + } + } +} + +#[cfg(not(target_arch = "wasm32"))] +pub use default::DefaultCredentials; + +#[cfg(test)] +mod tests { + use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering}; + use std::sync::Arc; + + use web_time::UNIX_EPOCH; + + use super::*; + + #[test] + fn debug_hides_the_token() { + let text = format!("{:?}", AccessToken::new("the-access-token")); + assert!(!text.contains("the-access-token")); + } + + #[test] + fn the_resource_is_the_scope_without_default() { + assert_eq!( + resource_of("https://vault.azure.net/.default"), + "https://vault.azure.net" + ); + assert_eq!(resource_of("https://x"), "https://x"); + } + + struct Counting { + calls: AtomicUsize, + fail: AtomicBool, + life: Option, + } + + impl TokenProvider for Counting { + async fn token(&self, scope: &str) -> Result { + let call = self.calls.fetch_add(1, Ordering::SeqCst); + if self.fail.load(Ordering::SeqCst) { + return Err(AuthError::unavailable("down")); + } + let token = AccessToken::new(format!("{scope}-{call}")); + Ok(match self.life { + Some(life) => token.with_expires_in(life), + None => token, + }) + } + } + + fn cached(life: Option) -> (CachedToken, Arc) { + let now = Arc::new(AtomicU64::new(1_000_000)); + let clock_now = Arc::clone(&now); + let clock: Clock = + Arc::new(move || UNIX_EPOCH + Duration::from_secs(clock_now.load(Ordering::SeqCst))); + let provider = Counting { + calls: AtomicUsize::new(0), + fail: false.into(), + life, + }; + (CachedToken::new(provider).with_clock(clock), now) + } + + async fn token_of(cache: &CachedToken, scope: &str) -> String { + cache.token(scope).await.unwrap().token().to_owned() + } + + #[tokio::test] + async fn a_token_is_got_again_before_it_expires() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + assert_eq!(token_of(&cache, "v").await, "v-0"); + now.fetch_add(3600 - 301, Ordering::SeqCst); + assert_eq!(token_of(&cache, "v").await, "v-0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(token_of(&cache, "v").await, "v-1"); + assert_eq!(cache.inner().calls.load(Ordering::SeqCst), 2); + } + + #[tokio::test] + async fn each_scope_has_its_own_token() { + let (cache, _) = cached(Some(Duration::from_secs(3600))); + assert_eq!(token_of(&cache, "vault").await, "vault-0"); + assert_eq!(token_of(&cache, "hsm").await, "hsm-1"); + assert_eq!(token_of(&cache, "vault").await, "vault-0"); + assert_eq!(token_of(&cache, "hsm").await, "hsm-1"); + cache.invalidate(); + assert_eq!(token_of(&cache, "hsm").await, "hsm-2"); + assert_eq!(token_of(&cache, "vault").await, "vault-3"); + } + + #[tokio::test] + async fn a_short_life_refreshes_a_third_before_its_end() { + let (cache, now) = cached(Some(Duration::from_secs(60))); + assert_eq!(token_of(&cache, "v").await, "v-0"); + now.fetch_add(39, Ordering::SeqCst); + assert_eq!(token_of(&cache, "v").await, "v-0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(token_of(&cache, "v").await, "v-1"); + } + + #[tokio::test] + async fn a_token_with_no_expiry_is_kept_until_invalidated() { + let (cache, now) = cached(None); + assert_eq!(token_of(&cache, "v").await, "v-0"); + now.fetch_add(10 * 365 * 86_400, Ordering::SeqCst); + assert_eq!(token_of(&cache, "v").await, "v-0"); + cache.invalidate(); + assert_eq!(token_of(&cache, "v").await, "v-1"); + } + + #[tokio::test] + async fn a_failed_fetch_keeps_a_token_that_has_not_expired() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + assert_eq!(token_of(&cache, "v").await, "v-0"); + cache.inner().fail.store(true, Ordering::SeqCst); + now.fetch_add(3500, Ordering::SeqCst); + assert_eq!(token_of(&cache, "v").await, "v-0"); + now.fetch_add(100, Ordering::SeqCst); + let error = cache.token("v").await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); + // Another scope has nothing cached to fall back on. + let error = cache.token("w").await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); + } + + #[tokio::test] + async fn the_refresh_window_is_configurable() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + let cache = cache.with_refresh_before(Duration::ZERO); + assert_eq!(token_of(&cache, "v").await, "v-0"); + now.fetch_add(3599, Ordering::SeqCst); + assert_eq!(token_of(&cache, "v").await, "v-0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(token_of(&cache, "v").await, "v-1"); + } + + #[test] + fn token_responses_are_read_and_failures_classified() { + let now = UNIX_EPOCH + Duration::from_secs(1_000_000); + for (body, life) in [ + ( + &br#"{"token_type":"Bearer","expires_in":3599,"access_token":"t"}"#[..], + Some(3599), + ), + (br#"{"access_token":"t","expires_in":"86399"}"#, Some(86399)), + ( + br#"{"access_token":"t","expires_on":"1003600"}"#, + Some(3600), + ), + (br#"{"access_token":"t","expires_on":1000060}"#, Some(60)), + (br#"{"access_token":"t","expires_on":"999999"}"#, None), + (br#"{"access_token":"t"}"#, None), + ] { + let token = + read_token_response("x", &VendorResponse::new(200, body.to_vec()), now).unwrap(); + assert_eq!(token.token(), "t"); + assert_eq!(token.expires_in(), life.map(Duration::from_secs)); + } + for (status, body, kind) in [ + ( + 400, + &br#"{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided."}"#[..], + BackendErrorKind::AccessDenied, + ), + (401, b"{}", BackendErrorKind::AccessDenied), + (404, b"not found", BackendErrorKind::AccessDenied), + (410, b"", BackendErrorKind::Unavailable), + (429, b"", BackendErrorKind::Throttled), + (503, b"", BackendErrorKind::Unavailable), + (200, b"not json", BackendErrorKind::Unavailable), + ( + 200, + br#"{"access_token":""}"#, + BackendErrorKind::Unavailable, + ), + ] { + let error = + read_token_response("x", &VendorResponse::new(status, body.to_vec()), now) + .unwrap_err(); + assert_eq!(error.kind(), kind, "{status}"); + } + let error = read_token_response( + "the token endpoint", + &VendorResponse::new( + 401, + br#"{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided."}"# + .to_vec(), + ), + now, + ) + .unwrap_err(); + assert_eq!( + error.to_string(), + "Azure credentials: the token endpoint failed with HTTP 401: \ + invalid_client: AADSTS7000215: Invalid client secret provided." + ); + } +} diff --git a/packages/stack-kms-azure/src/auth/workload_identity.rs b/packages/stack-kms-azure/src/auth/workload_identity.rs new file mode 100644 index 0000000000..b4d0ce7efa --- /dev/null +++ b/packages/stack-kms-azure/src/auth/workload_identity.rs @@ -0,0 +1,105 @@ +//! Workload identity federation: a federated token from a file, exchanged +//! at the Entra ID token endpoint. Native only. + +use std::fmt; +use std::path::PathBuf; + +use stack_kms_vendor::VendorTransport; +use zeroize::Zeroizing; + +use super::client_secret::{EntraApp, JWT_BEARER_ASSERTION_TYPE}; +use super::{AccessToken, AuthError, TokenProvider}; +use crate::time::{system_clock, Clock}; + +/// A federated token (for example the service account token that AKS +/// workload identity projects at `AZURE_FEDERATED_TOKEN_FILE`), exchanged at +/// the Entra ID token endpoint as a client assertion. The file is read on +/// each login, because the platform rotates it. Wrap it in a +/// [`CachedToken`](super::CachedToken). Native only. +pub struct WorkloadIdentityLogin { + app: EntraApp, + token_file: PathBuf, + transport: T, + clock: Clock, +} + +impl fmt::Debug for WorkloadIdentityLogin { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("WorkloadIdentityLogin") + .field("tenant_id", &self.app.tenant_id()) + .field("client_id", &self.app.client_id()) + .field("token_file", &self.token_file) + .finish_non_exhaustive() + } +} + +impl WorkloadIdentityLogin { + /// Log in as `client_id` of `tenant_id` with the federated token in + /// `token_file`, through `transport`. + pub fn new( + tenant_id: &str, + client_id: &str, + token_file: impl Into, + transport: T, + ) -> Result { + Ok(Self { + app: EntraApp::new(tenant_id, client_id)?, + token_file: token_file.into(), + transport, + clock: system_clock(), + }) + } + + /// Post to the Entra ID host `host` instead of + /// [`DEFAULT_AUTHORITY_HOST`](super::DEFAULT_AUTHORITY_HOST). + pub fn with_authority_host(mut self, host: impl Into) -> Result { + self.app = self.app.with_authority_host(host.into())?; + Ok(self) + } + + /// Read the time from `clock`. + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = clock; + self + } + + /// The token endpoint URL. + pub fn token_url(&self) -> String { + self.app.token_url() + } + + /// The transport. + pub fn transport(&self) -> &T { + &self.transport + } +} + +impl TokenProvider for WorkloadIdentityLogin { + async fn token(&self, scope: &str) -> Result { + let assertion = + Zeroizing::new(std::fs::read_to_string(&self.token_file).map_err(|_| { + AuthError::access_denied(format!( + "cannot read the federated token file {}", + self.token_file.display() + )) + })?); + let assertion = assertion.trim(); + if assertion.is_empty() { + return Err(AuthError::access_denied( + "the federated token file is empty", + )); + } + self.app + .post( + &self.transport, + &self.clock, + "the workload identity login", + scope, + &[ + ("client_assertion_type", JWT_BEARER_ASSERTION_TYPE), + ("client_assertion", assertion), + ], + ) + .await + } +} diff --git a/packages/stack-kms-azure/src/client.rs b/packages/stack-kms-azure/src/client.rs new file mode 100644 index 0000000000..e3a7887227 --- /dev/null +++ b/packages/stack-kms-azure/src/client.rs @@ -0,0 +1,822 @@ +//! The Key Vault client and its [`KeyVendorBackend`] implementation. + +use std::collections::HashMap; +use std::sync::{Mutex, PoisonError}; +use std::time::Duration; + +use base64ct::{Base64UrlUnpadded, Encoding}; +use serde::de::DeserializeOwned; +use serde::{Deserialize, Serialize}; +use stack_kms::{BackendErrorKind, Error, KeyVendor}; +use stack_kms_vendor::bound::{bind_key, open_bound_key, BOUND_KEY_LEN}; +use stack_kms_vendor::{ + classify_status, KeyBytes, KeyVendorBackend, TransportError, VendorRequest, VendorResponse, + VendorTransport, +}; +use zeroize::Zeroizing; + +use crate::auth::{AuthError, TokenProvider}; +use crate::config::{is_version, AzureKeyVaultConfig, KeyId, WrapAlgorithm, MAX_VERSION_LEN}; + +/// The vendor name in errors ([`KeyVendor::name`] of [`KeyVendor::Azure`]). +const VENDOR: &str = "azure-key-vault"; + +/// The first byte of a wrapped key this client stores: the format of the +/// rest (`algorithm(1) ‖ version length(1) ‖ version ‖ Azure ciphertext`). +pub const WRAPPED_KEY_FORMAT: u8 = 1; + +/// The length of the plaintext the client wraps: the bound key +/// ([`BOUND_KEY_LEN`], 65 bytes) and 7 zero bytes. A256KW needs a multiple +/// of 8 bytes; RSA-OAEP-256 with a 2048-bit key takes up to 190. +pub const WRAP_PLAINTEXT_LEN: usize = 72; + +/// The longest `Retry-After` the client waits for before it reports +/// `Throttled`. The adapter then backs off on its own as well. +pub const MAX_RETRY_AFTER: Duration = Duration::from_secs(30); + +/// A failure of a Key Vault call. It is the `source` of the +/// [`Error::Backend`] the client returns; the kind is on the +/// `Error::Backend`. +#[derive(Debug, thiserror::Error)] +pub enum AzureKeyVaultError { + /// Key Vault answered with an error. + #[error("Azure Key Vault {operation} failed with HTTP {status}: {code}: {message}")] + Service { + /// The operation (`unwrapkey`). + operation: &'static str, + /// The HTTP status. + status: u16, + /// The error code (`Forbidden`), with the inner code after a `/` + /// when there is one (`Forbidden/KeyDisabled`), or `unknown`. + code: String, + /// Azure's error message. + message: String, + }, + /// The request got no response. + #[error("Azure Key Vault {operation} got no response: {source}")] + Transport { + /// The operation. + operation: &'static str, + /// The transport failure. + source: TransportError, + }, + /// No access token could be got. + #[error(transparent)] + Auth(#[from] AuthError), + /// Key Vault answered with a success that this client cannot read or + /// does not trust. + #[error("Azure Key Vault {operation} returned a response this client cannot read: {reason}")] + InvalidResponse { + /// The operation. + operation: &'static str, + /// What is wrong with it. + reason: &'static str, + }, + /// The client refused the call before or after it reached Key Vault. + #[error("{0}")] + Refused(&'static str), +} + +fn backend(kind: BackendErrorKind, error: AzureKeyVaultError) -> Error { + Error::Backend { + vendor: VENDOR, + kind, + source: Box::new(error), + } +} + +fn invalid(operation: &'static str, reason: &'static str) -> Error { + backend( + BackendErrorKind::Unavailable, + AzureKeyVaultError::InvalidResponse { operation, reason }, + ) +} + +fn refused(kind: BackendErrorKind, reason: &'static str) -> Error { + backend(kind, AzureKeyVaultError::Refused(reason)) +} + +/// The error codes (`error.code`, or `error.innererror.code`) that mean a +/// key is disabled, expired or not yet valid. They map to `Unavailable`, as +/// AWS KMS's `DisabledException` does: the key exists, and an operator can +/// enable it again. +pub const KEY_STATE_CODES: [&str; 3] = ["KeyDisabled", "KeyExpired", "KeyNotYetValid"]; + +/// The failure kind of a Key Vault error response, by the HTTP `status` +/// and the error codes `codes` (`error.code` and `error.innererror.code`). +/// `unwrap` is true for an `unwrapkey` call. +/// +/// * The codes of [`KEY_STATE_CODES`]: `Unavailable`. +/// * 429, or `Throttled`: `Throttled` (the client first waits for +/// `Retry-After`, up to [`MAX_RETRY_AFTER`]). +/// * 401, 403, `Unauthorized`, `Forbidden`: `AccessDenied` (a 401 also +/// drops the cached token). +/// * 404, `KeyNotFound`, `NotFound`: `NotFound`. +/// * 5xx: `Unavailable`. +/// * 400 `BadParameter` on an `unwrapkey`: `InvalidCiphertext` (Azure +/// could not decrypt the bytes: damaged, or wrapped by another key or +/// version). +/// * Any other 4xx is a request Key Vault refused (`AccessDenied`), as for +/// AWS KMS, Vault and Cloud KMS. +pub fn error_kind(status: u16, codes: &[&str], unwrap: bool) -> BackendErrorKind { + let has = |wanted: &[&str]| codes.iter().any(|code| wanted.contains(code)); + if has(&KEY_STATE_CODES) { + return BackendErrorKind::Unavailable; + } + if status == 429 || has(&["Throttled"]) { + return BackendErrorKind::Throttled; + } + if matches!(status, 401 | 403) || has(&["Unauthorized", "Forbidden"]) { + return BackendErrorKind::AccessDenied; + } + if status == 404 || has(&["KeyNotFound", "NotFound"]) { + return BackendErrorKind::NotFound; + } + if unwrap && status == 400 && has(&["BadParameter"]) { + return BackendErrorKind::InvalidCiphertext; + } + match classify_status(status) { + Some(BackendErrorKind::InvalidCiphertext) | None => BackendErrorKind::AccessDenied, + Some(kind) => kind, + } +} + +#[derive(Deserialize)] +struct ErrorBody { + error: Option, +} + +#[derive(Deserialize)] +struct ErrorDetail { + code: Option, + message: Option, + innererror: Option, +} + +#[derive(Deserialize)] +struct InnerError { + code: Option, +} + +fn service_error(operation: &'static str, response: &VendorResponse) -> Error { + let detail = serde_json::from_slice::(&response.body) + .ok() + .and_then(|body| body.error); + let (code, inner, message) = match detail { + Some(detail) => ( + detail.code, + detail.innererror.and_then(|inner| inner.code), + detail.message.unwrap_or_default(), + ), + None => (None, None, String::new()), + }; + let codes: Vec<&str> = [code.as_deref(), inner.as_deref()] + .into_iter() + .flatten() + .collect(); + let kind = error_kind(response.status, &codes, operation == "unwrapkey"); + let code = match codes.as_slice() { + [] => "unknown".to_owned(), + codes => codes.join("/"), + }; + backend( + kind, + AzureKeyVaultError::Service { + operation, + status: response.status, + code, + message, + }, + ) +} + +/// The seconds of a `Retry-After` header, capped at [`MAX_RETRY_AFTER`]. +/// `None` when there is none or it is not a number of seconds. +pub fn retry_after(response: &VendorResponse) -> Option { + let seconds: u64 = response.header_value("retry-after")?.trim().parse().ok()?; + Some(Duration::from_secs(seconds).min(MAX_RETRY_AFTER)) +} + +/// Decode base64url, with or without padding (Azure sends none). +fn decode_base64url(text: &str) -> Option> { + Base64UrlUnpadded::decode_vec(text.trim_end_matches('=')).ok() +} + +fn serialize_secret( + value: &Zeroizing, + serializer: S, +) -> Result { + serializer.serialize_str(value) +} + +#[derive(Serialize)] +struct KeyOperationRequest { + alg: &'static str, + #[serde(serialize_with = "serialize_secret")] + value: Zeroizing, +} + +#[derive(Deserialize)] +struct KeyOperationResponse { + kid: Option, + value: Option, +} + +#[derive(Deserialize)] +struct KeyBundle { + key: Option, +} + +#[derive(Deserialize)] +struct JsonWebKey { + kid: Option, + kty: Option, +} + +/// The stored form of an Azure wrapped key: [`WRAPPED_KEY_FORMAT`], the +/// algorithm byte ([`WrapAlgorithm::byte`]), the length of the key version +/// (1 byte), the key version (ASCII), and the Azure ciphertext. +/// +/// A version is at most [`MAX_VERSION_LEN`] bytes ([`is_version`]); a +/// longer one is cut, and the result then does not split. +pub fn compact_ciphertext(algorithm: WrapAlgorithm, version: &str, ciphertext: &[u8]) -> Vec { + let version = &version.as_bytes()[..version.len().min(MAX_VERSION_LEN)]; + let mut out = Vec::with_capacity(3 + version.len() + ciphertext.len()); + out.push(WRAPPED_KEY_FORMAT); + out.push(algorithm.byte()); + out.push(version.len() as u8); + out.extend_from_slice(version); + out.extend_from_slice(ciphertext); + out +} + +/// A stored wrapped key, split into its parts. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct WrappedKeyParts<'a> { + /// The algorithm that wrapped it. + pub algorithm: WrapAlgorithm, + /// The key version that wrapped it. + pub version: &'a str, + /// The Azure ciphertext. + pub ciphertext: &'a [u8], +} + +/// The parts of a stored wrapped key. `None` if the bytes are not one. +pub fn split_wrapped_key(wrapped: &[u8]) -> Option> { + let (&format, rest) = wrapped.split_first()?; + if format != WRAPPED_KEY_FORMAT { + return None; + } + let (&algorithm, rest) = rest.split_first()?; + let algorithm = WrapAlgorithm::from_byte(algorithm)?; + let (&len, rest) = rest.split_first()?; + let (version, ciphertext) = rest.split_at_checked(usize::from(len))?; + let version = std::str::from_utf8(version).ok()?; + if !is_version(version) || ciphertext.is_empty() { + return None; + } + Some(WrappedKeyParts { + algorithm, + version, + ciphertext, + }) +} + +/// The key version that wrapped `wrapped`, or `None` if the bytes are not a +/// wrapped key of this client. Use it to find which versions stored keys +/// still need before you disable a version. +pub fn wrapped_key_version(wrapped: &[u8]) -> Option<&str> { + split_wrapped_key(wrapped).map(|parts| parts.version) +} + +/// The plaintext the client wraps: the bound key of `key` under `aad`, +/// padded to [`WRAP_PLAINTEXT_LEN`] with zeros. +pub fn wrap_plaintext(key: &KeyBytes, aad: &[u8]) -> Zeroizing> { + let mut plaintext = bind_key(key, aad); + plaintext.resize(WRAP_PLAINTEXT_LEN, 0); + plaintext +} + +/// The key in an unwrapped plaintext, if it is one this client made under +/// `aad`. Anything else is `InvalidCiphertext`. +fn open_plaintext(plaintext: &[u8], aad: &[u8]) -> Result { + if plaintext.len() != WRAP_PLAINTEXT_LEN + || plaintext[BOUND_KEY_LEN..].iter().any(|&byte| byte != 0) + { + return Err(refused( + BackendErrorKind::InvalidCiphertext, + "the unwrapped key is not a bound key of this client", + )); + } + open_bound_key(&plaintext[..BOUND_KEY_LEN], aad).map_err(|_| { + refused( + BackendErrorKind::InvalidCiphertext, + "the wrapped key belongs to other associated data", + ) + }) +} + +/// The version and the algorithm a wrap uses for one key reference. +#[derive(Clone, Debug)] +struct Resolved { + version: String, + algorithm: WrapAlgorithm, +} + +/// An Azure Key Vault or Managed HSM client, and the Azure key vendor +/// backend ([`KeyVendor::Azure`]). +/// +/// It speaks the Key Vault REST API ([`API_VERSION`](crate::API_VERSION): +/// `GET /keys/{name}`, `wrapkey` and `unwrapkey`) with an Entra ID access +/// token, over a [`VendorTransport`], so the same code runs on native Rust +/// and in the WASI guest. +/// +/// * `key_ref` (from the keyset record) is the key identifier, +/// `https://{vault}.vault.azure.net/keys/{name}` or +/// `https://{hsm}.managedhsm.azure.net/keys/{name}`, optionally with +/// `/{version}`. A Key Vault `RSA` or `RSA-HSM` key wraps with +/// RSA-OAEP-256; a Managed HSM `oct-HSM` key wraps with A256KW. +/// * **Versions.** Azure needs the key version to unwrap, so every stored +/// wrapped key names the version that wrapped it +/// ([`wrapped_key_version`]) and its algorithm. An unwrap reads no key. +/// With no version in `key_ref`, the client reads the key once +/// (`GET /keys/{name}`) to learn its current version and type, and wraps +/// under that version for the life of the client. After a failed wrap it +/// reads the key again, so a disabled old version is left behind. With a +/// version, wraps use that version. +/// * **Associated data.** Key Vault's `wrapkey` takes no associated data. +/// The client wraps a bound key ([`stack_kms_vendor::bound`]): a SHA-256 +/// digest of the whole associated data (keyset id and, for a wrapped data +/// key, the iv and the descriptor) beside the key, which an unwrap checks. +/// This is sound because RSA-OAEP and AES key wrap are non-malleable. It +/// is the Azure trade-off: Azure does not see or log the binding, so a +/// wrapped key moved to another value unwraps at Azure (and is then +/// refused here as `InvalidCiphertext`). +/// * Key Vault has no data key generation and no batch operation: each +/// wrapped data key is a key from the local CSPRNG and one `wrapkey`. +/// * Errors map to [`BackendErrorKind`] by [`error_kind`]. A 401 also tells +/// the token provider to drop its tokens ([`TokenProvider::invalidate`]). +pub struct AzureKeyVault { + config: AzureKeyVaultConfig, + transport: T, + auth: A, + resolved: Mutex>, +} + +impl std::fmt::Debug for AzureKeyVault { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("AzureKeyVault") + .field("config", &self.config) + .finish_non_exhaustive() + } +} + +/// The key identifier of `key_ref`. +fn key_id(key_ref: &str) -> Result { + KeyId::parse(key_ref).ok_or_else(|| { + refused( + BackendErrorKind::NotFound, + "the key reference is not an Azure Key Vault or Managed HSM key identifier", + ) + }) +} + +fn to_json(operation: &'static str, value: &S) -> Result>, Error> { + serde_json::to_vec(value) + .map(Zeroizing::new) + .map_err(|_| invalid(operation, "unserializable request")) +} + +impl AzureKeyVault { + /// A client for `config`, sending through `transport`, with the access + /// tokens of `auth`. + pub fn new(config: AzureKeyVaultConfig, transport: T, auth: A) -> Self { + Self { + config, + transport, + auth, + resolved: Mutex::new(HashMap::new()), + } + } + + /// The configuration. + pub fn config(&self) -> &AzureKeyVaultConfig { + &self.config + } + + /// The transport. + pub fn transport(&self) -> &T { + &self.transport + } + + /// The token provider. + pub fn auth(&self) -> &A { + &self.auth + } + + /// Send one request to `url` with a token for `scope`, and return the + /// response of a success. + async fn request( + &self, + operation: &'static str, + method: &str, + url: String, + scope: &str, + body: Option>>, + ) -> Result { + let token = self.auth.token(scope).await.map_err(|error| { + let kind = error.kind(); + backend(kind, AzureKeyVaultError::Auth(error)) + })?; + let mut request = VendorRequest::new(method, url) + .header("authorization", format!("Bearer {}", token.token())); + if let Some(body) = body { + request = request.header("content-type", "application/json"); + request.body = body; + } + let response = self.transport.send(request).await.map_err(|source| { + backend( + BackendErrorKind::Unavailable, + AzureKeyVaultError::Transport { operation, source }, + ) + })?; + if classify_status(response.status).is_some() { + if response.status == 401 { + self.auth.invalidate(); + } + if response.status == 429 { + if let Some(wait) = retry_after(&response) { + self.transport.sleep(wait).await; + } + } + return Err(service_error(operation, &response)); + } + Ok(response) + } + + /// Send one request to the key identifier `key_ref` (with no + /// operation: the key itself), and read the JSON of a success. With + /// `operation` (`wrapkey`, `create`, `rotate`) it posts to + /// `{key}/{operation}`. Any key operation can be called this way (for + /// example `create` in a test); [`KeyVendorBackend`] uses `GET`, + /// `wrapkey` and `unwrapkey`. The body is wiped after it is sent. + pub async fn send_json( + &self, + method: &str, + key_ref: &str, + operation: Option<&str>, + body: Option>>, + ) -> Result { + let id = key_id(key_ref)?; + if operation.is_some_and(|op| !op.bytes().all(|b| b.is_ascii_lowercase())) { + return Err(refused( + BackendErrorKind::AccessDenied, + "an operation is lower-case letters only", + )); + } + let response = self + .request("request", method, id.url(operation), id.scope(), body) + .await?; + serde_json::from_slice(&response.body).map_err(|_| invalid("request", "invalid JSON")) + } + + /// The version and algorithm a wrap under `id` uses: from the key + /// identifier and the configuration when both say, else from one key + /// read, kept for the life of the client. + async fn resolve(&self, key_ref: &str, id: &KeyId) -> Result { + if let (Some(version), Some(algorithm)) = (id.version(), self.config.algorithm()) { + return Ok(Resolved { + version: version.to_owned(), + algorithm, + }); + } + if let Some(resolved) = self + .resolved + .lock() + .unwrap_or_else(PoisonError::into_inner) + .get(key_ref) + { + return Ok(resolved.clone()); + } + let response = self + .request("get key", "GET", id.url(None), id.scope(), None) + .await?; + let bundle: KeyBundle = serde_json::from_slice(&response.body) + .map_err(|_| invalid("get key", "invalid JSON"))?; + let key = bundle + .key + .ok_or_else(|| invalid("get key", "no key in the response"))?; + let version = key + .kid + .as_deref() + .and_then(KeyId::parse) + .filter(|kid| kid.vault() == id.vault() && kid.name() == id.name()) + .and_then(|kid| kid.version().map(str::to_owned)) + .ok_or_else(|| invalid("get key", "the kid is not a version of the key"))?; + if id.version().is_some_and(|pinned| pinned != version) { + return Err(invalid("get key", "the kid is not the requested version")); + } + let algorithm = key + .kty + .as_deref() + .and_then(WrapAlgorithm::for_key_type) + .ok_or_else(|| { + refused( + BackendErrorKind::AccessDenied, + "the key type cannot wrap keys (use an RSA, RSA-HSM or oct-HSM key)", + ) + })?; + if self + .config + .algorithm() + .is_some_and(|configured| configured != algorithm) + { + return Err(refused( + BackendErrorKind::AccessDenied, + "the key type does not match the configured algorithm", + )); + } + let resolved = Resolved { version, algorithm }; + let _ = self + .resolved + .lock() + .unwrap_or_else(PoisonError::into_inner) + .insert(key_ref.to_owned(), resolved.clone()); + Ok(resolved) + } + + async fn wrap_resolved( + &self, + id: &KeyId, + resolved: &Resolved, + key: &KeyBytes, + aad: &[u8], + ) -> Result, Error> { + let plaintext = wrap_plaintext(key, aad); + let body = to_json( + "wrapkey", + &KeyOperationRequest { + alg: resolved.algorithm.name(), + value: Zeroizing::new(Base64UrlUnpadded::encode_string(&plaintext)), + }, + )?; + let versioned = id.with_version(&resolved.version); + let response = self + .request( + "wrapkey", + "POST", + versioned.url(Some("wrapkey")), + id.scope(), + Some(body), + ) + .await?; + let parsed: KeyOperationResponse = serde_json::from_slice(&response.body) + .map_err(|_| invalid("wrapkey", "invalid JSON"))?; + let kid = parsed.kid.as_deref().and_then(KeyId::parse); + if kid.as_ref() != Some(&versioned) { + return Err(invalid( + "wrapkey", + "the kid is not the version of the key that was asked for", + )); + } + let ciphertext = parsed + .value + .as_deref() + .and_then(decode_base64url) + .filter(|ciphertext| !ciphertext.is_empty()) + .ok_or_else(|| invalid("wrapkey", "no base64url wrapped key"))?; + Ok(compact_ciphertext( + resolved.algorithm, + &resolved.version, + &ciphertext, + )) + } +} + +impl KeyVendorBackend for AzureKeyVault { + fn vendor(&self) -> KeyVendor { + KeyVendor::Azure + } + + /// The same key in the same vault or HSM, whatever version either key + /// identifier names. An identifier that does not parse is compared as + /// text. + fn same_key(&self, a: &str, b: &str) -> bool { + match (KeyId::parse(a), KeyId::parse(b)) { + (Some(a), Some(b)) => a.vault() == b.vault() && a.name() == b.name(), + _ => a == b, + } + } + + async fn wrap(&self, key_ref: &str, key: &KeyBytes, aad: &[u8]) -> Result, Error> { + let id = key_id(key_ref)?; + let resolved = self.resolve(key_ref, &id).await?; + let result = self.wrap_resolved(&id, &resolved, key, aad).await; + if result.is_err() { + // The version may be disabled or gone: read the key again next + // time. + let _ = self + .resolved + .lock() + .unwrap_or_else(PoisonError::into_inner) + .remove(key_ref); + } + result + } + + async fn unwrap(&self, key_ref: &str, wrapped: &[u8], aad: &[u8]) -> Result { + let id = key_id(key_ref)?; + let parts = split_wrapped_key(wrapped).ok_or_else(|| { + refused( + BackendErrorKind::InvalidCiphertext, + "the wrapped key is not an Azure Key Vault wrapped key", + ) + })?; + let body = to_json( + "unwrapkey", + &KeyOperationRequest { + alg: parts.algorithm.name(), + value: Zeroizing::new(Base64UrlUnpadded::encode_string(parts.ciphertext)), + }, + )?; + let versioned = id.with_version(parts.version); + let response = self + .request( + "unwrapkey", + "POST", + versioned.url(Some("unwrapkey")), + id.scope(), + Some(body), + ) + .await?; + let parsed: KeyOperationResponse = serde_json::from_slice(&response.body) + .map_err(|_| invalid("unwrapkey", "invalid JSON"))?; + let encoded = Zeroizing::new( + parsed + .value + .ok_or_else(|| invalid("unwrapkey", "no unwrapped key"))?, + ); + let plaintext = Zeroizing::new( + decode_base64url(&encoded) + .ok_or_else(|| invalid("unwrapkey", "the unwrapped key is not base64url"))?, + ); + open_plaintext(&plaintext, aad) + } + + async fn sleep(&self, duration: Duration) { + self.transport.sleep(duration).await; + } +} + +#[cfg(test)] +mod tests { + use vitaminc::protected::{Controlled, Protected}; + + use super::*; + + #[test] + fn errors_map_by_code_then_status() { + use BackendErrorKind::*; + for (status, codes, unwrap, kind) in [ + (403, &["Forbidden"][..], false, AccessDenied), + ( + 403, + &["Forbidden", "ForbiddenByPolicy"], + false, + AccessDenied, + ), + (403, &["Forbidden", "KeyDisabled"], false, Unavailable), + (400, &["BadParameter", "KeyExpired"], true, Unavailable), + (401, &["Unauthorized"], false, AccessDenied), + (401, &[], false, AccessDenied), + (404, &["KeyNotFound"], false, NotFound), + (404, &[], true, NotFound), + (429, &["Throttled"], false, Throttled), + (429, &[], false, Throttled), + (503, &["ServiceUnavailable"], false, Unavailable), + (500, &[], true, Unavailable), + (400, &["BadParameter"], true, InvalidCiphertext), + (400, &["BadParameter"], false, AccessDenied), + (400, &["BadRequest"], true, AccessDenied), + (409, &["Conflict"], false, AccessDenied), + (302, &[], false, AccessDenied), + ] { + assert_eq!( + error_kind(status, codes, unwrap), + kind, + "{status} {codes:?} {unwrap}" + ); + } + } + + #[test] + fn service_errors_read_both_codes() { + let response = VendorResponse::new( + 403, + br#"{"error":{"code":"Forbidden","message":"Operation wrapKey is not allowed on a disabled key.","innererror":{"code":"KeyDisabled"}}}"#.to_vec(), + ); + let error = service_error("wrapkey", &response); + let Error::Backend { kind, source, .. } = error else { + panic!("not a backend error"); + }; + assert_eq!(kind, BackendErrorKind::Unavailable); + assert_eq!( + source.to_string(), + "Azure Key Vault wrapkey failed with HTTP 403: Forbidden/KeyDisabled: \ + Operation wrapKey is not allowed on a disabled key." + ); + let error = service_error("unwrapkey", &VendorResponse::new(502, b"".to_vec())); + let Error::Backend { kind, source, .. } = error else { + panic!("not a backend error"); + }; + assert_eq!(kind, BackendErrorKind::Unavailable); + assert!(source.to_string().contains("HTTP 502: unknown")); + } + + #[test] + fn retry_after_is_read_and_capped() { + let response = + |value: &str| VendorResponse::new(429, Vec::new()).header("retry-after", value); + assert_eq!(retry_after(&response("3")), Some(Duration::from_secs(3))); + assert_eq!(retry_after(&response("3600")), Some(MAX_RETRY_AFTER)); + assert_eq!( + retry_after(&response("Wed, 21 Oct 2015 07:28:00 GMT")), + None + ); + assert_eq!(retry_after(&VendorResponse::new(429, Vec::new())), None); + } + + #[test] + fn wrapped_keys_split_losslessly() { + let version = "0123456789abcdef0123456789abcdef"; + let wrapped = compact_ciphertext(WrapAlgorithm::A256Kw, version, b"ciphertext"); + assert_eq!(&wrapped[..3], &[WRAPPED_KEY_FORMAT, 2, 32]); + assert_eq!( + split_wrapped_key(&wrapped), + Some(WrappedKeyParts { + algorithm: WrapAlgorithm::A256Kw, + version, + ciphertext: b"ciphertext", + }) + ); + assert_eq!(wrapped_key_version(&wrapped), Some(version)); + for bad in [ + &[][..], + &[1], + &[1, 1], + &[1, 1, 1], + &[1, 1, 1, b'a'], + &[2, 1, 1, b'a', 9], + &[1, 3, 1, b'a', 9], + &[1, 0, 1, b'a', 9], + &[1, 1, 0, 9], + &[1, 1, 1, b'-', 9], + &[1, 1, 5, b'a', 9], + &[1, 1, 1, 0xff, 9], + ] { + assert_eq!(split_wrapped_key(bad), None, "{bad:?}"); + } + } + + #[test] + fn the_plaintext_is_a_padded_bound_key() { + let key = Protected::new([5u8; 32]); + let plaintext = wrap_plaintext(&key, b"aad"); + assert_eq!(plaintext.len(), WRAP_PLAINTEXT_LEN); + assert_eq!(plaintext.len() % 8, 0); + assert_eq!( + &plaintext[..BOUND_KEY_LEN], + bind_key(&key, b"aad").as_slice() + ); + assert_eq!(&plaintext[BOUND_KEY_LEN..], &[0u8; 7]); + assert_eq!( + open_plaintext(&plaintext, b"aad").unwrap().risky_ref(), + &[5u8; 32] + ); + let kind = |result: Result| match result { + Err(Error::Backend { kind, .. }) => kind, + _ => panic!("not refused"), + }; + assert_eq!( + kind(open_plaintext(&plaintext, b"other")), + BackendErrorKind::InvalidCiphertext + ); + assert_eq!( + kind(open_plaintext(&plaintext[..BOUND_KEY_LEN], b"aad")), + BackendErrorKind::InvalidCiphertext + ); + let mut padded = plaintext.to_vec(); + padded[WRAP_PLAINTEXT_LEN - 1] = 1; + assert_eq!( + kind(open_plaintext(&padded, b"aad")), + BackendErrorKind::InvalidCiphertext + ); + } + + #[test] + fn base64url_decodes_with_or_without_padding() { + assert_eq!(decode_base64url("AQI"), Some(vec![1, 2])); + assert_eq!(decode_base64url("AQI="), Some(vec![1, 2])); + assert_eq!(decode_base64url("+/8"), None); + } +} diff --git a/packages/stack-kms-azure/src/config.rs b/packages/stack-kms-azure/src/config.rs new file mode 100644 index 0000000000..d63513486a --- /dev/null +++ b/packages/stack-kms-azure/src/config.rs @@ -0,0 +1,362 @@ +//! The key identifiers a client accepts, the Azure clouds it knows, and its +//! settings. + +/// How an environment variable is read: the process environment unless a +/// test gives another. +pub type EnvLookup = std::sync::Arc Option + Send + Sync>; + +/// The process environment. An empty variable counts as unset. +pub fn process_env() -> EnvLookup { + std::sync::Arc::new(|name| std::env::var(name).ok().filter(|value| !value.is_empty())) +} + +/// The Key Vault REST API version of every request. +/// +/// 7.4 is a GA version that Key Vault and Managed HSM both serve, and it has +/// every operation this client uses (`GET /keys`, `wrapkey`, `unwrapkey`). +pub const API_VERSION: &str = "7.4"; + +/// Key Vault or Managed HSM. Both speak the same REST API; they differ in +/// the DNS suffix and the token scope. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub enum Flavour { + /// Azure Key Vault (`{vault}.vault.azure.net`): RSA keys, so + /// RSA-OAEP-256. + KeyVault, + /// Azure Key Vault Managed HSM (`{hsm}.managedhsm.azure.net`): `oct-HSM` + /// keys, so A256KW (or RSA-HSM keys, so RSA-OAEP-256). + ManagedHsm, +} + +/// The DNS suffixes of the Azure clouds this client sends to, with the +/// flavour and the token scope of each. A key identifier on any other host +/// is refused, so a keyset record cannot send a bearer token elsewhere. +pub const KNOWN_SUFFIXES: [(&str, Flavour, &str); 6] = [ + ( + "vault.azure.net", + Flavour::KeyVault, + "https://vault.azure.net/.default", + ), + ( + "vault.azure.cn", + Flavour::KeyVault, + "https://vault.azure.cn/.default", + ), + ( + "vault.usgovcloudapi.net", + Flavour::KeyVault, + "https://vault.usgovcloudapi.net/.default", + ), + ( + "managedhsm.azure.net", + Flavour::ManagedHsm, + "https://managedhsm.azure.net/.default", + ), + ( + "managedhsm.azure.cn", + Flavour::ManagedHsm, + "https://managedhsm.azure.cn/.default", + ), + ( + "managedhsm.usgovcloudapi.net", + Flavour::ManagedHsm, + "https://managedhsm.usgovcloudapi.net/.default", + ), +]; + +/// The token scope of Azure Key Vault in the public cloud. +pub const KEY_VAULT_SCOPE: &str = "https://vault.azure.net/.default"; +/// The token scope of Managed HSM in the public cloud. +pub const MANAGED_HSM_SCOPE: &str = "https://managedhsm.azure.net/.default"; + +/// A parsed Azure key identifier: +/// `https://{vault}.vault.azure.net/keys/{name}` or +/// `https://{hsm}.managedhsm.azure.net/keys/{name}`, optionally followed by +/// `/{version}`. +#[derive(Clone, Debug, PartialEq, Eq, Hash)] +pub struct KeyId { + vault: String, + name: String, + version: Option, + flavour: Flavour, + scope: &'static str, +} + +/// A vault, HSM or key name: `[A-Za-z0-9-]`, 1 to 127 characters. +fn is_name(text: &str) -> bool { + !text.is_empty() + && text.len() <= 127 + && text.bytes().all(|b| b.is_ascii_alphanumeric() || b == b'-') +} + +/// A key version: `[A-Za-z0-9]`, 1 to [`MAX_VERSION_LEN`] characters. Azure +/// issues 32 hexadecimal digits. +pub fn is_version(text: &str) -> bool { + !text.is_empty() + && text.len() <= MAX_VERSION_LEN + && text.bytes().all(|b| b.is_ascii_alphanumeric()) +} + +/// The longest key version this client accepts. +pub const MAX_VERSION_LEN: usize = 64; + +impl KeyId { + /// Parse `text`. `None` if it is not an Azure key identifier on a known + /// host ([`KNOWN_SUFFIXES`]) with no port, query, fragment or + /// credentials. The client refuses anything else before any call, so a + /// key reference cannot change the host or the request path. + pub fn parse(text: &str) -> Option { + let rest = text.strip_prefix("https://")?; + let (host, path) = rest.split_once('/')?; + let host = host.to_ascii_lowercase(); + let (vault_name, suffix) = host.split_once('.')?; + if !is_name(vault_name) || vault_name.len() > 63 { + return None; + } + let (_, flavour, scope) = KNOWN_SUFFIXES + .iter() + .find(|(known, _, _)| *known == suffix)?; + let parts: Vec<&str> = path.split('/').collect(); + let (name, version) = match parts.as_slice() { + ["keys", name] => (*name, None), + ["keys", name, version] => (*name, Some(*version)), + _ => return None, + }; + if !is_name(name) || version.is_some_and(|v| !is_version(v)) { + return None; + } + Some(Self { + vault: format!("https://{host}"), + name: name.to_owned(), + version: version.map(str::to_owned), + flavour: *flavour, + scope, + }) + } + + /// The vault or HSM origin, `https://{host}`, lower case. + pub fn vault(&self) -> &str { + &self.vault + } + + /// The key name. + pub fn name(&self) -> &str { + &self.name + } + + /// The pinned version, if the identifier names one. + pub fn version(&self) -> Option<&str> { + self.version.as_deref() + } + + /// Key Vault or Managed HSM. + pub fn flavour(&self) -> Flavour { + self.flavour + } + + /// The token scope of the host: [`KEY_VAULT_SCOPE`], + /// [`MANAGED_HSM_SCOPE`], or the one of a national cloud. + pub fn scope(&self) -> &'static str { + self.scope + } + + /// The same key at `version`. + pub fn with_version(&self, version: &str) -> Self { + Self { + version: Some(version.to_owned()), + ..self.clone() + } + } + + /// The URL of the key (with its version, if any) and `operation` + /// (`wrapkey`), or of the key itself when `operation` is `None`, with + /// the API version. + pub fn url(&self, operation: Option<&str>) -> String { + let mut url = format!("{}/keys/{}", self.vault, self.name); + if let Some(version) = &self.version { + url.push('/'); + url.push_str(version); + } + if let Some(operation) = operation { + url.push('/'); + url.push_str(operation); + } + url.push_str("?api-version="); + url.push_str(API_VERSION); + url + } + + /// The identifier text: `{vault}/keys/{name}[/{version}]`. + pub fn to_kid(&self) -> String { + match &self.version { + Some(version) => format!("{}/keys/{}/{version}", self.vault, self.name), + None => format!("{}/keys/{}", self.vault, self.name), + } + } +} + +/// The key wrap algorithm of a key. The stored wrapped key names it, so an +/// unwrap needs no key read. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub enum WrapAlgorithm { + /// RSA-OAEP with SHA-256 and MGF1 with SHA-256, for `RSA` and `RSA-HSM` + /// keys. Stored as `1`. + RsaOaep256, + /// AES key wrap (RFC 3394) with a 256-bit key, for Managed HSM `oct-HSM` + /// keys. Stored as `2`. + A256Kw, +} + +impl WrapAlgorithm { + /// The JSON Web Algorithm name Azure takes (`RSA-OAEP-256`, `A256KW`). + pub fn name(self) -> &'static str { + match self { + Self::RsaOaep256 => "RSA-OAEP-256", + Self::A256Kw => "A256KW", + } + } + + /// The byte of the stored wrapped key. + pub fn byte(self) -> u8 { + match self { + Self::RsaOaep256 => 1, + Self::A256Kw => 2, + } + } + + /// The algorithm of a stored byte. + pub fn from_byte(byte: u8) -> Option { + match byte { + 1 => Some(Self::RsaOaep256), + 2 => Some(Self::A256Kw), + _ => None, + } + } + + /// The algorithm of a key of JSON Web Key type `kty`, or `None` when the + /// type cannot wrap a key here (for example `EC`). + pub fn for_key_type(kty: &str) -> Option { + match kty { + "RSA" | "RSA-HSM" => Some(Self::RsaOaep256), + "oct" | "oct-HSM" => Some(Self::A256Kw), + _ => None, + } + } +} + +/// The settings of an [`AzureKeyVault`](crate::AzureKeyVault) client. +#[derive(Clone, Debug, Default, PartialEq, Eq)] +pub struct AzureKeyVaultConfig { + algorithm: Option, +} + +impl AzureKeyVaultConfig { + /// The default: the client reads each key once (`GET /keys/{name}`) to + /// learn its type, and its current version when the key identifier has + /// none. + pub fn new() -> Self { + Self::default() + } + + /// Use `algorithm` for every key. With a versioned key identifier the + /// client then reads no key before a wrap. A key whose type does not + /// match is refused. + pub fn with_algorithm(mut self, algorithm: WrapAlgorithm) -> Self { + self.algorithm = Some(algorithm); + self + } + + /// The configured algorithm, if any. + pub fn algorithm(&self) -> Option { + self.algorithm + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const VAULT_KEY: &str = "https://my-vault.vault.azure.net/keys/users"; + const HSM_KEY: &str = + "https://my-hsm.managedhsm.azure.net/keys/users/0123456789abcdef0123456789abcdef"; + + #[test] + fn key_ids_parse() { + let id = KeyId::parse(VAULT_KEY).unwrap(); + assert_eq!(id.vault(), "https://my-vault.vault.azure.net"); + assert_eq!(id.name(), "users"); + assert_eq!(id.version(), None); + assert_eq!(id.flavour(), Flavour::KeyVault); + assert_eq!(id.scope(), KEY_VAULT_SCOPE); + assert_eq!(id.to_kid(), VAULT_KEY); + assert_eq!( + id.url(None), + "https://my-vault.vault.azure.net/keys/users?api-version=7.4" + ); + assert_eq!( + id.with_version("abc").url(Some("wrapkey")), + "https://my-vault.vault.azure.net/keys/users/abc/wrapkey?api-version=7.4" + ); + + let id = KeyId::parse(HSM_KEY).unwrap(); + assert_eq!(id.flavour(), Flavour::ManagedHsm); + assert_eq!(id.scope(), MANAGED_HSM_SCOPE); + assert_eq!(id.version(), Some("0123456789abcdef0123456789abcdef")); + assert_eq!(id.to_kid(), HSM_KEY); + + let id = KeyId::parse("https://My-Vault.VAULT.azure.cn/keys/k").unwrap(); + assert_eq!(id.vault(), "https://my-vault.vault.azure.cn"); + assert_eq!(id.scope(), "https://vault.azure.cn/.default"); + let id = KeyId::parse("https://h.managedhsm.usgovcloudapi.net/keys/k").unwrap(); + assert_eq!(id.scope(), "https://managedhsm.usgovcloudapi.net/.default"); + } + + #[test] + fn anything_else_is_refused() { + for bad in [ + "", + "users", + "http://my-vault.vault.azure.net/keys/users", + "https://my-vault.vault.azure.net/keys/", + "https://my-vault.vault.azure.net/keys/users/", + "https://my-vault.vault.azure.net/keys/users/v1/wrapkey", + "https://my-vault.vault.azure.net/secrets/users", + "https://my-vault.vault.azure.net/keys/us_ers", + "https://my-vault.vault.azure.net/keys/users?x=1", + "https://my-vault.vault.azure.net/keys/users#x", + "https://my-vault.vault.azure.net/keys/users/ab-cd", + "https://my-vault.vault.azure.net:443/keys/users", + "https://user@my-vault.vault.azure.net/keys/users", + "https://my-vault.vault.azure.net.evil.example/keys/users", + "https://evil.example/keys/users", + "https://vault.azure.net/keys/users", + "https://a.b.vault.azure.net/keys/users", + "https://my-vault.vault.azure.net/keys/../users", + "https://my-vault.vault.azure.net//keys/users", + ] { + assert_eq!(KeyId::parse(bad), None, "{bad:?}"); + } + let long = format!("https://v.vault.azure.net/keys/k/{}", "a".repeat(65)); + assert_eq!(KeyId::parse(&long), None); + } + + #[test] + fn algorithms_round_trip() { + for algorithm in [WrapAlgorithm::RsaOaep256, WrapAlgorithm::A256Kw] { + assert_eq!(WrapAlgorithm::from_byte(algorithm.byte()), Some(algorithm)); + } + assert_eq!(WrapAlgorithm::from_byte(0), None); + assert_eq!(WrapAlgorithm::from_byte(3), None); + assert_eq!(WrapAlgorithm::RsaOaep256.name(), "RSA-OAEP-256"); + assert_eq!(WrapAlgorithm::A256Kw.name(), "A256KW"); + assert_eq!( + WrapAlgorithm::for_key_type("RSA-HSM"), + Some(WrapAlgorithm::RsaOaep256) + ); + assert_eq!( + WrapAlgorithm::for_key_type("oct-HSM"), + Some(WrapAlgorithm::A256Kw) + ); + assert_eq!(WrapAlgorithm::for_key_type("EC"), None); + } +} diff --git a/packages/stack-kms-azure/src/lib.rs b/packages/stack-kms-azure/src/lib.rs new file mode 100644 index 0000000000..a1fba313e1 --- /dev/null +++ b/packages/stack-kms-azure/src/lib.rs @@ -0,0 +1,160 @@ +#![doc(html_favicon_url = "https://cipherstash.com/favicon.ico")] +//! The Azure Key Vault and Managed HSM key vendor backend for Stack Encrypt +//! (ADR-0009, ADR-0011). +//! +//! [`AzureKeyVault`] is a small, hand-written client for the Key Vault REST +//! API ([`API_VERSION`]): `wrapkey` and `unwrapkey`, with RSA-OAEP-256 for a +//! Key Vault RSA key and A256KW for a Managed HSM `oct-HSM` key. It sends +//! through a [`VendorTransport`](stack_kms_vendor::VendorTransport) and +//! implements [`KeyVendorBackend`](stack_kms_vendor::KeyVendorBackend), so +//! [`VendorKeySource`](stack_kms_vendor::VendorKeySource) builds both key +//! modes on it. No Azure SDK crate is used, so this crate also builds for +//! wasm32-wasip1, the Go module's WASI guest. It does no RSA itself: Azure +//! holds the key and does the RSA operation. +//! +//! Key Vault has no data key generation, no batch operation and **no +//! associated data** on `wrapkey`. Each wrapped data key is a key from the +//! local CSPRNG and one `wrapkey` call, and the associated data is bound by +//! a digest inside the wrapped plaintext ([`stack_kms_vendor::bound`]); see +//! [`AzureKeyVault`] for the trade-off. +//! +//! # Native Rust +//! +//! ```no_run +//! # async fn example() -> Result<(), Box> { +//! use stack_kms_azure::AzureKeyVault; +//! use stack_kms_vendor::{KeysetRecord, VendorKeySource}; +//! use stack_kms::KeyMode; +//! +//! // AZURE_TENANT_ID, AZURE_CLIENT_ID and AZURE_CLIENT_SECRET, workload +//! // identity, or the managed identity on Azure. +//! let kms = AzureKeyVault::from_env()?; +//! let record = KeysetRecord::create( +//! &kms, +//! "https://my-vault.vault.azure.net/keys/cipherstash-users", +//! KeyMode::Wrapped, +//! Some("users".to_owned()), +//! ) +//! .await?; +//! let source = VendorKeySource::new(kms, [record])?; +//! # let _ = source; +//! # Ok(()) +//! # } +//! ``` +//! +//! # Capabilities and limits +//! +//! The [capability matrix](stack_kms_vendor#capability-matrix) of +//! `stack-kms-vendor` compares this backend with ZeroKMS and the other +//! key vendor backends: key lifetime, vendor calls, what the vendor sees +//! and can refuse, credentials, rotation and known limits. +//! +//! # Access +//! +//! The identity needs the `wrapKey` and `unwrapKey` key permissions and, +//! unless the key identifier names a version and +//! [`AzureKeyVaultConfig::with_algorithm`] is set, `get`. With Azure RBAC, +//! the built-in role "Key Vault Crypto Service Encryption User" has all +//! three (on Managed HSM, "Managed HSM Crypto Service Encryption User"). The +//! key's permitted operations must include `wrapKey` and `unwrapKey`. +//! Rotate the key in Key Vault: stored keys name the version that wrapped +//! them ([`wrapped_key_version`]), and an unwrap still works under an old +//! version until you disable it. +//! +//! # The WASI guest +//! +//! The guest builds [`AzureKeyVault::new`] with its host transport and a +//! [`TokenProvider`] over a host import. Workload identity and managed +//! identity are native only: in the guest, the host's own credentials +//! supply the token. + +#![deny(unsafe_code)] +#![warn(clippy::unwrap_used)] +#![warn(clippy::expect_used)] +#![warn(clippy::panic)] +#![warn(clippy::mem_forget)] +#![warn(clippy::print_stdout)] +#![warn(clippy::print_stderr)] +#![warn(clippy::dbg_macro)] +#![warn(unreachable_pub)] +#![warn(unused_results)] +#![cfg_attr(test, allow(clippy::unwrap_used))] +#![cfg_attr(test, allow(clippy::expect_used))] +#![cfg_attr(test, allow(clippy::panic))] +#![cfg_attr(test, allow(unused_results))] + +pub mod auth; +mod client; +mod config; +mod time; + +pub use auth::{ + AccessToken, AuthError, CachedToken, ClientSecretLogin, TokenProvider, ACCESS_TOKEN_VARIABLE, + DEFAULT_AUTHORITY_HOST, DEFAULT_REFRESH_BEFORE_EXPIRY, +}; +#[cfg(not(target_arch = "wasm32"))] +pub use auth::{DefaultCredentials, ManagedIdentityLogin, WorkloadIdentityLogin}; +pub use client::{ + compact_ciphertext, error_kind, retry_after, split_wrapped_key, wrap_plaintext, + wrapped_key_version, AzureKeyVault, AzureKeyVaultError, WrappedKeyParts, KEY_STATE_CODES, + MAX_RETRY_AFTER, WRAPPED_KEY_FORMAT, WRAP_PLAINTEXT_LEN, +}; +pub use config::{ + is_version, process_env, AzureKeyVaultConfig, EnvLookup, Flavour, KeyId, WrapAlgorithm, + API_VERSION, KEY_VAULT_SCOPE, KNOWN_SUFFIXES, MANAGED_HSM_SCOPE, MAX_VERSION_LEN, +}; +pub use time::{system_clock, Clock}; + +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +mod native { + use std::time::Duration; + + use stack_kms_vendor::{HttpVendorTransport, HttpVendorTransportOptions}; + + use crate::auth::DefaultCredentials; + use crate::{AzureKeyVault, AzureKeyVaultConfig}; + + /// [`AzureKeyVault::from_env`] failed. + #[derive(Debug, thiserror::Error)] + pub enum FromEnvError { + /// The credentials. + #[error(transparent)] + Auth(#[from] crate::AuthError), + /// The HTTP client. + #[error(transparent)] + Transport(#[from] stack_kms_vendor::TransportError), + } + + /// The client [`AzureKeyVault::from_env`] builds. + pub type DefaultAzureKeyVault = + AzureKeyVault>; + + impl DefaultAzureKeyVault { + /// A client with the credentials of [`DefaultCredentials::from_env`]. + /// Native only. + /// + /// The credentials transport opens connections with a 1 second + /// limit, so that off Azure the IMDS probe fails fast. + pub fn from_env() -> Result { + Self::from_env_with(AzureKeyVaultConfig::new()) + } + + /// As [`from_env`](Self::from_env), with `config`. + pub fn from_env_with(config: AzureKeyVaultConfig) -> Result { + let credentials_transport = + HttpVendorTransport::with_options(HttpVendorTransportOptions { + timeout: Duration::from_secs(10), + connect_timeout: Some(Duration::from_secs(1)), + })?; + let auth = DefaultCredentials::from_env(credentials_transport)?; + Ok(AzureKeyVault::new( + config, + HttpVendorTransport::new()?, + auth, + )) + } + } +} + +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +pub use native::{DefaultAzureKeyVault, FromEnvError}; diff --git a/packages/stack-kms-azure/src/time.rs b/packages/stack-kms-azure/src/time.rs new file mode 100644 index 0000000000..6a027a9fa6 --- /dev/null +++ b/packages/stack-kms-azure/src/time.rs @@ -0,0 +1,19 @@ +//! The time source of a client. + +use web_time::{SystemTime, UNIX_EPOCH}; + +/// The time source of a token cache: +/// `SystemTime::now` unless a test sets another. +pub type Clock = std::sync::Arc SystemTime + Send + Sync>; + +/// The system clock. +pub fn system_clock() -> Clock { + std::sync::Arc::new(SystemTime::now) +} + +/// Seconds since the Unix epoch. A time before the epoch is 0. +pub(crate) fn unix_seconds(time: SystemTime) -> u64 { + time.duration_since(UNIX_EPOCH) + .map(|elapsed| elapsed.as_secs()) + .unwrap_or(0) +} diff --git a/packages/stack-kms-azure/tasks.toml b/packages/stack-kms-azure/tasks.toml new file mode 100644 index 0000000000..92577c2eb1 --- /dev/null +++ b/packages/stack-kms-azure/tasks.toml @@ -0,0 +1,32 @@ +# Rustdoc with warnings as errors: a broken intra-doc link or a rustdoc +# warning fails the build. The root `doc` task fans out over every +# `doc:`; the root `test:doc` runs the doc examples. +["doc:stack-kms-azure"] +description = "Build docs for stack-kms-azure with all features (warnings are errors)" +env = { RUSTDOCFLAGS = "-D warnings" } +run = "cargo doc -p stack-kms-azure --no-deps --all-features" + +# The Azure backend against real Azure Key Vault (tests/live.rs). Microsoft +# publishes no Key Vault emulator, so this needs a real key and credentials. +# The test skips itself when STACK_KMS_AZURE_LIVE_KEY is unset, which is right +# for `cargo nextest run --workspace` and wrong here: a skip would read as a +# pass. So this task refuses to run without the key and the credentials. +# integration-azure-key-vault.yml (opt-in, credentialed) calls this task. +# +# STACK_KMS_AZURE_LIVE_KEY=https://.vault.azure.net/keys/ \ +# STACK_KMS_AZURE_LIVE_HSM_KEY=https://.managedhsm.azure.net/keys/ \ # optional +# AZURE_TENANT_ID=... AZURE_CLIENT_ID=... AZURE_CLIENT_SECRET=... \ +# mise run test:live:stack-kms-azure +["test:live:stack-kms-azure"] +description = "Run the stack-kms-azure tests against real Azure Key Vault (needs STACK_KMS_AZURE_LIVE_KEY and credentials)" +run = """ +if [ -z "${STACK_KMS_AZURE_LIVE_KEY:-}" ]; then + echo "error: set STACK_KMS_AZURE_LIVE_KEY to a key identifier (https://.vault.azure.net/keys/)" >&2 + exit 1 +fi +if [ -z "${AZURE_CLIENT_SECRET:-}" ] && [ -z "${AZURE_KEY_VAULT_ACCESS_TOKEN:-}" ] && [ -z "${AZURE_FEDERATED_TOKEN_FILE:-}" ]; then + echo "error: set AZURE_TENANT_ID, AZURE_CLIENT_ID and AZURE_CLIENT_SECRET (or AZURE_KEY_VAULT_ACCESS_TOKEN)" >&2 + exit 1 +fi +mise x --env test -- cargo nextest run --locked -p stack-kms-azure --all-features --test live --no-tests=fail +""" diff --git a/packages/stack-kms-azure/tests/common/mod.rs b/packages/stack-kms-azure/tests/common/mod.rs new file mode 100644 index 0000000000..e08aec4557 --- /dev/null +++ b/packages/stack-kms-azure/tests/common/mod.rs @@ -0,0 +1,637 @@ +//! Shared test support: fixtures, a replaying transport, the test-only RSA +//! key, an RFC 3394 key wrap, and an in-process emulation of the Key Vault +//! and Managed HSM key API and the Entra ID token endpoint. + +#![allow(dead_code, clippy::unwrap_used, clippy::expect_used, clippy::panic)] + +use std::collections::{HashMap, VecDeque}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use aes::cipher::generic_array::GenericArray; +use aes::cipher::{BlockDecrypt, BlockEncrypt, KeyInit}; +use aes::Aes256; +use base64ct::{Base64, Base64UrlUnpadded, Encoding}; +use rsa::pkcs8::DecodePrivateKey; +use rsa::sha2::Sha256; +use rsa::{Oaep, RsaPrivateKey}; +use serde::{Deserialize, Serialize}; +use serde_json::{json, Value}; +use stack_kms_azure::{AccessToken, AzureKeyVault, AzureKeyVaultConfig, KeyId}; +use stack_kms_vendor::{TransportError, VendorRequest, VendorResponse, VendorTransport}; +use web_time::{SystemTime, UNIX_EPOCH}; + +/// The Key Vault of the fixtures. +pub const VAULT: &str = "https://cipherstash-fixture.vault.azure.net"; +/// The Managed HSM of the fixtures. +pub const HSM: &str = "https://cipherstash-fixture.managedhsm.azure.net"; +/// The Key Vault RSA key of the fixtures (no version). +pub const RSA_KEY: &str = "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key"; +/// Its current version in the fixtures. +pub const RSA_VERSION: &str = "1a2b3c4d5e6f708192a3b4c5d6e7f801"; +/// An older version, which `vault_wrap_pinned.json` names. +pub const RSA_OLD_VERSION: &str = "0a1b2c3d4e5f60718293a4b5c6d7e8f9"; +/// The Managed HSM `oct-HSM` key of the fixtures (no version). +pub const OCT_KEY: &str = "https://cipherstash-fixture.managedhsm.azure.net/keys/fixture-oct-key"; +/// Its current version in the fixtures. +pub const OCT_VERSION: &str = "5c6d7e8f90a1b2c3d4e5f60718293a4b"; +/// The static access token of the fixtures. +pub const TOKEN: &str = "fixture-access-token"; +/// The fixture service principal. +pub const TENANT: &str = "test-only-tenant"; +pub const CLIENT_ID: &str = "test-only-client-id"; +pub const CLIENT_SECRET: &str = "fixture-client-secret"; +/// The Entra ID token endpoint of the fixture tenant. +pub const TOKEN_URL: &str = "https://login.microsoftonline.com/test-only-tenant/oauth2/v2.0/token"; +/// The time of the credential fixtures: 2026-11-05T12:00:00Z. +pub const ISSUED_AT: u64 = 1_793_880_000; + +pub fn issued_at() -> SystemTime { + UNIX_EPOCH + Duration::from_secs(ISSUED_AT) +} + +/// The TEST-ONLY 256-bit AES key of the fixture `oct-HSM` key version: the +/// RFC 3394 example key-encryption key (`00 01 .. 1F`). It is no one's key. +pub const TEST_ONLY_KEK: [u8; 32] = [ + 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, + 26, 27, 28, 29, 30, 31, +]; + +/// The TEST-ONLY 2048-bit RSA key of the fixture RSA key, generated for +/// these tests with `openssl genpkey -algorithm RSA -pkeyopt +/// rsa_keygen_bits:2048`. It is no one's key. It is kept as base64 PKCS #8 +/// DER, not as PEM or in a key file, so it does not look like a leaked +/// credential. +pub fn test_only_rsa_key() -> RsaPrivateKey { + let path = format!( + "{}/tests/fixtures/test_only_rsa_key.pkcs8.b64", + env!("CARGO_MANIFEST_DIR") + ); + let text = std::fs::read_to_string(path).unwrap(); + let body: String = text.split_whitespace().collect(); + RsaPrivateKey::from_pkcs8_der(&Base64::decode_vec(&body).unwrap()).unwrap() +} + +const KW_IV: [u8; 8] = [0xA6; 8]; + +/// AES key wrap (RFC 3394) under a 256-bit key: an implementation for the +/// emulator, pinned by the RFC's own example in `tests/fixtures.rs`. +pub fn kw_wrap(kek: &[u8; 32], plaintext: &[u8]) -> Vec { + assert!(plaintext.len().is_multiple_of(8) && plaintext.len() >= 16); + let cipher = Aes256::new(GenericArray::from_slice(kek)); + let n = plaintext.len() / 8; + let mut a = KW_IV; + let mut r: Vec<[u8; 8]> = plaintext.chunks(8).map(|c| c.try_into().unwrap()).collect(); + for j in 0..6 { + for (i, block) in r.iter_mut().enumerate() { + let mut b = GenericArray::clone_from_slice(&[a, *block].concat()); + cipher.encrypt_block(&mut b); + let t = (n * j + i + 1) as u64; + a = (u64::from_be_bytes(b[..8].try_into().unwrap()) ^ t).to_be_bytes(); + block.copy_from_slice(&b[8..]); + } + } + let mut out = a.to_vec(); + for block in r { + out.extend_from_slice(&block); + } + out +} + +/// AES key unwrap (RFC 3394). `None` when the integrity check fails. +pub fn kw_unwrap(kek: &[u8; 32], ciphertext: &[u8]) -> Option> { + if !ciphertext.len().is_multiple_of(8) || ciphertext.len() < 24 { + return None; + } + let cipher = Aes256::new(GenericArray::from_slice(kek)); + let n = ciphertext.len() / 8 - 1; + let mut a: [u8; 8] = ciphertext[..8].try_into().unwrap(); + let mut r: Vec<[u8; 8]> = ciphertext[8..] + .chunks(8) + .map(|c| c.try_into().unwrap()) + .collect(); + for j in (0..6).rev() { + for i in (0..n).rev() { + let t = (n * j + i + 1) as u64; + let a_t = (u64::from_be_bytes(a) ^ t).to_be_bytes(); + let mut b = GenericArray::clone_from_slice(&[a_t, r[i]].concat()); + cipher.decrypt_block(&mut b); + a.copy_from_slice(&b[..8]); + r[i].copy_from_slice(&b[8..]); + } + } + (a == KW_IV).then(|| r.concat()) +} + +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct FixtureRequest { + pub method: String, + pub url: String, + pub headers: Vec<(String, String)>, + pub body: Value, +} + +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct FixtureResponse { + pub status: u16, + #[serde(default)] + pub headers: Vec<(String, String)>, + pub body: Value, +} + +impl FixtureResponse { + pub fn to_response(&self) -> VendorResponse { + let body = match &self.body { + Value::String(text) => text.clone().into_bytes(), + Value::Null => Vec::new(), + other => serde_json::to_vec(other).unwrap(), + }; + let mut response = VendorResponse::new(self.status, body); + for (name, value) in &self.headers { + response = response.header(name, value); + } + response + } +} + +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct Fixture { + pub request: FixtureRequest, + pub response: FixtureResponse, +} + +pub fn load Deserialize<'de>>(name: &str) -> T { + let path = format!("{}/tests/fixtures/{name}", env!("CARGO_MANIFEST_DIR")); + let text = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}")); + serde_json::from_str(&text).unwrap_or_else(|e| panic!("{path}: {e}")) +} + +fn body_json(body: &[u8]) -> Value { + if body.is_empty() { + Value::Null + } else { + serde_json::from_slice(body) + .unwrap_or_else(|_| Value::String(String::from_utf8_lossy(body).into())) + } +} + +/// A transport that checks each request against a fixed one and answers +/// with the fixed response. +#[derive(Default)] +pub struct Replay { + exchanges: Mutex, VendorResponse)>>, + sleeps: Mutex>, +} + +impl Replay { + pub fn new() -> Self { + Self::default() + } + + /// Expect each exchange of a fixture file, in order. + pub fn expect_all(self, fixtures: Vec) -> Self { + for fixture in fixtures { + let response = fixture.response.to_response(); + self.exchanges + .lock() + .unwrap() + .push_back((Some(fixture.request), response)); + } + self + } + + /// Answer the next request, whatever it is, with `response`. + pub fn answer(self, response: VendorResponse) -> Self { + self.exchanges.lock().unwrap().push_back((None, response)); + self + } + + pub fn is_done(&self) -> bool { + self.exchanges.lock().unwrap().is_empty() + } + + pub fn sleeps(&self) -> Vec { + self.sleeps.lock().unwrap().clone() + } +} + +impl VendorTransport for Replay { + async fn send(&self, request: VendorRequest) -> Result { + let (expected, response) = self + .exchanges + .lock() + .unwrap() + .pop_front() + .expect("a request the fixtures do not expect"); + if let Some(expected) = expected { + let actual = json!({ + "method": request.method, + "url": request.url, + "headers": request.headers, + "body": body_json(&request.body), + }); + let wanted = json!({ + "method": expected.method, + "url": expected.url, + "headers": expected.headers, + "body": expected.body, + }); + assert_eq!( + actual, + wanted, + "the request differs from the fixture; actual:\n{}", + serde_json::to_string_pretty(&actual).unwrap() + ); + } + Ok(response) + } + + async fn sleep(&self, duration: Duration) { + self.sleeps.lock().unwrap().push(duration); + } +} + +pub fn replay(name: &str) -> Replay { + Replay::new().expect_all(load(name)) +} + +/// A client with the default configuration over `transport`, with the +/// fixture token. +pub fn fixture_client( + transport: T, +) -> AzureKeyVault { + AzureKeyVault::new( + AzureKeyVaultConfig::new(), + transport, + AccessToken::new(TOKEN), + ) +} + +/// The key material of one key version in the emulator. +#[derive(Clone)] +enum Material { + /// The test-only RSA key: every RSA version uses it. + Rsa, + Aes([u8; 32]), +} + +#[derive(Clone)] +struct EmulatedKey { + rsa: bool, + enabled: bool, + /// Versions, oldest first; the last is current. + versions: Vec<(String, Material, bool)>, +} + +#[derive(Default)] +struct EmulatorState { + /// Keys by `{vault}/keys/{name}`. + keys: HashMap, + /// Access tokens and their scope (`None`: every scope). + tokens: HashMap>, + exchanges: usize, + requests: Vec<(String, String, Value)>, + sleeps: Vec, + throttle: usize, + versions: u32, +} + +/// An in-process emulation of the Key Vault and Managed HSM key API (`GET +/// /keys`, `wrapkey`, `unwrapkey`; RSA-OAEP-256 with the test-only RSA key, +/// A256KW with per-version AES keys; versions, rotation and disabled +/// versions; Azure's error shapes) and of the Entra ID token endpoint (the +/// client credentials grant with [`CLIENT_SECRET`]). It checks the bearer +/// token and its scope on every key request. +/// +/// The contract suite uses random keys and ids, so fixed exchanges cannot +/// serve it: this answers it instead. Clones share one state. +#[derive(Clone)] +pub struct KeyVaultEmulator { + state: Arc>, + rsa: Arc, +} + +impl std::fmt::Debug for KeyVaultEmulator { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str("KeyVaultEmulator") + } +} + +impl Default for KeyVaultEmulator { + fn default() -> Self { + Self::new() + } +} + +impl KeyVaultEmulator { + /// An emulator holding [`RSA_KEY`] and [`OCT_KEY`] (one version each), + /// that accepts [`TOKEN`] for every scope. + pub fn new() -> Self { + let emulator = Self { + state: Arc::default(), + rsa: Arc::new(test_only_rsa_key()), + }; + emulator.add_key(RSA_KEY, true); + emulator.add_key(OCT_KEY, false); + let _ = emulator + .state + .lock() + .unwrap() + .tokens + .insert(TOKEN.to_owned(), None); + emulator + } + + /// Add a key (`{vault}/keys/{name}`) with one version. + pub fn add_key(&self, key: &str, rsa: bool) { + let _ = self.state.lock().unwrap().keys.insert( + key.to_owned(), + EmulatedKey { + rsa, + enabled: true, + versions: Vec::new(), + }, + ); + self.rotate(key); + } + + /// Add a version to `key` and make it current. Returns the version. + pub fn rotate(&self, key: &str) -> String { + let mut state = self.state.lock().unwrap(); + state.versions += 1; + let version = format!("{:032x}", 0xabc0_0000u64 + u64::from(state.versions)); + let seed = state.versions as u8; + let entry = state.keys.get_mut(key).unwrap(); + let material = if entry.rsa { + Material::Rsa + } else { + Material::Aes([seed ^ 0x5A; 32]) + }; + entry.versions.push((version.clone(), material, true)); + version + } + + /// Disable one version of `key`. + pub fn disable(&self, key: &str, version: &str) { + let mut state = self.state.lock().unwrap(); + for entry in &mut state.keys.get_mut(key).unwrap().versions { + if entry.0 == version { + entry.2 = false; + } + } + } + + /// Answer the next `n` key requests with HTTP 429 and `Retry-After: 2`. + pub fn throttle(&self, n: usize) { + self.state.lock().unwrap().throttle = n; + } + + /// Revoke every access token. + pub fn revoke_tokens(&self) { + self.state.lock().unwrap().tokens.clear(); + } + + /// Token endpoint exchanges so far. + pub fn exchanges(&self) -> usize { + self.state.lock().unwrap().exchanges + } + + /// Every key request: method, URL, body. + pub fn requests(&self) -> Vec<(String, String, Value)> { + self.state.lock().unwrap().requests.clone() + } + + pub fn sleeps(&self) -> Vec { + self.state.lock().unwrap().sleeps.clone() + } + + fn json(status: u16, body: Value) -> VendorResponse { + VendorResponse::new(status, body.to_string().into_bytes()) + .header("content-type", "application/json; charset=utf-8") + } + + fn error(status: u16, code: &str, inner: Option<&str>, message: &str) -> VendorResponse { + let mut error = json!({ "code": code, "message": message }); + if let Some(inner) = inner { + error["innererror"] = json!({ "code": inner }); + } + Self::json(status, json!({ "error": error })) + } + + fn token_exchange(&self, body: &[u8]) -> VendorResponse { + let form: HashMap = + url::form_urlencoded::parse(body).into_owned().collect(); + let field = |name: &str| form.get(name).map(String::as_str); + if field("grant_type") != Some("client_credentials") + || field("client_id") != Some(CLIENT_ID) + { + return Self::json(400, json!({ "error": "unauthorized_client" })); + } + if field("client_secret") != Some(CLIENT_SECRET) { + return Self::json( + 401, + json!({ + "error": "invalid_client", + "error_description": "AADSTS7000215: Invalid client secret provided.", + }), + ); + } + let Some(scope) = field("scope").map(str::to_owned) else { + return Self::json(400, json!({ "error": "invalid_scope" })); + }; + let mut state = self.state.lock().unwrap(); + state.exchanges += 1; + let token = format!("fixture-exchanged-token-{}", state.exchanges); + let _ = state.tokens.insert(token.clone(), Some(scope)); + Self::json( + 200, + json!({ "token_type": "Bearer", "expires_in": 3599, "access_token": token }), + ) + } + + fn get_key(&self, key: &str, version: Option<&str>) -> VendorResponse { + let state = self.state.lock().unwrap(); + let Some(entry) = state.keys.get(key) else { + return Self::error( + 404, + "KeyNotFound", + None, + "A key with (name/id) was not found.", + ); + }; + let found = match version { + Some(version) => entry.versions.iter().find(|v| v.0 == version), + None => entry.versions.last(), + }; + let Some((version, _, enabled)) = found else { + return Self::error( + 404, + "KeyNotFound", + None, + "A key with (name/id) was not found.", + ); + }; + let kty = if entry.rsa { "RSA" } else { "oct-HSM" }; + Self::json( + 200, + json!({ + "key": { + "kid": format!("{key}/{version}"), + "kty": kty, + "key_ops": ["wrapKey", "unwrapKey"], + }, + "attributes": { "enabled": *enabled && entry.enabled }, + }), + ) + } + + fn key_operation( + &self, + key: &str, + version: &str, + operation: &str, + body: &Value, + ) -> VendorResponse { + let (rsa, material, enabled) = { + let state = self.state.lock().unwrap(); + let Some(entry) = state.keys.get(key) else { + return Self::error( + 404, + "KeyNotFound", + None, + "A key with (name/id) was not found.", + ); + }; + let Some((_, material, enabled)) = entry.versions.iter().find(|v| v.0 == version) + else { + return Self::error( + 404, + "KeyNotFound", + None, + "A key with (name/id) was not found.", + ); + }; + (entry.rsa, material.clone(), *enabled) + }; + if !enabled { + return Self::error( + 403, + "Forbidden", + Some("KeyDisabled"), + "Operation is not allowed on a disabled key.", + ); + } + let expected_alg = if rsa { "RSA-OAEP-256" } else { "A256KW" }; + if body["alg"] != expected_alg { + return Self::error(400, "BadParameter", None, "Invalid algorithm for this key."); + } + let Some(value) = body["value"] + .as_str() + .and_then(|v| Base64UrlUnpadded::decode_vec(v).ok()) + else { + return Self::error( + 400, + "BadParameter", + None, + "Property value is not base64url.", + ); + }; + let result = match (operation, &material) { + ("wrapkey", Material::Rsa) => Some( + self.rsa + .to_public_key() + .encrypt(&mut rand_core::OsRng, Oaep::new::(), &value) + .unwrap(), + ), + ("unwrapkey", Material::Rsa) => self.rsa.decrypt(Oaep::new::(), &value).ok(), + ("wrapkey", Material::Aes(kek)) => { + (value.len().is_multiple_of(8) && value.len() >= 16).then(|| kw_wrap(kek, &value)) + } + ("unwrapkey", Material::Aes(kek)) => kw_unwrap(kek, &value), + _ => None, + }; + match result { + Some(out) => Self::json( + 200, + json!({ + "kid": format!("{key}/{version}"), + "value": Base64UrlUnpadded::encode_string(&out), + }), + ), + None => Self::error( + 400, + "BadParameter", + None, + "The value could not be unwrapped.", + ), + } + } +} + +impl VendorTransport for KeyVaultEmulator { + async fn send(&self, request: VendorRequest) -> Result { + if request.url == TOKEN_URL { + assert_eq!(request.method, "POST"); + return Ok(self.token_exchange(&request.body)); + } + let (path, query) = request.url.split_once('?').expect("an api-version"); + assert_eq!(query, "api-version=7.4"); + let body = body_json(&request.body); + let (kid, operation) = match path.rsplit_once('/') { + Some((kid, op @ ("wrapkey" | "unwrapkey"))) => (kid, Some(op)), + _ => (path, None), + }; + let id = KeyId::parse(kid).expect("a key identifier"); + { + let mut state = self.state.lock().unwrap(); + state + .requests + .push((request.method.clone(), request.url.clone(), body.clone())); + let token = request + .header_value("authorization") + .and_then(|value| value.strip_prefix("Bearer ")) + .unwrap_or(""); + match state.tokens.get(token) { + Some(None) => {} + Some(Some(scope)) if scope == id.scope() => {} + Some(Some(_)) => { + return Ok(Self::error( + 401, + "Unauthorized", + None, + "AKV10022: Invalid audience.", + )) + } + None => { + return Ok(Self::error( + 401, + "Unauthorized", + None, + "AKV10032: Invalid issuer.", + )) + } + } + if state.throttle > 0 { + state.throttle -= 1; + return Ok(Self::error(429, "Throttled", None, "Too many requests.") + .header("retry-after", "2")); + } + } + let key = format!("{}/keys/{}", id.vault(), id.name()); + Ok(match (request.method.as_str(), operation) { + ("GET", None) => self.get_key(&key, id.version()), + ("POST", Some(operation)) => { + assert_eq!( + request.header_value("content-type"), + Some("application/json") + ); + self.key_operation(&key, id.version().expect("a version"), operation, &body) + } + _ => Self::error(405, "MethodNotAllowed", None, "Method not allowed."), + }) + } + + async fn sleep(&self, duration: Duration) { + self.state.lock().unwrap().sleeps.push(duration); + } +} diff --git a/packages/stack-kms-azure/tests/contract.rs b/packages/stack-kms-azure/tests/contract.rs new file mode 100644 index 0000000000..528dc2170a --- /dev/null +++ b/packages/stack-kms-azure/tests/contract.rs @@ -0,0 +1,276 @@ +//! The shared contract suite, run against the Azure backend over an +//! in-process emulation of the Key Vault key API, in both key modes, for a +//! Key Vault RSA key (RSA-OAEP-256) and a Managed HSM `oct-HSM` key +//! (A256KW). +//! +//! The suite generates random keys and keyset ids, so fixed exchanges +//! cannot answer it; the emulator does (see `common::KeyVaultEmulator`). +//! The request shapes are pinned by `tests/fixtures.rs`, and real Azure Key +//! Vault is exercised by `tests/live.rs`. + +mod common; + +use common::{ + fixture_client, KeyVaultEmulator, CLIENT_ID, CLIENT_SECRET, OCT_KEY, RSA_KEY, RSA_OLD_VERSION, + RSA_VERSION, TENANT, +}; +use stack_encrypt::StackCipher; +use stack_kms::KeyMode; +use stack_kms::{BackendErrorKind, Error}; +use stack_kms_azure::{ + wrapped_key_version, AzureKeyVault, AzureKeyVaultConfig, CachedToken, ClientSecretLogin, +}; +use stack_kms_vendor::derive::wrapped_data_key_aad; +use stack_kms_vendor::{contract, KeyVendorBackend, KeysetRecord, RecordError, VendorKeySource}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; + +fn kind_of(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "azure-key-vault"); + kind + } + other => panic!("not a backend error: {other:?}"), + } +} + +#[tokio::test] +async fn a_key_vault_rsa_key_meets_the_contract() { + let emulator = KeyVaultEmulator::new(); + contract::run(|| fixture_client(emulator.clone()), RSA_KEY).await; +} + +#[tokio::test] +async fn a_managed_hsm_oct_key_meets_the_contract() { + let emulator = KeyVaultEmulator::new(); + contract::run(|| fixture_client(emulator.clone()), OCT_KEY).await; +} + +#[tokio::test] +async fn both_flavours_meet_the_contract_with_client_secret_credentials() { + let emulator = KeyVaultEmulator::new(); + let make = || { + AzureKeyVault::new( + AzureKeyVaultConfig::new(), + emulator.clone(), + CachedToken::new( + ClientSecretLogin::new(TENANT, CLIENT_ID, CLIENT_SECRET, emulator.clone()).unwrap(), + ), + ) + }; + contract::run(make, RSA_KEY).await; + contract::run(make, OCT_KEY).await; + // The emulator checks the audience: a vault token does not open the HSM. + assert!(emulator.exchanges() >= 2); +} + +#[tokio::test] +async fn a_wrapped_key_opens_only_under_its_own_aad() { + for key in [RSA_KEY, OCT_KEY] { + let emulator = KeyVaultEmulator::new(); + let kms = fixture_client(emulator.clone()); + let id = Uuid::from_u128(1); + let aad = wrapped_data_key_aad(id, &[1u8; 16], "users/email"); + let wrapped = kms + .wrap(key, &Protected::new([9u8; 32]), &aad) + .await + .unwrap(); + assert_eq!( + kms.unwrap(key, &wrapped, &aad).await.unwrap().risky_ref(), + &[9u8; 32] + ); + // Azure unwraps it (it sees no associated data); the client refuses. + let other = wrapped_data_key_aad(id, &[1u8; 16], "users/name"); + let error = kms.unwrap(key, &wrapped, &other).await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + // A damaged ciphertext does not unwrap at Azure. + let mut damaged = wrapped.clone(); + let last = damaged.len() - 1; + damaged[last] ^= 1; + let error = kms.unwrap(key, &damaged, &aad).await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + } +} + +#[tokio::test] +async fn rotation_needs_no_change_to_the_key_reference() { + let emulator = KeyVaultEmulator::new(); + let old = fixture_client(emulator.clone()) + .wrap(OCT_KEY, &Protected::new([5u8; 32]), b"aad") + .await + .unwrap(); + let new_version = emulator.rotate(OCT_KEY); + // A new client reads the key again and wraps under the new version. + let kms = fixture_client(emulator.clone()); + let new = kms + .wrap(OCT_KEY, &Protected::new([6u8; 32]), b"aad") + .await + .unwrap(); + assert_ne!(wrapped_key_version(&old), wrapped_key_version(&new)); + assert_eq!(wrapped_key_version(&new), Some(new_version.as_str())); + assert_eq!( + kms.unwrap(OCT_KEY, &old, b"aad").await.unwrap().risky_ref(), + &[5u8; 32] + ); + assert_eq!( + kms.unwrap(OCT_KEY, &new, b"aad").await.unwrap().risky_ref(), + &[6u8; 32] + ); +} + +#[tokio::test] +async fn a_failed_wrap_reads_the_key_again() { + let emulator = KeyVaultEmulator::new(); + let kms = fixture_client(emulator.clone()); + let first = kms + .wrap(OCT_KEY, &Protected::new([1u8; 32]), b"a") + .await + .unwrap(); + let old_version = wrapped_key_version(&first).unwrap().to_owned(); + // An operator rotates the key and disables the old version. + let new_version = emulator.rotate(OCT_KEY); + emulator.disable(OCT_KEY, &old_version); + let error = kms + .wrap(OCT_KEY, &Protected::new([2u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); + let next = kms + .wrap(OCT_KEY, &Protected::new([2u8; 32]), b"a") + .await + .unwrap(); + assert_eq!(wrapped_key_version(&next), Some(new_version.as_str())); + // The old wrapped key does not unwrap while its version is disabled. + let error = kms.unwrap(OCT_KEY, &first, b"a").await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); +} + +#[tokio::test] +async fn a_throttle_waits_for_retry_after_then_reports_throttled() { + let emulator = KeyVaultEmulator::new(); + let kms = fixture_client(emulator.clone()); + emulator.throttle(1); + let error = kms + .wrap(RSA_KEY, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Throttled); + assert_eq!(emulator.sleeps(), [std::time::Duration::from_secs(2)]); + // The adapter retries a throttle; the next call goes through. + let wrapped = kms + .wrap(RSA_KEY, &Protected::new([1u8; 32]), b"a") + .await + .unwrap(); + assert!(wrapped_key_version(&wrapped).is_some()); +} + +#[tokio::test] +async fn a_missing_key_is_not_found() { + let emulator = KeyVaultEmulator::new(); + let kms = fixture_client(emulator); + let missing = "https://cipherstash-fixture.vault.azure.net/keys/missing"; + let error = kms + .wrap(missing, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::NotFound); +} + +#[test] +fn the_same_key_ignores_the_version() { + let kms = fixture_client(KeyVaultEmulator::new()); + let v1 = format!("{RSA_KEY}/{RSA_VERSION}"); + let v2 = format!("{RSA_KEY}/{RSA_OLD_VERSION}"); + assert!(kms.same_key(RSA_KEY, RSA_KEY)); + assert!(kms.same_key(RSA_KEY, &v1)); + assert!(kms.same_key(&v1, &v2)); + // The host is case-insensitive. + assert!(kms.same_key( + RSA_KEY, + &RSA_KEY.replace("cipherstash-fixture", "CIPHERSTASH-FIXTURE") + )); + assert!(!kms.same_key(RSA_KEY, OCT_KEY)); + assert!(!kms.same_key(RSA_KEY, &RSA_KEY.replace("fixture-rsa-key", "other-key"))); + assert!(!kms.same_key( + RSA_KEY, + &RSA_KEY.replace("cipherstash-fixture.vault", "other-vault.vault") + )); + assert!(!kms.same_key(RSA_KEY, "not a key identifier")); +} + +#[tokio::test] +async fn a_wrapped_record_rewraps_to_a_new_version_and_keeps_old_values() { + for key in [RSA_KEY, OCT_KEY] { + for pin in [false, true] { + let emulator = KeyVaultEmulator::new(); + let record = KeysetRecord::create( + &fixture_client(emulator.clone()), + key, + KeyMode::Wrapped, + None, + ) + .await + .unwrap(); + let old = StackCipher::builder() + .kms( + VendorKeySource::new(fixture_client(emulator.clone()), [record.clone()]) + .unwrap(), + ) + .init() + .await + .unwrap(); + let sealed = old + .default_keyset() + .encrypt_tree("v".to_string(), b"ctx") + .await + .unwrap(); + let version = emulator.rotate(key); + let new_ref = if pin { + format!("{key}/{version}") + } else { + key.to_owned() + }; + // A new client, so it reads the current version. + let rewrapped = record + .rewrap(&fixture_client(emulator.clone()), new_ref) + .await + .unwrap(); + assert_eq!( + wrapped_key_version(rewrapped.wrapped_key()), + Some(version.as_str()) + ); + let new = StackCipher::builder() + .kms(VendorKeySource::new(fixture_client(emulator.clone()), [rewrapped]).unwrap()) + .init() + .await + .unwrap(); + let opened: String = new.decrypt(sealed, b"ctx").await.unwrap(); + assert_eq!(opened, "v"); + } + } +} + +#[tokio::test] +async fn a_wrapped_record_does_not_rewrap_to_another_key() { + let emulator = KeyVaultEmulator::new(); + let other = RSA_KEY.replace("fixture-rsa-key", "other-rsa-key"); + emulator.add_key(&other, true); + let kms = fixture_client(emulator.clone()); + let record = KeysetRecord::create(&kms, RSA_KEY, KeyMode::Wrapped, None) + .await + .unwrap(); + let before = emulator.requests().len(); + assert!(matches!( + record.rewrap(&kms, other.as_str()).await, + Err(RecordError::RewrapToAnotherKey { .. }) + )); + assert_eq!(emulator.requests().len(), before, "no vendor call"); + + // A derived record may move to another key. + let record = KeysetRecord::create(&kms, RSA_KEY, KeyMode::Derived, None) + .await + .unwrap(); + let moved = record.rewrap(&kms, other.as_str()).await.unwrap(); + assert_eq!(moved.key_ref(), other); +} diff --git a/packages/stack-kms-azure/tests/fixtures.rs b/packages/stack-kms-azure/tests/fixtures.rs new file mode 100644 index 0000000000..f395a78953 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures.rs @@ -0,0 +1,709 @@ +//! Each Key Vault call, each credential exchange and each error mapping, +//! against fixed HTTP exchanges (`tests/fixtures/*.json`). +//! +//! Microsoft publishes no Key Vault emulator, so no fixture was recorded +//! from Azure. Every response is written by hand in the documented Key +//! Vault, Entra ID and managed identity shapes. Every request side was +//! computed by a separate script, independently of this client (base64url, +//! the associated data frame, the bound key digest, the form bodies), and +//! the client must send exactly those bytes. The RSA-OAEP-256 ciphertexts +//! were made by `openssl pkeyutl` under a TEST-ONLY RSA key generated for +//! these tests (`tests/fixtures/test_only_rsa_key.pkcs8.b64`), and the +//! A256KW ones by an RFC 3394 implementation over `openssl enc +//! -aes-256-ecb`, under the RFC's example key. Neither is anyone's key. +//! `tests/live.rs` runs against real Azure Key Vault. + +mod common; + +use std::collections::HashMap; +use std::sync::Arc; +use std::time::Duration; + +use base64ct::{Base64UrlUnpadded, Encoding}; +use common::{ + fixture_client, issued_at, kw_unwrap, kw_wrap, load, replay, test_only_rsa_key, Fixture, + FixtureResponse, KeyVaultEmulator, Replay, CLIENT_ID, CLIENT_SECRET, OCT_KEY, OCT_VERSION, + RSA_KEY, RSA_OLD_VERSION, RSA_VERSION, TENANT, TEST_ONLY_KEK, TOKEN, +}; +use rsa::sha2::Sha256; +use rsa::Oaep; +use serde::Deserialize; +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor}; +use stack_kms_azure::{ + compact_ciphertext, split_wrapped_key, wrap_plaintext, wrapped_key_version, AccessToken, + AuthError, AzureKeyVault, AzureKeyVaultConfig, CachedToken, ClientSecretLogin, + DefaultCredentials, EnvLookup, ManagedIdentityLogin, TokenProvider, WorkloadIdentityLogin, + WrapAlgorithm, KEY_VAULT_SCOPE, MANAGED_HSM_SCOPE, +}; +use stack_kms_vendor::derive::wrapped_data_key_aad; +use stack_kms_vendor::{ + KeyVendorBackend, TransportError, VendorRequest, VendorResponse, VendorTransport, +}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; + +const KEYSET: Uuid = Uuid::from_u128(0x0f1e2d3c_4b5a_6978_8796_a5b4c3d2e1f0); + +fn data_aad(n: u8) -> Vec { + wrapped_data_key_aad(KEYSET, &[n; 16], "users/email") +} + +fn kind_of(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "azure-key-vault"); + kind + } + other => panic!("not a backend error: {other:?}"), + } +} + +fn b64(value: &serde_json::Value) -> Vec { + Base64UrlUnpadded::decode_vec(value.as_str().unwrap()).unwrap() +} + +/// The stored wrapped key of a fixture's last (wrapkey) exchange. +fn wrapped_of(name: &str, algorithm: WrapAlgorithm, version: &str) -> Vec { + let fixtures: Vec = load(name); + let body = &fixtures.last().unwrap().response.body; + compact_ciphertext(algorithm, version, &b64(&body["value"])) +} + +fn rsa_wrapped() -> Vec { + wrapped_of("vault_wrap.json", WrapAlgorithm::RsaOaep256, RSA_VERSION) +} + +#[tokio::test] +async fn a_key_vault_rsa_key_wraps_and_unwraps() { + let kms = fixture_client(replay("vault_wrap.json")); + let wrapped = kms + .wrap(RSA_KEY, &Protected::new([0x42u8; 32]), &data_aad(0)) + .await + .unwrap(); + assert_eq!(wrapped, rsa_wrapped()); + assert_eq!(wrapped_key_version(&wrapped), Some(RSA_VERSION)); + assert_eq!( + split_wrapped_key(&wrapped).unwrap().algorithm, + WrapAlgorithm::RsaOaep256 + ); + assert!(kms.transport().is_done()); + + let kms = fixture_client(replay("vault_unwrap.json")); + let opened = kms.unwrap(RSA_KEY, &wrapped, &data_aad(0)).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x42u8; 32]); + assert!(kms.transport().is_done()); + + // The unwrap names the stored version, whatever the key reference pins. + let kms = fixture_client(replay("vault_unwrap.json")); + let pinned = format!("{RSA_KEY}/{RSA_OLD_VERSION}"); + let opened = kms.unwrap(&pinned, &wrapped, &data_aad(0)).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x42u8; 32]); +} + +#[tokio::test] +async fn the_key_is_read_once_per_client() { + let fixtures: Vec = load("vault_wrap.json"); + let wrap_again = fixtures[1].clone(); + let kms = fixture_client( + Replay::new() + .expect_all(fixtures) + .expect_all(vec![wrap_again]), + ); + for _ in 0..2 { + let wrapped = kms + .wrap(RSA_KEY, &Protected::new([0x42u8; 32]), &data_aad(0)) + .await + .unwrap(); + assert_eq!(wrapped, rsa_wrapped()); + } + assert!(kms.transport().is_done()); +} + +#[tokio::test] +async fn a_pinned_version_with_a_configured_algorithm_reads_no_key() { + let kms = AzureKeyVault::new( + AzureKeyVaultConfig::new().with_algorithm(WrapAlgorithm::RsaOaep256), + replay("vault_wrap_pinned.json"), + AccessToken::new(TOKEN), + ); + let pinned = format!("{RSA_KEY}/{RSA_OLD_VERSION}"); + let wrapped = kms + .wrap(&pinned, &Protected::new([0x43u8; 32]), &data_aad(1)) + .await + .unwrap(); + assert_eq!( + wrapped, + wrapped_of( + "vault_wrap_pinned.json", + WrapAlgorithm::RsaOaep256, + RSA_OLD_VERSION + ) + ); + assert!(kms.transport().is_done()); + + let kms = fixture_client(replay("vault_unwrap_pinned.json")); + let opened = kms.unwrap(RSA_KEY, &wrapped, &data_aad(1)).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x43u8; 32]); +} + +#[tokio::test] +async fn a_managed_hsm_oct_key_wraps_with_a256kw() { + let kms = fixture_client(replay("hsm_wrap.json")); + let wrapped = kms + .wrap(OCT_KEY, &Protected::new([0x44u8; 32]), &data_aad(2)) + .await + .unwrap(); + assert_eq!( + wrapped, + wrapped_of("hsm_wrap.json", WrapAlgorithm::A256Kw, OCT_VERSION) + ); + // 72 bytes wrapped by A256KW are 80. + assert_eq!(split_wrapped_key(&wrapped).unwrap().ciphertext.len(), 80); + assert!(kms.transport().is_done()); + + let kms = fixture_client(replay("hsm_unwrap.json")); + let opened = kms.unwrap(OCT_KEY, &wrapped, &data_aad(2)).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x44u8; 32]); + assert!(kms.transport().is_done()); +} + +/// The fixture ciphertexts are real: the test-only keys open them to the +/// plaintext the client sends, and the emulator's key wrap is RFC 3394's. +#[test] +fn the_fixture_ciphertexts_open_under_the_test_only_keys() { + let fixtures: Vec = load("vault_wrap.json"); + let plaintext = b64(&fixtures[1].request.body["value"]); + assert_eq!( + plaintext, + wrap_plaintext(&Protected::new([0x42u8; 32]), &data_aad(0)).to_vec() + ); + let ciphertext = b64(&fixtures[1].response.body["value"]); + let opened = test_only_rsa_key() + .decrypt(Oaep::new::(), &ciphertext) + .unwrap(); + assert_eq!(opened, plaintext); + + let fixtures: Vec = load("hsm_wrap.json"); + let plaintext = b64(&fixtures[1].request.body["value"]); + let ciphertext = b64(&fixtures[1].response.body["value"]); + assert_eq!(kw_unwrap(&TEST_ONLY_KEK, &ciphertext).unwrap(), plaintext); + assert_eq!(kw_wrap(&TEST_ONLY_KEK, &plaintext), ciphertext); + + // RFC 3394, 4.6: 256 bits of key data with a 256-bit KEK. + let key_data: Vec = (0..16u8) + .map(|i| i * 0x11) + .chain(0..16u8) + .collect::>(); + let expected = [ + 0x28, 0xC9, 0xF4, 0x04, 0xC4, 0xB8, 0x10, 0xF4, 0xCB, 0xCC, 0xB3, 0x5C, 0xFB, 0x87, 0xF8, + 0x26, 0x3F, 0x57, 0x86, 0xE2, 0xD8, 0x0E, 0xD3, 0x26, 0xCB, 0xC7, 0xF0, 0xE7, 0x1A, 0x99, + 0xF4, 0x3B, 0xFB, 0x98, 0x8B, 0x9B, 0x7A, 0x02, 0xDD, 0x21, + ]; + assert_eq!(kw_wrap(&TEST_ONLY_KEK, &key_data), expected); + assert_eq!(kw_unwrap(&TEST_ONLY_KEK, &expected).unwrap(), key_data); +} + +fn fixed_clock() -> stack_kms_azure::Clock { + Arc::new(issued_at) +} + +#[tokio::test] +async fn a_client_secret_is_exchanged_for_a_token() { + let login = ClientSecretLogin::new( + TENANT, + CLIENT_ID, + CLIENT_SECRET, + replay("client_secret_token.json"), + ) + .unwrap(); + let token = login.token(KEY_VAULT_SCOPE).await.unwrap(); + assert_eq!(token.token(), "fixture-client-secret-token"); + assert_eq!(token.expires_in(), Some(Duration::from_secs(3599))); + assert!(login.transport().is_done()); +} + +#[tokio::test] +async fn a_federated_token_is_exchanged_for_a_token() { + let path = std::env::temp_dir().join(format!("stack-kms-azure-fed-{}", Uuid::new_v4())); + std::fs::write(&path, "fixture.federated.token\n").unwrap(); + let login = WorkloadIdentityLogin::new( + TENANT, + CLIENT_ID, + &path, + replay("workload_identity_token.json"), + ) + .unwrap(); + let token = login.token(MANAGED_HSM_SCOPE).await.unwrap(); + assert_eq!(token.token(), "fixture-workload-identity-token"); + assert!(login.transport().is_done()); + + std::fs::remove_file(&path).unwrap(); + let error = login.token(MANAGED_HSM_SCOPE).await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert!(error.to_string().contains("cannot read")); +} + +#[tokio::test] +async fn imds_gives_a_user_assigned_identity_token() { + let login = ManagedIdentityLogin::imds(replay("imds_token.json")) + .with_client_id("test-only-user-assigned-id"); + let token = login.token(KEY_VAULT_SCOPE).await.unwrap(); + assert_eq!(token.token(), "fixture-imds-token"); + assert_eq!(token.expires_in(), Some(Duration::from_secs(86399))); + assert!(login.transport().is_done()); +} + +#[tokio::test] +async fn app_service_gives_a_token_that_expires_on_a_time() { + let login = ManagedIdentityLogin::app_service( + "http://127.0.0.1:41056/msi/token", + "fixture-identity-header", + replay("app_service_token.json"), + ) + .unwrap() + .with_clock(fixed_clock()); + let token = login.token(MANAGED_HSM_SCOPE).await.unwrap(); + assert_eq!(token.token(), "fixture-app-service-token"); + assert_eq!(token.expires_in(), Some(Duration::from_secs(3600))); + assert!(login.transport().is_done()); +} + +#[tokio::test] +async fn a_refused_secret_is_access_denied_and_says_why() { + let login = ClientSecretLogin::new( + TENANT, + CLIENT_ID, + "the-wrong-secret", + Replay::new().answer(VendorResponse::new( + 401, + br#"{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided."}"#.to_vec(), + )), + ) + .unwrap(); + let error = login.token(KEY_VAULT_SCOPE).await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert!(error.to_string().contains("AADSTS7000215")); + assert!(!error.to_string().contains("the-wrong-secret")); +} + +fn env(pairs: &[(&str, &str)]) -> EnvLookup { + let map: HashMap = pairs + .iter() + .map(|(k, v)| ((*k).to_owned(), (*v).to_owned())) + .collect(); + Arc::new(move |name| map.get(name).cloned()) +} + +#[tokio::test] +async fn the_default_credentials_come_from_the_environment() { + let secret = [ + ("AZURE_TENANT_ID", TENANT), + ("AZURE_CLIENT_ID", CLIENT_ID), + ("AZURE_CLIENT_SECRET", CLIENT_SECRET), + ]; + + let credentials = DefaultCredentials::from_env_lookup( + KeyVaultEmulator::new(), + &env(&[ + ("AZURE_KEY_VAULT_ACCESS_TOKEN", "static-token"), + secret[0], + secret[1], + secret[2], + ]), + ) + .unwrap(); + assert!(matches!(credentials, DefaultCredentials::Token(_))); + assert_eq!( + credentials.token(KEY_VAULT_SCOPE).await.unwrap().token(), + "static-token" + ); + + let emulator = KeyVaultEmulator::new(); + let credentials = DefaultCredentials::from_env_lookup(emulator.clone(), &env(&secret)).unwrap(); + assert!(matches!(credentials, DefaultCredentials::ClientSecret(_))); + assert_eq!( + credentials.token(KEY_VAULT_SCOPE).await.unwrap().token(), + "fixture-exchanged-token-1" + ); + // Cached per scope. + let _ = credentials.token(KEY_VAULT_SCOPE).await.unwrap(); + assert_eq!(emulator.exchanges(), 1); + let _ = credentials.token(MANAGED_HSM_SCOPE).await.unwrap(); + assert_eq!(emulator.exchanges(), 2); + + let credentials = DefaultCredentials::from_env_lookup( + KeyVaultEmulator::new(), + &env(&[ + ("AZURE_TENANT_ID", TENANT), + ("AZURE_CLIENT_ID", CLIENT_ID), + ("AZURE_FEDERATED_TOKEN_FILE", "/var/run/secrets/token"), + ]), + ) + .unwrap(); + assert!(matches!( + credentials, + DefaultCredentials::WorkloadIdentity(_) + )); + + let error = DefaultCredentials::from_env_lookup( + KeyVaultEmulator::new(), + &env(&[ + ("AZURE_TENANT_ID", TENANT), + ("AZURE_CLIENT_ID", CLIENT_ID), + ("AZURE_CLIENT_CERTIFICATE_PATH", "/tmp/cert.pem"), + ]), + ) + .unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert!(error.to_string().contains("not supported")); + + let credentials = DefaultCredentials::from_env_lookup( + KeyVaultEmulator::new(), + &env(&[ + ("IDENTITY_ENDPOINT", "http://127.0.0.1:41056/msi/token"), + ("IDENTITY_HEADER", "h"), + ]), + ) + .unwrap(); + let DefaultCredentials::ManagedIdentity(login) = credentials else { + panic!("not a managed identity"); + }; + assert!(login + .inner() + .url(KEY_VAULT_SCOPE) + .starts_with("http://127.0.0.1:41056/msi/token?api-version=2019-08-01")); + + let credentials = DefaultCredentials::from_env_lookup( + KeyVaultEmulator::new(), + &env(&[("AZURE_CLIENT_ID", "user-assigned")]), + ) + .unwrap(); + let DefaultCredentials::ManagedIdentity(login) = credentials else { + panic!("not a managed identity"); + }; + assert_eq!( + login.inner().url(KEY_VAULT_SCOPE), + "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01\ + &resource=https%3A%2F%2Fvault.azure.net&client_id=user-assigned" + ); + + for bad in [ + vec![ + ("IDENTITY_ENDPOINT", "http://evil.example/token"), + ("IDENTITY_HEADER", "h"), + ], + vec![ + ("AZURE_TENANT_ID", "a/b"), + ("AZURE_CLIENT_ID", CLIENT_ID), + ("AZURE_CLIENT_SECRET", CLIENT_SECRET), + ], + vec![ + secret[0], + secret[1], + secret[2], + ("AZURE_AUTHORITY_HOST", "http://login.example"), + ], + ] { + let error = + DefaultCredentials::from_env_lookup(KeyVaultEmulator::new(), &env(&bad)).unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied, "{bad:?}"); + } +} + +#[derive(Deserialize)] +struct ErrorCase { + name: String, + response: FixtureResponse, + kind: String, +} + +#[tokio::test] +async fn every_written_error_maps_to_its_kind() { + let cases: Vec = load("errors.json"); + assert!(cases.len() >= 13); + let wrapped = rsa_wrapped(); + for case in cases { + let kms = fixture_client(Replay::new().answer(case.response.to_response())); + let error = kms + .unwrap(RSA_KEY, &wrapped, &data_aad(0)) + .await + .unwrap_err(); + let message = format!("{error:?}"); + assert!(!message.contains(TOKEN), "{message}"); + assert_eq!( + format!("{:?}", kind_of(error)), + case.kind, + "{}: {message}", + case.name + ); + } +} + +#[tokio::test] +async fn a_throttle_waits_for_retry_after_first() { + let cases: Vec = load("errors.json"); + let throttled = cases.iter().find(|case| case.name == "throttled").unwrap(); + let kms = fixture_client(Replay::new().answer(throttled.response.to_response())); + let error = kms + .unwrap(RSA_KEY, &rsa_wrapped(), &data_aad(0)) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Throttled); + assert_eq!(kms.transport().sleeps(), [Duration::from_secs(1)]); + + // A 503 with Retry-After does not wait: the adapter does not retry it. + let unavailable = cases + .iter() + .find(|case| case.name == "service unavailable") + .unwrap(); + let kms = fixture_client(Replay::new().answer(unavailable.response.to_response())); + let _ = kms + .unwrap(RSA_KEY, &rsa_wrapped(), &data_aad(0)) + .await + .unwrap_err(); + assert!(kms.transport().sleeps().is_empty()); +} + +/// A transport that never gets a response. +struct Down; + +impl VendorTransport for Down { + async fn send(&self, _request: VendorRequest) -> Result { + Err(TransportError("connection refused".into())) + } + + async fn sleep(&self, _duration: Duration) {} +} + +#[tokio::test] +async fn no_response_is_unavailable() { + let error = fixture_client(Down) + .unwrap(RSA_KEY, &rsa_wrapped(), &data_aad(0)) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); + let error = ManagedIdentityLogin::imds(Down) + .token(KEY_VAULT_SCOPE) + .await + .unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); +} + +struct NoToken; + +impl TokenProvider for NoToken { + async fn token(&self, _scope: &str) -> Result { + Err(AuthError::access_denied("none configured")) + } +} + +#[tokio::test] +async fn no_token_is_access_denied_and_sends_nothing() { + let kms = AzureKeyVault::new(AzureKeyVaultConfig::new(), Replay::new(), NoToken); + let error = kms + .wrap(RSA_KEY, &Protected::new([1u8; 32]), &data_aad(0)) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::AccessDenied); +} + +#[tokio::test] +async fn a_key_reference_that_is_not_a_key_id_is_refused_before_any_call() { + let emulator = KeyVaultEmulator::new(); + let kms = fixture_client(emulator.clone()); + for bad in [ + "", + "fixture-rsa-key", + "https://evil.example/keys/fixture-rsa-key", + "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/v1/unwrapkey", + "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key?x=1", + "http://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key", + ] { + let error = kms + .wrap(bad, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::NotFound, "{bad:?}"); + let error = kms.unwrap(bad, &rsa_wrapped(), b"a").await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::NotFound, "{bad:?}"); + } + assert!(emulator.requests().is_empty()); +} + +#[tokio::test] +async fn a_wrapped_key_that_is_not_one_is_refused_before_any_call() { + let emulator = KeyVaultEmulator::new(); + let kms = fixture_client(emulator.clone()); + for bad in [&b""[..], &[1, 1, 1, b'a'], &[1, 9, 1, b'a', 7], b"CiQA"] { + let error = kms.unwrap(RSA_KEY, bad, b"a").await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + } + assert!(emulator.requests().is_empty()); +} + +fn ok_json(body: serde_json::Value) -> VendorResponse { + VendorResponse::new(200, body.to_string().into_bytes()) +} + +fn key_bundle(kid: &str, kty: &str) -> VendorResponse { + ok_json(serde_json::json!({ "key": { "kid": kid, "kty": kty } })) +} + +#[tokio::test] +async fn a_key_read_that_cannot_be_used_is_refused() { + let current = format!("{RSA_KEY}/{RSA_VERSION}"); + for (response, kind) in [ + (key_bundle(¤t, "EC"), BackendErrorKind::AccessDenied), + ( + key_bundle(&format!("{OCT_KEY}/{OCT_VERSION}"), "RSA"), + BackendErrorKind::Unavailable, + ), + (key_bundle(RSA_KEY, "RSA"), BackendErrorKind::Unavailable), + ( + ok_json(serde_json::json!({})), + BackendErrorKind::Unavailable, + ), + ( + VendorResponse::new(200, b"not json".to_vec()), + BackendErrorKind::Unavailable, + ), + ] { + let kms = fixture_client(Replay::new().answer(response)); + let error = kms + .wrap(RSA_KEY, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), kind); + } + + // A key whose type is not the configured algorithm's. + let kms = AzureKeyVault::new( + AzureKeyVaultConfig::new().with_algorithm(WrapAlgorithm::A256Kw), + Replay::new().answer(key_bundle(¤t, "RSA")), + AccessToken::new(TOKEN), + ); + let error = kms + .wrap(RSA_KEY, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::AccessDenied); + + // A pinned version must be the version Azure reads. + let kms = fixture_client(Replay::new().answer(key_bundle(¤t, "RSA"))); + let error = kms + .wrap( + &format!("{RSA_KEY}/{RSA_OLD_VERSION}"), + &Protected::new([1u8; 32]), + b"a", + ) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); +} + +#[tokio::test] +async fn a_wrap_response_that_cannot_be_trusted_is_unavailable() { + let current = format!("{RSA_KEY}/{RSA_VERSION}"); + for body in [ + serde_json::json!({ "kid": current, "value": "" }), + serde_json::json!({ "kid": current, "value": "+/+/" }), + serde_json::json!({ "kid": current }), + serde_json::json!({ "kid": RSA_KEY, "value": "AQID" }), + serde_json::json!({ "kid": format!("{RSA_KEY}/{RSA_OLD_VERSION}"), "value": "AQID" }), + serde_json::json!({ "value": "AQID" }), + ] { + let kms = fixture_client( + Replay::new() + .answer(key_bundle(¤t, "RSA")) + .answer(ok_json(body.clone())), + ); + let error = kms + .wrap(RSA_KEY, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable, "{body}"); + } +} + +#[tokio::test] +async fn an_unwrap_response_that_cannot_be_trusted_is_refused() { + let plaintext = wrap_plaintext(&Protected::new([7u8; 32]), &data_aad(0)); + let mut padding = plaintext.to_vec(); + padding[71] = 1; + for (value, kind) in [ + (None, BackendErrorKind::Unavailable), + (Some("+/+/".to_owned()), BackendErrorKind::Unavailable), + ( + Some(Base64UrlUnpadded::encode_string(&[7u8; 32])), + BackendErrorKind::InvalidCiphertext, + ), + ( + Some(Base64UrlUnpadded::encode_string(&padding)), + BackendErrorKind::InvalidCiphertext, + ), + ( + Some(Base64UrlUnpadded::encode_string(&wrap_plaintext( + &Protected::new([7u8; 32]), + &data_aad(1), + ))), + BackendErrorKind::InvalidCiphertext, + ), + ] { + let body = match &value { + Some(value) => serde_json::json!({ "kid": RSA_KEY, "value": value }), + None => serde_json::json!({ "kid": RSA_KEY }), + }; + let kms = fixture_client(Replay::new().answer(ok_json(body))); + let error = kms + .unwrap(RSA_KEY, &rsa_wrapped(), &data_aad(0)) + .await + .unwrap_err(); + assert_eq!(kind_of(error), kind, "{value:?}"); + } + // The right plaintext, padded or not on the wire, opens. + let kms = fixture_client(Replay::new().answer(ok_json(serde_json::json!({ + "kid": RSA_KEY, + "value": format!("{}=", Base64UrlUnpadded::encode_string(&plaintext)), + })))); + let opened = kms + .unwrap(RSA_KEY, &rsa_wrapped(), &data_aad(0)) + .await + .unwrap(); + assert_eq!(opened.risky_ref(), &[7u8; 32]); +} + +#[tokio::test] +async fn a_401_drops_the_cached_token() { + let emulator = KeyVaultEmulator::new(); + let kms = AzureKeyVault::new( + AzureKeyVaultConfig::new(), + emulator.clone(), + CachedToken::new( + ClientSecretLogin::new(TENANT, CLIENT_ID, CLIENT_SECRET, emulator.clone()).unwrap(), + ), + ); + let wrapped = kms + .wrap(RSA_KEY, &Protected::new([2u8; 32]), b"a") + .await + .unwrap(); + assert_eq!(emulator.exchanges(), 1); + emulator.revoke_tokens(); + let error = kms.unwrap(RSA_KEY, &wrapped, b"a").await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::AccessDenied); + // The next call gets a new token. + let opened = kms.unwrap(RSA_KEY, &wrapped, b"a").await.unwrap(); + assert_eq!(opened.risky_ref(), &[2u8; 32]); + assert_eq!(emulator.exchanges(), 2); +} + +#[tokio::test] +async fn the_backend_reports_azure_and_sleeps_on_its_transport() { + let emulator = KeyVaultEmulator::new(); + let kms = fixture_client(emulator.clone()); + assert_eq!(kms.vendor(), KeyVendor::Azure); + assert!(kms.supports_mode(KeyMode::Wrapped)); + assert!(kms.supports_mode(KeyMode::Derived)); + assert_eq!(kms.batch_limit(), None); + kms.sleep(Duration::from_millis(50)).await; + assert_eq!(emulator.sleeps(), [Duration::from_millis(50)]); +} diff --git a/packages/stack-kms-azure/tests/fixtures/app_service_token.json b/packages/stack-kms-azure/tests/fixtures/app_service_token.json new file mode 100644 index 0000000000..151fb03cc3 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/app_service_token.json @@ -0,0 +1,31 @@ +[ + { + "request": { + "method": "GET", + "url": "http://127.0.0.1:41056/msi/token?api-version=2019-08-01&resource=https%3A%2F%2Fmanagedhsm.azure.net", + "headers": [ + [ + "x-identity-header", + "fixture-identity-header" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "access_token": "fixture-app-service-token", + "expires_on": "1793883600", + "resource": "https://managedhsm.azure.net", + "token_type": "Bearer", + "client_id": "test-only-system-assigned-id" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/client_secret_token.json b/packages/stack-kms-azure/tests/fixtures/client_secret_token.json new file mode 100644 index 0000000000..ebd61b3797 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/client_secret_token.json @@ -0,0 +1,30 @@ +[ + { + "request": { + "method": "POST", + "url": "https://login.microsoftonline.com/test-only-tenant/oauth2/v2.0/token", + "headers": [ + [ + "content-type", + "application/x-www-form-urlencoded" + ] + ], + "body": "grant_type=client_credentials&client_id=test-only-client-id&scope=https%3A%2F%2Fvault.azure.net%2F.default&client_secret=fixture-client-secret" + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "token_type": "Bearer", + "expires_in": 3599, + "ext_expires_in": 3599, + "access_token": "fixture-client-secret-token" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/errors.json b/packages/stack-kms-azure/tests/fixtures/errors.json new file mode 100644 index 0000000000..3b7f365083 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/errors.json @@ -0,0 +1,167 @@ +[ + { + "name": "an access policy without unwrapKey", + "kind": "AccessDenied", + "response": { + "status": 403, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { + "error": { + "code": "Forbidden", + "message": "The user, group or application 'appid=test-only-client-id;oid=00000000-0000-0000-0000-000000000000;iss=https://sts.windows.net/test-only-tenant/' does not have keys unwrapKey permission on key vault 'cipherstash-fixture;location=eastus'. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125287", + "innererror": { "code": "AccessDenied" } + } + } + } + }, + { + "name": "an RBAC role without unwrapKey", + "kind": "AccessDenied", + "response": { + "status": 403, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { + "error": { + "code": "Forbidden", + "message": "Caller is not authorized to perform action on resource. Action: 'Microsoft.KeyVault/vaults/keys/unwrap/action'", + "innererror": { "code": "ForbiddenByRbac" } + } + } + } + }, + { + "name": "a disabled key version", + "kind": "Unavailable", + "response": { + "status": 403, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { + "error": { + "code": "Forbidden", + "message": "Operation \"unwrapKey\" is not allowed on a disabled key.", + "innererror": { "code": "KeyDisabled" } + } + } + } + }, + { + "name": "a token for the wrong audience", + "kind": "AccessDenied", + "response": { + "status": 401, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { + "error": { + "code": "Unauthorized", + "message": "AKV10022: Invalid audience. Expected https://vault.azure.net, found: https://managedhsm.azure.net." + } + } + } + }, + { + "name": "a 401 challenge with no body", + "kind": "AccessDenied", + "response": { + "status": 401, + "headers": [["www-authenticate", "Bearer authorization=\"https://login.microsoftonline.com/test-only-tenant\", resource=\"https://vault.azure.net\""]], + "body": null + } + }, + { + "name": "a key that does not exist", + "kind": "NotFound", + "response": { + "status": 404, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { + "error": { + "code": "KeyNotFound", + "message": "A key with (name/id) fixture-rsa-key/1a2b3c4d5e6f708192a3b4c5d6e7f801 was not found in this key vault. If you recently deleted this key you may be able to recover it using the correct recovery command. For help resolving this issue, please see https://go.microsoft.com/fwlink/?linkid=2125182" + } + } + } + }, + { + "name": "a Managed HSM resource that does not exist", + "kind": "NotFound", + "response": { + "status": 404, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { "error": { "code": "NotFound", "message": "The requested resource was not found." } } + } + }, + { + "name": "throttled", + "kind": "Throttled", + "response": { + "status": 429, + "headers": [["content-type", "application/json; charset=utf-8"], ["retry-after", "1"]], + "body": { + "error": { + "code": "Throttled", + "message": "Request was not processed because too many requests were received. Reason: A generic reason for the throttling" + } + } + } + }, + { + "name": "a ciphertext that does not unwrap", + "kind": "InvalidCiphertext", + "response": { + "status": 400, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { "error": { "code": "BadParameter", "message": "Error occurred while decrypting the value." } } + } + }, + { + "name": "an expired key", + "kind": "Unavailable", + "response": { + "status": 403, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { + "error": { + "code": "Forbidden", + "message": "Operation \"unwrapKey\" is not allowed on an expired key.", + "innererror": { "code": "KeyExpired" } + } + } + } + }, + { + "name": "another refused request", + "kind": "AccessDenied", + "response": { + "status": 400, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { "error": { "code": "BadRequest", "message": "The request is not valid." } } + } + }, + { + "name": "an internal error", + "kind": "Unavailable", + "response": { + "status": 500, + "headers": [["content-type", "application/json; charset=utf-8"]], + "body": { "error": { "code": "InternalServerError", "message": "An unexpected error occurred." } } + } + }, + { + "name": "service unavailable", + "kind": "Unavailable", + "response": { + "status": 503, + "headers": [["content-type", "application/json; charset=utf-8"], ["retry-after", "5"]], + "body": { "error": { "code": "ServiceUnavailable", "message": "The service is unavailable." } } + } + }, + { + "name": "a gateway error page", + "kind": "Unavailable", + "response": { + "status": 502, + "headers": [["content-type", "text/html"]], + "body": "502 Bad Gateway" + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/hsm_unwrap.json b/packages/stack-kms-azure/tests/fixtures/hsm_unwrap.json new file mode 100644 index 0000000000..b4828c72f5 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/hsm_unwrap.json @@ -0,0 +1,35 @@ +[ + { + "request": { + "method": "POST", + "url": "https://cipherstash-fixture.managedhsm.azure.net/keys/fixture-oct-key/5c6d7e8f90a1b2c3d4e5f60718293a4b/unwrapkey?api-version=7.4", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "alg": "A256KW", + "value": "vfEFCXGY0Avo7uIbmTLXPwgesyrn3PCZTXKIku-sZ2F6SRDA-PMGPsu7P_obZk6xWi8PM8gxTYBuVuKW-hMp_GMe3f_CYmLRcVRPb5-DYTE" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "kid": "https://cipherstash-fixture.managedhsm.azure.net/keys/fixture-oct-key/5c6d7e8f90a1b2c3d4e5f60718293a4b", + "value": "AZNt220tYX6khetAWOuhDwYVOlXKerI-xWj6h4juOyYOREREREREREREREREREREREREREREREREREREREREREQAAAAAAAAA" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/hsm_wrap.json b/packages/stack-kms-azure/tests/fixtures/hsm_wrap.json new file mode 100644 index 0000000000..e2fa554105 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/hsm_wrap.json @@ -0,0 +1,78 @@ +[ + { + "request": { + "method": "GET", + "url": "https://cipherstash-fixture.managedhsm.azure.net/keys/fixture-oct-key?api-version=7.4", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "key": { + "kid": "https://cipherstash-fixture.managedhsm.azure.net/keys/fixture-oct-key/5c6d7e8f90a1b2c3d4e5f60718293a4b", + "kty": "oct-HSM", + "key_ops": [ + "wrapKey", + "unwrapKey", + "encrypt", + "decrypt" + ] + }, + "attributes": { + "enabled": true, + "created": 1793793600, + "updated": 1793793600, + "recoveryLevel": "Recoverable+Purgeable", + "recoverableDays": 90, + "exportable": false + }, + "release_policy": null + } + } + }, + { + "request": { + "method": "POST", + "url": "https://cipherstash-fixture.managedhsm.azure.net/keys/fixture-oct-key/5c6d7e8f90a1b2c3d4e5f60718293a4b/wrapkey?api-version=7.4", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "alg": "A256KW", + "value": "AZNt220tYX6khetAWOuhDwYVOlXKerI-xWj6h4juOyYOREREREREREREREREREREREREREREREREREREREREREQAAAAAAAAA" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "kid": "https://cipherstash-fixture.managedhsm.azure.net/keys/fixture-oct-key/5c6d7e8f90a1b2c3d4e5f60718293a4b", + "value": "vfEFCXGY0Avo7uIbmTLXPwgesyrn3PCZTXKIku-sZ2F6SRDA-PMGPsu7P_obZk6xWi8PM8gxTYBuVuKW-hMp_GMe3f_CYmLRcVRPb5-DYTE" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/imds_token.json b/packages/stack-kms-azure/tests/fixtures/imds_token.json new file mode 100644 index 0000000000..c45d1638c6 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/imds_token.json @@ -0,0 +1,34 @@ +[ + { + "request": { + "method": "GET", + "url": "http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https%3A%2F%2Fvault.azure.net&client_id=test-only-user-assigned-id", + "headers": [ + [ + "metadata", + "true" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "access_token": "fixture-imds-token", + "client_id": "test-only-user-assigned-id", + "expires_in": "86399", + "expires_on": "1793966399", + "ext_expires_in": "86399", + "not_before": "1793880000", + "resource": "https://vault.azure.net", + "token_type": "Bearer" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/test_only_rsa_key.pkcs8.b64 b/packages/stack-kms-azure/tests/fixtures/test_only_rsa_key.pkcs8.b64 new file mode 100644 index 0000000000..c86c7ee745 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/test_only_rsa_key.pkcs8.b64 @@ -0,0 +1,22 @@ +MIIEvAIBADANBgkqhkiG9w0BAQEFAASCBKYwggSiAgEAAoIBAQDK5SgDqGCCorYMpB7z6I+6nxXN +U6plIhXmB9+nUGD/+YFAM5Q0RjfD7vWE25Fe8y6GhoeVm10+K+KOJxbF6lNYtmXJ+nfMOMiAkXpi +xJA7nGKUfC/tSWmRQLZko3O5/hjf1gl+BwQCCRUZK8EFc5ZZUL2dx3s6JW9PF84VRkl4C4LZ6sOh +SinKMXOb0URZUrZv4oT2NWxG7Ssoz8vJe/qxtiiNinAa5cHNd/Ji6W0vOdu6QGTyI4DGk3zdzg5G +PvTuODyaQ8+3yno00Bv76wkqhlLRD4lD8oQQRHIpcYO21c5gi+VDKICUIzq0m+QBHCqtvCSb6bYj +41NC7KLYc9wrAgMBAAECggEAVuIKYIaZzVcf3gnpt0i/Q4scKvDI5sXR2s9EVsF0yChv0K5FqJzp +BF4vhNUBS0eXhMPXC5mcNCNWwMi8V8QgLLzuma9H3YvzUBr77Y9+tAgjrrH/U/0715d+g1n7FIpa +3H8rUWxvqu94uW6jGVKF73lDuWzSt/OP1q668SFIFL8V825MSvmR8Eaz+3Ezc0CAOe+SDJEOdxH/ +QbRIs+RkAYpsTjS3ZY5lsO7CIqKZgHsPPmBEeAVszK3I3jkm9RJb5LqM/66+taGyM4ThqJVC5m1+ +aY9Kb6edZkf80u7tIu23QXs3w4NVcx+8IPSfWV6fOn4GzJUKpJ+ThjuAznguAQKBgQDnXEpScDJW +yCg8Gw9udSI10f+ciu8GJrtMngh6807A7grE83VU4Z4ewp4eMJ9Znz6RCrnNxxjqULe1f9NoKvNo +BDlYZ6TUfeOXdXayDkzMORRFWVlRPtvgRwtqp4x4LJbI+YfgQ0msUGkmuklAgJDKBXa3jBFDTlsW +Mpl//uBiGQKBgQDggMyFr7nb95g2BhwwogmAYCaFNPPV2dd3nLLJnEKbykBtApMoaWQkEfZC9h0o +jSq6i3rjbOvk0UYglIv4Szp3Gj89krJC/qRClyXYb+Pl/rk7q4u11xhFqoYQT5R6l7iBnW7rCBal +pZ0KVrQhxN5Yi/tFRCJhYK1hymws+b/g4wKBgCLkjGD6gt0LuuW1vxINoGc1kLNvULVAyia3/tds +MohuXjlt7lgq3soM2AP9mUrO9YSydWtPh7ZvDgXVjXC8aZFacbufhgB2LAj/sjMQbQyv3xrGau4R +6N9eV55yPoExWFxfJdrwyv+t8O3iUoHj4fJV+J2eBkx8WMh0QSpIqlahAoGAdKX3nq7Dn2tyhv99 +vjczQKS03xm4aDEcJ2Yf72llopTgDMP9uk2oXqWGkxxV4jAXN34ph+mcV/P0wBDu+T/9TM0ydse2 +H+E3b/IhE+hvPZ2wmTR1DzKHxgroRalTuK8EizRF4ZD+U2tNwYVzauhjrxZspHzWcEpPTDRlKKlL +7YkCgYAhll5tqi/dNBGSOi1ipJLbUqQywkfFxxc0RkBLmUMhgjyQVNvcnAHl5cwt9zL5bxDOLO7e +iRQmMb/rq4+OD6q9Pf54f7EP2kBpB+fsERR+vwS0Qrgcn7sQ3POI0jyKwVj01C5uZNSmLs0vM5cu +L1DOuSEC7XBACmG3FfNCZQ5uKQ== diff --git a/packages/stack-kms-azure/tests/fixtures/vault_unwrap.json b/packages/stack-kms-azure/tests/fixtures/vault_unwrap.json new file mode 100644 index 0000000000..faae0a1b47 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/vault_unwrap.json @@ -0,0 +1,35 @@ +[ + { + "request": { + "method": "POST", + "url": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/1a2b3c4d5e6f708192a3b4c5d6e7f801/unwrapkey?api-version=7.4", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "alg": "RSA-OAEP-256", + "value": "keiy6g23HAB9rfi3cXEajutz7geL-m05U7NdS_yi-_Eb4Fqh9m-Y8lhzR5UftrLDUdU_v9VFetCD5HdqN_ExYmSLmwRCKfnhcGFQw55V8tqCZSdJkl8IwqYiFi0K2YkMnYMn3X85dzO3pE_4dVPeQ5wWZJnNwrT5zhnA83cL3VOHl-zMiWj93Lf5SKlouiNwCdRenSJKeJ55v4cfSKLAACwG2ksQcjI-pvAacfKBGxe3oz3wPtEsAEDa6fbXtyCLWiCA6FPOQJCAOOvYVj9mE7YvppEpXAJIwvUOYV0h1yUltb5jzSTyB8ffXnXuAerZNAJbxqpNqIRgpi-hShwEyQ" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "kid": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/1a2b3c4d5e6f708192a3b4c5d6e7f801", + "value": "AZlF03MkT4XV5cJBgwxOtLGKAfREkT2P6GPu03ek2dKRQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkIAAAAAAAAA" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/vault_unwrap_pinned.json b/packages/stack-kms-azure/tests/fixtures/vault_unwrap_pinned.json new file mode 100644 index 0000000000..9f99c34cf0 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/vault_unwrap_pinned.json @@ -0,0 +1,35 @@ +[ + { + "request": { + "method": "POST", + "url": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/0a1b2c3d4e5f60718293a4b5c6d7e8f9/unwrapkey?api-version=7.4", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "alg": "RSA-OAEP-256", + "value": "kUo3PSIGHcHrkMTimf86-mwyOA3lDhEtesdBotERyA1G0udxTs1ktCcMjajthyBQA9bBWreaYbxBnYLqk4_jjWf75vTiNBM9G2ibCL5LkgCk8HyOCHFccZO4ZW5qJZbgg8ZW5bt3UFFurpuDYxsagtiOfE-I4NpOEYd8xHJad_-woX5Zn5thY0qspFC3F2jJKyx_RqcH1E7t-ibkX01IpW8r5zjlc8156vQc70jUKDA_8z1Ztkb1JftTdGi0X3b6idQ8jojNnQh7xGd4R-HIe3qHFuFn4ijeay-XsEqPATO4FqZ71JTETzibYk0ZwOlJ2wvVKYNXYYbAz_e6GoOOtQ" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "kid": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/0a1b2c3d4e5f60718293a4b5c6d7e8f9", + "value": "AYMOBWlv4yz-mSGK1iKQB-ZgCqLsWJwttCwwiMtJEAyDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0MAAAAAAAAA" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/vault_wrap.json b/packages/stack-kms-azure/tests/fixtures/vault_wrap.json new file mode 100644 index 0000000000..199c015be9 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/vault_wrap.json @@ -0,0 +1,80 @@ +[ + { + "request": { + "method": "GET", + "url": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key?api-version=7.4", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "key": { + "kid": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/1a2b3c4d5e6f708192a3b4c5d6e7f801", + "kty": "RSA", + "key_ops": [ + "encrypt", + "decrypt", + "wrapKey", + "unwrapKey" + ], + "n": "yuUoA6hggqK2DKQe8-iPup8VzVOqZSIV5gffp1Bg__mBQDOUNEY3w-71hNuRXvMuhoaHlZtdPivijicWxepTWLZlyfp3zDjIgJF6YsSQO5xilHwv7UlpkUC2ZKNzuf4Y39YJfgcEAgkVGSvBBXOWWVC9ncd7OiVvTxfOFUZJeAuC2erDoUopyjFzm9FEWVK2b-KE9jVsRu0rKM_LyXv6sbYojYpwGuXBzXfyYultLznbukBk8iOAxpN83c4ORj707jg8mkPPt8p6NNAb--sJKoZS0Q-JQ_KEEERyKXGDttXOYIvlQyiAlCM6tJvkARwqrbwkm-m2I-NTQuyi2HPcKw", + "e": "AQAB" + }, + "attributes": { + "enabled": true, + "created": 1793793600, + "updated": 1793793600, + "recoveryLevel": "Recoverable+Purgeable", + "recoverableDays": 90, + "exportable": false + }, + "tags": {} + } + } + }, + { + "request": { + "method": "POST", + "url": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/1a2b3c4d5e6f708192a3b4c5d6e7f801/wrapkey?api-version=7.4", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "alg": "RSA-OAEP-256", + "value": "AZlF03MkT4XV5cJBgwxOtLGKAfREkT2P6GPu03ek2dKRQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkIAAAAAAAAA" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "kid": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/1a2b3c4d5e6f708192a3b4c5d6e7f801", + "value": "keiy6g23HAB9rfi3cXEajutz7geL-m05U7NdS_yi-_Eb4Fqh9m-Y8lhzR5UftrLDUdU_v9VFetCD5HdqN_ExYmSLmwRCKfnhcGFQw55V8tqCZSdJkl8IwqYiFi0K2YkMnYMn3X85dzO3pE_4dVPeQ5wWZJnNwrT5zhnA83cL3VOHl-zMiWj93Lf5SKlouiNwCdRenSJKeJ55v4cfSKLAACwG2ksQcjI-pvAacfKBGxe3oz3wPtEsAEDa6fbXtyCLWiCA6FPOQJCAOOvYVj9mE7YvppEpXAJIwvUOYV0h1yUltb5jzSTyB8ffXnXuAerZNAJbxqpNqIRgpi-hShwEyQ" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/vault_wrap_pinned.json b/packages/stack-kms-azure/tests/fixtures/vault_wrap_pinned.json new file mode 100644 index 0000000000..10221d86f1 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/vault_wrap_pinned.json @@ -0,0 +1,35 @@ +[ + { + "request": { + "method": "POST", + "url": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/0a1b2c3d4e5f60718293a4b5c6d7e8f9/wrapkey?api-version=7.4", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "alg": "RSA-OAEP-256", + "value": "AYMOBWlv4yz-mSGK1iKQB-ZgCqLsWJwttCwwiMtJEAyDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0MAAAAAAAAA" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "kid": "https://cipherstash-fixture.vault.azure.net/keys/fixture-rsa-key/0a1b2c3d4e5f60718293a4b5c6d7e8f9", + "value": "kUo3PSIGHcHrkMTimf86-mwyOA3lDhEtesdBotERyA1G0udxTs1ktCcMjajthyBQA9bBWreaYbxBnYLqk4_jjWf75vTiNBM9G2ibCL5LkgCk8HyOCHFccZO4ZW5qJZbgg8ZW5bt3UFFurpuDYxsagtiOfE-I4NpOEYd8xHJad_-woX5Zn5thY0qspFC3F2jJKyx_RqcH1E7t-ibkX01IpW8r5zjlc8156vQc70jUKDA_8z1Ztkb1JftTdGi0X3b6idQ8jojNnQh7xGd4R-HIe3qHFuFn4ijeay-XsEqPATO4FqZ71JTETzibYk0ZwOlJ2wvVKYNXYYbAz_e6GoOOtQ" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/fixtures/workload_identity_token.json b/packages/stack-kms-azure/tests/fixtures/workload_identity_token.json new file mode 100644 index 0000000000..9d639a37e1 --- /dev/null +++ b/packages/stack-kms-azure/tests/fixtures/workload_identity_token.json @@ -0,0 +1,30 @@ +[ + { + "request": { + "method": "POST", + "url": "https://login.microsoftonline.com/test-only-tenant/oauth2/v2.0/token", + "headers": [ + [ + "content-type", + "application/x-www-form-urlencoded" + ] + ], + "body": "grant_type=client_credentials&client_id=test-only-client-id&scope=https%3A%2F%2Fmanagedhsm.azure.net%2F.default&client_assertion_type=urn%3Aietf%3Aparams%3Aoauth%3Aclient-assertion-type%3Ajwt-bearer&client_assertion=fixture.federated.token" + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "token_type": "Bearer", + "expires_in": 3599, + "ext_expires_in": 3599, + "access_token": "fixture-workload-identity-token" + } + } + } +] diff --git a/packages/stack-kms-azure/tests/live.rs b/packages/stack-kms-azure/tests/live.rs new file mode 100644 index 0000000000..ba105cff05 --- /dev/null +++ b/packages/stack-kms-azure/tests/live.rs @@ -0,0 +1,118 @@ +//! The Azure backend against real Azure Key Vault and Managed HSM. +//! +//! Microsoft publishes no Key Vault emulator, so this is the only test of +//! the real service. It runs when `STACK_KMS_AZURE_LIVE_KEY` names a Key +//! Vault RSA key (`https://{vault}.vault.azure.net/keys/{name}`) and skips +//! itself otherwise. `STACK_KMS_AZURE_LIVE_HSM_KEY`, if set, names a +//! Managed HSM `oct-HSM` key that is tested the same way. The credentials +//! come from the environment +//! ([`DefaultCredentials`](stack_kms_azure::DefaultCredentials)): normally +//! `AZURE_TENANT_ID`, `AZURE_CLIENT_ID` and `AZURE_CLIENT_SECRET` of a +//! service principal with "Key Vault Crypto Service Encryption User" on the +//! key. +//! +//! ```text +//! STACK_KMS_AZURE_LIVE_KEY=https://.vault.azure.net/keys/stack-kms-azure-live \ +//! AZURE_TENANT_ID=... AZURE_CLIENT_ID=... AZURE_CLIENT_SECRET=... \ +//! mise run test:live:stack-kms-azure +//! ``` +//! +//! The opt-in `integration-azure-key-vault.yml` workflow runs it in CI. + +use stack_kms::{BackendErrorKind, Error}; +use stack_kms_azure::{wrapped_key_version, AzureKeyVault, KeyId}; +use stack_kms_vendor::derive::wrapped_data_key_aad; +use stack_kms_vendor::{contract, KeyVendorBackend}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; + +fn live_key(variable: &str) -> Option { + let key = std::env::var(variable).ok().filter(|key| !key.is_empty())?; + assert!( + KeyId::parse(&key).is_some(), + "{variable} is not an Azure key identifier: {key}" + ); + Some(key) +} + +fn live_keys() -> Vec { + ["STACK_KMS_AZURE_LIVE_KEY", "STACK_KMS_AZURE_LIVE_HSM_KEY"] + .into_iter() + .filter_map(live_key) + .collect() +} + +fn kind_of(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "azure-key-vault"); + kind + } + other => panic!("not a backend error: {other:?}"), + } +} + +#[tokio::test] +async fn azure_key_vault_meets_the_contract() { + let keys = live_keys(); + if keys.is_empty() { + eprintln!("skipped: set STACK_KMS_AZURE_LIVE_KEY to run against Azure Key Vault"); + return; + } + for key in keys { + contract::run(|| AzureKeyVault::from_env().expect("credentials"), &key).await; + } +} + +#[tokio::test] +async fn azure_key_vault_binds_the_associated_data_and_maps_errors() { + let keys = live_keys(); + if keys.is_empty() { + eprintln!("skipped: set STACK_KMS_AZURE_LIVE_KEY to run against Azure Key Vault"); + return; + } + for key in keys { + let kms = AzureKeyVault::from_env().expect("credentials"); + let id = Uuid::new_v4(); + let aad = wrapped_data_key_aad(id, &[3u8; 16], "users/email"); + let wrapped = kms + .wrap(&key, &Protected::new([0x5Au8; 32]), &aad) + .await + .unwrap(); + assert!(wrapped_key_version(&wrapped).is_some()); + let opened = kms.unwrap(&key, &wrapped, &aad).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x5Au8; 32]); + + // Other associated data: Azure unwraps, the client refuses. + let other = wrapped_data_key_aad(id, &[3u8; 16], "users/name"); + let error = kms.unwrap(&key, &wrapped, &other).await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + + // A damaged ciphertext. + let mut damaged = wrapped.clone(); + let last = damaged.len() - 1; + damaged[last] ^= 0x01; + let error = kms.unwrap(&key, &damaged, &aad).await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + + // A key that does not exist. + let parsed = KeyId::parse(&key).unwrap(); + let missing = format!( + "{}/keys/stack-kms-azure-no-such-key-{}", + parsed.vault(), + Uuid::new_v4().simple() + ); + let error = kms + .wrap(&missing, &Protected::new([1u8; 32]), &aad) + .await + .unwrap_err(); + let kind = kind_of(error); + assert!( + matches!( + kind, + BackendErrorKind::NotFound | BackendErrorKind::AccessDenied + ), + "{kind:?}" + ); + } +} diff --git a/packages/stack-kms-gcp/Cargo.toml b/packages/stack-kms-gcp/Cargo.toml new file mode 100644 index 0000000000..0ec8dd7e16 --- /dev/null +++ b/packages/stack-kms-gcp/Cargo.toml @@ -0,0 +1,71 @@ +[package] +name = "stack-kms-gcp" +description = "The Google Cloud KMS key vendor backend for Stack Encrypt: a hand-written Cloud KMS client with service account and metadata server credentials, for native Rust and the WASI guest" +version = "0.1.0" +edition.workspace = true +authors.workspace = true +repository.workspace = true +homepage.workspace = true +keywords.workspace = true +categories.workspace = true +license-file = "LICENSE" +# Publishable, in the `stack-kms-vendor` version group with the vendor seam it +# implements. release-plz.toml keeps `release = false` until the first version +# is on crates.io (published by hand; see stack-kms-vendor's Cargo.toml). + +[features] +default = ["http", "service-account"] +# `GcpKms::from_env`: the native transport (`HttpVendorTransport`) and the +# default credentials over it. Native only; on wasm32 it adds nothing. +http = ["stack-kms-vendor/http", "service-account"] +# `ServiceAccountKey` and `ServiceAccountLogin`: the RS256 JWT exchange. Off, +# the crate has no RSA code and no `rsa` dependency. The Go WASI guest turns +# it off: its host supplies the access token (ADR-0011), so the guest never +# signs with a service account key (RUSTSEC-2023-0071, see CIP-5311 in the +# decision log). +service-account = ["dep:rsa", "dep:rand_core"] + +[dependencies] +stack-kms = { path = "../stack-kms", version = "0.2.0", default-features = false } +stack-kms-vendor = { path = "../stack-kms-vendor", version = "0.1.0" } + +vitaminc = { workspace = true, features = ["protected"] } +base64ct = { version = "1.7", features = ["alloc"] } +futures = "0.3.25" +# RS256 for the service account JWT. Pure Rust, so it builds for +# wasm32-wasip1 (ring and aws-lc-rs need a C toolchain there). See the +# CIP-5308 section of docs/plans/2026-11-05-key-vendor-backends-decisions.md +# for RUSTSEC-2023-0071. +rsa = { version = "0.9.10", default-features = false, features = ["std", "sha2", "pem", "u64_digit"], optional = true } +# The blinding RNG of the RSA signature (rsa 0.9 takes rand_core 0.6). +rand_core = { version = "0.6.4", features = ["getrandom"], optional = true } +serde = { workspace = true } +serde_json = { workspace = true } +thiserror = { workspace = true } +url = { workspace = true } +web-time = { workspace = true } +zeroize = { workspace = true } + +[dev-dependencies] +stack-kms-vendor = { path = "../stack-kms-vendor", features = ["test-support", "http"] } +stack-encrypt = { path = "../stack-encrypt", default-features = false } +aes-gcm = { version = "0.10.3", features = ["zeroize"] } +tokio = { workspace = true, features = ["rt", "macros", "time"] } +uuid = { workspace = true } +vitaminc = { workspace = true, features = ["protected", "random"] } + +# Builds `GcpKms::from_env`, which needs the native transport. +[[test]] +name = "live" +required-features = ["http"] + +[[test]] +name = "contract" +required-features = ["service-account"] + +[[test]] +name = "fixtures" +required-features = ["service-account"] + +[package.metadata.docs.rs] +all-features = true diff --git a/packages/stack-kms-gcp/LICENSE b/packages/stack-kms-gcp/LICENSE new file mode 100644 index 0000000000..2cbd67a66d --- /dev/null +++ b/packages/stack-kms-gcp/LICENSE @@ -0,0 +1,96 @@ +# PolyForm Internal Use License 1.0.0 + + + +## Acceptance + +In order to get any license under these terms, you must agree +to them as both strict obligations and conditions to all +your licenses. + +## Copyright License + +The licensor grants you a copyright license for the software +to do everything you might do with the software that would +otherwise infringe the licensor's copyright in it for any +permitted purpose. However, you may only make changes or +new works based on the software according to [Changes and New +Works License](#changes-and-new-works-license), and you may +not distribute the software. + +## Changes and New Works License + +The licensor grants you an additional copyright license to +make changes and new works based on the software for any +permitted purpose. + +## Patent License + +The licensor grants you a patent license for the software that +covers patent claims the licensor can license, or becomes able +to license, that you would infringe by using the software. + +## Fair Use + +You may have "fair use" rights for the software under the +law. These terms do not limit them. + +## Internal Business Use + +Use of the software for the internal business operations of +you and your company is use for a permitted purpose. + +## No Other Rights + +These terms do not allow you to sublicense or transfer any of +your licenses to anyone else, or prevent the licensor from +granting licenses to anyone else. These terms do not imply +any other licenses. + +## Patent Defense + +If you make any written claim that the software infringes or +contributes to infringement of any patent, your patent license +for the software granted under these terms ends immediately. If +your company makes such a claim, your patent license ends +immediately for work on behalf of your company. + +## Violations + +The first time you are notified in writing that you have +violated any of these terms, or done anything with the software +not covered by your licenses, your licenses can nonetheless +continue if you come into full compliance with these terms, +and take practical steps to correct past violations, within +32 days of receiving notice. Otherwise, all your licenses +end immediately. + +## No Liability + +***As far as the law allows, the software comes as is, without +any warranty or condition, and the licensor will not be liable +to you for any damages arising out of these terms or the use +or nature of the software, under any kind of legal claim.*** + +## Definitions + +The **licensor** is the individual or entity offering these +terms, and the **software** is the software the licensor makes +available under these terms. + +**You** refers to the individual or entity agreeing to these +terms. + +**Your company** is any legal entity, sole proprietorship, +or other kind of organization that you work for, plus all +organizations that have control over, are under the control of, +or are under common control with that organization. **Control** +means ownership of substantially all the assets of an entity, +or the power to direct its management and policies by vote, +contract, or otherwise. Control can be direct or indirect. + +**Your licenses** are all the licenses granted to you for the +software under these terms. + +**Use** means anything you do with the software requiring one +of your licenses. diff --git a/packages/stack-kms-gcp/src/auth/metadata.rs b/packages/stack-kms-gcp/src/auth/metadata.rs new file mode 100644 index 0000000000..e283c83090 --- /dev/null +++ b/packages/stack-kms-gcp/src/auth/metadata.rs @@ -0,0 +1,137 @@ +//! The metadata server's token for the attached service account. Native +//! only. + +use std::fmt; + +use stack_kms_vendor::{VendorRequest, VendorTransport}; + +use super::{read_token_response, AccessToken, AuthError, TokenProvider}; +use crate::config::EnvLookup; + +/// The host of the metadata server on Google Cloud. +pub const DEFAULT_METADATA_HOST: &str = "metadata.google.internal"; + +/// The path of the default service account's access token. +pub const METADATA_TOKEN_PATH: &str = "/computeMetadata/v1/instance/service-accounts/default/token"; + +/// The access token of the service account attached to the Compute Engine +/// instance, GKE workload (Workload Identity), Cloud Run service or Cloud +/// Functions function, from the metadata server. The request carries +/// `Metadata-Flavor: Google`. Native only. Wrap it in a +/// [`CachedToken`](super::CachedToken): the server caches the token too and +/// says how long it has left. +/// +/// The token has the scopes of the attached service account (on Compute +/// Engine, the instance's access scopes must include Cloud KMS or +/// `cloud-platform`). +pub struct MetadataServerLogin { + transport: T, + host: String, +} + +impl fmt::Debug for MetadataServerLogin { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("MetadataServerLogin") + .field("host", &self.host) + .finish_non_exhaustive() + } +} + +/// A host, with an optional port: `[A-Za-z0-9.-]` and at most one `:` +/// before digits. Nothing that could change the request path. +fn is_host(text: &str) -> bool { + let (host, port) = match text.split_once(':') { + Some((host, port)) => (host, Some(port)), + None => (text, None), + }; + !host.is_empty() + && host + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'-')) + && port.is_none_or(|port| { + !port.is_empty() && port.len() <= 5 && port.bytes().all(|b| b.is_ascii_digit()) + }) +} + +impl MetadataServerLogin { + /// Ask the metadata server at [`DEFAULT_METADATA_HOST`] through + /// `transport`. + pub fn new(transport: T) -> Self { + Self { + transport, + host: DEFAULT_METADATA_HOST.to_owned(), + } + } + + /// Ask the metadata server at `GCE_METADATA_HOST` (`host` or + /// `host:port`, the name the Google SDKs read) if it is set. + pub fn from_env_lookup(transport: T, env: &EnvLookup) -> Result { + let login = Self::new(transport); + match env("GCE_METADATA_HOST") { + Some(host) => login.with_host(host), + None => Ok(login), + } + } + + /// Ask the metadata server at `host` (`host` or `host:port`) instead. + pub fn with_host(mut self, host: impl Into) -> Result { + let host = host.into(); + if !is_host(&host) { + return Err(AuthError::access_denied(format!( + "{host:?} is not a metadata server host" + ))); + } + self.host = host; + Ok(self) + } + + /// The URL of the token. + pub fn url(&self) -> String { + format!("http://{}{METADATA_TOKEN_PATH}", self.host) + } + + /// The transport. + pub fn transport(&self) -> &T { + &self.transport + } +} + +impl TokenProvider for MetadataServerLogin { + async fn token(&self) -> Result { + let request = VendorRequest::new("GET", self.url()).header("metadata-flavor", "Google"); + let response = self.transport.send(request).await.map_err(|error| { + AuthError::unavailable(format!( + "the metadata server got no response (not on Google Cloud?): {error}" + )) + })?; + read_token_response("the metadata server", &response) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn hosts_are_checked() { + for good in [ + "metadata.google.internal", + "169.254.169.254", + "127.0.0.1:8080", + ] { + assert!(is_host(good), "{good}"); + } + for bad in [ + "", + "a/b", + "a:b", + "a:", + "a:123456", + "user@host", + "a b", + "a?b", + ] { + assert!(!is_host(bad), "{bad:?}"); + } + } +} diff --git a/packages/stack-kms-gcp/src/auth/mod.rs b/packages/stack-kms-gcp/src/auth/mod.rs new file mode 100644 index 0000000000..502995b9f4 --- /dev/null +++ b/packages/stack-kms-gcp/src/auth/mod.rs @@ -0,0 +1,588 @@ +//! OAuth 2.0 access tokens for Cloud KMS, and the credentials that get them. +//! +//! [`TokenProvider`] is the seam: the client asks it for an [`AccessToken`] +//! before each request. This crate provides: +//! +//! * [`AccessToken`] itself: a static token, for example from +//! `gcloud auth print-access-token`. Everywhere. It is used until Google +//! refuses it. +//! * [`ServiceAccountLogin`]: a service account key (the JSON file Google +//! issues, [`ServiceAccountKey`]) signs a JWT with RS256 and exchanges it +//! at the token endpoint (RFC 7523). Everywhere: the key can come from +//! bytes. Feature `service-account` (on by default). +//! * [`MetadataServerLogin`]: the token of the attached service account, +//! from the metadata server of Compute Engine, GKE, Cloud Run and Cloud +//! Functions. **Native only.** +//! * [`CachedToken`]: keeps the token a login returns and gets a new one +//! before it expires. +//! * [`DefaultCredentials`] (native, feature `service-account`): picks one +//! from the environment. +//! +//! In the WASI guest the host supplies the token (ADR-0011): the guest +//! implements [`TokenProvider`] over a host import. +//! +//! **Not supported:** the `authorized_user` file that +//! `gcloud auth application-default login` writes, `external_account` +//! (workload identity federation) files, and service account +//! impersonation. Use a service account key, the metadata server, or a +//! static token. + +use std::fmt; +use std::future::Future; +use std::sync::{Mutex as SyncMutex, PoisonError}; +use std::time::Duration; + +use futures::lock::Mutex; +#[cfg(any(feature = "service-account", not(target_arch = "wasm32")))] +use serde::Deserialize; +use stack_kms::{BackendErrorKind, MaybeSend}; +#[cfg(any(feature = "service-account", not(target_arch = "wasm32")))] +use stack_kms_vendor::{classify_status, VendorResponse}; +use web_time::SystemTime; +use zeroize::Zeroizing; + +use crate::config::EnvLookup; +use crate::time::{system_clock, Clock}; + +#[cfg(not(target_arch = "wasm32"))] +mod metadata; +#[cfg(feature = "service-account")] +mod service_account; + +#[cfg(not(target_arch = "wasm32"))] +pub use metadata::{MetadataServerLogin, DEFAULT_METADATA_HOST, METADATA_TOKEN_PATH}; +#[cfg(feature = "service-account")] +pub use service_account::{ + ServiceAccountKey, ServiceAccountLogin, ASSERTION_LIFETIME, DEFAULT_TOKEN_URI, + JWT_BEARER_GRANT_TYPE, +}; + +/// The OAuth scope of Cloud KMS. +pub const CLOUD_KMS_SCOPE: &str = "https://www.googleapis.com/auth/cloudkms"; + +/// An OAuth 2.0 access token, and how long it lasts from when it was +/// issued. +/// +/// The token is wiped on drop and never printed. +#[derive(Clone)] +pub struct AccessToken { + token: Zeroizing, + expires_in: Option, +} + +impl fmt::Debug for AccessToken { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("AccessToken") + .field("token", &"") + .field("expires_in", &self.expires_in) + .finish() + } +} + +impl AccessToken { + /// A token with no known expiry: it is used until Google refuses it. + pub fn new(token: impl Into) -> Self { + Self { + token: Zeroizing::new(token.into()), + expires_in: None, + } + } + + /// The same token, which expires `expires_in` from now. + pub fn with_expires_in(mut self, expires_in: Duration) -> Self { + self.expires_in = Some(expires_in); + self + } + + /// The token from `GOOGLE_OAUTH_ACCESS_TOKEN`, or `None` when it is not + /// set. + pub fn from_env_lookup(env: &EnvLookup) -> Option { + env("GOOGLE_OAUTH_ACCESS_TOKEN").map(Self::new) + } + + /// The token. + pub fn token(&self) -> &str { + &self.token + } + + /// How long the token lasts from when it was issued. `None` for a token + /// with no known expiry. + pub fn expires_in(&self) -> Option { + self.expires_in + } +} + +/// A token could not be got. +#[derive(Debug, thiserror::Error)] +#[error("Google credentials: {message}")] +pub struct AuthError { + kind: BackendErrorKind, + message: String, +} + +impl AuthError { + /// A failure of `kind`, described by `message`. Never put a secret in + /// the message. + pub fn new(kind: BackendErrorKind, message: impl Into) -> Self { + Self { + kind, + message: message.into(), + } + } + + /// No credentials are configured, or they were refused. + pub fn access_denied(message: impl Into) -> Self { + Self::new(BackendErrorKind::AccessDenied, message) + } + + /// The token endpoint or the metadata server did not answer. + pub fn unavailable(message: impl Into) -> Self { + Self::new(BackendErrorKind::Unavailable, message) + } + + /// The failure kind the client reports. + pub fn kind(&self) -> BackendErrorKind { + self.kind + } +} + +/// Where a client gets its access token. The client asks before each +/// request, so a provider that logs in should cache ([`CachedToken`]). +/// +/// The returned future is bounded by [`MaybeSend`]: `Send` on native +/// targets, unbounded on wasm32. +pub trait TokenProvider: Send + Sync { + /// The current token. + fn token(&self) -> impl Future> + MaybeSend; + + /// Google refused the current token (HTTP 401). A caching provider + /// drops it, so that the next request gets a new one. The default does + /// nothing. + fn invalidate(&self) {} +} + +impl TokenProvider for AccessToken { + fn token(&self) -> impl Future> + MaybeSend { + std::future::ready(Ok(self.clone())) + } +} + +#[cfg(any(feature = "service-account", not(target_arch = "wasm32")))] +#[derive(Deserialize)] +struct TokenResponse { + access_token: Option, + #[serde(default)] + expires_in: u64, +} + +#[cfg(any(feature = "service-account", not(target_arch = "wasm32")))] +#[derive(Deserialize)] +struct TokenErrorBody { + error: Option, + error_description: Option, +} + +/// The access token of a token endpoint or metadata server response. +/// `source` names it in errors. +#[cfg(any(feature = "service-account", not(target_arch = "wasm32")))] +pub(crate) fn read_token_response( + source: &str, + response: &VendorResponse, +) -> Result { + if let Some(kind) = classify_status(response.status) { + // A refused grant is HTTP 400 (`invalid_grant`); a metadata server + // with no service account answers 404. Neither is a missing key or + // a damaged ciphertext. + let kind = match kind { + BackendErrorKind::InvalidCiphertext | BackendErrorKind::NotFound => { + BackendErrorKind::AccessDenied + } + kind => kind, + }; + let detail = serde_json::from_slice::(&response.body) + .ok() + .map(|body| { + let error = match body.error { + Some(serde_json::Value::String(text)) => text, + Some(serde_json::Value::Object(object)) => object + .get("message") + .and_then(|m| m.as_str()) + .unwrap_or_default() + .to_owned(), + _ => String::new(), + }; + match body.error_description { + Some(description) => format!("{error}: {description}"), + None => error, + } + }) + .unwrap_or_default(); + return Err(AuthError::new( + kind, + format!("{source} failed with HTTP {}: {detail}", response.status), + )); + } + let parsed: TokenResponse = serde_json::from_slice(&response.body).map_err(|_| { + AuthError::unavailable(format!("{source} returned a response that is not JSON")) + })?; + let token = Zeroizing::new( + parsed + .access_token + .filter(|token| !token.is_empty()) + .ok_or_else(|| AuthError::unavailable(format!("{source} returned no access token")))?, + ); + let token = AccessToken { + token, + expires_in: None, + }; + Ok(match parsed.expires_in { + 0 => token, + seconds => token.with_expires_in(Duration::from_secs(seconds)), + }) +} + +/// The default time before a token expires at which [`CachedToken`] gets a +/// new one. For a short-lived token the window is a third of its life. +pub const DEFAULT_REFRESH_BEFORE_EXPIRY: Duration = Duration::from_secs(5 * 60); + +/// A token in the cache, and when to get the next one. +struct Cached { + token: AccessToken, + /// When to get a new token. `None`: never. + refresh_at: Option, + /// When the token expires. `None`: never. + expires_at: Option, +} + +/// Keeps the token `inner` returns, and gets a new one when it is near its +/// expiry: [`DEFAULT_REFRESH_BEFORE_EXPIRY`] before, or a third of its life +/// before when it lives less than three times that. +/// +/// One fetch runs at a time: callers that arrive during a fetch wait for it +/// and share its result. If a fetch fails while the cached token has not +/// expired, the cached token is still returned. A token with no expiry is +/// kept until Google refuses it ([`TokenProvider::invalidate`]). +pub struct CachedToken

{ + inner: P, + cached: SyncMutex>, + fetch: Mutex<()>, + refresh_before: Duration, + clock: Clock, +} + +impl fmt::Debug for CachedToken

{ + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("CachedToken") + .field("inner", &self.inner) + .field("refresh_before", &self.refresh_before) + .finish_non_exhaustive() + } +} + +impl CachedToken

{ + /// Cache the token of `inner`. + pub fn new(inner: P) -> Self { + Self { + inner, + cached: SyncMutex::new(None), + fetch: Mutex::new(()), + refresh_before: DEFAULT_REFRESH_BEFORE_EXPIRY, + clock: system_clock(), + } + } + + /// Get a new token at most `refresh_before` before the token expires. + pub fn with_refresh_before(mut self, refresh_before: Duration) -> Self { + self.refresh_before = refresh_before; + self + } + + /// Read the time from `clock`. + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = clock; + self + } + + /// The provider whose token is cached. + pub fn inner(&self) -> &P { + &self.inner + } + + /// The cached token, if it does not need a refresh yet. + fn fresh(&self, now: SystemTime) -> Option { + let cached = self.cached.lock().unwrap_or_else(PoisonError::into_inner); + cached + .as_ref() + .filter(|cached| cached.refresh_at.is_none_or(|at| now < at)) + .map(|cached| cached.token.clone()) + } +} + +impl TokenProvider for CachedToken

{ + async fn token(&self) -> Result { + if let Some(token) = self.fresh((self.clock)()) { + return Ok(token); + } + let _fetch = self.fetch.lock().await; + // Another caller may have got a token while this one waited. + let now = (self.clock)(); + if let Some(token) = self.fresh(now) { + return Ok(token); + } + match self.inner.token().await { + Ok(token) => { + let (refresh_at, expires_at) = match token.expires_in { + Some(life) => { + let window = self.refresh_before.min(life / 3); + (now.checked_add(life - window), now.checked_add(life)) + } + None => (None, None), + }; + *self.cached.lock().unwrap_or_else(PoisonError::into_inner) = Some(Cached { + token: token.clone(), + refresh_at, + expires_at, + }); + Ok(token) + } + Err(error) => { + let cached = self.cached.lock().unwrap_or_else(PoisonError::into_inner); + match cached.as_ref() { + Some(cached) if cached.expires_at.is_some_and(|end| now < end) => { + Ok(cached.token.clone()) + } + _ => Err(error), + } + } + } + } + + fn invalidate(&self) { + *self.cached.lock().unwrap_or_else(PoisonError::into_inner) = None; + self.inner.invalidate(); + } +} + +#[cfg(all(feature = "service-account", not(target_arch = "wasm32")))] +mod default { + use stack_kms_vendor::VendorTransport; + + use super::*; + use crate::config::process_env; + + /// The credentials [`DefaultCredentials::from_env`] picks. + #[derive(Debug)] + pub enum DefaultCredentials { + /// `GOOGLE_OAUTH_ACCESS_TOKEN`. + Token(AccessToken), + /// The service account key file at `GOOGLE_APPLICATION_CREDENTIALS`. + ServiceAccount(Box>>), + /// The metadata server. + MetadataServer(CachedToken>), + } + + impl DefaultCredentials { + /// Pick credentials from the environment, in this order: + /// + /// 1. `GOOGLE_OAUTH_ACCESS_TOKEN`: a static access token. + /// 2. `GOOGLE_APPLICATION_CREDENTIALS`: the path of a service + /// account key file. A file of another type (an + /// `authorized_user` or `external_account` file) is refused. + /// 3. Otherwise the metadata server, at `GCE_METADATA_HOST` if set. + /// + /// The logins send through `transport` and are cached. + pub fn from_env(transport: T) -> Result { + Self::from_env_lookup(transport, &process_env()) + } + + /// As [`from_env`](Self::from_env), reading the variables with `env`. + pub fn from_env_lookup(transport: T, env: &EnvLookup) -> Result { + if let Some(token) = AccessToken::from_env_lookup(env) { + return Ok(Self::Token(token)); + } + if let Some(path) = env("GOOGLE_APPLICATION_CREDENTIALS") { + let key = ServiceAccountKey::from_file(path)?; + return Ok(Self::ServiceAccount(Box::new(CachedToken::new( + ServiceAccountLogin::new(key, transport), + )))); + } + Ok(Self::MetadataServer(CachedToken::new( + MetadataServerLogin::from_env_lookup(transport, env)?, + ))) + } + } + + impl TokenProvider for DefaultCredentials { + async fn token(&self) -> Result { + match self { + Self::Token(token) => Ok(token.clone()), + Self::ServiceAccount(login) => login.token().await, + Self::MetadataServer(login) => login.token().await, + } + } + + fn invalidate(&self) { + match self { + Self::Token(_) => {} + Self::ServiceAccount(login) => login.invalidate(), + Self::MetadataServer(login) => login.invalidate(), + } + } + } +} + +#[cfg(all(feature = "service-account", not(target_arch = "wasm32")))] +pub use default::DefaultCredentials; + +#[cfg(test)] +mod tests { + use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering}; + use std::sync::Arc; + + use web_time::UNIX_EPOCH; + + use super::*; + + #[test] + fn debug_hides_the_token() { + let text = format!("{:?}", AccessToken::new("the-access-token")); + assert!(!text.contains("the-access-token")); + } + + struct Counting { + calls: AtomicUsize, + fail: AtomicBool, + life: Option, + } + + impl TokenProvider for Counting { + async fn token(&self) -> Result { + let call = self.calls.fetch_add(1, Ordering::SeqCst); + if self.fail.load(Ordering::SeqCst) { + return Err(AuthError::unavailable("down")); + } + let token = AccessToken::new(format!("token-{call}")); + Ok(match self.life { + Some(life) => token.with_expires_in(life), + None => token, + }) + } + } + + fn cached(life: Option) -> (CachedToken, Arc) { + let now = Arc::new(AtomicU64::new(1_000_000)); + let clock_now = Arc::clone(&now); + let clock: Clock = + Arc::new(move || UNIX_EPOCH + Duration::from_secs(clock_now.load(Ordering::SeqCst))); + let provider = Counting { + calls: AtomicUsize::new(0), + fail: false.into(), + life, + }; + (CachedToken::new(provider).with_clock(clock), now) + } + + async fn token_of(cache: &CachedToken) -> String { + cache.token().await.unwrap().token().to_owned() + } + + #[tokio::test] + async fn a_token_is_got_again_before_it_expires() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(3600 - 301, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-1"); + assert_eq!(cache.inner().calls.load(Ordering::SeqCst), 2); + } + + #[tokio::test] + async fn a_short_life_refreshes_a_third_before_its_end() { + let (cache, now) = cached(Some(Duration::from_secs(60))); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(39, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-1"); + } + + #[tokio::test] + async fn a_token_with_no_expiry_is_kept_until_invalidated() { + let (cache, now) = cached(None); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(10 * 365 * 86_400, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + cache.invalidate(); + assert_eq!(token_of(&cache).await, "token-1"); + } + + #[tokio::test] + async fn a_failed_fetch_keeps_a_token_that_has_not_expired() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + assert_eq!(token_of(&cache).await, "token-0"); + cache.inner().fail.store(true, Ordering::SeqCst); + now.fetch_add(3500, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(100, Ordering::SeqCst); + let error = cache.token().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); + } + + #[tokio::test] + async fn the_refresh_window_is_configurable() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + let cache = cache.with_refresh_before(Duration::ZERO); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(3599, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-1"); + } + + #[test] + fn token_responses_are_read_and_failures_classified() { + let ok = VendorResponse::new( + 200, + br#"{"access_token":"t","expires_in":3599,"token_type":"Bearer"}"#.to_vec(), + ); + let token = read_token_response("x", &ok).unwrap(); + assert_eq!(token.token(), "t"); + assert_eq!(token.expires_in(), Some(Duration::from_secs(3599))); + for (status, body, kind) in [ + ( + 400, + &br#"{"error":"invalid_grant","error_description":"Invalid JWT Signature."}"#[..], + BackendErrorKind::AccessDenied, + ), + (401, b"{}", BackendErrorKind::AccessDenied), + (404, b"not found", BackendErrorKind::AccessDenied), + (429, b"", BackendErrorKind::Throttled), + (503, b"", BackendErrorKind::Unavailable), + (200, b"not json", BackendErrorKind::Unavailable), + ( + 200, + br#"{"access_token":""}"#, + BackendErrorKind::Unavailable, + ), + ] { + let error = + read_token_response("x", &VendorResponse::new(status, body.to_vec())).unwrap_err(); + assert_eq!(error.kind(), kind, "{status}"); + } + let error = read_token_response( + "the token endpoint", + &VendorResponse::new( + 400, + br#"{"error":"invalid_grant","error_description":"Invalid JWT Signature."}"# + .to_vec(), + ), + ) + .unwrap_err(); + assert_eq!( + error.to_string(), + "Google credentials: the token endpoint failed with HTTP 400: \ + invalid_grant: Invalid JWT Signature." + ); + } +} diff --git a/packages/stack-kms-gcp/src/auth/service_account.rs b/packages/stack-kms-gcp/src/auth/service_account.rs new file mode 100644 index 0000000000..b7cacb6122 --- /dev/null +++ b/packages/stack-kms-gcp/src/auth/service_account.rs @@ -0,0 +1,279 @@ +//! Service account keys and the JWT bearer token exchange (RFC 7523). + +use std::fmt; +use std::time::Duration; + +use base64ct::{Base64UrlUnpadded, Encoding}; +use rsa::pkcs1v15::SigningKey; +use rsa::pkcs8::DecodePrivateKey; +use rsa::sha2::Sha256; +use rsa::signature::{RandomizedSigner, SignatureEncoding}; +use rsa::RsaPrivateKey; +use serde::{Deserialize, Serialize}; +use stack_kms_vendor::{VendorRequest, VendorTransport}; +use web_time::SystemTime; +use zeroize::Zeroizing; + +use super::{read_token_response, AccessToken, AuthError, TokenProvider, CLOUD_KMS_SCOPE}; +use crate::time::{system_clock, unix_seconds, Clock}; + +/// Google's OAuth 2.0 token endpoint, the `token_uri` of every service +/// account key Google issues. +pub const DEFAULT_TOKEN_URI: &str = "https://oauth2.googleapis.com/token"; + +/// The `grant_type` of the JWT bearer token exchange (RFC 7523). +pub const JWT_BEARER_GRANT_TYPE: &str = "urn:ietf:params:oauth:grant-type:jwt-bearer"; + +/// How long a signed assertion is valid: one hour, the most Google accepts. +pub const ASSERTION_LIFETIME: Duration = Duration::from_secs(3600); + +/// A service account key: the JSON file Google issues for a service +/// account (`"type": "service_account"`). +/// +/// The private key is wiped on drop and never printed. Only the fields the +/// token exchange needs are read: `client_email`, `private_key`, +/// `private_key_id` and `token_uri`. +pub struct ServiceAccountKey { + client_email: String, + private_key_id: Option, + token_uri: String, + signing_key: SigningKey, +} + +impl fmt::Debug for ServiceAccountKey { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("ServiceAccountKey") + .field("client_email", &self.client_email) + .field("private_key_id", &self.private_key_id) + .field("token_uri", &self.token_uri) + .field("private_key", &"") + .finish() + } +} + +#[derive(Deserialize)] +struct KeyFile { + #[serde(rename = "type")] + kind: Option, + client_email: Option, + private_key: Option, + private_key_id: Option, + token_uri: Option, +} + +#[derive(Serialize)] +struct Header<'a> { + alg: &'static str, + #[serde(skip_serializing_if = "Option::is_none")] + kid: Option<&'a str>, + typ: &'static str, +} + +#[derive(Serialize)] +struct Claims<'a> { + iss: &'a str, + scope: &'a str, + aud: &'a str, + iat: u64, + exp: u64, +} + +impl ServiceAccountKey { + /// Read a service account key from the bytes of its JSON file. Refuses + /// another type of credentials file, a key that is not an RSA PKCS #8 + /// PEM key, and a `token_uri` that is not HTTPS. + pub fn from_json(json: &[u8]) -> Result { + let file: KeyFile = serde_json::from_slice(json) + .map_err(|_| AuthError::access_denied("the service account key is not JSON"))?; + match file.kind.as_deref() { + Some("service_account") => {} + Some(other) => { + return Err(AuthError::access_denied(format!( + "a credentials file of type {other:?} is not supported: \ + use a service account key (\"service_account\")" + ))) + } + None => { + return Err(AuthError::access_denied( + "the credentials file has no \"type\"", + )) + } + } + let client_email = file + .client_email + .filter(|email| !email.is_empty()) + .ok_or_else(|| { + AuthError::access_denied("the service account key has no client_email") + })?; + let pem = Zeroizing::new(file.private_key.ok_or_else(|| { + AuthError::access_denied("the service account key has no private_key") + })?); + let private_key = RsaPrivateKey::from_pkcs8_pem(&pem).map_err(|_| { + AuthError::access_denied("the private_key is not an RSA PKCS #8 PEM key") + })?; + let token_uri = file + .token_uri + .unwrap_or_else(|| DEFAULT_TOKEN_URI.to_owned()); + let parsed = url::Url::parse(&token_uri) + .map_err(|_| AuthError::access_denied("the token_uri is not a URL"))?; + if parsed.scheme() != "https" || parsed.host().is_none() { + return Err(AuthError::access_denied( + "the token_uri is not an HTTPS URL", + )); + } + Ok(Self { + client_email, + private_key_id: file.private_key_id.filter(|id| !id.is_empty()), + token_uri, + signing_key: SigningKey::::new(private_key), + }) + } + + /// Read a service account key file. Native only. + #[cfg(not(target_arch = "wasm32"))] + pub fn from_file(path: impl AsRef) -> Result { + let path = path.as_ref(); + let json = Zeroizing::new(std::fs::read(path).map_err(|error| { + AuthError::access_denied(format!( + "cannot read the service account key at {}: {}", + path.display(), + error.kind() + )) + })?); + Self::from_json(&json) + } + + /// The service account's email address. + pub fn client_email(&self) -> &str { + &self.client_email + } + + /// The id of the key, if the file names one. It is the `kid` of the + /// JWT header. + pub fn private_key_id(&self) -> Option<&str> { + self.private_key_id.as_deref() + } + + /// The token endpoint. + pub fn token_uri(&self) -> &str { + &self.token_uri + } + + /// The signed JWT assertion for `scope`, issued at `issued_at` and + /// valid for [`ASSERTION_LIFETIME`]: `{"alg":"RS256","kid":…,"typ":"JWT"}` + /// and `{"iss":…,"scope":…,"aud":,"iat":…,"exp":…}`, signed + /// with RSASSA-PKCS1-v1_5 and SHA-256 (RS256). The signature is + /// deterministic; the private key operation is blinded. + pub fn assertion( + &self, + scope: &str, + issued_at: SystemTime, + ) -> Result, AuthError> { + let iat = unix_seconds(issued_at); + let header = serde_json::to_vec(&Header { + alg: "RS256", + kid: self.private_key_id.as_deref(), + typ: "JWT", + }) + .map_err(|_| AuthError::access_denied("the JWT header"))?; + let claims = serde_json::to_vec(&Claims { + iss: &self.client_email, + scope, + aud: &self.token_uri, + iat, + exp: iat + ASSERTION_LIFETIME.as_secs(), + }) + .map_err(|_| AuthError::access_denied("the JWT claims"))?; + let mut jwt = Zeroizing::new(format!( + "{}.{}", + Base64UrlUnpadded::encode_string(&header), + Base64UrlUnpadded::encode_string(&claims) + )); + let signature = self + .signing_key + .try_sign_with_rng(&mut rand_core::OsRng, jwt.as_bytes()) + .map_err(|_| AuthError::unavailable("the JWT could not be signed"))?; + jwt.push('.'); + jwt.push_str(&Base64UrlUnpadded::encode_string(&signature.to_bytes())); + Ok(jwt) + } +} + +/// A service account key's token: a signed JWT assertion exchanged at the +/// key's `token_uri` for an access token (RFC 7523), with the Cloud KMS +/// scope. Google issues the token for one hour. Wrap it in a +/// [`CachedToken`](super::CachedToken). +pub struct ServiceAccountLogin { + key: ServiceAccountKey, + transport: T, + scope: String, + clock: Clock, +} + +impl fmt::Debug for ServiceAccountLogin { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("ServiceAccountLogin") + .field("key", &self.key) + .field("scope", &self.scope) + .finish_non_exhaustive() + } +} + +impl ServiceAccountLogin { + /// Exchange assertions signed by `key` through `transport`, for + /// [`CLOUD_KMS_SCOPE`]. + pub fn new(key: ServiceAccountKey, transport: T) -> Self { + Self { + key, + transport, + scope: CLOUD_KMS_SCOPE.to_owned(), + clock: system_clock(), + } + } + + /// Ask for `scope` instead (space-separated scopes), for example + /// `https://www.googleapis.com/auth/cloud-platform`. + pub fn with_scope(mut self, scope: impl Into) -> Self { + self.scope = scope.into(); + self + } + + /// Read the issue time of each assertion from `clock`. + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = clock; + self + } + + /// The key. + pub fn key(&self) -> &ServiceAccountKey { + &self.key + } + + /// The transport. + pub fn transport(&self) -> &T { + &self.transport + } +} + +impl TokenProvider for ServiceAccountLogin { + async fn token(&self) -> Result { + let assertion = self.key.assertion(&self.scope, (self.clock)())?; + // The assertion is base64url and dots, so it needs no form escape. + let body = Zeroizing::new( + format!( + "grant_type={}&assertion={}", + url::form_urlencoded::byte_serialize(JWT_BEARER_GRANT_TYPE.as_bytes()) + .collect::(), + assertion.as_str() + ) + .into_bytes(), + ); + let mut request = VendorRequest::new("POST", self.key.token_uri()) + .header("content-type", "application/x-www-form-urlencoded"); + request.body = body; + let response = self.transport.send(request).await.map_err(|error| { + AuthError::unavailable(format!("the token endpoint got no response: {error}")) + })?; + read_token_response("the service account token exchange", &response) + } +} diff --git a/packages/stack-kms-gcp/src/client.rs b/packages/stack-kms-gcp/src/client.rs new file mode 100644 index 0000000000..20c7ea266e --- /dev/null +++ b/packages/stack-kms-gcp/src/client.rs @@ -0,0 +1,614 @@ +//! The Cloud KMS client and its [`KeyVendorBackend`] implementation. + +use std::time::Duration; + +use base64ct::{Base64, Encoding}; +use serde::de::DeserializeOwned; +use serde::{Deserialize, Serialize}; +use stack_kms::{BackendErrorKind, Error, KeyVendor}; +use stack_kms_vendor::{ + classify_status, KeyBytes, KeyVendorBackend, TransportError, VendorRequest, VendorResponse, + VendorTransport, +}; +use vitaminc::protected::{Controlled, Protected}; +use zeroize::Zeroizing; + +use crate::auth::{AuthError, TokenProvider}; +use crate::config::{parse_key_name, GcpKmsConfig}; +use crate::crc32c::crc32c; + +/// The vendor name in errors ([`KeyVendor::name`] of [`KeyVendor::Gcp`]). +const VENDOR: &str = "gcp-kms"; + +/// The first byte of a wrapped key this client stores: the format of the +/// rest (`key version(4, BE) ‖ Cloud KMS ciphertext`). +pub const WRAPPED_KEY_FORMAT: u8 = 1; + +/// The text in an `INVALID_ARGUMENT` error that means "this ciphertext does +/// not open" ([`BackendErrorKind::InvalidCiphertext`]), lower case. Cloud +/// KMS says `Decryption failed: the ciphertext is invalid.` for damaged +/// bytes and other associated data, and `Decryption failed: verify that +/// 'name' refers to the correct CryptoKey.` for another key. +pub const INVALID_CIPHERTEXT_MARKERS: [&str; 2] = ["decryption failed", "ciphertext is invalid"]; + +/// The text in an `INVALID_ARGUMENT` error that means "a CRC32C did not +/// match the data" ([`BackendErrorKind::Unavailable`]), lower case: the +/// request was damaged on the way, so the same request can succeed later. +pub const CHECKSUM_MARKERS: [&str; 2] = ["checksum", "crc32c"]; + +/// A failure of a Cloud KMS call. It is the `source` of the +/// [`Error::Backend`] the client returns; the kind is on the +/// `Error::Backend`. +#[derive(Debug, thiserror::Error)] +pub enum GcpKmsError { + /// Cloud KMS answered with an error. + #[error("Cloud KMS {operation} failed with HTTP {status}: {code}: {message}")] + Service { + /// The operation (`decrypt`). + operation: &'static str, + /// The HTTP status. + status: u16, + /// The `google.rpc.Code` name (`PERMISSION_DENIED`), or `unknown`. + code: String, + /// Google's error message. + message: String, + }, + /// The request got no response. + #[error("Cloud KMS {operation} got no response: {source}")] + Transport { + /// The operation. + operation: &'static str, + /// The transport failure. + source: TransportError, + }, + /// No access token could be got. + #[error(transparent)] + Auth(#[from] AuthError), + /// Cloud KMS answered with a success that this client cannot read or + /// does not trust (a CRC32C that does not match). + #[error("Cloud KMS {operation} returned a response this client cannot read: {reason}")] + InvalidResponse { + /// The operation. + operation: &'static str, + /// What is wrong with it. + reason: &'static str, + }, + /// The client refused the call before or after it reached Cloud KMS. + #[error("{0}")] + Refused(&'static str), +} + +fn backend(kind: BackendErrorKind, error: GcpKmsError) -> Error { + Error::Backend { + vendor: VENDOR, + kind, + source: Box::new(error), + } +} + +fn invalid(operation: &'static str, reason: &'static str) -> Error { + backend( + BackendErrorKind::Unavailable, + GcpKmsError::InvalidResponse { operation, reason }, + ) +} + +fn refused(kind: BackendErrorKind, reason: &'static str) -> Error { + backend(kind, GcpKmsError::Refused(reason)) +} + +/// The failure kind of a Cloud KMS error response: by the `google.rpc.Code` +/// name in `error.status` (`code`), then by the HTTP status. +/// +/// * `PERMISSION_DENIED`, `UNAUTHENTICATED`: `AccessDenied`. Note that Cloud +/// KMS answers `PERMISSION_DENIED`, not `NOT_FOUND`, for a key the caller +/// may not see. +/// * `NOT_FOUND`: `NotFound`. +/// * `RESOURCE_EXHAUSTED`: `Throttled` (a quota; the adapter backs off). +/// * `UNAVAILABLE`, `DEADLINE_EXCEEDED`, `INTERNAL`, `UNKNOWN`, `ABORTED`, +/// `CANCELLED`: `Unavailable`. `FAILED_PRECONDITION` (a disabled or +/// destroyed key version, a key with no primary version) is `Unavailable` +/// too, as AWS KMS's `DisabledException` is. +/// * `INVALID_ARGUMENT`: by the message, lower case. [`CHECKSUM_MARKERS`] +/// first (`Unavailable`), then [`INVALID_CIPHERTEXT_MARKERS`] +/// (`InvalidCiphertext`). Any other `INVALID_ARGUMENT` is a request Cloud +/// KMS refused (`AccessDenied`), as for AWS KMS and Vault. +/// * No or another code: the HTTP status (401 and 403 `AccessDenied`, 404 +/// `NotFound`, 429 `Throttled`, 5xx `Unavailable`); any other 4xx is +/// `AccessDenied`. +pub fn error_kind(status: u16, code: &str, message: &str) -> BackendErrorKind { + match code { + "PERMISSION_DENIED" | "UNAUTHENTICATED" => BackendErrorKind::AccessDenied, + "NOT_FOUND" => BackendErrorKind::NotFound, + "RESOURCE_EXHAUSTED" => BackendErrorKind::Throttled, + "UNAVAILABLE" + | "DEADLINE_EXCEEDED" + | "INTERNAL" + | "UNKNOWN" + | "ABORTED" + | "CANCELLED" + | "FAILED_PRECONDITION" => BackendErrorKind::Unavailable, + "INVALID_ARGUMENT" => { + let message = message.to_ascii_lowercase(); + if CHECKSUM_MARKERS.iter().any(|m| message.contains(m)) { + BackendErrorKind::Unavailable + } else if INVALID_CIPHERTEXT_MARKERS + .iter() + .any(|m| message.contains(m)) + { + BackendErrorKind::InvalidCiphertext + } else { + BackendErrorKind::AccessDenied + } + } + _ => match classify_status(status) { + // Another 4xx is a refused request, not a damaged ciphertext. + Some(BackendErrorKind::InvalidCiphertext) => BackendErrorKind::AccessDenied, + Some(kind) => kind, + None => BackendErrorKind::Unavailable, + }, + } +} + +#[derive(Deserialize)] +struct ErrorBody { + error: Option, +} + +#[derive(Deserialize)] +struct ErrorStatus { + message: Option, + status: Option, +} + +fn service_error(operation: &'static str, response: &VendorResponse) -> Error { + let status = serde_json::from_slice::(&response.body) + .ok() + .and_then(|body| body.error); + let (code, message) = match status { + Some(status) => ( + status.status.unwrap_or_else(|| "unknown".to_owned()), + status.message.unwrap_or_default(), + ), + None => ("unknown".to_owned(), String::new()), + }; + backend( + error_kind(response.status, &code, &message), + GcpKmsError::Service { + operation, + status: response.status, + code, + message, + }, + ) +} + +/// An `int64` of a JSON response: protobuf's JSON form is a string, but a +/// number is accepted too. +fn read_u64(value: &serde_json::Value) -> Option { + match value { + serde_json::Value::String(text) => text.parse().ok(), + serde_json::Value::Number(number) => number.as_u64(), + _ => None, + } +} + +fn serialize_secret( + value: &Zeroizing, + serializer: S, +) -> Result { + serializer.serialize_str(value) +} + +#[derive(Serialize)] +#[serde(rename_all = "camelCase")] +struct EncryptRequest { + #[serde(serialize_with = "serialize_secret")] + plaintext: Zeroizing, + additional_authenticated_data: String, + plaintext_crc32c: String, + additional_authenticated_data_crc32c: String, +} + +#[derive(Deserialize)] +#[serde(rename_all = "camelCase")] +struct EncryptResponse { + name: Option, + ciphertext: Option, + ciphertext_crc32c: Option, + #[serde(default)] + verified_plaintext_crc32c: bool, + #[serde(default)] + verified_additional_authenticated_data_crc32c: bool, +} + +#[derive(Serialize)] +#[serde(rename_all = "camelCase")] +struct DecryptRequest { + ciphertext: String, + additional_authenticated_data: String, + ciphertext_crc32c: String, + additional_authenticated_data_crc32c: String, +} + +#[derive(Deserialize)] +#[serde(rename_all = "camelCase")] +struct DecryptResponse { + plaintext: Option, + plaintext_crc32c: Option, +} + +/// The stored form of a Cloud KMS ciphertext: [`WRAPPED_KEY_FORMAT`], the +/// number of the key version that encrypted it (4 bytes, big-endian), and +/// the ciphertext. +pub fn compact_ciphertext(version: u32, ciphertext: &[u8]) -> Vec { + let mut out = Vec::with_capacity(5 + ciphertext.len()); + out.push(WRAPPED_KEY_FORMAT); + out.extend_from_slice(&version.to_be_bytes()); + out.extend_from_slice(ciphertext); + out +} + +/// The key version and the Cloud KMS ciphertext of a stored wrapped key. +/// `None` if the bytes are not one. +pub fn split_wrapped_key(wrapped: &[u8]) -> Option<(u32, &[u8])> { + let (&format, rest) = wrapped.split_first()?; + if format != WRAPPED_KEY_FORMAT { + return None; + } + let (version, ciphertext) = rest.split_at_checked(4)?; + if ciphertext.is_empty() { + return None; + } + Some((u32::from_be_bytes(version.try_into().ok()?), ciphertext)) +} + +/// The number of the crypto key version that wrapped `wrapped`, or `None` +/// if the bytes are not a wrapped key of this client. Use it to find which +/// versions stored keys still need before you disable or destroy one. +pub fn wrapped_key_version(wrapped: &[u8]) -> Option { + split_wrapped_key(wrapped).map(|(version, _)| version) +} + +/// A Google Cloud KMS client, and the Google Cloud KMS key vendor backend +/// ([`KeyVendor::Gcp`]). +/// +/// It speaks the Cloud KMS REST API v1 (`cryptoKeys.encrypt` and +/// `cryptoKeys.decrypt`) with an OAuth 2.0 access token, over a +/// [`VendorTransport`], so the same code runs on native Rust and in the +/// WASI guest. +/// +/// * `key_ref` (from the keyset record) is the crypto key name, +/// `projects/{p}/locations/{l}/keyRings/{r}/cryptoKeys/{k}`, of a +/// symmetric `ENCRYPT_DECRYPT` key. Each `encrypt` uses the key's primary +/// version, and the client stores that version's number in the wrapped +/// key ([`wrapped_key_version`]); `decrypt` finds the version from the +/// ciphertext, so rotation needs no change to the record. A +/// `…/cryptoKeyVersions/{n}` name pins `encrypt` to that version. +/// * **Associated data.** The whole associated data (it holds the keyset +/// id and, for a wrapped data key, the iv and the descriptor) is sent as +/// `additionalAuthenticatedData`. Cloud KMS refuses a `decrypt` under +/// other associated data. +/// * **Integrity.** Every request carries the CRC32C of each field it +/// sends, and the client checks the CRC32C of each field Cloud KMS +/// returns, and that Cloud KMS verified the ones it was sent. +/// * Cloud KMS has no data key generation and no batch operation: each +/// wrapped data key is a key from the local CSPRNG and one `encrypt`. +/// * Errors map to [`BackendErrorKind`] by [`error_kind`]. A 401 also tells +/// the token provider to drop its token ([`TokenProvider::invalidate`]). +pub struct GcpKms { + config: GcpKmsConfig, + transport: T, + auth: A, +} + +impl std::fmt::Debug for GcpKms { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("GcpKms") + .field("config", &self.config) + .finish_non_exhaustive() + } +} + +impl GcpKms { + /// A client for `config`, sending through `transport`, with the access + /// tokens of `auth`. + pub fn new(config: GcpKmsConfig, transport: T, auth: A) -> Self { + Self { + config, + transport, + auth, + } + } + + /// The configuration. + pub fn config(&self) -> &GcpKmsConfig { + &self.config + } + + /// The transport. + pub fn transport(&self) -> &T { + &self.transport + } + + /// The token provider. + pub fn auth(&self) -> &A { + &self.auth + } + + /// Send one request to the API path `path` (after `/v1/`), with an + /// access token, and return the response of a success. + async fn request( + &self, + operation: &'static str, + method: &str, + path: &str, + body: Option>>, + ) -> Result { + let token = self.auth.token().await.map_err(|error| { + let kind = error.kind(); + backend(kind, GcpKmsError::Auth(error)) + })?; + let mut request = + VendorRequest::new(method, format!("{}/v1/{path}", self.config.endpoint())) + .header("authorization", format!("Bearer {}", token.token())); + if let Some(body) = body { + request = request.header("content-type", "application/json"); + request.body = body; + } + let response = self.transport.send(request).await.map_err(|source| { + backend( + BackendErrorKind::Unavailable, + GcpKmsError::Transport { operation, source }, + ) + })?; + if classify_status(response.status).is_some() { + if response.status == 401 { + self.auth.invalidate(); + } + return Err(service_error(operation, &response)); + } + Ok(response) + } + + /// Send one request to the API path `path` (after `/v1/`, for example + /// `projects/p/locations/global/keyRings/r/cryptoKeys/k`) and read the + /// JSON of a success. `method` is the HTTP method. Any Cloud KMS method + /// can be called this way (for example `cryptoKeys.get` in a test); + /// [`KeyVendorBackend`] uses `encrypt` and `decrypt`. The body is wiped + /// after it is sent. + pub async fn send_json( + &self, + method: &str, + path: &str, + body: Option>>, + ) -> Result { + let response = self.request("request", method, path, body).await?; + serde_json::from_slice(&response.body).map_err(|_| invalid("request", "invalid JSON")) + } +} + +/// The crypto key name and the pinned version of `key_ref`. +fn key_name(key_ref: &str) -> Result<(&str, Option), Error> { + parse_key_name(key_ref).ok_or_else(|| { + refused( + BackendErrorKind::NotFound, + "the key reference is not a Cloud KMS crypto key name", + ) + }) +} + +fn to_json(operation: &'static str, value: &S) -> Result>, Error> { + serde_json::to_vec(value) + .map(Zeroizing::new) + .map_err(|_| invalid(operation, "unserializable request")) +} + +impl KeyVendorBackend for GcpKms { + fn vendor(&self) -> KeyVendor { + KeyVendor::Gcp + } + + /// The same crypto key, whatever `…/cryptoKeyVersions/{n}` either + /// reference names. A reference that is not a crypto key name is + /// compared as text. + fn same_key(&self, a: &str, b: &str) -> bool { + match (parse_key_name(a), parse_key_name(b)) { + (Some((a, _)), Some((b, _))) => a == b, + _ => a == b, + } + } + + async fn wrap(&self, key_ref: &str, key: &KeyBytes, aad: &[u8]) -> Result, Error> { + let (crypto_key, _) = key_name(key_ref)?; + let body = to_json( + "encrypt", + &EncryptRequest { + plaintext: Zeroizing::new(Base64::encode_string(key.risky_ref())), + additional_authenticated_data: Base64::encode_string(aad), + plaintext_crc32c: crc32c(key.risky_ref()).to_string(), + additional_authenticated_data_crc32c: crc32c(aad).to_string(), + }, + )?; + let response = self + .request("encrypt", "POST", &format!("{key_ref}:encrypt"), Some(body)) + .await?; + let parsed: EncryptResponse = serde_json::from_slice(&response.body) + .map_err(|_| invalid("encrypt", "invalid JSON"))?; + if !parsed.verified_plaintext_crc32c + || !parsed.verified_additional_authenticated_data_crc32c + { + return Err(invalid( + "encrypt", + "Cloud KMS did not verify the CRC32C of the request", + )); + } + let ciphertext = Base64::decode_vec( + parsed + .ciphertext + .as_deref() + .ok_or_else(|| invalid("encrypt", "no ciphertext"))?, + ) + .map_err(|_| invalid("encrypt", "the ciphertext is not base64"))?; + if ciphertext.is_empty() { + return Err(invalid("encrypt", "an empty ciphertext")); + } + let expected = parsed + .ciphertext_crc32c + .as_ref() + .and_then(read_u64) + .ok_or_else(|| invalid("encrypt", "no ciphertext CRC32C"))?; + if u64::from(crc32c(&ciphertext)) != expected { + return Err(invalid("encrypt", "the ciphertext CRC32C does not match")); + } + let version = match parsed.name.as_deref().and_then(parse_key_name) { + Some((name, Some(version))) if name == crypto_key => version, + _ => { + return Err(invalid( + "encrypt", + "the response does not name a version of the key", + )) + } + }; + Ok(compact_ciphertext(version, &ciphertext)) + } + + async fn unwrap(&self, key_ref: &str, wrapped: &[u8], aad: &[u8]) -> Result { + let (crypto_key, _) = key_name(key_ref)?; + let (_, ciphertext) = split_wrapped_key(wrapped).ok_or_else(|| { + refused( + BackendErrorKind::InvalidCiphertext, + "the wrapped key is not a Cloud KMS wrapped key", + ) + })?; + let body = to_json( + "decrypt", + &DecryptRequest { + ciphertext: Base64::encode_string(ciphertext), + additional_authenticated_data: Base64::encode_string(aad), + ciphertext_crc32c: crc32c(ciphertext).to_string(), + additional_authenticated_data_crc32c: crc32c(aad).to_string(), + }, + )?; + let response = self + .request( + "decrypt", + "POST", + &format!("{crypto_key}:decrypt"), + Some(body), + ) + .await?; + let parsed: DecryptResponse = serde_json::from_slice(&response.body) + .map_err(|_| invalid("decrypt", "invalid JSON"))?; + let encoded = Zeroizing::new( + parsed + .plaintext + .ok_or_else(|| invalid("decrypt", "no plaintext"))?, + ); + let plaintext = Zeroizing::new( + Base64::decode_vec(&encoded) + .map_err(|_| invalid("decrypt", "the plaintext is not base64"))?, + ); + let expected = parsed + .plaintext_crc32c + .as_ref() + .and_then(read_u64) + .ok_or_else(|| invalid("decrypt", "no plaintext CRC32C"))?; + if u64::from(crc32c(&plaintext)) != expected { + return Err(invalid("decrypt", "the plaintext CRC32C does not match")); + } + let key: [u8; 32] = plaintext.as_slice().try_into().map_err(|_| { + refused( + BackendErrorKind::InvalidCiphertext, + "the wrapped key does not hold a 256-bit key", + ) + })?; + Ok(Protected::new(key)) + } + + async fn sleep(&self, duration: Duration) { + self.transport.sleep(duration).await; + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn errors_map_by_code_then_status() { + use BackendErrorKind::*; + for (status, code, message, kind) in [ + (403, "PERMISSION_DENIED", "Permission denied", AccessDenied), + (401, "UNAUTHENTICATED", "", AccessDenied), + (404, "NOT_FOUND", "CryptoKey not found.", NotFound), + (429, "RESOURCE_EXHAUSTED", "Quota exceeded", Throttled), + (503, "UNAVAILABLE", "", Unavailable), + (504, "DEADLINE_EXCEEDED", "", Unavailable), + (500, "INTERNAL", "", Unavailable), + (500, "UNKNOWN", "", Unavailable), + (409, "ABORTED", "", Unavailable), + (499, "CANCELLED", "", Unavailable), + (400, "FAILED_PRECONDITION", "is not enabled", Unavailable), + ( + 400, + "INVALID_ARGUMENT", + "Decryption failed: the ciphertext is invalid.", + InvalidCiphertext, + ), + ( + 400, + "INVALID_ARGUMENT", + "Decryption failed: verify that 'name' refers to the correct CryptoKey.", + InvalidCiphertext, + ), + ( + 400, + "INVALID_ARGUMENT", + "The checksum in field ciphertext_crc32c did not match the data in field ciphertext.", + Unavailable, + ), + ( + 400, + "INVALID_ARGUMENT", + "Plaintext must be at most 65536 bytes.", + AccessDenied, + ), + (403, "unknown", "", AccessDenied), + (404, "unknown", "", NotFound), + (429, "unknown", "", Throttled), + (502, "unknown", "", Unavailable), + (412, "unknown", "", AccessDenied), + (302, "unknown", "", AccessDenied), + ] { + assert_eq!( + error_kind(status, code, message), + kind, + "{status} {code} {message}" + ); + } + } + + #[test] + fn wrapped_keys_split_losslessly() { + let wrapped = compact_ciphertext(7, b"ciphertext"); + assert_eq!(wrapped[0], WRAPPED_KEY_FORMAT); + assert_eq!(&wrapped[1..5], &7u32.to_be_bytes()); + assert_eq!(split_wrapped_key(&wrapped), Some((7, &b"ciphertext"[..]))); + assert_eq!(wrapped_key_version(&wrapped), Some(7)); + for bad in [&[][..], &[1, 0, 0, 0, 7], &[2, 0, 0, 0, 7, 1], &[1, 0, 0]] { + assert_eq!(split_wrapped_key(bad), None, "{bad:?}"); + } + } + + #[test] + fn integers_read_from_strings_and_numbers() { + assert_eq!( + read_u64(&serde_json::json!("4294967295")), + Some(4_294_967_295) + ); + assert_eq!(read_u64(&serde_json::json!(12)), Some(12)); + assert_eq!(read_u64(&serde_json::json!("x")), None); + assert_eq!(read_u64(&serde_json::json!(null)), None); + } +} diff --git a/packages/stack-kms-gcp/src/config.rs b/packages/stack-kms-gcp/src/config.rs new file mode 100644 index 0000000000..c0aa8a1880 --- /dev/null +++ b/packages/stack-kms-gcp/src/config.rs @@ -0,0 +1,184 @@ +//! The endpoint of a Cloud KMS client, and the crypto key names it accepts. + +/// How an environment variable is read: the process environment unless a +/// test gives another. +pub type EnvLookup = std::sync::Arc Option + Send + Sync>; + +/// The process environment. An empty variable counts as unset. +pub fn process_env() -> EnvLookup { + std::sync::Arc::new(|name| std::env::var(name).ok().filter(|value| !value.is_empty())) +} + +/// The Cloud KMS API endpoint. +pub const DEFAULT_ENDPOINT: &str = "https://cloudkms.googleapis.com"; + +/// The configuration is not usable. +#[derive(Debug, thiserror::Error, PartialEq, Eq)] +pub enum ConfigError { + /// The endpoint is not an absolute HTTP or HTTPS URL with no path, + /// query or credentials. + #[error("{0:?} is not an HTTP or HTTPS Cloud KMS endpoint")] + InvalidEndpoint(String), +} + +/// The endpoint of a [`GcpKms`](crate::GcpKms) client. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct GcpKmsConfig { + endpoint: String, +} + +impl Default for GcpKmsConfig { + fn default() -> Self { + Self::new() + } +} + +impl GcpKmsConfig { + /// The public Cloud KMS endpoint, [`DEFAULT_ENDPOINT`]. + pub fn new() -> Self { + Self { + endpoint: DEFAULT_ENDPOINT.to_owned(), + } + } + + /// Send to `endpoint` instead: a regional or Private Service Connect + /// endpoint (`https://cloudkms.europe-west1.rep.googleapis.com`), or a + /// test server. A trailing `/` is removed. + pub fn with_endpoint(mut self, endpoint: impl Into) -> Result { + let endpoint = endpoint.into(); + let parsed = url::Url::parse(&endpoint) + .map_err(|_| ConfigError::InvalidEndpoint(endpoint.clone()))?; + if !matches!(parsed.scheme(), "http" | "https") + || parsed.host().is_none() + || parsed.path() != "/" + || parsed.query().is_some() + || parsed.fragment().is_some() + || !parsed.username().is_empty() + || parsed.password().is_some() + { + return Err(ConfigError::InvalidEndpoint(endpoint)); + } + self.endpoint = parsed.as_str().trim_end_matches('/').to_owned(); + Ok(self) + } + + /// The endpoint, with no trailing `/`. + pub fn endpoint(&self) -> &str { + &self.endpoint + } + + /// The URL of the method `method` (`encrypt`) on the resource `name`. + pub fn url(&self, name: &str, method: &str) -> String { + format!("{}/v1/{name}:{method}", self.endpoint) + } +} + +/// One resource id of a Cloud KMS name: `[A-Za-z0-9._-]`, not `.` or `..`. +/// Only a project id may hold a `:`, because it can be domain-scoped +/// (`example.com:project`); anywhere else a `:` would name a method. +fn is_id(text: &str, project: bool) -> bool { + !text.is_empty() + && text.len() <= 128 + && text != "." + && text != ".." + && text.bytes().all(|b| { + b.is_ascii_alphanumeric() || matches!(b, b'.' | b'_' | b'-') || (project && b == b':') + }) +} + +/// A crypto key name, `projects/{p}/locations/{l}/keyRings/{r}/cryptoKeys/{k}`, +/// optionally followed by `/cryptoKeyVersions/{n}`. +/// +/// Returns the crypto key name and the version, or `None` if `key_ref` is not +/// such a name. The client refuses anything else before any call, so a key +/// reference cannot change the request path. +pub fn parse_key_name(key_ref: &str) -> Option<(&str, Option)> { + let parts: Vec<&str> = key_ref.split('/').collect(); + let fixed = ["projects", "locations", "keyRings", "cryptoKeys"]; + if parts.len() != 8 && parts.len() != 10 { + return None; + } + for (index, collection) in fixed.iter().enumerate() { + if parts[2 * index] != *collection || !is_id(parts[2 * index + 1], index == 0) { + return None; + } + } + if parts.len() == 8 { + return Some((key_ref, None)); + } + let version = parts[9]; + if parts[8] != "cryptoKeyVersions" + || version.is_empty() + || version.len() > 10 + || !version.bytes().all(|b| b.is_ascii_digit()) + { + return None; + } + let version: u32 = version.parse().ok()?; + let key_len = key_ref.len() - "/cryptoKeyVersions/".len() - parts[9].len(); + Some((&key_ref[..key_len], Some(version))) +} + +#[cfg(test)] +mod tests { + use super::*; + + const KEY: &str = "projects/p-1/locations/global/keyRings/ring/cryptoKeys/key"; + + #[test] + fn the_endpoint_is_normalised_and_checked() { + let config = GcpKmsConfig::new(); + assert_eq!(config.endpoint(), DEFAULT_ENDPOINT); + assert_eq!( + config.url(KEY, "encrypt"), + format!("https://cloudkms.googleapis.com/v1/{KEY}:encrypt") + ); + let config = GcpKmsConfig::new() + .with_endpoint("http://127.0.0.1:9011/") + .unwrap(); + assert_eq!(config.endpoint(), "http://127.0.0.1:9011"); + for bad in [ + "cloudkms.googleapis.com", + "ftp://kms/", + "https://kms/v1", + "https://kms/?x=1", + "https://kms/#x", + "https://user:pass@kms/", + "not a url", + ] { + assert_eq!( + GcpKmsConfig::new().with_endpoint(bad), + Err(ConfigError::InvalidEndpoint(bad.to_owned())), + "{bad}" + ); + } + } + + #[test] + fn key_names_parse() { + assert_eq!(parse_key_name(KEY), Some((KEY, None))); + let version = format!("{KEY}/cryptoKeyVersions/12"); + assert_eq!(parse_key_name(&version), Some((KEY, Some(12)))); + let scoped = "projects/example.com:p/locations/us-east1/keyRings/r_1/cryptoKeys/k-2"; + assert_eq!(parse_key_name(scoped), Some((scoped, None))); + for bad in [ + "", + "key", + "projects/p/locations/l/keyRings/r", + "projects/p/locations/l/keyRings/r/cryptoKeys/", + "projects/p/locations/l/keyRings/r/cryptoKeys/k/", + "projects/p/locations/l/keyRings/r/cryptoKeys/k/cryptoKeyVersions", + "projects/p/locations/l/keyRings/r/cryptoKeys/k/cryptoKeyVersions/x", + "projects/p/locations/l/keyRings/r/cryptoKeys/k/cryptoKeyVersions/99999999999", + "projects/p/locations/l/keyRings/r/cryptoKeys/k/versions/1", + "projects/p/locations/l/keyRings/../cryptoKeys/k", + "projects/p/locations/l/keyRings/r/cryptoKeys/k?x", + "projects/p/locations/l/keyRings/r/cryptoKeys/k:decrypt", + "projects/p/locations/l/keyRings/r/cryptoKeys/a b", + "folders/p/locations/l/keyRings/r/cryptoKeys/k", + "/projects/p/locations/l/keyRings/r/cryptoKeys/k", + ] { + assert_eq!(parse_key_name(bad), None, "{bad:?}"); + } + } +} diff --git a/packages/stack-kms-gcp/src/crc32c.rs b/packages/stack-kms-gcp/src/crc32c.rs new file mode 100644 index 0000000000..ff25e92446 --- /dev/null +++ b/packages/stack-kms-gcp/src/crc32c.rs @@ -0,0 +1,58 @@ +//! CRC32C (Castagnoli), for the Cloud KMS integrity fields. +//! +//! Cloud KMS checks a CRC32C of each request field it is given, and returns +//! one of each response field. This is the table-driven, reflected CRC with +//! polynomial `0x82F63B78`, initial value and final XOR `0xFFFFFFFF` +//! (RFC 3720, appendix B.4). + +/// The reflected Castagnoli polynomial. +const POLYNOMIAL: u32 = 0x82F6_3B78; + +const fn table() -> [u32; 256] { + let mut table = [0u32; 256]; + let mut n = 0; + while n < 256 { + let mut crc = n as u32; + let mut bit = 0; + while bit < 8 { + crc = if crc & 1 == 1 { + (crc >> 1) ^ POLYNOMIAL + } else { + crc >> 1 + }; + bit += 1; + } + table[n] = crc; + n += 1; + } + table +} + +const TABLE: [u32; 256] = table(); + +/// The CRC32C of `bytes`. +pub fn crc32c(bytes: &[u8]) -> u32 { + let mut crc = u32::MAX; + for &byte in bytes { + crc = TABLE[((crc ^ u32::from(byte)) & 0xFF) as usize] ^ (crc >> 8); + } + !crc +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn known_answers() { + // The check value of CRC-32C, and RFC 3720 B.4's test vectors. + assert_eq!(crc32c(b"123456789"), 0xE306_9283); + assert_eq!(crc32c(b""), 0); + assert_eq!(crc32c(&[0u8; 32]), 0x8A91_36AA); + assert_eq!(crc32c(&[0xFFu8; 32]), 0x62A8_AB43); + let ascending: Vec = (0u8..32).collect(); + assert_eq!(crc32c(&ascending), 0x46DD_794E); + let descending: Vec = (0u8..32).rev().collect(); + assert_eq!(crc32c(&descending), 0x113F_DB5C); + } +} diff --git a/packages/stack-kms-gcp/src/lib.rs b/packages/stack-kms-gcp/src/lib.rs new file mode 100644 index 0000000000..8d892d3fdc --- /dev/null +++ b/packages/stack-kms-gcp/src/lib.rs @@ -0,0 +1,155 @@ +#![doc(html_favicon_url = "https://cipherstash.com/favicon.ico")] +//! The Google Cloud KMS key vendor backend for Stack Encrypt (ADR-0009, +//! ADR-0011). +//! +//! [`GcpKms`] is a small, hand-written client for the Cloud KMS REST API v1: +//! `cryptoKeys.encrypt` and `cryptoKeys.decrypt`, with the associated data +//! as `additionalAuthenticatedData` and CRC32C integrity checks on every +//! field. It sends through a +//! [`VendorTransport`](stack_kms_vendor::VendorTransport) and implements +//! [`KeyVendorBackend`](stack_kms_vendor::KeyVendorBackend), so +//! [`VendorKeySource`](stack_kms_vendor::VendorKeySource) builds both key +//! modes on it. No Google SDK crate is used, so this crate also builds for +//! wasm32-wasip1, the Go module's WASI guest. +//! +//! Cloud KMS has no data key generation and no batch operation: under +//! wrapped data keys each value's data key comes from the local CSPRNG and +//! costs one `encrypt` call. +//! +//! # Native Rust +//! +//! ```no_run +//! # async fn example() -> Result<(), Box> { +//! use stack_kms_gcp::GcpKms; +//! use stack_kms_vendor::{KeysetRecord, VendorKeySource}; +//! use stack_kms::KeyMode; +//! +//! // GOOGLE_APPLICATION_CREDENTIALS (a service account key file), or the +//! // metadata server on Google Cloud. +//! let kms = GcpKms::from_env()?; +//! let record = KeysetRecord::create( +//! &kms, +//! "projects/my-project/locations/global/keyRings/cipherstash/cryptoKeys/users", +//! KeyMode::Wrapped, +//! Some("users".to_owned()), +//! ) +//! .await?; +//! let source = VendorKeySource::new(kms, [record])?; +//! # let _ = source; +//! # Ok(()) +//! # } +//! ``` +//! +//! # Capabilities and limits +//! +//! The [capability matrix](stack_kms_vendor#capability-matrix) of +//! `stack-kms-vendor` compares this backend with ZeroKMS and the other +//! key vendor backends: key lifetime, vendor calls, what the vendor sees +//! and can refuse, credentials, rotation and known limits. +//! +//! # IAM +//! +//! The service account needs `roles/cloudkms.cryptoKeyEncrypterDecrypter` on +//! the crypto key (the permissions `cloudkms.cryptoKeyVersions.useToEncrypt` +//! and `useToDecrypt`). The key must be a symmetric `ENCRYPT_DECRYPT` key. +//! Rotate it in Cloud KMS: stored keys name the version that wrapped them +//! ([`wrapped_key_version`]), and decrypt still works under an old version +//! until you disable or destroy it. +//! +//! # The WASI guest +//! +//! The guest builds [`GcpKms::new`] with its host transport and a +//! [`TokenProvider`] over a host import. The metadata server and key files +//! are native only: in the guest, the host's own credentials supply the +//! token. + +#![deny(unsafe_code)] +#![warn(clippy::unwrap_used)] +#![warn(clippy::expect_used)] +#![warn(clippy::panic)] +#![warn(clippy::mem_forget)] +#![warn(clippy::print_stdout)] +#![warn(clippy::print_stderr)] +#![warn(clippy::dbg_macro)] +#![warn(unreachable_pub)] +#![warn(unused_results)] +#![cfg_attr(test, allow(clippy::unwrap_used))] +#![cfg_attr(test, allow(clippy::expect_used))] +#![cfg_attr(test, allow(clippy::panic))] +#![cfg_attr(test, allow(unused_results))] + +pub mod auth; +mod client; +mod config; +pub mod crc32c; +mod time; + +#[cfg(all(feature = "service-account", not(target_arch = "wasm32")))] +pub use auth::DefaultCredentials; +#[cfg(not(target_arch = "wasm32"))] +pub use auth::MetadataServerLogin; +pub use auth::{ + AccessToken, AuthError, CachedToken, TokenProvider, CLOUD_KMS_SCOPE, + DEFAULT_REFRESH_BEFORE_EXPIRY, +}; +#[cfg(feature = "service-account")] +pub use auth::{ServiceAccountKey, ServiceAccountLogin, DEFAULT_TOKEN_URI}; +pub use client::{ + compact_ciphertext, error_kind, split_wrapped_key, wrapped_key_version, GcpKms, GcpKmsError, + CHECKSUM_MARKERS, INVALID_CIPHERTEXT_MARKERS, WRAPPED_KEY_FORMAT, +}; +pub use config::{ + parse_key_name, process_env, ConfigError, EnvLookup, GcpKmsConfig, DEFAULT_ENDPOINT, +}; +pub use time::{system_clock, Clock}; + +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +mod native { + use std::time::Duration; + + use stack_kms_vendor::{HttpVendorTransport, HttpVendorTransportOptions}; + + use crate::auth::DefaultCredentials; + use crate::{GcpKms, GcpKmsConfig}; + + /// [`GcpKms::from_env`] failed. + #[derive(Debug, thiserror::Error)] + pub enum FromEnvError { + /// The credentials. + #[error(transparent)] + Auth(#[from] crate::AuthError), + /// The HTTP client. + #[error(transparent)] + Transport(#[from] stack_kms_vendor::TransportError), + } + + /// The client [`GcpKms::from_env`] builds. + pub type DefaultGcpKms = GcpKms>; + + impl DefaultGcpKms { + /// A client of the public Cloud KMS endpoint, with the credentials + /// of [`DefaultCredentials::from_env`]. Native only. + /// + /// The credentials transport opens connections with a 1 second + /// limit, so that off Google Cloud the metadata server probe fails + /// fast. + pub fn from_env() -> Result { + Self::from_env_with(GcpKmsConfig::new()) + } + + /// As [`from_env`](Self::from_env), with `config` (for example + /// another endpoint). + pub fn from_env_with(config: GcpKmsConfig) -> Result { + let credentials_transport = + HttpVendorTransport::with_options(HttpVendorTransportOptions { + timeout: Duration::from_secs(10), + connect_timeout: Some(Duration::from_secs(1)), + })?; + let auth = DefaultCredentials::from_env(credentials_transport)?; + Ok(GcpKms::new(config, HttpVendorTransport::new()?, auth)) + } + } +} + +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +pub use native::{DefaultGcpKms, FromEnvError}; diff --git a/packages/stack-kms-gcp/src/time.rs b/packages/stack-kms-gcp/src/time.rs new file mode 100644 index 0000000000..d7085fea0b --- /dev/null +++ b/packages/stack-kms-gcp/src/time.rs @@ -0,0 +1,20 @@ +//! The time source of a client. + +use web_time::SystemTime; + +/// The time source of a token cache and of the service account JWT: +/// `SystemTime::now` unless a test sets another. +pub type Clock = std::sync::Arc SystemTime + Send + Sync>; + +/// The system clock. +pub fn system_clock() -> Clock { + std::sync::Arc::new(SystemTime::now) +} + +/// Seconds since the Unix epoch. A time before the epoch is 0. +#[cfg(feature = "service-account")] +pub(crate) fn unix_seconds(time: SystemTime) -> u64 { + time.duration_since(web_time::UNIX_EPOCH) + .map(|elapsed| elapsed.as_secs()) + .unwrap_or(0) +} diff --git a/packages/stack-kms-gcp/tasks.toml b/packages/stack-kms-gcp/tasks.toml new file mode 100644 index 0000000000..ec1fa21dd3 --- /dev/null +++ b/packages/stack-kms-gcp/tasks.toml @@ -0,0 +1,31 @@ +# Rustdoc with warnings as errors: a broken intra-doc link or a rustdoc +# warning fails the build. The root `doc` task fans out over every +# `doc:`; the root `test:doc` runs the doc examples. +["doc:stack-kms-gcp"] +description = "Build docs for stack-kms-gcp with all features (warnings are errors)" +env = { RUSTDOCFLAGS = "-D warnings" } +run = "cargo doc -p stack-kms-gcp --no-deps --all-features" + +# The Cloud KMS backend against real Google Cloud KMS (tests/live.rs). Google +# publishes no Cloud KMS emulator, so this needs a real key and credentials. +# The test skips itself when STACK_KMS_GCP_LIVE_KEY is unset, which is right +# for `cargo nextest run --workspace` and wrong here: a skip would read as a +# pass. So this task refuses to run without the key and the credentials. +# integration-gcp-kms.yml (opt-in, credentialed) calls this task. +# +# STACK_KMS_GCP_LIVE_KEY=projects/

/locations//keyRings//cryptoKeys/ \ +# GOOGLE_APPLICATION_CREDENTIALS= \ +# mise run test:live:stack-kms-gcp +["test:live:stack-kms-gcp"] +description = "Run the stack-kms-gcp tests against real Cloud KMS (needs STACK_KMS_GCP_LIVE_KEY and credentials)" +run = """ +if [ -z "${STACK_KMS_GCP_LIVE_KEY:-}" ]; then + echo "error: set STACK_KMS_GCP_LIVE_KEY to a crypto key name (projects/.../cryptoKeys/...)" >&2 + exit 1 +fi +if [ -z "${GOOGLE_APPLICATION_CREDENTIALS:-}" ] && [ -z "${GOOGLE_OAUTH_ACCESS_TOKEN:-}" ]; then + echo "error: set GOOGLE_APPLICATION_CREDENTIALS (a service account key file) or GOOGLE_OAUTH_ACCESS_TOKEN" >&2 + exit 1 +fi +mise x --env test -- cargo nextest run --locked -p stack-kms-gcp --all-features --test live --no-tests=fail +""" diff --git a/packages/stack-kms-gcp/tests/common/mod.rs b/packages/stack-kms-gcp/tests/common/mod.rs new file mode 100644 index 0000000000..e4c507c7a2 --- /dev/null +++ b/packages/stack-kms-gcp/tests/common/mod.rs @@ -0,0 +1,549 @@ +//! Shared test support: fixtures, a replaying transport, the test-only +//! service account, and an in-process emulation of the Cloud KMS API, the +//! token endpoint and the metadata server. + +#![allow(dead_code, clippy::unwrap_used, clippy::expect_used, clippy::panic)] + +use std::collections::{HashMap, VecDeque}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use aes_gcm::aead::{Aead, KeyInit, Payload}; +use aes_gcm::{Aes256Gcm, Nonce}; +use base64ct::{Base64, Base64UrlUnpadded, Encoding}; +use rsa::pkcs1v15::{Signature, VerifyingKey}; +use rsa::pkcs8::DecodePrivateKey; +use rsa::sha2::Sha256; +use rsa::signature::Verifier; +use serde::{Deserialize, Serialize}; +use serde_json::{json, Value}; +use stack_kms_gcp::crc32c::crc32c; +use stack_kms_gcp::{parse_key_name, AccessToken, GcpKms, GcpKmsConfig, ServiceAccountKey}; +use stack_kms_vendor::{TransportError, VendorRequest, VendorResponse, VendorTransport}; +use web_time::{SystemTime, UNIX_EPOCH}; + +/// The crypto key of every fixture. +pub const KEY: &str = + "projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key"; +/// The static access token of the fixtures. +pub const TOKEN: &str = "fixture-access-token"; +/// The email of the test-only service account. +pub const EMAIL: &str = "stack-kms-gcp-fixture@test-only-project.iam.gserviceaccount.com"; +/// The key id of the test-only service account key. +pub const KEY_ID: &str = "test-only-fixture-key-id"; +/// The time every fixture JWT is issued at: 2026-11-05T12:00:00Z. +pub const ISSUED_AT: u64 = 1_793_880_000; +/// The Cloud KMS endpoint. +pub const ENDPOINT: &str = "https://cloudkms.googleapis.com"; +/// The token endpoint. +pub const TOKEN_URI: &str = "https://oauth2.googleapis.com/token"; +/// The metadata server's token URL. +pub const METADATA_URL: &str = + "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token"; + +/// The time of the fixture JWT. +pub fn issued_at() -> SystemTime { + UNIX_EPOCH + Duration::from_secs(ISSUED_AT) +} + +/// The PEM of the TEST-ONLY RSA key, generated for these tests with +/// `openssl genpkey -algorithm RSA -pkeyopt rsa_keygen_bits:2048`. It is no +/// one's service account key. It is kept as base64 PKCS #8 DER, not as PEM +/// or in a key file, so it does not look like a leaked credential. +pub fn test_only_key_pem() -> String { + let path = format!( + "{}/tests/fixtures/test_only_rsa_key.pkcs8.b64", + env!("CARGO_MANIFEST_DIR") + ); + let text = std::fs::read_to_string(path).unwrap(); + let body: String = text.split_whitespace().collect(); + let lines: Vec = body + .as_bytes() + .chunks(64) + .map(|chunk| String::from_utf8(chunk.to_vec()).unwrap()) + .collect(); + // The PEM label is built in pieces so that no file in the tree holds + // a PEM private key block. + let label = ["PRIVATE", "KEY"].join(" "); + format!( + "-----BEGIN {label}-----\n{}\n-----END {label}-----\n", + lines.join("\n") + ) +} + +/// A service account key file for the test-only key, built at run time. +pub fn test_only_service_account_json() -> Vec { + serde_json::to_vec(&json!({ + "type": "service_account", + "project_id": "test-only-project", + "private_key_id": KEY_ID, + "private_key": test_only_key_pem(), + "client_email": EMAIL, + "client_id": "000000000000000000000", + "auth_uri": "https://accounts.google.com/o/oauth2/auth", + "token_uri": TOKEN_URI, + })) + .unwrap() +} + +pub fn test_only_service_account() -> ServiceAccountKey { + ServiceAccountKey::from_json(&test_only_service_account_json()).unwrap() +} + +/// Verify a JWT's RS256 signature with the test-only key, and return its +/// header and claims. +pub fn verify_jwt(jwt: &str) -> Option<(Value, Value)> { + let (signing_input, signature) = jwt.rsplit_once('.')?; + let (header, claims) = signing_input.split_once('.')?; + let private = rsa::RsaPrivateKey::from_pkcs8_pem(&test_only_key_pem()).unwrap(); + let verifying = VerifyingKey::::new(private.to_public_key()); + let signature = + Signature::try_from(Base64UrlUnpadded::decode_vec(signature).ok()?.as_slice()).ok()?; + verifying + .verify(signing_input.as_bytes(), &signature) + .ok()?; + let header = serde_json::from_slice(&Base64UrlUnpadded::decode_vec(header).ok()?).ok()?; + let claims = serde_json::from_slice(&Base64UrlUnpadded::decode_vec(claims).ok()?).ok()?; + Some((header, claims)) +} + +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct FixtureRequest { + pub method: String, + pub url: String, + pub headers: Vec<(String, String)>, + pub body: Value, +} + +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct FixtureResponse { + pub status: u16, + #[serde(default)] + pub headers: Vec<(String, String)>, + pub body: Value, +} + +impl FixtureResponse { + pub fn to_response(&self) -> VendorResponse { + let body = match &self.body { + Value::String(text) => text.clone().into_bytes(), + Value::Null => Vec::new(), + other => serde_json::to_vec(other).unwrap(), + }; + let mut response = VendorResponse::new(self.status, body); + for (name, value) in &self.headers { + response = response.header(name, value); + } + response + } +} + +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct Fixture { + pub request: FixtureRequest, + pub response: FixtureResponse, +} + +pub fn load Deserialize<'de>>(name: &str) -> T { + let path = format!("{}/tests/fixtures/{name}", env!("CARGO_MANIFEST_DIR")); + let text = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}")); + serde_json::from_str(&text).unwrap_or_else(|e| panic!("{path}: {e}")) +} + +fn body_json(body: &[u8]) -> Value { + if body.is_empty() { + Value::Null + } else { + serde_json::from_slice(body) + .unwrap_or_else(|_| Value::String(String::from_utf8_lossy(body).into())) + } +} + +/// A transport that checks each request against a recorded one and +/// answers with the recorded response. +#[derive(Default)] +pub struct Replay { + exchanges: Mutex, VendorResponse)>>, +} + +impl Replay { + pub fn new() -> Self { + Self::default() + } + + /// Expect each exchange of a fixture file, in order. + pub fn expect_all(self, fixtures: Vec) -> Self { + for fixture in fixtures { + let response = fixture.response.to_response(); + self.exchanges + .lock() + .unwrap() + .push_back((Some(fixture.request), response)); + } + self + } + + /// Answer the next request, whatever it is, with `response`. + pub fn answer(self, response: VendorResponse) -> Self { + self.exchanges.lock().unwrap().push_back((None, response)); + self + } + + pub fn is_done(&self) -> bool { + self.exchanges.lock().unwrap().is_empty() + } +} + +impl VendorTransport for Replay { + async fn send(&self, request: VendorRequest) -> Result { + let (expected, response) = self + .exchanges + .lock() + .unwrap() + .pop_front() + .expect("a request the fixtures do not expect"); + if let Some(expected) = expected { + let actual = json!({ + "method": request.method, + "url": request.url, + "headers": request.headers, + "body": body_json(&request.body), + }); + let wanted = json!({ + "method": expected.method, + "url": expected.url, + "headers": expected.headers, + "body": expected.body, + }); + assert_eq!( + actual, + wanted, + "the request differs from the fixture; actual:\n{}", + serde_json::to_string_pretty(&actual).unwrap() + ); + } + Ok(response) + } + + async fn sleep(&self, _duration: Duration) {} +} + +pub fn replay(name: &str) -> Replay { + Replay::new().expect_all(load(name)) +} + +/// A client of the public endpoint over `transport`, with the fixture +/// token. +pub fn fixture_client(transport: T) -> GcpKms { + GcpKms::new(GcpKmsConfig::new(), transport, AccessToken::new(TOKEN)) +} + +#[derive(Debug, Default)] +struct EmulatorState { + /// Each crypto key's versions; the last is the primary. + keys: HashMap>, + tokens: Vec, + exchanges: usize, + requests: Vec<(String, String, Value)>, + sleeps: Vec, + nonce: u32, +} + +/// An in-process emulation of the Cloud KMS REST API (`encrypt` and +/// `decrypt` of symmetric keys with versions, AES-256-GCM with the +/// associated data bound, the CRC32C checks, Google's error shapes), the +/// OAuth token endpoint (which verifies the JWT's RS256 signature with the +/// test-only key) and the metadata server. It checks the bearer token on +/// every Cloud KMS request. +/// +/// The contract suite uses random keys and ids, so fixed bytes cannot serve +/// it: this answers it instead. Clones share one state. +#[derive(Clone, Debug, Default)] +pub struct KmsEmulator { + state: Arc>, +} + +impl KmsEmulator { + /// An emulator holding [`KEY`] (one version), that accepts [`TOKEN`]. + pub fn new() -> Self { + let emulator = Self::default(); + emulator.add_key(KEY); + emulator.state.lock().unwrap().tokens.push(TOKEN.to_owned()); + emulator + } + + pub fn add_key(&self, name: &str) { + let mut state = self.state.lock().unwrap(); + let seed = state.keys.len() as u8; + state.keys.insert(name.to_owned(), vec![[seed ^ 0x5A; 32]]); + } + + /// Add a version to `name` and make it the primary. + pub fn rotate(&self, name: &str) { + let mut state = self.state.lock().unwrap(); + let versions = state.keys.get_mut(name).unwrap(); + let next = [versions.len() as u8 ^ 0xA5; 32]; + versions.push(next); + } + + /// Revoke every access token. + pub fn revoke_tokens(&self) { + self.state.lock().unwrap().tokens.clear(); + } + + /// Token endpoint and metadata server exchanges so far. + pub fn exchanges(&self) -> usize { + self.state.lock().unwrap().exchanges + } + + /// Every Cloud KMS request: method, path after `/v1/`, body. + pub fn requests(&self) -> Vec<(String, String, Value)> { + self.state.lock().unwrap().requests.clone() + } + + pub fn sleeps(&self) -> Vec { + self.state.lock().unwrap().sleeps.clone() + } + + fn json(status: u16, body: Value) -> VendorResponse { + VendorResponse::new(status, body.to_string().into_bytes()) + .header("content-type", "application/json; charset=UTF-8") + } + + fn error(status: u16, code: &str, message: &str) -> VendorResponse { + Self::json( + status, + json!({ "error": { "code": status, "message": message, "status": code } }), + ) + } + + fn issue(&self, prefix: &str) -> VendorResponse { + let mut state = self.state.lock().unwrap(); + state.exchanges += 1; + let token = format!("{prefix}-{}", state.exchanges); + state.tokens.push(token.clone()); + Self::json( + 200, + json!({ "access_token": token, "expires_in": 3599, "token_type": "Bearer" }), + ) + } + + fn token_exchange(&self, body: &[u8]) -> VendorResponse { + let form: HashMap = + url::form_urlencoded::parse(body).into_owned().collect(); + if form.get("grant_type").map(String::as_str) + != Some("urn:ietf:params:oauth:grant-type:jwt-bearer") + { + return Self::json(400, json!({ "error": "unsupported_grant_type" })); + } + let Some((header, claims)) = form.get("assertion").and_then(|jwt| verify_jwt(jwt)) else { + return Self::json( + 400, + json!({ "error": "invalid_grant", "error_description": "Invalid JWT Signature." }), + ); + }; + let valid = header["alg"] == "RS256" + && header["kid"] == KEY_ID + && claims["iss"] == EMAIL + && claims["aud"] == TOKEN_URI + && claims["scope"] == "https://www.googleapis.com/auth/cloudkms" + && claims["exp"].as_u64() == claims["iat"].as_u64().map(|iat| iat + 3600); + if !valid { + return Self::json( + 400, + json!({ "error": "invalid_grant", "error_description": "Invalid JWT claims." }), + ); + } + self.issue("fixture-exchanged-token") + } + + fn check_crc(body: &Value, field: &str, crc_field: &str) -> Result, VendorResponse> { + let bytes = Base64::decode_vec(body[field].as_str().unwrap_or("")).map_err(|_| { + Self::error( + 400, + "INVALID_ARGUMENT", + &format!("Invalid value at '{field}'"), + ) + })?; + let sent = body[crc_field] + .as_str() + .and_then(|crc| crc.parse::().ok()); + if sent != Some(u64::from(crc32c(&bytes))) { + return Err(Self::error( + 400, + "INVALID_ARGUMENT", + &format!( + "The checksum in field {crc_field} did not match the data in field {field}." + ), + )); + } + Ok(bytes) + } + + fn encrypt(&self, name: &str, body: &Value) -> VendorResponse { + let Some((key_name, pinned)) = parse_key_name(name) else { + return Self::error(400, "INVALID_ARGUMENT", "Invalid resource name"); + }; + let mut state = self.state.lock().unwrap(); + let Some(versions) = state.keys.get(key_name).cloned() else { + return Self::error(404, "NOT_FOUND", &format!("{key_name} not found.")); + }; + let plaintext = match Self::check_crc(body, "plaintext", "plaintextCrc32c") { + Ok(bytes) => bytes, + Err(response) => return response, + }; + let aad = match Self::check_crc( + body, + "additionalAuthenticatedData", + "additionalAuthenticatedDataCrc32c", + ) { + Ok(bytes) => bytes, + Err(response) => return response, + }; + let version = pinned.unwrap_or(versions.len() as u32); + let Some(key) = versions.get(version as usize - 1) else { + return Self::error(404, "NOT_FOUND", "CryptoKeyVersion not found."); + }; + state.nonce += 1; + let mut nonce = [0u8; 12]; + nonce[..4].copy_from_slice(&state.nonce.to_be_bytes()); + let mut ciphertext = version.to_be_bytes().to_vec(); + ciphertext.extend_from_slice(&nonce); + ciphertext.extend( + Aes256Gcm::new_from_slice(key) + .unwrap() + .encrypt( + Nonce::from_slice(&nonce), + Payload { + msg: &plaintext, + aad: &aad, + }, + ) + .unwrap(), + ); + Self::json( + 200, + json!({ + "name": format!("{key_name}/cryptoKeyVersions/{version}"), + "ciphertext": Base64::encode_string(&ciphertext), + "ciphertextCrc32c": crc32c(&ciphertext).to_string(), + "verifiedPlaintextCrc32c": true, + "verifiedAdditionalAuthenticatedDataCrc32c": true, + "protectionLevel": "SOFTWARE", + }), + ) + } + + fn decrypt(&self, name: &str, body: &Value) -> VendorResponse { + match parse_key_name(name) { + Some((_, None)) => {} + _ => return Self::error(400, "INVALID_ARGUMENT", "Invalid resource name"), + } + let Some(versions) = self.state.lock().unwrap().keys.get(name).cloned() else { + return Self::error(404, "NOT_FOUND", &format!("{name} not found.")); + }; + let ciphertext = match Self::check_crc(body, "ciphertext", "ciphertextCrc32c") { + Ok(bytes) => bytes, + Err(response) => return response, + }; + let aad = match Self::check_crc( + body, + "additionalAuthenticatedData", + "additionalAuthenticatedDataCrc32c", + ) { + Ok(bytes) => bytes, + Err(response) => return response, + }; + let invalid = || { + Self::error( + 400, + "INVALID_ARGUMENT", + "Decryption failed: the ciphertext is invalid.", + ) + }; + if ciphertext.len() < 16 { + return invalid(); + } + let version = u32::from_be_bytes(ciphertext[..4].try_into().unwrap()); + let Some(key) = versions.get((version as usize).wrapping_sub(1)) else { + return invalid(); + }; + let Ok(plaintext) = Aes256Gcm::new_from_slice(key).unwrap().decrypt( + Nonce::from_slice(&ciphertext[4..16]), + Payload { + msg: &ciphertext[16..], + aad: &aad, + }, + ) else { + return invalid(); + }; + Self::json( + 200, + json!({ + "plaintext": Base64::encode_string(&plaintext), + "plaintextCrc32c": crc32c(&plaintext).to_string(), + "usedPrimary": version as usize == versions.len(), + "protectionLevel": "SOFTWARE", + }), + ) + } +} + +impl VendorTransport for KmsEmulator { + async fn send(&self, request: VendorRequest) -> Result { + if request.url == TOKEN_URI { + assert_eq!(request.method, "POST"); + assert_eq!( + request.header_value("content-type"), + Some("application/x-www-form-urlencoded") + ); + return Ok(self.token_exchange(&request.body)); + } + if request.url == METADATA_URL { + if request.header_value("metadata-flavor") != Some("Google") { + return Ok(VendorResponse::new( + 403, + b"Missing Metadata-Flavor".to_vec(), + )); + } + return Ok(self.issue("fixture-metadata-token")); + } + let path = request + .url + .strip_prefix(&format!("{ENDPOINT}/v1/")) + .expect("a Cloud KMS URL") + .to_owned(); + let body = body_json(&request.body); + { + let mut state = self.state.lock().unwrap(); + state + .requests + .push((request.method.clone(), path.clone(), body.clone())); + let token = request + .header_value("authorization") + .and_then(|value| value.strip_prefix("Bearer ")) + .unwrap_or(""); + if !state.tokens.iter().any(|t| t == token) { + return Ok(Self::error( + 401, + "UNAUTHENTICATED", + "Request had invalid authentication credentials.", + )); + } + } + assert_eq!(request.method, "POST"); + assert_eq!( + request.header_value("content-type"), + Some("application/json") + ); + Ok(match path.rsplit_once(':') { + Some((name, "encrypt")) => self.encrypt(name, &body), + Some((name, "decrypt")) => self.decrypt(name, &body), + _ => Self::error(404, "NOT_FOUND", "Method not found."), + }) + } + + async fn sleep(&self, duration: Duration) { + self.state.lock().unwrap().sleeps.push(duration); + } +} diff --git a/packages/stack-kms-gcp/tests/contract.rs b/packages/stack-kms-gcp/tests/contract.rs new file mode 100644 index 0000000000..1ba29b0fbd --- /dev/null +++ b/packages/stack-kms-gcp/tests/contract.rs @@ -0,0 +1,202 @@ +//! The shared contract suite, run against the Cloud KMS backend over an +//! in-process emulation of the Cloud KMS API, in both key modes. +//! +//! The suite generates random keys and keyset ids, so fixed exchanges +//! cannot answer it; the emulator does (see `common::KmsEmulator`). The +//! request shapes are pinned by `tests/fixtures.rs`, and real Cloud KMS is +//! exercised by `tests/live.rs`. + +mod common; + +use common::{fixture_client, test_only_service_account, KmsEmulator, KEY}; +use stack_encrypt::StackCipher; +use stack_kms::KeyMode; +use stack_kms::{BackendErrorKind, Error}; +use stack_kms_gcp::{wrapped_key_version, CachedToken, GcpKms, GcpKmsConfig, ServiceAccountLogin}; +use stack_kms_vendor::derive::wrapped_data_key_aad; +use stack_kms_vendor::{contract, KeyVendorBackend, KeysetRecord, RecordError, VendorKeySource}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; + +fn kind_of(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "gcp-kms"); + kind + } + other => panic!("not a backend error: {other:?}"), + } +} + +#[tokio::test] +async fn gcp_kms_meets_the_contract() { + let emulator = KmsEmulator::new(); + contract::run(|| fixture_client(emulator.clone()), KEY).await; +} + +#[tokio::test] +async fn gcp_kms_meets_the_contract_with_service_account_credentials() { + let emulator = KmsEmulator::new(); + contract::run( + || { + GcpKms::new( + GcpKmsConfig::new(), + emulator.clone(), + CachedToken::new(ServiceAccountLogin::new( + test_only_service_account(), + emulator.clone(), + )), + ) + }, + KEY, + ) + .await; + // Each client caches its own token. + assert!(emulator.exchanges() >= 1); +} + +#[tokio::test] +async fn a_wrapped_key_opens_only_under_its_own_aad() { + let emulator = KmsEmulator::new(); + let kms = fixture_client(emulator.clone()); + let id = Uuid::from_u128(1); + let aad = wrapped_data_key_aad(id, &[1u8; 16], "users/email"); + let wrapped = kms + .wrap(KEY, &Protected::new([9u8; 32]), &aad) + .await + .unwrap(); + assert_eq!( + kms.unwrap(KEY, &wrapped, &aad).await.unwrap().risky_ref(), + &[9u8; 32] + ); + let other = wrapped_data_key_aad(id, &[1u8; 16], "users/name"); + let error = kms.unwrap(KEY, &wrapped, &other).await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + // The associated data travels whole. + let requests = emulator.requests(); + let (_, path, body) = &requests[0]; + assert_eq!(path, &format!("{KEY}:encrypt")); + assert_eq!( + body["additionalAuthenticatedData"], + ::encode_string(&aad) + ); +} + +#[tokio::test] +async fn rotation_needs_no_change_to_the_key_reference() { + let emulator = KmsEmulator::new(); + let kms = fixture_client(emulator.clone()); + let old = kms + .wrap(KEY, &Protected::new([5u8; 32]), b"aad") + .await + .unwrap(); + assert_eq!(wrapped_key_version(&old), Some(1)); + emulator.rotate(KEY); + let new = kms + .wrap(KEY, &Protected::new([6u8; 32]), b"aad") + .await + .unwrap(); + assert_eq!(wrapped_key_version(&new), Some(2)); + assert_eq!( + kms.unwrap(KEY, &old, b"aad").await.unwrap().risky_ref(), + &[5u8; 32] + ); + assert_eq!( + kms.unwrap(KEY, &new, b"aad").await.unwrap().risky_ref(), + &[6u8; 32] + ); +} + +#[tokio::test] +async fn a_missing_key_is_not_found() { + let emulator = KmsEmulator::new(); + let kms = fixture_client(emulator); + let missing = "projects/test-only-project/locations/global/keyRings/r/cryptoKeys/missing"; + let error = kms + .wrap(missing, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::NotFound); +} + +/// Seal a value under `record`, then rewrap the record to `new_ref` and +/// open the value through the rewrapped record. +async fn rewrap_and_open(emulator: &KmsEmulator, record: KeysetRecord, new_ref: &str) { + let old = StackCipher::builder() + .kms(VendorKeySource::new(fixture_client(emulator.clone()), [record.clone()]).unwrap()) + .init() + .await + .unwrap(); + let sealed = old + .default_keyset() + .encrypt_tree("v".to_string(), b"ctx") + .await + .unwrap(); + emulator.rotate(KEY); + let rewrapped = record + .rewrap(&fixture_client(emulator.clone()), new_ref) + .await + .unwrap(); + assert_eq!(wrapped_key_version(rewrapped.wrapped_key()), Some(2)); + let new = StackCipher::builder() + .kms(VendorKeySource::new(fixture_client(emulator.clone()), [rewrapped]).unwrap()) + .init() + .await + .unwrap(); + let opened: String = new.decrypt(sealed, b"ctx").await.unwrap(); + assert_eq!(opened, "v"); +} + +#[test] +fn the_same_crypto_key_ignores_the_version() { + let kms = fixture_client(KmsEmulator::new()); + let v2 = format!("{KEY}/cryptoKeyVersions/2"); + let v7 = format!("{KEY}/cryptoKeyVersions/7"); + assert!(kms.same_key(KEY, KEY)); + assert!(kms.same_key(KEY, &v2)); + assert!(kms.same_key(&v2, &v7)); + let other = KEY.replace("fixture-key", "other-key"); + assert!(!kms.same_key(KEY, &other)); + assert!(!kms.same_key(&v2, &format!("{other}/cryptoKeyVersions/2"))); + assert!(!kms.same_key(KEY, "not a key name")); +} + +#[tokio::test] +async fn a_wrapped_record_rewraps_to_a_new_version_and_keeps_old_values() { + for new_ref in [KEY.to_owned(), format!("{KEY}/cryptoKeyVersions/2")] { + let emulator = KmsEmulator::new(); + let record = KeysetRecord::create( + &fixture_client(emulator.clone()), + KEY, + KeyMode::Wrapped, + None, + ) + .await + .unwrap(); + rewrap_and_open(&emulator, record, &new_ref).await; + } +} + +#[tokio::test] +async fn a_wrapped_record_does_not_rewrap_to_another_crypto_key() { + let emulator = KmsEmulator::new(); + let other = KEY.replace("fixture-key", "other-key"); + emulator.add_key(&other); + let kms = fixture_client(emulator.clone()); + let record = KeysetRecord::create(&kms, KEY, KeyMode::Wrapped, None) + .await + .unwrap(); + let before = emulator.requests().len(); + assert!(matches!( + record.rewrap(&kms, other.as_str()).await, + Err(RecordError::RewrapToAnotherKey { .. }) + )); + assert_eq!(emulator.requests().len(), before, "no vendor call"); + + // A derived record may move to another crypto key. + let record = KeysetRecord::create(&kms, KEY, KeyMode::Derived, None) + .await + .unwrap(); + let moved = record.rewrap(&kms, other.as_str()).await.unwrap(); + assert_eq!(moved.key_ref(), other); +} diff --git a/packages/stack-kms-gcp/tests/fixtures.rs b/packages/stack-kms-gcp/tests/fixtures.rs new file mode 100644 index 0000000000..e39a81887c --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures.rs @@ -0,0 +1,541 @@ +//! Each Cloud KMS call, each credential exchange and each error mapping, +//! against fixed HTTP exchanges (`tests/fixtures/*.json`). +//! +//! Google publishes no Cloud KMS emulator, so no fixture was recorded from +//! Google Cloud. Every response is written by hand in the documented +//! Cloud KMS and OAuth 2.0 shapes. Every request side was computed by a +//! separate script, independently of this client (base64, CRC32C, the +//! associated data frame, and the JWT signed by `openssl dgst -sha256 +//! -sign`), and the client must send exactly those bytes. The key that +//! signs the JWT is a TEST-ONLY RSA key generated for these tests +//! (`tests/fixtures/test_only_rsa_key.pkcs8.b64`); it is no one's service +//! account key. `tests/live.rs` runs against real Cloud KMS. + +mod common; + +use std::collections::HashMap; +use std::sync::Arc; +use std::time::Duration; + +use base64ct::{Base64, Encoding}; +use common::{ + fixture_client, issued_at, load, replay, test_only_key_pem, test_only_service_account, + test_only_service_account_json, Fixture, FixtureResponse, KmsEmulator, Replay, EMAIL, KEY, + KEY_ID, +}; +use serde::Deserialize; +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor}; +use stack_kms_gcp::crc32c::crc32c; +use stack_kms_gcp::{ + compact_ciphertext, wrapped_key_version, AccessToken, AuthError, CachedToken, Clock, + DefaultCredentials, EnvLookup, GcpKms, GcpKmsConfig, MetadataServerLogin, ServiceAccountKey, + ServiceAccountLogin, TokenProvider, CLOUD_KMS_SCOPE, +}; +use stack_kms_vendor::derive::wrapped_data_key_aad; +use stack_kms_vendor::{ + KeyVendorBackend, TransportError, VendorRequest, VendorResponse, VendorTransport, +}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; + +const KEYSET: Uuid = Uuid::from_u128(0x0f1e2d3c_4b5a_6978_8796_a5b4c3d2e1f0); + +fn data_aad(n: u8) -> Vec { + wrapped_data_key_aad(KEYSET, &[n; 16], "users/email") +} + +fn kind_of(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "gcp-kms"); + kind + } + other => panic!("not a backend error: {other:?}"), + } +} + +/// The stored wrapped key of a fixture's encrypt response. +fn wrapped_of(name: &str) -> Vec { + let fixtures: Vec = load(name); + let body = &fixtures[0].response.body; + let ciphertext = Base64::decode_vec(body["ciphertext"].as_str().unwrap()).unwrap(); + let version = body["name"] + .as_str() + .unwrap() + .rsplit('/') + .next() + .unwrap() + .parse() + .unwrap(); + compact_ciphertext(version, &ciphertext) +} + +fn fixed_clock() -> Clock { + Arc::new(issued_at) +} + +#[tokio::test] +async fn encrypt_and_decrypt() { + let kms = fixture_client(replay("encrypt.json")); + let wrapped = kms + .wrap(KEY, &Protected::new([0x42u8; 32]), &data_aad(0)) + .await + .unwrap(); + assert_eq!(wrapped, wrapped_of("encrypt.json")); + assert_eq!(wrapped_key_version(&wrapped), Some(3)); + assert!(kms.transport().is_done()); + + let kms = fixture_client(replay("decrypt.json")); + let opened = kms.unwrap(KEY, &wrapped, &data_aad(0)).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x42u8; 32]); + assert!(kms.transport().is_done()); +} + +#[tokio::test] +async fn a_version_name_pins_encrypt_and_decrypt_uses_the_key() { + let pinned = format!("{KEY}/cryptoKeyVersions/2"); + let kms = fixture_client(replay("encrypt_pinned_version.json")); + let wrapped = kms + .wrap(&pinned, &Protected::new([0x43u8; 32]), &data_aad(1)) + .await + .unwrap(); + assert_eq!(wrapped, wrapped_of("encrypt_pinned_version.json")); + assert_eq!(wrapped_key_version(&wrapped), Some(2)); + + // Decrypt names the crypto key, not the version, with either key_ref. + for key_ref in [KEY, pinned.as_str()] { + let kms = fixture_client(replay("decrypt_old_version.json")); + let opened = kms.unwrap(key_ref, &wrapped, &data_aad(1)).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x43u8; 32]); + assert!(kms.transport().is_done()); + } +} + +#[test] +fn the_jwt_assertion_is_the_openssl_known_answer() { + let expected = std::fs::read_to_string(format!( + "{}/tests/fixtures/test_only_jwt_assertion.txt", + env!("CARGO_MANIFEST_DIR") + )) + .unwrap(); + let key = test_only_service_account(); + let assertion = key.assertion(CLOUD_KMS_SCOPE, issued_at()).unwrap(); + assert_eq!(assertion.as_str(), expected.trim()); + // And it verifies with the public key. + let (header, claims) = common::verify_jwt(&assertion).unwrap(); + assert_eq!(header["kid"], KEY_ID); + assert_eq!(claims["iss"], EMAIL); + assert_eq!(claims["iat"], 1_793_880_000u64); + assert_eq!(claims["exp"], 1_793_883_600u64); +} + +#[tokio::test] +async fn a_service_account_key_is_exchanged_for_a_token() { + let login = ServiceAccountLogin::new( + test_only_service_account(), + replay("service_account_token.json"), + ) + .with_clock(fixed_clock()); + let token = login.token().await.unwrap(); + assert_eq!(token.token(), "fixture-service-account-token"); + assert_eq!(token.expires_in(), Some(Duration::from_secs(3599))); + assert!(login.transport().is_done()); +} + +#[tokio::test] +async fn the_metadata_server_gives_a_token() { + let login = MetadataServerLogin::new(replay("metadata_token.json")); + let token = login.token().await.unwrap(); + assert_eq!(token.token(), "fixture-metadata-token"); + assert_eq!(token.expires_in(), Some(Duration::from_secs(3420))); + assert!(login.transport().is_done()); +} + +#[tokio::test] +async fn a_refused_exchange_is_access_denied_and_says_why() { + let login = ServiceAccountLogin::new( + test_only_service_account(), + Replay::new().answer(VendorResponse::new( + 400, + br#"{"error":"invalid_grant","error_description":"Invalid JWT Signature."}"#.to_vec(), + )), + ); + let error = login.token().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert!(error.to_string().contains("Invalid JWT Signature.")); + assert!(!error.to_string().contains("PRIVATE KEY")); +} + +#[test] +fn a_service_account_key_file_is_checked() { + let key = test_only_service_account(); + assert_eq!(key.client_email(), EMAIL); + assert_eq!(key.private_key_id(), Some(KEY_ID)); + assert_eq!(key.token_uri(), "https://oauth2.googleapis.com/token"); + let debug = format!("{key:?}"); + assert!(debug.contains(EMAIL)); + assert!(!debug.contains("PRIVATE KEY")); + + let pem = test_only_key_pem(); + let file = |fields: serde_json::Value| serde_json::to_vec(&fields).unwrap(); + for (json, says) in [ + (b"not json".to_vec(), "not JSON"), + ( + file(serde_json::json!({ "type": "authorized_user", "refresh_token": "x" })), + "\"authorized_user\" is not supported", + ), + ( + file(serde_json::json!({ "client_email": EMAIL, "private_key": pem })), + "no \"type\"", + ), + ( + file(serde_json::json!({ "type": "service_account", "private_key": pem })), + "no client_email", + ), + ( + file(serde_json::json!({ "type": "service_account", "client_email": EMAIL })), + "no private_key", + ), + ( + file(serde_json::json!({ + "type": "service_account", "client_email": EMAIL, "private_key": "nope" + })), + "not an RSA PKCS #8 PEM key", + ), + ( + file(serde_json::json!({ + "type": "service_account", "client_email": EMAIL, "private_key": pem, + "token_uri": "http://oauth2.googleapis.com/token" + })), + "not an HTTPS URL", + ), + ] { + let error = ServiceAccountKey::from_json(&json).unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert!(error.to_string().contains(says), "{error}"); + assert!(!error.to_string().contains("PRIVATE KEY")); + } + + // No token_uri means Google's. + let key = ServiceAccountKey::from_json(&file(serde_json::json!({ + "type": "service_account", "client_email": EMAIL, "private_key": pem + }))) + .unwrap(); + assert_eq!(key.token_uri(), "https://oauth2.googleapis.com/token"); + assert_eq!(key.private_key_id(), None); +} + +fn env(pairs: &[(&str, &str)]) -> EnvLookup { + let map: HashMap = pairs + .iter() + .map(|(k, v)| ((*k).to_owned(), (*v).to_owned())) + .collect(); + Arc::new(move |name| map.get(name).cloned()) +} + +#[tokio::test] +async fn the_default_credentials_come_from_the_environment() { + let path = std::env::temp_dir().join(format!("stack-kms-gcp-sa-{}.json", Uuid::new_v4())); + std::fs::write(&path, test_only_service_account_json()).unwrap(); + let path_text = path.to_str().unwrap(); + + let credentials = DefaultCredentials::from_env_lookup( + KmsEmulator::new(), + &env(&[ + ("GOOGLE_OAUTH_ACCESS_TOKEN", "static-token"), + ("GOOGLE_APPLICATION_CREDENTIALS", path_text), + ]), + ) + .unwrap(); + assert!(matches!(credentials, DefaultCredentials::Token(_))); + assert_eq!(credentials.token().await.unwrap().token(), "static-token"); + + let emulator = KmsEmulator::new(); + let credentials = DefaultCredentials::from_env_lookup( + emulator.clone(), + &env(&[("GOOGLE_APPLICATION_CREDENTIALS", path_text)]), + ) + .unwrap(); + assert!(matches!(credentials, DefaultCredentials::ServiceAccount(_))); + let token = credentials.token().await.unwrap(); + assert_eq!(token.token(), "fixture-exchanged-token-1"); + // Cached. + let _ = credentials.token().await.unwrap(); + assert_eq!(emulator.exchanges(), 1); + + let emulator = KmsEmulator::new(); + let credentials = DefaultCredentials::from_env_lookup(emulator.clone(), &env(&[])).unwrap(); + assert!(matches!(credentials, DefaultCredentials::MetadataServer(_))); + assert_eq!( + credentials.token().await.unwrap().token(), + "fixture-metadata-token-1" + ); + + let error = DefaultCredentials::from_env_lookup( + KmsEmulator::new(), + &env(&[("GCE_METADATA_HOST", "evil/path")]), + ) + .unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + + std::fs::remove_file(&path).unwrap(); + let error = DefaultCredentials::from_env_lookup( + KmsEmulator::new(), + &env(&[("GOOGLE_APPLICATION_CREDENTIALS", path_text)]), + ) + .unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert!(error.to_string().contains("cannot read")); +} + +#[test] +fn the_metadata_host_comes_from_the_environment() { + let login = + MetadataServerLogin::from_env_lookup((), &env(&[("GCE_METADATA_HOST", "127.0.0.1:8080")])) + .unwrap(); + assert_eq!( + login.url(), + "http://127.0.0.1:8080/computeMetadata/v1/instance/service-accounts/default/token" + ); +} + +#[derive(Deserialize)] +struct ErrorCase { + name: String, + response: FixtureResponse, + kind: String, +} + +#[tokio::test] +async fn every_written_error_maps_to_its_kind() { + let cases: Vec = load("errors.json"); + assert!(cases.len() >= 13); + let wrapped = wrapped_of("encrypt.json"); + for case in cases { + let kms = fixture_client(Replay::new().answer(case.response.to_response())); + let error = kms.unwrap(KEY, &wrapped, &data_aad(0)).await.unwrap_err(); + let message = format!("{error:?}"); + assert_eq!( + format!("{:?}", kind_of(error)), + case.kind, + "{}: {message}", + case.name + ); + } +} + +/// A transport that never gets a response. +struct Down; + +impl VendorTransport for Down { + async fn send(&self, _request: VendorRequest) -> Result { + Err(TransportError("connection refused".into())) + } + + async fn sleep(&self, _duration: Duration) {} +} + +#[tokio::test] +async fn no_response_is_unavailable() { + let error = fixture_client(Down) + .unwrap(KEY, &wrapped_of("encrypt.json"), &data_aad(0)) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); + let error = MetadataServerLogin::new(Down).token().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); +} + +struct NoToken; + +impl TokenProvider for NoToken { + async fn token(&self) -> Result { + Err(AuthError::access_denied("none configured")) + } +} + +#[tokio::test] +async fn no_token_is_access_denied_and_sends_nothing() { + let kms = GcpKms::new(GcpKmsConfig::new(), Replay::new(), NoToken); + let error = kms + .wrap(KEY, &Protected::new([1u8; 32]), &data_aad(0)) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::AccessDenied); +} + +#[tokio::test] +async fn a_key_reference_that_is_not_a_key_name_is_refused_before_any_call() { + let emulator = KmsEmulator::new(); + let kms = fixture_client(emulator.clone()); + for bad in [ + "", + "key", + "projects/p/locations/l/keyRings/r", + "projects/p/locations/l/keyRings/r/cryptoKeys/k:decrypt", + "projects/p/locations/l/keyRings/../cryptoKeys/k", + "projects/p/locations/l/keyRings/r/cryptoKeys/k?alt=media", + ] { + let error = kms + .wrap(bad, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::NotFound, "{bad:?}"); + let error = kms + .unwrap(bad, &[1, 0, 0, 0, 1, 9], b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::NotFound, "{bad:?}"); + } + assert!(emulator.requests().is_empty()); +} + +#[tokio::test] +async fn a_wrapped_key_that_is_not_one_is_refused_before_any_call() { + let emulator = KmsEmulator::new(); + let kms = fixture_client(emulator.clone()); + for bad in [&b""[..], &[1, 0, 0, 0, 3], &[2, 0, 0, 0, 1, 7], b"CiQA"] { + let error = kms.unwrap(KEY, bad, b"a").await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + } + assert!(emulator.requests().is_empty()); +} + +fn encrypt_response(fields: serde_json::Value) -> VendorResponse { + VendorResponse::new(200, fields.to_string().into_bytes()) +} + +#[tokio::test] +async fn an_encrypt_response_that_cannot_be_trusted_is_unavailable() { + let ciphertext = b"some ciphertext"; + let good = serde_json::json!({ + "name": format!("{KEY}/cryptoKeyVersions/1"), + "ciphertext": Base64::encode_string(ciphertext), + "ciphertextCrc32c": crc32c(ciphertext).to_string(), + "verifiedPlaintextCrc32c": true, + "verifiedAdditionalAuthenticatedDataCrc32c": true, + }); + // The good one is accepted, with the CRC32C as a string or a number. + let mut as_number = good.clone(); + as_number["ciphertextCrc32c"] = serde_json::json!(crc32c(ciphertext)); + for body in [good.clone(), as_number] { + let kms = fixture_client(Replay::new().answer(encrypt_response(body))); + let wrapped = kms + .wrap(KEY, &Protected::new([1u8; 32]), b"a") + .await + .unwrap(); + assert_eq!(wrapped, compact_ciphertext(1, ciphertext)); + } + let mut bad_bodies = Vec::new(); + for (field, value) in [ + ("ciphertextCrc32c", serde_json::json!("1")), + ("ciphertextCrc32c", serde_json::Value::Null), + ("verifiedPlaintextCrc32c", serde_json::json!(false)), + ( + "verifiedAdditionalAuthenticatedDataCrc32c", + serde_json::Value::Null, + ), + ("ciphertext", serde_json::json!("not base64!")), + ("ciphertext", serde_json::json!("")), + ("ciphertext", serde_json::Value::Null), + ("name", serde_json::Value::Null), + ("name", serde_json::json!(KEY)), + ( + "name", + serde_json::json!( + "projects/p/locations/global/keyRings/r/cryptoKeys/other/cryptoKeyVersions/1" + ), + ), + ] { + let mut body = good.clone(); + body[field] = value; + bad_bodies.push(body.to_string().into_bytes()); + } + bad_bodies.push(b"not json".to_vec()); + for body in bad_bodies { + let kms = fixture_client(Replay::new().answer(VendorResponse::new(200, body.clone()))); + let error = kms + .wrap(KEY, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!( + kind_of(error), + BackendErrorKind::Unavailable, + "{}", + String::from_utf8_lossy(&body) + ); + } +} + +#[tokio::test] +async fn a_decrypt_response_that_cannot_be_trusted_is_refused() { + let wrapped = wrapped_of("encrypt.json"); + let key = [7u8; 32]; + let short = [7u8; 16]; + for (body, kind) in [ + ( + serde_json::json!({ "plaintext": Base64::encode_string(&key), "plaintextCrc32c": "1" }), + BackendErrorKind::Unavailable, + ), + ( + serde_json::json!({ "plaintext": Base64::encode_string(&key) }), + BackendErrorKind::Unavailable, + ), + ( + serde_json::json!({ "plaintextCrc32c": "0" }), + BackendErrorKind::Unavailable, + ), + ( + serde_json::json!({ "plaintext": "not base64!", "plaintextCrc32c": "0" }), + BackendErrorKind::Unavailable, + ), + ( + serde_json::json!({ + "plaintext": Base64::encode_string(&short), + "plaintextCrc32c": crc32c(&short).to_string(), + }), + BackendErrorKind::InvalidCiphertext, + ), + ] { + let kms = fixture_client(Replay::new().answer(encrypt_response(body.clone()))); + let error = kms.unwrap(KEY, &wrapped, &data_aad(0)).await.unwrap_err(); + assert_eq!(kind_of(error), kind, "{body}"); + } +} + +#[tokio::test] +async fn a_401_drops_the_cached_token() { + let emulator = KmsEmulator::new(); + let kms = GcpKms::new( + GcpKmsConfig::new(), + emulator.clone(), + CachedToken::new(ServiceAccountLogin::new( + test_only_service_account(), + emulator.clone(), + )), + ); + let wrapped = kms + .wrap(KEY, &Protected::new([2u8; 32]), b"a") + .await + .unwrap(); + assert_eq!(emulator.exchanges(), 1); + emulator.revoke_tokens(); + let error = kms.unwrap(KEY, &wrapped, b"a").await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::AccessDenied); + // The next call gets a new token. + let opened = kms.unwrap(KEY, &wrapped, b"a").await.unwrap(); + assert_eq!(opened.risky_ref(), &[2u8; 32]); + assert_eq!(emulator.exchanges(), 2); +} + +#[tokio::test] +async fn the_backend_reports_gcp_and_sleeps_on_its_transport() { + let emulator = KmsEmulator::new(); + let kms = fixture_client(emulator.clone()); + assert_eq!(kms.vendor(), KeyVendor::Gcp); + assert!(kms.supports_mode(KeyMode::Wrapped)); + assert!(kms.supports_mode(KeyMode::Derived)); + assert_eq!(kms.batch_limit(), None); + kms.sleep(Duration::from_millis(50)).await; + assert_eq!(emulator.sleeps(), [Duration::from_millis(50)]); +} diff --git a/packages/stack-kms-gcp/tests/fixtures/decrypt.json b/packages/stack-kms-gcp/tests/fixtures/decrypt.json new file mode 100644 index 0000000000..07f3bb6271 --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures/decrypt.json @@ -0,0 +1,39 @@ +[ + { + "request": { + "method": "POST", + "url": "https://cloudkms.googleapis.com/v1/projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key:decrypt", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "ciphertext": "CiQAEmyBUPjIK2G75Uos0XBTQANVyK01UXZ3Jn9jPlVg64fGByFw11jlNY1xc2CCm9H5sWA7NVtff+N10aq9ynog3jga0HlcanMRFECWVec4zyjit5k0fMesDWSCdGxGFZQSHHCOhVW+", + "additionalAuthenticatedData": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAdXNlcnMvZW1haWw=", + "ciphertextCrc32c": "222503263", + "additionalAuthenticatedDataCrc32c": "4091397523" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "plaintext": "QkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=", + "plaintextCrc32c": "2134019127", + "usedPrimary": true, + "protectionLevel": "SOFTWARE" + } + } + } +] diff --git a/packages/stack-kms-gcp/tests/fixtures/decrypt_old_version.json b/packages/stack-kms-gcp/tests/fixtures/decrypt_old_version.json new file mode 100644 index 0000000000..2278431738 --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures/decrypt_old_version.json @@ -0,0 +1,39 @@ +[ + { + "request": { + "method": "POST", + "url": "https://cloudkms.googleapis.com/v1/projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key:decrypt", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "ciphertext": "CiQAJGkadvNFiRYxQukGpPXsH5mn+UILBKXRpfGmC4+IrCQ/q1wYG9KKCbZDl992riv68erBgpebX9t6NChYAE82r0cYIwZ+w5BcZ963iIyCwVmuqBRxcsEdBT4j1zKcVVun5CBPLuCO", + "additionalAuthenticatedData": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAQEBAQEBAQEBAQEBAQEBAQsAAAAAAAAAdXNlcnMvZW1haWw=", + "ciphertextCrc32c": "2110219672", + "additionalAuthenticatedDataCrc32c": "130468208" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "plaintext": "Q0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0M=", + "plaintextCrc32c": "1618493874", + "usedPrimary": false, + "protectionLevel": "SOFTWARE" + } + } + } +] diff --git a/packages/stack-kms-gcp/tests/fixtures/encrypt.json b/packages/stack-kms-gcp/tests/fixtures/encrypt.json new file mode 100644 index 0000000000..4d3ffd1ec0 --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures/encrypt.json @@ -0,0 +1,41 @@ +[ + { + "request": { + "method": "POST", + "url": "https://cloudkms.googleapis.com/v1/projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key:encrypt", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "plaintext": "QkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=", + "additionalAuthenticatedData": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAdXNlcnMvZW1haWw=", + "plaintextCrc32c": "2134019127", + "additionalAuthenticatedDataCrc32c": "4091397523" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "name": "projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key/cryptoKeyVersions/3", + "ciphertext": "CiQAEmyBUPjIK2G75Uos0XBTQANVyK01UXZ3Jn9jPlVg64fGByFw11jlNY1xc2CCm9H5sWA7NVtff+N10aq9ynog3jga0HlcanMRFECWVec4zyjit5k0fMesDWSCdGxGFZQSHHCOhVW+", + "ciphertextCrc32c": "222503263", + "verifiedPlaintextCrc32c": true, + "verifiedAdditionalAuthenticatedDataCrc32c": true, + "protectionLevel": "SOFTWARE" + } + } + } +] diff --git a/packages/stack-kms-gcp/tests/fixtures/encrypt_pinned_version.json b/packages/stack-kms-gcp/tests/fixtures/encrypt_pinned_version.json new file mode 100644 index 0000000000..24352cb779 --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures/encrypt_pinned_version.json @@ -0,0 +1,41 @@ +[ + { + "request": { + "method": "POST", + "url": "https://cloudkms.googleapis.com/v1/projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key/cryptoKeyVersions/2:encrypt", + "headers": [ + [ + "authorization", + "Bearer fixture-access-token" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "plaintext": "Q0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0NDQ0M=", + "additionalAuthenticatedData": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAQEBAQEBAQEBAQEBAQEBAQsAAAAAAAAAdXNlcnMvZW1haWw=", + "plaintextCrc32c": "1618493874", + "additionalAuthenticatedDataCrc32c": "130468208" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "name": "projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key/cryptoKeyVersions/2", + "ciphertext": "CiQAJGkadvNFiRYxQukGpPXsH5mn+UILBKXRpfGmC4+IrCQ/q1wYG9KKCbZDl992riv68erBgpebX9t6NChYAE82r0cYIwZ+w5BcZ963iIyCwVmuqBRxcsEdBT4j1zKcVVun5CBPLuCO", + "ciphertextCrc32c": "2110219672", + "verifiedPlaintextCrc32c": true, + "verifiedAdditionalAuthenticatedDataCrc32c": true, + "protectionLevel": "SOFTWARE" + } + } + } +] diff --git a/packages/stack-kms-gcp/tests/fixtures/errors.json b/packages/stack-kms-gcp/tests/fixtures/errors.json new file mode 100644 index 0000000000..828e164a26 --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures/errors.json @@ -0,0 +1,267 @@ +[ + { + "name": "permission denied on the key", + "kind": "AccessDenied", + "response": { + "status": 403, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 403, + "message": "Permission 'cloudkms.cryptoKeyVersions.useToDecrypt' denied on resource 'projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key' (or it may not exist).", + "status": "PERMISSION_DENIED", + "details": [ + { + "@type": "type.googleapis.com/google.rpc.ErrorInfo", + "reason": "IAM_PERMISSION_DENIED", + "domain": "cloudkms.googleapis.com", + "metadata": { + "permission": "cloudkms.cryptoKeyVersions.useToDecrypt", + "resource": "projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key" + } + } + ] + } + } + } + }, + { + "name": "an expired access token", + "kind": "AccessDenied", + "response": { + "status": 401, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 401, + "message": "Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential.", + "status": "UNAUTHENTICATED" + } + } + } + }, + { + "name": "a key that does not exist", + "kind": "NotFound", + "response": { + "status": 404, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 404, + "message": "projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key not found.", + "status": "NOT_FOUND" + } + } + } + }, + { + "name": "a quota", + "kind": "Throttled", + "response": { + "status": 429, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 429, + "message": "Quota exceeded for quota metric 'Cryptographic requests' and limit 'Cryptographic requests per minute' of service 'cloudkms.googleapis.com'.", + "status": "RESOURCE_EXHAUSTED" + } + } + } + }, + { + "name": "the service is unavailable", + "kind": "Unavailable", + "response": { + "status": 503, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 503, + "message": "The service is currently unavailable.", + "status": "UNAVAILABLE" + } + } + } + }, + { + "name": "a deadline", + "kind": "Unavailable", + "response": { + "status": 504, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 504, + "message": "Deadline expired before operation could complete.", + "status": "DEADLINE_EXCEEDED" + } + } + } + }, + { + "name": "an internal error", + "kind": "Unavailable", + "response": { + "status": 500, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 500, + "message": "Internal error encountered.", + "status": "INTERNAL" + } + } + } + }, + { + "name": "a disabled key version", + "kind": "Unavailable", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 400, + "message": "projects/test-only-project/locations/global/keyRings/cipherstash-fixture/cryptoKeys/fixture-key/cryptoKeyVersions/3 is not enabled, current state is: DISABLED.", + "status": "FAILED_PRECONDITION" + } + } + } + }, + { + "name": "a damaged ciphertext or other associated data", + "kind": "InvalidCiphertext", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 400, + "message": "Decryption failed: the ciphertext is invalid.", + "status": "INVALID_ARGUMENT" + } + } + } + }, + { + "name": "a ciphertext of another key", + "kind": "InvalidCiphertext", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 400, + "message": "Decryption failed: verify that 'name' refers to the correct CryptoKey.", + "status": "INVALID_ARGUMENT" + } + } + } + }, + { + "name": "a CRC32C that does not match", + "kind": "Unavailable", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 400, + "message": "The checksum in field ciphertext_crc32c did not match the data in field ciphertext.", + "status": "INVALID_ARGUMENT" + } + } + } + }, + { + "name": "another invalid argument", + "kind": "AccessDenied", + "response": { + "status": 400, + "headers": [ + [ + "content-type", + "application/json; charset=UTF-8" + ] + ], + "body": { + "error": { + "code": 400, + "message": "Ciphertext must be at most 65536 bytes.", + "status": "INVALID_ARGUMENT" + } + } + } + }, + { + "name": "a gateway error with no JSON", + "kind": "Unavailable", + "response": { + "status": 502, + "headers": [ + [ + "content-type", + "text/html; charset=UTF-8" + ] + ], + "body": "Error 502 (Server Error)!!1" + } + } +] diff --git a/packages/stack-kms-gcp/tests/fixtures/metadata_token.json b/packages/stack-kms-gcp/tests/fixtures/metadata_token.json new file mode 100644 index 0000000000..a31bbd4466 --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures/metadata_token.json @@ -0,0 +1,33 @@ +[ + { + "request": { + "method": "GET", + "url": "http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/token", + "headers": [ + [ + "metadata-flavor", + "Google" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ], + [ + "metadata-flavor", + "Google" + ] + ], + "body": { + "access_token": "fixture-metadata-token", + "expires_in": 3420, + "token_type": "Bearer" + } + } + } +] diff --git a/packages/stack-kms-gcp/tests/fixtures/service_account_token.json b/packages/stack-kms-gcp/tests/fixtures/service_account_token.json new file mode 100644 index 0000000000..02e467c80c --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures/service_account_token.json @@ -0,0 +1,29 @@ +[ + { + "request": { + "method": "POST", + "url": "https://oauth2.googleapis.com/token", + "headers": [ + [ + "content-type", + "application/x-www-form-urlencoded" + ] + ], + "body": "grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=eyJhbGciOiJSUzI1NiIsImtpZCI6InRlc3Qtb25seS1maXh0dXJlLWtleS1pZCIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdGFjay1rbXMtZ2NwLWZpeHR1cmVAdGVzdC1vbmx5LXByb2plY3QuaWFtLmdzZXJ2aWNlYWNjb3VudC5jb20iLCJzY29wZSI6Imh0dHBzOi8vd3d3Lmdvb2dsZWFwaXMuY29tL2F1dGgvY2xvdWRrbXMiLCJhdWQiOiJodHRwczovL29hdXRoMi5nb29nbGVhcGlzLmNvbS90b2tlbiIsImlhdCI6MTc5Mzg4MDAwMCwiZXhwIjoxNzkzODgzNjAwfQ.XJBEwxYG-9iqjRMPihdEoOehSRmBqPGO-gu4f6pFGpqCkWGEOh2VgmwXwrquAx7JqMdl-ctShZML8bFymQ2V_RYuAuOxmQy6kqDOjmVrCFre0mxhAJNDQWuxTBIbABftcay_I7t5m0rX3gpbmM5NnfNJw23JAgbu4EUk2RBaHutGXF8O8YUgjuEUOFr5I_3xmVSw5FNe_dAq9JmV3xHJuI2TvvTfBFJ-KY4xTccHZbdb7YH4y6OQuyq1U679rjOZ0bdqapx3ayvedkL5nQ4zS43Xh8kLulBERJ_X-V6Gupjktq53DwoXOxDSdY5aQBKSP0B6_bj4qIOHQzceISNbRw" + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json; charset=utf-8" + ] + ], + "body": { + "access_token": "fixture-service-account-token", + "expires_in": 3599, + "token_type": "Bearer" + } + } + } +] diff --git a/packages/stack-kms-gcp/tests/fixtures/test_only_jwt_assertion.txt b/packages/stack-kms-gcp/tests/fixtures/test_only_jwt_assertion.txt new file mode 100644 index 0000000000..4692211a2d --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures/test_only_jwt_assertion.txt @@ -0,0 +1 @@ +eyJhbGciOiJSUzI1NiIsImtpZCI6InRlc3Qtb25seS1maXh0dXJlLWtleS1pZCIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdGFjay1rbXMtZ2NwLWZpeHR1cmVAdGVzdC1vbmx5LXByb2plY3QuaWFtLmdzZXJ2aWNlYWNjb3VudC5jb20iLCJzY29wZSI6Imh0dHBzOi8vd3d3Lmdvb2dsZWFwaXMuY29tL2F1dGgvY2xvdWRrbXMiLCJhdWQiOiJodHRwczovL29hdXRoMi5nb29nbGVhcGlzLmNvbS90b2tlbiIsImlhdCI6MTc5Mzg4MDAwMCwiZXhwIjoxNzkzODgzNjAwfQ.XJBEwxYG-9iqjRMPihdEoOehSRmBqPGO-gu4f6pFGpqCkWGEOh2VgmwXwrquAx7JqMdl-ctShZML8bFymQ2V_RYuAuOxmQy6kqDOjmVrCFre0mxhAJNDQWuxTBIbABftcay_I7t5m0rX3gpbmM5NnfNJw23JAgbu4EUk2RBaHutGXF8O8YUgjuEUOFr5I_3xmVSw5FNe_dAq9JmV3xHJuI2TvvTfBFJ-KY4xTccHZbdb7YH4y6OQuyq1U679rjOZ0bdqapx3ayvedkL5nQ4zS43Xh8kLulBERJ_X-V6Gupjktq53DwoXOxDSdY5aQBKSP0B6_bj4qIOHQzceISNbRw diff --git a/packages/stack-kms-gcp/tests/fixtures/test_only_rsa_key.pkcs8.b64 b/packages/stack-kms-gcp/tests/fixtures/test_only_rsa_key.pkcs8.b64 new file mode 100644 index 0000000000..e2a1d80209 --- /dev/null +++ b/packages/stack-kms-gcp/tests/fixtures/test_only_rsa_key.pkcs8.b64 @@ -0,0 +1,22 @@ +MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQCPqbq3hUOm4E3TomwpjulEakQd +AVec6b2FtTtdhEZIEJ7Zp1AlV8KiDxZbv3GcO+mlh7kGHCCScXFkWt2UpVL9/PwBWB8gFl5+UBcP +wC8DgKEHBT3c319mcPGU35O3yVLrphFfhaRG/xU6VqTstD4WwV3C3ebYCa6SUViIjDDR2t35HPfa +duT5RYs7sa1r7rCcwyxQIGRUtB1NSYEZB0ZLadpu7hPSKpO49Uky0x8t4PolE+qsIfB1gAYaJ9hH +CY45cdRe31pe+NOfU+DmvYWZcLHlsCzIBcFGRChxqHgRf51DGp5Mk2elpa5ruLafDgOeBHvBMV7c +heNwMJJdpMINAgMBAAECggEAOT9d986jeEauzLRPHLplqw0bimFkC9O64aKn/up8ZTA2VK5PCipw +FUsk0je2KpdXUk6ExPIFasXI9UAeRyMgMLUUqOZ4NL2lNvlsTyr6sQ8bW+RTVAYxCm3cXsI/J4AS +ynOZPbLQFTiFuYMYYniquz+fwMzTNeiZx5WhzXvr4luXrbFD6uvkvp4ZKys2klM+2Ki1M1tL3Zew +tm+NvaE2RdOhmY1CZY8vOcaNUmNSJ+AmGgxnWTZLdAXXyh3p6eEBem2Ig9qTzyxxX4k1EMCOe8Tf +EeAC018dTmeLj7kYylms+Yt/pOKnLqyKKEHg/IkmQS0qyRpkejx1EwjEf0LPkwKBgQDFFwQVe6AK +g1sCq6OMfksWs/z4LZNS0S9IjqnmYi52DUhAPWoQDv9M5RFwNbYYfGUSUjybDc/qB6406Dh1AO8A +bcnUgR58J32oT3e9Ouyh7TBYFMapG3uSBghACXc8zAvGbIXTtYZrK4BlMxER8OHzPdZegfSNEFWD +enWV+HzYxwKBgQC6mpPzNGGRhPCvykYnL9Z+omuef134OB3dyUp90bsllXVvWCx0EpgIp7E3WCTW +oRJK4IQBKsLOsXHkBDa4l8ZinF0LXaRnkYK9whqG7EbXgS9ekWK0+PJhMKs/Bm3qGK8vmABq0JbQ +rQYeyu8VEW6XZ0cqSWfx/3eQMFT6zAqCiwKBgCHY5QuncoG7s0qPHYRIQC8C9CPO9+kWIa+tSIJX +RDFj8I+PvIJLgQ68nDUQFFi6LEVsOBpM0fMYJJPTebamVWVWdX34OrsJsSvRkfe+gdsBN/c4gjGS +lbMrNAmDVqXrSO5nvsfv+ci37NH0Zi8VlkhGy2Tpno0HTBhLVJxP02kLAoGBAIwIhnCjZXjLt8dj +1yN/LcsYXgSydjVY/2i5rvIemv/IWjl1hN6SOY99MGtk9Tgv5cdVFxq9GTX9SYegdeQu2cyUpOpr +mn2yX9fBOyD3XO5KXt7r4ZQHofS3jjscuw1K+uSPf/iokhUFUyt2hNZdn411Dp24Oo4LdZE6A/u1 +dUmjAoGBALjrI2/EwDIJ2etEyDeR9+TUsfFxDhKGV5voWcIsh8jj1Q1FivBqSeHXgngZM3SZ9cgp +k5dwf3Eusct+24jv+nJFo6I+hDR4UitYzUG2E105+F/CXA4aX46K3q26IYzvnMGvK158AogkqfI3 +3QgwyO+dn+s/hbLy/d/PCw/YhS1r diff --git a/packages/stack-kms-gcp/tests/live.rs b/packages/stack-kms-gcp/tests/live.rs new file mode 100644 index 0000000000..6241b10a9a --- /dev/null +++ b/packages/stack-kms-gcp/tests/live.rs @@ -0,0 +1,107 @@ +//! The Cloud KMS backend against real Google Cloud KMS. +//! +//! Google publishes no Cloud KMS emulator, so this is the only test of the +//! real service. It runs when `STACK_KMS_GCP_LIVE_KEY` names a symmetric +//! `ENCRYPT_DECRYPT` crypto key +//! (`projects/{p}/locations/{l}/keyRings/{r}/cryptoKeys/{k}`) and skips +//! itself otherwise. The credentials come from the environment +//! ([`DefaultCredentials`](stack_kms_gcp::DefaultCredentials)): normally +//! `GOOGLE_APPLICATION_CREDENTIALS`, the path of a service account key file +//! with `roles/cloudkms.cryptoKeyEncrypterDecrypter` on the key. +//! +//! ```text +//! STACK_KMS_GCP_LIVE_KEY=projects/.../cryptoKeys/stack-kms-gcp-live \ +//! GOOGLE_APPLICATION_CREDENTIALS=$HOME/stack-kms-gcp-live.json \ +//! mise run test:live:stack-kms-gcp +//! ``` +//! +//! The opt-in `integration-gcp-kms.yml` workflow runs it in CI. + +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor}; +use stack_kms_gcp::{parse_key_name, wrapped_key_version, GcpKms}; +use stack_kms_vendor::derive::{record_key_aad, wrapped_data_key_aad}; +use stack_kms_vendor::{contract, KeyVendorBackend}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; + +fn live_key() -> Option { + let key = std::env::var("STACK_KMS_GCP_LIVE_KEY") + .ok() + .filter(|key| !key.is_empty())?; + assert!( + matches!(parse_key_name(&key), Some((_, None))), + "STACK_KMS_GCP_LIVE_KEY is not a crypto key name: {key}" + ); + Some(key) +} + +fn kind_of(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "gcp-kms"); + kind + } + other => panic!("not a backend error: {other:?}"), + } +} + +#[tokio::test] +async fn cloud_kms_meets_the_contract() { + let Some(key) = live_key() else { + eprintln!("skipped: set STACK_KMS_GCP_LIVE_KEY to run against Cloud KMS"); + return; + }; + contract::run(|| GcpKms::from_env().expect("credentials"), &key).await; +} + +#[tokio::test] +async fn cloud_kms_binds_the_associated_data_and_maps_errors() { + let Some(key) = live_key() else { + eprintln!("skipped: set STACK_KMS_GCP_LIVE_KEY to run against Cloud KMS"); + return; + }; + let kms = GcpKms::from_env().expect("credentials"); + let id = Uuid::new_v4(); + let aad = wrapped_data_key_aad(id, &[3u8; 16], "users/email"); + let wrapped = kms + .wrap(&key, &Protected::new([0x5Au8; 32]), &aad) + .await + .unwrap(); + assert!(wrapped_key_version(&wrapped).is_some_and(|version| version >= 1)); + let opened = kms.unwrap(&key, &wrapped, &aad).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x5Au8; 32]); + + // Other associated data. + let other = record_key_aad(id, KeyVendor::Gcp, KeyMode::Wrapped); + let error = kms.unwrap(&key, &wrapped, &other).await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + + // A damaged ciphertext. + let mut damaged = wrapped.clone(); + let last = damaged.len() - 1; + damaged[last] ^= 0x01; + let error = kms.unwrap(&key, &damaged, &aad).await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + + // A key that does not exist. Cloud KMS answers PERMISSION_DENIED, not + // NOT_FOUND, when the caller may not list the key ring, so either is + // right here. + let (name, _) = parse_key_name(&key).unwrap(); + let missing = format!( + "{}/stack-kms-gcp-no-such-key-{}", + name.rsplit_once('/').unwrap().0, + Uuid::new_v4().simple() + ); + let error = kms + .wrap(&missing, &Protected::new([1u8; 32]), &aad) + .await + .unwrap_err(); + let kind = kind_of(error); + assert!( + matches!( + kind, + BackendErrorKind::NotFound | BackendErrorKind::AccessDenied + ), + "{kind:?}" + ); +} diff --git a/packages/stack-kms-vault/Cargo.toml b/packages/stack-kms-vault/Cargo.toml new file mode 100644 index 0000000000..50f27467d4 --- /dev/null +++ b/packages/stack-kms-vault/Cargo.toml @@ -0,0 +1,46 @@ +[package] +name = "stack-kms-vault" +description = "The HashiCorp Vault Transit key vendor backend for Stack Encrypt: a hand-written Transit client with batch operations, for native Rust and the WASI guest" +version = "0.1.0" +edition.workspace = true +authors.workspace = true +repository.workspace = true +homepage.workspace = true +keywords.workspace = true +categories.workspace = true +license-file = "LICENSE" +# Publishable, in the `stack-kms-vendor` version group with the vendor seam it +# implements. release-plz.toml keeps `release = false` until the first version +# is on crates.io (published by hand; see stack-kms-vendor's Cargo.toml). + +[features] +default = ["http"] +# `VaultTransit::from_env`: the native transport (`HttpVendorTransport`) and +# the default auth method over it. Native only; on wasm32 it adds nothing. +http = ["stack-kms-vendor/http"] + +[dependencies] +stack-kms = { path = "../stack-kms", version = "0.2.0", default-features = false } +stack-kms-vendor = { path = "../stack-kms-vendor", version = "0.1.0" } + +vitaminc = { workspace = true, features = ["protected"] } +base64ct = { version = "1.7", features = ["alloc"] } +futures = "0.3.25" +serde = { workspace = true } +serde_json = { workspace = true } +thiserror = { workspace = true } +url = { workspace = true } +web-time = { workspace = true } +zeroize = { workspace = true } + +[dev-dependencies] +stack-kms-vendor = { path = "../stack-kms-vendor", features = ["test-support", "http"] } +stack-encrypt = { path = "../stack-encrypt", default-features = false } +aes-gcm = { version = "0.10.3", features = ["zeroize"] } +sha2 = "0.11" +tokio = { workspace = true, features = ["rt", "macros", "time"] } +uuid = { workspace = true } +vitaminc = { workspace = true, features = ["protected", "random"] } + +[package.metadata.docs.rs] +all-features = true diff --git a/packages/stack-kms-vault/LICENSE b/packages/stack-kms-vault/LICENSE new file mode 100644 index 0000000000..2cbd67a66d --- /dev/null +++ b/packages/stack-kms-vault/LICENSE @@ -0,0 +1,96 @@ +# PolyForm Internal Use License 1.0.0 + + + +## Acceptance + +In order to get any license under these terms, you must agree +to them as both strict obligations and conditions to all +your licenses. + +## Copyright License + +The licensor grants you a copyright license for the software +to do everything you might do with the software that would +otherwise infringe the licensor's copyright in it for any +permitted purpose. However, you may only make changes or +new works based on the software according to [Changes and New +Works License](#changes-and-new-works-license), and you may +not distribute the software. + +## Changes and New Works License + +The licensor grants you an additional copyright license to +make changes and new works based on the software for any +permitted purpose. + +## Patent License + +The licensor grants you a patent license for the software that +covers patent claims the licensor can license, or becomes able +to license, that you would infringe by using the software. + +## Fair Use + +You may have "fair use" rights for the software under the +law. These terms do not limit them. + +## Internal Business Use + +Use of the software for the internal business operations of +you and your company is use for a permitted purpose. + +## No Other Rights + +These terms do not allow you to sublicense or transfer any of +your licenses to anyone else, or prevent the licensor from +granting licenses to anyone else. These terms do not imply +any other licenses. + +## Patent Defense + +If you make any written claim that the software infringes or +contributes to infringement of any patent, your patent license +for the software granted under these terms ends immediately. If +your company makes such a claim, your patent license ends +immediately for work on behalf of your company. + +## Violations + +The first time you are notified in writing that you have +violated any of these terms, or done anything with the software +not covered by your licenses, your licenses can nonetheless +continue if you come into full compliance with these terms, +and take practical steps to correct past violations, within +32 days of receiving notice. Otherwise, all your licenses +end immediately. + +## No Liability + +***As far as the law allows, the software comes as is, without +any warranty or condition, and the licensor will not be liable +to you for any damages arising out of these terms or the use +or nature of the software, under any kind of legal claim.*** + +## Definitions + +The **licensor** is the individual or entity offering these +terms, and the **software** is the software the licensor makes +available under these terms. + +**You** refers to the individual or entity agreeing to these +terms. + +**Your company** is any legal entity, sole proprietorship, +or other kind of organization that you work for, plus all +organizations that have control over, are under the control of, +or are under common control with that organization. **Control** +means ownership of substantially all the assets of an entity, +or the power to direct its management and policies by vote, +contract, or otherwise. Control can be direct or indirect. + +**Your licenses** are all the licenses granted to you for the +software under these terms. + +**Use** means anything you do with the software requiring one +of your licenses. diff --git a/packages/stack-kms-vault/src/auth.rs b/packages/stack-kms-vault/src/auth.rs new file mode 100644 index 0000000000..b7fb79beeb --- /dev/null +++ b/packages/stack-kms-vault/src/auth.rs @@ -0,0 +1,729 @@ +//! Vault tokens and the auth methods that get them. +//! +//! [`TokenProvider`] is the seam: the client asks it for a [`VaultToken`] +//! before each request. This crate provides: +//! +//! * [`VaultToken`] itself: a static token, from an explicit value or from +//! `VAULT_TOKEN` ([`VaultToken::from_env`]). Everywhere. The operator +//! renews it. +//! * [`AppRoleLogin`]: AppRole auth (`role_id` and `secret_id` to +//! `/v1/auth/{mount}/login`). Everywhere. +//! * [`KubernetesLogin`]: Kubernetes auth (the pod's service account token +//! to `/v1/auth/{mount}/login`). **Native only**: it reads the token file. +//! * [`CachedToken`]: keeps the token a login returns and logs in again +//! before its lease ends. +//! * [`DefaultAuth`] (native): picks one from the environment. +//! +//! In the WASI guest the host supplies the token (ADR-0011): the guest +//! implements [`TokenProvider`] over a host import. +//! +//! A login always makes a new token. The client does not call +//! `renew-self`: a renewal cannot pass the token's max TTL, so a client +//! that renews must also log in again, and one path is simpler to test. +//! Note that each AppRole login uses up one use of a `secret_id` with a +//! use limit (`secret_id_num_uses`). + +use std::fmt; +use std::future::Future; +use std::sync::{Mutex as SyncMutex, PoisonError}; +use std::time::Duration; + +use futures::lock::Mutex; +use serde::{Deserialize, Serialize}; +use stack_kms::{BackendErrorKind, MaybeSend}; +use stack_kms_vendor::{classify_status, VendorRequest, VendorTransport}; +use web_time::SystemTime; +use zeroize::Zeroizing; + +use crate::config::{is_path, process_env, EnvLookup, VaultConfig}; +use crate::time::{system_clock, Clock}; + +/// A Vault token, and how long its lease lasts from when it was issued. +/// +/// The token is wiped on drop and never printed. +#[derive(Clone)] +pub struct VaultToken { + token: Zeroizing, + lease: Option, +} + +impl fmt::Debug for VaultToken { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("VaultToken") + .field("token", &"") + .field("lease", &self.lease) + .finish() + } +} + +impl VaultToken { + /// A token with no known lease: it is used until Vault refuses it. + pub fn new(token: impl Into) -> Self { + Self { + token: Zeroizing::new(token.into()), + lease: None, + } + } + + /// The same token, whose lease lasts `lease` from now. + pub fn with_lease(mut self, lease: Duration) -> Self { + self.lease = Some(lease); + self + } + + /// The token from `VAULT_TOKEN`, or `None` when it is not set. + pub fn from_env() -> Option { + Self::from_env_lookup(&process_env()) + } + + /// As [`from_env`](Self::from_env), reading the variable with `env`. + pub fn from_env_lookup(env: &EnvLookup) -> Option { + env("VAULT_TOKEN").map(Self::new) + } + + /// The token. + pub fn token(&self) -> &str { + &self.token + } + + /// How long the lease lasts from when the token was issued. `None` for + /// a token with no known lease. + pub fn lease(&self) -> Option { + self.lease + } +} + +/// A token could not be got. +#[derive(Debug, thiserror::Error)] +#[error("Vault auth: {message}")] +pub struct AuthError { + kind: BackendErrorKind, + message: String, +} + +impl AuthError { + /// A failure of `kind`, described by `message`. Never put a secret in + /// the message. + pub fn new(kind: BackendErrorKind, message: impl Into) -> Self { + Self { + kind, + message: message.into(), + } + } + + /// No auth method is configured, or the login was refused. + pub fn access_denied(message: impl Into) -> Self { + Self::new(BackendErrorKind::AccessDenied, message) + } + + /// The auth method did not answer. + pub fn unavailable(message: impl Into) -> Self { + Self::new(BackendErrorKind::Unavailable, message) + } + + /// The failure kind the client reports. + pub fn kind(&self) -> BackendErrorKind { + self.kind + } +} + +/// Where a client gets its Vault token. The client asks before each +/// request, so a provider that logs in should cache ([`CachedToken`]). +/// +/// The returned future is bounded by [`MaybeSend`]: `Send` on native +/// targets, unbounded on wasm32. +pub trait TokenProvider: Send + Sync { + /// The current token. + fn token(&self) -> impl Future> + MaybeSend; + + /// Vault refused the current token (HTTP 403). A caching provider drops + /// it, so that the next request logs in again. The default does + /// nothing. + fn invalidate(&self) {} +} + +impl TokenProvider for VaultToken { + fn token(&self) -> impl Future> + MaybeSend { + std::future::ready(Ok(self.clone())) + } +} + +/// The default time before the end of a lease at which [`CachedToken`] +/// logs in again. For a short lease the window is a third of the lease. +pub const DEFAULT_REFRESH_BEFORE_EXPIRY: Duration = Duration::from_secs(5 * 60); + +/// A token in the cache, and when to get the next one. +struct Cached { + token: VaultToken, + /// When to log in again. `None`: never. + refresh_at: Option, + /// When the lease ends. `None`: never. + expires_at: Option, +} + +/// Keeps the token `inner` returns, and gets a new one when its lease is +/// near its end: [`DEFAULT_REFRESH_BEFORE_EXPIRY`] before, or a third of +/// the lease before when the lease is shorter than three times that. +/// +/// One login runs at a time: callers that arrive during a login wait for +/// it and share its result. If a login fails while the cached token's lease +/// has not ended, the cached token is still returned. A token with no +/// lease is kept until Vault refuses it ([`TokenProvider::invalidate`]). +pub struct CachedToken

{ + inner: P, + cached: SyncMutex>, + fetch: Mutex<()>, + refresh_before: Duration, + clock: Clock, +} + +impl fmt::Debug for CachedToken

{ + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("CachedToken") + .field("inner", &self.inner) + .field("refresh_before", &self.refresh_before) + .finish_non_exhaustive() + } +} + +impl CachedToken

{ + /// Cache the token of `inner`. + pub fn new(inner: P) -> Self { + Self { + inner, + cached: SyncMutex::new(None), + fetch: Mutex::new(()), + refresh_before: DEFAULT_REFRESH_BEFORE_EXPIRY, + clock: system_clock(), + } + } + + /// Log in again at most `refresh_before` before the lease ends. + pub fn with_refresh_before(mut self, refresh_before: Duration) -> Self { + self.refresh_before = refresh_before; + self + } + + /// Read the time from `clock`. + pub fn with_clock(mut self, clock: Clock) -> Self { + self.clock = clock; + self + } + + /// The provider whose token is cached. + pub fn inner(&self) -> &P { + &self.inner + } + + /// The cached token, if it does not need a login yet. + fn fresh(&self, now: SystemTime) -> Option { + let cached = self.cached.lock().unwrap_or_else(PoisonError::into_inner); + cached + .as_ref() + .filter(|cached| cached.refresh_at.is_none_or(|at| now < at)) + .map(|cached| cached.token.clone()) + } +} + +impl TokenProvider for CachedToken

{ + async fn token(&self) -> Result { + if let Some(token) = self.fresh((self.clock)()) { + return Ok(token); + } + let _fetch = self.fetch.lock().await; + // Another caller may have logged in while this one waited. + let now = (self.clock)(); + if let Some(token) = self.fresh(now) { + return Ok(token); + } + match self.inner.token().await { + Ok(token) => { + let (refresh_at, expires_at) = match token.lease { + Some(lease) => { + let window = self.refresh_before.min(lease / 3); + (now.checked_add(lease - window), now.checked_add(lease)) + } + None => (None, None), + }; + *self.cached.lock().unwrap_or_else(PoisonError::into_inner) = Some(Cached { + token: token.clone(), + refresh_at, + expires_at, + }); + Ok(token) + } + Err(error) => { + let cached = self.cached.lock().unwrap_or_else(PoisonError::into_inner); + match cached.as_ref() { + Some(cached) if cached.expires_at.is_some_and(|end| now < end) => { + Ok(cached.token.clone()) + } + _ => Err(error), + } + } + } + } + + fn invalidate(&self) { + *self.cached.lock().unwrap_or_else(PoisonError::into_inner) = None; + self.inner.invalidate(); + } +} + +#[derive(Deserialize)] +struct LoginResponse { + auth: Option, +} + +#[derive(Deserialize)] +struct LoginAuth { + client_token: Option, + #[serde(default)] + lease_duration: u64, +} + +#[derive(Deserialize)] +struct ErrorBody { + #[serde(default)] + errors: Vec, +} + +/// Log in at `/v1/auth/{mount}/login` with the JSON `body`. The body is +/// wiped after it is sent. +async fn login( + config: &VaultConfig, + transport: &T, + method: &'static str, + mount: &str, + body: Zeroizing>, +) -> Result { + let mut request = VendorRequest::new("POST", config.url(&format!("auth/{mount}/login"))) + .header("content-type", "application/json") + .header("x-vault-request", "true"); + request.body = body; + if let Some(namespace) = config.namespace() { + request = request.header("x-vault-namespace", namespace); + } + let response = transport.send(request).await.map_err(|error| { + AuthError::unavailable(format!("{method} login got no response: {error}")) + })?; + if let Some(kind) = classify_status(response.status) { + // A refused login is HTTP 400 ("invalid role or secret ID") or 403. + let kind = match kind { + BackendErrorKind::InvalidCiphertext => BackendErrorKind::AccessDenied, + kind => kind, + }; + let errors = serde_json::from_slice::(&response.body) + .map(|body| body.errors.join("; ")) + .unwrap_or_default(); + return Err(AuthError::new( + kind, + format!( + "{method} login failed with HTTP {}: {errors}", + response.status + ), + )); + } + let parsed: LoginResponse = serde_json::from_slice(&response.body).map_err(|_| { + AuthError::unavailable(format!( + "{method} login returned a response that is not JSON" + )) + })?; + let auth = parsed + .auth + .ok_or_else(|| AuthError::unavailable(format!("{method} login returned no auth")))?; + let token = Zeroizing::new( + auth.client_token + .ok_or_else(|| AuthError::unavailable(format!("{method} login returned no token")))?, + ); + let token = VaultToken { token, lease: None }; + Ok(match auth.lease_duration { + 0 => token, + seconds => token.with_lease(Duration::from_secs(seconds)), + }) +} + +fn check_mount(method: &str, mount: &str) -> Result<(), AuthError> { + if is_path(mount) { + Ok(()) + } else { + Err(AuthError::access_denied(format!( + "{mount:?} is not a {method} auth mount path" + ))) + } +} + +/// The default mount path of AppRole auth. +pub const DEFAULT_APPROLE_MOUNT: &str = "approle"; + +/// AppRole auth: `role_id` and `secret_id` to `/v1/auth/{mount}/login`. +/// Wrap it in a [`CachedToken`]. +pub struct AppRoleLogin { + config: VaultConfig, + transport: T, + mount: String, + role_id: String, + secret_id: Zeroizing, +} + +impl fmt::Debug for AppRoleLogin { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("AppRoleLogin") + .field("mount", &self.mount) + .field("role_id", &self.role_id) + .field("secret_id", &"") + .finish_non_exhaustive() + } +} + +impl AppRoleLogin { + /// Log in to the server of `config` (address and namespace) through + /// `transport`, at [`DEFAULT_APPROLE_MOUNT`]. + pub fn new( + config: &VaultConfig, + transport: T, + role_id: impl Into, + secret_id: impl Into, + ) -> Self { + Self { + config: config.clone(), + transport, + mount: DEFAULT_APPROLE_MOUNT.to_owned(), + role_id: role_id.into(), + secret_id: Zeroizing::new(secret_id.into()), + } + } + + /// Log in at the AppRole auth mount `mount` instead. + pub fn with_mount(mut self, mount: impl Into) -> Self { + self.mount = mount.into().trim_matches('/').to_owned(); + self + } + + /// The transport. + pub fn transport(&self) -> &T { + &self.transport + } +} + +#[derive(Serialize)] +struct AppRoleBody<'a> { + role_id: &'a str, + secret_id: &'a str, +} + +impl TokenProvider for AppRoleLogin { + async fn token(&self) -> Result { + check_mount("AppRole", &self.mount)?; + let body = Zeroizing::new( + serde_json::to_vec(&AppRoleBody { + role_id: &self.role_id, + secret_id: &self.secret_id, + }) + .map_err(|_| AuthError::unavailable("AppRole login body"))?, + ); + login(&self.config, &self.transport, "AppRole", &self.mount, body).await + } +} + +#[cfg(not(target_arch = "wasm32"))] +mod kubernetes { + use std::path::PathBuf; + + use super::*; + + /// The default mount path of Kubernetes auth. + pub const DEFAULT_KUBERNETES_MOUNT: &str = "kubernetes"; + + /// Where Kubernetes mounts the pod's service account token. + pub const DEFAULT_SERVICE_ACCOUNT_TOKEN_PATH: &str = + "/var/run/secrets/kubernetes.io/serviceaccount/token"; + + /// Kubernetes auth: the pod's service account token (a JWT) and a + /// Vault role to `/v1/auth/{mount}/login`. The token file is read on + /// each login, because Kubernetes rotates it. Native only. Wrap it in a + /// [`CachedToken`]. + pub struct KubernetesLogin { + config: VaultConfig, + transport: T, + mount: String, + role: String, + token_path: PathBuf, + } + + impl fmt::Debug for KubernetesLogin { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("KubernetesLogin") + .field("mount", &self.mount) + .field("role", &self.role) + .field("token_path", &self.token_path) + .finish_non_exhaustive() + } + } + + impl KubernetesLogin { + /// Log in as `role` to the server of `config` through `transport`, + /// at [`DEFAULT_KUBERNETES_MOUNT`], with the token at + /// [`DEFAULT_SERVICE_ACCOUNT_TOKEN_PATH`]. + pub fn new(config: &VaultConfig, transport: T, role: impl Into) -> Self { + Self { + config: config.clone(), + transport, + mount: DEFAULT_KUBERNETES_MOUNT.to_owned(), + role: role.into(), + token_path: PathBuf::from(DEFAULT_SERVICE_ACCOUNT_TOKEN_PATH), + } + } + + /// Log in at the Kubernetes auth mount `mount` instead. + pub fn with_mount(mut self, mount: impl Into) -> Self { + self.mount = mount.into().trim_matches('/').to_owned(); + self + } + + /// Read the service account token from `path` instead. + pub fn with_token_path(mut self, path: impl Into) -> Self { + self.token_path = path.into(); + self + } + } + + #[derive(Serialize)] + struct KubernetesBody<'a> { + role: &'a str, + jwt: &'a str, + } + + impl TokenProvider for KubernetesLogin { + async fn token(&self) -> Result { + check_mount("Kubernetes", &self.mount)?; + let jwt = + Zeroizing::new(std::fs::read_to_string(&self.token_path).map_err(|error| { + AuthError::access_denied(format!( + "cannot read the service account token at {}: {}", + self.token_path.display(), + error.kind() + )) + })?); + let body = Zeroizing::new( + serde_json::to_vec(&KubernetesBody { + role: &self.role, + jwt: jwt.trim(), + }) + .map_err(|_| AuthError::unavailable("Kubernetes login body"))?, + ); + login( + &self.config, + &self.transport, + "Kubernetes", + &self.mount, + body, + ) + .await + } + } + + /// The auth method [`DefaultAuth::from_env`] picks. + #[derive(Debug)] + pub enum DefaultAuth { + /// `VAULT_TOKEN`. + Token(VaultToken), + /// `VAULT_APPROLE_ROLE_ID` and `VAULT_APPROLE_SECRET_ID`. + AppRole(CachedToken>), + /// `VAULT_KUBERNETES_ROLE`. + Kubernetes(CachedToken>), + } + + impl DefaultAuth { + /// Pick an auth method from the environment, in this order: + /// + /// 1. `VAULT_TOKEN`: a static token. + /// 2. `VAULT_APPROLE_ROLE_ID` and `VAULT_APPROLE_SECRET_ID`: AppRole, + /// at `VAULT_APPROLE_MOUNT` (default `approle`). + /// 3. `VAULT_KUBERNETES_ROLE`: Kubernetes, at + /// `VAULT_KUBERNETES_MOUNT` (default `kubernetes`), with the token + /// at `VAULT_KUBERNETES_TOKEN_PATH` (default the service account + /// token path). + /// + /// The login methods log in through `transport` and are cached. + pub fn from_env(config: &VaultConfig, transport: T) -> Result { + Self::from_env_lookup(config, transport, &process_env()) + } + + /// As [`from_env`](Self::from_env), reading the variables with `env`. + pub fn from_env_lookup( + config: &VaultConfig, + transport: T, + env: &EnvLookup, + ) -> Result { + if let Some(token) = VaultToken::from_env_lookup(env) { + return Ok(Self::Token(token)); + } + if let (Some(role_id), Some(secret_id)) = ( + env("VAULT_APPROLE_ROLE_ID"), + env("VAULT_APPROLE_SECRET_ID").map(Zeroizing::new), + ) { + let mut login = AppRoleLogin::new(config, transport, role_id, secret_id.as_str()); + if let Some(mount) = env("VAULT_APPROLE_MOUNT") { + login = login.with_mount(mount); + } + return Ok(Self::AppRole(CachedToken::new(login))); + } + if let Some(role) = env("VAULT_KUBERNETES_ROLE") { + let mut login = KubernetesLogin::new(config, transport, role); + if let Some(mount) = env("VAULT_KUBERNETES_MOUNT") { + login = login.with_mount(mount); + } + if let Some(path) = env("VAULT_KUBERNETES_TOKEN_PATH") { + login = login.with_token_path(path); + } + return Ok(Self::Kubernetes(CachedToken::new(login))); + } + Err(AuthError::access_denied( + "no Vault auth: set VAULT_TOKEN, VAULT_APPROLE_ROLE_ID and \ + VAULT_APPROLE_SECRET_ID, or VAULT_KUBERNETES_ROLE", + )) + } + } + + impl TokenProvider for DefaultAuth { + async fn token(&self) -> Result { + match self { + Self::Token(token) => Ok(token.clone()), + Self::AppRole(login) => login.token().await, + Self::Kubernetes(login) => login.token().await, + } + } + + fn invalidate(&self) { + match self { + Self::Token(_) => {} + Self::AppRole(login) => login.invalidate(), + Self::Kubernetes(login) => login.invalidate(), + } + } + } +} + +#[cfg(not(target_arch = "wasm32"))] +pub use kubernetes::{ + DefaultAuth, KubernetesLogin, DEFAULT_KUBERNETES_MOUNT, DEFAULT_SERVICE_ACCOUNT_TOKEN_PATH, +}; + +#[cfg(test)] +mod tests { + use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering}; + use std::sync::Arc; + + use web_time::UNIX_EPOCH; + + use super::*; + + #[test] + fn debug_hides_the_token() { + let text = format!("{:?}", VaultToken::new("hvs.secret")); + assert!(!text.contains("hvs.secret")); + let text = format!( + "{:?}", + AppRoleLogin::new( + &VaultConfig::new("http://vault:8200").unwrap(), + (), + "role", + "the-secret-id" + ) + ); + assert!(text.contains("role")); + assert!(!text.contains("the-secret-id")); + } + + struct Counting { + calls: AtomicUsize, + fail: AtomicBool, + lease: Option, + } + + impl TokenProvider for Counting { + async fn token(&self) -> Result { + let call = self.calls.fetch_add(1, Ordering::SeqCst); + if self.fail.load(Ordering::SeqCst) { + return Err(AuthError::unavailable("down")); + } + let token = VaultToken::new(format!("token-{call}")); + Ok(match self.lease { + Some(lease) => token.with_lease(lease), + None => token, + }) + } + } + + fn cached(lease: Option) -> (CachedToken, Arc) { + let now = Arc::new(AtomicU64::new(1_000_000)); + let clock_now = Arc::clone(&now); + let clock: Clock = + Arc::new(move || UNIX_EPOCH + Duration::from_secs(clock_now.load(Ordering::SeqCst))); + let provider = Counting { + calls: AtomicUsize::new(0), + fail: false.into(), + lease, + }; + (CachedToken::new(provider).with_clock(clock), now) + } + + async fn token_of(cache: &CachedToken) -> String { + cache.token().await.unwrap().token().to_owned() + } + + #[tokio::test] + async fn a_token_is_got_again_before_its_lease_ends() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(3600 - 301, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-1"); + assert_eq!(cache.inner().calls.load(Ordering::SeqCst), 2); + } + + #[tokio::test] + async fn a_short_lease_refreshes_a_third_before_its_end() { + let (cache, now) = cached(Some(Duration::from_secs(60))); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(39, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-1"); + } + + #[tokio::test] + async fn a_token_with_no_lease_is_kept_until_invalidated() { + let (cache, now) = cached(None); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(10 * 365 * 86_400, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + cache.invalidate(); + assert_eq!(token_of(&cache).await, "token-1"); + } + + #[tokio::test] + async fn a_failed_login_keeps_a_token_whose_lease_has_not_ended() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + assert_eq!(token_of(&cache).await, "token-0"); + cache.inner().fail.store(true, Ordering::SeqCst); + now.fetch_add(3500, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(100, Ordering::SeqCst); + let error = cache.token().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::Unavailable); + } + + #[tokio::test] + async fn the_refresh_window_is_configurable() { + let (cache, now) = cached(Some(Duration::from_secs(3600))); + let cache = cache.with_refresh_before(Duration::ZERO); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(3599, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-0"); + now.fetch_add(1, Ordering::SeqCst); + assert_eq!(token_of(&cache).await, "token-1"); + } +} diff --git a/packages/stack-kms-vault/src/client.rs b/packages/stack-kms-vault/src/client.rs new file mode 100644 index 0000000000..3e804adbac --- /dev/null +++ b/packages/stack-kms-vault/src/client.rs @@ -0,0 +1,710 @@ +//! The Vault Transit client and its [`KeyVendorBackend`] implementation. + +use std::collections::HashMap; +use std::num::NonZeroUsize; +use std::sync::{Mutex, PoisonError}; +use std::time::Duration; + +use base64ct::{Base64, Encoding}; +use serde::de::DeserializeOwned; +use serde::{Deserialize, Serialize}; +use stack_kms::{BackendErrorKind, Error, KeyVendor}; +use stack_kms_vendor::bound::{bind_key, open_bound_key}; +use stack_kms_vendor::{ + classify_status, KeyBytes, KeyVendorBackend, TransportError, UnwrapItem, VendorRequest, + VendorResponse, VendorTransport, WrapItem, +}; +use zeroize::Zeroizing; + +use crate::auth::{AuthError, TokenProvider}; +use crate::config::{is_segment, KeyDerivation, VaultConfig}; + +/// The vendor name in errors ([`KeyVendor::name`] of [`KeyVendor::Vault`]). +const VENDOR: &str = "vault-transit"; + +/// The first byte of a wrapped key this client stores: the format of the +/// rest (`version(4, BE) ‖ Vault ciphertext bytes`). +pub const WRAPPED_KEY_FORMAT: u8 = 1; + +/// The Transit key types whose encryption is an AEAD. The client binds the +/// associated data inside the plaintext +/// ([`stack_kms_vendor::bound`]), which is sound only under a +/// non-malleable cipher, so [`KeyDerivation::Detect`] refuses other types. +pub const AEAD_KEY_TYPES: [&str; 3] = ["aes256-gcm96", "aes128-gcm96", "chacha20-poly1305"]; + +/// The text in a Vault HTTP 400 error that means "this ciphertext does not +/// open" ([`BackendErrorKind::InvalidCiphertext`]), lower case. Checked +/// after [`NOT_FOUND_MARKERS`]. +pub const INVALID_CIPHERTEXT_MARKERS: [&str; 6] = [ + "invalid ciphertext", + "message authentication failed", + "unable to decrypt", + "could not decode", + "ciphertext version", + "version is too new", +]; + +/// The text in a Vault HTTP 400 error that means "no such key" +/// ([`BackendErrorKind::NotFound`]), lower case. +pub const NOT_FOUND_MARKERS: [&str; 2] = ["encryption key not found", "key not found"]; + +/// A failure of a Vault call. It is the `source` of the [`Error::Backend`] +/// the client returns; the kind is on the `Error::Backend`. +#[derive(Debug, thiserror::Error)] +pub enum VaultError { + /// Vault answered with an error. + #[error("Vault {operation} failed with HTTP {status}: {message}")] + Service { + /// The operation (`decrypt`). + operation: &'static str, + /// The HTTP status. + status: u16, + /// Vault's error text. + message: String, + }, + /// The request got no response. + #[error("Vault {operation} got no response: {source}")] + Transport { + /// The operation. + operation: &'static str, + /// The transport failure. + source: TransportError, + }, + /// No token could be got. + #[error(transparent)] + Auth(#[from] AuthError), + /// Vault answered with a success that this client cannot read. + #[error("Vault {operation} returned a response this client cannot read: {reason}")] + InvalidResponse { + /// The operation. + operation: &'static str, + /// What is wrong with it. + reason: &'static str, + }, + /// The client refused the call before or after it reached Vault. + #[error("{0}")] + Refused(&'static str), +} + +fn backend(kind: BackendErrorKind, error: VaultError) -> Error { + Error::Backend { + vendor: VENDOR, + kind, + source: Box::new(error), + } +} + +fn invalid(operation: &'static str, reason: &'static str) -> Error { + backend( + BackendErrorKind::Unavailable, + VaultError::InvalidResponse { operation, reason }, + ) +} + +fn refused(kind: BackendErrorKind, reason: &'static str) -> Error { + backend(kind, VaultError::Refused(reason)) +} + +/// The failure kind of a Vault error response. +/// +/// * 401 and 403: `AccessDenied`. 404: `NotFound` (a mount or key that does +/// not exist). 429: `Throttled` (a rate limit quota). 5xx: `Unavailable` +/// (503 is a sealed or standby Vault). +/// * 400: by the error text, lower case. [`NOT_FOUND_MARKERS`] first +/// (`NotFound`), then [`INVALID_CIPHERTEXT_MARKERS`] +/// (`InvalidCiphertext`). Any other 400 is a request Vault refused +/// (`AccessDenied`), for example a derived key with no `context`. +/// * Any other 4xx: `AccessDenied`. +pub fn error_kind(status: u16, message: &str) -> BackendErrorKind { + if status == 400 { + let message = message.to_ascii_lowercase(); + return if NOT_FOUND_MARKERS.iter().any(|m| message.contains(m)) { + BackendErrorKind::NotFound + } else if INVALID_CIPHERTEXT_MARKERS + .iter() + .any(|m| message.contains(m)) + { + BackendErrorKind::InvalidCiphertext + } else { + BackendErrorKind::AccessDenied + }; + } + match classify_status(status) { + // Another 4xx is a refused request, not a damaged ciphertext. + Some(BackendErrorKind::InvalidCiphertext) => BackendErrorKind::AccessDenied, + Some(kind) => kind, + None => BackendErrorKind::Unavailable, + } +} + +#[derive(Deserialize)] +struct ErrorBody { + #[serde(default)] + errors: Vec, + data: Option, +} + +/// Vault's error text: the first failed batch item's, else the `errors` +/// list. +fn failure_message(body: &[u8]) -> String { + let Ok(body) = serde_json::from_slice::(body) else { + return String::new(); + }; + body.data + .and_then(|data| { + data.batch_results + .into_iter() + .find_map(|result| result.error.filter(|e| !e.is_empty())) + }) + .unwrap_or_else(|| body.errors.join("; ")) +} + +fn service_error(operation: &'static str, status: u16, message: String) -> Error { + backend( + error_kind(status, &message), + VaultError::Service { + operation, + status, + message, + }, + ) +} + +#[derive(Deserialize)] +struct BatchResponse { + data: Option, +} + +#[derive(Deserialize)] +struct BatchData { + #[serde(default)] + batch_results: Vec, +} + +#[derive(Deserialize)] +struct BatchResult { + ciphertext: Option, + plaintext: Option, + error: Option, +} + +#[derive(Serialize)] +struct BatchRequest<'a, I> { + batch_input: &'a [I], +} + +#[derive(Serialize)] +struct EncryptItem { + #[serde(serialize_with = "serialize_secret")] + plaintext: Zeroizing, + #[serde(skip_serializing_if = "Option::is_none")] + context: Option, +} + +fn serialize_secret( + value: &Zeroizing, + serializer: S, +) -> Result { + serializer.serialize_str(value) +} + +#[derive(Serialize)] +struct CiphertextItem { + ciphertext: String, + #[serde(skip_serializing_if = "Option::is_none")] + context: Option, +} + +#[derive(Deserialize)] +struct KeyRead { + data: Option, +} + +#[derive(Deserialize)] +struct KeyData { + #[serde(default)] + derived: bool, + #[serde(rename = "type")] + key_type: Option, + #[serde(default)] + supports_encryption: bool, + #[serde(default)] + supports_decryption: bool, +} + +/// The stored form of a Vault ciphertext (`vault:v:`): +/// [`WRAPPED_KEY_FORMAT`], the key version (4 bytes, big-endian), and the +/// decoded bytes. About 40 bytes shorter than the text. +pub fn compact_ciphertext(ciphertext: &str) -> Option> { + let rest = ciphertext.strip_prefix("vault:v")?; + let (version, encoded) = rest.split_once(':')?; + if version.is_empty() || !version.bytes().all(|b| b.is_ascii_digit()) { + return None; + } + let version: u32 = version.parse().ok()?; + let bytes = Base64::decode_vec(encoded).ok()?; + if bytes.is_empty() || Base64::encode_string(&bytes) != encoded { + return None; + } + let mut out = Vec::with_capacity(5 + bytes.len()); + out.push(WRAPPED_KEY_FORMAT); + out.extend_from_slice(&version.to_be_bytes()); + out.extend_from_slice(&bytes); + Some(out) +} + +/// The Vault ciphertext text of a stored wrapped key. `None` if the bytes +/// are not one. +pub fn expand_ciphertext(wrapped: &[u8]) -> Option { + let (&format, rest) = wrapped.split_first()?; + if format != WRAPPED_KEY_FORMAT { + return None; + } + let (version, bytes) = rest.split_at_checked(4)?; + if bytes.is_empty() { + return None; + } + let version = u32::from_be_bytes(version.try_into().ok()?); + Some(format!("vault:v{version}:{}", Base64::encode_string(bytes))) +} + +/// A HashiCorp Vault Transit client, and the Vault Transit key vendor +/// backend ([`KeyVendor::Vault`]). +/// +/// It speaks the Transit secrets engine API (`encrypt`, `decrypt` and +/// `rewrap`, always with `batch_input`) over a [`VendorTransport`], so the +/// same code runs on native Rust and in the WASI guest. +/// +/// * `key_ref` (from the keyset record) is the Transit key name. Vault +/// ciphertexts name the key version (`vault:v3:`), so rotation needs no +/// change to the record; [`rewrap`](Self::rewrap) moves a wrapped key to +/// the latest version. The key reference names no version, so +/// [`same_key`](KeyVendorBackend::same_key) is string equality: a record +/// in the wrapped data key mode rewraps only under its own key name. +/// * **Batches.** Under wrapped data keys the adapter sends the keys of a +/// batch in chunks of at most [`VaultConfig::batch_size`]: N values cost +/// one HTTP call per chunk. +/// * **Associated data.** The client always encrypts a bound key +/// ([`stack_kms_vendor::bound`]): the digest of the associated data and +/// the key. For a derived key (`derived=true`) it also sends the +/// associated data as `context`, which Vault requires and which binds the +/// keyset id and descriptor in Vault's own key derivation. The client +/// learns which keys are derived from [`KeyDerivation`]. +/// * Errors map to [`BackendErrorKind`] by [`error_kind`]. A 403 also tells +/// the token provider to drop its token ([`TokenProvider::invalidate`]). +pub struct VaultTransit { + config: VaultConfig, + transport: T, + auth: A, + /// Whether each key is derived, for [`KeyDerivation::Detect`]. + derived: Mutex>, +} + +impl std::fmt::Debug for VaultTransit { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("VaultTransit") + .field("config", &self.config) + .finish_non_exhaustive() + } +} + +impl VaultTransit { + /// A client for `config`, sending through `transport`, with the tokens + /// of `auth`. + pub fn new(config: VaultConfig, transport: T, auth: A) -> Self { + Self { + config, + transport, + auth, + derived: Mutex::new(HashMap::new()), + } + } + + /// The configuration. + pub fn config(&self) -> &VaultConfig { + &self.config + } + + /// The transport. + pub fn transport(&self) -> &T { + &self.transport + } + + /// The token provider. + pub fn auth(&self) -> &A { + &self.auth + } + + /// Send one request to the API path `path` (`transit/keys/my-key`, no + /// `v1/`), with a token, and return the response of a success. + async fn request( + &self, + operation: &'static str, + method: &str, + path: &str, + body: Option>>, + ) -> Result { + let token = self.auth.token().await.map_err(|error| { + let kind = error.kind(); + backend(kind, VaultError::Auth(error)) + })?; + let mut request = VendorRequest::new(method, self.config.url(path)) + .header("x-vault-token", token.token()) + .header("x-vault-request", "true"); + if let Some(namespace) = self.config.namespace() { + request = request.header("x-vault-namespace", namespace); + } + if let Some(body) = body { + request = request.header("content-type", "application/json"); + request.body = body; + } + let response = self.transport.send(request).await.map_err(|source| { + backend( + BackendErrorKind::Unavailable, + VaultError::Transport { operation, source }, + ) + })?; + if classify_status(response.status).is_some() { + if response.status == 403 { + self.auth.invalidate(); + } + return Err(service_error( + operation, + response.status, + failure_message(&response.body), + )); + } + Ok(response) + } + + /// Send one request to the API path `path` (no leading `/`, no `v1/`) + /// and read the JSON of a success. `method` is the HTTP method. Any + /// Vault API can be called this way (for example to enable the Transit + /// engine in a test); [`KeyVendorBackend`] uses `encrypt`, `decrypt` and + /// `keys`. The body is wiped after it is sent. + pub async fn send_json( + &self, + method: &str, + path: &str, + body: Option>>, + ) -> Result { + let response = self.request("request", method, path, body).await?; + serde_json::from_slice(&response.body).map_err(|_| invalid("request", "invalid JSON")) + } + + /// The API path of `operation` on the key `key_ref`. + fn key_path(&self, operation: &str, key_ref: &str) -> Result { + if !is_segment(key_ref) { + return Err(refused( + BackendErrorKind::NotFound, + "the key reference is not a Vault Transit key name", + )); + } + Ok(format!("{}/{operation}/{key_ref}", self.config.mount())) + } + + /// Whether the key `key_ref` is derived: from the config, or read from + /// Vault once per key ([`KeyDerivation::Detect`]). + pub async fn is_derived(&self, key_ref: &str) -> Result { + match self.config.key_derivation() { + KeyDerivation::Derived => return Ok(true), + KeyDerivation::NotDerived => return Ok(false), + KeyDerivation::Detect => {} + } + let path = self.key_path("keys", key_ref)?; + if let Some(derived) = self + .derived + .lock() + .unwrap_or_else(PoisonError::into_inner) + .get(key_ref) + { + return Ok(*derived); + } + let response = self.request("read key", "GET", &path, None).await?; + let read: KeyRead = serde_json::from_slice(&response.body) + .map_err(|_| invalid("read key", "invalid JSON"))?; + let data = read.data.ok_or_else(|| invalid("read key", "no data"))?; + let aead = data + .key_type + .as_deref() + .is_some_and(|key_type| AEAD_KEY_TYPES.contains(&key_type)); + if !aead || !data.supports_encryption || !data.supports_decryption { + return Err(refused( + BackendErrorKind::AccessDenied, + "the Transit key is not an AEAD encryption key \ + (aes256-gcm96, aes128-gcm96 or chacha20-poly1305)", + )); + } + let _ = self + .derived + .lock() + .unwrap_or_else(PoisonError::into_inner) + .insert(key_ref.to_owned(), data.derived); + Ok(data.derived) + } + + /// Send a batch to `operation` on `key_ref` and return each result, in + /// order. A failed item fails the whole batch. + async fn batch( + &self, + operation: &'static str, + key_ref: &str, + items: &[I], + ) -> Result, Error> { + let path = self.key_path(operation, key_ref)?; + let body = Zeroizing::new( + serde_json::to_vec(&BatchRequest { batch_input: items }) + .map_err(|_| invalid(operation, "unserializable request"))?, + ); + let response = self.request(operation, "POST", &path, Some(body)).await?; + let parsed: BatchResponse = serde_json::from_slice(&response.body) + .map_err(|_| invalid(operation, "invalid JSON"))?; + let results = parsed + .data + .ok_or_else(|| invalid(operation, "no data"))? + .batch_results; + if let Some(error) = results + .iter() + .find_map(|result| result.error.as_deref().filter(|e| !e.is_empty())) + { + return Err(service_error(operation, 400, error.to_owned())); + } + if results.len() != items.len() { + return Err(invalid(operation, "another count of batch results")); + } + Ok(results) + } + + /// The `context` of `aad` for a derived key. + fn context(derived: bool, aad: &[u8]) -> Option { + derived.then(|| Base64::encode_string(aad)) + } + + /// Move a wrapped key to the latest version of its Transit key + /// (`rewrap`), without the key leaving Vault. `aad` is the associated + /// data it was wrapped with. Old versions can then be retired with the + /// key's `min_decryption_version`. + pub async fn rewrap( + &self, + key_ref: &str, + wrapped: &[u8], + aad: &[u8], + ) -> Result, Error> { + let derived = self.is_derived(key_ref).await?; + let item = CiphertextItem { + ciphertext: expand(wrapped)?, + context: Self::context(derived, aad), + }; + let mut results = self.batch("rewrap", key_ref, &[item]).await?; + let result = results + .pop() + .ok_or_else(|| invalid("rewrap", "no result"))?; + stored("rewrap", result) + } +} + +/// The Vault ciphertext of a stored wrapped key. +fn expand(wrapped: &[u8]) -> Result { + expand_ciphertext(wrapped).ok_or_else(|| { + refused( + BackendErrorKind::InvalidCiphertext, + "the wrapped key is not a Vault Transit wrapped key", + ) + }) +} + +/// The stored form of a batch result's ciphertext. +fn stored(operation: &'static str, result: BatchResult) -> Result, Error> { + let ciphertext = result + .ciphertext + .ok_or_else(|| invalid(operation, "no ciphertext"))?; + compact_ciphertext(&ciphertext) + .ok_or_else(|| invalid(operation, "the ciphertext is not a Vault ciphertext")) +} + +impl KeyVendorBackend for VaultTransit { + fn vendor(&self) -> KeyVendor { + KeyVendor::Vault + } + + async fn wrap(&self, key_ref: &str, key: &KeyBytes, aad: &[u8]) -> Result, Error> { + let mut wrapped = self.wrap_batch(key_ref, &[WrapItem { key, aad }]).await?; + wrapped.pop().ok_or_else(|| invalid("encrypt", "no result")) + } + + async fn unwrap(&self, key_ref: &str, wrapped: &[u8], aad: &[u8]) -> Result { + let mut keys = self + .unwrap_batch(key_ref, &[UnwrapItem { wrapped, aad }]) + .await?; + keys.pop().ok_or_else(|| invalid("decrypt", "no result")) + } + + fn batch_limit(&self) -> Option { + Some(self.config.batch_size()) + } + + async fn wrap_batch( + &self, + key_ref: &str, + items: &[WrapItem<'_>], + ) -> Result>, Error> { + let derived = self.is_derived(key_ref).await?; + let input: Vec = items + .iter() + .map(|item| { + let bound = bind_key(item.key, item.aad); + EncryptItem { + plaintext: Zeroizing::new(Base64::encode_string(&bound)), + context: Self::context(derived, item.aad), + } + }) + .collect(); + self.batch("encrypt", key_ref, &input) + .await? + .into_iter() + .map(|result| stored("encrypt", result)) + .collect() + } + + async fn unwrap_batch( + &self, + key_ref: &str, + items: &[UnwrapItem<'_>], + ) -> Result, Error> { + let derived = self.is_derived(key_ref).await?; + let input = items + .iter() + .map(|item| { + Ok(CiphertextItem { + ciphertext: expand(item.wrapped)?, + context: Self::context(derived, item.aad), + }) + }) + .collect::, Error>>()?; + self.batch("decrypt", key_ref, &input) + .await? + .into_iter() + .zip(items) + .map(|(result, item)| { + let encoded = Zeroizing::new( + result + .plaintext + .ok_or_else(|| invalid("decrypt", "no plaintext"))?, + ); + let bound = Zeroizing::new( + Base64::decode_vec(&encoded) + .map_err(|_| invalid("decrypt", "the plaintext is not base64"))?, + ); + open_bound_key(&bound, item.aad).map_err(|_| { + refused( + BackendErrorKind::InvalidCiphertext, + "the wrapped key does not belong to this associated data", + ) + }) + }) + .collect() + } + + async fn sleep(&self, duration: Duration) { + self.transport.sleep(duration).await; + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn errors_map_by_status_and_text() { + for (status, message, kind) in [ + (400, "encryption key not found", BackendErrorKind::NotFound), + ( + 400, + "invalid ciphertext: no prefix", + BackendErrorKind::InvalidCiphertext, + ), + ( + 400, + "cipher: message authentication failed", + BackendErrorKind::InvalidCiphertext, + ), + ( + 400, + "invalid ciphertext: could not decode base64", + BackendErrorKind::InvalidCiphertext, + ), + ( + 400, + "invalid ciphertext: version is too new", + BackendErrorKind::InvalidCiphertext, + ), + ( + 400, + "ciphertext version is disallowed by policy (too old)", + BackendErrorKind::InvalidCiphertext, + ), + ( + 400, + "missing 'context' for key derivation", + BackendErrorKind::AccessDenied, + ), + (403, "permission denied", BackendErrorKind::AccessDenied), + (401, "", BackendErrorKind::AccessDenied), + (404, "", BackendErrorKind::NotFound), + ( + 429, + "rate limit quota exceeded", + BackendErrorKind::Throttled, + ), + (500, "", BackendErrorKind::Unavailable), + (503, "Vault is sealed", BackendErrorKind::Unavailable), + (412, "", BackendErrorKind::AccessDenied), + ] { + assert_eq!(error_kind(status, message), kind, "{status} {message}"); + } + } + + #[test] + fn the_message_comes_from_a_failed_item_or_the_errors() { + assert_eq!( + failure_message(br#"{"errors":["a","b"]}"#), + "a; b".to_owned() + ); + assert_eq!( + failure_message( + br#"{"data":{"batch_results":[{"plaintext":"x"},{"error":"cipher: message authentication failed"}]}}"# + ), + "cipher: message authentication failed" + ); + assert_eq!(failure_message(b"not json"), ""); + } + + #[test] + fn ciphertexts_compact_and_expand_losslessly() { + let text = "vault:v12:uN5CE4+rarIuFgLc3Etlpo1eG8jFreK4KUzYvlTMZQ=="; + let compact = compact_ciphertext(text).unwrap(); + assert_eq!(compact[0], WRAPPED_KEY_FORMAT); + assert_eq!(&compact[1..5], &12u32.to_be_bytes()); + assert_eq!(expand_ciphertext(&compact).unwrap(), text); + for bad in [ + "vault:v1:", + "vault:1:AAAA", + "vault:v:AAAA", + "vault:v-1:AAAA", + "vault:v1:!!!!", + "vault:v1:AAA", + "vault:v99999999999:AAAA", + "AAAA", + ] { + assert_eq!(compact_ciphertext(bad), None, "{bad}"); + } + for bad in [ + &[][..], + &[2, 0, 0, 0, 1, 9], + &[1, 0, 0, 1], + &[1, 0, 0, 0, 1], + ] { + assert_eq!(expand_ciphertext(bad), None, "{bad:?}"); + } + } +} diff --git a/packages/stack-kms-vault/src/config.rs b/packages/stack-kms-vault/src/config.rs new file mode 100644 index 0000000000..f988e6e1f4 --- /dev/null +++ b/packages/stack-kms-vault/src/config.rs @@ -0,0 +1,299 @@ +//! The address, namespace, mount and batch size of a Vault Transit client. + +use std::num::NonZeroUsize; + +/// How an environment variable is read: the process environment unless a +/// test gives another. +pub type EnvLookup = std::sync::Arc Option + Send + Sync>; + +/// The process environment. An empty variable counts as unset. +pub fn process_env() -> EnvLookup { + std::sync::Arc::new(|name| std::env::var(name).ok().filter(|value| !value.is_empty())) +} + +/// The default mount path of the Transit secrets engine. +pub const DEFAULT_MOUNT: &str = "transit"; + +/// The default batch size: the most keys in one `batch_input`. +pub const DEFAULT_BATCH_SIZE: usize = 250; + +/// The configuration is not usable. +#[derive(Debug, thiserror::Error, PartialEq, Eq)] +pub enum ConfigError { + /// No address was given, and `VAULT_ADDR` is not set. + #[error("no Vault address: set VAULT_ADDR or give one in the config")] + NoAddress, + /// The address is not an absolute HTTP or HTTPS URL with no query. + #[error("{0:?} is not an HTTP or HTTPS Vault address")] + InvalidAddress(String), + /// The mount path is not a Vault path. + #[error("{0:?} is not a Vault mount path")] + InvalidMount(String), + /// The namespace is not a Vault namespace path. + #[error("{0:?} is not a Vault namespace")] + InvalidNamespace(String), +} + +/// Whether a Transit key derives a key per request from `context` +/// (`derived=true`). +/// +/// Vault requires `context` for a derived key and silently ignores it for +/// a key that is not derived, so the client must know which. Either way the +/// client also binds the associated data itself +/// ([`stack_kms_vendor::bound`]), so a wrong answer can make a call fail but +/// never makes a wrapped key open under other associated data. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub enum KeyDerivation { + /// Read the key's configuration (`GET /v1/{mount}/keys/{name}`) on the + /// first use of each key, and keep the answer. The token needs `read` on + /// that path. This also refuses a key that does not exist (Transit's + /// `encrypt` would otherwise create it) and a key type that is not an + /// AEAD. The default. + #[default] + Detect, + /// Every key is derived: send the associated data as `context`. No key + /// read. + Derived, + /// No key is derived: send no `context`. No key read. + NotDerived, +} + +/// A path of one or more segments of `[A-Za-z0-9._-]`, none `.` or `..`. +pub(crate) fn is_path(text: &str) -> bool { + !text.is_empty() && text.split('/').all(is_segment) +} + +/// One path segment of `[A-Za-z0-9._-]`, not `.` or `..`. +pub(crate) fn is_segment(text: &str) -> bool { + !text.is_empty() + && text != "." + && text != ".." + && text + .bytes() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'_' | b'-')) +} + +/// The address, namespace, mount path and batch size of a +/// [`VaultTransit`](crate::VaultTransit) client. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct VaultConfig { + address: String, + namespace: Option, + mount: String, + batch_size: NonZeroUsize, + derivation: KeyDerivation, +} + +impl VaultConfig { + /// The Vault server at `address` (`https://vault.example.com:8200`), + /// with the Transit engine at [`DEFAULT_MOUNT`], no namespace and + /// [`DEFAULT_BATCH_SIZE`]. + pub fn new(address: impl Into) -> Result { + let address = address.into(); + let parsed = + url::Url::parse(&address).map_err(|_| ConfigError::InvalidAddress(address.clone()))?; + if !matches!(parsed.scheme(), "http" | "https") + || parsed.host().is_none() + || parsed.query().is_some() + || parsed.fragment().is_some() + || !parsed.username().is_empty() + || parsed.password().is_some() + { + return Err(ConfigError::InvalidAddress(address)); + } + let address = parsed.as_str().trim_end_matches('/').to_owned(); + Ok(Self { + address, + namespace: None, + mount: DEFAULT_MOUNT.to_owned(), + batch_size: NonZeroUsize::new(DEFAULT_BATCH_SIZE).unwrap_or(NonZeroUsize::MIN), + derivation: KeyDerivation::Detect, + }) + } + + /// The address from `VAULT_ADDR`, the namespace from `VAULT_NAMESPACE`, + /// and the mount path from `VAULT_TRANSIT_MOUNT` (default + /// [`DEFAULT_MOUNT`]). + pub fn from_env() -> Result { + Self::from_env_lookup(&process_env()) + } + + /// As [`from_env`](Self::from_env), reading the variables with `env`. + pub fn from_env_lookup(env: &EnvLookup) -> Result { + let mut config = Self::new(env("VAULT_ADDR").ok_or(ConfigError::NoAddress)?)?; + if let Some(namespace) = env("VAULT_NAMESPACE") { + config = config.with_namespace(namespace)?; + } + if let Some(mount) = env("VAULT_TRANSIT_MOUNT") { + config = config.with_mount(mount)?; + } + Ok(config) + } + + /// Send every request in `namespace` (Vault Enterprise and HCP Vault), + /// as the `X-Vault-Namespace` header. A leading or trailing `/` is + /// removed. + pub fn with_namespace(mut self, namespace: impl Into) -> Result { + let namespace = namespace.into(); + let trimmed = namespace.trim_matches('/'); + if !is_path(trimmed) { + return Err(ConfigError::InvalidNamespace(namespace)); + } + self.namespace = Some(trimmed.to_owned()); + Ok(self) + } + + /// Use the Transit engine mounted at `mount` (`transit`, + /// `encryption/transit`). A leading or trailing `/` is removed. + pub fn with_mount(mut self, mount: impl Into) -> Result { + let mount = mount.into(); + let trimmed = mount.trim_matches('/'); + if !is_path(trimmed) { + return Err(ConfigError::InvalidMount(mount)); + } + self.mount = trimmed.to_owned(); + Ok(self) + } + + /// Send at most `batch_size` keys in one `batch_input`. + pub fn with_batch_size(mut self, batch_size: NonZeroUsize) -> Self { + self.batch_size = batch_size; + self + } + + /// How the client learns whether a key is derived. Default + /// [`KeyDerivation::Detect`]. + pub fn with_key_derivation(mut self, derivation: KeyDerivation) -> Self { + self.derivation = derivation; + self + } + + /// The address, with no trailing `/`. + pub fn address(&self) -> &str { + &self.address + } + + /// The namespace, if any. + pub fn namespace(&self) -> Option<&str> { + self.namespace.as_deref() + } + + /// The mount path of the Transit engine. + pub fn mount(&self) -> &str { + &self.mount + } + + /// The most keys in one `batch_input`. + pub fn batch_size(&self) -> NonZeroUsize { + self.batch_size + } + + /// How the client learns whether a key is derived. + pub fn key_derivation(&self) -> KeyDerivation { + self.derivation + } + + /// The URL of the API path `path` (no leading `/`, no `v1/`). + pub fn url(&self, path: &str) -> String { + format!("{}/v1/{path}", self.address) + } +} + +#[cfg(test)] +mod tests { + use std::collections::HashMap; + use std::sync::Arc; + + use super::*; + + fn env(pairs: &[(&str, &str)]) -> EnvLookup { + let map: HashMap = pairs + .iter() + .map(|(k, v)| ((*k).to_owned(), (*v).to_owned())) + .collect(); + Arc::new(move |name| map.get(name).cloned()) + } + + #[test] + fn the_address_is_normalised_and_checked() { + let config = VaultConfig::new("https://vault.example.com:8200/").unwrap(); + assert_eq!(config.address(), "https://vault.example.com:8200"); + assert_eq!( + config.url("transit/encrypt/k"), + "https://vault.example.com:8200/v1/transit/encrypt/k" + ); + assert_eq!(config.mount(), "transit"); + assert_eq!(config.namespace(), None); + assert_eq!(config.batch_size().get(), DEFAULT_BATCH_SIZE); + assert_eq!(config.key_derivation(), KeyDerivation::Detect); + for bad in [ + "vault:8200", + "ftp://vault/", + "https://vault/?x=1", + "https://vault/#x", + "https://user:pass@vault/", + "not a url", + ] { + assert_eq!( + VaultConfig::new(bad), + Err(ConfigError::InvalidAddress(bad.to_owned())), + "{bad}" + ); + } + } + + #[test] + fn mounts_and_namespaces_are_paths() { + let config = VaultConfig::new("http://127.0.0.1:8200").unwrap(); + let config = config + .with_mount("/encryption/transit/") + .unwrap() + .with_namespace("admin/team-a/") + .unwrap(); + assert_eq!(config.mount(), "encryption/transit"); + assert_eq!(config.namespace(), Some("admin/team-a")); + for bad in ["", "a//b", "../sys", "a/./b", "a b", "a?b", "a\nb", "a%2fb"] { + assert_eq!( + config.clone().with_mount(bad), + Err(ConfigError::InvalidMount(bad.to_owned())), + "{bad:?}" + ); + assert_eq!( + config.clone().with_namespace(bad), + Err(ConfigError::InvalidNamespace(bad.to_owned())), + "{bad:?}" + ); + } + } + + #[test] + fn the_config_comes_from_the_environment() { + assert_eq!( + VaultConfig::from_env_lookup(&env(&[])), + Err(ConfigError::NoAddress) + ); + let config = VaultConfig::from_env_lookup(&env(&[ + ("VAULT_ADDR", "https://vault:8200"), + ("VAULT_NAMESPACE", "admin"), + ("VAULT_TRANSIT_MOUNT", "kms"), + ])) + .unwrap(); + assert_eq!(config.address(), "https://vault:8200"); + assert_eq!(config.namespace(), Some("admin")); + assert_eq!(config.mount(), "kms"); + assert_eq!( + VaultConfig::from_env_lookup(&env(&[("VAULT_ADDR", "vault")])), + Err(ConfigError::InvalidAddress("vault".to_owned())) + ); + } + + #[test] + fn the_batch_size_and_derivation_are_set() { + let config = VaultConfig::new("http://127.0.0.1:8200") + .unwrap() + .with_batch_size(NonZeroUsize::new(7).unwrap()) + .with_key_derivation(KeyDerivation::Derived); + assert_eq!(config.batch_size().get(), 7); + assert_eq!(config.key_derivation(), KeyDerivation::Derived); + } +} diff --git a/packages/stack-kms-vault/src/lib.rs b/packages/stack-kms-vault/src/lib.rs new file mode 100644 index 0000000000..a923327679 --- /dev/null +++ b/packages/stack-kms-vault/src/lib.rs @@ -0,0 +1,138 @@ +#![doc(html_favicon_url = "https://cipherstash.com/favicon.ico")] +//! The HashiCorp Vault Transit key vendor backend for Stack Encrypt +//! (ADR-0009, ADR-0011). +//! +//! [`VaultTransit`] is a small, hand-written client for the Transit secrets +//! engine: `encrypt`, `decrypt` and `rewrap`, each with `batch_input`, and +//! a key read. It sends through a +//! [`VendorTransport`](stack_kms_vendor::VendorTransport) and implements +//! [`KeyVendorBackend`](stack_kms_vendor::KeyVendorBackend), so +//! [`VendorKeySource`](stack_kms_vendor::VendorKeySource) builds both key +//! modes on it. No Vault SDK crate is used, so this crate also builds for +//! wasm32-wasip1, the Go module's WASI guest. +//! +//! Under wrapped data keys, Transit's batch operations make Vault the +//! cheapest key vendor backend: the wrapped data keys of N values cost one +//! HTTP call for each [`VaultConfig::batch_size`] values. +//! +//! # Native Rust +//! +//! ```no_run +//! # async fn example() -> Result<(), Box> { +//! use stack_kms_vault::VaultTransit; +//! use stack_kms_vendor::{KeysetRecord, VendorKeySource}; +//! use stack_kms::KeyMode; +//! +//! // VAULT_ADDR, VAULT_NAMESPACE, and a token from VAULT_TOKEN, AppRole +//! // or Kubernetes auth. +//! let vault = VaultTransit::from_env()?; +//! let record = KeysetRecord::create( +//! &vault, +//! "cipherstash-users", +//! KeyMode::Wrapped, +//! Some("users".to_owned()), +//! ) +//! .await?; +//! let source = VendorKeySource::new(vault, [record])?; +//! # let _ = source; +//! # Ok(()) +//! # } +//! ``` +//! +//! # Capabilities and limits +//! +//! The [capability matrix](stack_kms_vendor#capability-matrix) of +//! `stack-kms-vendor` compares this backend with ZeroKMS and the other +//! key vendor backends: key lifetime, vendor calls, what the vendor sees +//! and can refuse, credentials, rotation and known limits. +//! +//! # Vault policy +//! +//! The token needs `update` on `/encrypt/` and +//! `/decrypt/`, and (with the default +//! [`KeyDerivation::Detect`]) `read` on `/keys/`. Give it +//! `update` on `/rewrap/` to use [`VaultTransit::rewrap`]. Do +//! not give it `create` on `/encrypt/`: with it, Transit +//! creates a missing key on the first `encrypt`. +//! +//! # The WASI guest +//! +//! The guest builds [`VaultTransit::new`] with its host transport and a +//! [`TokenProvider`] over a host import. Kubernetes auth reads a file and is +//! native only: in the guest, the host's own auth supplies the token. + +#![deny(unsafe_code)] +#![warn(clippy::unwrap_used)] +#![warn(clippy::expect_used)] +#![warn(clippy::panic)] +#![warn(clippy::mem_forget)] +#![warn(clippy::print_stdout)] +#![warn(clippy::print_stderr)] +#![warn(clippy::dbg_macro)] +#![warn(unreachable_pub)] +#![warn(unused_results)] +#![cfg_attr(test, allow(clippy::unwrap_used))] +#![cfg_attr(test, allow(clippy::expect_used))] +#![cfg_attr(test, allow(clippy::panic))] +#![cfg_attr(test, allow(unused_results))] + +pub mod auth; +mod client; +mod config; +mod time; + +pub use auth::{ + AppRoleLogin, AuthError, CachedToken, TokenProvider, VaultToken, DEFAULT_APPROLE_MOUNT, + DEFAULT_REFRESH_BEFORE_EXPIRY, +}; +#[cfg(not(target_arch = "wasm32"))] +pub use auth::{DefaultAuth, KubernetesLogin}; +pub use client::{ + compact_ciphertext, error_kind, expand_ciphertext, VaultError, VaultTransit, AEAD_KEY_TYPES, + INVALID_CIPHERTEXT_MARKERS, NOT_FOUND_MARKERS, WRAPPED_KEY_FORMAT, +}; +pub use config::{ + process_env, ConfigError, EnvLookup, KeyDerivation, VaultConfig, DEFAULT_BATCH_SIZE, + DEFAULT_MOUNT, +}; +pub use time::{system_clock, Clock}; + +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +mod native { + use stack_kms_vendor::HttpVendorTransport; + + use crate::auth::DefaultAuth; + use crate::{VaultConfig, VaultTransit}; + + /// [`VaultTransit::from_env`] failed. + #[derive(Debug, thiserror::Error)] + pub enum FromEnvError { + /// The address, namespace or mount. + #[error(transparent)] + Config(#[from] crate::ConfigError), + /// The auth method. + #[error(transparent)] + Auth(#[from] crate::AuthError), + /// The HTTP client. + #[error(transparent)] + Transport(#[from] stack_kms_vendor::TransportError), + } + + /// The client [`VaultTransit::from_env`] builds. + pub type DefaultVaultTransit = + VaultTransit>; + + impl DefaultVaultTransit { + /// A client configured from the environment: the address, namespace + /// and mount from [`VaultConfig::from_env`], and the auth method + /// from [`DefaultAuth::from_env`]. Native only. + pub fn from_env() -> Result { + let config = VaultConfig::from_env()?; + let auth = DefaultAuth::from_env(&config, HttpVendorTransport::new()?)?; + Ok(VaultTransit::new(config, HttpVendorTransport::new()?, auth)) + } + } +} + +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +pub use native::{DefaultVaultTransit, FromEnvError}; diff --git a/packages/stack-kms-vault/src/time.rs b/packages/stack-kms-vault/src/time.rs new file mode 100644 index 0000000000..7487a4a511 --- /dev/null +++ b/packages/stack-kms-vault/src/time.rs @@ -0,0 +1,12 @@ +//! The time source of a client. + +use web_time::SystemTime; + +/// The time source of a token cache: `SystemTime::now` unless a test sets +/// another. +pub type Clock = std::sync::Arc SystemTime + Send + Sync>; + +/// The system clock. +pub fn system_clock() -> Clock { + std::sync::Arc::new(SystemTime::now) +} diff --git a/packages/stack-kms-vault/tasks.toml b/packages/stack-kms-vault/tasks.toml new file mode 100644 index 0000000000..9fda7a4545 --- /dev/null +++ b/packages/stack-kms-vault/tasks.toml @@ -0,0 +1,25 @@ +# Rustdoc with warnings as errors: a broken intra-doc link or a rustdoc +# warning fails the build. The root `doc` task fans out over every +# `doc:`; the root `test:doc` runs the doc examples. +["doc:stack-kms-vault"] +description = "Build docs for stack-kms-vault with all features (warnings are errors)" +env = { RUSTDOCFLAGS = "-D warnings" } +run = "cargo doc -p stack-kms-vault --no-deps --all-features" + +# The Vault Transit backend against a running `vault server -dev` +# (tests/vault_dev.rs). The test skips itself when STACK_KMS_VAULT_ADDR is +# unset, which is right for `cargo nextest run --workspace` and wrong here: a +# skip would read as a pass. So this task refuses to run without it. +# tests-crates.yml's `vault` job runs Vault as a service and calls this task. +# +# docker run -d --rm -p 8200:8200 -e VAULT_DEV_ROOT_TOKEN_ID=root hashicorp/vault:1.20 +# STACK_KMS_VAULT_ADDR=http://127.0.0.1:8200 mise run test:vault:stack-kms-vault +["test:vault:stack-kms-vault"] +description = "Run the stack-kms-vault tests against vault server -dev (needs STACK_KMS_VAULT_ADDR)" +run = """ +if [ -z "${STACK_KMS_VAULT_ADDR:-}" ]; then + echo "error: set STACK_KMS_VAULT_ADDR (for example http://127.0.0.1:8200)" >&2 + exit 1 +fi +mise x --env test -- cargo nextest run --locked -p stack-kms-vault --all-features --test vault_dev --no-tests=fail +""" diff --git a/packages/stack-kms-vault/tests/common/mod.rs b/packages/stack-kms-vault/tests/common/mod.rs new file mode 100644 index 0000000000..63b5852ad8 --- /dev/null +++ b/packages/stack-kms-vault/tests/common/mod.rs @@ -0,0 +1,498 @@ +//! Shared test support: recorded fixtures, a replaying transport, a +//! recording transport, and an in-process emulation of the Vault Transit +//! API. + +#![allow(dead_code, clippy::unwrap_used, clippy::expect_used, clippy::panic)] + +use std::collections::{HashMap, VecDeque}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use aes_gcm::aead::{Aead, KeyInit}; +use aes_gcm::{Aes256Gcm, Nonce}; +use base64ct::{Base64, Encoding}; +use serde::{Deserialize, Serialize}; +use serde_json::{json, Value}; +use sha2::{Digest, Sha256}; +use stack_kms_vault::{VaultConfig, VaultToken, VaultTransit}; +use stack_kms_vendor::{TransportError, VendorRequest, VendorResponse, VendorTransport}; + +/// The address of every fixture: a local `vault server -dev`. +pub const ADDRESS: &str = "http://127.0.0.1:8200"; +/// The dev root token of every fixture. +pub const TOKEN: &str = "fixture-root-token"; +/// The non-derived Transit key of the fixtures. +pub const KEY: &str = "cipherstash-fixture"; +/// The derived Transit key of the fixtures. +pub const DERIVED_KEY: &str = "cipherstash-fixture-derived"; + +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct FixtureRequest { + pub method: String, + pub url: String, + pub headers: Vec<(String, String)>, + pub body: Value, +} + +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct FixtureResponse { + pub status: u16, + #[serde(default)] + pub headers: Vec<(String, String)>, + pub body: Value, +} + +impl FixtureResponse { + pub fn to_response(&self) -> VendorResponse { + let body = match &self.body { + Value::String(text) => text.clone().into_bytes(), + Value::Null => Vec::new(), + other => serde_json::to_vec(other).unwrap(), + }; + let mut response = VendorResponse::new(self.status, body); + for (name, value) in &self.headers { + response = response.header(name, value); + } + response + } +} + +#[derive(Clone, Debug, Deserialize, Serialize)] +pub struct Fixture { + pub request: FixtureRequest, + pub response: FixtureResponse, +} + +pub fn fixture_path(name: &str) -> String { + format!("{}/tests/fixtures/{name}", env!("CARGO_MANIFEST_DIR")) +} + +pub fn load Deserialize<'de>>(name: &str) -> T { + let path = fixture_path(name); + let text = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}")); + serde_json::from_str(&text).unwrap_or_else(|e| panic!("{path}: {e}")) +} + +fn body_json(body: &[u8]) -> Value { + if body.is_empty() { + Value::Null + } else { + serde_json::from_slice(body) + .unwrap_or_else(|_| Value::String(String::from_utf8_lossy(body).into())) + } +} + +/// A transport that checks each request against a recorded one and +/// answers with the recorded response. +#[derive(Default)] +pub struct Replay { + exchanges: Mutex, VendorResponse)>>, +} + +impl Replay { + pub fn new() -> Self { + Self::default() + } + + /// Expect `fixture`'s request and answer with its response. + pub fn expect(self, fixture: Fixture) -> Self { + let response = fixture.response.to_response(); + self.exchanges + .lock() + .unwrap() + .push_back((Some(fixture.request), response)); + self + } + + /// Expect each exchange of a recorded sequence, in order. + pub fn expect_all(mut self, fixtures: Vec) -> Self { + for fixture in fixtures { + self = self.expect(fixture); + } + self + } + + /// Answer the next request, whatever it is, with `response`. + pub fn answer(self, response: VendorResponse) -> Self { + self.exchanges.lock().unwrap().push_back((None, response)); + self + } + + pub fn is_done(&self) -> bool { + self.exchanges.lock().unwrap().is_empty() + } +} + +impl VendorTransport for Replay { + async fn send(&self, request: VendorRequest) -> Result { + let (expected, response) = self + .exchanges + .lock() + .unwrap() + .pop_front() + .expect("a request the fixtures do not expect"); + if let Some(expected) = expected { + let actual = json!({ + "method": request.method, + "url": request.url, + "headers": request.headers, + "body": body_json(&request.body), + }); + let wanted = json!({ + "method": expected.method, + "url": expected.url, + "headers": expected.headers, + "body": expected.body, + }); + assert_eq!( + actual, + wanted, + "the request differs from the fixture; actual:\n{}", + serde_json::to_string_pretty(&actual).unwrap() + ); + } + Ok(response) + } + + async fn sleep(&self, _duration: Duration) {} +} + +/// A transport that sends through `inner` and keeps every exchange, to +/// write fixtures. +pub struct Recording { + pub inner: T, + pub exchanges: Mutex>, +} + +impl Recording { + pub fn new(inner: T) -> Self { + Self { + inner, + exchanges: Mutex::new(Vec::new()), + } + } + + /// The exchanges so far, and forget them. + pub fn take(&self) -> Vec { + std::mem::take(&mut *self.exchanges.lock().unwrap()) + } +} + +impl VendorTransport for Recording { + async fn send(&self, request: VendorRequest) -> Result { + let recorded = FixtureRequest { + method: request.method.clone(), + url: request.url.clone(), + headers: request.headers.clone(), + body: body_json(&request.body), + }; + let response = self.inner.send(request).await?; + let headers = response + .header_value("content-type") + .map(|value| vec![("content-type".to_owned(), value.to_owned())]) + .unwrap_or_default(); + self.exchanges.lock().unwrap().push(Fixture { + request: recorded, + response: FixtureResponse { + status: response.status, + headers, + body: body_json(&response.body), + }, + }); + Ok(response) + } + + async fn sleep(&self, duration: Duration) { + self.inner.sleep(duration).await; + } +} + +/// A client over `transport` with the fixture address and token. +pub fn fixture_client( + transport: T, +) -> VaultTransit { + VaultTransit::new( + VaultConfig::new(ADDRESS).unwrap(), + transport, + VaultToken::new(TOKEN), + ) +} + +#[derive(Clone)] +struct EmulatedKey { + key: [u8; 32], + derived: bool, + key_type: &'static str, +} + +/// An in-process emulation of the Vault Transit API: key reads, `encrypt`, +/// `decrypt` and `rewrap` with `batch_input`, derived keys (which require +/// `context`, while other keys ignore it, as Vault does), AppRole and +/// Kubernetes logins, and Vault's error shapes. It checks the token and +/// the namespace on every request. +/// +/// The contract suite uses random keys and ids, so recorded bytes cannot +/// serve it: this answers it instead. Clones share one state. +#[derive(Clone, Default)] +pub struct TransitEmulator { + keys: Arc>>, + tokens: Arc>>, + logins: Arc>, + login_lease: Arc>, + namespace: Arc>>, + requests: Arc>>, + sleeps: Arc>>, +} + +impl TransitEmulator { + /// An emulator holding the non-derived [`KEY`] and the derived + /// [`DERIVED_KEY`], that accepts [`TOKEN`]. + pub fn new() -> Self { + let emulator = Self::default(); + emulator.add_key(KEY, false, "aes256-gcm96"); + emulator.add_key(DERIVED_KEY, true, "aes256-gcm96"); + emulator.tokens.lock().unwrap().push(TOKEN.to_owned()); + *emulator.login_lease.lock().unwrap() = 3600; + emulator + } + + pub fn add_key(&self, name: &str, derived: bool, key_type: &'static str) { + let mut key = [0u8; 32]; + for (i, byte) in key.iter_mut().enumerate() { + *byte = (i as u8) ^ name.len() as u8 ^ u8::from(derived); + } + self.keys.lock().unwrap().insert( + name.to_owned(), + EmulatedKey { + key, + derived, + key_type, + }, + ); + } + + /// Require the `X-Vault-Namespace` header `namespace`. + pub fn with_namespace(self, namespace: &str) -> Self { + *self.namespace.lock().unwrap() = Some(namespace.to_owned()); + self + } + + /// Revoke every token. + pub fn revoke_tokens(&self) { + self.tokens.lock().unwrap().clear(); + } + + pub fn logins(&self) -> usize { + *self.logins.lock().unwrap() + } + + /// Every request: method, path after `/v1/`, body. + pub fn requests(&self) -> Vec<(String, String, Value)> { + self.requests.lock().unwrap().clone() + } + + pub fn sleeps(&self) -> Vec { + self.sleeps.lock().unwrap().clone() + } + + fn error(status: u16, message: &str) -> VendorResponse { + VendorResponse::new( + status, + json!({ "errors": [message] }).to_string().into_bytes(), + ) + .header("content-type", "application/json") + } + + fn ok(data: Value) -> VendorResponse { + VendorResponse::new( + 200, + json!({ "request_id": "emulated", "data": data }) + .to_string() + .into_bytes(), + ) + .header("content-type", "application/json") + } + + fn cipher(key: &EmulatedKey, context: Option<&str>) -> Result { + let key_bytes = if key.derived { + let context = context.filter(|c| !c.is_empty()).ok_or( + "missing 'context' for key derivation; the key was created using a derived key", + )?; + let mut hasher = Sha256::new(); + hasher.update(key.key); + hasher.update( + Base64::decode_vec(context).map_err(|_| "failed to base64-decode context")?, + ); + hasher.finalize().into() + } else { + key.key + }; + Ok(Aes256Gcm::new_from_slice(&key_bytes).unwrap()) + } + + fn encrypt(key: &EmulatedKey, item: &Value, counter: &mut u8) -> Value { + let cipher = match Self::cipher(key, item["context"].as_str()) { + Ok(cipher) => cipher, + Err(error) => return json!({ "error": error }), + }; + let Ok(plaintext) = Base64::decode_vec(item["plaintext"].as_str().unwrap_or("")) else { + return json!({ "error": "failed to base64-decode plaintext" }); + }; + *counter = counter.wrapping_add(1); + let nonce = [*counter; 12]; + let mut out = nonce.to_vec(); + out.extend( + cipher + .encrypt(Nonce::from_slice(&nonce), plaintext.as_slice()) + .unwrap(), + ); + json!({ "ciphertext": format!("vault:v1:{}", Base64::encode_string(&out)), "key_version": 1 }) + } + + fn decrypt(key: &EmulatedKey, item: &Value) -> Result, String> { + let cipher = Self::cipher(key, item["context"].as_str())?; + let text = item["ciphertext"].as_str().unwrap_or(""); + let encoded = text + .strip_prefix("vault:v1:") + .ok_or("invalid ciphertext: no prefix")?; + let bytes = Base64::decode_vec(encoded) + .map_err(|_| "invalid ciphertext: could not decode base64")?; + if bytes.len() < 12 { + return Err("invalid ciphertext length".into()); + } + let (nonce, ct) = bytes.split_at(12); + cipher + .decrypt(Nonce::from_slice(nonce), ct) + .map_err(|_| "cipher: message authentication failed".into()) + } + + fn login(&self, path: &str, body: &Value) -> VendorResponse { + let ok = match path { + "auth/approle/login" => { + body["role_id"] == "fixture-role-id" && body["secret_id"] == "fixture-secret-id" + } + "auth/kubernetes/login" => { + body["role"] == "cipherstash" && body["jwt"] == "fixture.jwt.token" + } + _ => return Self::error(404, "no handler for route"), + }; + if !ok { + return Self::error(400, "invalid role or secret ID"); + } + let mut logins = self.logins.lock().unwrap(); + *logins += 1; + let token = format!("hvs.emulated-{logins}"); + self.tokens.lock().unwrap().push(token.clone()); + VendorResponse::new( + 200, + json!({ + "auth": { + "client_token": token, + "lease_duration": *self.login_lease.lock().unwrap(), + "renewable": true + } + }) + .to_string() + .into_bytes(), + ) + } +} + +impl VendorTransport for TransitEmulator { + async fn send(&self, request: VendorRequest) -> Result { + let path = request + .url + .split_once("/v1/") + .map(|(_, path)| path.to_owned()) + .expect("a /v1/ URL"); + let body = body_json(&request.body); + self.requests + .lock() + .unwrap() + .push((request.method.clone(), path.clone(), body.clone())); + let namespace = self.namespace.lock().unwrap().clone(); + assert_eq!( + request.header_value("x-vault-namespace").map(str::to_owned), + namespace, + "the namespace header" + ); + assert_eq!(request.header_value("x-vault-request"), Some("true")); + if path.starts_with("auth/") { + return Ok(self.login(&path, &body)); + } + let token = request + .header_value("x-vault-token") + .unwrap_or("") + .to_owned(); + if !self.tokens.lock().unwrap().contains(&token) { + return Ok(Self::error(403, "permission denied")); + } + let mut parts = path.splitn(3, '/'); + let (Some("transit"), Some(operation), Some(name)) = + (parts.next(), parts.next(), parts.next()) + else { + return Ok(Self::error(404, "no handler for route")); + }; + let key = self.keys.lock().unwrap().get(name).cloned(); + if operation == "keys" && request.method == "GET" { + let Some(key) = key else { + return Ok(Self::error(404, "")); + }; + return Ok(Self::ok(json!({ + "name": name, + "derived": key.derived, + "type": key.key_type, + "supports_encryption": true, + "supports_decryption": true, + "latest_version": 1 + }))); + } + let Some(key) = key else { + return Ok(Self::error(400, "encryption key not found")); + }; + let items = body["batch_input"].as_array().cloned().unwrap_or_default(); + let has_context = items.iter().filter(|i| i.get("context").is_some()).count(); + if has_context != 0 && has_context != items.len() { + return Ok(Self::error( + 400, + "context should be set either in all the request blocks or in none", + )); + } + let mut counter = 0u8; + let results: Vec = match operation { + "encrypt" => items + .iter() + .map(|i| Self::encrypt(&key, i, &mut counter)) + .collect(), + "decrypt" => items + .iter() + .map(|i| match Self::decrypt(&key, i) { + Ok(plaintext) => json!({ "plaintext": Base64::encode_string(&plaintext) }), + Err(error) => json!({ "plaintext": "", "error": error }), + }) + .collect(), + "rewrap" => items + .iter() + .map(|i| match Self::decrypt(&key, i) { + Ok(plaintext) => { + let mut item = i.clone(); + item["plaintext"] = Value::String(Base64::encode_string(&plaintext)); + Self::encrypt(&key, &item, &mut counter) + } + Err(error) => json!({ "error": error }), + }) + .collect(), + _ => return Ok(Self::error(404, "no handler for route")), + }; + let failed = results.iter().any(|r| r.get("error").is_some()); + let mut response = Self::ok(json!({ "batch_results": results })); + if failed { + response.status = 400; + } + Ok(response) + } + + async fn sleep(&self, duration: Duration) { + self.sleeps.lock().unwrap().push(duration); + } +} diff --git a/packages/stack-kms-vault/tests/contract.rs b/packages/stack-kms-vault/tests/contract.rs new file mode 100644 index 0000000000..8c7a28a4bc --- /dev/null +++ b/packages/stack-kms-vault/tests/contract.rs @@ -0,0 +1,243 @@ +//! The shared contract suite, run against the Vault Transit backend over an +//! in-process emulation of the Transit API, in both key modes, for a +//! derived and a non-derived key. +//! +//! The suite generates random keys and keyset ids, so recorded exchanges +//! cannot answer it; the emulator does (see `common::TransitEmulator`). The +//! request shapes are pinned by `tests/fixtures.rs`, and a real Vault is +//! exercised by `tests/vault_dev.rs`. + +mod common; + +use std::num::NonZeroUsize; + +use common::{fixture_client, TransitEmulator, ADDRESS, DERIVED_KEY, KEY, TOKEN}; +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor}; +use stack_kms_vault::{KeyDerivation, VaultConfig, VaultToken, VaultTransit}; +use stack_kms_vendor::derive::wrapped_data_key_aad; +use stack_kms_vendor::{contract, KeyVendorBackend}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; + +fn client_with( + emulator: &TransitEmulator, + derivation: KeyDerivation, +) -> VaultTransit { + VaultTransit::new( + VaultConfig::new(ADDRESS) + .unwrap() + .with_batch_size(NonZeroUsize::new(4).unwrap()) + .with_key_derivation(derivation), + emulator.clone(), + VaultToken::new(TOKEN), + ) +} + +fn kind_of(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "vault-transit"); + kind + } + other => panic!("not a backend error: {other:?}"), + } +} + +#[tokio::test] +async fn vault_transit_meets_the_contract_with_a_non_derived_key() { + let emulator = TransitEmulator::new(); + contract::run(|| client_with(&emulator, KeyDerivation::Detect), KEY).await; +} + +#[tokio::test] +async fn vault_transit_meets_the_contract_with_a_derived_key() { + let emulator = TransitEmulator::new(); + contract::run( + || client_with(&emulator, KeyDerivation::Detect), + DERIVED_KEY, + ) + .await; +} + +#[tokio::test] +async fn vault_transit_meets_the_contract_with_the_derivation_configured() { + let emulator = TransitEmulator::new(); + contract::run( + || client_with(&emulator, KeyDerivation::Derived), + DERIVED_KEY, + ) + .await; + contract::run(|| client_with(&emulator, KeyDerivation::NotDerived), KEY).await; +} + +#[tokio::test] +async fn a_wrapped_key_opens_only_under_its_own_aad() { + let emulator = TransitEmulator::new(); + // A non-derived key configured as derived: Vault ignores the context, + // so only the bound key refuses other associated data. + for (key, derivation) in [ + (KEY, KeyDerivation::Detect), + (DERIVED_KEY, KeyDerivation::Detect), + (KEY, KeyDerivation::Derived), + ] { + let vault = client_with(&emulator, derivation); + let id = Uuid::from_u128(1); + let aad = wrapped_data_key_aad(id, &[1u8; 16], "users/email"); + let wrapped = vault + .wrap(key, &Protected::new([9u8; 32]), &aad) + .await + .unwrap(); + assert_eq!( + vault.unwrap(key, &wrapped, &aad).await.unwrap().risky_ref(), + &[9u8; 32] + ); + let other = wrapped_data_key_aad(id, &[1u8; 16], "users/name"); + let error = vault.unwrap(key, &wrapped, &other).await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext, "{key}"); + } +} + +#[tokio::test] +async fn a_derived_key_gets_the_aad_as_context_and_another_key_gets_none() { + let emulator = TransitEmulator::new(); + let vault = client_with(&emulator, KeyDerivation::Detect); + let aad = b"the associated data"; + let _ = vault + .wrap(DERIVED_KEY, &Protected::new([1u8; 32]), aad) + .await + .unwrap(); + let _ = vault + .wrap(KEY, &Protected::new([1u8; 32]), aad) + .await + .unwrap(); + let requests = emulator.requests(); + let encrypts: Vec<_> = requests + .iter() + .filter(|(_, path, _)| path.starts_with("transit/encrypt/")) + .collect(); + assert_eq!(encrypts.len(), 2); + assert_eq!( + encrypts[0].2["batch_input"][0]["context"], + "dGhlIGFzc29jaWF0ZWQgZGF0YQ==" + ); + assert!(encrypts[1].2["batch_input"][0].get("context").is_none()); + // The key's derivation is read once per key. + let reads = requests + .iter() + .filter(|(method, path, _)| method == "GET" && path.starts_with("transit/keys/")) + .count(); + assert_eq!(reads, 2); + let _ = vault + .wrap(KEY, &Protected::new([1u8; 32]), aad) + .await + .unwrap(); + assert_eq!( + emulator + .requests() + .iter() + .filter(|(method, _, _)| method == "GET") + .count(), + 2 + ); +} + +#[tokio::test] +async fn detect_refuses_a_missing_key_and_a_key_that_is_not_an_aead() { + let emulator = TransitEmulator::new(); + emulator.add_key("rsa", false, "rsa-2048"); + let vault = client_with(&emulator, KeyDerivation::Detect); + let error = vault + .wrap("no-such-key", &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::NotFound); + let error = vault + .wrap("rsa", &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::AccessDenied); + // Nothing was encrypted. + assert!(emulator + .requests() + .iter() + .all(|(_, path, _)| !path.contains("/encrypt/"))); +} + +#[tokio::test] +async fn a_key_reference_that_is_not_a_key_name_is_refused_before_any_call() { + let emulator = TransitEmulator::new(); + let vault = client_with(&emulator, KeyDerivation::NotDerived); + for bad in ["", "a/b", "..", "a?b", "a b", "../../sys/seal"] { + let error = vault + .wrap(bad, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::NotFound, "{bad:?}"); + } + assert!(emulator.requests().is_empty()); +} + +#[tokio::test] +async fn a_wrapped_key_that_is_not_one_is_refused_before_any_call() { + let emulator = TransitEmulator::new(); + let vault = client_with(&emulator, KeyDerivation::NotDerived); + for bad in [&b""[..], b"vault:v1:AAAA", &[2, 0, 0, 0, 1, 7]] { + let error = vault.unwrap(KEY, bad, b"a").await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); + } + assert!(emulator.requests().is_empty()); +} + +#[tokio::test] +async fn rewrap_keeps_the_key_and_its_binding() { + let emulator = TransitEmulator::new(); + for key in [KEY, DERIVED_KEY] { + let vault = client_with(&emulator, KeyDerivation::Detect); + let wrapped = vault + .wrap(key, &Protected::new([4u8; 32]), b"aad") + .await + .unwrap(); + let rewrapped = vault.rewrap(key, &wrapped, b"aad").await.unwrap(); + assert_eq!( + vault + .unwrap(key, &rewrapped, b"aad") + .await + .unwrap() + .risky_ref(), + &[4u8; 32] + ); + assert!(vault.unwrap(key, &rewrapped, b"other").await.is_err()); + } +} + +#[tokio::test] +async fn the_namespace_is_sent_on_every_request() { + let emulator = TransitEmulator::new().with_namespace("admin/team-a"); + let vault = VaultTransit::new( + VaultConfig::new(ADDRESS) + .unwrap() + .with_namespace("admin/team-a") + .unwrap(), + emulator.clone(), + VaultToken::new(TOKEN), + ); + let wrapped = vault + .wrap(KEY, &Protected::new([3u8; 32]), b"a") + .await + .unwrap(); + let _ = vault.unwrap(KEY, &wrapped, b"a").await.unwrap(); + // The emulator asserts the header on each request. + assert_eq!(emulator.requests().len(), 3); +} + +#[tokio::test] +async fn the_backend_reports_vault_and_batches_and_sleeps_on_its_transport() { + let emulator = TransitEmulator::new(); + let vault = fixture_client(emulator.clone()); + assert_eq!(vault.vendor(), KeyVendor::Vault); + assert!(vault.supports_mode(KeyMode::Wrapped)); + assert!(vault.supports_mode(KeyMode::Derived)); + assert_eq!(vault.batch_limit().unwrap().get(), 250); + vault.sleep(std::time::Duration::from_millis(50)).await; + assert_eq!(emulator.sleeps(), [std::time::Duration::from_millis(50)]); +} diff --git a/packages/stack-kms-vault/tests/fixtures.rs b/packages/stack-kms-vault/tests/fixtures.rs new file mode 100644 index 0000000000..b89186a01a --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures.rs @@ -0,0 +1,403 @@ +//! Each Vault call and each error mapping, against HTTP exchanges recorded +//! from `vault server -dev` 1.20 (`tests/fixtures/*.json`). +//! +//! Every fixture was recorded by `tests/vault_dev.rs`'s +//! `records_the_fixtures`, except `kubernetes_login.json` (a dev server has +//! no Kubernetes; the response is the AppRole login response in the +//! Kubernetes auth shape) and the errors whose name says "written by hand". +//! Each request side is pinned: the URL, the headers and the body, which +//! includes the bound key and the context. + +mod common; + +use common::{fixture_client, load, Fixture, FixtureResponse, Replay, DERIVED_KEY, KEY, TOKEN}; +use serde::Deserialize; +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor}; +use stack_kms_vault::{ + AppRoleLogin, AuthError, CachedToken, KeyDerivation, KubernetesLogin, TokenProvider, + VaultConfig, VaultToken, VaultTransit, +}; +use stack_kms_vendor::derive::{record_key_aad, wrapped_data_key_aad}; +use stack_kms_vendor::{ + KeyVendorBackend, TransportError, UnwrapItem, VendorRequest, VendorResponse, VendorTransport, + WrapItem, +}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; + +const KEYSET: Uuid = Uuid::from_u128(0x0f1e2d3c_4b5a_6978_8796_a5b4c3d2e1f0); + +fn record_aad() -> Vec { + record_key_aad(KEYSET, KeyVendor::Vault, KeyMode::Wrapped) +} + +fn data_aads() -> Vec> { + (0u8..3) + .map(|n| wrapped_data_key_aad(KEYSET, &[n; 16], "users/email")) + .collect() +} + +fn kind_of(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "vault-transit"); + kind + } + other => panic!("not a backend error: {other:?}"), + } +} + +fn replay(name: &str) -> Replay { + Replay::new().expect_all(load(name)) +} + +/// The stored wrapped key of the `index`th ciphertext a fixture's last +/// response returns. +fn wrapped_of(name: &str, index: usize) -> Vec { + let fixtures: Vec = load(name); + let body = &fixtures.last().unwrap().response.body; + let text = body["data"]["batch_results"][index]["ciphertext"] + .as_str() + .unwrap(); + stack_kms_vault::compact_ciphertext(text).unwrap() +} + +async fn check_single(prefix: &str, key: &str) { + let vault = fixture_client(replay(&format!("{prefix}encrypt.json"))); + let wrapped = vault + .wrap(key, &Protected::new([0x42u8; 32]), &record_aad()) + .await + .unwrap(); + assert_eq!(wrapped, wrapped_of(&format!("{prefix}encrypt.json"), 0)); + assert!(vault.transport().is_done()); + + let vault = fixture_client(replay(&format!("{prefix}decrypt.json"))); + let opened = vault.unwrap(key, &wrapped, &record_aad()).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x42u8; 32]); + assert!(vault.transport().is_done()); +} + +async fn check_batch(prefix: &str, key: &str) { + let aads = data_aads(); + let keys: Vec<_> = (0u8..3).map(|n| Protected::new([0x40 + n; 32])).collect(); + let vault = fixture_client(replay(&format!("{prefix}encrypt_batch.json"))); + let items: Vec<_> = keys + .iter() + .zip(&aads) + .map(|(key, aad)| WrapItem { key, aad }) + .collect(); + let wrapped = vault.wrap_batch(key, &items).await.unwrap(); + assert_eq!(wrapped.len(), 3); + for (index, wrapped) in wrapped.iter().enumerate() { + assert_eq!( + wrapped, + &wrapped_of(&format!("{prefix}encrypt_batch.json"), index) + ); + } + + let vault = fixture_client(replay(&format!("{prefix}decrypt_batch.json"))); + let items: Vec<_> = wrapped + .iter() + .zip(&aads) + .map(|(wrapped, aad)| UnwrapItem { wrapped, aad }) + .collect(); + let opened = vault.unwrap_batch(key, &items).await.unwrap(); + for (n, key) in opened.iter().enumerate() { + assert_eq!(key.risky_ref(), &[0x40 + n as u8; 32]); + } + assert!(vault.transport().is_done()); +} + +#[tokio::test] +async fn encrypt_and_decrypt_with_a_non_derived_key() { + check_single("", KEY).await; +} + +#[tokio::test] +async fn encrypt_and_decrypt_with_a_derived_key() { + check_single("derived_", DERIVED_KEY).await; +} + +#[tokio::test] +async fn a_batch_is_one_call() { + check_batch("", KEY).await; +} + +#[tokio::test] +async fn a_batch_with_a_derived_key_is_one_call() { + check_batch("derived_", DERIVED_KEY).await; +} + +#[tokio::test] +async fn the_fixture_ciphertexts_name_the_key_version() { + let fixtures: Vec = load("encrypt.json"); + let text = fixtures[1].response.body["data"]["batch_results"][0]["ciphertext"] + .as_str() + .unwrap(); + assert!(text.starts_with("vault:v1:")); +} + +#[tokio::test] +async fn a_decrypt_under_other_associated_data_is_refused_by_the_bound_key() { + // Vault decrypts it (a non-derived key has no context to check); the + // bound key does not open under other associated data. + let wrapped = wrapped_of("encrypt.json", 0); + let other = record_key_aad(KEYSET, KeyVendor::Vault, KeyMode::Derived); + let fixtures: Vec = load("decrypt.json"); + let vault = VaultTransit::new( + VaultConfig::new(common::ADDRESS) + .unwrap() + .with_key_derivation(KeyDerivation::NotDerived), + Replay::new().answer(fixtures[1].response.to_response()), + VaultToken::new(TOKEN), + ); + let error = vault.unwrap(KEY, &wrapped, &other).await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); +} + +#[tokio::test] +async fn rewrap_moves_to_the_latest_version() { + let fixtures: Vec = load("rewrap.json"); + let text = fixtures[0].request.body["batch_input"][0]["ciphertext"] + .as_str() + .unwrap(); + let wrapped = stack_kms_vault::compact_ciphertext(text).unwrap(); + let vault = VaultTransit::new( + VaultConfig::new(common::ADDRESS) + .unwrap() + .with_key_derivation(KeyDerivation::NotDerived), + Replay::new().expect_all(fixtures), + VaultToken::new(TOKEN), + ); + let rewrapped = vault.rewrap(KEY, &wrapped, &record_aad()).await.unwrap(); + assert_eq!(&wrapped[1..5], &1u32.to_be_bytes()); + assert_eq!(&rewrapped[1..5], &2u32.to_be_bytes()); +} + +#[tokio::test] +async fn approle_login() { + let login = AppRoleLogin::new( + &VaultConfig::new(common::ADDRESS).unwrap(), + replay("approle_login.json"), + "fixture-role-id", + "fixture-secret-id", + ); + let token = login.token().await.unwrap(); + assert!(token.token().starts_with("hvs.")); + assert_eq!(token.lease(), Some(std::time::Duration::from_secs(20 * 60))); + assert!(login.transport().is_done()); +} + +#[tokio::test] +async fn kubernetes_login_reads_the_token_file_and_sends_the_namespace() { + let path = std::env::temp_dir().join(format!("stack-kms-vault-jwt-{}", Uuid::new_v4())); + std::fs::write(&path, "fixture.jwt.token\n").unwrap(); + let config = VaultConfig::new(common::ADDRESS) + .unwrap() + .with_namespace("admin") + .unwrap(); + let login = KubernetesLogin::new(&config, replay("kubernetes_login.json"), "cipherstash") + .with_token_path(&path); + let token = login.token().await.unwrap(); + assert_eq!(token.token(), "hvs.fixture-kubernetes-token"); + assert_eq!(token.lease(), Some(std::time::Duration::from_secs(3600))); + std::fs::remove_file(&path).unwrap(); + + let error = login.token().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert!(!error.to_string().contains("fixture.jwt.token")); +} + +#[tokio::test] +async fn a_refused_login_is_access_denied_and_says_why() { + let login = AppRoleLogin::new( + &VaultConfig::new(common::ADDRESS).unwrap(), + Replay::new().answer(VendorResponse::new( + 400, + br#"{"errors":["invalid role or secret ID"]}"#.to_vec(), + )), + "role", + "the-secret-id", + ); + let error = login.token().await.unwrap_err(); + assert_eq!(error.kind(), BackendErrorKind::AccessDenied); + assert!(error.to_string().contains("invalid role or secret ID")); + assert!(!error.to_string().contains("the-secret-id")); + + for (status, kind) in [ + (429, BackendErrorKind::Throttled), + (503, BackendErrorKind::Unavailable), + ] { + let login = AppRoleLogin::new( + &VaultConfig::new(common::ADDRESS).unwrap(), + Replay::new().answer(VendorResponse::new(status, b"{}".to_vec())), + "role", + "secret", + ); + assert_eq!(login.token().await.unwrap_err().kind(), kind); + } +} + +#[derive(Deserialize)] +struct ErrorCase { + name: String, + response: FixtureResponse, + kind: String, +} + +#[tokio::test] +async fn every_recorded_error_maps_to_its_kind() { + let cases: Vec = load("errors.json"); + assert!(cases.len() >= 12); + let wrapped = wrapped_of("encrypt.json", 0); + for case in cases { + let vault = VaultTransit::new( + VaultConfig::new(common::ADDRESS) + .unwrap() + .with_key_derivation(KeyDerivation::NotDerived), + Replay::new().answer(case.response.to_response()), + VaultToken::new(TOKEN), + ); + let error = vault + .unwrap(KEY, &wrapped, &record_aad()) + .await + .unwrap_err(); + let message = format!("{error:?}"); + assert_eq!( + format!("{:?}", kind_of(error)), + case.kind, + "{}: {message}", + case.name + ); + } +} + +/// A transport that never gets a response. +struct Down; + +impl VendorTransport for Down { + async fn send(&self, _request: VendorRequest) -> Result { + Err(TransportError("connection refused".into())) + } + + async fn sleep(&self, _duration: std::time::Duration) {} +} + +#[tokio::test] +async fn no_response_is_unavailable() { + let error = fixture_client(Down) + .unwrap(KEY, &wrapped_of("encrypt.json", 0), &record_aad()) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); +} + +struct NoToken; + +impl TokenProvider for NoToken { + async fn token(&self) -> Result { + Err(AuthError::access_denied("none configured")) + } +} + +#[tokio::test] +async fn no_token_is_access_denied_and_sends_nothing() { + let vault = VaultTransit::new( + VaultConfig::new(common::ADDRESS).unwrap(), + Replay::new(), + NoToken, + ); + let error = vault + .wrap(KEY, &Protected::new([1u8; 32]), &record_aad()) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::AccessDenied); +} + +#[tokio::test] +async fn an_unreadable_success_is_unavailable() { + let wrapped = wrapped_of("encrypt.json", 0); + for body in [ + b"not json".to_vec(), + br#"{"data":null}"#.to_vec(), + br#"{"data":{"batch_results":[]}}"#.to_vec(), + br#"{"data":{"batch_results":[{"plaintext":"not base64!"}]}}"#.to_vec(), + br#"{"data":{"batch_results":[{}]}}"#.to_vec(), + ] { + let vault = VaultTransit::new( + VaultConfig::new(common::ADDRESS) + .unwrap() + .with_key_derivation(KeyDerivation::NotDerived), + Replay::new().answer(VendorResponse::new(200, body)), + VaultToken::new(TOKEN), + ); + let error = vault + .unwrap(KEY, &wrapped, &record_aad()) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); + } + for body in [ + br#"{"data":{"batch_results":[{"ciphertext":"not a vault ciphertext"}]}}"#.to_vec(), + br#"{"data":{"batch_results":[{}]}}"#.to_vec(), + ] { + let vault = VaultTransit::new( + VaultConfig::new(common::ADDRESS) + .unwrap() + .with_key_derivation(KeyDerivation::NotDerived), + Replay::new().answer(VendorResponse::new(200, body)), + VaultToken::new(TOKEN), + ); + let error = vault + .wrap(KEY, &Protected::new([1u8; 32]), b"a") + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::Unavailable); + } +} + +#[tokio::test] +async fn a_plaintext_that_is_not_a_bound_key_is_refused() { + // 32 bytes: a bare key, not a bound key this crate wrapped. + let body = br#"{"data":{"batch_results":[{"plaintext":"AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8="}]}}"#; + let vault = VaultTransit::new( + VaultConfig::new(common::ADDRESS) + .unwrap() + .with_key_derivation(KeyDerivation::NotDerived), + Replay::new().answer(VendorResponse::new(200, body.to_vec())), + VaultToken::new(TOKEN), + ); + let error = vault + .unwrap(KEY, &wrapped_of("encrypt.json", 0), &record_aad()) + .await + .unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::InvalidCiphertext); +} + +#[tokio::test] +async fn a_403_drops_the_cached_token() { + let emulator = common::TransitEmulator::new(); + let config = VaultConfig::new(common::ADDRESS).unwrap(); + let vault = VaultTransit::new( + config.clone(), + emulator.clone(), + CachedToken::new(AppRoleLogin::new( + &config, + emulator.clone(), + "fixture-role-id", + "fixture-secret-id", + )), + ); + let wrapped = vault + .wrap(KEY, &Protected::new([2u8; 32]), b"a") + .await + .unwrap(); + assert_eq!(emulator.logins(), 1); + emulator.revoke_tokens(); + let error = vault.unwrap(KEY, &wrapped, b"a").await.unwrap_err(); + assert_eq!(kind_of(error), BackendErrorKind::AccessDenied); + // The next call logs in again. + let opened = vault.unwrap(KEY, &wrapped, b"a").await.unwrap(); + assert_eq!(opened.risky_ref(), &[2u8; 32]); + assert_eq!(emulator.logins(), 2); +} diff --git a/packages/stack-kms-vault/tests/fixtures/approle_login.json b/packages/stack-kms-vault/tests/fixtures/approle_login.json new file mode 100644 index 0000000000..7580792225 --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/approle_login.json @@ -0,0 +1,63 @@ +[ + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/auth/approle/login", + "headers": [ + [ + "content-type", + "application/json" + ], + [ + "x-vault-request", + "true" + ] + ], + "body": { + "role_id": "fixture-role-id", + "secret_id": "fixture-secret-id" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": { + "accessor": "HjtgtYyKbCqHkdof10qacF2T", + "client_token": "hvs.fixture-approle-token", + "entity_id": "829b5b18-c7ea-dc88-fb8e-d062e4bc22b9", + "lease_duration": 1200, + "metadata": { + "role_name": "cipherstash-fixture" + }, + "mfa_requirement": null, + "num_uses": 0, + "orphan": true, + "policies": [ + "cipherstash-fixture", + "default" + ], + "renewable": true, + "token_policies": [ + "cipherstash-fixture", + "default" + ], + "token_type": "service" + }, + "data": null, + "lease_duration": 0, + "lease_id": "", + "mount_type": "", + "renewable": false, + "request_id": "0d334ea8-3d6c-570f-9d25-a6e781cab3c7", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/decrypt.json b/packages/stack-kms-vault/tests/fixtures/decrypt.json new file mode 100644 index 0000000000..b1ec7822e9 --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/decrypt.json @@ -0,0 +1,113 @@ +[ + { + "request": { + "method": "GET", + "url": "http://127.0.0.1:8200/v1/transit/keys/cipherstash-fixture", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "allow_plaintext_backup": false, + "auto_rotate_period": 0, + "deletion_allowed": false, + "derived": false, + "exportable": false, + "imported_key": false, + "keys": { + "1": 1793922877 + }, + "latest_version": 1, + "min_available_version": 0, + "min_decryption_version": 1, + "min_encryption_version": 0, + "name": "cipherstash-fixture", + "supports_decryption": true, + "supports_derivation": true, + "supports_encryption": true, + "supports_signing": false, + "type": "aes256-gcm96" + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "d5bfa463-1286-ae7d-0322-17207c6c6966", + "warnings": null, + "wrap_info": null + } + } + }, + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/transit/decrypt/cipherstash-fixture", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "batch_input": [ + { + "ciphertext": "vault:v1:I7N6/bDFCgIaJWv46169sRJOh5ctmsb2LDGtmCVkwHkUX4nCpjazGZh41Dp2OsDbIg47UsbWinnYfBFVneQ8IXBf4kJCRg/aOlrXw5DR6lq+UzNpicyn2F0f2Xo4" + } + ] + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "plaintext": "AfsodyYVMevqYNXYFf5JEoGe5JCoMcypmaAU/FSu7HTmQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=", + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "d26bd1b2-d066-b11b-7b20-0aba779d3cc3", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/decrypt_batch.json b/packages/stack-kms-vault/tests/fixtures/decrypt_batch.json new file mode 100644 index 0000000000..13875eb7f9 --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/decrypt_batch.json @@ -0,0 +1,127 @@ +[ + { + "request": { + "method": "GET", + "url": "http://127.0.0.1:8200/v1/transit/keys/cipherstash-fixture", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "allow_plaintext_backup": false, + "auto_rotate_period": 0, + "deletion_allowed": false, + "derived": false, + "exportable": false, + "imported_key": false, + "keys": { + "1": 1793922877 + }, + "latest_version": 1, + "min_available_version": 0, + "min_decryption_version": 1, + "min_encryption_version": 0, + "name": "cipherstash-fixture", + "supports_decryption": true, + "supports_derivation": true, + "supports_encryption": true, + "supports_signing": false, + "type": "aes256-gcm96" + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "c455572a-2ca7-eae1-ce8d-2d0403dd99a0", + "warnings": null, + "wrap_info": null + } + } + }, + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/transit/decrypt/cipherstash-fixture", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "batch_input": [ + { + "ciphertext": "vault:v1:qtbTUkudNVlrA8q3k2TsdeREewI5Q1Cy3/9EyvadgrMXRSPJ+UUkrmBV9v6guOeAHI0GVkZ2g219XOGKo1aJyQvKBM93I2q77oMyAVcljwXMLRM23AtCX0fGpH2E" + }, + { + "ciphertext": "vault:v1:SOonZWYwK42Yw4XvZ9aUI0FbTGwejkrKwDYu/+sGs5e2pObSMuHzjBAVfbdvCwYkM/BFYOO2IlM4+6E7aovOdPUXj8pehEcooAoEYk/HCsBofzhAQGMwK5MOxe9k" + }, + { + "ciphertext": "vault:v1:W1YvhTo4lMzGoxDKFjkFiTtoqehUXG5eOv4h1/bu0mfCW9A2Pzh4eDl0CrJ5rOGdFAxblEZX4HG4gxM2b0v1sbnglWSUAUNo7qF3EifWd5zWx6wmNuDGVLsQ26xi" + } + ] + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "plaintext": "AZlF03MkT4XV5cJBgwxOtLGKAfREkT2P6GPu03ek2dKRQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEA=", + "reference": "" + }, + { + "plaintext": "AYMOBWlv4yz+mSGK1iKQB+ZgCqLsWJwttCwwiMtJEAyDQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUE=", + "reference": "" + }, + { + "plaintext": "AZNt220tYX6khetAWOuhDwYVOlXKerI+xWj6h4juOyYOQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=", + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "8b096624-2285-a2d2-f0bc-00fd5e69cd42", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/derived_decrypt.json b/packages/stack-kms-vault/tests/fixtures/derived_decrypt.json new file mode 100644 index 0000000000..4af8d56b0f --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/derived_decrypt.json @@ -0,0 +1,116 @@ +[ + { + "request": { + "method": "GET", + "url": "http://127.0.0.1:8200/v1/transit/keys/cipherstash-fixture-derived", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "allow_plaintext_backup": false, + "auto_rotate_period": 0, + "convergent_encryption": false, + "deletion_allowed": false, + "derived": true, + "exportable": false, + "imported_key": false, + "kdf": "hkdf_sha256", + "keys": { + "1": 1793922877 + }, + "latest_version": 1, + "min_available_version": 0, + "min_decryption_version": 1, + "min_encryption_version": 0, + "name": "cipherstash-fixture-derived", + "supports_decryption": true, + "supports_derivation": true, + "supports_encryption": true, + "supports_signing": false, + "type": "aes256-gcm96" + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "7c7109b4-a9c2-0e0d-74aa-17825004bde1", + "warnings": null, + "wrap_info": null + } + } + }, + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/transit/decrypt/cipherstash-fixture-derived", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "batch_input": [ + { + "ciphertext": "vault:v1:1zE8ZkJpGw5Se+1Z6JXEVCSFa67w84Ho9TZTuTJunMkPxDzvqHCp+bc7NmjKiGJPg9rC5ZIClAK7y+6UWQIInUXg7cdlVbXfNtsbLbR76Fzs4P3j8dj2yKB57cTo", + "context": "BQAAAAAAAAAlAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Iva2V5c2V0LXJlY29yZC1rZXkvdjEJAAAAAAAAAGluZGV4LWtleRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8AEAAAAAAAAABAEAAAAAAAAAAQ==" + } + ] + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "plaintext": "AfsodyYVMevqYNXYFf5JEoGe5JCoMcypmaAU/FSu7HTmQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=", + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "60038f1b-e789-60a7-6884-85396e98e318", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/derived_decrypt_batch.json b/packages/stack-kms-vault/tests/fixtures/derived_decrypt_batch.json new file mode 100644 index 0000000000..9cc635f695 --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/derived_decrypt_batch.json @@ -0,0 +1,132 @@ +[ + { + "request": { + "method": "GET", + "url": "http://127.0.0.1:8200/v1/transit/keys/cipherstash-fixture-derived", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "allow_plaintext_backup": false, + "auto_rotate_period": 0, + "convergent_encryption": false, + "deletion_allowed": false, + "derived": true, + "exportable": false, + "imported_key": false, + "kdf": "hkdf_sha256", + "keys": { + "1": 1793922877 + }, + "latest_version": 1, + "min_available_version": 0, + "min_decryption_version": 1, + "min_encryption_version": 0, + "name": "cipherstash-fixture-derived", + "supports_decryption": true, + "supports_derivation": true, + "supports_encryption": true, + "supports_signing": false, + "type": "aes256-gcm96" + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "d0be64d4-8638-c496-2e29-416c0af7eb32", + "warnings": null, + "wrap_info": null + } + } + }, + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/transit/decrypt/cipherstash-fixture-derived", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "batch_input": [ + { + "ciphertext": "vault:v1:yZKm4bni817QZprSElt6ooOJtZSl3IdxdtHnjjTCEum/nXRX99/wWkbaxNY0h6FIi3F2OCZeVJbkrHiO7jAGjUHodOCkzFCAwbp23QvS5n/aowZbbxfSpgiwhtNz", + "context": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAdXNlcnMvZW1haWw=" + }, + { + "ciphertext": "vault:v1:JddnL0W4uTcrWE5XqxkxjB4WO/ivgY2bVqZDVkFFe7fXMKGCXOz218InkbroMMuiGSTxQzy5BvSVKPsEzZ14n3XJZXYWGE9QXyH4v8Esbw5H133v/icJrplPVDcZ", + "context": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAQEBAQEBAQEBAQEBAQEBAQsAAAAAAAAAdXNlcnMvZW1haWw=" + }, + { + "ciphertext": "vault:v1:lkunFMYAYgKZm0oQzncqPkJnATrnSat76l0wF3/d55bpbU57OpCBdAFMLir1RtJ32Fd9hn/FnKp6vxofXuVI1i3pLTQTQFeB8o9OmOx6w8qwVhUmSIDBQC5BltWS", + "context": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAgICAgICAgICAgICAgICAgsAAAAAAAAAdXNlcnMvZW1haWw=" + } + ] + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "plaintext": "AZlF03MkT4XV5cJBgwxOtLGKAfREkT2P6GPu03ek2dKRQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEA=", + "reference": "" + }, + { + "plaintext": "AYMOBWlv4yz+mSGK1iKQB+ZgCqLsWJwttCwwiMtJEAyDQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUE=", + "reference": "" + }, + { + "plaintext": "AZNt220tYX6khetAWOuhDwYVOlXKerI+xWj6h4juOyYOQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=", + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "399709ab-8b80-2c4b-dccf-3b25164d3b41", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/derived_encrypt.json b/packages/stack-kms-vault/tests/fixtures/derived_encrypt.json new file mode 100644 index 0000000000..4e338465ef --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/derived_encrypt.json @@ -0,0 +1,117 @@ +[ + { + "request": { + "method": "GET", + "url": "http://127.0.0.1:8200/v1/transit/keys/cipherstash-fixture-derived", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "allow_plaintext_backup": false, + "auto_rotate_period": 0, + "convergent_encryption": false, + "deletion_allowed": false, + "derived": true, + "exportable": false, + "imported_key": false, + "kdf": "hkdf_sha256", + "keys": { + "1": 1793922877 + }, + "latest_version": 1, + "min_available_version": 0, + "min_decryption_version": 1, + "min_encryption_version": 0, + "name": "cipherstash-fixture-derived", + "supports_decryption": true, + "supports_derivation": true, + "supports_encryption": true, + "supports_signing": false, + "type": "aes256-gcm96" + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "8fb8fa30-f9de-4ff4-fb9c-a27a2a703e55", + "warnings": null, + "wrap_info": null + } + } + }, + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/transit/encrypt/cipherstash-fixture-derived", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "batch_input": [ + { + "context": "BQAAAAAAAAAlAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Iva2V5c2V0LXJlY29yZC1rZXkvdjEJAAAAAAAAAGluZGV4LWtleRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8AEAAAAAAAAABAEAAAAAAAAAAQ==", + "plaintext": "AfsodyYVMevqYNXYFf5JEoGe5JCoMcypmaAU/FSu7HTmQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=" + } + ] + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "ciphertext": "vault:v1:1zE8ZkJpGw5Se+1Z6JXEVCSFa67w84Ho9TZTuTJunMkPxDzvqHCp+bc7NmjKiGJPg9rC5ZIClAK7y+6UWQIInUXg7cdlVbXfNtsbLbR76Fzs4P3j8dj2yKB57cTo", + "key_version": 1, + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "97b691f6-ad20-3b36-5eea-d1ec9fbea242", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/derived_encrypt_batch.json b/packages/stack-kms-vault/tests/fixtures/derived_encrypt_batch.json new file mode 100644 index 0000000000..b5ee01a7eb --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/derived_encrypt_batch.json @@ -0,0 +1,135 @@ +[ + { + "request": { + "method": "GET", + "url": "http://127.0.0.1:8200/v1/transit/keys/cipherstash-fixture-derived", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "allow_plaintext_backup": false, + "auto_rotate_period": 0, + "convergent_encryption": false, + "deletion_allowed": false, + "derived": true, + "exportable": false, + "imported_key": false, + "kdf": "hkdf_sha256", + "keys": { + "1": 1793922877 + }, + "latest_version": 1, + "min_available_version": 0, + "min_decryption_version": 1, + "min_encryption_version": 0, + "name": "cipherstash-fixture-derived", + "supports_decryption": true, + "supports_derivation": true, + "supports_encryption": true, + "supports_signing": false, + "type": "aes256-gcm96" + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "cb14c656-8698-4b6f-c912-7f88d4b899c9", + "warnings": null, + "wrap_info": null + } + } + }, + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/transit/encrypt/cipherstash-fixture-derived", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "batch_input": [ + { + "context": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAsAAAAAAAAAdXNlcnMvZW1haWw=", + "plaintext": "AZlF03MkT4XV5cJBgwxOtLGKAfREkT2P6GPu03ek2dKRQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEA=" + }, + { + "context": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAQEBAQEBAQEBAQEBAQEBAQsAAAAAAAAAdXNlcnMvZW1haWw=", + "plaintext": "AYMOBWlv4yz+mSGK1iKQB+ZgCqLsWJwttCwwiMtJEAyDQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUE=" + }, + { + "context": "BAAAAAAAAAAkAAAAAAAAAHN0YWNrLWttcy12ZW5kb3Ivd3JhcHBlZC1kYXRhLWtleS92MRAAAAAAAAAADx4tPEtaaXiHlqW0w9Lh8BAAAAAAAAAAAgICAgICAgICAgICAgICAgsAAAAAAAAAdXNlcnMvZW1haWw=", + "plaintext": "AZNt220tYX6khetAWOuhDwYVOlXKerI+xWj6h4juOyYOQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=" + } + ] + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "ciphertext": "vault:v1:yZKm4bni817QZprSElt6ooOJtZSl3IdxdtHnjjTCEum/nXRX99/wWkbaxNY0h6FIi3F2OCZeVJbkrHiO7jAGjUHodOCkzFCAwbp23QvS5n/aowZbbxfSpgiwhtNz", + "key_version": 1, + "reference": "" + }, + { + "ciphertext": "vault:v1:JddnL0W4uTcrWE5XqxkxjB4WO/ivgY2bVqZDVkFFe7fXMKGCXOz218InkbroMMuiGSTxQzy5BvSVKPsEzZ14n3XJZXYWGE9QXyH4v8Esbw5H133v/icJrplPVDcZ", + "key_version": 1, + "reference": "" + }, + { + "ciphertext": "vault:v1:lkunFMYAYgKZm0oQzncqPkJnATrnSat76l0wF3/d55bpbU57OpCBdAFMLir1RtJ32Fd9hn/FnKp6vxofXuVI1i3pLTQTQFeB8o9OmOx6w8qwVhUmSIDBQC5BltWS", + "key_version": 1, + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "3a4c04eb-c423-36d9-d59d-22930682caf7", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/encrypt.json b/packages/stack-kms-vault/tests/fixtures/encrypt.json new file mode 100644 index 0000000000..9b886c3849 --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/encrypt.json @@ -0,0 +1,114 @@ +[ + { + "request": { + "method": "GET", + "url": "http://127.0.0.1:8200/v1/transit/keys/cipherstash-fixture", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "allow_plaintext_backup": false, + "auto_rotate_period": 0, + "deletion_allowed": false, + "derived": false, + "exportable": false, + "imported_key": false, + "keys": { + "1": 1793922877 + }, + "latest_version": 1, + "min_available_version": 0, + "min_decryption_version": 1, + "min_encryption_version": 0, + "name": "cipherstash-fixture", + "supports_decryption": true, + "supports_derivation": true, + "supports_encryption": true, + "supports_signing": false, + "type": "aes256-gcm96" + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "ec81819a-825e-6189-202c-896abde0d02e", + "warnings": null, + "wrap_info": null + } + } + }, + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/transit/encrypt/cipherstash-fixture", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "batch_input": [ + { + "plaintext": "AfsodyYVMevqYNXYFf5JEoGe5JCoMcypmaAU/FSu7HTmQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=" + } + ] + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "ciphertext": "vault:v1:I7N6/bDFCgIaJWv46169sRJOh5ctmsb2LDGtmCVkwHkUX4nCpjazGZh41Dp2OsDbIg47UsbWinnYfBFVneQ8IXBf4kJCRg/aOlrXw5DR6lq+UzNpicyn2F0f2Xo4", + "key_version": 1, + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "4f8414c1-34ef-0b27-f6c0-1253eb1aa958", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/encrypt_batch.json b/packages/stack-kms-vault/tests/fixtures/encrypt_batch.json new file mode 100644 index 0000000000..eec5dcea0d --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/encrypt_batch.json @@ -0,0 +1,130 @@ +[ + { + "request": { + "method": "GET", + "url": "http://127.0.0.1:8200/v1/transit/keys/cipherstash-fixture", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ] + ], + "body": null + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "allow_plaintext_backup": false, + "auto_rotate_period": 0, + "deletion_allowed": false, + "derived": false, + "exportable": false, + "imported_key": false, + "keys": { + "1": 1793922877 + }, + "latest_version": 1, + "min_available_version": 0, + "min_decryption_version": 1, + "min_encryption_version": 0, + "name": "cipherstash-fixture", + "supports_decryption": true, + "supports_derivation": true, + "supports_encryption": true, + "supports_signing": false, + "type": "aes256-gcm96" + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "4e32f585-4efe-9656-324d-e4d0aa615638", + "warnings": null, + "wrap_info": null + } + } + }, + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/transit/encrypt/cipherstash-fixture", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "batch_input": [ + { + "plaintext": "AZlF03MkT4XV5cJBgwxOtLGKAfREkT2P6GPu03ek2dKRQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEA=" + }, + { + "plaintext": "AYMOBWlv4yz+mSGK1iKQB+ZgCqLsWJwttCwwiMtJEAyDQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUFBQUE=" + }, + { + "plaintext": "AZNt220tYX6khetAWOuhDwYVOlXKerI+xWj6h4juOyYOQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkJCQkI=" + } + ] + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "ciphertext": "vault:v1:qtbTUkudNVlrA8q3k2TsdeREewI5Q1Cy3/9EyvadgrMXRSPJ+UUkrmBV9v6guOeAHI0GVkZ2g219XOGKo1aJyQvKBM93I2q77oMyAVcljwXMLRM23AtCX0fGpH2E", + "key_version": 1, + "reference": "" + }, + { + "ciphertext": "vault:v1:SOonZWYwK42Yw4XvZ9aUI0FbTGwejkrKwDYu/+sGs5e2pObSMuHzjBAVfbdvCwYkM/BFYOO2IlM4+6E7aovOdPUXj8pehEcooAoEYk/HCsBofzhAQGMwK5MOxe9k", + "key_version": 1, + "reference": "" + }, + { + "ciphertext": "vault:v1:W1YvhTo4lMzGoxDKFjkFiTtoqehUXG5eOv4h1/bu0mfCW9A2Pzh4eDl0CrJ5rOGdFAxblEZX4HG4gxM2b0v1sbnglWSUAUNo7qF3EifWd5zWx6wmNuDGVLsQ26xi", + "key_version": 1, + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "f5a5ed47-04e7-db59-2377-5b7f73fbccc5", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/errors.json b/packages/stack-kms-vault/tests/fixtures/errors.json new file mode 100644 index 0000000000..0df8b694ab --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/errors.json @@ -0,0 +1,289 @@ +[ + { + "kind": "InvalidCiphertext", + "name": "decrypt under another context", + "response": { + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "error": "cipher: message authentication failed", + "plaintext": "", + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "", + "renewable": false, + "request_id": "3d7b1d8d-57af-8910-f284-1242fd2df87c", + "warnings": null, + "wrap_info": null + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 400 + } + }, + { + "kind": "InvalidCiphertext", + "name": "a key version that is too new", + "response": { + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "error": "invalid ciphertext: version is too new", + "plaintext": "", + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "", + "renewable": false, + "request_id": "7d99e008-a905-abce-1fd5-c6409d737251", + "warnings": null, + "wrap_info": null + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 400 + } + }, + { + "kind": "InvalidCiphertext", + "name": "a ciphertext that is too short", + "response": { + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "error": "invalid ciphertext length", + "plaintext": "", + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "", + "renewable": false, + "request_id": "4350d912-0652-624f-b30d-84d70962295c", + "warnings": null, + "wrap_info": null + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 400 + } + }, + { + "kind": "NotFound", + "name": "decrypt with a key that does not exist", + "response": { + "body": { + "errors": [ + "encryption key not found" + ] + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 400 + } + }, + { + "kind": "NotFound", + "name": "read a key that does not exist", + "response": { + "body": { + "errors": [] + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 404 + } + }, + { + "kind": "AccessDenied", + "name": "a derived key with no context", + "response": { + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "error": "missing 'context' for key derivation; the key was created using a derived key, which means additional, per-request information must be included in order to perform operations with the key", + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "", + "renewable": false, + "request_id": "f0c099e7-45ad-988d-2fd1-e75a6baa6f2c", + "warnings": null, + "wrap_info": null + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 400 + } + }, + { + "kind": "AccessDenied", + "name": "a bad token", + "response": { + "body": { + "errors": [ + "2 errors occurred:\n\t* permission denied\n\t* invalid token\n\n" + ] + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 403 + } + }, + { + "kind": "NotFound", + "name": "a mount that does not exist", + "response": { + "body": { + "errors": [ + "no handler for route \"no-such-mount/decrypt/cipherstash-fixture\". route entry not found." + ] + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 404 + } + }, + { + "kind": "Throttled", + "name": "a rate limit quota (written by hand)", + "response": { + "body": { + "errors": [ + "request path \"transit/decrypt/cipherstash-fixture\": rate limit quota exceeded" + ] + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 429 + } + }, + { + "kind": "Unavailable", + "name": "a sealed Vault (written by hand)", + "response": { + "body": { + "errors": [ + "Vault is sealed" + ] + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 503 + } + }, + { + "kind": "Unavailable", + "name": "an internal error (written by hand)", + "response": { + "body": { + "errors": [ + "internal error" + ] + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 500 + } + }, + { + "kind": "AccessDenied", + "name": "a policy that denies (written by hand)", + "response": { + "body": { + "errors": [ + "1 error occurred:\n\t* permission denied\n\n" + ] + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 403 + } + }, + { + "kind": "InvalidCiphertext", + "name": "a retired key version (written by hand)", + "response": { + "body": { + "errors": [ + "ciphertext version is disallowed by policy (too old)" + ] + }, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "status": 400 + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/kubernetes_login.json b/packages/stack-kms-vault/tests/fixtures/kubernetes_login.json new file mode 100644 index 0000000000..536339f6da --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/kubernetes_login.json @@ -0,0 +1,71 @@ +[ + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/auth/kubernetes/login", + "headers": [ + [ + "content-type", + "application/json" + ], + [ + "x-vault-request", + "true" + ], + [ + "x-vault-namespace", + "admin" + ] + ], + "body": { + "role": "cipherstash", + "jwt": "fixture.jwt.token" + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": { + "accessor": "HjtgtYyKbCqHkdof10qacF2T", + "client_token": "hvs.fixture-kubernetes-token", + "entity_id": "829b5b18-c7ea-dc88-fb8e-d062e4bc22b9", + "lease_duration": 3600, + "metadata": { + "role": "cipherstash", + "service_account_name": "app", + "service_account_namespace": "default", + "service_account_secret_name": "", + "service_account_uid": "5b1c2c9f-0000-4000-8000-000000000001" + }, + "mfa_requirement": null, + "num_uses": 0, + "orphan": true, + "policies": [ + "cipherstash-fixture", + "default" + ], + "renewable": true, + "token_policies": [ + "cipherstash-fixture", + "default" + ], + "token_type": "service" + }, + "data": null, + "lease_duration": 0, + "lease_id": "", + "mount_type": "", + "renewable": false, + "request_id": "0d334ea8-3d6c-570f-9d25-a6e781cab3c7", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/fixtures/rewrap.json b/packages/stack-kms-vault/tests/fixtures/rewrap.json new file mode 100644 index 0000000000..e25db255b0 --- /dev/null +++ b/packages/stack-kms-vault/tests/fixtures/rewrap.json @@ -0,0 +1,57 @@ +[ + { + "request": { + "method": "POST", + "url": "http://127.0.0.1:8200/v1/transit/rewrap/cipherstash-fixture", + "headers": [ + [ + "x-vault-token", + "fixture-root-token" + ], + [ + "x-vault-request", + "true" + ], + [ + "content-type", + "application/json" + ] + ], + "body": { + "batch_input": [ + { + "ciphertext": "vault:v1:uyjPyhnCJTiF/N/7UijXui3P3F3aynaSFB0VHRXIcPoVyf97omLk30xBtmdCpsmZGBThE173nMMJiJ25eNf70m7BPZ1KRcckfTxe/vP5K5QJnF6GmfU9zyc/4tty" + } + ] + } + }, + "response": { + "status": 200, + "headers": [ + [ + "content-type", + "application/json" + ] + ], + "body": { + "auth": null, + "data": { + "batch_results": [ + { + "ciphertext": "vault:v2:XO/+V+UpFufNSUAIIKHIFzdO+nPfJeapoUo8OCNAkW6enJeNgtc+HArZQb+eForzaSVXf1oUlbEHTkjcEazsE1pHc0Uk3aVxLe+oy1hzSRMgEtuwIWPsdGEjt8mG", + "key_version": 2, + "reference": "" + } + ] + }, + "lease_duration": 0, + "lease_id": "", + "mount_type": "transit", + "renewable": false, + "request_id": "2dc6ea03-7263-5dc8-5836-953f6cb1e2e1", + "warnings": null, + "wrap_info": null + } + } + } +] diff --git a/packages/stack-kms-vault/tests/vault_dev.rs b/packages/stack-kms-vault/tests/vault_dev.rs new file mode 100644 index 0000000000..68ecf857e2 --- /dev/null +++ b/packages/stack-kms-vault/tests/vault_dev.rs @@ -0,0 +1,664 @@ +//! The Vault Transit backend against a real `vault server -dev`, over the +//! native transport. +//! +//! Runs only when `STACK_KMS_VAULT_ADDR` is set (for example +//! `http://127.0.0.1:8200`); otherwise each test passes at once and says it +//! was skipped. The token is `STACK_KMS_VAULT_TOKEN` (default `root`). CI +//! runs Vault as a service and sets both. +//! +//! ```sh +//! docker run -d --rm -p 8200:8200 -e VAULT_DEV_ROOT_TOKEN_ID=root hashicorp/vault:1.20 +//! STACK_KMS_VAULT_ADDR=http://127.0.0.1:8200 \ +//! cargo nextest run -p stack-kms-vault --all-features --test vault_dev +//! ``` +//! +//! `records_the_fixtures` writes `tests/fixtures/*.json` again. It runs +//! only when `STACK_KMS_VAULT_RECORD_FIXTURES=1` is also set, against a dev +//! server at `common::ADDRESS` whose root token is `common::TOKEN`: +//! +//! ```sh +//! docker run -d --rm -p 8200:8200 -e VAULT_DEV_ROOT_TOKEN_ID=fixture-root-token hashicorp/vault:1.20 +//! STACK_KMS_VAULT_ADDR=http://127.0.0.1:8200 STACK_KMS_VAULT_TOKEN=fixture-root-token \ +//! STACK_KMS_VAULT_RECORD_FIXTURES=1 \ +//! cargo nextest run -p stack-kms-vault --all-features --test vault_dev records_the_fixtures +//! ``` + +#![cfg(not(target_arch = "wasm32"))] + +mod common; + +use std::num::NonZeroUsize; + +use common::{Fixture, Recording, DERIVED_KEY, KEY}; +use serde_json::{json, Value}; +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor}; +use stack_kms_vault::{ + AppRoleLogin, CachedToken, KeyDerivation, TokenProvider, VaultConfig, VaultToken, VaultTransit, +}; +use stack_kms_vendor::derive::{record_key_aad, wrapped_data_key_aad}; +use stack_kms_vendor::{contract, HttpVendorTransport, KeyVendorBackend}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; +use zeroize::Zeroizing; + +const ADDR_VAR: &str = "STACK_KMS_VAULT_ADDR"; +const TOKEN_VAR: &str = "STACK_KMS_VAULT_TOKEN"; +const RECORD_VAR: &str = "STACK_KMS_VAULT_RECORD_FIXTURES"; + +type Client = VaultTransit; + +struct Server { + address: String, + token: String, +} + +fn server() -> Option { + let address = std::env::var(ADDR_VAR).ok().filter(|e| !e.is_empty()); + let Some(address) = address else { + eprintln!("skipped: set {ADDR_VAR} to run against vault server -dev"); + return None; + }; + let token = std::env::var(TOKEN_VAR) + .ok() + .filter(|e| !e.is_empty()) + .unwrap_or_else(|| "root".to_owned()); + Some(Server { address, token }) +} + +fn config(server: &Server) -> VaultConfig { + VaultConfig::new(&server.address).unwrap() +} + +fn client(server: &Server) -> Client { + VaultTransit::new( + config(server), + HttpVendorTransport::new().unwrap(), + VaultToken::new(&server.token), + ) +} + +fn body(value: Value) -> Option>> { + Some(Zeroizing::new(serde_json::to_vec(&value).unwrap())) +} + +/// Send an admin request, ignoring a failure (the mount or key may exist). +async fn admin(vault: &Client, method: &str, path: &str, value: Option) -> Option { + vault + .send_json::(method, path, value.and_then(body)) + .await + .ok() +} + +/// Enable the Transit engine (if needed) and create a key. +async fn create_key(vault: &Client, name: &str, derived: bool) { + let _ = admin( + vault, + "POST", + "sys/mounts/transit", + Some(json!({"type": "transit"})), + ) + .await; + let _ = admin( + vault, + "POST", + &format!("transit/keys/{name}"), + Some(json!({"derived": derived})), + ) + .await; +} + +fn kind(error: Error) -> BackendErrorKind { + match error { + Error::Backend { kind, .. } => kind, + other => panic!("{other:?}"), + } +} + +fn unique(prefix: &str) -> String { + format!("{prefix}-{}", Uuid::new_v4().simple()) +} + +#[tokio::test] +async fn vault_transit_meets_the_contract_on_vault_dev() { + let Some(server) = server() else { return }; + let admin_client = client(&server); + for derived in [false, true] { + let key = unique(if derived { + "contract-derived" + } else { + "contract" + }); + create_key(&admin_client, &key, derived).await; + let make = || { + VaultTransit::new( + config(&server).with_batch_size(NonZeroUsize::new(4).unwrap()), + HttpVendorTransport::new().unwrap(), + VaultToken::new(&server.token), + ) + }; + contract::run(make, &key).await; + } +} + +#[tokio::test] +async fn vault_dev_errors_map_to_kinds() { + let Some(server) = server() else { return }; + let vault = client(&server); + let key = unique("errors"); + let derived = unique("errors-derived"); + create_key(&vault, &key, false).await; + create_key(&vault, &derived, true).await; + let aad = wrapped_data_key_aad(Uuid::new_v4(), &[3u8; 16], "users/email"); + let wrapped = vault + .wrap(&derived, &Protected::new([8u8; 32]), &aad) + .await + .unwrap(); + assert_eq!( + vault + .unwrap(&derived, &wrapped, &aad) + .await + .unwrap() + .risky_ref(), + &[8u8; 32] + ); + + // Another context: Vault itself refuses it for a derived key. + let other = wrapped_data_key_aad(Uuid::new_v4(), &[3u8; 16], "users/email"); + let error = vault.unwrap(&derived, &wrapped, &other).await.unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::InvalidCiphertext); + + // A damaged ciphertext. + let mut damaged = wrapped.clone(); + if let Some(last) = damaged.last_mut() { + *last ^= 1; + } + let error = vault.unwrap(&derived, &damaged, &aad).await.unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::InvalidCiphertext); + + // A key version that does not exist yet. + let mut future = wrapped.clone(); + future[1..5].copy_from_slice(&9u32.to_be_bytes()); + let error = vault.unwrap(&derived, &future, &aad).await.unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::InvalidCiphertext); + + // Another key's ciphertext. + let error = vault.unwrap(&key, &wrapped, &aad).await.unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::InvalidCiphertext); + + // A key that does not exist. + let error = vault + .wrap(&unique("missing"), &Protected::new([1u8; 32]), &aad) + .await + .unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::NotFound); + + // A derived key told it is not derived. + let not_derived = VaultTransit::new( + config(&server).with_key_derivation(KeyDerivation::NotDerived), + HttpVendorTransport::new().unwrap(), + VaultToken::new(&server.token), + ); + let error = not_derived + .wrap(&derived, &Protected::new([1u8; 32]), &aad) + .await + .unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::AccessDenied); + + // A bad token. + let bad = VaultTransit::new( + config(&server), + HttpVendorTransport::new().unwrap(), + VaultToken::new("not-a-token"), + ); + let error = bad + .wrap(&key, &Protected::new([1u8; 32]), &aad) + .await + .unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::AccessDenied); + + // A mount that does not exist. + let no_mount = VaultTransit::new( + config(&server).with_mount("no-such-mount").unwrap(), + HttpVendorTransport::new().unwrap(), + VaultToken::new(&server.token), + ); + let error = no_mount + .wrap(&key, &Protected::new([1u8; 32]), &aad) + .await + .unwrap_err(); + assert_eq!(kind(error), BackendErrorKind::NotFound); +} + +#[tokio::test] +async fn rotation_and_rewrap_on_vault_dev() { + let Some(server) = server() else { return }; + let vault = client(&server); + let key = unique("rotate"); + create_key(&vault, &key, true).await; + let wrapped = vault + .wrap(&key, &Protected::new([6u8; 32]), b"aad") + .await + .unwrap(); + let _ = admin(&vault, "POST", &format!("transit/keys/{key}/rotate"), None).await; + // The old version still opens; rewrap moves it to the new one. + assert_eq!( + vault + .unwrap(&key, &wrapped, b"aad") + .await + .unwrap() + .risky_ref(), + &[6u8; 32] + ); + let rewrapped = vault.rewrap(&key, &wrapped, b"aad").await.unwrap(); + assert_eq!(&rewrapped[1..5], &2u32.to_be_bytes()); + assert_eq!( + vault + .unwrap(&key, &rewrapped, b"aad") + .await + .unwrap() + .risky_ref(), + &[6u8; 32] + ); +} + +/// Enable AppRole auth and make a role that may use `key`; its role id and +/// a secret id. With `fixed`, the ids are the fixtures' own. +async fn approle(vault: &Client, role: &str, key: &str, fixed: bool) -> (String, String) { + let _ = admin( + vault, + "POST", + "sys/auth/approle", + Some(json!({"type": "approle"})), + ) + .await; + let policy = format!( + "path \"transit/encrypt/{key}\" {{ capabilities = [\"update\"] }}\n\ + path \"transit/decrypt/{key}\" {{ capabilities = [\"update\"] }}\n\ + path \"transit/keys/{key}\" {{ capabilities = [\"read\"] }}\n" + ); + let _ = admin( + vault, + "PUT", + &format!("sys/policies/acl/{role}"), + Some(json!({ "policy": policy })), + ) + .await; + let _ = admin( + vault, + "POST", + &format!("auth/approle/role/{role}"), + Some(json!({ "token_policies": [role], "token_ttl": "20m" })), + ) + .await; + if fixed { + let _ = admin( + vault, + "POST", + &format!("auth/approle/role/{role}/role-id"), + Some(json!({ "role_id": "fixture-role-id" })), + ) + .await; + let _ = admin( + vault, + "POST", + &format!("auth/approle/role/{role}/custom-secret-id"), + Some(json!({ "secret_id": "fixture-secret-id" })), + ) + .await; + return ("fixture-role-id".into(), "fixture-secret-id".into()); + } + let role_id = admin( + vault, + "GET", + &format!("auth/approle/role/{role}/role-id"), + None, + ) + .await + .unwrap(); + let secret = admin( + vault, + "POST", + &format!("auth/approle/role/{role}/secret-id"), + Some(json!({})), + ) + .await + .unwrap(); + ( + role_id["data"]["role_id"].as_str().unwrap().to_owned(), + secret["data"]["secret_id"].as_str().unwrap().to_owned(), + ) +} + +#[tokio::test] +async fn approle_login_on_vault_dev() { + let Some(server) = server() else { return }; + let admin_client = client(&server); + let key = unique("approle"); + create_key(&admin_client, &key, false).await; + let role = unique("role"); + let (role_id, secret_id) = approle(&admin_client, &role, &key, false).await; + + let auth = CachedToken::new(AppRoleLogin::new( + &config(&server), + HttpVendorTransport::new().unwrap(), + role_id, + secret_id, + )); + let token = auth.token().await.unwrap(); + assert_eq!(token.lease(), Some(std::time::Duration::from_secs(20 * 60))); + let vault = VaultTransit::new(config(&server), HttpVendorTransport::new().unwrap(), auth); + let aad = wrapped_data_key_aad(Uuid::new_v4(), &[5u8; 16], "users/email"); + let wrapped = vault + .wrap(&key, &Protected::new([5u8; 32]), &aad) + .await + .unwrap(); + assert_eq!( + vault + .unwrap(&key, &wrapped, &aad) + .await + .unwrap() + .risky_ref(), + &[5u8; 32] + ); + + // A wrong secret id is refused. + let wrong = AppRoleLogin::new( + &config(&server), + HttpVendorTransport::new().unwrap(), + "fixture-role-id-that-does-not-exist", + "nope", + ); + assert_eq!( + wrong.token().await.unwrap_err().kind(), + BackendErrorKind::AccessDenied + ); +} + +/// Write `tests/fixtures/` (an exchange sequence). +fn write(name: &str, exchanges: Vec) { + let text = serde_json::to_string_pretty(&exchanges).unwrap() + "\n"; + std::fs::write(common::fixture_path(name), text).unwrap(); +} + +fn recorded_client( + derivation: KeyDerivation, +) -> VaultTransit, VaultToken> { + VaultTransit::new( + VaultConfig::new(common::ADDRESS) + .unwrap() + .with_key_derivation(derivation), + Recording::new(HttpVendorTransport::new().unwrap()), + VaultToken::new(common::TOKEN), + ) +} + +/// The record key fixtures wrap: a fixed keyset id. +pub const FIXTURE_KEYSET: Uuid = Uuid::from_u128(0x0f1e2d3c_4b5a_6978_8796_a5b4c3d2e1f0); + +#[tokio::test] +async fn records_the_fixtures() { + let Some(server) = server() else { return }; + if std::env::var(RECORD_VAR).as_deref() != Ok("1") { + eprintln!("skipped: set {RECORD_VAR}=1 to record the fixtures"); + return; + } + assert_eq!(server.address.trim_end_matches('/'), common::ADDRESS); + assert_eq!(server.token, common::TOKEN); + let admin_client = client(&server); + create_key(&admin_client, KEY, false).await; + create_key(&admin_client, DERIVED_KEY, true).await; + + let record_aad = record_key_aad(FIXTURE_KEYSET, KeyVendor::Vault, KeyMode::Wrapped); + let data_aads: Vec> = (0u8..3) + .map(|n| wrapped_data_key_aad(FIXTURE_KEYSET, &[n; 16], "users/email")) + .collect(); + let keys: Vec<_> = (0u8..3).map(|n| Protected::new([0x40 + n; 32])).collect(); + + for (key, prefix) in [(KEY, ""), (DERIVED_KEY, "derived_")] { + // One key: a key read, then encrypt. + let vault = recorded_client(KeyDerivation::Detect); + let wrapped = vault + .wrap(key, &Protected::new([0x42u8; 32]), &record_aad) + .await + .unwrap(); + write(&format!("{prefix}encrypt.json"), vault.transport_take()); + let vault = recorded_client(KeyDerivation::Detect); + let opened = vault.unwrap(key, &wrapped, &record_aad).await.unwrap(); + assert_eq!(opened.risky_ref(), &[0x42u8; 32]); + write(&format!("{prefix}decrypt.json"), vault.transport_take()); + + // A batch of three. + let vault = recorded_client(KeyDerivation::Detect); + let items: Vec<_> = keys + .iter() + .zip(&data_aads) + .map(|(key, aad)| stack_kms_vendor::WrapItem { key, aad }) + .collect(); + let wrapped = vault.wrap_batch(key, &items).await.unwrap(); + write( + &format!("{prefix}encrypt_batch.json"), + vault.transport_take(), + ); + let vault = recorded_client(KeyDerivation::Detect); + let items: Vec<_> = wrapped + .iter() + .zip(&data_aads) + .map(|(wrapped, aad)| stack_kms_vendor::UnwrapItem { wrapped, aad }) + .collect(); + let opened = vault.unwrap_batch(key, &items).await.unwrap(); + assert_eq!(opened.len(), 3); + write( + &format!("{prefix}decrypt_batch.json"), + vault.transport_take(), + ); + } + + // Rewrap after a rotation. + let vault = recorded_client(KeyDerivation::NotDerived); + let wrapped = vault + .wrap(KEY, &Protected::new([0x42u8; 32]), &record_aad) + .await + .unwrap(); + let _ = vault.transport_take(); + let _ = admin( + &admin_client, + "POST", + &format!("transit/keys/{KEY}/rotate"), + None, + ) + .await; + let rewrapped = vault.rewrap(KEY, &wrapped, &record_aad).await.unwrap(); + write("rewrap.json", vec![vault.transport_take().pop().unwrap()]); + let _ = rewrapped; + + // An AppRole login, then a key read and an encrypt with its token. + let _ = approle(&admin_client, "cipherstash-fixture", KEY, true).await; + let login = AppRoleLogin::new( + &VaultConfig::new(common::ADDRESS).unwrap(), + Recording::new(HttpVendorTransport::new().unwrap()), + "fixture-role-id", + "fixture-secret-id", + ); + let _ = login.token().await.unwrap(); + write("approle_login.json", login.transport().take()); + + // Errors, each one exchange. + let mut errors = Vec::new(); + let mut case = |name: &str, kind: &str, exchanges: Vec| { + let exchange = exchanges.into_iter().last().unwrap(); + errors.push(json!({ + "name": name, + "kind": kind, + "response": exchange.response, + })); + }; + let vault = recorded_client(KeyDerivation::NotDerived); + let derived_wrapped = recorded_client(KeyDerivation::Derived) + .wrap(DERIVED_KEY, &Protected::new([1u8; 32]), &record_aad) + .await + .unwrap(); + let other_aad = record_key_aad(FIXTURE_KEYSET, KeyVendor::Vault, KeyMode::Derived); + let derived = recorded_client(KeyDerivation::Derived); + assert_eq!( + kind( + derived + .unwrap(DERIVED_KEY, &derived_wrapped, &other_aad) + .await + .unwrap_err() + ), + BackendErrorKind::InvalidCiphertext + ); + case( + "decrypt under another context", + "InvalidCiphertext", + derived.transport_take(), + ); + let mut future = derived_wrapped.clone(); + future[1..5].copy_from_slice(&9u32.to_be_bytes()); + let derived = recorded_client(KeyDerivation::Derived); + assert_eq!( + kind( + derived + .unwrap(DERIVED_KEY, &future, &record_aad) + .await + .unwrap_err() + ), + BackendErrorKind::InvalidCiphertext + ); + case( + "a key version that is too new", + "InvalidCiphertext", + derived.transport_take(), + ); + let short = stack_kms_vault::compact_ciphertext("vault:v1:AAAA").unwrap(); + assert_eq!( + kind(vault.unwrap(KEY, &short, &record_aad).await.unwrap_err()), + BackendErrorKind::InvalidCiphertext + ); + case( + "a ciphertext that is too short", + "InvalidCiphertext", + vault.transport_take(), + ); + assert_eq!( + kind( + vault + .unwrap("no-such-key", &short, &record_aad) + .await + .unwrap_err() + ), + BackendErrorKind::NotFound + ); + case( + "decrypt with a key that does not exist", + "NotFound", + vault.transport_take(), + ); + let detect = recorded_client(KeyDerivation::Detect); + assert_eq!( + kind( + detect + .wrap("no-such-key", &Protected::new([1u8; 32]), &record_aad) + .await + .unwrap_err() + ), + BackendErrorKind::NotFound + ); + case( + "read a key that does not exist", + "NotFound", + detect.transport_take(), + ); + assert_eq!( + kind( + vault + .wrap(DERIVED_KEY, &Protected::new([1u8; 32]), &record_aad) + .await + .unwrap_err() + ), + BackendErrorKind::AccessDenied + ); + case( + "a derived key with no context", + "AccessDenied", + vault.transport_take(), + ); + let bad = VaultTransit::new( + VaultConfig::new(common::ADDRESS) + .unwrap() + .with_key_derivation(KeyDerivation::NotDerived), + Recording::new(HttpVendorTransport::new().unwrap()), + VaultToken::new("not-a-token"), + ); + assert_eq!( + kind(bad.unwrap(KEY, &short, &record_aad).await.unwrap_err()), + BackendErrorKind::AccessDenied + ); + case("a bad token", "AccessDenied", bad.transport_take()); + let no_mount = VaultTransit::new( + VaultConfig::new(common::ADDRESS) + .unwrap() + .with_mount("no-such-mount") + .unwrap() + .with_key_derivation(KeyDerivation::NotDerived), + Recording::new(HttpVendorTransport::new().unwrap()), + VaultToken::new(common::TOKEN), + ); + assert_eq!( + kind(no_mount.unwrap(KEY, &short, &record_aad).await.unwrap_err()), + BackendErrorKind::NotFound + ); + case( + "a mount that does not exist", + "NotFound", + no_mount.transport_take(), + ); + // Responses a dev server does not give, in Vault's shapes. + for (name, status, message, kind) in [ + ( + "a rate limit quota", + 429, + "request path \"transit/decrypt/cipherstash-fixture\": rate limit quota exceeded", + "Throttled", + ), + ("a sealed Vault", 503, "Vault is sealed", "Unavailable"), + ("an internal error", 500, "internal error", "Unavailable"), + ( + "a policy that denies", + 403, + "1 error occurred:\n\t* permission denied\n\n", + "AccessDenied", + ), + ( + "a retired key version", + 400, + "ciphertext version is disallowed by policy (too old)", + "InvalidCiphertext", + ), + ] { + errors.push(json!({ + "name": format!("{name} (written by hand)"), + "kind": kind, + "response": { + "status": status, + "headers": [["content-type", "application/json"]], + "body": { "errors": [message] } + }, + })); + } + std::fs::write( + common::fixture_path("errors.json"), + serde_json::to_string_pretty(&errors).unwrap() + "\n", + ) + .unwrap(); +} + +trait TakeExchanges { + fn transport_take(&self) -> Vec; +} + +impl TakeExchanges for VaultTransit, A> { + fn transport_take(&self) -> Vec { + self.transport().take() + } +} diff --git a/packages/stack-kms-vendor/Cargo.toml b/packages/stack-kms-vendor/Cargo.toml new file mode 100644 index 0000000000..8e0d77a9ae --- /dev/null +++ b/packages/stack-kms-vendor/Cargo.toml @@ -0,0 +1,72 @@ +[package] +name = "stack-kms-vendor" +description = "Key vendor backends for Stack Encrypt: keyset records, the two key modes, and the static key backend" +version = "0.1.0" +edition.workspace = true +authors.workspace = true +repository.workspace = true +homepage.workspace = true +keywords.workspace = true +categories.workspace = true +license-file = "LICENSE" +# Publishable, in the `stack-kms-vendor` version group with every vendor crate +# (stack-kms-aws, ...). release-plz.toml keeps `release = false` until the +# first version is on crates.io: crates.io Trusted Publishing needs an +# existing crate, so the first version is published by hand (see +# docs/plans/2026-11-05-key-vendor-backends-decisions.md, CIP-5307). + +[features] +# `stack_kms_vendor::contract` (the shared contract suite every key vendor +# backend runs) and `FakeVendor` (an in-memory key vendor backend). The +# contract suite drives a real `StackCipher`, so this pulls stack-encrypt in. +test-support = ["dep:stack-encrypt", "stack-kms/test-support"] +# `HttpVendorTransport`: the native `VendorTransport`, over the reqwest client +# and the TLS configuration stack-kms uses for ZeroKMS. Native only: on wasm32 +# the dependencies are not in the graph and the type does not exist, because +# the WASI guest sends through its `transport_send` host import instead. +http = ["dep:reqwest", "dep:stack-auth", "dep:tokio"] + +[dependencies] +stack-kms = { path = "../stack-kms", version = "0.2.0", default-features = false } +# The contract suite only: it seals and opens values through a cipher. +stack-encrypt = { path = "../stack-encrypt", version = "0.3.0", default-features = false, optional = true } + +vitaminc = { workspace = true, features = ["protected", "random"] } +aes-gcm = { version = "0.10.3", features = ["zeroize"] } +base64ct = { version = "1.7", features = ["alloc"] } +futures = "0.3.25" +hkdf = "0.13" +sha2 = "0.11" +subtle = "2.6" +miette = { workspace = true } +serde_json = { workspace = true } +thiserror = { workspace = true } +uuid = { workspace = true } +web-time = { workspace = true } +zeroize = { workspace = true } + +[target.'cfg(not(target_arch = "wasm32"))'.dependencies] +reqwest = { workspace = true, optional = true } +# Only for `stack_auth::tls::client_config`: the platform verifier with the +# compromised-CA blocklist that every bundled reqwest client in the stack +# crates uses. +stack-auth = { workspace = true, optional = true, features = ["http"] } +# The backoff timer of `HttpVendorTransport::sleep`. reqwest already runs on +# tokio, so this adds no runtime. +tokio = { version = "1.47.1", optional = true, default-features = false, features = ["time"] } + +[dev-dependencies] +stack-kms = { path = "../stack-kms", default-features = false, features = ["test-support"] } +stack-encrypt = { path = "../stack-encrypt", default-features = false } +tokio = { workspace = true, features = ["rt", "macros", "time", "net", "io-util"] } + +[package.metadata.docs.rs] +all-features = true + +[[test]] +name = "contract" +required-features = ["test-support"] + +[[test]] +name = "router" +required-features = ["test-support"] diff --git a/packages/stack-kms-vendor/LICENSE b/packages/stack-kms-vendor/LICENSE new file mode 100644 index 0000000000..2cbd67a66d --- /dev/null +++ b/packages/stack-kms-vendor/LICENSE @@ -0,0 +1,96 @@ +# PolyForm Internal Use License 1.0.0 + + + +## Acceptance + +In order to get any license under these terms, you must agree +to them as both strict obligations and conditions to all +your licenses. + +## Copyright License + +The licensor grants you a copyright license for the software +to do everything you might do with the software that would +otherwise infringe the licensor's copyright in it for any +permitted purpose. However, you may only make changes or +new works based on the software according to [Changes and New +Works License](#changes-and-new-works-license), and you may +not distribute the software. + +## Changes and New Works License + +The licensor grants you an additional copyright license to +make changes and new works based on the software for any +permitted purpose. + +## Patent License + +The licensor grants you a patent license for the software that +covers patent claims the licensor can license, or becomes able +to license, that you would infringe by using the software. + +## Fair Use + +You may have "fair use" rights for the software under the +law. These terms do not limit them. + +## Internal Business Use + +Use of the software for the internal business operations of +you and your company is use for a permitted purpose. + +## No Other Rights + +These terms do not allow you to sublicense or transfer any of +your licenses to anyone else, or prevent the licensor from +granting licenses to anyone else. These terms do not imply +any other licenses. + +## Patent Defense + +If you make any written claim that the software infringes or +contributes to infringement of any patent, your patent license +for the software granted under these terms ends immediately. If +your company makes such a claim, your patent license ends +immediately for work on behalf of your company. + +## Violations + +The first time you are notified in writing that you have +violated any of these terms, or done anything with the software +not covered by your licenses, your licenses can nonetheless +continue if you come into full compliance with these terms, +and take practical steps to correct past violations, within +32 days of receiving notice. Otherwise, all your licenses +end immediately. + +## No Liability + +***As far as the law allows, the software comes as is, without +any warranty or condition, and the licensor will not be liable +to you for any damages arising out of these terms or the use +or nature of the software, under any kind of legal claim.*** + +## Definitions + +The **licensor** is the individual or entity offering these +terms, and the **software** is the software the licensor makes +available under these terms. + +**You** refers to the individual or entity agreeing to these +terms. + +**Your company** is any legal entity, sole proprietorship, +or other kind of organization that you work for, plus all +organizations that have control over, are under the control of, +or are under common control with that organization. **Control** +means ownership of substantially all the assets of an entity, +or the power to direct its management and policies by vote, +contract, or otherwise. Control can be direct or indirect. + +**Your licenses** are all the licenses granted to you for the +software under these terms. + +**Use** means anything you do with the software requiring one +of your licenses. diff --git a/packages/stack-kms-vendor/src/batch.rs b/packages/stack-kms-vendor/src/batch.rs new file mode 100644 index 0000000000..324880ee87 --- /dev/null +++ b/packages/stack-kms-vendor/src/batch.rs @@ -0,0 +1,263 @@ +//! N vendor calls at once, with a limit, backoff and fail-whole-batch. + +use std::future::Future; +use std::num::NonZeroUsize; +use std::time::Duration; + +use futures::{StreamExt, TryStreamExt}; +use stack_kms::{BackendErrorKind, Error}; + +/// How [`run_batch`] sends a batch of vendor calls. +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct BatchOptions { + /// The most calls in flight at once. Default 16. + pub concurrency: NonZeroUsize, + /// The most attempts for one call, the first one included. A call is + /// tried again only after a `Throttled` failure. Default 5. + pub max_attempts: u32, + /// The wait before the second attempt. Each later wait is double the + /// one before it, up to `max_delay`. Default 50 ms. + pub base_delay: Duration, + /// The longest wait between two attempts. Default 2 s. + pub max_delay: Duration, +} + +impl Default for BatchOptions { + fn default() -> Self { + Self { + concurrency: NonZeroUsize::new(16).unwrap_or(NonZeroUsize::MIN), + max_attempts: 5, + base_delay: Duration::from_millis(50), + max_delay: Duration::from_secs(2), + } + } +} + +impl BatchOptions { + /// The wait after the failed attempt number `attempt` (1-based). + fn delay_after(&self, attempt: u32) -> Duration { + let factor = 1u32 + .checked_shl(attempt.saturating_sub(1)) + .unwrap_or(u32::MAX); + self.base_delay + .checked_mul(factor) + .map_or(self.max_delay, |delay| delay.min(self.max_delay)) + } +} + +fn is_throttled(error: &Error) -> bool { + matches!( + error, + Error::Backend { + kind: BackendErrorKind::Throttled, + .. + } + ) +} + +/// Run `call(0)` to `call(count - 1)`, at most +/// [`concurrency`](BatchOptions::concurrency) at once, and return the +/// results in index order. +/// +/// A call that fails with +/// [`Throttled`](stack_kms::BackendErrorKind::Throttled) is tried again +/// after `sleep(delay)`, with an exponential delay, up to +/// [`max_attempts`](BatchOptions::max_attempts) in all. Any other failure, +/// or a call that is still throttled after the last attempt, fails the +/// whole batch: the calls still in flight are dropped and no partial result +/// is returned. +/// +/// Runtime-agnostic: it uses no timer and no executor of its own. `sleep` +/// is the caller's timer ([`KeyVendorBackend::sleep`](crate::KeyVendorBackend::sleep)). +pub async fn run_batch( + options: &BatchOptions, + count: usize, + call: F, + sleep: S, +) -> Result, Error> +where + F: Fn(usize) -> Fut, + Fut: Future>, + S: Fn(Duration) -> SFut, + SFut: Future, +{ + let attempt = |index: usize| { + let call = &call; + let sleep = &sleep; + async move { + let mut attempt = 1; + loop { + match call(index).await { + Err(error) if is_throttled(&error) && attempt < options.max_attempts => { + sleep(options.delay_after(attempt)).await; + attempt += 1; + } + result => return result, + } + } + } + }; + futures::stream::iter(0..count) + .map(attempt) + .buffered(options.concurrency.get()) + .try_collect() + .await +} + +#[cfg(test)] +mod tests { + use std::sync::atomic::{AtomicUsize, Ordering}; + use std::sync::Mutex; + + use super::*; + + fn throttled() -> Error { + Error::Backend { + vendor: "test-vendor", + kind: BackendErrorKind::Throttled, + source: "slow down".into(), + } + } + + fn options() -> BatchOptions { + BatchOptions { + concurrency: NonZeroUsize::new(3).unwrap(), + ..BatchOptions::default() + } + } + + #[tokio::test] + async fn results_keep_index_order() { + let results = run_batch( + &options(), + 10, + |index| async move { + tokio::time::sleep(Duration::from_millis(10 - index as u64)).await; + Ok(index) + }, + |_| async {}, + ) + .await + .unwrap(); + assert_eq!(results, (0..10).collect::>()); + } + + #[tokio::test] + async fn no_more_than_the_limit_runs_at_once() { + let in_flight = AtomicUsize::new(0); + let peak = AtomicUsize::new(0); + let _ = run_batch( + &options(), + 12, + |_| async { + let now = in_flight.fetch_add(1, Ordering::SeqCst) + 1; + let _ = peak.fetch_max(now, Ordering::SeqCst); + tokio::time::sleep(Duration::from_millis(5)).await; + let _ = in_flight.fetch_sub(1, Ordering::SeqCst); + Ok(()) + }, + |_| async {}, + ) + .await + .unwrap(); + assert_eq!(peak.load(Ordering::SeqCst), 3); + } + + #[tokio::test] + async fn a_throttled_call_backs_off_and_is_tried_again() { + let attempts = AtomicUsize::new(0); + let waits = Mutex::new(Vec::new()); + let results = run_batch( + &options(), + 1, + |_| async { + if attempts.fetch_add(1, Ordering::SeqCst) < 3 { + Err(throttled()) + } else { + Ok("done") + } + }, + |delay| { + waits.lock().unwrap().push(delay); + async {} + }, + ) + .await + .unwrap(); + assert_eq!(results, ["done"]); + assert_eq!( + *waits.lock().unwrap(), + [ + Duration::from_millis(50), + Duration::from_millis(100), + Duration::from_millis(200) + ] + ); + } + + #[tokio::test] + async fn still_throttled_after_the_last_attempt_fails_the_batch() { + let attempts = AtomicUsize::new(0); + let result = run_batch( + &options(), + 1, + |_| async { + let _ = attempts.fetch_add(1, Ordering::SeqCst); + Err::<(), _>(throttled()) + }, + |_| async {}, + ) + .await; + assert!(matches!( + result, + Err(Error::Backend { + kind: BackendErrorKind::Throttled, + .. + }) + )); + assert_eq!(attempts.load(Ordering::SeqCst), 5); + } + + #[tokio::test] + async fn any_other_failure_fails_the_whole_batch_without_a_retry() { + let attempts = AtomicUsize::new(0); + let result = run_batch( + &options(), + 6, + |index| { + let _ = attempts.fetch_add(1, Ordering::SeqCst); + async move { + if index == 2 { + Err(Error::Backend { + vendor: "test-vendor", + kind: BackendErrorKind::AccessDenied, + source: "no".into(), + }) + } else { + Ok(index) + } + } + }, + |_| async {}, + ) + .await; + assert!(matches!( + result, + Err(Error::Backend { + kind: BackendErrorKind::AccessDenied, + .. + }) + )); + } + + #[test] + fn the_delay_doubles_up_to_the_cap() { + let options = BatchOptions::default(); + assert_eq!(options.delay_after(1), Duration::from_millis(50)); + assert_eq!(options.delay_after(2), Duration::from_millis(100)); + assert_eq!(options.delay_after(6), Duration::from_millis(1600)); + assert_eq!(options.delay_after(7), Duration::from_secs(2)); + assert_eq!(options.delay_after(40), Duration::from_secs(2)); + assert_eq!(options.concurrency.get(), 16); + } +} diff --git a/packages/stack-kms-vendor/src/bound.rs b/packages/stack-kms-vendor/src/bound.rs new file mode 100644 index 0000000000..b94635231a --- /dev/null +++ b/packages/stack-kms-vendor/src/bound.rs @@ -0,0 +1,150 @@ +//! Associated data bound by us, for a vendor key that cannot bind it. +//! +//! [`KeyVendorBackend`](crate::KeyVendorBackend) requires that `unwrap` +//! fails unless its `aad` is the `aad` given to `wrap`. Some vendor keys +//! have no associated data of their own: an Azure Key Vault RSA key, or a +//! HashiCorp Vault Transit key that is not derived (Vault ignores +//! `context` on such a key, and older Vault versions ignore +//! `associated_data`). For these, the vendor encrypts a **bound key** +//! instead of the bare key: +//! +//! ```text +//! bound key = version(1) = 1 ‖ SHA-256(frame[BOUND_KEY_LABEL, aad]) (32) ‖ key (32) +//! ``` +//! +//! On unwrap, [`open_bound_key`] recomputes the digest from the `aad` it is +//! given and compares it in constant time. A wrapped key moved to another +//! keyset, value or field decrypts, but does not open. +//! +//! # The rule for the vendor +//! +//! This is sound only if the vendor's encryption is **non-malleable**: an +//! attacker who holds a wrapped key must not be able to change its digest +//! without the vendor key. An AEAD (AES-GCM, ChaCha20-Poly1305) and RSA-OAEP +//! are non-malleable. Unauthenticated modes (AES-CBC, AES-CTR, RSA PKCS#1 +//! v1.5) are not, and must not be used. +//! +//! The bound key is 65 bytes, small enough for RSA-OAEP-256 with a 2048-bit +//! key (190 bytes at most). + +use sha2::{Digest, Sha256}; +use subtle::ConstantTimeEq; +use vitaminc::protected::{Controlled, Protected}; +use zeroize::Zeroizing; + +use crate::derive::frame; +use crate::vendor::KeyBytes; + +/// The label of the associated-data digest in a bound key. +pub const BOUND_KEY_LABEL: &[u8] = b"stack-kms-vendor/bound-key/v1"; +/// The version byte of a bound key. +pub const BOUND_KEY_VERSION: u8 = 1; +/// The length of a bound key: version, digest, key. +pub const BOUND_KEY_LEN: usize = 1 + 32 + 32; + +/// A bound key did not open. +#[derive(Clone, Copy, Debug, PartialEq, Eq, thiserror::Error)] +pub enum BoundKeyError { + /// The bytes are not a bound key: wrong length or version. + #[error("the vendor plaintext is not a bound key")] + Malformed, + /// The key was wrapped with other associated data. + #[error("the wrapped key belongs to other associated data")] + WrongAad, +} + +/// The digest of `aad` that a bound key carries. +fn aad_digest(aad: &[u8]) -> [u8; 32] { + Sha256::digest(frame(&[BOUND_KEY_LABEL, aad])).into() +} + +/// The bound key of `key` under `aad`: the plaintext a vendor encrypts in +/// place of `key`. Wiped on drop. +pub fn bind_key(key: &KeyBytes, aad: &[u8]) -> Zeroizing> { + let mut out = Zeroizing::new(Vec::with_capacity(BOUND_KEY_LEN)); + out.push(BOUND_KEY_VERSION); + out.extend_from_slice(&aad_digest(aad)); + out.extend_from_slice(key.risky_ref()); + out +} + +/// The key in `bound`, if `bound` is a bound key made under `aad`. +pub fn open_bound_key(bound: &[u8], aad: &[u8]) -> Result { + if bound.len() != BOUND_KEY_LEN || bound[0] != BOUND_KEY_VERSION { + return Err(BoundKeyError::Malformed); + } + let (digest, key) = bound[1..].split_at(32); + if !bool::from(digest.ct_eq(&aad_digest(aad))) { + return Err(BoundKeyError::WrongAad); + } + let mut out = Zeroizing::new([0u8; 32]); + out.copy_from_slice(key); + Ok(Protected::new(*out)) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn hex(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02x}")).collect() + } + + #[test] + fn a_bound_key_opens_under_its_own_aad_only() { + let key = Protected::new([5u8; 32]); + let bound = bind_key(&key, b"aad"); + assert_eq!(bound.len(), BOUND_KEY_LEN); + assert_eq!( + open_bound_key(&bound, b"aad").unwrap().risky_ref(), + &[5u8; 32] + ); + assert_eq!( + open_bound_key(&bound, b"other").unwrap_err(), + BoundKeyError::WrongAad + ); + assert_eq!( + open_bound_key(&bound, b"").unwrap_err(), + BoundKeyError::WrongAad + ); + } + + #[test] + fn anything_but_a_bound_key_is_malformed() { + let bound = bind_key(&Protected::new([5u8; 32]), b"aad"); + assert_eq!( + open_bound_key(&bound[..64], b"aad").unwrap_err(), + BoundKeyError::Malformed + ); + let mut long = bound.to_vec(); + long.push(0); + assert_eq!( + open_bound_key(&long, b"aad").unwrap_err(), + BoundKeyError::Malformed + ); + let mut version = bound.to_vec(); + version[0] = 2; + assert_eq!( + open_bound_key(&version, b"aad").unwrap_err(), + BoundKeyError::Malformed + ); + assert_eq!( + open_bound_key(&[5u8; 32], b"aad").unwrap_err(), + BoundKeyError::Malformed + ); + } + + /// A storage commitment: a wrapped key bound today must open later. + #[test] + fn the_bound_key_is_pinned() { + let bound = bind_key(&Protected::new([5u8; 32]), b"aad"); + assert_eq!( + hex(&bound), + concat!( + "01", + "ae51a3bebe8d3505fbec2c4045a899e0ffc51efd1fa0b91d1ce298ceb042e693", + "0505050505050505050505050505050505050505050505050505050505050505" + ) + ); + } +} diff --git a/packages/stack-kms-vendor/src/contract.rs b/packages/stack-kms-vendor/src/contract.rs new file mode 100644 index 0000000000..558151351e --- /dev/null +++ b/packages/stack-kms-vendor/src/contract.rs @@ -0,0 +1,668 @@ +//! The shared contract suite every key vendor backend runs. +//! +//! A vendor crate calls [`run`] (or each check alone) from its own tests +//! with a factory for its backend and a vendor key reference it may use. +//! Every check drives the backend through [`VendorKeySource`] and a real +//! [`StackCipher`], so it tests what a customer gets, not the trait alone. +//! A check panics with a message that names the broken rule. +//! +//! ```ignore +//! #[tokio::test] +//! async fn aws_kms_meets_the_contract() { +//! stack_kms_vendor::contract::run(|| AwsKms::new(localstack()), KEY_ARN).await; +//! } +//! ``` +//! +//! The checks, for each key mode the vendor offers: +//! +//! * a value and an index term round-trip, as a v2 leaf that names the +//! vendor and the mode, and a second cipher derives the same term; +//! * a value opened under another context (descriptor) is refused; +//! * a leaf of another key backend is refused before any key call; +//! * ZeroKMS-only inputs are refused, never ignored; +//! * the key lifetime is obeyed: under wrapped data keys each settle that +//! builds a term unwraps the index key again and the handle holds none; +//! under derived data keys the root key is unwrapped once per TTL; +//! * a batch keeps its order; +//! * a vendor with a batch operation wraps and unwraps the wrapped data +//! keys of a batch in one call per chunk of at most its batch limit; +//! * a throttled call is retried, and a call still throttled after the last +//! attempt fails the whole batch. +//! +//! Plus, for every vendor: a record refuses a mode the vendor does not +//! offer. + +// A contract check is a test: it panics with the rule it found broken. +#![allow(clippy::panic, clippy::unwrap_used, clippy::expect_used)] + +use std::borrow::Cow; +use std::num::NonZeroUsize; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::Arc; +use std::time::Duration; + +use stack_encrypt::kms::{ + BackendErrorKind, Context, DataKeySource, DecryptionPolicy, Error as KmsError, + FakeDataKeySource, GenerateKeyPayload, KeyBackend, KeyMode, KeyVendor, RetrieveKeyPayload, + UnverifiedContext, ZeroKmsFeature, +}; +use stack_encrypt::{nonempty, CipherText, Error, StackCipher, StackCipherText}; + +use crate::{ + BatchOptions, KeyBytes, KeyVendorBackend, KeysetRecord, RecordError, UnwrapItem, + VendorKeySource, WrapItem, +}; + +/// Run every check of the contract against the backends `make` builds, for +/// each key mode the backend offers. `key_ref` names a vendor key the +/// backend may wrap and unwrap under. +pub async fn run(make: F, key_ref: &str) +where + B: KeyVendorBackend + 'static, + F: Fn() -> B, +{ + refuses_a_mode_it_does_not_offer(make(), key_ref).await; + for mode in KeyMode::ALL { + if !make().supports_mode(mode) { + continue; + } + round_trip(&make, key_ref, mode).await; + refuses_another_context(make(), key_ref, mode).await; + refuses_another_key_backend(make(), key_ref, mode).await; + refuses_zerokms_features(make(), key_ref, mode).await; + obeys_the_key_lifetime(&make, key_ref, mode).await; + keeps_batch_order(make(), key_ref, mode).await; + uses_vendor_batches(make(), key_ref, mode).await; + backs_off_when_throttled(&make, key_ref, mode).await; + } +} + +/// A key vendor backend that counts the calls it forwards, and can refuse +/// the next calls as throttled. +pub struct Observed { + inner: B, + wraps: Arc, + unwraps: Arc, + batches: Arc, + throttle_next: Arc, +} + +/// The counters and the throttle of an [`Observed`] backend, kept by the +/// test after the backend moves into a key source. +#[derive(Clone)] +pub struct Observer { + wraps: Arc, + unwraps: Arc, + batches: Arc, + throttle_next: Arc, +} + +impl Observer { + /// Batch calls forwarded so far ([`KeyVendorBackend::wrap_batch`] and + /// [`KeyVendorBackend::unwrap_batch`]). Their keys also count as wraps + /// and unwraps. + pub fn batches(&self) -> usize { + self.batches.load(Ordering::SeqCst) + } + + /// Wrap calls forwarded so far (`generate_wrapped` included). + pub fn wraps(&self) -> usize { + self.wraps.load(Ordering::SeqCst) + } + + /// Unwrap calls forwarded so far. + pub fn unwraps(&self) -> usize { + self.unwraps.load(Ordering::SeqCst) + } + + /// Refuse the next `count` calls as throttled, without forwarding them. + pub fn throttle_next(&self, count: usize) { + self.throttle_next.store(count, Ordering::SeqCst); + } +} + +impl Observed { + /// Observe `inner`. + pub fn new(inner: B) -> (Self, Observer) { + let observer = Observer { + wraps: Arc::default(), + unwraps: Arc::default(), + batches: Arc::default(), + throttle_next: Arc::default(), + }; + ( + Self { + inner, + wraps: Arc::clone(&observer.wraps), + unwraps: Arc::clone(&observer.unwraps), + batches: Arc::clone(&observer.batches), + throttle_next: Arc::clone(&observer.throttle_next), + }, + observer, + ) + } + + fn throttle(&self) -> Result<(), KmsError> { + let pending = self + .throttle_next + .fetch_update(Ordering::SeqCst, Ordering::SeqCst, |n| n.checked_sub(1)); + match pending { + Ok(_) => Err(KmsError::Backend { + vendor: self.inner.vendor().name(), + kind: BackendErrorKind::Throttled, + source: "throttled by the contract suite".into(), + }), + Err(_) => Ok(()), + } + } +} + +impl KeyVendorBackend for Observed { + fn vendor(&self) -> KeyVendor { + self.inner.vendor() + } + + fn supports_mode(&self, mode: KeyMode) -> bool { + self.inner.supports_mode(mode) + } + + fn same_key(&self, a: &str, b: &str) -> bool { + self.inner.same_key(a, b) + } + + async fn wrap(&self, key_ref: &str, key: &KeyBytes, aad: &[u8]) -> Result, KmsError> { + self.throttle()?; + let _ = self.wraps.fetch_add(1, Ordering::SeqCst); + self.inner.wrap(key_ref, key, aad).await + } + + async fn unwrap( + &self, + key_ref: &str, + wrapped: &[u8], + aad: &[u8], + ) -> Result { + self.throttle()?; + let _ = self.unwraps.fetch_add(1, Ordering::SeqCst); + self.inner.unwrap(key_ref, wrapped, aad).await + } + + fn batch_limit(&self) -> Option { + self.inner.batch_limit() + } + + async fn wrap_batch( + &self, + key_ref: &str, + items: &[WrapItem<'_>], + ) -> Result>, KmsError> { + self.throttle()?; + let _ = self.batches.fetch_add(1, Ordering::SeqCst); + let _ = self.wraps.fetch_add(items.len(), Ordering::SeqCst); + self.inner.wrap_batch(key_ref, items).await + } + + async fn unwrap_batch( + &self, + key_ref: &str, + items: &[UnwrapItem<'_>], + ) -> Result, KmsError> { + self.throttle()?; + let _ = self.batches.fetch_add(1, Ordering::SeqCst); + let _ = self.unwraps.fetch_add(items.len(), Ordering::SeqCst); + self.inner.unwrap_batch(key_ref, items).await + } + + // `generate_wrapped` is not forwarded: the default (a local key and + // one `wrap`) is what this suite counts, and a vendor's own override is + // covered by the round trip. + + async fn sleep(&self, duration: Duration) { + self.inner.sleep(duration).await; + } +} + +/// A cipher over one fresh keyset of `backend` in `mode`, with its record. +async fn cipher_over( + backend: B, + key_ref: &str, + mode: KeyMode, + configure: impl FnOnce(VendorKeySource>) -> VendorKeySource>, +) -> ( + StackCipher>>, + KeysetRecord, + Observer, +) { + let (backend, observer) = Observed::new(backend); + let record = KeysetRecord::create(&backend, key_ref, mode, Some("contract".into())) + .await + .unwrap_or_else(|error| panic!("creating a {mode} keyset record failed: {error}")); + let source = configure( + VendorKeySource::new(backend, [record.clone()]) + .unwrap_or_else(|error| panic!("loading the {mode} record failed: {error}")), + ); + let cipher = StackCipher::builder() + .kms(source) + .init() + .await + .unwrap_or_else(|error| panic!("a cipher over a {mode} keyset failed to init: {error}")); + (cipher, record, observer) +} + +fn single(ciphertext: &StackCipherText) -> &stack_encrypt::SealedValue { + match ciphertext { + CipherText::Single(leaf) => leaf, + _ => panic!("a string seals as one leaf"), + } +} + +/// A value and an index term round-trip, the leaf is v2, and a second +/// cipher over the same record (a backend from `make`) derives the same +/// term and opens the same value. +pub async fn round_trip(make: &F, key_ref: &str, mode: KeyMode) +where + B: KeyVendorBackend, + F: Fn() -> B, +{ + let backend = make(); + let vendor = backend.vendor(); + let (cipher, record, _) = cipher_over(backend, key_ref, mode, |s| s).await; + let keyset = cipher.default_keyset(); + assert_eq!( + keyset.keyset_id(), + record.id(), + "the only record is the default keyset" + ); + let context = nonempty!("users").with("email"); + + let sealed = keyset + .encrypt_tree("alice@example.com".to_string(), context) + .await + .unwrap_or_else(|error| panic!("{mode}: sealing failed: {error}")); + let leaf = single(&sealed); + assert_eq!( + leaf.format_version(), + 2, + "{mode}: a key vendor backend seals v2 leaves" + ); + assert_eq!(leaf.key_backend(), KeyBackend::Vendor(vendor)); + assert_eq!(leaf.key_mode(), Some(mode)); + assert_eq!(leaf.keyset_id(), record.id()); + match mode { + KeyMode::Derived => assert!(leaf.tag().is_empty(), "a derived data key has no tag"), + KeyMode::Wrapped => assert!(!leaf.tag().is_empty(), "the tag holds the wrapped key"), + } + let bytes = leaf.to_bytes(); + let reread = stack_encrypt::SealedValue::from_bytes(&bytes).expect("a v2 leaf decodes"); + let opened: String = cipher + .decrypt(CipherText::Single(reread.clone()), context) + .await + .unwrap_or_else(|error| panic!("{mode}: opening failed: {error}")); + assert_eq!(opened, "alice@example.com"); + + let term = keyset + .equality_term("alice@example.com", context) + .await + .unwrap_or_else(|error| panic!("{mode}: an equality term failed: {error}")); + let again = keyset + .equality_term("alice@example.com", context) + .await + .expect("a second term"); + assert_eq!( + term.as_bytes(), + again.as_bytes(), + "{mode}: terms are deterministic" + ); + let other = keyset + .equality_term("bob@example.com", context) + .await + .expect("a term for another value"); + assert_ne!(term.as_bytes(), other.as_bytes()); + + // Another process, with the same record, derives the same term and + // opens the same value. + let other = StackCipher::builder() + .kms(VendorKeySource::new(make(), [record.clone()]).expect("load the record again")) + .init() + .await + .expect("a second cipher"); + let term_elsewhere = other + .default_keyset() + .equality_term("alice@example.com", context) + .await + .expect("a term in the second cipher"); + assert_eq!( + term.as_bytes(), + term_elsewhere.as_bytes(), + "{mode}: another cipher with the same record derives the same term" + ); + let opened: String = other + .decrypt(sealed, context) + .await + .unwrap_or_else(|error| panic!("{mode}: another cipher must open the value: {error}")); + assert_eq!(opened, "alice@example.com"); + + let ore = keyset + .ore_term(42u64, context) + .await + .unwrap_or_else(|error| panic!("{mode}: an ORE term failed: {error}")); + let ore_again = keyset.ore_term(42u64, context).await.expect("ORE again"); + assert_eq!(ore, ore_again); +} + +/// A value opened under another context is refused. +pub async fn refuses_another_context( + backend: B, + key_ref: &str, + mode: KeyMode, +) { + let (cipher, _, _) = cipher_over(backend, key_ref, mode, |s| s).await; + let sealed = cipher + .default_keyset() + .encrypt_tree("secret".to_string(), nonempty!("users").with("email")) + .await + .expect("seal"); + let result = cipher + .decrypt::(sealed, nonempty!("users").with("phone")) + .await; + assert!( + result.is_err(), + "{mode}: a value opened under another descriptor must be refused" + ); +} + +/// A leaf of another key backend is refused before any key call. +pub async fn refuses_another_key_backend( + backend: B, + key_ref: &str, + mode: KeyMode, +) { + let vendor = backend.vendor(); + let (cipher, _, observer) = cipher_over(backend, key_ref, mode, |s| s).await; + let zerokms = StackCipher::builder() + .kms(FakeDataKeySource::new()) + .init() + .await + .expect("a ZeroKMS cipher"); + let sealed = zerokms + .default_keyset() + .encrypt_tree("zerokms".to_string(), b"ctx") + .await + .expect("seal under ZeroKMS"); + let before = observer.unwraps(); + match cipher.decrypt::(sealed, b"ctx").await { + Err(Error::ForeignBackend { expected, found }) => { + assert_eq!(expected, KeyBackend::Vendor(vendor)); + assert_eq!(found, KeyBackend::ZeroKms); + } + other => panic!("{mode}: a ZeroKMS leaf must be refused, got {other:?}"), + } + assert_eq!( + observer.unwraps(), + before, + "{mode}: refused before any key call" + ); + + // And the other way: a ZeroKMS cipher refuses this vendor's leaf. + let sealed = cipher + .default_keyset() + .encrypt_tree("vendor".to_string(), b"ctx") + .await + .expect("seal under the vendor"); + assert!(matches!( + zerokms.decrypt::(sealed, b"ctx").await, + Err(Error::ForeignBackend { .. }) + )); +} + +fn assert_zerokms_only(result: Result, feature: ZeroKmsFeature) { + match result { + Err(KmsError::ZeroKmsOnly { feature: found, .. }) => assert_eq!(found, feature), + other => panic!("expected a ZeroKmsOnly refusal of {feature}, got {other:?}"), + } +} + +/// ZeroKMS-only inputs are refused, never ignored. +pub async fn refuses_zerokms_features( + backend: B, + key_ref: &str, + mode: KeyMode, +) { + let (cipher, record, _) = cipher_over(backend, key_ref, mode, |s| s).await; + let source = cipher.kms(); + let keyset = Some(record.id()); + let tag = [Context::Tag("tenant".into())]; + assert_zerokms_only( + source + .generate_keys( + vec![GenerateKeyPayload::new("d", Cow::Borrowed(&tag))], + keyset, + None, + ) + .await, + ZeroKmsFeature::Context, + ); + assert_zerokms_only( + source + .generate_keys( + vec![GenerateKeyPayload::new("d", Cow::Owned(vec![])) + .with_decryption_policy(DecryptionPolicy { conditions: vec![] })], + keyset, + None, + ) + .await, + ZeroKmsFeature::DecryptionPolicy, + ); + let mut claims = std::collections::HashMap::new(); + let _ = claims.insert("sub".to_string(), None); + let unverified = UnverifiedContext(claims); + assert_zerokms_only( + source + .retrieve_keys( + vec![RetrieveKeyPayload::new([0u8; 16], "d", b"")], + keyset, + Some(&unverified), + ) + .await, + ZeroKmsFeature::UnverifiedContext, + ); +} + +/// The key lifetime is obeyed. +pub async fn obeys_the_key_lifetime(make: &F, key_ref: &str, mode: KeyMode) +where + B: KeyVendorBackend, + F: Fn() -> B, +{ + let context = nonempty!("users").with("email"); + match mode { + KeyMode::Wrapped => { + let (cipher, record, observer) = cipher_over(make(), key_ref, mode, |s| s).await; + let selected = cipher.keyset(record.id()).await.expect("select by id"); + assert!( + selected.prf().is_none(), + "a wrapped-data-key handle holds no index key" + ); + let after_select = observer.unwraps(); + let named = cipher + .keyset(stack_encrypt::kms::IdentifiedBy::Name( + "contract".to_string().into(), + )) + .await + .expect("select by name"); + assert_eq!(named.keyset_id(), record.id(), "a name resolves locally"); + assert_eq!( + observer.unwraps(), + after_select, + "selection makes no vendor call" + ); + for settle in 1..=3 { + let _ = selected.equality_term("v", context).await.expect("term"); + assert_eq!( + observer.unwraps(), + after_select + settle, + "each settle that builds a term unwraps the index key again" + ); + } + // Two terms in one settle share one unwrap. + let _ = selected + .equality_term("a", context) + .zip(selected.equality_term("b", context)) + .await + .expect("two terms"); + assert_eq!( + observer.unwraps(), + after_select + 4, + "one unwrap per settle" + ); + } + KeyMode::Derived => { + let (cipher, _, observer) = cipher_over(make(), key_ref, mode, |s| s).await; + let keyset = cipher.default_keyset(); + assert!(keyset.prf().is_none(), "the handle holds no index key"); + for _ in 0..3 { + let _ = keyset.equality_term("v", context).await.expect("term"); + let _ = keyset + .encrypt_tree("v".to_string(), context) + .await + .expect("seal"); + } + assert_eq!( + observer.unwraps(), + 1, + "the root key is unwrapped once and kept for its TTL" + ); + + let (cipher, _, observer) = cipher_over(make(), key_ref, mode, |source| { + source.with_root_key_ttl(Duration::ZERO) + }) + .await; + let keyset = cipher.default_keyset(); + for settle in 1..=3 { + let _ = keyset.equality_term("v", context).await.expect("term"); + assert_eq!( + observer.unwraps(), + settle, + "an expired root key is unwrapped again" + ); + } + } + } +} + +/// A batch keeps its order. +pub async fn keeps_batch_order(backend: B, key_ref: &str, mode: KeyMode) { + let (cipher, _, _) = cipher_over(backend, key_ref, mode, |source| { + source.with_batch_options(BatchOptions { + concurrency: std::num::NonZeroUsize::new(4).unwrap_or(std::num::NonZeroUsize::MIN), + ..BatchOptions::default() + }) + }) + .await; + let values: Vec = (0..25).map(|n| format!("value-{n}")).collect(); + let sealed = cipher + .default_keyset() + .encrypt_tree(values.clone(), b"batch") + .await + .unwrap_or_else(|error| panic!("{mode}: sealing a batch failed: {error}")); + let opened: Vec = cipher + .decrypt(sealed, b"batch") + .await + .unwrap_or_else(|error| panic!("{mode}: opening a batch failed: {error}")); + assert_eq!(opened, values, "{mode}: a batch keeps its order"); +} + +/// A vendor with a batch operation makes one call per chunk of wrapped data +/// keys, under wrapped data keys. Nothing to check for another vendor or +/// mode. +pub async fn uses_vendor_batches(backend: B, key_ref: &str, mode: KeyMode) { + let Some(limit) = backend.batch_limit() else { + return; + }; + if mode != KeyMode::Wrapped { + return; + } + let limit = limit.get(); + let (cipher, _, observer) = cipher_over(backend, key_ref, mode, |s| s).await; + let count = 2 * limit + 1; + let values: Vec = (0..count).map(|n| format!("value-{n}")).collect(); + let before = observer.batches(); + let sealed = cipher + .default_keyset() + .encrypt_tree(values.clone(), b"batch") + .await + .unwrap_or_else(|error| panic!("{mode}: sealing in vendor batches failed: {error}")); + assert_eq!( + observer.batches() - before, + 3, + "{mode}: {count} values seal in three batch calls of at most {limit}" + ); + let before = observer.batches(); + let opened: Vec = cipher + .decrypt(sealed, b"batch") + .await + .unwrap_or_else(|error| panic!("{mode}: opening in vendor batches failed: {error}")); + assert_eq!(opened, values, "{mode}: vendor batches keep their order"); + assert_eq!( + observer.batches() - before, + 3, + "{mode}: {count} values open in three batch calls of at most {limit}" + ); +} + +/// A throttled call is retried; one throttled past the last attempt fails +/// the batch. +pub async fn backs_off_when_throttled(make: &F, key_ref: &str, mode: KeyMode) +where + B: KeyVendorBackend, + F: Fn() -> B, +{ + let options = BatchOptions { + max_attempts: 3, + base_delay: Duration::from_millis(1), + ..BatchOptions::default() + }; + let (cipher, _, observer) = cipher_over(make(), key_ref, mode, |source| { + source + .with_batch_options(options.clone()) + .with_root_key_ttl(Duration::ZERO) + }) + .await; + let keyset = cipher.default_keyset(); + // Derived data keys make one call per settle (the root key); wrapped + // data keys one per value. One throttle either way is retried. + observer.throttle_next(1); + let sealed = keyset + .encrypt_tree("v".to_string(), b"ctx") + .await + .unwrap_or_else(|error| panic!("{mode}: a throttled call must be retried: {error}")); + let opened: String = cipher.decrypt(sealed, b"ctx").await.expect("open"); + assert_eq!(opened, "v"); + + if mode == KeyMode::Wrapped { + observer.throttle_next(options.max_attempts as usize); + match keyset.encrypt_tree("v".to_string(), b"ctx").await { + Err(Error::Kms(KmsError::Backend { + kind: BackendErrorKind::Throttled, + .. + })) => {} + other => panic!("{mode}: still throttled must fail the batch, got {other:?}"), + } + observer.throttle_next(0); + } +} + +/// A record refuses a mode the vendor does not offer. +pub async fn refuses_a_mode_it_does_not_offer(backend: B, key_ref: &str) { + for mode in KeyMode::ALL { + if backend.supports_mode(mode) { + continue; + } + match KeysetRecord::create(&backend, key_ref, mode, None).await { + Err(RecordError::ModeUnsupported { mode: refused, .. }) => assert_eq!(refused, mode), + other => { + panic!("a record in an unsupported mode {mode} must be refused, got {other:?}") + } + } + } +} diff --git a/packages/stack-kms-vendor/src/derive.rs b/packages/stack-kms-vendor/src/derive.rs new file mode 100644 index 0000000000..f2131f03b7 --- /dev/null +++ b/packages/stack-kms-vendor/src/derive.rs @@ -0,0 +1,369 @@ +//! The key derivations and the associated data of a key vendor backend. +//! +//! These bytes are a storage commitment: a stored value opens only if its +//! data key derives, or unwraps, exactly as it did when it was sealed. The +//! tests pin them. +//! +//! # Framing +//! +//! Every label, info and associated data is a list of byte strings framed +//! with [`frame`]: the count of pieces, then each piece as its length and +//! its bytes, every number a `u64` little-endian. The framing is +//! prefix-free, so two different lists never give the same bytes. +//! +//! # Derived data keys +//! +//! * data key = `HKDF-SHA256(ikm = root key, salt = iv, info = +//! frame[DATA_KEY_LABEL, keyset id, descriptor])`, 32 bytes. +//! * index key = `HKDF-SHA256(ikm = root key, salt = none, info = +//! frame[INDEX_KEY_LABEL, keyset id])`, 32 bytes. +//! +//! The iv is 16 random bytes per value, stored in the leaf, and the leaf's +//! tag is empty. +//! +//! # Associated data +//! +//! * The wrapped key in a keyset record (the root key under derived data +//! keys, the index key under wrapped data keys) is wrapped with +//! `frame[RECORD_KEY_LABEL, purpose, keyset id, vendor byte, mode byte]`. +//! A record whose id, vendor or mode was changed does not unwrap. +//! * A wrapped data key is wrapped with `frame[WRAPPED_DATA_KEY_LABEL, +//! keyset id, iv, descriptor]`. A wrapped data key moved to another +//! keyset, value or field does not unwrap. + +use hkdf::Hkdf; +use sha2::Sha256; +use stack_kms::{KeyMode, KeyVendor}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; +use zeroize::Zeroizing; + +use crate::vendor::KeyBytes; + +/// The HKDF info label of a derived data key. +pub const DATA_KEY_LABEL: &[u8] = b"stack-kms-vendor/derived-data-key/v1"; +/// The HKDF info label of a derived index key. +pub const INDEX_KEY_LABEL: &[u8] = b"stack-kms-vendor/derived-index-key/v1"; +/// The associated-data label of the key a keyset record wraps. +pub const RECORD_KEY_LABEL: &[u8] = b"stack-kms-vendor/keyset-record-key/v1"; +/// The associated-data label of a wrapped data key. +pub const WRAPPED_DATA_KEY_LABEL: &[u8] = b"stack-kms-vendor/wrapped-data-key/v1"; + +/// Frame a list of byte strings without ambiguity (see the module docs). +pub fn frame(pieces: &[&[u8]]) -> Vec { + let len = 8 + pieces.iter().map(|piece| 8 + piece.len()).sum::(); + let mut out = Vec::with_capacity(len); + out.extend_from_slice(&(pieces.len() as u64).to_le_bytes()); + for piece in pieces { + out.extend_from_slice(&(piece.len() as u64).to_le_bytes()); + out.extend_from_slice(piece); + } + out +} + +/// What the key in a keyset record is, by key mode: the root key under +/// derived data keys, the index key under wrapped data keys. +pub fn record_key_purpose(mode: KeyMode) -> &'static [u8] { + match mode { + KeyMode::Derived => b"root-key", + KeyMode::Wrapped => b"index-key", + } +} + +/// The associated data the key in a keyset record is wrapped with. +pub fn record_key_aad(keyset_id: Uuid, vendor: KeyVendor, mode: KeyMode) -> Vec { + frame(&[ + RECORD_KEY_LABEL, + record_key_purpose(mode), + keyset_id.as_bytes(), + &[vendor.byte()], + &[mode.byte()], + ]) +} + +/// The associated data a wrapped data key is wrapped with. +pub fn wrapped_data_key_aad(keyset_id: Uuid, iv: &[u8; 16], descriptor: &str) -> Vec { + frame(&[ + WRAPPED_DATA_KEY_LABEL, + keyset_id.as_bytes(), + iv, + descriptor.as_bytes(), + ]) +} + +/// Split bytes made by [`frame`] back into their pieces. `None` if the +/// bytes are not exactly one frame. +pub fn unframe(bytes: &[u8]) -> Option> { + fn take_u64(bytes: &[u8]) -> Option<(usize, &[u8])> { + let (head, rest) = bytes.split_at_checked(8)?; + let value = u64::from_le_bytes(head.try_into().ok()?); + Some((usize::try_from(value).ok()?, rest)) + } + let (count, mut rest) = take_u64(bytes)?; + // Each piece takes at least 8 bytes, so a larger count cannot fit. + if count > rest.len() / 8 { + return None; + } + let mut pieces = Vec::with_capacity(count); + for _ in 0..count { + let (len, after) = take_u64(rest)?; + let (piece, after) = after.split_at_checked(len)?; + pieces.push(piece); + rest = after; + } + rest.is_empty().then_some(pieces) +} + +/// What a vendor may show of the associated data it is given, for audit +/// logs and access policies: the vendor's own context fields (AWS KMS +/// encryption context, for example). See [`describe_aad`]. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct AadDescription<'a> { + /// What the wrapped key is: `root-key` or `index-key` (the key in a + /// keyset record), or `data-key` (a wrapped data key). + pub purpose: &'a str, + /// The keyset the key belongs to. + pub keyset_id: Uuid, + /// The descriptor of the value, for a wrapped data key. `None` for the + /// key in a keyset record. + pub descriptor: Option<&'a str>, +} + +/// Describe associated data made by [`record_key_aad`] or +/// [`wrapped_data_key_aad`]. `None` for any other bytes. +/// +/// The description is for a vendor that shows its associated data as +/// fields. It does not replace the associated data: the vendor must still +/// bind all of `aad` (see +/// [`KeyVendorBackend`](crate::KeyVendorBackend)), because the description +/// leaves out the iv, the vendor byte and the mode byte. +pub fn describe_aad(aad: &[u8]) -> Option> { + let pieces = unframe(aad)?; + let keyset_id = |bytes: &[u8]| Uuid::from_slice(bytes).ok(); + match pieces.as_slice() { + [label, purpose, id, vendor, mode] + if *label == RECORD_KEY_LABEL && vendor.len() == 1 && mode.len() == 1 => + { + let purpose = match *purpose { + b"root-key" => "root-key", + b"index-key" => "index-key", + _ => return None, + }; + Some(AadDescription { + purpose, + keyset_id: keyset_id(id)?, + descriptor: None, + }) + } + [label, id, iv, descriptor] if *label == WRAPPED_DATA_KEY_LABEL && iv.len() == 16 => { + Some(AadDescription { + purpose: "data-key", + keyset_id: keyset_id(id)?, + descriptor: Some(std::str::from_utf8(descriptor).ok()?), + }) + } + _ => None, + } +} + +/// `HKDF-SHA256(ikm, salt, info)`, 32 bytes, into a key that wipes itself. +fn hkdf(ikm: &KeyBytes, salt: Option<&[u8]>, info: &[u8]) -> KeyBytes { + let hkdf = Hkdf::::new(salt, ikm.risky_ref()); + let mut okm = Zeroizing::new([0u8; 32]); + // 32 bytes is far below HKDF-SHA256's limit of 255 * 32. + #[allow(clippy::expect_used)] + hkdf.expand(info, okm.as_mut()) + .expect("32 bytes is a valid HKDF-SHA256 output length"); + Protected::new(*okm) +} + +/// The data key of one value under derived data keys. +pub fn derived_data_key( + root: &KeyBytes, + keyset_id: Uuid, + iv: &[u8; 16], + descriptor: &str, +) -> KeyBytes { + let info = frame(&[DATA_KEY_LABEL, keyset_id.as_bytes(), descriptor.as_bytes()]); + hkdf(root, Some(iv), &info) +} + +/// The index key of a keyset under derived data keys. +pub fn derived_index_key(root: &KeyBytes, keyset_id: Uuid) -> KeyBytes { + let info = frame(&[INDEX_KEY_LABEL, keyset_id.as_bytes()]); + hkdf(root, None, &info) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn hex(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02x}")).collect() + } + + /// RFC 5869 test case 1, cut to the first 32 bytes of output (one + /// HMAC block, which is all a 32-byte key uses). Proves the HKDF here + /// is plain HKDF-SHA256. + #[test] + fn hkdf_matches_rfc_5869_test_case_1() { + let ikm = Protected::new([0x0bu8; 22]); + let hk = Hkdf::::new( + Some(&[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]), + ikm.risky_ref(), + ); + let mut okm = [0u8; 32]; + hk.expand( + &[0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7, 0xf8, 0xf9], + &mut okm, + ) + .unwrap(); + assert_eq!( + hex(&okm), + "3cb25f25faacd57a90434f64d0362f2a2d2d0a90cf1a5a4c5db02d56ecc4c5bf" + ); + } + + #[test] + fn framing_is_prefix_free() { + assert_ne!(frame(&[b"ab", b"c"]), frame(&[b"a", b"bc"])); + assert_ne!(frame(&[b"abc"]), frame(&[b"abc", b""])); + assert_eq!(hex(&frame(&[b"a"])), "0100000000000000010000000000000061"); + } + + /// The derivations are a storage commitment: these pins fail if one + /// moves. + #[test] + fn the_derivations_are_pinned() { + let root = Protected::new([7u8; 32]); + let id = Uuid::from_u128(0x0102_0304_0506_0708_090a_0b0c_0d0e_0f10); + let data = derived_data_key(&root, id, &[9u8; 16], "users/email"); + let index = derived_index_key(&root, id); + assert_eq!( + hex(data.risky_ref()), + hex(derived_data_key(&root, id, &[9u8; 16], "users/email").risky_ref()) + ); + assert_ne!(data.risky_ref(), index.risky_ref()); + assert_eq!( + hex(data.risky_ref()), + "93a33099260b4c29872102658969dfe9f28622e17875f26afdd5d1895f638c83" + ); + assert_eq!( + hex(index.risky_ref()), + "80fd8f8c664080afae35d02ebb8b34cb1f6c90643dbd83fec572c0686f8eee25" + ); + } + + #[test] + fn every_input_changes_the_data_key() { + let root = Protected::new([7u8; 32]); + let id = Uuid::from_u128(1); + let base = derived_data_key(&root, id, &[1u8; 16], "d"); + let others = [ + derived_data_key(&Protected::new([8u8; 32]), id, &[1u8; 16], "d"), + derived_data_key(&root, Uuid::from_u128(2), &[1u8; 16], "d"), + derived_data_key(&root, id, &[2u8; 16], "d"), + derived_data_key(&root, id, &[1u8; 16], "e"), + ]; + for other in others { + assert_ne!(base.risky_ref(), other.risky_ref()); + } + assert_ne!( + derived_index_key(&root, id).risky_ref(), + derived_index_key(&root, Uuid::from_u128(2)).risky_ref() + ); + } + + #[test] + fn unframe_reverses_frame_and_refuses_anything_else() { + let pieces: [&[u8]; 3] = [b"ab", b"", b"cde"]; + let framed = frame(&pieces); + assert_eq!(unframe(&framed).unwrap(), pieces); + assert_eq!(unframe(&frame(&[])).unwrap(), Vec::<&[u8]>::new()); + let mut long = framed.clone(); + long.push(0); + assert_eq!(unframe(&long), None); + assert_eq!(unframe(&framed[..framed.len() - 1]), None); + assert_eq!(unframe(&[1, 0, 0]), None); + assert_eq!(unframe(&u64::MAX.to_le_bytes()), None); + } + + #[test] + fn describe_aad_names_the_purpose_keyset_and_descriptor() { + let id = Uuid::from_u128(7); + assert_eq!( + describe_aad(&record_key_aad(id, KeyVendor::Aws, KeyMode::Derived)), + Some(AadDescription { + purpose: "root-key", + keyset_id: id, + descriptor: None + }) + ); + assert_eq!( + describe_aad(&record_key_aad(id, KeyVendor::Aws, KeyMode::Wrapped)) + .unwrap() + .purpose, + "index-key" + ); + assert_eq!( + describe_aad(&wrapped_data_key_aad(id, &[1u8; 16], "users/email")), + Some(AadDescription { + purpose: "data-key", + keyset_id: id, + descriptor: Some("users/email") + }) + ); + assert_eq!(describe_aad(b"not a frame"), None); + assert_eq!(describe_aad(&frame(&[b"other-label", id.as_bytes()])), None); + assert_eq!( + describe_aad(&frame(&[RECORD_KEY_LABEL, b"x", id.as_bytes(), &[1], &[1]])), + None + ); + assert_eq!( + describe_aad(&frame(&[ + WRAPPED_DATA_KEY_LABEL, + b"short", + &[1u8; 16], + b"d" + ])), + None + ); + assert_eq!( + describe_aad(&frame(&[ + WRAPPED_DATA_KEY_LABEL, + id.as_bytes(), + &[1u8; 15], + b"d" + ])), + None + ); + assert_eq!( + describe_aad(&frame(&[ + WRAPPED_DATA_KEY_LABEL, + id.as_bytes(), + &[1u8; 16], + &[0xff] + ])), + None + ); + } + + #[test] + fn the_record_aad_binds_every_field() { + let id = Uuid::from_u128(1); + let base = record_key_aad(id, KeyVendor::Aws, KeyMode::Derived); + assert_ne!( + base, + record_key_aad(Uuid::from_u128(2), KeyVendor::Aws, KeyMode::Derived) + ); + assert_ne!(base, record_key_aad(id, KeyVendor::Vault, KeyMode::Derived)); + assert_ne!(base, record_key_aad(id, KeyVendor::Aws, KeyMode::Wrapped)); + let data = wrapped_data_key_aad(id, &[1u8; 16], "d"); + assert_ne!(data, wrapped_data_key_aad(id, &[2u8; 16], "d")); + assert_ne!(data, wrapped_data_key_aad(id, &[1u8; 16], "e")); + assert_ne!( + data, + wrapped_data_key_aad(Uuid::from_u128(2), &[1u8; 16], "d") + ); + } +} diff --git a/packages/stack-kms-vendor/src/error.rs b/packages/stack-kms-vendor/src/error.rs new file mode 100644 index 0000000000..7d73376992 --- /dev/null +++ b/packages/stack-kms-vendor/src/error.rs @@ -0,0 +1,188 @@ +//! Errors from keyset records and from building a key vendor backend. + +use stack_kms::{diagnostic::payload, ErrorPayload, KeyMode, KeyVendor}; +use uuid::Uuid; + +/// A keyset record could not be parsed, created or loaded. +/// +/// No message carries key material or a wrapped key. A parse error names +/// what is wrong with the record, not the record. +#[derive(Debug, thiserror::Error, miette::Diagnostic)] +#[non_exhaustive] +pub enum RecordError { + /// The text does not start with [`RECORD_PREFIX`](crate::RECORD_PREFIX). + #[error("This is not a keyset record: it does not start with `stack-keyset:`")] + #[diagnostic( + code(stack_kms_vendor::record_prefix), + help("Copy the whole keyset record, as `stash keyset create` printed it.") + )] + NotARecord, + + /// The text after the prefix is not unpadded base64url. + #[error("The keyset record is not valid base64url")] + #[diagnostic(code(stack_kms_vendor::record_encoding))] + Encoding, + + /// The record has a format version this build does not know. + #[error("The keyset record has format version {0}, which this build does not know")] + #[diagnostic( + code(stack_kms_vendor::record_version), + help("Upgrade to a release that reads this record version.") + )] + UnknownVersion(u8), + + /// The record names a key backend this build does not know. + #[error("The keyset record names key backend {0}, which this build does not know")] + #[diagnostic(code(stack_kms_vendor::record_backend))] + UnknownBackend(u8), + + /// The record names a key mode this build does not know. + #[error("The keyset record names key mode {0}, which this build does not know")] + #[diagnostic(code(stack_kms_vendor::record_mode))] + UnknownMode(u8), + + /// The record is shorter than its fields say, or has bytes after them. + #[error("The keyset record is damaged: its length does not match its fields")] + #[diagnostic(code(stack_kms_vendor::record_length))] + Length, + + /// A field holds a value it may not hold. + #[error("The keyset record field `{field}` is not valid")] + #[diagnostic( + code(stack_kms_vendor::record_field), + help("A name is 1 to 255 bytes of UTF-8. A vendor key reference and a wrapped key are 1 to 65535 bytes.") + )] + InvalidField { + /// The field. + field: &'static str, + }, + + /// The key vendor backend does not offer this key mode. The static key + /// backend offers derived data keys only. + #[error("Key backend {vendor} does not offer the {mode} key mode")] + #[diagnostic( + code(stack_kms_vendor::mode_unsupported), + help("Use the derived key mode with the static key backend.") + )] + ModeUnsupported { + /// The vendor. + vendor: KeyVendor, + /// The mode it refused. + mode: KeyMode, + }, + + /// A record names another key vendor backend than the one it was given + /// to. + #[error("The keyset record {keyset} names key backend {found}, not {expected}")] + #[diagnostic(code(stack_kms_vendor::foreign_vendor))] + ForeignVendor { + /// The keyset. + keyset: Uuid, + /// The vendor of the backend. + expected: KeyVendor, + /// The vendor the record names. + found: KeyVendor, + }, + + /// Two records name the same keyset id. + #[error("Two keyset records have the id {0}")] + #[diagnostic(code(stack_kms_vendor::duplicate_keyset))] + DuplicateKeyset(Uuid), + + /// Two records have the same name. + #[error("Two keyset records have the same name")] + #[diagnostic( + code(stack_kms_vendor::duplicate_name), + help("A keyset name resolves locally, so it must be unique among the loaded records.") + )] + DuplicateName, + + /// The records name more than one key vendor backend. A cipher holds + /// ZeroKMS and at most one key vendor backend (ADR-0009). + #[error("The keyset records name two key vendor backends, {first} and {second}")] + #[diagnostic( + code(stack_kms_vendor::second_vendor), + help("A cipher holds ZeroKMS and at most one key vendor backend. Load the records of one vendor.") + )] + SecondVendor { + /// The vendor named first. + first: KeyVendor, + /// Another vendor. + second: KeyVendor, + }, + + /// A rewrap would move a record in the wrapped data key mode to another + /// vendor key. Each stored value's data key stays wrapped under the key + /// that sealed it, so the values sealed before the move would not open. + /// A new version of the same key is allowed, and so is any key in the + /// derived data key mode. + #[error( + "The keyset {keyset} uses wrapped data keys, so its record cannot move to another vendor key" + )] + #[diagnostic( + code(stack_kms_vendor::rewrap_another_key), + help( + "Each stored value's data key stays wrapped under the vendor key that sealed it. \ + Rewrap under a new version of the same key, or create a new keyset under the other key \ + and re-encrypt the data into it." + ) + )] + RewrapToAnotherKey { + /// The keyset. + keyset: Uuid, + /// The record's key reference. + from: String, + /// The key reference the rewrap named. + to: String, + }, + + /// The default keyset is not one of the loaded records. + #[error("The default keyset is not one of the loaded keyset records")] + #[diagnostic(code(stack_kms_vendor::unknown_default))] + UnknownDefault, + + /// The vendor call that created or rewrapped the record failed. + #[error(transparent)] + #[diagnostic(transparent)] + Kms(#[from] stack_kms::Error), +} + +impl ErrorPayload for RecordError { + fn payload(&self) -> serde_json::Map { + match self { + Self::UnknownVersion(byte) | Self::UnknownBackend(byte) | Self::UnknownMode(byte) => { + payload([("byte", (*byte).into())]) + } + Self::InvalidField { field } => payload([("field", (*field).into())]), + Self::ModeUnsupported { vendor, mode } => payload([ + ("vendor", vendor.name().into()), + ("mode", mode.name().into()), + ]), + Self::ForeignVendor { + keyset, + expected, + found, + } => payload([ + ("keyset", keyset.to_string().into()), + ("expected", expected.name().into()), + ("found", found.name().into()), + ]), + Self::DuplicateKeyset(keyset) => payload([("keyset", keyset.to_string().into())]), + Self::RewrapToAnotherKey { keyset, from, to } => payload([ + ("keyset", keyset.to_string().into()), + ("from", from.as_str().into()), + ("to", to.as_str().into()), + ]), + Self::SecondVendor { first, second } => payload([ + ("first", first.name().into()), + ("second", second.name().into()), + ]), + Self::Kms(error) => error.payload(), + Self::NotARecord + | Self::Encoding + | Self::Length + | Self::DuplicateName + | Self::UnknownDefault => serde_json::Map::new(), + } + } +} diff --git a/packages/stack-kms-vendor/src/fake.rs b/packages/stack-kms-vendor/src/fake.rs new file mode 100644 index 0000000000..c9c945dada --- /dev/null +++ b/packages/stack-kms-vendor/src/fake.rs @@ -0,0 +1,171 @@ +//! An in-memory key vendor backend for tests. + +use std::num::NonZeroUsize; +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::Mutex; +use std::time::Duration; + +use stack_kms::{BackendErrorKind, Error, KeyVendor}; +use vitaminc::protected::Protected; + +use crate::static_key::StaticKeyVendor; +use crate::vendor::{KeyBytes, KeyVendorBackend}; + +/// How many calls a [`FakeVendor`] has answered, by operation. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub struct FakeVendorCalls { + /// Calls to [`wrap`](KeyVendorBackend::wrap), `generate_wrapped` + /// included. + pub wrap: usize, + /// Calls to [`unwrap`](KeyVendorBackend::unwrap). + pub unwrap: usize, + /// Calls refused as throttled. + pub throttled: usize, +} + +/// An in-memory key vendor backend that offers both key modes, for tests +/// and for the [contract suite](crate::contract). +/// +/// It wraps with AES-256-GCM under in-memory vendor keys, as +/// [`StaticKeyVendor`] does, but reports itself as the vendor it was built +/// with, counts its calls, can refuse the next calls as throttled, and +/// records the waits the adapter asks for. It does no I/O. +pub struct FakeVendor { + vendor: KeyVendor, + inner: StaticKeyVendor, + wraps: AtomicUsize, + unwraps: AtomicUsize, + throttled: AtomicUsize, + throttle_next: AtomicUsize, + sleeps: Mutex>, + batch_limit: Option, +} + +impl std::fmt::Debug for FakeVendor { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("FakeVendor") + .field("vendor", &self.vendor) + .field("calls", &self.calls()) + .finish() + } +} + +impl FakeVendor { + /// The vendor key reference every fake vendor holds. + pub const KEY_REF: &'static str = "fake-key/v1"; + + /// A fake that reports itself as `vendor`, with one vendor key, + /// [`KEY_REF`](Self::KEY_REF). + pub fn new(vendor: KeyVendor) -> Self { + Self { + vendor, + inner: StaticKeyVendor::new(Self::KEY_REF, Protected::new([0x5au8; 32])), + wraps: AtomicUsize::new(0), + unwraps: AtomicUsize::new(0), + throttled: AtomicUsize::new(0), + throttle_next: AtomicUsize::new(0), + sleeps: Mutex::new(Vec::new()), + batch_limit: None, + } + } + + /// Offer batch calls of at most `limit` keys + /// ([`KeyVendorBackend::batch_limit`]). Each batch call wraps or unwraps + /// its keys one at a time (the trait's default), so the counts stay per + /// key. + pub fn with_batch_limit(mut self, limit: NonZeroUsize) -> Self { + self.batch_limit = Some(limit); + self + } + + /// Add another vendor key, for rewrap tests. + pub fn with_key(mut self, key_ref: &str, key: KeyBytes) -> Self { + self.inner = self.inner.with_key(key_ref, key); + self + } + + /// Refuse the next `count` calls as throttled. + pub fn throttle_next(&self, count: usize) { + self.throttle_next.store(count, Ordering::SeqCst); + } + + /// The calls answered so far. + pub fn calls(&self) -> FakeVendorCalls { + FakeVendorCalls { + wrap: self.wraps.load(Ordering::SeqCst), + unwrap: self.unwraps.load(Ordering::SeqCst), + throttled: self.throttled.load(Ordering::SeqCst), + } + } + + /// The waits the adapter asked for, in order. + pub fn sleeps(&self) -> Vec { + self.sleeps + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .clone() + } + + /// Refuse this call as throttled if a throttle is pending. + fn throttle(&self) -> Result<(), Error> { + let pending = self + .throttle_next + .fetch_update(Ordering::SeqCst, Ordering::SeqCst, |n| n.checked_sub(1)); + if pending.is_ok() { + let _ = self.throttled.fetch_add(1, Ordering::SeqCst); + return Err(Error::Backend { + vendor: self.vendor.name(), + kind: BackendErrorKind::Throttled, + source: "the fake vendor is throttling".into(), + }); + } + Ok(()) + } + + /// Report the inner static vendor's failure as this vendor's. + fn rename(&self, error: Error) -> Error { + match error { + Error::Backend { kind, source, .. } => Error::Backend { + vendor: self.vendor.name(), + kind, + source, + }, + other => other, + } + } +} + +impl KeyVendorBackend for FakeVendor { + fn vendor(&self) -> KeyVendor { + self.vendor + } + + async fn wrap(&self, key_ref: &str, key: &KeyBytes, aad: &[u8]) -> Result, Error> { + self.throttle()?; + let _ = self.wraps.fetch_add(1, Ordering::SeqCst); + self.inner + .wrap(key_ref, key, aad) + .await + .map_err(|error| self.rename(error)) + } + + async fn unwrap(&self, key_ref: &str, wrapped: &[u8], aad: &[u8]) -> Result { + self.throttle()?; + let _ = self.unwraps.fetch_add(1, Ordering::SeqCst); + self.inner + .unwrap(key_ref, wrapped, aad) + .await + .map_err(|error| self.rename(error)) + } + + fn batch_limit(&self) -> Option { + self.batch_limit + } + + async fn sleep(&self, duration: Duration) { + self.sleeps + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner) + .push(duration); + } +} diff --git a/packages/stack-kms-vendor/src/http.rs b/packages/stack-kms-vendor/src/http.rs new file mode 100644 index 0000000000..74ee23e584 --- /dev/null +++ b/packages/stack-kms-vendor/src/http.rs @@ -0,0 +1,239 @@ +//! The native [`VendorTransport`]: HTTPS through reqwest. +//! +//! It uses the same HTTP client and TLS configuration as stack-kms's +//! ZeroKMS transport: reqwest with rustls, and the platform verifier with +//! the compromised-CA blocklist (`stack_auth::tls::client_config`). Native +//! only. The WASI guest sends through its `transport_send` host import. + +use std::time::Duration; + +use zeroize::Zeroizing; + +use crate::transport::{TransportError, VendorRequest, VendorResponse, VendorTransport}; + +/// The default time limit for one request, connection included. +pub const DEFAULT_REQUEST_TIMEOUT: Duration = Duration::from_secs(30); + +/// The native [`VendorTransport`]: HTTPS (or plain HTTP, for a local +/// emulator or an instance metadata service) through reqwest. +/// +/// Clone it freely: clones share one connection pool. +#[derive(Clone, Debug)] +pub struct HttpVendorTransport { + client: reqwest::Client, +} + +/// Options for [`HttpVendorTransport`]. +#[derive(Clone, Debug)] +pub struct HttpVendorTransportOptions { + /// The time limit for one request, connection included. Default + /// [`DEFAULT_REQUEST_TIMEOUT`]. + pub timeout: Duration, + /// The time limit to open a connection. `None` keeps the operating + /// system's limit. Set a short one for an instance metadata service, + /// which does not answer off the cloud. + pub connect_timeout: Option, +} + +impl Default for HttpVendorTransportOptions { + fn default() -> Self { + Self { + timeout: DEFAULT_REQUEST_TIMEOUT, + connect_timeout: None, + } + } +} + +impl HttpVendorTransport { + /// A transport with the default options. + pub fn new() -> Result { + Self::with_options(HttpVendorTransportOptions::default()) + } + + /// A transport with `options`. + pub fn with_options(options: HttpVendorTransportOptions) -> Result { + let tls = stack_auth::tls::client_config(false) + .map_err(|error| TransportError(format!("TLS configuration failed: {error}")))?; + let mut builder = reqwest::ClientBuilder::new() + .user_agent(concat!("stack-kms-vendor/", env!("CARGO_PKG_VERSION"))) + .tls_backend_preconfigured(tls) + .timeout(options.timeout); + if let Some(connect_timeout) = options.connect_timeout { + builder = builder.connect_timeout(connect_timeout); + } + let client = builder + .build() + .map_err(|error| TransportError(format!("HTTP client failed to build: {error}")))?; + Ok(Self { client }) + } +} + +impl VendorTransport for HttpVendorTransport { + async fn send(&self, request: VendorRequest) -> Result { + let method = reqwest::Method::from_bytes(request.method.as_bytes()) + .map_err(|_| TransportError(format!("invalid HTTP method {:?}", request.method)))?; + let mut builder = self.client.request(method, request.url.as_str()); + for (name, value) in &request.headers { + builder = builder.header(name.as_str(), value.as_str()); + } + // reqwest takes the body by value: the copy it holds is not wiped. + // The `Zeroizing` copy in `request` is wiped when it drops. + let response = builder + .body(request.body.to_vec()) + .send() + .await + .map_err(|error| TransportError(transport_failure(&error)))?; + let status = response.status().as_u16(); + let headers = response + .headers() + .iter() + .filter_map(|(name, value)| { + value + .to_str() + .ok() + .map(|value| (name.as_str().to_owned(), value.to_owned())) + }) + .collect(); + let body = response + .bytes() + .await + .map_err(|error| TransportError(transport_failure(&error)))?; + Ok(VendorResponse { + status, + headers, + body: Zeroizing::new(body.to_vec()), + }) + } + + async fn sleep(&self, duration: Duration) { + tokio::time::sleep(duration).await; + } +} + +/// The text of a reqwest failure, without the URL: a URL can hold a +/// credential in its query. +fn transport_failure(error: &reqwest::Error) -> String { + let kind = if error.is_timeout() { + "timed out" + } else if error.is_connect() { + "could not connect" + } else if error.is_body() || error.is_decode() { + "could not read the response" + } else { + "failed" + }; + let mut text = format!("the request {kind}"); + // reqwest's own text names the URL; only its causes are kept. + let mut source = std::error::Error::source(error); + while let Some(cause) = source { + text.push_str(": "); + text.push_str(&cause.to_string()); + source = cause.source(); + } + text +} + +#[cfg(test)] +mod tests { + use tokio::io::{AsyncReadExt, AsyncWriteExt}; + use tokio::net::TcpListener; + + use super::*; + + /// One HTTP/1.1 exchange on a local socket: read the request, answer + /// with `response`, return what was read. + async fn serve_once(listener: TcpListener, response: &'static [u8]) -> String { + let (mut socket, _) = listener.accept().await.unwrap(); + let mut received = Vec::new(); + let mut buffer = [0u8; 4096]; + loop { + let n = socket.read(&mut buffer).await.unwrap(); + received.extend_from_slice(&buffer[..n]); + let text = String::from_utf8_lossy(&received); + if let Some(end) = text.find("\r\n\r\n") { + let length = text[..end] + .lines() + .find_map(|line| { + let (name, value) = line.split_once(':')?; + name.eq_ignore_ascii_case("content-length") + .then(|| value.trim().parse::().ok())? + }) + .unwrap_or(0); + if received.len() >= end + 4 + length { + break; + } + } + } + socket.write_all(response).await.unwrap(); + socket.shutdown().await.unwrap(); + String::from_utf8(received).unwrap() + } + + #[tokio::test] + async fn sends_the_request_and_returns_any_status() { + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + let server = tokio::spawn(serve_once( + listener, + b"HTTP/1.1 400 Bad Request\r\ncontent-type: application/x-amz-json-1.1\r\ncontent-length: 2\r\n\r\n{}", + )); + let transport = HttpVendorTransport::new().unwrap(); + let response = transport + .send( + VendorRequest::new("POST", format!("http://{address}/")) + .header("x-amz-target", "TrentService.Decrypt") + .body(b"{\"a\":1}".to_vec()), + ) + .await + .unwrap(); + assert_eq!(response.status, 400); + assert_eq!( + response.header_value("content-type"), + Some("application/x-amz-json-1.1") + ); + assert_eq!(response.body.as_slice(), b"{}"); + let received = server.await.unwrap(); + assert!(received.starts_with("POST / HTTP/1.1\r\n")); + assert!(received.contains("x-amz-target: TrentService.Decrypt\r\n")); + assert!(received.ends_with("{\"a\":1}")); + } + + #[tokio::test] + async fn a_refused_connection_is_a_transport_error() { + let listener = TcpListener::bind("127.0.0.1:0").await.unwrap(); + let address = listener.local_addr().unwrap(); + drop(listener); + let transport = HttpVendorTransport::with_options(HttpVendorTransportOptions { + connect_timeout: Some(Duration::from_secs(1)), + ..HttpVendorTransportOptions::default() + }) + .unwrap(); + let error = transport + .send(VendorRequest::new( + "GET", + format!("http://{address}/?secret=1"), + )) + .await + .unwrap_err(); + assert!(error.0.contains("could not connect"), "{}", error.0); + assert!(!error.0.contains("secret"), "{}", error.0); + } + + #[tokio::test] + async fn an_invalid_method_is_refused() { + let transport = HttpVendorTransport::new().unwrap(); + let error = transport + .send(VendorRequest::new("BAD METHOD", "http://127.0.0.1:1/")) + .await + .unwrap_err(); + assert!(error.0.contains("invalid HTTP method")); + } + + #[tokio::test] + async fn sleep_waits() { + let transport = HttpVendorTransport::new().unwrap(); + let start = std::time::Instant::now(); + transport.sleep(Duration::from_millis(20)).await; + assert!(start.elapsed() >= Duration::from_millis(20)); + } +} diff --git a/packages/stack-kms-vendor/src/lib.rs b/packages/stack-kms-vendor/src/lib.rs new file mode 100644 index 0000000000..78fdf1d5ee --- /dev/null +++ b/packages/stack-kms-vendor/src/lib.rs @@ -0,0 +1,270 @@ +#![doc(html_favicon_url = "https://cipherstash.com/favicon.ico")] +//! Key vendor backends for Stack Encrypt (ADR-0009, ADR-0010, ADR-0011). +//! +//! A **key vendor backend** is a key backend that is not ZeroKMS: AWS KMS, +//! Azure Key Vault, HashiCorp Vault Transit, Google Cloud KMS, or a static +//! key. This crate holds what every vendor shares, so that a vendor crate +//! implements one small trait and nothing else: +//! +//! * [`KeyVendorBackend`]: the vendor seam. A vendor wraps and unwraps a +//! key under its vendor key, with associated data. That is all. +//! * [`KeysetRecord`]: the customer-held description of a keyset in a key +//! vendor backend. It is an opaque, versioned string that the customer +//! keeps in config or in an environment variable. +//! * [`VendorKeySource`]: the adapter that turns one vendor and its keyset +//! records into a [`DataKeySource`](stack_kms::DataKeySource) and an +//! [`IndexKeySource`](stack_kms::IndexKeySource), in either key mode. +//! Vendor crates never implement the stack-kms traits themselves. +//! * [`KeyRouter`]: ZeroKMS and at most one key vendor backend in one key +//! source, routed by keyset. +//! * [`StaticKeyVendor`]: the static key backend, for development or for +//! customers who manage the key themselves. Derived data keys only. +//! * [`bound`]: associated data bound by us, for a vendor key that has +//! none of its own. The Vault Transit and Azure Key Vault clients use it +//! for every key. +//! * [`run_batch`]: N vendor calls with a concurrency limit, backoff on +//! [`Throttled`](stack_kms::BackendErrorKind::Throttled), and +//! fail-whole-batch. +//! * [`VendorTransport`]: the HTTP seam a vendor client sends through, of +//! the same shape as [`ZeroKMSConnection`](stack_kms::ZeroKMSConnection), +//! so the same vendor code runs on native Rust and in the WASI guest. +//! `HttpVendorTransport` (feature `http`, native only) is the native +//! implementation; the WASI guest's lives in `stack-guest-abi`. +//! +//! # The two key modes +//! +//! * **Wrapped data key** ([`KeyMode::Wrapped`](stack_kms::KeyMode::Wrapped)): +//! each value has its own data key, generated and wrapped by the vendor +//! key and stored in the leaf's tag. The keyset record holds the wrapped +//! index key, and the cipher unwraps it inside each settle that needs it. +//! No key outlives one settle. +//! * **Derived data key** ([`KeyMode::Derived`](stack_kms::KeyMode::Derived)): +//! the keyset record holds the wrapped root key. The adapter unwraps it +//! and keeps it for a TTL ([`DEFAULT_ROOT_KEY_TTL`]). Each data key is +//! `HKDF-SHA256(root key, salt = iv, info = keyset id ‖ descriptor)`, and +//! the index key is derived from the root key under a separate label. +//! +//! See [`derive`](mod@derive) for the exact derivations and associated data. +//! +//! # Capability matrix +//! +//! A key vendor backend works at less than ZeroKMS parity. This section +//! states each difference. It describes the code of this crate, of the +//! vendor crates (`stack-kms-aws`, `stack-kms-gcp`, `stack-kms-azure`, +//! `stack-kms-vault`) and of the Go module, whose WASI guest runs the same +//! vendor code. +//! +//! **The trade-off.** In wrapped data key mode, no key outlives the settle +//! that needed it. In derived data key mode, the root key is cached for the +//! TTL. In both modes, Postgres holds no usable key material. +//! +//! **Latency.** Under wrapped data keys, each separate settle costs vendor +//! round trips: one call for each value it seals or opens, and one more for +//! the index key when it derives a search term. Two queries in two settles +//! cost two index key unwraps. To pay once, compose the terms and values +//! into one settle: `stack_encrypt::all`, `Pending::zip` or `Pending::all` +//! in Rust, or one `Seal` or `Open` call for many rows in Go. +//! +//! ## Key modes +//! +//! | | ZeroKMS | Wrapped data key | Derived data key | +//! |---|---|---|---| +//! | Keys in application memory | The index key of a keyset, until the cipher's bounded cache evicts it ([`KeyLifetime::UntilEvicted`](stack_kms::KeyLifetime::UntilEvicted)). Each data key, for one settle. | The index key, for one settle ([`KeyLifetime::Settle`](stack_kms::KeyLifetime::Settle)): the cipher loads it inside the settle and drops it at the end. Each data key, for one settle. | The root key of a keyset, for the root key TTL ([`DEFAULT_ROOT_KEY_TTL`], 5 minutes; set it with [`VendorKeySource::with_root_key_ttl`]). The index key and the data keys are derived from it in each settle and dropped at its end. | +//! | Key backend calls for a settle of N values in one keyset | One request for each 500 keys (`stack_kms::DEFAULT_KEYS_PER_REQ`). The index key loads once, then stays in the cache. | N to seal and N to open (one per value), or one per chunk on a vendor with batch calls. Plus 1 index key unwrap when the settle derives a search term. | 0 while the root key is cached, else 1 root key unwrap. With a TTL of zero: 1 for the data keys, and 1 more when the settle derives a search term. | +//! | What the key backend can refuse | Each request. ZeroKMS also checks the key context (lock context) and the decryption policy of each data key. | Every value it seals or opens, and each settle that derives a search term. | The root key unwrap only. A running process sees a refusal when its cached root key expires: at most one TTL later. | +//! +//! The vendor calls of one settle run through [`run_batch`]: at most 16 at +//! once on native Rust ([`BatchOptions`]), up to 5 attempts with backoff on +//! [`Throttled`](stack_kms::BackendErrorKind::Throttled), and the first +//! other failure fails the whole settle. The Go guest sends its vendor calls +//! one at a time. Every keyset record key (the index key of a wrapped +//! record, the root key of a derived one) is one `unwrap` call, never a +//! batch call. +//! +//! ## Backends +//! +//! | Backend ([`KeyVendor::name`](stack_kms::KeyVendor::name)) | Key modes | Wrapped data keys: calls to seal or open N values | How the associated data is bound | +//! |---|---|---|---| +//! | ZeroKMS (`zerokms`, v1 leaves) | n/a | n/a | n/a | +//! | AWS KMS (`aws-kms`) | both | N `GenerateDataKey` to seal, N `Decrypt` to open | Natively, as the encryption context: `cipherstash:aad` is all of the associated data; `cipherstash:purpose`, `cipherstash:keyset-id` and `cipherstash:descriptor` are readable copies. | +//! | Google Cloud KMS (`gcp-kms`) | both | N `encrypt` to seal, N `decrypt` to open | Natively, as `additionalAuthenticatedData`, with CRC32C checks on every field. | +//! | Azure Key Vault, RSA key (`azure-key-vault`) | both | N `wrapkey` (RSA-OAEP-256) to seal, N `unwrapkey` to open | By us ([`bound`]): Azure wraps the key with a digest of the associated data, and the client checks the digest after the unwrap. | +//! | Azure Managed HSM, `oct-HSM` key (`azure-key-vault`) | both | N `wrapkey` (A256KW) to seal, N `unwrapkey` to open | By us ([`bound`]), as for an RSA key. | +//! | Vault Transit, derived key (`vault-transit`) | both | ⌈N / 250⌉ batch `encrypt` to seal, ⌈N / 250⌉ batch `decrypt` to open (`VaultConfig::batch_size`) | By us ([`bound`]), and also natively: the client sends all of the associated data as the key derivation `context`. | +//! | Vault Transit, non-derived key (`vault-transit`) | both | as for a derived key | By us ([`bound`]) only. Vault ignores `context` on a non-derived key. | +//! | Static key (`static`) | derived only | n/a | Natively, as the AES-256-GCM associated data. | +//! +//! First use of a key costs one more call: with the default +//! `KeyDerivation::Detect`, the Vault client reads each Transit key once +//! per process; the Azure client reads each key once per client to learn +//! its current version and type, unless the key reference names a version +//! and `AzureKeyVaultConfig::with_algorithm` is set. +//! +//! **What the vendor sees.** A vendor sees each key it wraps or unwraps. +//! It holds the vendor key, so it could unwrap that key anyway. On AWS KMS, +//! `GenerateDataKey` makes the data key. No vendor ever sees a plaintext +//! value or a search term. Of the associated data, AWS KMS gets all of it, +//! with the keyset id, the purpose and the descriptor in readable form for +//! CloudTrail and for `kms:EncryptionContext:` key policy conditions. Google +//! Cloud KMS gets all of it. Vault gets all of it as `context` for a derived +//! key, and only its digest inside the wrapped plaintext for a non-derived +//! key. Azure gets only its digest inside the wrapped plaintext: Azure does +//! not see or log the binding, so a wrapped key moved to another value +//! unwraps at Azure and is then refused by the client as +//! [`InvalidCiphertext`](stack_kms::BackendErrorKind::InvalidCiphertext). +//! The static key backend runs in the process: nothing outside it can see, +//! refuse or audit a use of the key. +//! +//! ## ZeroKMS-only features +//! +//! A key vendor backend refuses these with +//! [`Error::ZeroKmsOnly`](stack_kms::Error::ZeroKmsOnly). It never ignores +//! them ([`ZeroKmsFeature`](stack_kms::ZeroKmsFeature)): +//! +//! * a non-empty key context: a lock context, or tags and identity claims +//! bound into the data key; +//! * a decryption policy; +//! * a non-empty unverified context. +//! +//! A cipher holds ZeroKMS and at most one key vendor backend ([`KeyRouter`]). +//! A value sealed under one backend never opens under another, and there is +//! no migration between backends. +//! +//! ## Credentials +//! +//! | Backend | Native Rust (`from_env`) | Go (the host supplies short-lived credentials before each request) | +//! |---|---|---| +//! | AWS KMS | Static keys (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`), then web identity (IRSA), then the ECS or EKS container endpoint, then IMDSv2. Not read: `~/.aws` config and credentials files, profiles, SSO, `credential_process`. | `AWSCredentialSource`: `AWSCredentialsFunc` over the AWS SDK's chain, or `AWSCredentialsFromEnv`. | +//! | Google Cloud KMS | `GOOGLE_OAUTH_ACCESS_TOKEN`, then a service account key file (`GOOGLE_APPLICATION_CREDENTIALS`, feature `service-account`), then the metadata server. Not supported: `authorized_user` and `external_account` files, impersonation. | `AccessTokenSource` for the Cloud KMS scope: `OAuth2AccessTokens` or `AccessTokenFunc`. | +//! | Azure Key Vault and Managed HSM | `AZURE_KEY_VAULT_ACCESS_TOKEN`, then a client secret, then workload identity (`AZURE_FEDERATED_TOKEN_FILE`), then managed identity (App Service, else IMDS). Not supported: client certificates, Azure CLI, Azure Arc, Service Fabric. | `AccessTokenSource` for the scope the client names (Key Vault or Managed HSM): `AccessTokenFunc` over azidentity. | +//! | Vault Transit | `VAULT_TOKEN`, then AppRole (`VAULT_APPROLE_ROLE_ID` and `VAULT_APPROLE_SECRET_ID`), then Kubernetes auth (`VAULT_KUBERNETES_ROLE`). | `AccessTokenSource` with an empty scope: `AccessTokenFunc` over the Vault API client. | +//! | Static key | [`StaticKeyVendor::new`] takes the key. | `StaticKey.Key`, which `NewClient` wipes. | +//! +//! The web identity, container, IMDS, metadata server, workload identity, +//! managed identity and Kubernetes sources read files or reach instance +//! metadata, so they are native only. The Go guest holds no credential +//! between requests and never signs with a long-lived key: it does no RSA +//! and reads no service account key. `stash keyset` uses the native chains, +//! except for Google Cloud, where it reads `GOOGLE_OAUTH_ACCESS_TOKEN` or the +//! metadata server and refuses a service account key file. +//! +//! ## Rotation +//! +//! * **AWS KMS.** AWS rotates the key material behind one key ARN. A stored +//! wrapped key names the material that made it, so nothing changes in the +//! records or the data. +//! * **Google Cloud KMS.** `encrypt` uses the primary version, or the +//! version a `…/cryptoKeyVersions/{n}` key reference names. Each stored +//! wrapped key holds its version (`stack_kms_gcp::wrapped_key_version`). +//! Keep a version enabled while stored keys use it. +//! * **Azure.** Each stored wrapped key holds its version and algorithm +//! (`stack_kms_azure::wrapped_key_version`). A client reads the current +//! version of an unversioned key reference once and wraps under it for +//! its life, so it sees a new version after a restart, or after a wrap +//! fails and it reads the key again. Keep a version enabled while stored +//! keys use it. +//! * **Vault Transit.** `encrypt` uses the latest key version. Each stored +//! wrapped key holds its version. `VaultTransit::rewrap` moves one wrapped +//! key to the latest version inside Vault. Do not raise the key's minimum +//! decryption version past a version that stored keys still use. +//! * **Static key.** Hold the old and the new key +//! ([`StaticKeyVendor::with_key`]), rewrap every record under the new +//! key reference, then drop the old key. +//! +//! [`KeysetRecord::rewrap`] (and `stash keyset rewrap`) wraps the record +//! key again. It does not re-key stored values, so the rule depends on the +//! key mode. Under derived data keys a record may move to any key of its +//! backend: every data key comes from the root key. Under wrapped data keys +//! each stored value's data key stays wrapped under the vendor key that +//! sealed it, and the adapter unwraps it with the record's current key +//! reference. So a record may move only to a new version of its own vendor +//! key ([`KeyVendorBackend::same_key`]); another key is refused with +//! [`RecordError::RewrapToAnotherKey`], before any vendor call. Google +//! Cloud KMS and Azure ignore the version a key reference names. AWS KMS +//! and Vault Transit compare the references as text, so an AWS alias and +//! the key ARN it points to count as different keys. +//! +//! ## Known limits +//! +//! * **All vendors.** TypeScript (`@cipherstash/stack`) cannot use a key +//! vendor backend yet; only Rust and Go can. No SDK reads the +//! `CS_KEYSET_RECORD*` variables that `stash keyset` prints: pass the +//! record to [`KeysetRecord::parse`] or Go's `WithKeysetRecords`. +//! * **AWS KMS.** An AWS error code with no fixed mapping on a 4xx status is +//! [`AccessDenied`](stack_kms::BackendErrorKind::AccessDenied), so a +//! malformed request reads as a refusal. `LimitExceededException` is a +//! quota, not a throttle, and is not retried. +//! * **Google Cloud KMS.** The service account login (feature +//! `service-account`, a default feature) signs a JWT with the `rsa` crate, +//! which has the open advisory RUSTSEC-2023-0071 (the Marvin timing +//! attack). A build without default features has no RSA code; the Go +//! guest and `@cipherstash/auth` build it that way. Cloud KMS answers +//! `PERMISSION_DENIED` for a key the caller may not see, so a missing key +//! often reads as `AccessDenied`, not `NotFound`. The fixtures are +//! written by hand, and the live test has not run yet. +//! * **Azure.** A 400 `BadParameter` on an `unwrapkey` is +//! [`InvalidCiphertext`](stack_kms::BackendErrorKind::InvalidCiphertext), +//! because Azure gives no finer code; a `BadParameter` for another reason +//! reads the same. A key reference must be on one of the six Azure Key +//! Vault and Managed HSM DNS suffixes. A 429 waits for `Retry-After`, at +//! most 30 seconds, before the retry. The fixtures are written by hand, +//! and the live test has not run yet. +//! * **Vault Transit.** The native transport trusts the operating system's +//! certificate store only. It does not read `VAULT_CACERT` and sends no +//! mTLS client certificate, so a Vault with a private CA works only when +//! that CA is in the system store (or with your own [`VendorTransport`]; +//! in Go, the client's `WithTransport` round tripper). A 400 maps to a +//! kind by Vault's error text. Do not give the token `create` on +//! `/encrypt/`: Transit then creates a missing key. +//! * **Static key.** Derived data keys only +//! ([`RecordError::ModeUnsupported`]). The key is always in memory. + +#![deny(unsafe_code)] +#![warn(clippy::unwrap_used)] +#![warn(clippy::expect_used)] +#![warn(clippy::panic)] +#![warn(clippy::mem_forget)] +#![warn(clippy::print_stdout)] +#![warn(clippy::print_stderr)] +#![warn(clippy::dbg_macro)] +#![warn(unreachable_pub)] +#![warn(unused_results)] +#![cfg_attr(test, allow(clippy::unwrap_used))] +#![cfg_attr(test, allow(clippy::expect_used))] +#![cfg_attr(test, allow(clippy::panic))] +#![cfg_attr(test, allow(unused_results))] + +mod batch; +pub mod bound; +pub mod derive; +mod error; +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +mod http; +mod record; +mod router; +mod source; +mod static_key; +mod transport; +mod vendor; + +#[cfg(feature = "test-support")] +pub mod contract; +#[cfg(feature = "test-support")] +mod fake; + +pub use batch::{run_batch, BatchOptions}; +pub use error::RecordError; +pub use record::{KeysetRecord, RECORD_FORMAT_VERSION, RECORD_PREFIX}; +pub use router::KeyRouter; +pub use source::{VendorKeySource, DEFAULT_ROOT_KEY_TTL}; +pub use static_key::StaticKeyVendor; +pub use transport::{ + classify_status, TransportError, VendorRequest, VendorResponse, VendorTransport, +}; +pub use vendor::{KeyBytes, KeyVendorBackend, UnwrapItem, WrapItem}; + +#[cfg(all(feature = "http", not(target_arch = "wasm32")))] +pub use http::{HttpVendorTransport, HttpVendorTransportOptions, DEFAULT_REQUEST_TIMEOUT}; + +#[cfg(feature = "test-support")] +pub use fake::{FakeVendor, FakeVendorCalls}; diff --git a/packages/stack-kms-vendor/src/record.rs b/packages/stack-kms-vendor/src/record.rs new file mode 100644 index 0000000000..2afbc1741c --- /dev/null +++ b/packages/stack-kms-vendor/src/record.rs @@ -0,0 +1,521 @@ +//! Keyset records: the customer-held description of a keyset in a key +//! vendor backend. + +use std::fmt; +use std::str::FromStr; + +use base64ct::{Base64UrlUnpadded, Encoding}; +use stack_kms::{KeyMode, KeyVendor}; +use uuid::Uuid; + +use crate::derive::record_key_aad; +use crate::error::RecordError; +use crate::vendor::{random_key, KeyBytes, KeyVendorBackend}; + +/// The text every keyset record starts with. +pub const RECORD_PREFIX: &str = "stack-keyset:"; + +/// The format version of the keyset records this build writes. A record of +/// another version is refused ([`RecordError::UnknownVersion`]). +pub const RECORD_FORMAT_VERSION: u8 = 1; + +/// The longest keyset name, in bytes. +const MAX_NAME_LEN: usize = u8::MAX as usize; + +/// The customer-held description of a keyset in a key vendor backend: its +/// id, an optional name, its backend, its key mode, the vendor key that +/// protects it, and the wrapped key. +/// +/// The wrapped key is the root key under derived data keys and the index +/// key under wrapped data keys. Only the vendor key can unwrap it, so a +/// record is not secret: keep it in config or in an environment variable. +/// The wrapped key is bound to the id, the backend and the mode +/// ([`derive::record_key_aad`](crate::derive::record_key_aad)), so a record +/// edited in any of them does not unwrap. The name is a local label and is +/// not bound: rename a keyset by editing its record. +/// +/// # Text form +/// +/// [`RECORD_PREFIX`] followed by the unpadded base64url of: +/// +/// ```text +/// version (1) ‖ backend (1) ‖ mode (1) ‖ keyset id (16) +/// ‖ name length (1, 0 = no name) ‖ name +/// ‖ key ref length (2, big-endian) ‖ key ref +/// ‖ wrapped key length (2, big-endian) ‖ wrapped key +/// ``` +/// +/// The backend and mode bytes are the v2 leaf's ([`KeyVendor::byte`], +/// [`KeyMode::byte`]). +/// +/// # Creating a record +/// +/// [`KeysetRecord::create`] generates the keyset id and the key, and wraps +/// the key with the vendor. [`KeysetRecord::rewrap`] moves a record to +/// another vendor key. Both are what `stash keyset` calls. +#[derive(Clone, PartialEq, Eq)] +pub struct KeysetRecord { + id: Uuid, + name: Option, + vendor: KeyVendor, + mode: KeyMode, + key_ref: String, + wrapped_key: Vec, +} + +impl fmt::Debug for KeysetRecord { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("KeysetRecord") + .field("id", &self.id) + .field("name", &self.name) + .field("vendor", &self.vendor) + .field("mode", &self.mode) + .field("key_ref", &self.key_ref) + .field("wrapped_key_len", &self.wrapped_key.len()) + .finish() + } +} + +impl KeysetRecord { + /// Create a keyset in `backend`: generate a keyset id and a key (the + /// root key under derived data keys, the index key under wrapped data + /// keys), and wrap the key under the vendor key `key_ref`. + /// + /// One vendor call. Refuses a mode the vendor does not offer + /// ([`RecordError::ModeUnsupported`]): the static key backend offers + /// derived data keys only. + pub async fn create( + backend: &B, + key_ref: impl Into, + mode: KeyMode, + name: Option, + ) -> Result { + let vendor = backend.vendor(); + if !backend.supports_mode(mode) { + return Err(RecordError::ModeUnsupported { vendor, mode }); + } + let key_ref = key_ref.into(); + check_key_ref(&key_ref)?; + if let Some(name) = &name { + check_name(name)?; + } + let id = Uuid::new_v4(); + let key = random_key(vendor)?; + let wrapped_key = backend + .wrap(&key_ref, &key, &record_key_aad(id, vendor, mode)) + .await?; + check_wrapped(&wrapped_key)?; + Ok(Self { + id, + name, + vendor, + mode, + key_ref, + wrapped_key, + }) + } + + /// Wrap this record's key under a vendor key of the same backend. The + /// keyset id, name and mode stay the same. Stored values are not + /// re-keyed. + /// + /// * **Derived data keys:** `key_ref` may name any key of the backend. + /// Every data key comes from the root key, so every value sealed under + /// the keyset still opens. + /// * **Wrapped data keys:** `key_ref` must name the record's own vendor + /// key ([`KeyVendorBackend::same_key`]), for example its newest version + /// after a rotation. Each stored value's data key stays wrapped under + /// the key that sealed it, so a move to another key is refused with + /// [`RecordError::RewrapToAnotherKey`], before any vendor call. See the + /// [capability matrix](crate#rotation). + /// + /// Two vendor calls: unwrap under the old key, wrap under the new one. + pub async fn rewrap( + &self, + backend: &B, + key_ref: impl Into, + ) -> Result { + self.check_vendor(backend.vendor())?; + let key_ref = key_ref.into(); + check_key_ref(&key_ref)?; + if self.mode == KeyMode::Wrapped && !backend.same_key(&self.key_ref, &key_ref) { + return Err(RecordError::RewrapToAnotherKey { + keyset: self.id, + from: self.key_ref.clone(), + to: key_ref, + }); + } + let aad = self.key_aad(); + let key = self.unwrap_key(backend).await?; + let wrapped_key = backend.wrap(&key_ref, &key, &aad).await?; + check_wrapped(&wrapped_key)?; + Ok(Self { + key_ref, + wrapped_key, + ..self.clone() + }) + } + + /// Build a record from its parts, as a vendor crate's own tests or a + /// migration might. Prefer [`create`](Self::create). + pub fn from_parts( + id: Uuid, + name: Option, + vendor: KeyVendor, + mode: KeyMode, + key_ref: String, + wrapped_key: Vec, + ) -> Result { + if let Some(name) = &name { + check_name(name)?; + } + check_key_ref(&key_ref)?; + check_wrapped(&wrapped_key)?; + Ok(Self { + id, + name, + vendor, + mode, + key_ref, + wrapped_key, + }) + } + + /// Parse a record from its text form. Refuses an unknown version, + /// backend or mode, and a record whose length does not match its + /// fields. + pub fn parse(text: &str) -> Result { + let body = text + .trim() + .strip_prefix(RECORD_PREFIX) + .ok_or(RecordError::NotARecord)?; + let bytes = Base64UrlUnpadded::decode_vec(body).map_err(|_| RecordError::Encoding)?; + Self::from_bytes(&bytes) + } + + fn from_bytes(bytes: &[u8]) -> Result { + let mut reader = Reader(bytes); + let version = reader.u8()?; + if version != RECORD_FORMAT_VERSION { + return Err(RecordError::UnknownVersion(version)); + } + let vendor_byte = reader.u8()?; + let vendor = + KeyVendor::from_byte(vendor_byte).ok_or(RecordError::UnknownBackend(vendor_byte))?; + let mode_byte = reader.u8()?; + let mode = KeyMode::from_byte(mode_byte).ok_or(RecordError::UnknownMode(mode_byte))?; + let id = Uuid::from_bytes(reader.array::<16>()?); + let name_len = usize::from(reader.u8()?); + let name = match name_len { + 0 => None, + len => Some(utf8(reader.take(len)?, "name")?), + }; + let key_ref_len = usize::from(reader.u16()?); + let key_ref = utf8(reader.take(key_ref_len)?, "key_ref")?; + let wrapped_len = usize::from(reader.u16()?); + let wrapped_key = reader.take(wrapped_len)?.to_vec(); + if !reader.0.is_empty() { + return Err(RecordError::Length); + } + Self::from_parts(id, name, vendor, mode, key_ref, wrapped_key) + } + + /// The binary form, before base64url. + fn to_bytes(&self) -> Vec { + let name = self.name.as_deref().unwrap_or("").as_bytes(); + let mut out = Vec::with_capacity( + 3 + 16 + 1 + name.len() + 2 + self.key_ref.len() + 2 + self.wrapped_key.len(), + ); + out.extend_from_slice(&[RECORD_FORMAT_VERSION, self.vendor.byte(), self.mode.byte()]); + out.extend_from_slice(self.id.as_bytes()); + // The lengths were checked at construction, so the casts are exact. + out.push(name.len() as u8); + out.extend_from_slice(name); + out.extend_from_slice(&(self.key_ref.len() as u16).to_be_bytes()); + out.extend_from_slice(self.key_ref.as_bytes()); + out.extend_from_slice(&(self.wrapped_key.len() as u16).to_be_bytes()); + out.extend_from_slice(&self.wrapped_key); + out + } + + /// The keyset id. + pub fn id(&self) -> Uuid { + self.id + } + + /// The keyset name, if it has one. + pub fn name(&self) -> Option<&str> { + self.name.as_deref() + } + + /// The key vendor backend that holds the keyset. + pub fn vendor(&self) -> KeyVendor { + self.vendor + } + + /// The key mode of the keyset. + pub fn mode(&self) -> KeyMode { + self.mode + } + + /// The versioned vendor key reference. + pub fn key_ref(&self) -> &str { + &self.key_ref + } + + /// The wrapped key: the root key under derived data keys, the index key + /// under wrapped data keys. + pub fn wrapped_key(&self) -> &[u8] { + &self.wrapped_key + } + + /// The associated data the wrapped key is bound to. + pub(crate) fn key_aad(&self) -> Vec { + record_key_aad(self.id, self.vendor, self.mode) + } + + /// Unwrap this record's key with `backend`. + pub(crate) async fn unwrap_key( + &self, + backend: &B, + ) -> Result { + backend + .unwrap(&self.key_ref, &self.wrapped_key, &self.key_aad()) + .await + } + + pub(crate) fn check_vendor(&self, expected: KeyVendor) -> Result<(), RecordError> { + if self.vendor == expected { + Ok(()) + } else { + Err(RecordError::ForeignVendor { + keyset: self.id, + expected, + found: self.vendor, + }) + } + } +} + +/// The text form: [`RECORD_PREFIX`] and the unpadded base64url of the +/// binary form. +impl fmt::Display for KeysetRecord { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(RECORD_PREFIX)?; + f.write_str(&Base64UrlUnpadded::encode_string(&self.to_bytes())) + } +} + +impl FromStr for KeysetRecord { + type Err = RecordError; + + fn from_str(text: &str) -> Result { + Self::parse(text) + } +} + +fn check_name(name: &str) -> Result<(), RecordError> { + if name.is_empty() || name.len() > MAX_NAME_LEN { + return Err(RecordError::InvalidField { field: "name" }); + } + Ok(()) +} + +fn check_key_ref(key_ref: &str) -> Result<(), RecordError> { + if key_ref.is_empty() || key_ref.len() > usize::from(u16::MAX) { + return Err(RecordError::InvalidField { field: "key_ref" }); + } + Ok(()) +} + +fn check_wrapped(wrapped: &[u8]) -> Result<(), RecordError> { + if wrapped.is_empty() || wrapped.len() > usize::from(u16::MAX) { + return Err(RecordError::InvalidField { + field: "wrapped_key", + }); + } + Ok(()) +} + +fn utf8(bytes: &[u8], field: &'static str) -> Result { + String::from_utf8(bytes.to_vec()).map_err(|_| RecordError::InvalidField { field }) +} + +/// A cursor over the binary form. Every read past the end is +/// [`RecordError::Length`]. +struct Reader<'a>(&'a [u8]); + +impl<'a> Reader<'a> { + fn take(&mut self, len: usize) -> Result<&'a [u8], RecordError> { + if self.0.len() < len { + return Err(RecordError::Length); + } + let (head, rest) = self.0.split_at(len); + self.0 = rest; + Ok(head) + } + + fn array(&mut self) -> Result<[u8; N], RecordError> { + let mut out = [0u8; N]; + out.copy_from_slice(self.take(N)?); + Ok(out) + } + + fn u8(&mut self) -> Result { + Ok(self.array::<1>()?[0]) + } + + fn u16(&mut self) -> Result { + Ok(u16::from_be_bytes(self.array::<2>()?)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn record() -> KeysetRecord { + KeysetRecord::from_parts( + Uuid::from_u128(0x0102_0304_0506_0708_090a_0b0c_0d0e_0f10), + Some("acme".into()), + KeyVendor::Aws, + KeyMode::Wrapped, + "arn:aws:kms:us-east-1:111122223333:key/k".into(), + vec![1, 2, 3], + ) + .unwrap() + } + + #[test] + fn a_record_round_trips_through_its_text_form() { + let record = record(); + let text = record.to_string(); + assert!(text.starts_with(RECORD_PREFIX)); + assert_eq!(KeysetRecord::parse(&text).unwrap(), record); + assert_eq!(text.parse::().unwrap(), record); + // Whitespace around a value pasted into an env var is not an error. + assert_eq!(KeysetRecord::parse(&format!(" {text}\n")).unwrap(), record); + + let unnamed = KeysetRecord { + name: None, + ..record + }; + assert_eq!(KeysetRecord::parse(&unnamed.to_string()).unwrap(), unnamed); + } + + /// The text form is a storage commitment: this pin fails if it moves. + #[test] + fn the_text_form_is_pinned() { + assert_eq!(record().to_string(), "stack-keyset:AQEBAQIDBAUGBwgJCgsMDQ4PEARhY21lAChhcm46YXdzOmttczp1cy1lYXN0LTE6MTExMTIyMjIzMzMzOmtleS9rAAMBAgM"); + } + + fn reencode(bytes: &[u8]) -> String { + format!("{RECORD_PREFIX}{}", Base64UrlUnpadded::encode_string(bytes)) + } + + #[test] + fn parsing_refuses_what_it_does_not_know() { + let bytes = record().to_bytes(); + let refuse = |edit: &dyn Fn(&mut Vec)| { + let mut bytes = bytes.clone(); + edit(&mut bytes); + KeysetRecord::parse(&reencode(&bytes)).unwrap_err() + }; + assert!(matches!( + refuse(&|b| b[0] = 2), + RecordError::UnknownVersion(2) + )); + assert!(matches!( + refuse(&|b| b[0] = 0), + RecordError::UnknownVersion(0) + )); + assert!(matches!( + refuse(&|b| b[1] = 0), + RecordError::UnknownBackend(0) + )); + assert!(matches!( + refuse(&|b| b[1] = 99), + RecordError::UnknownBackend(99) + )); + assert!(matches!(refuse(&|b| b[2] = 3), RecordError::UnknownMode(3))); + assert!(matches!(refuse(&|b| b.push(0)), RecordError::Length)); + assert!(matches!( + refuse(&|b| { + let _ = b.pop(); + }), + RecordError::Length + )); + assert!(matches!(refuse(&|b| b.truncate(10)), RecordError::Length)); + assert!(matches!( + KeysetRecord::parse("stack-keyset:***"), + Err(RecordError::Encoding) + )); + assert!(matches!( + KeysetRecord::parse("not-a-record"), + Err(RecordError::NotARecord) + )); + assert!(matches!( + KeysetRecord::parse(""), + Err(RecordError::NotARecord) + )); + } + + #[test] + fn fields_are_checked() { + let base = record(); + let bad_name = KeysetRecord::from_parts( + base.id, + Some(String::new()), + base.vendor, + base.mode, + base.key_ref.clone(), + base.wrapped_key.clone(), + ); + assert!(matches!( + bad_name, + Err(RecordError::InvalidField { field: "name" }) + )); + let long_name = KeysetRecord::from_parts( + base.id, + Some("n".repeat(256)), + base.vendor, + base.mode, + base.key_ref.clone(), + base.wrapped_key.clone(), + ); + assert!(long_name.is_err()); + assert!(KeysetRecord::from_parts( + base.id, + Some("n".repeat(255)), + base.vendor, + base.mode, + base.key_ref.clone(), + base.wrapped_key.clone(), + ) + .is_ok()); + assert!(matches!( + KeysetRecord::from_parts( + base.id, + None, + base.vendor, + base.mode, + String::new(), + vec![1] + ), + Err(RecordError::InvalidField { field: "key_ref" }) + )); + assert!(matches!( + KeysetRecord::from_parts(base.id, None, base.vendor, base.mode, "k".into(), vec![]), + Err(RecordError::InvalidField { + field: "wrapped_key" + }) + )); + } + + #[test] + fn debug_shows_no_wrapped_key() { + let shown = format!("{:?}", record()); + assert!(shown.contains("wrapped_key_len: 3"), "{shown}"); + assert!(!shown.contains("[1, 2, 3]"), "{shown}"); + } +} diff --git a/packages/stack-kms-vendor/src/router.rs b/packages/stack-kms-vendor/src/router.rs new file mode 100644 index 0000000000..c4cb550c0f --- /dev/null +++ b/packages/stack-kms-vendor/src/router.rs @@ -0,0 +1,211 @@ +//! ZeroKMS and at most one key vendor backend in one key source. + +use std::borrow::Cow; + +use stack_kms::{ + DataKey, DataKeySource, DataKeyWithTag, Error, GenerateKeyPayload, IdentifiedBy, IndexKey, + IndexKeySource, KeyBackend, KeyMode, LoadedIndexKey, RetrieveKeyPayload, SelectedKeyset, + UnverifiedContext, +}; +use uuid::Uuid; + +use crate::source::VendorKeySource; +use crate::vendor::KeyVendorBackend; + +/// ZeroKMS and one key vendor backend, as one key source that routes each +/// request by its keyset (ADR-0009). +/// +/// A cipher holds ZeroKMS and **at most one** key vendor backend. The type +/// has room for exactly one, and [`VendorKeySource::new`] refuses records +/// that name a second vendor +/// ([`RecordError::SecondVendor`](crate::RecordError::SecondVendor)). +/// +/// Routing: +/// +/// * A keyset id that one of the vendor's records names goes to the vendor. +/// Every other id goes to ZeroKMS. +/// * A keyset name that one of the vendor's records has goes to the vendor. +/// A local record name wins over a ZeroKMS keyset of the same name. Every +/// other name goes to ZeroKMS. +/// * Naming no keyset is ZeroKMS's default keyset, or the vendor's default +/// keyset after [`with_vendor_default`](Self::with_vendor_default). +/// +/// The cipher reads [`keyset_backend`](DataKeySource::keyset_backend) to +/// refuse a leaf that names the wrong backend before any key call, and to +/// seal a vendor keyset's values as v2 leaves. There is no migration +/// between the two backends in either direction. +pub struct KeyRouter { + zerokms: Z, + vendor: VendorKeySource, + vendor_default: bool, +} + +impl std::fmt::Debug for KeyRouter { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("KeyRouter") + .field("zerokms", &std::any::type_name::()) + .field("vendor", &self.vendor) + .field("vendor_default", &self.vendor_default) + .finish() + } +} + +impl KeyRouter { + /// Route between `zerokms` and `vendor`. + pub fn new(zerokms: Z, vendor: VendorKeySource) -> Self { + Self { + zerokms, + vendor, + vendor_default: false, + } + } + + /// Make naming no keyset select the vendor's default keyset (its only + /// record, or the one set with + /// [`VendorKeySource::with_default_keyset`]), not ZeroKMS's. For a + /// client that has a key vendor backend and no ZeroKMS credentials: its + /// ZeroKMS side is then a source that refuses every request. + pub fn with_vendor_default(mut self) -> Self { + self.vendor_default = true; + self + } + + /// The ZeroKMS key source. + pub fn zerokms(&self) -> &Z { + &self.zerokms + } + + /// The key vendor backend's key source. + pub fn vendor(&self) -> &VendorKeySource { + &self.vendor + } + + fn is_vendor_id(&self, keyset: Option) -> bool { + match keyset { + None => self.vendor_default, + Some(keyset) => self.vendor.has_keyset(keyset), + } + } + + fn is_vendor_selection(&self, keyset: Option<&IdentifiedBy>) -> bool { + match keyset { + None => self.vendor_default, + Some(IdentifiedBy::Uuid(id)) => self.vendor.has_keyset(*id), + Some(IdentifiedBy::Name(name)) => self.vendor.has_name(name), + } + } +} + +impl DataKeySource for KeyRouter +where + Z: DataKeySource + Send + Sync, + B: KeyVendorBackend, +{ + async fn generate_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option>, + ) -> Result, Error> { + if self.is_vendor_id(keyset_id) { + self.vendor + .generate_keys(payloads, keyset_id, unverified_context) + .await + } else { + self.zerokms + .generate_keys(payloads, keyset_id, unverified_context) + .await + } + } + + async fn retrieve_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option<&UnverifiedContext>, + ) -> Result, Error> { + if self.is_vendor_id(keyset_id) { + self.vendor + .retrieve_keys(payloads, keyset_id, unverified_context) + .await + } else { + self.zerokms + .retrieve_keys(payloads, keyset_id, unverified_context) + .await + } + } + + /// The backend of the default keyset: ZeroKMS, or the vendor after + /// [`with_vendor_default`](Self::with_vendor_default). Per keyset, see + /// [`keyset_backend`](DataKeySource::keyset_backend). + fn key_backend(&self) -> KeyBackend { + if self.vendor_default { + self.vendor.key_backend() + } else { + self.zerokms.key_backend() + } + } + + fn keyset_backend(&self, keyset_id: Uuid) -> KeyBackend { + if self.vendor.has_keyset(keyset_id) { + self.vendor.key_backend() + } else { + self.zerokms.keyset_backend(keyset_id) + } + } + + fn key_mode(&self, keyset_id: Uuid) -> Option { + if self.vendor.has_keyset(keyset_id) { + self.vendor.key_mode(keyset_id) + } else { + self.zerokms.key_mode(keyset_id) + } + } + + async fn load_settle_index_key(&self, keyset_id: Uuid) -> Result { + if self.vendor.has_keyset(keyset_id) { + self.vendor.load_settle_index_key(keyset_id).await + } else { + self.zerokms.load_settle_index_key(keyset_id).await + } + } +} + +impl IndexKeySource for KeyRouter +where + Z: IndexKeySource + Send + Sync, + B: KeyVendorBackend, +{ + async fn load_index_key( + &self, + keyset_id: Option, + ) -> Result<(Uuid, IndexKey), Error> { + if self.is_vendor_selection(keyset_id.as_ref()) { + self.vendor.load_index_key(keyset_id).await + } else { + self.zerokms.load_index_key(keyset_id).await + } + } + + async fn load_index_key_with_lifetime( + &self, + keyset_id: Option, + ) -> Result { + if self.is_vendor_selection(keyset_id.as_ref()) { + self.vendor.load_index_key_with_lifetime(keyset_id).await + } else { + self.zerokms.load_index_key_with_lifetime(keyset_id).await + } + } + + async fn select_keyset( + &self, + keyset_id: Option, + ) -> Result { + if self.is_vendor_selection(keyset_id.as_ref()) { + self.vendor.select_keyset(keyset_id).await + } else { + self.zerokms.select_keyset(keyset_id).await + } + } +} diff --git a/packages/stack-kms-vendor/src/source.rs b/packages/stack-kms-vendor/src/source.rs new file mode 100644 index 0000000000..a400cbab45 --- /dev/null +++ b/packages/stack-kms-vendor/src/source.rs @@ -0,0 +1,614 @@ +//! The adapter from one key vendor backend and its keyset records to the +//! stack-kms key source traits. + +use std::borrow::Cow; +use std::collections::HashMap; +use std::num::NonZeroUsize; +use std::sync::{Mutex, PoisonError}; +use std::time::Duration; + +use stack_kms::{ + refuse_zerokms_extensions, BackendErrorKind, DataKey, DataKeySource, DataKeyWithTag, Error, + GenerateKeyPayload, IdentifiedBy, IndexKey, IndexKeySource, KeyBackend, KeyLifetime, KeyMode, + KeyVendor, LoadedIndexKey, RetrieveKeyPayload, SelectedKeyset, UnverifiedContext, +}; +use uuid::Uuid; +use vitaminc::protected::{Controlled, Protected}; +use web_time::Instant; + +use crate::batch::{run_batch, BatchOptions}; +use crate::derive::{derived_data_key, derived_index_key, wrapped_data_key_aad}; +use crate::error::RecordError; +use crate::record::KeysetRecord; +use crate::vendor::{random_bytes, random_key, KeyBytes, KeyVendorBackend, UnwrapItem, WrapItem}; + +/// How long the adapter keeps an unwrapped root key (derived data keys) +/// before it asks the vendor again: five minutes. +pub const DEFAULT_ROOT_KEY_TTL: Duration = Duration::from_secs(5 * 60); + +/// A root key the adapter holds, and when it must drop it. +struct CachedRoot { + key: KeyBytes, + until: Instant, +} + +/// One key vendor backend and its keyset records, as a +/// [`DataKeySource`] and an [`IndexKeySource`]. +/// +/// This is the adapter every vendor shares. A vendor crate implements +/// [`KeyVendorBackend`]; this type builds both key modes on it: +/// +/// * **Wrapped data key.** Each generate makes one vendor call per value +/// ([`KeyVendorBackend::generate_wrapped`]) and puts the wrapped data key +/// in the leaf's tag. Each retrieve makes one vendor call per value. A +/// vendor with a batch operation ([`KeyVendorBackend::batch_limit`]) +/// instead gets one call per chunk of values +/// ([`KeyVendorBackend::wrap_batch`], [`KeyVendorBackend::unwrap_batch`]). +/// The calls of one batch run through [`run_batch`]. The index key is +/// unwrapped from the record once per settle and never kept. +/// * **Derived data key.** The adapter unwraps the root key once, keeps it +/// for the root key TTL ([`DEFAULT_ROOT_KEY_TTL`]), and derives every data +/// key and the index key from it with no vendor call. The root key is +/// wiped when it expires or when the adapter drops. +/// +/// Either way, a cipher holds no index key between settles: the adapter +/// answers [`select_keyset`](IndexKeySource::select_keyset) with +/// [`SelectedKeyset::PerSettle`], and the cipher asks +/// [`load_settle_index_key`](DataKeySource::load_settle_index_key) inside +/// each settle that builds a term. +/// +/// Keyset names resolve locally from the loaded records, with no freshness +/// window: a record is the customer's statement of what the name means. +/// +/// The adapter refuses every ZeroKMS-only input (a lock context, a +/// decryption policy, an unverified context) with +/// [`Error::ZeroKmsOnly`]. +pub struct VendorKeySource { + backend: B, + records: HashMap, + names: HashMap, + default: Option, + root_ttl: Duration, + batch: BatchOptions, + roots: Mutex>, +} + +impl std::fmt::Debug for VendorKeySource { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("VendorKeySource") + .field("keysets", &self.records.keys().collect::>()) + .field("default", &self.default) + .field("root_ttl", &self.root_ttl) + .finish_non_exhaustive() + } +} + +/// The one vendor a set of records names, or `None` for no records. +/// Two vendors is [`RecordError::SecondVendor`]: a cipher holds ZeroKMS and +/// at most one key vendor backend. +pub(crate) fn vendor_of<'a>( + records: impl IntoIterator, +) -> Result, RecordError> { + let mut found: Option = None; + for record in records { + match found { + None => found = Some(record.vendor()), + Some(first) if first != record.vendor() => { + return Err(RecordError::SecondVendor { + first, + second: record.vendor(), + }) + } + Some(_) => {} + } + } + Ok(found) +} + +fn not_found(vendor: KeyVendor) -> Error { + Error::Backend { + vendor: vendor.name(), + kind: BackendErrorKind::NotFound, + source: "no keyset record for this keyset".into(), + } +} + +fn unavailable(vendor: KeyVendor, message: &'static str) -> Error { + Error::Backend { + vendor: vendor.name(), + kind: BackendErrorKind::Unavailable, + source: message.into(), + } +} + +fn invalid(vendor: KeyVendor, message: &'static str) -> Error { + Error::Backend { + vendor: vendor.name(), + kind: BackendErrorKind::InvalidCiphertext, + source: message.into(), + } +} + +impl VendorKeySource { + /// The adapter over `backend` and `records`. + /// + /// Refuses records that name two vendors ([`RecordError::SecondVendor`]), + /// a record of another vendor than `backend` + /// ([`RecordError::ForeignVendor`]), a record whose mode the vendor + /// does not offer ([`RecordError::ModeUnsupported`]), and two records + /// with the same id or name. + /// + /// The default keyset (naming no keyset) is the only record when there + /// is exactly one, else none until + /// [`with_default_keyset`](Self::with_default_keyset) sets it. + pub fn new( + backend: B, + records: impl IntoIterator, + ) -> Result { + let records: Vec = records.into_iter().collect(); + let vendor = backend.vendor(); + let _ = vendor_of(&records)?; + let mut by_id = HashMap::with_capacity(records.len()); + let mut names = HashMap::new(); + for record in records { + record.check_vendor(vendor)?; + if !backend.supports_mode(record.mode()) { + return Err(RecordError::ModeUnsupported { + vendor, + mode: record.mode(), + }); + } + if let Some(name) = record.name() { + if names.insert(name.to_owned(), record.id()).is_some() { + return Err(RecordError::DuplicateName); + } + } + let id = record.id(); + if by_id.insert(id, record).is_some() { + return Err(RecordError::DuplicateKeyset(id)); + } + } + let default = match by_id.len() { + 1 => by_id.keys().next().copied(), + _ => None, + }; + Ok(Self { + backend, + records: by_id, + names, + default, + root_ttl: DEFAULT_ROOT_KEY_TTL, + batch: BatchOptions::default(), + roots: Mutex::new(HashMap::new()), + }) + } + + /// Parse each text record and build the adapter, as [`new`](Self::new) + /// does. For records held in config or environment variables. + pub fn from_texts<'t>( + backend: B, + texts: impl IntoIterator, + ) -> Result { + let records = texts + .into_iter() + .map(KeysetRecord::parse) + .collect::, _>>()?; + Self::new(backend, records) + } + + /// Make `keyset` the default keyset: the one naming no keyset selects. + pub fn with_default_keyset(mut self, keyset: Uuid) -> Result { + if !self.records.contains_key(&keyset) { + return Err(RecordError::UnknownDefault); + } + self.default = Some(keyset); + Ok(self) + } + + /// How long to keep an unwrapped root key (derived data keys). Default + /// [`DEFAULT_ROOT_KEY_TTL`]. `Duration::ZERO` keeps none: every settle + /// unwraps again. + pub fn with_root_key_ttl(mut self, ttl: Duration) -> Self { + self.root_ttl = ttl; + self + } + + /// How vendor calls are batched (wrapped data keys). + pub fn with_batch_options(mut self, options: BatchOptions) -> Self { + self.batch = options; + self + } + + /// The key vendor backend. + pub fn backend(&self) -> &B { + &self.backend + } + + /// Whether a record for `keyset` is loaded. + pub fn has_keyset(&self, keyset: Uuid) -> bool { + self.records.contains_key(&keyset) + } + + /// Whether a loaded record has the name `name`. + pub fn has_name(&self, name: &str) -> bool { + self.names.contains_key(name) + } + + /// The record of `keyset`, if it is loaded. + pub fn record(&self, keyset: Uuid) -> Option<&KeysetRecord> { + self.records.get(&keyset) + } + + fn vendor(&self) -> KeyVendor { + self.backend.vendor() + } + + /// The record a selection names: an id, a local name, or the default. + fn resolve(&self, keyset: Option<&IdentifiedBy>) -> Result<&KeysetRecord, Error> { + let id = match keyset { + None => self.default, + Some(IdentifiedBy::Uuid(id)) => Some(*id), + Some(IdentifiedBy::Name(name)) => self.names.get(&**name).copied(), + }; + id.and_then(|id| self.records.get(&id)) + .ok_or_else(|| not_found(self.vendor())) + } + + /// The record a data-key request names (`None` is the default keyset). + fn resolve_id(&self, keyset: Option) -> Result<&KeysetRecord, Error> { + self.resolve(keyset.map(IdentifiedBy::Uuid).as_ref()) + } + + /// The root key of a derived-data-key record: from the cache while it + /// is live, else unwrapped (one vendor call) and cached for the TTL. + async fn root_key(&self, record: &KeysetRecord) -> Result { + let now = Instant::now(); + { + let mut roots = self.roots.lock().unwrap_or_else(PoisonError::into_inner); + // Every lookup drops every expired root key, so none outlives + // its TTL by more than the time to the next lookup. + roots.retain(|_, cached| now < cached.until); + if let Some(cached) = roots.get(&record.id()) { + return Ok(Protected::new(*cached.key.risky_ref())); + } + } + let key = self.unwrap_record(record).await?; + if !self.root_ttl.is_zero() { + if let Some(until) = Instant::now().checked_add(self.root_ttl) { + let mut roots = self.roots.lock().unwrap_or_else(PoisonError::into_inner); + let _ = roots.insert( + record.id(), + CachedRoot { + key: Protected::new(*key.risky_ref()), + until, + }, + ); + } + } + Ok(key) + } + + /// Unwrap the key in `record` (one vendor call), with the batch's + /// backoff on a throttle. + async fn unwrap_record(&self, record: &KeysetRecord) -> Result { + let mut keys = run_batch( + &self.batch, + 1, + |_| record.unwrap_key(&self.backend), + |delay| self.backend.sleep(delay), + ) + .await?; + keys.pop() + .ok_or_else(|| invalid(self.vendor(), "the vendor returned no key")) + } + + /// The index key of `record`: derived from the root key, or unwrapped + /// from the record. + async fn index_key(&self, record: &KeysetRecord) -> Result { + let key = match record.mode() { + KeyMode::Derived => derived_index_key(&self.root_key(record).await?, record.id()), + KeyMode::Wrapped => self.unwrap_record(record).await?, + }; + Ok(IndexKey::from_key_bytes(key)) + } + + /// Generate one wrapped data key for each descriptor, in vendor batch + /// calls of at most [`KeyVendorBackend::batch_limit`] keys. Each data key + /// comes from the CSPRNG. The chunks run through [`run_batch`], so a + /// throttled chunk backs off and is sent again whole. + async fn generate_in_vendor_batches( + &self, + record: &KeysetRecord, + descriptors: &[&str], + ) -> Result, Error> { + let vendor = self.vendor(); + let limit = self.backend.batch_limit().map_or(1, NonZeroUsize::get); + let chunks: Vec<&[&str]> = descriptors.chunks(limit).collect(); + let results = run_batch( + &self.batch, + chunks.len(), + |chunk| { + let chunk = chunks[chunk]; + async move { + let mut ivs = Vec::with_capacity(chunk.len()); + let mut keys = Vec::with_capacity(chunk.len()); + let mut aads = Vec::with_capacity(chunk.len()); + for descriptor in chunk { + let iv = random_bytes::<16>(vendor)?; + aads.push(wrapped_data_key_aad(record.id(), &iv, descriptor)); + ivs.push(iv); + keys.push(random_key(vendor)?); + } + let items: Vec> = keys + .iter() + .zip(&aads) + .map(|(key, aad)| WrapItem { key, aad }) + .collect(); + let wrapped = self.backend.wrap_batch(record.key_ref(), &items).await?; + if wrapped.len() != chunk.len() { + return Err(unavailable( + vendor, + "a batch returned another count of keys", + )); + } + Ok(ivs + .into_iter() + .zip(keys) + .zip(wrapped) + .map(|((iv, key), tag)| DataKeyWithTag { + key: DataKey { + iv, + key: *key.risky_ref(), + }, + tag, + decryption_policy: None, + }) + .collect::>()) + } + }, + |delay| self.backend.sleep(delay), + ) + .await?; + Ok(results.into_iter().flatten().collect()) + } + + /// Unwrap the wrapped data key of each payload, in vendor batch calls + /// of at most [`KeyVendorBackend::batch_limit`] keys. + async fn retrieve_in_vendor_batches( + &self, + record: &KeysetRecord, + payloads: &[RetrieveKeyPayload<'_>], + ) -> Result, Error> { + let vendor = self.vendor(); + let limit = self.backend.batch_limit().map_or(1, NonZeroUsize::get); + let chunks: Vec<&[RetrieveKeyPayload<'_>]> = payloads.chunks(limit).collect(); + let results = run_batch( + &self.batch, + chunks.len(), + |chunk| { + let chunk = chunks[chunk]; + async move { + let ivs: Vec<[u8; 16]> = chunk.iter().map(|p| p.iv_bytes()).collect(); + let aads: Vec> = chunk + .iter() + .zip(&ivs) + .map(|(payload, iv)| { + wrapped_data_key_aad(record.id(), iv, payload.descriptor) + }) + .collect(); + let items: Vec> = chunk + .iter() + .zip(&aads) + .map(|(payload, aad)| UnwrapItem { + wrapped: payload.tag, + aad, + }) + .collect(); + let keys = self.backend.unwrap_batch(record.key_ref(), &items).await?; + if keys.len() != chunk.len() { + return Err(unavailable( + vendor, + "a batch returned another count of keys", + )); + } + Ok(ivs + .into_iter() + .zip(keys) + .map(|(iv, key)| DataKey { + iv, + key: *key.risky_ref(), + }) + .collect::>()) + } + }, + |delay| self.backend.sleep(delay), + ) + .await?; + Ok(results.into_iter().flatten().collect()) + } + + /// The lifetime of `record`'s index key, for a caller of + /// [`IndexKeySource::load_index_key_with_lifetime`]. + fn lifetime(&self, record: &KeysetRecord) -> KeyLifetime { + match record.mode() { + KeyMode::Wrapped => KeyLifetime::Settle, + KeyMode::Derived => KeyLifetime::Ttl(self.root_ttl), + } + } +} + +impl DataKeySource for VendorKeySource { + async fn generate_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option>, + ) -> Result, Error> { + let vendor = self.vendor(); + refuse_zerokms_extensions(vendor.name(), &payloads, unverified_context.as_deref())?; + let record = self.resolve_id(keyset_id)?; + match record.mode() { + KeyMode::Derived => { + let root = self.root_key(record).await?; + payloads + .iter() + .map(|payload| { + let iv = random_bytes::<16>(vendor)?; + let key = derived_data_key(&root, record.id(), &iv, payload.descriptor()); + Ok(DataKeyWithTag { + key: DataKey { + iv, + key: *key.risky_ref(), + }, + tag: Vec::new(), + decryption_policy: None, + }) + }) + .collect() + } + KeyMode::Wrapped if self.backend.batch_limit().is_some() => { + let descriptors: Vec<&str> = payloads + .iter() + .map(GenerateKeyPayload::descriptor) + .collect(); + self.generate_in_vendor_batches(record, &descriptors).await + } + KeyMode::Wrapped => { + run_batch( + &self.batch, + payloads.len(), + |index| { + let descriptor = payloads[index].descriptor(); + async move { + let iv = random_bytes::<16>(vendor)?; + let aad = wrapped_data_key_aad(record.id(), &iv, descriptor); + let (key, wrapped) = self + .backend + .generate_wrapped(record.key_ref(), &aad) + .await?; + Ok(DataKeyWithTag { + key: DataKey { + iv, + key: *key.risky_ref(), + }, + tag: wrapped, + decryption_policy: None, + }) + } + }, + |delay| self.backend.sleep(delay), + ) + .await + } + } + } + + async fn retrieve_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option<&UnverifiedContext>, + ) -> Result, Error> { + let vendor = self.vendor(); + refuse_zerokms_extensions(vendor.name(), &payloads, unverified_context)?; + let record = self.resolve_id(keyset_id)?; + if payloads + .iter() + .any(|payload| payload.key_mode.is_some_and(|mode| mode != record.mode())) + { + return Err(invalid( + vendor, + "the leaf names another key mode than its keyset", + )); + } + match record.mode() { + KeyMode::Derived => { + if payloads.iter().any(|payload| !payload.tag.is_empty()) { + return Err(invalid(vendor, "a derived data key has an empty tag")); + } + let root = self.root_key(record).await?; + Ok(payloads + .iter() + .map(|payload| { + let iv = payload.iv_bytes(); + let key = derived_data_key(&root, record.id(), &iv, payload.descriptor); + DataKey { + iv, + key: *key.risky_ref(), + } + }) + .collect()) + } + KeyMode::Wrapped if self.backend.batch_limit().is_some() => { + self.retrieve_in_vendor_batches(record, &payloads).await + } + KeyMode::Wrapped => { + run_batch( + &self.batch, + payloads.len(), + |index| { + let payload = &payloads[index]; + async move { + let iv = payload.iv_bytes(); + let aad = wrapped_data_key_aad(record.id(), &iv, payload.descriptor); + let key = self + .backend + .unwrap(record.key_ref(), payload.tag, &aad) + .await?; + Ok(DataKey { + iv, + key: *key.risky_ref(), + }) + } + }, + |delay| self.backend.sleep(delay), + ) + .await + } + } + } + + fn key_backend(&self) -> KeyBackend { + KeyBackend::Vendor(self.vendor()) + } + + fn key_mode(&self, keyset_id: Uuid) -> Option { + self.records.get(&keyset_id).map(KeysetRecord::mode) + } + + async fn load_settle_index_key(&self, keyset_id: Uuid) -> Result { + let record = self.resolve_id(Some(keyset_id))?; + self.index_key(record).await + } +} + +impl IndexKeySource for VendorKeySource { + async fn load_index_key( + &self, + keyset_id: Option, + ) -> Result<(Uuid, IndexKey), Error> { + let record = self.resolve(keyset_id.as_ref())?; + Ok((record.id(), self.index_key(record).await?)) + } + + async fn load_index_key_with_lifetime( + &self, + keyset_id: Option, + ) -> Result { + let record = self.resolve(keyset_id.as_ref())?; + let key = self.index_key(record).await?; + Ok(LoadedIndexKey::new(record.id(), key, self.lifetime(record))) + } + + /// Resolve the keyset locally, with no vendor call. The cipher loads + /// the index key inside each settle. + async fn select_keyset( + &self, + keyset_id: Option, + ) -> Result { + let record = self.resolve(keyset_id.as_ref())?; + Ok(SelectedKeyset::PerSettle { + keyset_id: record.id(), + }) + } +} diff --git a/packages/stack-kms-vendor/src/static_key.rs b/packages/stack-kms-vendor/src/static_key.rs new file mode 100644 index 0000000000..cd0d292865 --- /dev/null +++ b/packages/stack-kms-vendor/src/static_key.rs @@ -0,0 +1,225 @@ +//! The static key backend: a key held in memory. + +use std::collections::BTreeMap; +use std::fmt; + +use aes_gcm::aead::{Aead, KeyInit, Payload}; +use aes_gcm::{Aes256Gcm, Nonce}; +use stack_kms::{BackendErrorKind, Error, KeyMode, KeyVendor}; +use vitaminc::protected::{Controlled, Protected}; +use zeroize::Zeroizing; + +use crate::vendor::{random_bytes, KeyBytes, KeyVendorBackend}; + +/// The first byte of every key the static key backend wraps. +const WRAP_VERSION: u8 = 1; +/// The AES-GCM nonce length. +const NONCE_LEN: usize = 12; + +/// The static key backend ([`KeyVendor::Static`]): a 256-bit vendor key +/// held in the application's memory. +/// +/// **For development, or for customers who manage the key themselves.** +/// The vendor key is always in memory, so there is no vendor that can see +/// or refuse an unwrap. For that reason it offers derived data keys only +/// (ADR-0009): [`KeysetRecord::create`](crate::KeysetRecord::create) +/// refuses wrapped data keys with +/// [`RecordError::ModeUnsupported`](crate::RecordError::ModeUnsupported). +/// +/// Each vendor key has a reference (`key_ref`), which the keyset record +/// stores. Hold more than one key ([`with_key`](Self::with_key)) to rotate: +/// rewrap each record under the new reference, then drop the old key. +/// +/// A key is wrapped with AES-256-GCM under a random 96-bit nonce, with the +/// associated data bound: `version (1) ‖ nonce (12) ‖ ciphertext ‖ tag`. +pub struct StaticKeyVendor { + keys: BTreeMap, +} + +impl fmt::Debug for StaticKeyVendor { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.debug_struct("StaticKeyVendor") + .field("key_refs", &self.keys.keys().collect::>()) + .finish() + } +} + +impl StaticKeyVendor { + /// A static key backend with one vendor key, named `key_ref`. + pub fn new(key_ref: impl Into, key: KeyBytes) -> Self { + Self { + keys: BTreeMap::new(), + } + .with_key(key_ref, key) + } + + /// Add another vendor key, named `key_ref`. A key of the same name is + /// replaced. + pub fn with_key(mut self, key_ref: impl Into, key: KeyBytes) -> Self { + let _ = self.keys.insert(key_ref.into(), key); + self + } + + fn cipher(&self, key_ref: &str) -> Result { + let key = self.keys.get(key_ref).ok_or_else(|| { + failure( + BackendErrorKind::NotFound, + "no static key has this reference", + ) + })?; + Aes256Gcm::new_from_slice(key.risky_ref()).map_err(|_| { + failure( + BackendErrorKind::NotFound, + "the static key has a bad length", + ) + }) + } +} + +fn failure(kind: BackendErrorKind, message: &'static str) -> Error { + Error::Backend { + vendor: KeyVendor::Static.name(), + kind, + source: message.into(), + } +} + +impl KeyVendorBackend for StaticKeyVendor { + fn vendor(&self) -> KeyVendor { + KeyVendor::Static + } + + fn supports_mode(&self, mode: KeyMode) -> bool { + mode == KeyMode::Derived + } + + async fn wrap(&self, key_ref: &str, key: &KeyBytes, aad: &[u8]) -> Result, Error> { + let cipher = self.cipher(key_ref)?; + let nonce = random_bytes::(KeyVendor::Static)?; + let sealed = cipher + .encrypt( + Nonce::from_slice(&nonce), + Payload { + msg: key.risky_ref(), + aad, + }, + ) + .map_err(|_| { + failure( + BackendErrorKind::Unavailable, + "the static key could not wrap", + ) + })?; + let mut out = Vec::with_capacity(1 + NONCE_LEN + sealed.len()); + out.push(WRAP_VERSION); + out.extend_from_slice(&nonce); + out.extend_from_slice(&sealed); + Ok(out) + } + + async fn unwrap(&self, key_ref: &str, wrapped: &[u8], aad: &[u8]) -> Result { + let invalid = || { + failure( + BackendErrorKind::InvalidCiphertext, + "the wrapped key does not open", + ) + }; + let cipher = self.cipher(key_ref)?; + let (version, rest) = wrapped.split_first().ok_or_else(invalid)?; + if *version != WRAP_VERSION || rest.len() < NONCE_LEN { + return Err(invalid()); + } + let (nonce, sealed) = rest.split_at(NONCE_LEN); + let opened = Zeroizing::new( + cipher + .decrypt(Nonce::from_slice(nonce), Payload { msg: sealed, aad }) + .map_err(|_| invalid())?, + ); + let key: [u8; 32] = opened.as_slice().try_into().map_err(|_| invalid())?; + Ok(Protected::new(key)) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn vendor() -> StaticKeyVendor { + StaticKeyVendor::new("dev", Protected::new([3u8; 32])) + } + + fn kind(error: Error) -> BackendErrorKind { + match error { + Error::Backend { vendor, kind, .. } => { + assert_eq!(vendor, "static"); + kind + } + other => panic!("expected a backend error, got {other:?}"), + } + } + + #[tokio::test] + async fn a_key_round_trips_and_binds_its_aad() { + let vendor = vendor(); + let key = Protected::new([9u8; 32]); + let wrapped = vendor.wrap("dev", &key, b"aad").await.unwrap(); + assert_eq!(wrapped.len(), 1 + 12 + 32 + 16); + assert_eq!(wrapped[0], WRAP_VERSION); + let opened = vendor.unwrap("dev", &wrapped, b"aad").await.unwrap(); + assert_eq!(opened.risky_ref(), &[9u8; 32]); + // A fresh nonce each time. + assert_ne!(wrapped, vendor.wrap("dev", &key, b"aad").await.unwrap()); + + let refused = vendor.unwrap("dev", &wrapped, b"other").await.unwrap_err(); + assert_eq!(kind(refused), BackendErrorKind::InvalidCiphertext); + let mut damaged = wrapped.clone(); + damaged[20] ^= 1; + assert_eq!( + kind(vendor.unwrap("dev", &damaged, b"aad").await.unwrap_err()), + BackendErrorKind::InvalidCiphertext + ); + let mut relabelled = wrapped.clone(); + relabelled[0] = 2; + assert_eq!( + kind(vendor.unwrap("dev", &relabelled, b"aad").await.unwrap_err()), + BackendErrorKind::InvalidCiphertext + ); + assert_eq!( + kind(vendor.unwrap("dev", &[1, 2], b"aad").await.unwrap_err()), + BackendErrorKind::InvalidCiphertext + ); + assert_eq!( + kind(vendor.unwrap("dev", &[], b"aad").await.unwrap_err()), + BackendErrorKind::InvalidCiphertext + ); + } + + #[tokio::test] + async fn another_key_does_not_unwrap_and_an_unknown_ref_is_not_found() { + let vendor = vendor().with_key("next", Protected::new([4u8; 32])); + let wrapped = vendor + .wrap("dev", &Protected::new([9u8; 32]), b"aad") + .await + .unwrap(); + assert_eq!( + kind(vendor.unwrap("next", &wrapped, b"aad").await.unwrap_err()), + BackendErrorKind::InvalidCiphertext + ); + assert_eq!( + kind(vendor.unwrap("gone", &wrapped, b"aad").await.unwrap_err()), + BackendErrorKind::NotFound + ); + } + + #[test] + fn it_offers_derived_data_keys_only_and_prints_no_key() { + let vendor = vendor(); + assert!(vendor.supports_mode(KeyMode::Derived)); + assert!(!vendor.supports_mode(KeyMode::Wrapped)); + assert_eq!(vendor.vendor(), KeyVendor::Static); + assert_eq!( + format!("{vendor:?}"), + r#"StaticKeyVendor { key_refs: ["dev"] }"# + ); + } +} diff --git a/packages/stack-kms-vendor/src/transport.rs b/packages/stack-kms-vendor/src/transport.rs new file mode 100644 index 0000000000..65aa47a8dd --- /dev/null +++ b/packages/stack-kms-vendor/src/transport.rs @@ -0,0 +1,256 @@ +//! The HTTP seam a vendor client sends through. +//! +//! The official vendor SDKs need tokio and hyper and do not build for +//! wasm32-wasip1 (ADR-0011). Each vendor client is therefore a small REST +//! client that signs its own requests and sends them through +//! [`VendorTransport`]. Native Rust implements it over an HTTP client; the +//! WASI guest implements it over its `transport_send` host import. The +//! vendor code is the same on both. + +use std::future::Future; +use std::time::Duration; + +use stack_kms::{BackendErrorKind, MaybeSend}; +use zeroize::Zeroizing; + +/// One signed HTTP request to a vendor. +/// +/// The body can hold a key in the clear (an AWS `Encrypt` request) and the +/// headers can hold a credential (a session token, a bearer token), so the +/// body is wiped on drop and `Debug` shows only the method, the URL and +/// the header names. +#[derive(Clone, PartialEq, Eq)] +pub struct VendorRequest { + /// The HTTP method, upper case (`POST`). + pub method: String, + /// The full URL. + pub url: String, + /// The headers, signature included, in the order they are sent. + pub headers: Vec<(String, String)>, + /// The body. Wiped on drop. + pub body: Zeroizing>, +} + +impl VendorRequest { + /// A request with no headers and an empty body. + pub fn new(method: impl Into, url: impl Into) -> Self { + Self { + method: method.into(), + url: url.into(), + headers: Vec::new(), + body: Zeroizing::new(Vec::new()), + } + } + + /// Add a header. + pub fn header(mut self, name: impl Into, value: impl Into) -> Self { + self.headers.push((name.into(), value.into())); + self + } + + /// Set the body. + pub fn body(mut self, body: Vec) -> Self { + self.body = Zeroizing::new(body); + self + } + + /// The value of the first header named `name`, compared without case. + pub fn header_value(&self, name: &str) -> Option<&str> { + header_value(&self.headers, name) + } +} + +impl std::fmt::Debug for VendorRequest { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("VendorRequest") + .field("method", &self.method) + .field("url", &self.url) + .field( + "headers", + &self.headers.iter().map(|(n, _)| n).collect::>(), + ) + .field("body_len", &self.body.len()) + .finish() + } +} + +/// A vendor's HTTP response. +/// +/// The body can hold a key in the clear (an AWS `Decrypt` response) or a +/// credential (an STS response), so it is wiped on drop and `Debug` shows +/// only the status and the header names. +#[derive(Clone, PartialEq, Eq)] +pub struct VendorResponse { + /// The HTTP status. + pub status: u16, + /// The headers. + pub headers: Vec<(String, String)>, + /// The body. Wiped on drop. + pub body: Zeroizing>, +} + +impl VendorResponse { + /// A response with `status`, no headers and `body`. + pub fn new(status: u16, body: Vec) -> Self { + Self { + status, + headers: Vec::new(), + body: Zeroizing::new(body), + } + } + + /// Add a header. + pub fn header(mut self, name: impl Into, value: impl Into) -> Self { + self.headers.push((name.into(), value.into())); + self + } + + /// The value of the first header named `name`, compared without case. + pub fn header_value(&self, name: &str) -> Option<&str> { + header_value(&self.headers, name) + } +} + +impl std::fmt::Debug for VendorResponse { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("VendorResponse") + .field("status", &self.status) + .field( + "headers", + &self.headers.iter().map(|(n, _)| n).collect::>(), + ) + .field("body_len", &self.body.len()) + .finish() + } +} + +fn header_value<'a>(headers: &'a [(String, String)], name: &str) -> Option<&'a str> { + headers + .iter() + .find(|(n, _)| n.eq_ignore_ascii_case(name)) + .map(|(_, v)| v.as_str()) +} + +/// The request did not get a response: the transport failed (DNS, TLS, +/// connection, timeout). +#[derive(Debug, thiserror::Error)] +#[error("the vendor request got no response: {0}")] +pub struct TransportError(pub String); + +/// The HTTP seam a vendor client sends through, of the same shape as +/// [`ZeroKMSConnection`](stack_kms::ZeroKMSConnection). +/// +/// The returned futures are bounded by [`MaybeSend`]: `Send` on native +/// targets, unbounded on wasm32. A transport gives the request no meaning: +/// it sends the bytes and returns the response. The vendor client signs +/// the request before, and classifies the response after +/// ([`classify_status`]). +pub trait VendorTransport { + /// Send `request` and return the response, whatever its status. + fn send( + &self, + request: VendorRequest, + ) -> impl Future> + MaybeSend; + + /// Wait for `duration`. A key vendor backend uses it to back off after + /// a rate limit ([`KeyVendorBackend::sleep`](crate::KeyVendorBackend::sleep)). + fn sleep(&self, duration: Duration) -> impl Future + MaybeSend; +} + +/// The failure kind of an HTTP status that is not a success, in terms that +/// hold for every vendor. `None` for a 2xx status. +/// +/// A vendor client may refine it from the response body (for example AWS +/// returns 400 for both a damaged ciphertext and a throttle). +pub fn classify_status(status: u16) -> Option { + match status { + 200..=299 => None, + 401 | 403 => Some(BackendErrorKind::AccessDenied), + 404 => Some(BackendErrorKind::NotFound), + 429 => Some(BackendErrorKind::Throttled), + 500..=599 => Some(BackendErrorKind::Unavailable), + _ => Some(BackendErrorKind::InvalidCiphertext), + } +} + +#[cfg(test)] +mod tests { + use std::sync::Mutex; + + use super::*; + + /// A transport that answers from a script, as the vendor crates' tests + /// will over recorded fixtures. + struct Scripted { + responses: Mutex>, + sent: Mutex>, + } + + impl VendorTransport for Scripted { + async fn send(&self, request: VendorRequest) -> Result { + self.sent.lock().unwrap().push(request); + self.responses + .lock() + .unwrap() + .pop() + .ok_or_else(|| TransportError("no response".into())) + } + + async fn sleep(&self, _duration: Duration) {} + } + + #[tokio::test] + async fn a_transport_sends_bytes_and_returns_the_response() { + let transport = Scripted { + responses: Mutex::new(vec![VendorResponse::new(200, b"{}".to_vec())]), + sent: Mutex::new(vec![]), + }; + let request = VendorRequest::new("POST", "https://kms.example/") + .header("x-amz-target", "TrentService.Decrypt") + .body(b"{}".to_vec()); + assert_eq!( + request.header_value("X-Amz-Target"), + Some("TrentService.Decrypt") + ); + let response = transport.send(request.clone()).await.unwrap(); + assert_eq!(response.status, 200); + assert_eq!( + *transport.sent.lock().unwrap(), + std::slice::from_ref(&request) + ); + assert!(transport.send(request).await.is_err()); + } + + #[test] + fn debug_never_shows_a_body_or_a_header_value() { + let request = VendorRequest::new("POST", "https://kms.example/") + .header("x-amz-security-token", "secret-token") + .body(b"secret-key".to_vec()); + let text = format!("{request:?}"); + assert!(text.contains("x-amz-security-token")); + assert!(!text.contains("secret-token")); + assert!(!text.contains("secret-key")); + let response = VendorResponse::new(200, b"secret-plaintext".to_vec()) + .header("set-cookie", "secret-cookie"); + let text = format!("{response:?}"); + assert!(!text.contains("secret-plaintext")); + assert!(!text.contains("secret-cookie")); + assert_eq!(response.header_value("Set-Cookie"), Some("secret-cookie")); + } + + #[test] + fn statuses_classify_the_same_for_every_vendor() { + assert_eq!(classify_status(200), None); + assert_eq!(classify_status(204), None); + assert_eq!(classify_status(401), Some(BackendErrorKind::AccessDenied)); + assert_eq!(classify_status(403), Some(BackendErrorKind::AccessDenied)); + assert_eq!(classify_status(404), Some(BackendErrorKind::NotFound)); + assert_eq!(classify_status(429), Some(BackendErrorKind::Throttled)); + assert_eq!(classify_status(500), Some(BackendErrorKind::Unavailable)); + assert_eq!(classify_status(503), Some(BackendErrorKind::Unavailable)); + assert_eq!( + classify_status(400), + Some(BackendErrorKind::InvalidCiphertext) + ); + } +} diff --git a/packages/stack-kms-vendor/src/vendor.rs b/packages/stack-kms-vendor/src/vendor.rs new file mode 100644 index 0000000000..cb838db2f2 --- /dev/null +++ b/packages/stack-kms-vendor/src/vendor.rs @@ -0,0 +1,233 @@ +//! The vendor seam: what a key vendor backend must provide. + +use std::future::Future; +use std::num::NonZeroUsize; +use std::time::Duration; + +use stack_kms::{Error, KeyMode, KeyVendor, MaybeSend}; +use vitaminc::protected::Protected; +use vitaminc::random::{Generatable, SafeRand}; + +/// A 256-bit key, wiped when it drops: a root key, an index key or a data +/// key, in the clear. +pub type KeyBytes = Protected<[u8; 32]>; + +/// What a key vendor backend must provide: wrap and unwrap a key under a +/// vendor key, with associated data. +/// +/// This is the whole contract of a vendor. The adapter +/// ([`VendorKeySource`](crate::VendorKeySource)) builds both key modes, the +/// keyset records, key lifetime and batching on top of it, so a vendor +/// crate (`stack-kms-aws` and so on) implements this trait and never +/// implements [`DataKeySource`](stack_kms::DataKeySource) or +/// [`IndexKeySource`](stack_kms::IndexKeySource) itself. +/// +/// # Rules for an implementation +/// +/// * **Bind the associated data.** `unwrap` must fail unless `aad` is the +/// `aad` given to `wrap`. A vendor with native associated data (AWS KMS +/// encryption context, Google Cloud KMS `additionalAuthenticatedData`) +/// passes it through. A vendor without it (Azure Key Vault), or with it +/// on some keys or versions only (Vault Transit), must bind it some other +/// way: wrap the bound key of +/// [`bound::bind_key`](crate::bound::bind_key) in place of the key, and +/// open it with [`bound::open_bound_key`](crate::bound::open_bound_key). +/// * **Report failures as [`Error::Backend`]**, with +/// [`KeyVendor::name`] as `vendor` and the nearest +/// [`BackendErrorKind`](stack_kms::BackendErrorKind). A wrong key, a wrong +/// `aad` or damaged bytes is `InvalidCiphertext`. A rate limit is +/// `Throttled`: the adapter backs off and retries it. Never put key +/// material or plaintext in an error. +/// * **`key_ref` is the versioned vendor key reference** from the keyset +/// record (an AWS key ARN, a Vault key name, a Google Cloud KMS crypto +/// key name, an Azure key id). Its format is the vendor's. A vendor key +/// that rotates behind one name is versioned by the vendor: its +/// ciphertexts name the version that made them. +pub trait KeyVendorBackend: Send + Sync { + /// Which key vendor backend this is. + fn vendor(&self) -> KeyVendor; + + /// Whether this vendor offers `mode`. The default is both modes. The + /// static key backend offers derived data keys only. + fn supports_mode(&self, mode: KeyMode) -> bool { + let _ = mode; + true + } + + /// Whether the key references `a` and `b` name the same vendor key, + /// perhaps at different versions of it. + /// + /// [`KeysetRecord::rewrap`](crate::KeysetRecord::rewrap) asks this for a + /// record in the wrapped data key mode. Such a record may move to a new + /// version of its key, but not to another key: each stored value's data + /// key stays wrapped under the key that sealed it, and the adapter + /// unwraps it with the record's current key reference. + /// + /// The default is string equality. A vendor whose key references can + /// name a version (Google Cloud KMS `…/cryptoKeyVersions/{n}`, an Azure + /// key identifier with `/{version}`) overrides it to ignore the version. + /// Answer `false` when unsure: a false `true` lets a rewrap make stored + /// values unreadable, and a false `false` only refuses a rewrap. + fn same_key(&self, a: &str, b: &str) -> bool { + a == b + } + + /// Wrap `key` under the vendor key `key_ref`, binding `aad`. + fn wrap( + &self, + key_ref: &str, + key: &KeyBytes, + aad: &[u8], + ) -> impl Future, Error>> + MaybeSend; + + /// Unwrap `wrapped` under the vendor key `key_ref`. Fails unless `aad` + /// is the `aad` the key was wrapped with. + fn unwrap( + &self, + key_ref: &str, + wrapped: &[u8], + aad: &[u8], + ) -> impl Future> + MaybeSend; + + /// Generate a fresh data key and wrap it under `key_ref`, binding `aad`. + /// Returns the key and its wrapped form. + /// + /// The default generates the key locally with a CSPRNG and calls + /// [`wrap`](Self::wrap): one vendor call either way. A vendor with a + /// native operation (AWS `GenerateDataKey`) may override it. + fn generate_wrapped( + &self, + key_ref: &str, + aad: &[u8], + ) -> impl Future), Error>> + MaybeSend { + async move { + let key = random_key(self.vendor())?; + let wrapped = self.wrap(key_ref, &key, aad).await?; + Ok((key, wrapped)) + } + } + + /// The most keys one [`wrap_batch`](Self::wrap_batch) or + /// [`unwrap_batch`](Self::unwrap_batch) call takes, or `None` (the + /// default) when the vendor has no batch operation. + /// + /// When it is `Some(n)`, the adapter wraps and unwraps the wrapped data + /// keys of a batch in chunks of at most `n` keys, one batch call per + /// chunk, and generates each data key locally with the CSPRNG. When it + /// is `None`, the adapter makes one call per key + /// ([`generate_wrapped`](Self::generate_wrapped) or + /// [`unwrap`](Self::unwrap)). The keyset record key is always one + /// [`unwrap`](Self::unwrap) call. + fn batch_limit(&self) -> Option { + None + } + + /// Wrap each key under `key_ref`, binding its associated data, in one + /// vendor call. Returns the wrapped keys in the order of `items`. + /// + /// The adapter calls it only when [`batch_limit`](Self::batch_limit) is + /// `Some(n)`, with at most `n` items. The rules of + /// [`wrap`](Self::wrap) apply to each item. A batch fails whole: return + /// the first failure. The default wraps the items one at a time. + fn wrap_batch( + &self, + key_ref: &str, + items: &[WrapItem<'_>], + ) -> impl Future>, Error>> + MaybeSend { + async move { + let mut out = Vec::with_capacity(items.len()); + for item in items { + out.push(self.wrap(key_ref, item.key, item.aad).await?); + } + Ok(out) + } + } + + /// Unwrap each wrapped key under `key_ref`, in one vendor call. Returns + /// the keys in the order of `items`. + /// + /// The adapter calls it only when [`batch_limit`](Self::batch_limit) is + /// `Some(n)`, with at most `n` items. The rules of + /// [`unwrap`](Self::unwrap) apply to each item. A batch fails whole: + /// return the first failure. The default unwraps the items one at a + /// time. + fn unwrap_batch( + &self, + key_ref: &str, + items: &[UnwrapItem<'_>], + ) -> impl Future, Error>> + MaybeSend { + async move { + let mut out = Vec::with_capacity(items.len()); + for item in items { + out.push(self.unwrap(key_ref, item.wrapped, item.aad).await?); + } + Ok(out) + } + } + + /// Wait for `duration` before a retry. + /// + /// The adapter calls this to back off after a `Throttled` failure. The + /// default returns at once, because this crate has no timer of its own + /// and must run on native Rust and in the WASI guest alike. A vendor + /// overrides it with its transport's timer + /// ([`VendorTransport::sleep`](crate::VendorTransport::sleep)). + fn sleep(&self, duration: Duration) -> impl Future + MaybeSend { + let _ = duration; + std::future::ready(()) + } +} + +/// One key of a [`KeyVendorBackend::wrap_batch`] call. +#[derive(Clone, Copy)] +pub struct WrapItem<'a> { + /// The key to wrap. + pub key: &'a KeyBytes, + /// The associated data to bind. + pub aad: &'a [u8], +} + +impl std::fmt::Debug for WrapItem<'_> { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("WrapItem") + .field("aad_len", &self.aad.len()) + .finish_non_exhaustive() + } +} + +/// One wrapped key of a [`KeyVendorBackend::unwrap_batch`] call. +#[derive(Clone, Copy, Debug)] +pub struct UnwrapItem<'a> { + /// The wrapped key. + pub wrapped: &'a [u8], + /// The associated data it was wrapped with. + pub aad: &'a [u8], +} + +/// A fresh random 256-bit key, from the CSPRNG. +pub(crate) fn random_key(vendor: KeyVendor) -> Result { + let mut rng = SafeRand::from_entropy().map_err(|error| Error::Backend { + vendor: vendor.name(), + kind: stack_kms::BackendErrorKind::Unavailable, + source: Box::new(error), + })?; + KeyBytes::random(&mut rng).map_err(|error| Error::Backend { + vendor: vendor.name(), + kind: stack_kms::BackendErrorKind::Unavailable, + source: Box::new(error), + }) +} + +/// Fresh random bytes, from the CSPRNG. +pub(crate) fn random_bytes(vendor: KeyVendor) -> Result<[u8; N], Error> { + let mut rng = SafeRand::from_entropy().map_err(|error| Error::Backend { + vendor: vendor.name(), + kind: stack_kms::BackendErrorKind::Unavailable, + source: Box::new(error), + })?; + <[u8; N]>::random(&mut rng).map_err(|error| Error::Backend { + vendor: vendor.name(), + kind: stack_kms::BackendErrorKind::Unavailable, + source: Box::new(error), + }) +} diff --git a/packages/stack-kms-vendor/tasks.toml b/packages/stack-kms-vendor/tasks.toml new file mode 100644 index 0000000000..4101d6382d --- /dev/null +++ b/packages/stack-kms-vendor/tasks.toml @@ -0,0 +1,7 @@ +# Rustdoc with warnings as errors: a broken intra-doc link or a rustdoc +# warning fails the build. The root `doc` task fans out over every +# `doc:`; the root `test:doc` runs the doc examples. +["doc:stack-kms-vendor"] +description = "Build docs for stack-kms-vendor with all features (warnings are errors)" +env = { RUSTDOCFLAGS = "-D warnings" } +run = "cargo doc -p stack-kms-vendor --no-deps --all-features" diff --git a/packages/stack-kms-vendor/tests/contract.rs b/packages/stack-kms-vendor/tests/contract.rs new file mode 100644 index 0000000000..e89115c102 --- /dev/null +++ b/packages/stack-kms-vendor/tests/contract.rs @@ -0,0 +1,33 @@ +//! The shared contract suite, run against the static key backend and the +//! in-memory fake vendor (both key modes). + +use stack_encrypt::kms::KeyVendor; +use stack_kms_vendor::contract; +use stack_kms_vendor::{FakeVendor, StaticKeyVendor}; +use vitaminc::protected::Protected; + +#[tokio::test] +async fn the_static_key_backend_meets_the_contract() { + contract::run( + || StaticKeyVendor::new("dev-key/v1", Protected::new([0x11u8; 32])), + "dev-key/v1", + ) + .await; +} + +#[tokio::test] +async fn the_fake_vendor_meets_the_contract_in_both_modes() { + for vendor in [KeyVendor::Aws, KeyVendor::Vault] { + contract::run(|| FakeVendor::new(vendor), FakeVendor::KEY_REF).await; + } +} + +#[tokio::test] +async fn the_fake_vendor_meets_the_contract_with_vendor_batches() { + let limit = std::num::NonZeroUsize::new(4).unwrap(); + contract::run( + || FakeVendor::new(KeyVendor::Vault).with_batch_limit(limit), + FakeVendor::KEY_REF, + ) + .await; +} diff --git a/packages/stack-kms-vendor/tests/router.rs b/packages/stack-kms-vendor/tests/router.rs new file mode 100644 index 0000000000..9c79ffc738 --- /dev/null +++ b/packages/stack-kms-vendor/tests/router.rs @@ -0,0 +1,380 @@ +//! ZeroKMS and one key vendor backend in one cipher, keyset records as +//! config, and the root key TTL in real time. + +use std::time::Duration; + +use stack_encrypt::kms::{ + BackendErrorKind, Error as KmsError, FakeDataKeySource, IdentifiedBy, KeyBackend, KeyMode, + KeyVendor, +}; +use stack_encrypt::{nonempty, CipherText, Error, StackCipher, StackCipherText}; +use stack_kms_vendor::contract::Observed; +use stack_kms_vendor::{ + FakeVendor, KeyRouter, KeysetRecord, RecordError, StaticKeyVendor, VendorKeySource, +}; +use vitaminc::protected::Protected; + +fn leaf_backend(sealed: &StackCipherText) -> (KeyBackend, Option) { + match sealed { + CipherText::Single(leaf) => (leaf.key_backend(), leaf.key_mode()), + _ => panic!("one leaf"), + } +} + +#[tokio::test] +async fn one_cipher_routes_between_zerokms_and_one_vendor() { + let vendor = FakeVendor::new(KeyVendor::Aws); + let wrapped = KeysetRecord::create( + &vendor, + FakeVendor::KEY_REF, + KeyMode::Wrapped, + Some("acme".into()), + ) + .await + .unwrap(); + let derived = KeysetRecord::create(&vendor, FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + let texts = [wrapped.to_string(), derived.to_string()]; + let source = VendorKeySource::from_texts(vendor, texts.iter().map(String::as_str)).unwrap(); + let cipher = StackCipher::builder() + .kms(KeyRouter::new(FakeDataKeySource::new(), source)) + .init() + .await + .unwrap(); + let context = nonempty!("users").with("email"); + + // Naming no keyset is ZeroKMS's default keyset, and seals v1. + let zerokms = cipher + .default_keyset() + .encrypt_tree("z".to_string(), context) + .await + .unwrap(); + assert_eq!(leaf_backend(&zerokms), (KeyBackend::ZeroKms, None)); + assert!(cipher.default_keyset().prf().is_some()); + + // A record's name and id select the vendor, and seal v2. + let acme = cipher + .keyset(IdentifiedBy::Name("acme".to_string().into())) + .await + .unwrap(); + assert_eq!(acme.keyset_id(), wrapped.id()); + let sealed_wrapped = acme.encrypt_tree("w".to_string(), context).await.unwrap(); + assert_eq!( + leaf_backend(&sealed_wrapped), + (KeyBackend::Vendor(KeyVendor::Aws), Some(KeyMode::Wrapped)) + ); + let other = cipher.keyset(derived.id()).await.unwrap(); + let sealed_derived = other.encrypt_tree("d".to_string(), context).await.unwrap(); + assert_eq!( + leaf_backend(&sealed_derived), + (KeyBackend::Vendor(KeyVendor::Aws), Some(KeyMode::Derived)) + ); + + // Each leaf opens through the backend that sealed it. + for (sealed, expected) in [(zerokms, "z"), (sealed_wrapped, "w"), (sealed_derived, "d")] { + let opened: String = cipher.decrypt(sealed, context).await.unwrap(); + assert_eq!(opened, expected); + } + + // A term through a vendor keyset is derived inside the settle, and + // differs from the ZeroKMS keyset's term for the same value. + let vendor_term = acme.equality_term("x", context).await.unwrap(); + let zerokms_term = cipher + .default_keyset() + .equality_term("x", context) + .await + .unwrap(); + assert_ne!(vendor_term.as_bytes(), zerokms_term.as_bytes()); +} + +#[tokio::test] +async fn a_vendor_leaf_relabelled_as_zerokms_is_refused() { + let vendor = FakeVendor::new(KeyVendor::Vault); + let record = KeysetRecord::create(&vendor, FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + let cipher = StackCipher::builder() + .kms(KeyRouter::new( + FakeDataKeySource::new(), + VendorKeySource::new(vendor, [record.clone()]).unwrap(), + )) + .init() + .await + .unwrap(); + let keyset = cipher.keyset(record.id()).await.unwrap(); + let sealed = keyset.encrypt_tree("v".to_string(), b"ctx").await.unwrap(); + let CipherText::Single(leaf) = sealed else { + panic!("one leaf") + }; + let (keyset_id, iv, tag, ct) = leaf.into_parts(); + let v1 = stack_encrypt::SealedValue::from_parts(keyset_id, iv, tag, ct).unwrap(); + match cipher + .decrypt::(CipherText::Single(v1), b"ctx") + .await + { + Err(Error::ForeignBackend { expected, found }) => { + assert_eq!(expected, KeyBackend::Vendor(KeyVendor::Vault)); + assert_eq!(found, KeyBackend::ZeroKms); + } + other => panic!("expected ForeignBackend, got {other:?}"), + } +} + +#[tokio::test] +async fn records_of_two_vendors_are_refused() { + let aws = FakeVendor::new(KeyVendor::Aws); + let vault = FakeVendor::new(KeyVendor::Vault); + let a = KeysetRecord::create(&aws, FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + let v = KeysetRecord::create(&vault, FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + match VendorKeySource::new(aws, [a.clone(), v.clone()]) { + Err(RecordError::SecondVendor { first, second }) => { + assert_eq!((first, second), (KeyVendor::Aws, KeyVendor::Vault)); + } + other => panic!("expected SecondVendor, got {other:?}"), + } + match VendorKeySource::new(vault, [a]) { + Err(RecordError::ForeignVendor { + expected, found, .. + }) => assert_eq!((expected, found), (KeyVendor::Vault, KeyVendor::Aws)), + other => panic!("expected ForeignVendor, got {other:?}"), + } +} + +#[tokio::test] +async fn the_static_backend_refuses_wrapped_data_keys() { + let vendor = StaticKeyVendor::new("k", Protected::new([1u8; 32])); + assert!(matches!( + KeysetRecord::create(&vendor, "k", KeyMode::Wrapped, None).await, + Err(RecordError::ModeUnsupported { + vendor: KeyVendor::Static, + mode: KeyMode::Wrapped + }) + )); + let forged = KeysetRecord::from_parts( + uuid::Uuid::new_v4(), + None, + KeyVendor::Static, + KeyMode::Wrapped, + "k".into(), + vec![1; 60], + ) + .unwrap(); + assert!(matches!( + VendorKeySource::new(vendor, [forged]), + Err(RecordError::ModeUnsupported { .. }) + )); +} + +#[tokio::test] +async fn a_record_edited_in_its_bound_fields_does_not_unwrap() { + let vendor = FakeVendor::new(KeyVendor::Aws); + let record = KeysetRecord::create(&vendor, FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + let moved = KeysetRecord::from_parts( + uuid::Uuid::new_v4(), + None, + record.vendor(), + record.mode(), + record.key_ref().into(), + record.wrapped_key().to_vec(), + ) + .unwrap(); + let cipher = StackCipher::builder() + .kms(VendorKeySource::new(vendor, [moved]).unwrap()) + .init() + .await + .unwrap(); + match cipher + .default_keyset() + .encrypt_tree("v".to_string(), b"ctx") + .await + { + Err(Error::Kms(KmsError::Backend { + kind: BackendErrorKind::InvalidCiphertext, + .. + })) => {} + other => panic!("a record moved to another id must not unwrap, got {other:?}"), + } +} + +/// Seal one value and derive one term under `record`, rewrap the record to +/// `new_ref`, and check the old value opens and the term is the same. +async fn rewrap_keeps_old_values( + make: impl Fn() -> B, + record: KeysetRecord, + new_ref: &str, +) { + let old = StackCipher::builder() + .kms(VendorKeySource::new(make(), [record.clone()]).unwrap()) + .init() + .await + .unwrap(); + let sealed = old + .default_keyset() + .encrypt_tree("v".to_string(), b"ctx") + .await + .unwrap(); + let term = old + .default_keyset() + .equality_term("v", nonempty!("ctx")) + .await + .unwrap(); + + let rewrapped = record.rewrap(&make(), new_ref).await.unwrap(); + assert_eq!(rewrapped.id(), record.id()); + assert_eq!(rewrapped.mode(), record.mode()); + assert_eq!(rewrapped.key_ref(), new_ref); + let new = StackCipher::builder() + .kms(VendorKeySource::new(make(), [rewrapped]).unwrap()) + .init() + .await + .unwrap(); + let opened: String = new.decrypt(sealed, b"ctx").await.unwrap(); + assert_eq!(opened, "v", "a value sealed before the rewrap opens"); + let again = new + .default_keyset() + .equality_term("v", nonempty!("ctx")) + .await + .unwrap(); + assert_eq!( + term.as_bytes(), + again.as_bytes(), + "the index key did not change" + ); +} + +#[tokio::test] +async fn a_wrapped_record_does_not_rewrap_to_another_key() { + let vendor = FakeVendor::new(KeyVendor::Aws).with_key("next/v2", Protected::new([2u8; 32])); + let record = KeysetRecord::create(&vendor, FakeVendor::KEY_REF, KeyMode::Wrapped, None) + .await + .unwrap(); + let before = vendor.calls(); + match record.rewrap(&vendor, "next/v2").await { + Err(RecordError::RewrapToAnotherKey { keyset, from, to }) => { + assert_eq!(keyset, record.id()); + assert_eq!(from, FakeVendor::KEY_REF); + assert_eq!(to, "next/v2"); + } + other => panic!("a wrapped record must not move to another key, got {other:?}"), + } + assert_eq!(vendor.calls(), before, "refused before any vendor call"); +} + +#[tokio::test] +async fn a_wrapped_record_rewraps_under_its_own_key_and_keeps_old_values() { + let vendor = || FakeVendor::new(KeyVendor::Aws); + let record = KeysetRecord::create(&vendor(), FakeVendor::KEY_REF, KeyMode::Wrapped, None) + .await + .unwrap(); + rewrap_keeps_old_values(vendor, record, FakeVendor::KEY_REF).await; +} + +#[tokio::test] +async fn a_derived_record_rewraps_to_another_key_and_keeps_old_values() { + let vendor = || { + StaticKeyVendor::new("key-a", Protected::new([1u8; 32])) + .with_key("key-b", Protected::new([2u8; 32])) + }; + let record = KeysetRecord::create(&vendor(), "key-a", KeyMode::Derived, None) + .await + .unwrap(); + rewrap_keeps_old_values(vendor, record.clone(), "key-b").await; + + // The fake vendor too, so a wrapped-capable vendor is covered. + let fake = || FakeVendor::new(KeyVendor::Aws).with_key("next/v2", Protected::new([2u8; 32])); + let record = KeysetRecord::create(&fake(), FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + rewrap_keeps_old_values(fake, record, "next/v2").await; +} + +#[tokio::test] +async fn an_expired_root_key_is_unwrapped_again() { + let (backend, observer) = Observed::new(FakeVendor::new(KeyVendor::Gcp)); + let record = KeysetRecord::create(&backend, FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + let cipher = StackCipher::builder() + .kms( + VendorKeySource::new(backend, [record]) + .unwrap() + .with_root_key_ttl(Duration::from_millis(100)), + ) + .init() + .await + .unwrap(); + let keyset = cipher.default_keyset(); + let _ = keyset.equality_term("v", nonempty!("ctx")).await.unwrap(); + let _ = keyset.equality_term("v", nonempty!("ctx")).await.unwrap(); + assert_eq!(observer.unwraps(), 1, "kept within its TTL"); + tokio::time::sleep(Duration::from_millis(150)).await; + let _ = keyset.equality_term("v", nonempty!("ctx")).await.unwrap(); + assert_eq!(observer.unwraps(), 2, "unwrapped again after its TTL"); +} + +#[tokio::test] +async fn several_records_need_a_named_default() { + let vendor = FakeVendor::new(KeyVendor::Aws); + let a = KeysetRecord::create(&vendor, FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + let b = KeysetRecord::create(&vendor, FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + let source = VendorKeySource::new(vendor, [a.clone(), b]).unwrap(); + assert!( + StackCipher::builder().kms(source).init().await.is_err(), + "no default keyset among several records" + ); + + let vendor = FakeVendor::new(KeyVendor::Aws); + let source = VendorKeySource::new(vendor, [a.clone()]) + .unwrap() + .with_default_keyset(a.id()) + .unwrap(); + let cipher = StackCipher::builder().kms(source).init().await.unwrap(); + assert_eq!(cipher.default_keyset().keyset_id(), a.id()); + assert!(matches!( + VendorKeySource::new(FakeVendor::new(KeyVendor::Aws), [a]) + .unwrap() + .with_default_keyset(uuid::Uuid::nil()), + Err(RecordError::UnknownDefault) + )); +} + +/// A client with a key vendor backend and no ZeroKMS: naming no keyset is +/// the vendor's default keyset, and anything else goes to the ZeroKMS side. +#[tokio::test] +async fn a_vendor_default_router_selects_the_vendor_for_no_keyset() { + let vendor = FakeVendor::new(KeyVendor::Aws); + let record = KeysetRecord::create(&vendor, FakeVendor::KEY_REF, KeyMode::Derived, None) + .await + .unwrap(); + let source = VendorKeySource::new(vendor, [record.clone()]).unwrap(); + let router = KeyRouter::new(FakeDataKeySource::new(), source).with_vendor_default(); + assert_eq!( + stack_encrypt::kms::DataKeySource::key_backend(&router), + KeyBackend::Vendor(KeyVendor::Aws) + ); + let cipher = StackCipher::builder().kms(router).init().await.unwrap(); + assert_eq!(cipher.default_keyset().keyset_id(), record.id()); + let context = nonempty!("users").with("email"); + let sealed = cipher + .default_keyset() + .encrypt_tree("v".to_string(), context) + .await + .unwrap(); + assert_eq!( + leaf_backend(&sealed), + (KeyBackend::Vendor(KeyVendor::Aws), Some(KeyMode::Derived)) + ); + let opened: String = cipher.decrypt(sealed, context).await.unwrap(); + assert_eq!(opened, "v"); +} diff --git a/packages/stack-kms/src/backend.rs b/packages/stack-kms/src/backend.rs new file mode 100644 index 0000000000..8efd91be3f --- /dev/null +++ b/packages/stack-kms/src/backend.rs @@ -0,0 +1,232 @@ +//! Which key backend a key source is, and the key mode of a key vendor +//! backend's keyset. +//! +//! A sealed leaf names its key backend so that a reader can route it, or +//! refuse it, before it asks for a key. A ZeroKMS leaf names its backend by +//! its format version alone (the v1 leaf). A leaf sealed by a key vendor +//! backend is a v2 leaf, and carries one byte for the vendor +//! ([`KeyVendor::byte`]) and one byte for the key mode ([`KeyMode::byte`]). +//! ADR-0010 has the layout. +//! +//! # Stable byte values +//! +//! The byte values are a storage commitment: a database column holds them. +//! A value is never reused and never renumbered. The value `0` is reserved +//! as invalid for both fields, so a zeroed header never decodes. +//! +//! | vendor | byte | +//! |----------------------|------| +//! | (invalid) | 0 | +//! | AWS KMS | 1 | +//! | Google Cloud KMS | 2 | +//! | Azure Key Vault | 3 | +//! | HashiCorp Vault | 4 | +//! | static key | 5 | +//! +//! | key mode | byte | +//! |----------------------|------| +//! | (invalid) | 0 | +//! | wrapped data key | 1 | +//! | derived data key | 2 | + +use std::fmt; + +/// The service a keyset's data keys and index key come from: ZeroKMS, or +/// one key vendor backend. +/// +/// [`DataKeySource::key_backend`](crate::DataKeySource::key_backend) +/// returns it, and a cipher compares it with the backend a sealed leaf +/// names before it asks for a key. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub enum KeyBackend { + /// ZeroKMS. Its leaves are v1 leaves, which have no backend byte. + ZeroKms, + /// A key vendor backend. Its leaves are v2 leaves. + Vendor(KeyVendor), +} + +impl KeyBackend { + /// A stable, lower-case name for logs and error payloads. + pub const fn name(self) -> &'static str { + match self { + Self::ZeroKms => "zerokms", + Self::Vendor(vendor) => vendor.name(), + } + } +} + +impl fmt::Display for KeyBackend { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(self.name()) + } +} + +/// A key vendor backend: a key backend that is not ZeroKMS. +/// +/// Each variant has a stable byte ([`byte`](Self::byte)), which a v2 sealed +/// leaf stores. See the module docs for the table. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[non_exhaustive] +#[repr(u8)] +pub enum KeyVendor { + /// AWS KMS. + Aws = 1, + /// Google Cloud KMS. + Gcp = 2, + /// Azure Key Vault. + Azure = 3, + /// HashiCorp Vault Transit. + Vault = 4, + /// A static key, held in memory. + Static = 5, +} + +impl KeyVendor { + /// Every vendor, in byte order. + pub const ALL: [Self; 5] = [Self::Aws, Self::Gcp, Self::Azure, Self::Vault, Self::Static]; + + /// The byte a v2 sealed leaf stores for this vendor. + pub const fn byte(self) -> u8 { + self as u8 + } + + /// The vendor a v2 leaf's backend byte names, or `None` for a byte that + /// names no vendor (`0` included). + pub const fn from_byte(byte: u8) -> Option { + match byte { + 1 => Some(Self::Aws), + 2 => Some(Self::Gcp), + 3 => Some(Self::Azure), + 4 => Some(Self::Vault), + 5 => Some(Self::Static), + _ => None, + } + } + + /// A stable, lower-case name. A key vendor backend uses it as the + /// `vendor` of its [`Error::Backend`](crate::Error::Backend) and + /// [`Error::ZeroKmsOnly`](crate::Error::ZeroKmsOnly). + pub const fn name(self) -> &'static str { + match self { + Self::Aws => "aws-kms", + Self::Gcp => "gcp-kms", + Self::Azure => "azure-key-vault", + Self::Vault => "vault-transit", + Self::Static => "static", + } + } +} + +impl fmt::Display for KeyVendor { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(self.name()) + } +} + +/// The key mode of a keyset in a key vendor backend (ADR-0009). +/// +/// Each variant has a stable byte ([`byte`](Self::byte)), which a v2 sealed +/// leaf stores. ZeroKMS has no key mode. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u8)] +pub enum KeyMode { + /// Wrapped data key: each value has its own data key, wrapped by the + /// vendor key. The leaf's tag holds the wrapped data key. + Wrapped = 1, + /// Derived data key: each data key is derived from the keyset's root + /// key. The leaf's tag is empty, and its iv is the derivation salt. + Derived = 2, +} + +impl KeyMode { + /// Both modes, in byte order. + pub const ALL: [Self; 2] = [Self::Wrapped, Self::Derived]; + + /// The byte a v2 sealed leaf stores for this mode. + pub const fn byte(self) -> u8 { + self as u8 + } + + /// The mode a v2 leaf's mode byte names, or `None` for a byte that + /// names no mode (`0` included). + pub const fn from_byte(byte: u8) -> Option { + match byte { + 1 => Some(Self::Wrapped), + 2 => Some(Self::Derived), + _ => None, + } + } + + /// A stable, lower-case name for logs and error payloads. + pub const fn name(self) -> &'static str { + match self { + Self::Wrapped => "wrapped", + Self::Derived => "derived", + } + } +} + +impl fmt::Display for KeyMode { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(self.name()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The byte values are a storage commitment. This pin fails if one + /// moves. + #[test] + fn the_byte_values_are_pinned() { + let vendors: Vec<(u8, &str)> = KeyVendor::ALL + .iter() + .map(|vendor| (vendor.byte(), vendor.name())) + .collect(); + assert_eq!( + vendors, + [ + (1, "aws-kms"), + (2, "gcp-kms"), + (3, "azure-key-vault"), + (4, "vault-transit"), + (5, "static"), + ] + ); + let modes: Vec<(u8, &str)> = KeyMode::ALL + .iter() + .map(|mode| (mode.byte(), mode.name())) + .collect(); + assert_eq!(modes, [(1, "wrapped"), (2, "derived")]); + } + + #[test] + fn from_byte_inverts_byte_and_refuses_every_other_value() { + for byte in 0..=u8::MAX { + match KeyVendor::from_byte(byte) { + Some(vendor) => assert_eq!(vendor.byte(), byte), + None => assert!(!KeyVendor::ALL.iter().any(|v| v.byte() == byte), "{byte}"), + } + match KeyMode::from_byte(byte) { + Some(mode) => assert_eq!(mode.byte(), byte), + None => assert!(!KeyMode::ALL.iter().any(|m| m.byte() == byte), "{byte}"), + } + } + assert_eq!(KeyVendor::from_byte(0), None); + assert_eq!(KeyMode::from_byte(0), None); + } + + #[test] + fn names_display_as_themselves() { + assert_eq!(KeyBackend::ZeroKms.to_string(), "zerokms"); + assert_eq!(KeyBackend::ZeroKms.name(), "zerokms"); + for vendor in KeyVendor::ALL { + assert_eq!(vendor.to_string(), vendor.name()); + assert_eq!(KeyBackend::Vendor(vendor).to_string(), vendor.name()); + } + for mode in KeyMode::ALL { + assert_eq!(mode.to_string(), mode.name()); + } + } +} diff --git a/packages/stack-kms/src/errors.rs b/packages/stack-kms/src/errors.rs index 9287e6f18e..6f11bef6cc 100644 --- a/packages/stack-kms/src/errors.rs +++ b/packages/stack-kms/src/errors.rs @@ -340,6 +340,15 @@ mod codes { }, Error::InvalidEndpoint(_) => Error::InvalidEndpoint(InvalidEndpoint::Userinfo), Error::Unexpected(_) => Error::Unexpected("unexpected".into()), + Error::Backend { .. } => Error::Backend { + vendor: "test-vendor", + kind: BackendErrorKind::NotFound, + source: Box::new(std::io::Error::other("no key")), + }, + Error::ZeroKmsOnly { .. } => Error::ZeroKmsOnly { + vendor: "test-vendor", + feature: crate::payload::ZeroKmsFeature::Context, + }, ])); errors.extend(boxed(variants![ InvalidEndpoint::Parse(_) => InvalidEndpoint::Parse(url::ParseError::EmptyHost), @@ -500,6 +509,53 @@ mod codes { } } + /// A backend failure gives its vendor and kind, never the vendor's own + /// error, which stays behind `source()`. + #[test] + fn a_backend_failure_gives_the_vendor_and_kind_and_not_the_source() { + use serde_json::json; + for kind in [ + BackendErrorKind::NotFound, + BackendErrorKind::AccessDenied, + BackendErrorKind::Throttled, + BackendErrorKind::Unavailable, + BackendErrorKind::InvalidCiphertext, + ] { + let error = Error::Backend { + vendor: "test-vendor", + kind, + source: Box::new(std::io::Error::other("marker-detail")), + }; + assert_eq!( + serde_json::Value::Object(error.payload()), + json!({ "vendor": "test-vendor", "kind": format!("{kind:?}") }) + ); + assert_eq!(kind.to_string(), format!("{kind:?}")); + let shown = error.to_string(); + assert_eq!(shown, format!("Key backend test-vendor failed: {kind:?}")); + assert!(!shown.contains("marker-detail"), "{shown}"); + assert!(std::error::Error::source(&error) + .is_some_and(|source| source.to_string() == "marker-detail")); + } + } + + #[test] + fn a_zerokms_only_refusal_names_the_vendor_and_the_feature() { + use serde_json::json; + let error = Error::ZeroKmsOnly { + vendor: "test-vendor", + feature: crate::payload::ZeroKmsFeature::DecryptionPolicy, + }; + assert_eq!( + serde_json::Value::Object(error.payload()), + json!({ "vendor": "test-vendor", "feature": "decryption_policy" }) + ); + assert_eq!( + error.to_string(), + "Key backend test-vendor does not support the ZeroKMS feature decryption_policy" + ); + } + /// The builder's help names the variable that is wrong. #[cfg(feature = "http")] #[test] @@ -751,6 +807,75 @@ pub enum Error { #[error("Unexpected error: {0}")] #[diagnostic(code(stack_kms::unexpected))] Unexpected(String), + + /// A key vendor backend failed. `vendor` names the backend (for example + /// `"aws-kms"`) and `kind` says what failed, in terms that hold for + /// every vendor. The vendor's own error is behind `source()`; the + /// message carries only the vendor and the kind. + #[error("Key backend {vendor} failed: {kind}")] + #[diagnostic( + code(stack_kms::backend), + help("Read the `kind` field. NotFound and AccessDenied are configuration; Throttled and Unavailable can be retried; InvalidCiphertext means the stored key or value is damaged, or belongs to another key.") + )] + Backend { + vendor: &'static str, + kind: BackendErrorKind, + #[source] + source: Box, + }, + + /// A key vendor backend was asked for an input that only ZeroKMS can + /// honour (a key context, a decryption policy, an unverified context). + /// The backend refuses it rather than ignore it. + #[error("Key backend {vendor} does not support the ZeroKMS feature {feature}")] + #[diagnostic( + code(stack_kms::zerokms_only), + help("Lock contexts, decryption policies and unverified contexts need ZeroKMS. Use a ZeroKMS keyset, or remove the feature from the request.") + )] + ZeroKmsOnly { + vendor: &'static str, + feature: crate::payload::ZeroKmsFeature, + }, +} + +/// What failed in a key vendor backend, in terms that hold for every vendor. +/// +/// Each vendor client maps its own errors to one of these, so that a caller +/// can act on a failure without knowing the vendor. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[non_exhaustive] +pub enum BackendErrorKind { + /// The vendor key, or the key material it protects, does not exist. + NotFound, + /// The vendor refused the credential, or the credential has no grant + /// for this key or operation. + AccessDenied, + /// The vendor rate-limited the request. Retry later. + Throttled, + /// The vendor could not be reached, or did not answer. Retry later. + Unavailable, + /// The vendor could not unwrap the key material: it is damaged, or it + /// was wrapped by another key. + InvalidCiphertext, +} + +impl BackendErrorKind { + /// The kind as error payloads spell it. + pub const fn as_str(self) -> &'static str { + match self { + Self::NotFound => "NotFound", + Self::AccessDenied => "AccessDenied", + Self::Throttled => "Throttled", + Self::Unavailable => "Unavailable", + Self::InvalidCiphertext => "InvalidCiphertext", + } + } +} + +impl std::fmt::Display for BackendErrorKind { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(self.as_str()) + } } impl ErrorPayload for Error { @@ -760,6 +885,13 @@ impl ErrorPayload for Error { Self::RetrieveKey(error) => error.payload(), Self::LoadKeyset(error) => error.payload(), Self::Auth(error) => error.payload(), + Self::Backend { vendor, kind, .. } => { + payload([("vendor", (*vendor).into()), ("kind", kind.as_str().into())]) + } + Self::ZeroKmsOnly { vendor, feature } => payload([ + ("vendor", (*vendor).into()), + ("feature", feature.as_str().into()), + ]), Self::ConnectionInit(_) | Self::InvalidEndpoint(_) | Self::Unexpected(_) => { serde_json::Map::new() } diff --git a/packages/stack-kms/src/key.rs b/packages/stack-kms/src/key.rs index c919ba9a17..33657a739f 100644 --- a/packages/stack-kms/src/key.rs +++ b/packages/stack-kms/src/key.rs @@ -8,7 +8,7 @@ use serde::{Deserialize, Deserializer, Serialize, Serializer}; use sha2::{Digest, Sha256}; use std::ops::Deref; use uuid::Uuid; -use vitaminc::protected::{OpaqueDebug, TimingSafeEq}; +use vitaminc::protected::{Controlled, OpaqueDebug, Protected, TimingSafeEq}; use zeroize::{Zeroize, ZeroizeOnDrop, Zeroizing}; use zerokms_protocol::{DecryptionPolicy, ViturKeyMaterial}; @@ -226,6 +226,19 @@ impl IndexKey { Ok(Self(key)) } + /// Build the index key from raw key material that a key vendor backend + /// unwrapped or derived. + /// + /// The bytes must be the keyset's index key exactly as every writer and + /// every reader of that keyset obtains it: a different key builds terms + /// that match nothing. A ZeroKMS keyset never needs this; use + /// [`from_key_material`](Self::from_key_material). The input is + /// [`Protected`], so the caller's copy is wiped when it drops, and the + /// index key wipes its own copy on drop. + pub fn from_key_bytes(key: Protected) -> Self { + Self(*key.risky_ref()) + } + pub fn key(&self) -> &Key { &self.0 } @@ -460,6 +473,17 @@ mod tests { ); } + #[test] + fn from_key_bytes_keeps_the_bytes_it_is_given() { + let index_key = + IndexKey::from_key_bytes(vitaminc::protected::Protected::new([9u8; 32])); + assert_eq!(index_key.key(), &[9u8; 32]); + assert!( + !format!("{index_key:?}").contains("9"), + "the key must not print" + ); + } + #[test] fn from_key_material_is_deterministic() { let a = IndexKey::from_key_material(&kat_client_key(), &kat_material()).unwrap(); diff --git a/packages/stack-kms/src/key_source.rs b/packages/stack-kms/src/key_source.rs index 844301eadb..1e007f7580 100644 --- a/packages/stack-kms/src/key_source.rs +++ b/packages/stack-kms/src/key_source.rs @@ -1,7 +1,18 @@ -//! An abstraction over the ZeroKMS data-key operations that higher-level -//! encryption layers (e.g. `stack-encrypt`) depend on. +//! An abstraction over the data-key operations of a **key backend** that +//! higher-level encryption layers (e.g. `stack-encrypt`) depend on. //! -//! [`StackKms`](crate::StackKms) is the production implementation. Downstream +//! A key backend is ZeroKMS or one key vendor backend (AWS KMS, Azure Key +//! Vault, HashiCorp Vault Transit, Google Cloud KMS, or a static key). The +//! vendor-neutral part of each request is its descriptor, its iv and tag, +//! and its keyset id. The key context, the decryption policy and the +//! [`UnverifiedContext`] are ZeroKMS-only extensions: a key vendor backend +//! refuses them with [`Error::ZeroKmsOnly`] (see +//! [`refuse_zerokms_extensions`](crate::refuse_zerokms_extensions)) and +//! never ignores them. A key vendor backend reports its own failures as +//! [`Error::Backend`], and says how long its index key may be kept with +//! [`IndexKeySource::load_index_key_with_lifetime`]. +//! +//! [`StackKms`](crate::StackKms) is the ZeroKMS implementation. Downstream //! crates take a `K: DataKeySource` rather than a concrete client so their //! encrypt/decrypt logic can be unit-tested against a deterministic in-memory //! fake (see [`FakeDataKeySource`], enabled by the `test-support` feature) @@ -13,6 +24,7 @@ use std::future::Future; use uuid::Uuid; use zerokms_protocol::{IdentifiedBy, UnverifiedContext}; +use crate::backend::{KeyBackend, KeyMode}; use crate::errors::Error; use crate::key::{DataKey, DataKeyWithTag, IndexKey}; use crate::maybe_send::MaybeSend; @@ -46,6 +58,60 @@ pub trait DataKeySource { keyset_id: Option, unverified_context: Option<&UnverifiedContext>, ) -> impl Future, Error>> + MaybeSend; + + /// The key backend this source is. + /// + /// A cipher reads it before any key call. It refuses a sealed leaf that + /// names another key backend, as it refuses a leaf from a foreign + /// keyset, so a leaf never reaches a backend that did not seal it. The + /// default is [`KeyBackend::ZeroKms`], so every existing source keeps + /// the ZeroKMS behaviour. A key vendor backend overrides it to return + /// [`KeyBackend::Vendor`]. A wrapper that forwards to another source + /// must forward this method too. + fn key_backend(&self) -> KeyBackend { + KeyBackend::ZeroKms + } + + /// The key backend that holds `keyset_id`. + /// + /// A source that routes between ZeroKMS and one key vendor backend + /// answers per keyset. The default is [`key_backend`](Self::key_backend), + /// which is correct for a source that is one backend. The cipher reads + /// it before any key call: it refuses a leaf that names another backend, + /// and it chooses the leaf format (v1 or v2) from it when it seals. + fn keyset_backend(&self, keyset_id: Uuid) -> KeyBackend { + let _ = keyset_id; + self.key_backend() + } + + /// The key mode of `keyset_id`, when a key vendor backend holds it. + /// + /// `None` for a ZeroKMS keyset, which has no key mode, and for a keyset + /// this source does not know. The cipher writes the mode into the v2 + /// leaf it seals, and refuses to seal under a key vendor backend that + /// names no mode. A wrapper that forwards to another source must + /// forward this method too. + fn key_mode(&self, keyset_id: Uuid) -> Option { + let _ = keyset_id; + None + } + + /// Load the index key of `keyset_id` for one settle only. + /// + /// The cipher calls this inside a settle for a keyset that + /// [`IndexKeySource::select_keyset`] reported as + /// [`SelectedKeyset::PerSettle`], and drops the key when the settle + /// ends. A source that never reports `PerSettle` never receives this + /// call; the default refuses it with [`Error::Unexpected`]. + fn load_settle_index_key( + &self, + keyset_id: Uuid, + ) -> impl Future> + MaybeSend { + let _ = keyset_id; + std::future::ready(Err(Error::Unexpected( + "this key source does not load an index key for one settle".into(), + ))) + } } /// The slice of ZeroKMS functionality required to *index* encrypted data: @@ -70,6 +136,123 @@ pub trait IndexKeySource { &self, keyset_id: Option, ) -> impl Future> + MaybeSend; + + /// Load the index key as [`load_index_key`](Self::load_index_key) does, + /// and say how long the caller may keep it ([`KeyLifetime`]). + /// + /// The cipher calls this method, not `load_index_key`, and its + /// per-keyset cache obeys the lifetime: the cipher enforces key + /// lifetime, so each backend does not have to. The default + /// implementation returns [`KeyLifetime::UntilEvicted`], which is the + /// ZeroKMS behaviour. A key vendor backend overrides it to return + /// [`KeyLifetime::Settle`] (wrapped data keys) or + /// [`KeyLifetime::Ttl`] (derived data keys). + fn load_index_key_with_lifetime( + &self, + keyset_id: Option, + ) -> impl Future> + MaybeSend { + let load = self.load_index_key(keyset_id); + async move { + let (keyset_id, key) = load.await?; + Ok(LoadedIndexKey { + keyset_id, + key, + lifetime: KeyLifetime::UntilEvicted, + }) + } + } + + /// Resolve a keyset for a cipher that selects it, and say how the + /// cipher gets its index key. + /// + /// The default loads the index key now + /// ([`load_index_key_with_lifetime`](Self::load_index_key_with_lifetime)) + /// and returns [`SelectedKeyset::Loaded`]. A key vendor backend returns + /// [`SelectedKeyset::PerSettle`] instead: it resolves the keyset with + /// no vendor call, and the cipher asks for the index key inside each + /// settle that needs it + /// ([`DataKeySource::load_settle_index_key`]), so no index key + /// outlives the settle that needed it. + fn select_keyset( + &self, + keyset_id: Option, + ) -> impl Future> + MaybeSend { + let load = self.load_index_key_with_lifetime(keyset_id); + async move { load.await.map(SelectedKeyset::Loaded) } + } +} + +/// How a cipher gets the index key of a keyset it selected: what +/// [`IndexKeySource::select_keyset`] returns. +#[derive(Debug)] +#[non_exhaustive] +pub enum SelectedKeyset { + /// The index key, loaded now, with how long the cipher may keep it. + Loaded(LoadedIndexKey), + /// The keyset is resolved, and its index key is loaded inside each + /// settle that needs it, through + /// [`DataKeySource::load_settle_index_key`]. The cipher holds no key + /// for the keyset between settles. + PerSettle { + /// The resolved keyset id. + keyset_id: Uuid, + }, +} + +impl SelectedKeyset { + /// The resolved keyset id. + pub fn keyset_id(&self) -> Uuid { + match self { + Self::Loaded(loaded) => loaded.keyset_id, + Self::PerSettle { keyset_id } => *keyset_id, + } + } +} + +/// How long a caller may keep an index key after its key backend returned +/// it. +/// +/// The unit of the shortest lifetime is the **settle**: one batched round +/// trip of a pending's key requests to the key backend. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] +#[non_exhaustive] +pub enum KeyLifetime { + /// Keep the key only for the settle that needed it. The cipher never + /// caches it, and loads it again for the next selection of the keyset. + /// Under wrapped data keys the vendor sees, and can refuse, each one. + Settle, + /// Keep the key for this long after it was loaded, then load it again. + /// A zero duration is the same as [`Settle`](Self::Settle). Under + /// derived data keys the vendor sees, and can refuse, each reload. + Ttl(std::time::Duration), + /// Keep the key until the cipher's bounded cache evicts it. This is the + /// ZeroKMS behaviour, and the default. + #[default] + UntilEvicted, +} + +/// An index key, the keyset it belongs to, and how long it may be kept: +/// what [`IndexKeySource::load_index_key_with_lifetime`] returns. +#[derive(Debug)] +#[non_exhaustive] +pub struct LoadedIndexKey { + /// The resolved keyset id. + pub keyset_id: Uuid, + /// The keyset's index key. + pub key: IndexKey, + /// How long the caller may keep `key`. + pub lifetime: KeyLifetime, +} + +impl LoadedIndexKey { + /// An index key for `keyset_id` that may be kept for `lifetime`. + pub fn new(keyset_id: Uuid, key: IndexKey, lifetime: KeyLifetime) -> Self { + Self { + keyset_id, + key, + lifetime, + } + } } impl DataKeySource for crate::StackKms @@ -722,6 +905,82 @@ mod tests { assert_eq!(dk.key.key(), retrieved.key()); } + /// The default lifetime is the ZeroKMS one, so a source that does not + /// say otherwise keeps today's caching. + #[tokio::test] + async fn a_source_that_names_no_lifetime_is_kept_until_evicted() { + let src = FakeDataKeySource::new(); + let (id, key) = src.load_index_key(None).await.unwrap(); + let loaded = src.load_index_key_with_lifetime(None).await.unwrap(); + assert_eq!(loaded.keyset_id, id); + assert_eq!(loaded.key.key(), key.key()); + assert_eq!(loaded.lifetime, KeyLifetime::UntilEvicted); + assert_eq!(KeyLifetime::default(), KeyLifetime::UntilEvicted); + } + + #[tokio::test] + async fn a_source_can_name_its_own_lifetime() { + struct SettleOnly; + impl IndexKeySource for SettleOnly { + async fn load_index_key( + &self, + _keyset_id: Option, + ) -> Result<(Uuid, IndexKey), Error> { + Ok(( + Uuid::nil(), + IndexKey::from_key_bytes(vitaminc::protected::Protected::new([1u8; 32])), + )) + } + + async fn load_index_key_with_lifetime( + &self, + keyset_id: Option, + ) -> Result { + let (id, key) = self.load_index_key(keyset_id).await?; + Ok(LoadedIndexKey::new(id, key, KeyLifetime::Settle)) + } + } + let loaded = SettleOnly.load_index_key_with_lifetime(None).await.unwrap(); + assert_eq!(loaded.lifetime, KeyLifetime::Settle); + assert_eq!(loaded.key.key(), &[1u8; 32]); + } + + /// The defaults keep a single-backend source as it was: it is ZeroKMS + /// for every keyset, names no mode, loads its index key at selection, + /// and refuses a per-settle load it never asked for. + #[tokio::test] + async fn the_defaults_describe_one_zerokms_backend() { + let src = FakeDataKeySource::new(); + let keyset = Uuid::from_u128(9); + assert_eq!(src.keyset_backend(keyset), KeyBackend::ZeroKms); + assert_eq!(src.key_mode(keyset), None); + match src.load_settle_index_key(keyset).await { + Err(Error::Unexpected(_)) => {} + other => panic!("expected a refusal, got {other:?}"), + } + let (id, key) = src.load_index_key(None).await.unwrap(); + match src.select_keyset(None).await.unwrap() { + SelectedKeyset::Loaded(loaded) => { + assert_eq!(loaded.keyset_id, id); + assert_eq!(loaded.key.key(), key.key()); + assert_eq!(loaded.lifetime, KeyLifetime::UntilEvicted); + } + other => panic!("expected a loaded key, got {other:?}"), + } + assert_eq!( + SelectedKeyset::PerSettle { keyset_id: keyset }.keyset_id(), + keyset + ); + } + + #[test] + fn a_retrieve_payload_carries_the_key_mode_it_is_given() { + let payload = RetrieveKeyPayload::new([0u8; 16], "d", b""); + assert_eq!(payload.key_mode, None); + let payload = payload.with_key_mode(KeyMode::Derived); + assert_eq!(payload.key_mode, Some(KeyMode::Derived)); + } + #[test] fn the_stub_is_send_and_sync() { fn assert_send_sync() {} diff --git a/packages/stack-kms/src/lib.rs b/packages/stack-kms/src/lib.rs index ee484f39e1..375a863e12 100644 --- a/packages/stack-kms/src/lib.rs +++ b/packages/stack-kms/src/lib.rs @@ -15,6 +15,14 @@ //! Encryption/decryption, keyset and client management, and config save/load //! all remain in `cipherstash-client` for now. //! +//! # Key backends +//! +//! ZeroKMS is one key backend. The `stack-kms-vendor` crate adds the key +//! vendor backends (AWS KMS, Google Cloud KMS, Azure Key Vault, HashiCorp +//! Vault Transit and a static key) on the [`DataKeySource`] and +//! [`IndexKeySource`] traits of this crate. Its crate documentation has a +//! capability matrix that compares each of them with ZeroKMS. +//! //! # Quick start //! // The quick start goes through `StackKmsBuilder`, which configures the default @@ -76,6 +84,7 @@ assert_eq!(keys.len(), 1); #![cfg_attr(test, allow(clippy::panic))] #![cfg_attr(test, allow(unused_results))] +mod backend; #[cfg(feature = "http")] mod builder; mod client; @@ -120,7 +129,8 @@ pub use maybe_send::MaybeSend; // Errors pub use errors::{ - Error, GenerateKeyError, InvalidKeyMaterialError, LoadKeysetError, RetrieveKeyError, + BackendErrorKind, Error, GenerateKeyError, InvalidKeyMaterialError, LoadKeysetError, + RetrieveKeyError, }; /// The trait every error from this crate implements to hand over its /// structured fields, with the rule for what an error may contain, and the @@ -131,12 +141,15 @@ pub use stack_auth::{diagnostic, ErrorPayload}; // Key material pub use key::{ClientKey, DataKey, DataKeyWithTag, IndexKey, V1KeySet}; +// Key backends and key modes (the v2 leaf's header bytes) +pub use backend::{KeyBackend, KeyMode, KeyVendor}; + // Key source abstractions (production = `StackKms`; tests = `FakeDataKeySource`) #[cfg(feature = "test-support")] pub use key_source::DeterministicSource; #[cfg(feature = "test-support")] pub use key_source::FakeDataKeySource; -pub use key_source::{DataKeySource, IndexKeySource}; +pub use key_source::{DataKeySource, IndexKeySource, KeyLifetime, LoadedIndexKey, SelectedKeyset}; // Key providers pub use key_provider::{ @@ -149,7 +162,10 @@ pub use secret_key::SecretKey; pub use stack_profile::ProfileStore; // Operation payloads -pub use payload::{GenerateKeyPayload, RetrieveKeyPayload}; +pub use payload::{ + refuse_zerokms_extensions, GenerateKeyPayload, RetrieveKeyPayload, ZeroKmsExtensions, + ZeroKmsFeature, +}; // Commonly needed re-exports from the protocol / crypto layers pub use recipher::key::{GenRandom, Iv}; diff --git a/packages/stack-kms/src/payload.rs b/packages/stack-kms/src/payload.rs index 0c4ada8225..2ad32cd397 100644 --- a/packages/stack-kms/src/payload.rs +++ b/packages/stack-kms/src/payload.rs @@ -1,8 +1,86 @@ use recipher::key::Iv; use std::borrow::Cow; -use zerokms_protocol::{Context, DecryptionPolicy, KeyId, RetrieveKeySpec}; +use zerokms_protocol::{Context, DecryptionPolicy, KeyId, RetrieveKeySpec, UnverifiedContext}; -/// The requirements for generating a data key from ZeroKMS. +/// An input on a data-key request that only ZeroKMS can honour. +/// +/// The vendor-neutral part of a request is its descriptor, its iv and tag, +/// and its keyset id. A key vendor backend works from these alone. The +/// inputs named here have no meaning there, so a key vendor backend refuses +/// a request that carries one ([`Error::ZeroKmsOnly`](crate::Error::ZeroKmsOnly)) +/// and never ignores it. Find them with [`ZeroKmsExtensions`], or refuse them +/// in one call with [`refuse_zerokms_extensions`]. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[non_exhaustive] +pub enum ZeroKmsFeature { + /// A non-empty key [`Context`]: tags and identity claims bound into the + /// data key. + Context, + /// A [`DecryptionPolicy`] that ZeroKMS checks on each retrieve. + DecryptionPolicy, + /// A non-empty [`UnverifiedContext`] sent with the request. + UnverifiedContext, +} + +impl ZeroKmsFeature { + /// The feature as error payloads spell it. + pub const fn as_str(self) -> &'static str { + match self { + Self::Context => "context", + Self::DecryptionPolicy => "decryption_policy", + Self::UnverifiedContext => "unverified_context", + } + } +} + +impl std::fmt::Display for ZeroKmsFeature { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(self.as_str()) + } +} + +/// A data-key payload that can carry ZeroKMS-only inputs. +/// +/// Implemented by [`GenerateKeyPayload`] and [`RetrieveKeyPayload`]. A key +/// vendor backend calls [`refuse_zerokms_extensions`] (or this trait +/// directly) before it sends anything. +pub trait ZeroKmsExtensions { + /// The first ZeroKMS-only input this payload carries, if any. An empty + /// context is not an input: it binds nothing. + fn zerokms_feature(&self) -> Option; +} + +/// Refuse a request that carries a ZeroKMS-only input, for the key vendor +/// backend `vendor`. +/// +/// Returns [`Error::ZeroKmsOnly`](crate::Error::ZeroKmsOnly) for the first +/// such input: a payload's context or decryption policy, else a non-empty +/// `unverified_context`. A key vendor backend calls this first in each +/// [`DataKeySource`](crate::DataKeySource) method, so that a lock context +/// or a policy is refused, not silently dropped. +pub fn refuse_zerokms_extensions( + vendor: &'static str, + payloads: &[P], + unverified_context: Option<&UnverifiedContext>, +) -> Result<(), crate::Error> { + let feature = payloads + .iter() + .find_map(ZeroKmsExtensions::zerokms_feature) + .or_else(|| { + unverified_context + .filter(|context| !context.0.is_empty()) + .map(|_| ZeroKmsFeature::UnverifiedContext) + }); + match feature { + Some(feature) => Err(crate::Error::ZeroKmsOnly { vendor, feature }), + None => Ok(()), + } +} + +/// The requirements for generating a data key. +/// +/// [`descriptor`](Self::descriptor) is vendor-neutral. The context and the +/// decryption policy are ZeroKMS-only extensions ([`ZeroKmsFeature`]). #[derive(Clone)] pub struct GenerateKeyPayload<'a> { pub descriptor: &'a str, @@ -24,15 +102,48 @@ impl<'a> GenerateKeyPayload<'a> { self.decryption_policy = Some(policy); self } + + /// The descriptor the data key is bound to. + pub fn descriptor(&self) -> &'a str { + self.descriptor + } +} + +impl ZeroKmsExtensions for GenerateKeyPayload<'_> { + fn zerokms_feature(&self) -> Option { + feature_of(&self.context, self.decryption_policy.as_ref()) + } +} + +/// The ZeroKMS-only input a context and a policy carry, context first. +fn feature_of( + context: &[Context], + decryption_policy: Option<&DecryptionPolicy>, +) -> Option { + if !context.is_empty() { + Some(ZeroKmsFeature::Context) + } else if decryption_policy.is_some() { + Some(ZeroKmsFeature::DecryptionPolicy) + } else { + None + } } -/// The requirements for retrieving a data key from ZeroKMS. +/// The requirements for retrieving a data key. +/// +/// The iv, the descriptor and the tag are vendor-neutral (see +/// [`iv_bytes`](Self::iv_bytes)). The context and the decryption policy are +/// ZeroKMS-only extensions ([`ZeroKmsFeature`]). pub struct RetrieveKeyPayload<'a> { pub iv: KeyId, pub descriptor: &'a str, pub tag: &'a [u8], pub context: Cow<'a, [Context]>, pub decryption_policy: Option, + /// The key mode the leaf names, when a key vendor backend sealed it + /// (a v2 leaf). `None` for a ZeroKMS leaf. A key vendor backend + /// refuses a payload whose mode is not its keyset's. + pub key_mode: Option, } impl<'a> RetrieveKeyPayload<'a> { @@ -44,9 +155,16 @@ impl<'a> RetrieveKeyPayload<'a> { tag, context: Default::default(), decryption_policy: None, + key_mode: None, } } + /// Name the key mode the leaf was sealed under (a v2 leaf). + pub fn with_key_mode(mut self, mode: crate::KeyMode) -> Self { + self.key_mode = Some(mode); + self + } + pub fn with_context(mut self, context: Cow<'a, [Context]>) -> Self { self.context = context; self @@ -56,6 +174,17 @@ impl<'a> RetrieveKeyPayload<'a> { self.decryption_policy = Some(policy); self } + + /// The iv the data key was generated with, as plain bytes. + pub fn iv_bytes(&self) -> Iv { + *self.iv.as_ref() + } +} + +impl ZeroKmsExtensions for RetrieveKeyPayload<'_> { + fn zerokms_feature(&self) -> Option { + feature_of(&self.context, self.decryption_policy.as_ref()) + } } impl<'a> From> for RetrieveKeySpec<'a> { @@ -66,6 +195,7 @@ impl<'a> From> for RetrieveKeySpec<'a> { tag, context, decryption_policy, + .. }: RetrieveKeyPayload<'a>, ) -> Self { let mut spec = Self::new(iv, tag, descriptor).with_context(context); @@ -120,6 +250,122 @@ mod tests { } } + mod zerokms_extensions { + use super::*; + use crate::Error; + use std::collections::HashMap; + + fn tag() -> Vec { + vec![Context::Tag("tenant-1".into())] + } + + #[test] + fn a_plain_payload_carries_no_zerokms_feature() { + let generate = GenerateKeyPayload::new("d", Cow::Owned(vec![])); + let retrieve = RetrieveKeyPayload::new([1u8; 16], "d", b"tag"); + assert_eq!(generate.zerokms_feature(), None); + assert_eq!(retrieve.zerokms_feature(), None); + assert_eq!(generate.descriptor(), "d"); + assert_eq!(retrieve.iv_bytes(), [1u8; 16]); + } + + #[test] + fn a_context_or_a_policy_is_a_zerokms_feature() { + let ctx = tag(); + assert_eq!( + GenerateKeyPayload::new("d", Cow::Borrowed(&ctx)).zerokms_feature(), + Some(ZeroKmsFeature::Context) + ); + assert_eq!( + GenerateKeyPayload::new("d", Cow::Owned(vec![])) + .with_decryption_policy(policy()) + .zerokms_feature(), + Some(ZeroKmsFeature::DecryptionPolicy) + ); + assert_eq!( + RetrieveKeyPayload::new([0u8; 16], "d", b"t") + .with_context(Cow::Borrowed(&ctx)) + .zerokms_feature(), + Some(ZeroKmsFeature::Context) + ); + assert_eq!( + RetrieveKeyPayload::new([0u8; 16], "d", b"t") + .with_decryption_policy(policy()) + .zerokms_feature(), + Some(ZeroKmsFeature::DecryptionPolicy) + ); + } + + fn refused(result: Result<(), Error>) -> ZeroKmsFeature { + match result { + Err(Error::ZeroKmsOnly { vendor, feature }) => { + assert_eq!(vendor, "test-vendor"); + feature + } + other => panic!("expected a ZeroKmsOnly refusal, got {other:?}"), + } + } + + #[test] + fn a_vendor_refuses_every_zerokms_feature() { + let ctx = tag(); + let plain = [GenerateKeyPayload::new("d", Cow::Owned(vec![]))]; + assert!(refuse_zerokms_extensions("test-vendor", &plain, None).is_ok()); + + let with_context = [ + GenerateKeyPayload::new("d", Cow::Owned(vec![])), + GenerateKeyPayload::new("d", Cow::Borrowed(&ctx)), + ]; + assert_eq!( + refused(refuse_zerokms_extensions( + "test-vendor", + &with_context, + None + )), + ZeroKmsFeature::Context + ); + + let with_policy = + [RetrieveKeyPayload::new([0u8; 16], "d", b"t").with_decryption_policy(policy())]; + assert_eq!( + refused(refuse_zerokms_extensions("test-vendor", &with_policy, None)), + ZeroKmsFeature::DecryptionPolicy + ); + + let mut claims = HashMap::new(); + let _ = claims.insert("sub".to_string(), None); + let unverified = UnverifiedContext(claims); + assert_eq!( + refused(refuse_zerokms_extensions( + "test-vendor", + &plain, + Some(&unverified) + )), + ZeroKmsFeature::UnverifiedContext + ); + } + + /// An empty unverified context carries nothing, so it is no input. + #[test] + fn an_empty_unverified_context_is_not_refused() { + let plain = [GenerateKeyPayload::new("d", Cow::Owned(vec![]))]; + let empty = UnverifiedContext(HashMap::new()); + assert!(refuse_zerokms_extensions("test-vendor", &plain, Some(&empty)).is_ok()); + } + + #[test] + fn every_feature_keeps_its_spelling() { + for (feature, text) in [ + (ZeroKmsFeature::Context, "context"), + (ZeroKmsFeature::DecryptionPolicy, "decryption_policy"), + (ZeroKmsFeature::UnverifiedContext, "unverified_context"), + ] { + assert_eq!(feature.as_str(), text); + assert_eq!(feature.to_string(), text); + } + } + } + #[test] fn generate_key_payload_with_decryption_policy_sets_the_policy() { let payload = GenerateKeyPayload::new("d", Cow::Owned(vec![])); diff --git a/release-plz.toml b/release-plz.toml index 382457a48c..a32d07fcf7 100644 --- a/release-plz.toml +++ b/release-plz.toml @@ -19,6 +19,17 @@ # feature), so a crates.io release of `eql-bindings` needs the matching # `stack-encrypt` published first. # +# Five more crates are publishable but STAGED: `stack-kms-vendor`, +# `stack-kms-aws`, `stack-kms-vault`, `stack-kms-gcp` and `stack-kms-azure`, the `stack-kms-vendor` version group (the key vendor seam +# and every vendor crate that implements it: a change to the seam moves every +# vendor). crates.io Trusted Publishing can only be configured on a crate that +# exists, so each crate's first version is published by hand, after the +# stack-kms and stack-encrypt it depends on are on crates.io. Until then they +# carry `release = false` and `publish = false` here; the PR that arms them +# deletes those two lines from each and adds each to release-plz.yml's +# preflight loop. Their Cargo.toml files are already publishable, so +# tests-crates.yml's `cargo publish --dry-run` packages them on every PR. +# # Every other workspace member is `publish = false` in its Cargo.toml and is # also listed below with `release = false`, so `release-plz update` does not # bump its version or write its changelog either. Each also repeats @@ -76,3 +87,38 @@ publish = false name = "stack-auth-wasm" release = false publish = false + +# Staged: see the header. +[[package]] +name = "stack-kms-vendor" +version_group = "stack-kms-vendor" +release = false +publish = false + +# Staged: see the header. +[[package]] +name = "stack-kms-aws" +version_group = "stack-kms-vendor" +release = false +publish = false + +# Staged: see the header. +[[package]] +name = "stack-kms-vault" +version_group = "stack-kms-vendor" +release = false +publish = false + +# Staged: see the header. +[[package]] +name = "stack-kms-gcp" +version_group = "stack-kms-vendor" +release = false +publish = false + +# Staged: see the header. +[[package]] +name = "stack-kms-azure" +version_group = "stack-kms-vendor" +release = false +publish = false diff --git a/scripts/__tests__/cargo-publish-opt-out.test.mjs b/scripts/__tests__/cargo-publish-opt-out.test.mjs index 6e1e465454..beadbfc73e 100644 --- a/scripts/__tests__/cargo-publish-opt-out.test.mjs +++ b/scripts/__tests__/cargo-publish-opt-out.test.mjs @@ -57,7 +57,10 @@ const WORKSPACES = [ // exactly these five: stack-auth and stack-profile, and stack-kms, // stack-encrypt and stack-encrypt-derive (which `eql-bindings`' // `stack-encrypt` feature depends on from crates.io). stack-guest-abi is - // the Go guests' internal ABI and stays unpublished. + // the Go guests' internal ABI and stays unpublished. stack-kms-vendor, + // stack-kms-aws, stack-kms-vault, stack-kms-gcp and stack-kms-azure are publishable but staged: release-plz.toml keeps them + // from releasing until their first version is published by hand (see + // release-plz-root-config.test.mjs). root: '.', publishable: new Set([ 'packages/stack-auth', @@ -65,6 +68,11 @@ const WORKSPACES = [ 'packages/stack-kms', 'packages/stack-encrypt', 'packages/stack-encrypt-derive', + 'packages/stack-kms-vendor', + 'packages/stack-kms-aws', + 'packages/stack-kms-vault', + 'packages/stack-kms-gcp', + 'packages/stack-kms-azure', ]), expects: 'packages/stack-auth', }, diff --git a/scripts/__tests__/check-auth-npm-changeset.test.mjs b/scripts/__tests__/check-auth-npm-changeset.test.mjs index 4554edd517..cb5e259f50 100644 --- a/scripts/__tests__/check-auth-npm-changeset.test.mjs +++ b/scripts/__tests__/check-auth-npm-changeset.test.mjs @@ -243,10 +243,20 @@ describe('require-auth-npm-changeset.yml', () => { const filters = workflow?.on?.pull_request?.paths ?? [] - it('diffs the crate and both binding folders', () => { + it('diffs the crates the binding compiles in and both binding folders', () => { expect(shipped).toEqual([ 'packages/stack-auth/Cargo.toml', 'packages/stack-auth/src', + 'packages/stack-kms-vendor/Cargo.toml', + 'packages/stack-kms-vendor/src', + 'packages/stack-kms-aws/Cargo.toml', + 'packages/stack-kms-aws/src', + 'packages/stack-kms-vault/Cargo.toml', + 'packages/stack-kms-vault/src', + 'packages/stack-kms-gcp/Cargo.toml', + 'packages/stack-kms-gcp/src', + 'packages/stack-kms-azure/Cargo.toml', + 'packages/stack-kms-azure/src', 'languages/typescript/packages/auth', 'languages/typescript/packages/stack-auth-wasm', ]) diff --git a/scripts/__tests__/crates-ci.test.mjs b/scripts/__tests__/crates-ci.test.mjs index 72c5046127..8937f5529d 100644 --- a/scripts/__tests__/crates-ci.test.mjs +++ b/scripts/__tests__/crates-ci.test.mjs @@ -16,7 +16,7 @@ import { readWorkflow, WORKFLOW_DIR, workflowFiles } from './lib/workflows.mjs' * EQL import, and `integrationSuiteCi.test.ts` for protect-ffi's before it. * This file is the same guard for this set, in two halves: * - * 1. Every mise task the set defines — the five crate `tasks.toml` files the + * 1. Every mise task the set defines — the ten crate `tasks.toml` files the * root `mise.toml` includes, and the root tasks — is reached from a root * workflow, or named as exempt with a reason. The scan is a smaller copy * of `eql-suite-ci.test.mjs`'s: the same `invokes` rule for "a command, @@ -259,13 +259,18 @@ const CI_EXEMPT_TASKS = new Map([ ]) describe('the scan sees every task mise sees', () => { - it('reads the five crate task files from the root mise.toml', () => { + it('reads the ten crate task files from the root mise.toml', () => { expect(INCLUDES.sort()).toEqual( [ 'packages/stack-auth/tasks.toml', 'packages/stack-encrypt/tasks.toml', 'packages/stack-guest-abi/tasks.toml', 'packages/stack-kms/tasks.toml', + 'packages/stack-kms-aws/tasks.toml', + 'packages/stack-kms-azure/tasks.toml', + 'packages/stack-kms-gcp/tasks.toml', + 'packages/stack-kms-vault/tasks.toml', + 'packages/stack-kms-vendor/tasks.toml', 'packages/stack-profile/tasks.toml', ].sort(), ) @@ -315,6 +320,11 @@ describe('the scan sees every task mise sees', () => { 'doc:stack-encrypt', 'doc:stack-guest-abi', 'doc:stack-kms', + 'doc:stack-kms-aws', + 'doc:stack-kms-azure', + 'doc:stack-kms-gcp', + 'doc:stack-kms-vault', + 'doc:stack-kms-vendor', 'doc:stack-profile', ].sort(), ) @@ -371,6 +381,8 @@ describe('every task the set defines is reached by a root workflow', () => { const SOLE_CALLER_WORKFLOWS = [ `${WORKFLOW_DIR}/crap-crates.yml`, // crap:* `${WORKFLOW_DIR}/fuzz.yml`, // fuzz:* + `${WORKFLOW_DIR}/integration-azure-key-vault.yml`, // test:live:stack-kms-azure + `${WORKFLOW_DIR}/integration-gcp-kms.yml`, // test:live:stack-kms-gcp `${WORKFLOW_DIR}/miri.yml`, // miri:stack-guest-abi `${WORKFLOW_DIR}/tests-crates.yml`, // doc, test:doc, test:integration:* `${WORKFLOW_DIR}/tests-golang.yml`, // wasm:*, go:test, go:lint diff --git a/scripts/__tests__/release-plz-root-config.test.mjs b/scripts/__tests__/release-plz-root-config.test.mjs index f0fc6997c1..7b026ea73a 100644 --- a/scripts/__tests__/release-plz-root-config.test.mjs +++ b/scripts/__tests__/release-plz-root-config.test.mjs @@ -38,6 +38,19 @@ const GROUPS = { 'stack-encrypt-derive': 'stack-encrypt', } const PUBLISHED = Object.keys(GROUPS).sort() +// Publishable crates whose first version is not on crates.io yet. crates.io +// Trusted Publishing needs an existing crate, so the first version is +// published by hand; until then release-plz.toml names their version group +// but keeps `release = false` and `publish = false`, so `release-crates` does +// not fail trying. Arming one moves it from here to GROUPS. +const STAGED = { + 'stack-kms-vendor': 'stack-kms-vendor', + 'stack-kms-aws': 'stack-kms-vendor', + 'stack-kms-vault': 'stack-kms-vendor', + 'stack-kms-gcp': 'stack-kms-vendor', + 'stack-kms-azure': 'stack-kms-vendor', +} +const PUBLISHABLE = [...PUBLISHED, ...Object.keys(STAGED)].sort() /** The root workspace's members, as `{ path, name, publish }`. */ function rootMembers() { @@ -93,7 +106,7 @@ describe('root release-plz.toml', () => { .filter((member) => member.publish) .map((member) => member.name) .sort() - expect(publishable).toEqual(PUBLISHED) + expect(publishable).toEqual(PUBLISHABLE) const released = [...packages] .filter(([, config]) => config.release) @@ -106,8 +119,20 @@ describe('root release-plz.toml', () => { } }) + it('stages each crate not yet on crates.io in its version group', () => { + for (const [name, group] of Object.entries(STAGED)) { + expect(packages.get(name), name).toEqual({ + versionGroup: group, + release: false, + publish: false, + }) + } + }) + it('opts every unpublished member out of publishing here too', () => { - const unpublished = members.filter((member) => !member.publish) + const unpublished = members.filter( + (member) => !member.publish && !(member.name in STAGED), + ) expect(unpublished.length).toBeGreaterThan(0) for (const { name } of unpublished) { expect(packages.get(name)?.publish, name).toBe(false) @@ -123,7 +148,7 @@ describe('root release-plz.toml', () => { // behind from building and releasing. const versions = new Map() for (const { name, version } of members) { - const group = GROUPS[name] + const group = GROUPS[name] ?? STAGED[name] if (!group) continue expect(version, name).toBeTruthy() versions.set(group, [ diff --git a/scripts/__tests__/workflow-dispatch-job-conditions.test.mjs b/scripts/__tests__/workflow-dispatch-job-conditions.test.mjs index 3b02b86e77..f26af0e9b1 100644 --- a/scripts/__tests__/workflow-dispatch-job-conditions.test.mjs +++ b/scripts/__tests__/workflow-dispatch-job-conditions.test.mjs @@ -68,6 +68,11 @@ const EXPECTED_DISPATCHABLE = [ // The same dry run for the @cipherstash/auth line. '.github/workflows/auth-preflight.yml', '.github/workflows/integration-protect-ffi.yml', + // Opt-in by the STACK_KMS_GCP_LIVE_KEY repository variable; dispatch runs it + // against a branch the path filter does not select. + '.github/workflows/integration-gcp-kms.yml', + // Opt-in by the STACK_KMS_AZURE_LIVE_KEY repository variable, as above. + '.github/workflows/integration-azure-key-vault.yml', // Path-filtered to the release machinery, so dispatch is how it gets run // against a branch that changed something the filter does not name. '.github/workflows/lint-release.yml', @@ -120,7 +125,11 @@ const EXPECTED_FORK_GUARDED_JOBS = [ '.github/workflows/claude-review.yml / review', // The routine token is a repository secret, which forks are not given. '.github/workflows/code-review.yml / request', + // A repository secret (the Azure service principal's client secret). + '.github/workflows/integration-azure-key-vault.yml / live', '.github/workflows/integration-drizzle.yml / integration', + // A repository secret (the Cloud KMS service account key). + '.github/workflows/integration-gcp-kms.yml / live', '.github/workflows/integration-prisma-next.yml / integration', '.github/workflows/integration-protect-ffi.yml / integration', '.github/workflows/integration-supabase.yml / integration', @@ -218,7 +227,15 @@ const PERMISSIVE_NEEDS = { * switched on by `CLAUDE_REVIEW_ENABLED`; left unset it would read as skipped * on every event, and the fork guard would be credited with the skip. */ -const PERMISSIVE_VARS = { CLAUDE_REVIEW_ENABLED: 'true' } +const PERMISSIVE_VARS = { + CLAUDE_REVIEW_ENABLED: 'true', + // integration-gcp-kms.yml's opt-in: the crypto key name of the live test. + STACK_KMS_GCP_LIVE_KEY: + 'projects/p/locations/global/keyRings/r/cryptoKeys/stack-kms-gcp-live', + // integration-azure-key-vault.yml's opt-in: the key identifier of the live test. + STACK_KMS_AZURE_LIVE_KEY: + 'https://stack-kms-live.vault.azure.net/keys/stack-kms-azure-live', +} /** * `github.event` carries the webhook payload of the triggering event, so on a diff --git a/skills/stash-cli/SKILL.md b/skills/stash-cli/SKILL.md index f45c156bf9..b7fe1bbcca 100644 --- a/skills/stash-cli/SKILL.md +++ b/skills/stash-cli/SKILL.md @@ -810,6 +810,79 @@ Things to know: Flags: `--name `, `--write [path]`, `--json`. +### Key vendor backends + +**ZeroKMS users never need these commands.** Skip this section unless the +project keeps a keyset in a key vendor backend instead of ZeroKMS: AWS KMS, +Google Cloud KMS, Azure Key Vault, HashiCorp Vault Transit, or the static key +backend. Today only the Rust and Go SDKs load such keysets. + +#### `keyset create` + +```bash +stash keyset create --backend aws --vendor-key arn:aws:kms:us-east-1:111122223333:key/ --mode wrapped --name users +stash keyset create --backend vault --vendor-key users --mode derived --out keyset.env +stash keyset create --backend static --vendor-key dev-key --mode derived --out .env.keyset.local +``` + +Generates a keyset id and a key, wraps the key under `--vendor-key` (one vendor +call), and prints a **keyset record** as a dotenv line: + +```dotenv +CS_KEYSET_RECORD_USERS=stack-keyset:… +``` + +- `--backend `, `--vendor-key ` and + `--mode ` are required. `--name ` labels the keyset and + names the variable (`CS_KEYSET_RECORD_`, else `CS_KEYSET_RECORD`). +- **The record is not secret.** Only the vendor key can unwrap its key. Keep it + in config or an env var. **No SDK reads `CS_KEYSET_RECORD*` by itself**: the + app passes the value to `encrypt.WithKeysetRecords` (Go) or + `KeysetRecord::parse` (Rust). +- **Credentials come from the vendor's own environment chain, never from a + flag** — the same chain the Rust and Go SDKs use. AWS: `AWS_REGION` plus + static keys, web identity, ECS or IMDS (no `~/.aws` profiles; export them + first with `aws configure export-credentials --format env`). Google Cloud: + `GOOGLE_OAUTH_ACCESS_TOKEN` (for example `gcloud auth print-access-token`), + else the metadata server; service account key files are **not** read. Azure: + `AZURE_KEY_VAULT_ACCESS_TOKEN`, client secret, workload identity or managed + identity. Vault: `VAULT_ADDR` with `VAULT_TOKEN`, AppRole or Kubernetes auth. + Do not read or print these values; ask the user to set them. +- **Static backend: derived mode only.** `--key-env ` names the variable + that holds the 32-byte key (base64 or hex) — the name, never the key. Without + it a new key is generated and printed as `CS_STATIC_KEY`, which **is a + secret and the only copy**: write it with `--out` and move it to a secret + store. Never commit it. +- `--out ` writes the block with mode 0600 and refuses an existing file + unless `--force`. Stdout carries only the block (or the `--json` event). + With `--json`, `--out` gives `{ status: "written", … }`, which never carries + the static key; failures are `{ status: "error", code, message, hint? }`. + +#### `keyset rewrap` + +```bash +stash keyset rewrap --in keyset.env --out keyset.env --force +stash keyset rewrap stack-keyset:… --vendor-key arn:aws:kms:us-east-1:111122223333:key/ +stash keyset rewrap --in .env.keyset.local --key-env CS_STATIC_KEY --vendor-key dev-key-2 --new-key-env CS_STATIC_KEY_2 +``` + +Unwraps the record's key and wraps it again (two vendor calls). With no +`--vendor-key` it uses the record's own key, which moves the record to that +key's current version after a rotation; with `--vendor-key` it moves to another +key of the same backend. The keyset id, name and mode do not change, and +**stored data is not re-keyed**, so stored values stay readable. The rule +depends on the mode. A `derived` record may move to any key of its backend. A +`wrapped` record may move only to a new version of its own vendor key, because +each stored value keeps its data key wrapped under the key that sealed it: a +`--vendor-key` that names another key is refused (`rewrap_refused`) before any +vendor call. To move `wrapped` data to another key, create a new keyset under +it and re-encrypt the data. Replace the old record in every app afterwards. The record is the argument or `--in ` (one +record per file). A static record needs `--key-env` for its current key, and +`--new-key-env` (or a generated key) when `--vendor-key` names another key. + +Flags: `--in `, `--vendor-key `, `--key-env `, +`--new-key-env `, `--out `, `--force`, `--json`. + ## Programmatic API ```typescript diff --git a/skills/stash-zerokms/SKILL.md b/skills/stash-zerokms/SKILL.md index dfee7d5f6d..24496d3790 100644 --- a/skills/stash-zerokms/SKILL.md +++ b/skills/stash-zerokms/SKILL.md @@ -215,6 +215,20 @@ const client = await Encryption({ why CLI operations (`stash encrypt backfill`, dev-time tooling) work without any keyset configuration. +## Key vendor backends (keysets outside ZeroKMS) + +A keyset can also live in a **key vendor backend** instead of ZeroKMS: AWS +KMS, Google Cloud KMS, Azure Key Vault, HashiCorp Vault Transit, or a static +key. The customer then holds a **keyset record** (a `stack-keyset:…` string), +and a key in their own KMS protects it. Clients, grants, lock context and +decryption policies are ZeroKMS features: a vendor keyset has none of them, and +it refuses a lock context with an error. Only the Rust crates and the Go SDK +support key vendor backends today. `@cipherstash/stack` (TypeScript) does not +support them yet, so a TypeScript app keeps its keysets in ZeroKMS. `stash +keyset create` makes a keyset record and `stash keyset rewrap` moves one to +another vendor key (see `stash-cli`: a `wrapped`-mode record moves only to a +new version of its own key). + ## Two gates, two very different failures Keyset access and lock context are **independent gates**, and their failures