diff --git a/docs/plans/2026-11-05-key-vendor-backends-decisions.md b/docs/plans/2026-11-05-key-vendor-backends-decisions.md new file mode 100644 index 0000000000..c4151ebda5 --- /dev/null +++ b/docs/plans/2026-11-05-key-vendor-backends-decisions.md @@ -0,0 +1,236 @@ +# Key vendor backends: decision log (CIP-5203) + +The design is in ADR-0009, ADR-0010 and ADR-0011 +(`packages/stack-encrypt/docs/adr/`) and in the CIP-5203 sub-issues. This +file records the decisions made **during implementation without asking**, so +that they can be revised later. Each entry names the ticket, the decision, +the alternatives and the reason. + +## Stack structure + +- **Decision:** the work is one stack of pull requests on `main`. The design + PR (#1521, ADRs and `CONTEXT.md`) is the bottom layer, rebased onto the + latest `origin/main`, so every code layer can refer to the ADRs. + **Alternative:** keep #1521 outside the stack. **Reason:** the code layers + cite the ADRs and glossary terms. +- **Decision:** the layer order is CIP-5304 (trait), CIP-5305 (v2 leaf), + CIP-5306 (keyset records, modes, static), CIP-5307 (AWS), CIP-5310 (Vault), + CIP-5308 (GCP), CIP-5309 (Azure), CIP-5311 (Go), CIP-5312 (`stash keyset`), + CIP-5313 (docs). **Reason:** AWS and Vault have local emulators (LocalStack, + `vault server -dev`), so they prove the vendor transport before the two + vendors that have only recorded fixtures. + +## CIP-5304: Make DataKeySource and IndexKeySource vendor-neutral + +- **Decision:** the trait signatures of `DataKeySource` and `IndexKeySource` + do not change. The payload structs keep their ZeroKMS fields (`context`, + `decryption_policy`, the `KeyId` iv) and the `UnverifiedContext` + argument stays. New: `ZeroKmsFeature`, the `ZeroKmsExtensions` trait on + both payloads, `refuse_zerokms_extensions(vendor, payloads, + unverified_context)`, and neutral accessors (`descriptor()`, + `iv_bytes()`). A key vendor backend calls the refusal first in each + method and gets `Error::ZeroKmsOnly { vendor, feature }`. + **Alternative:** remove the ZeroKMS types from the payloads and the + trait methods. **Reason:** that breaks 22 implementations (stack-encrypt + tests, the Go guest tests, the EQL encryption tests) for no gain in + safety. A backend can ignore a public field either way; the refusal + helper is what makes "refuse, never ignore" easy to do. The ticket text + says "remove"; this is the weaker reading, chosen with the orchestrator's + guidance. +- **Decision:** an empty key context and an empty `UnverifiedContext` are + not ZeroKMS features, and a key vendor backend accepts them. + **Alternative:** refuse `Some(empty)`. **Reason:** they bind nothing. + stack-encrypt always sends an empty context and `None`. +- **Decision:** the key lifetime comes from a new provided method, + `IndexKeySource::load_index_key_with_lifetime`, which returns + `LoadedIndexKey { keyset_id, key, lifetime }`. The default + implementation calls `load_index_key` and returns + `KeyLifetime::UntilEvicted`. The cipher calls only the new method. + **Alternative:** change the return type of `load_index_key`. + **Reason:** additive; every existing source keeps the ZeroKMS behaviour + with no edit. A wrapper that forwards only `load_index_key` reports + `UntilEvicted`, which is correct for every wrapper in the tree today. +- **Decision:** `KeyLifetime` is `Settle`, `Ttl(Duration)` and + `UntilEvicted` (the default), `#[non_exhaustive]`. `Ttl(ZERO)` is the + same as `Settle`. A TTL past the range of `Instant` is `UntilEvicted`. A + lifetime the cipher does not know is never cached. **Reason:** the short + answer is the safe one. +- **Decision:** the cipher enforces the lifetime in `KeysetCache` + (`KeysetState.retention`). A `Settle` key is never inserted: each + `StackCipher::keyset` call loads it again, and the `KeysetCipher` handle + (and the pendings built from it) holds the only reference. A TTL key is + evicted on the first lookup at or after its end, and every insert sweeps + out all expired entries. Expiry raises the watermark as LRU eviction + does. **Alternative:** load the index key inside each settle. + **Reason:** term derivation is synchronous and reads `KeysetCipher::prf` + while the target is built, before the settle. Moving the load into the + settle changes the public `Term`/`Pending` shape. So in this layer "one + settle" means "one keyset selection": a caller that keeps a handle keeps + the key. The docs on `StackCipher::keyset` say so. +- **Decision:** `StackCipherBuilder::init` refuses a default keyset whose + lifetime is not `UntilEvicted`, with the new + `stack_encrypt::Error::DefaultKeysetLifetime { keyset }` + (`stack_encrypt::default_keyset_lifetime`). **Alternative:** keep the + default anyway, or make `default_keyset()` async. **Reason:** + `default_keyset()` is synchronous and holds the default for the life of + the cipher, so the cipher cannot obey a shorter lifetime. Refusing is + "refuse, never ignore". Changing `default_keyset()` is a public API break. +- **Decision:** `stack_kms::Error::Backend { vendor: &'static str, kind: + BackendErrorKind, source }` has one code, `stack_kms::backend`, and the + payload fields `vendor` and `kind`. `BackendErrorKind` is `NotFound`, + `AccessDenied`, `Throttled`, `Unavailable`, `InvalidCiphertext`, + `#[non_exhaustive]`. The message is only the vendor and the kind. + **Alternative:** one code per kind; a vendor enum. **Reason:** the miette + derive gives one code per variant, and the kind is in the payload. A + vendor enum would fix the vendor list before CIP-5305 numbers the v2 + leaf's backend byte. +- **Decision:** `Error::Backend` and `Error::ZeroKmsOnly` are new variants of + the exhaustive `stack_kms::Error`. That is a breaking change for a caller + that matches it exhaustively (the Go guest's `status_for_kms` did). The + commit is marked breaking (`!`) so release-plz bumps stack-kms 0.1 to + 0.2. **Alternative:** make `Error` `#[non_exhaustive]` now. **Reason:** + the guest relies on exhaustiveness on purpose (see `status.rs`). +- **Decision:** the Go guest maps `Backend` kinds: `NotFound` to + `STATUS_KMS_NOT_FOUND`, `AccessDenied` to `STATUS_KMS_FORBIDDEN`, + `Throttled` and `Unavailable` to `STATUS_KMS_TRANSPORT`, + `InvalidCiphertext` to `STATUS_AUTH`; `ZeroKmsOnly` to `STATUS_ENCODING`. + **Reason:** the nearest existing status for each. CIP-5311 can revisit. +- **Decision:** `IndexKey::from_key_bytes(Protected<[u8; 32]>)` is the + public constructor from raw key material. `From` stays + `test-support` only. **Reason:** `Protected` wipes the caller's copy on + drop and does not print. +- **Decision:** `is_keyset_not_found` also matches `Error::Backend { kind: + NotFound }`, so a key vendor backend's "no such keyset" forgets a name + binding as ZeroKMS's does. +- **Decision:** `Descriptor::MAX_LEN` is the literal `512`, with a test that + it equals `stack_kms::MAX_DESCRIPTOR_LEN`. The `DescriptorTooLong` message + and help say "key descriptor" and "a key backend", not "ZeroKMS". + **Reason:** the limit stays the same for every backend; only the source + of the constant moves. +- **Decision:** `CONTEXT.md` gets a glossary entry for **key lifetime**. +- **Deferred to CIP-5306:** the rule "a cipher holds ZeroKMS and at most one + key vendor backend, a second vendor is a typed error". The trait layer + has no notion of vendor yet (a cipher holds one `K`), so there is nothing + to enforce until the builder takes a key vendor backend beside ZeroKMS. +- **Deferred to CIP-5306:** how a key vendor backend's default keyset is + chosen, given that `init` refuses a default with a short lifetime. +- **Deferred (no ticket yet):** binding a `Settle` index key to exactly one + settle inside a held `KeysetCipher` handle (see above). +- **Note:** `stack-encrypt::ui derive_diagnostics` (trybuild) fails on this + machine on the base commit too: the snapshots print relative paths and + the local run prints `/…` paths. It is not caused by this change. + +## Orchestrator review of CIP-5304 + +- **Decision:** accept that, in CIP-5304, a `Settle` key lifetime lasts as + long as one keyset selection (one `KeysetCipher` handle), not one settle. + **CIP-5306 must close this gap** before it ships wrapped data keys: for a + keyset whose index key has the `Settle` lifetime, the term PRF runs inside + the settle (the deferred-backend path that `sem/mod.rs` already describes), + and the handle does not hold the index key. **Alternative:** send CIP-5304 + back. **Reason:** the gap matters only once a backend returns `Settle`, and + no backend does until CIP-5306. ADR-0009's "no key outlives the settle that + needed it" is the target, and it is unchanged. +- **Decision:** accept that the trait signatures and payload fields keep their + ZeroKMS types, and that a vendor backend refuses them through + `refuse_zerokms_extensions`. **Alternative:** remove the types now, which + breaks 22 implementations. **Reason:** the refusal is what ADR-0009 needs, + and the change is additive. Revisit when TypeScript (CIP-5314) moves onto + stack-encrypt. +- **Note:** `stack-encrypt::ui derive_diagnostics` (trybuild) fails on `main` + on a developer machine whose paths print as ``. It is not caused by + this stack. CI is the judge. + +## CIP-5305: v2 sealed leaf with key backend and key mode + +- **Decision:** the backend and mode are two `stack-kms` enums, `KeyVendor` + and `KeyMode`, with stable bytes. `KeyVendor`: `Aws = 1`, `Gcp = 2`, + `Azure = 3`, `Vault = 4`, `Static = 5`. `KeyMode`: `Wrapped = 1`, + `Derived = 2`. The value `0` is invalid for both, so a zeroed header + never decodes. A value is never reused. `KeyBackend` is `ZeroKms` or + `Vendor(KeyVendor)`, so ZeroKMS has no backend byte and a v2 leaf cannot + name it. **Alternative:** one enum with a ZeroKMS byte; the enums in + `stack-encrypt`. **Reason:** the trait method (below) lives in + `stack-kms` and must return the type. Leaving ZeroKMS out of the byte + enum makes "ZeroKMS is never v2" a type fact. +- **Decision:** `KeyVendor` is `#[non_exhaustive]`; `KeyMode` and + `KeyBackend` are exhaustive. **Reason:** a new vendor is likely; ADR-0009 + rejects a third mode, and `KeyBackend` has exactly two shapes. +- **Decision:** `KeyVendor::name()` gives stable names (`aws-kms`, + `gcp-kms`, `azure-key-vault`, `vault-transit`, `static`), and a key + vendor backend should use it as the `vendor` of `Error::Backend` and + `Error::ZeroKmsOnly`. `KeyBackend::ZeroKms` is `zerokms`. +- **Decision:** the seam is a provided method, + `DataKeySource::key_backend(&self) -> KeyBackend`, default `ZeroKms`. + **Alternative:** a method on `IndexKeySource`, or a per-keyset answer. + **Reason:** additive; every existing source keeps ZeroKMS with no edit, + and ADR-0010 routes a leaf by the backend the cipher holds, not by the + keyset. A wrapper must forward it (the doc says so); no wrapper in the + tree needs to today. +- **Decision:** `SealedValue` keeps one type and gains a private + `vendor: Option` field. `None` is v1. New API: + `VENDOR_FORMAT_VERSION = 2`, `format_version()`, `key_backend()`, + `key_mode()`, `from_vendor_parts(vendor, mode, keyset_id, iv, tag, ct)`. + `FORMAT_VERSION` stays `1` and still names v1. `into_parts` is + unchanged and drops the v2 header (documented: persist v2 with + `to_bytes`). **Alternative:** a separate v2 type, or an enum leaf. + **Reason:** every tree, visitor and binding carries `SealedValue`; one + type keeps v2 opaque to all of them. +- **Decision:** the serde form of a v1 leaf does not change. A v2 leaf + adds a last field `vendor: [backend, mode]`, skipped when `None` and + defaulted on read. **Reason:** keeps stored v1 serde data readable; no + caller in the tree uses serde for leaves. +- **Decision:** the leaf AAD's second PAE piece is the envelope header: + `[0x01]` for v1 (byte for byte the old AAD; the existing pin is + unchanged) and `[0x02, backend, mode]` for v2. **Alternative:** two + extra PAE pieces for v2. **Reason:** PAE frames each piece with its + length, so a 3-byte header can never collide with a 1-byte one, and the + v1 derivation stays identical. A new pin covers the v2 AAD. +- **Decision:** the routing refusal is + `stack_encrypt::Error::ForeignBackend { expected, found }` + (`stack_encrypt::foreign_backend`), raised in `dispatch` before any + generate or retrieve call, for any retrieve whose leaf names another + backend than `cipher.kms().key_backend()`. It is symmetric: a vendor + cipher refuses a v1 leaf too. Each retrieve `Request` now carries the + leaf's backend; the new public `Request::retrieve_leaf_key(&leaf, + context)` sets it, and the old `retrieve_data_key` names ZeroKMS. + **Alternative:** check in `check_scope` at pending construction. + **Reason:** `check_scope` has no cipher; `dispatch` is the one gate + every request passes before I/O, and the refusal is still before any + key call. A whole tree is refused if one leaf is foreign. +- **Decision:** a generate request under a key vendor backend is refused + with `stack_encrypt::Error::SealUnsupported { backend }` + (`stack_encrypt::seal_unsupported`), before any key call. **Alternative:** + write v1 (it would name ZeroKMS, wrongly) or wire v2 sealing now. + **Reason:** the v2 seal needs the keyset's key mode, which is CIP-5306's + keyset record. The private `seal_leaf_as(Some(header), ..)` already + writes v2; the tests use it. +- **Decision:** `LeafBytesError` gains `UnknownBackend(u8)` + (`stack_encrypt::leaf_backend`) and `UnknownMode(u8)` + (`stack_encrypt::leaf_mode`). A v1-only reader (an older build) refuses + v2 with `UnknownVersion(2)`, as ADR-0010 says. +- **Decision:** the Go guest maps `ForeignBackend` to `STATUS_ENCODING` + (the caller's input; no keyset option opens it, so not + `STATUS_FOREIGN_KEYSET`). `SealUnsupported` falls to `STATUS_INTERNAL` + (the guest holds ZeroKMS, so it cannot occur). CIP-5311 can revisit. +- **Decision:** a v1 known-answer test (`a_pinned_v1_leaf_still_opens` in + `tests/frozen_bytes.rs`) pins the bytes of a real v1 leaf sealed under + `DeterministicSource([7; 32])` and opens it. The existing layout pin + (`sealed_value_layout_is_pinned`) is unchanged. +- **Decision:** `eql-bindings` needs no code change. Its `c` is still + `stack-encrypt:1:` + base64 of `SealedValue::to_bytes`; the `1` there is + the EQL producer marker, not the leaf version. A new test + (`a_v2_leaf_passes_through_the_ciphertext_as_opaque_bytes`) proves a v2 + leaf round-trips through `c`. +- **Decision:** fuzzing reuses the `sealed_value_decode` target (it already + checks "no panic" and "lossless" for every input) and adds two v2 seeds + (`valid-leaf-v2-aws-wrapped`, `valid-leaf-v2-static-derived-empty-tag`). + The seed replay and a 30 s run passed. +- **Deferred to CIP-5306:** the v2 seal path (choose the leaf header from + the keyset's backend and key mode), and passing the leaf's key mode to + the key source on retrieve (`RetrieveKeyPayload` does not carry it yet; + the cipher has it in `SealedValue::key_mode`). Remove or keep + `Error::SealUnsupported` then. +- **Deferred to CIP-5306:** a cipher that holds ZeroKMS and one key vendor + backend routes each leaf to the right one. Today a cipher holds one `K`, + so `ForeignBackend` compares with that one backend. diff --git a/languages/golang/encrypt/guest/src/status.rs b/languages/golang/encrypt/guest/src/status.rs index 7cfbdabd07..b2ddab6521 100644 --- a/languages/golang/encrypt/guest/src/status.rs +++ b/languages/golang/encrypt/guest/src/status.rs @@ -155,6 +155,11 @@ pub fn status_for_error(error: &stack_encrypt::Error) -> u32 { // children asked for as a scalar, or an altered entry. The client // cannot tell which, so neither STATUS_AUTH nor a claim of either. stack_encrypt::Error::JsonNotALeaf => STATUS_ENCODING, + // A leaf that names another key backend than this guest's (a v2 + // leaf of a key vendor backend, while this guest holds ZeroKMS) is + // refused before any key call. It is the caller's input, as a leaf + // this guest cannot decode is: no retry with other options opens it. + stack_encrypt::Error::ForeignBackend { .. } => STATUS_ENCODING, _ => STATUS_INTERNAL, } } @@ -240,6 +245,27 @@ fn status_for_kms(error: &stack_kms::Error) -> u32 { STATUS_KMS_TRANSPORT } stack_kms::Error::Unexpected(_) => STATUS_KMS_OTHER, + stack_kms::Error::Backend { kind, .. } => status_for_backend(*kind), + // A request for a lock context, a decryption policy or an + // unverified context against a key vendor backend: the caller's + // input, refused before any key call. + stack_kms::Error::ZeroKmsOnly { .. } => STATUS_ENCODING, + } +} + +/// A key vendor backend's failure as a status. `BackendErrorKind` is +/// `#[non_exhaustive]`, so a kind added upstream falls to +/// [`STATUS_KMS_OTHER`] until it is classified here. +fn status_for_backend(kind: stack_kms::BackendErrorKind) -> u32 { + use stack_kms::BackendErrorKind as Kind; + match kind { + Kind::NotFound => STATUS_KMS_NOT_FOUND, + Kind::AccessDenied => STATUS_KMS_FORBIDDEN, + Kind::Throttled | Kind::Unavailable => STATUS_KMS_TRANSPORT, + // The vendor could not unwrap the key: damaged, or wrapped by + // another key. Like an AEAD failure, the stored bytes do not open. + Kind::InvalidCiphertext => STATUS_AUTH, + _ => STATUS_KMS_OTHER, } } @@ -315,6 +341,30 @@ mod tests { assert_eq!(status_for_error(&load), STATUS_KMS_UNAUTHORIZED); } + #[test] + fn a_key_vendor_backend_failure_maps_by_its_kind() { + use stack_kms::BackendErrorKind as Kind; + let status = |kind| { + status_for_error(&stack_encrypt::Error::Kms(stack_kms::Error::Backend { + vendor: "test-vendor", + kind, + source: Box::new(std::io::Error::other("detail")), + })) + }; + assert_eq!(status(Kind::NotFound), STATUS_KMS_NOT_FOUND); + assert_eq!(status(Kind::AccessDenied), STATUS_KMS_FORBIDDEN); + assert_eq!(status(Kind::Throttled), STATUS_KMS_TRANSPORT); + assert_eq!(status(Kind::Unavailable), STATUS_KMS_TRANSPORT); + assert_eq!(status(Kind::InvalidCiphertext), STATUS_AUTH); + assert_eq!( + status_for_error(&stack_encrypt::Error::Kms(stack_kms::Error::ZeroKmsOnly { + vendor: "test-vendor", + feature: stack_kms::ZeroKmsFeature::DecryptionPolicy, + })), + STATUS_ENCODING + ); + } + #[test] fn aead_and_composition_errors_map_to_the_vitaminc_codes() { assert_eq!(status_for_error(&stack_encrypt::Error::Aead), STATUS_AUTH); @@ -433,6 +483,27 @@ mod tests { ); } + /// A leaf of another key backend is the caller's input, never a + /// foreign keyset (no keyset constraint would let it open). A seal under + /// a key vendor backend cannot happen here, so it is internal. + #[test] + fn a_foreign_backend_is_encoding_and_an_unsupported_seal_is_internal() { + use stack_encrypt::kms::{KeyBackend, KeyVendor}; + assert_eq!( + status_for_error(&stack_encrypt::Error::ForeignBackend { + expected: KeyBackend::ZeroKms, + found: KeyBackend::Vendor(KeyVendor::Aws), + }), + STATUS_ENCODING + ); + assert_eq!( + status_for_error(&stack_encrypt::Error::SealUnsupported { + backend: KeyBackend::Vendor(KeyVendor::Aws), + }), + STATUS_INTERNAL + ); + } + #[test] fn a_foreign_keyset_is_its_own_status_and_scope_bugs_are_internal() { let (a, b) = (uuid::Uuid::from_u128(1), uuid::Uuid::from_u128(2)); diff --git a/packages/eql/crates/eql-bindings/src/encryption.rs b/packages/eql/crates/eql-bindings/src/encryption.rs index 4baa22fcf6..1b0ed661ed 100644 --- a/packages/eql/crates/eql-bindings/src/encryption.rs +++ b/packages/eql/crates/eql-bindings/src/encryption.rs @@ -571,6 +571,37 @@ mod tests { ); } + /// EQL carries a leaf as its opaque `SealedValue` bytes, so a v2 leaf (a + /// key vendor backend's) passes through `c` unchanged: same prefix, same + /// payload shape, and the backend and mode read back as written. + #[test] + fn a_v2_leaf_passes_through_the_ciphertext_as_opaque_bytes() { + use stack_encrypt::kms::{KeyBackend, KeyMode, KeyVendor}; + + let leaf = SealedValue::from_vendor_parts( + KeyVendor::Aws, + KeyMode::Wrapped, + Default::default(), + [3; 16], + vec![9; 8], + vec![0xDE, 0xAD], + ) + .unwrap(); + let bytes = leaf.to_bytes(); + let ciphertext = CiphertextVisitor.sealed(leaf).unwrap(); + assert_eq!( + ciphertext.0, + format!("{CIPHERTEXT_PREFIX}{}", STANDARD.encode(&bytes)) + ); + + let CipherText::Single(read) = ciphertext.read(NativeCiphertextVisitor).unwrap() else { + panic!("a ciphertext reads back as one leaf"); + }; + assert_eq!(read.to_bytes(), bytes); + assert_eq!(read.key_backend(), KeyBackend::Vendor(KeyVendor::Aws)); + assert_eq!(read.key_mode(), Some(KeyMode::Wrapped)); + } + #[test] fn a_separator_in_a_name_is_escaped_never_a_third_part() { let odd = id("users/email", "x"); diff --git a/packages/stack-encrypt/CONTEXT.md b/packages/stack-encrypt/CONTEXT.md index ee3130de66..40beb9df7d 100644 --- a/packages/stack-encrypt/CONTEXT.md +++ b/packages/stack-encrypt/CONTEXT.md @@ -261,13 +261,72 @@ and whose ZeroKMS key requests are queued but not sent. Pendings compose (`zip`, `map`, `all`) so a whole struct or `Vec` settles in one batched call. _Avoid_: future, promise +**Settle**: +A pending's key requests going out and coming back as one batched round +trip to the key backend; composed pendings settle together. The unit a key +lifetime is measured in: under wrapped data keys, no key outlives one +settle. +_Avoid_: operation (that is what produces a target), request, flush + +**Key lifetime**: +How long the cipher may keep an index key after its key backend returned +it: one settle only, for a TTL, or until the bounded cache evicts it (the +ZeroKMS behaviour). The key backend states it (`KeyLifetime`); the cipher +enforces it. +_Avoid_: key TTL (a TTL is one of the lifetimes), cache policy + **Keyset**: -The ZeroKMS key domain a data key is minted under and an index key belongs -to — one per tenant is the common shape. A client may use any number; -`StackCipher` is scoped to the client, not to a keyset. Its **id** (a UUID) -is its identity: globally unique, carried in every sealed leaf, never -re-checked. -_Avoid_: dataset, key ring, tenant (a tenant *has* a keyset) +The key domain a data key is minted under and an index key belongs to — +one per tenant is the common shape. A keyset lives in exactly one key +backend: in ZeroKMS it is a ZeroKMS keyset; in a third-party vendor's KMS +it is a keyset record the customer creates, naming the vendor key that +protects it. A client may use any number; `StackCipher` is scoped to the +client, not to a keyset. Its **id** (a UUID) is its identity: globally +unique, carried in every sealed leaf, never re-checked. +_Avoid_: dataset, key ring, tenant (a tenant *has* a keyset), vendor key +(that is what protects a keyset, not the keyset) + +**Key backend**: +The service a keyset's data keys and index key come from: ZeroKMS, or one +key vendor backend. Every keyset lives in exactly one. +_Avoid_: KMS (ambiguous between ZeroKMS and a vendor's), key provider (that +is the source of ZeroKMS client credentials), key source + +**Key vendor backend**: +A key backend that is not ZeroKMS: AWS KMS, Azure Key Vault, HashiCorp +Vault Transit, Google Cloud KMS, or a static key. It works at less than +ZeroKMS parity, and each gap is a stated trade-off: what it cannot do it +refuses, never ignores. +_Avoid_: vendor KMS, third-party KMS, BYOK (that is ZeroKMS rooted in a +customer's KMS, not ZeroKMS replaced by it) + +**Keyset record**: +The customer-held description of a keyset in a key vendor backend: its id, +its backend, its key mode, the vendor key that protects it, and the wrapped +key material. ZeroKMS keeps its own keysets, so a ZeroKMS user never holds +or persists one. +_Avoid_: keyset config, key file + +**Wrapped data key**: +The key mode where no key outlives the settle that needed it. Each value +has its own data key, generated and wrapped by the vendor key and stored, +wrapped, with the value; the index key is wrapped in the keyset record and +unwrapped once per settle. Search terms are still built in the client. The +vendor sees, and can refuse, every settle and every value. +_Avoid_: envelope mode, DEK, no-key mode (keys exist, only never beyond one +settle) + +**Derived data key**: +The key mode where a keyset's root key is wrapped by the vendor key, and +each value's data key is derived from it locally. Opening a value costs no +vendor call once the root key is unwrapped; the vendor sees and can refuse +the keyset, not the value. +_Avoid_: local mode, cached mode + +**Root key**: +Under derived data keys, the keyset secret every data key is derived from. It is held unwrapped in the application's memory, for a +bounded time. +_Avoid_: master key, KEK (the vendor key is the one that wraps) **Keyset cipher**: `KeysetCipher`, the cipher bound to one keyset, and what every operation diff --git a/packages/stack-encrypt/docs/adr/0009-two-key-modes-for-key-vendor-backends.md b/packages/stack-encrypt/docs/adr/0009-two-key-modes-for-key-vendor-backends.md new file mode 100644 index 0000000000..eac845b12e --- /dev/null +++ b/packages/stack-encrypt/docs/adr/0009-two-key-modes-for-key-vendor-backends.md @@ -0,0 +1,67 @@ +--- +status: proposed +date: 2026-11-04 +--- + +# Two key modes for key vendor backends + +CIP-5203 makes Stack Encrypt usable with AWS KMS, Azure Key Vault, HashiCorp +Vault Transit and Google Cloud KMS as alternatives to ZeroKMS. An earlier +project tried to make KMS support fully general in the vitaminc layer. That +was too complex, so this work adds key vendor backends at the +`DataKeySource` / `IndexKeySource` seam in `stack-kms` and accepts less than +ZeroKMS parity. Each gap is a stated trade-off. + +ZeroKMS gives each value its own data key and settles a batch of N values in +one call. The vendors cannot do this. AWS KMS, Google Cloud KMS and Azure Key +Vault unwrap one key per call and have no batch operation. Only Vault Transit +batches. There is no single vendor design that is both "per-value keys" and +"few vendor calls". We therefore give each vendor backend two key modes, and +the customer chooses one per keyset: + +- **Wrapped data key.** No key outlives the settle that needed it. Each value + has its own data key, generated and wrapped by the vendor key, and stored + wrapped in the leaf. The index key is wrapped in the keyset record and + unwrapped once per settle. A settle of N values costs about 1 + N vendor + calls. The vendor sees, and can refuse, every settle and every value. +- **Derived data key.** The keyset record holds a root key wrapped by the + vendor key. The cipher unwraps it once and caches it for a TTL. Each data + key is HKDF-SHA256(root key, salt = iv, info = keyset id ‖ descriptor), and + the index key is derived from the same root key under a separate label. + Vendor calls are O(keysets), not O(values). The vendor sees, and can refuse, + the keyset unwrap, not the value. + +In both modes, search terms are built in the client, as they are today, and +Postgres holds no usable key material. + +## Considered options + +- **One mode per vendor.** Rejected. A customer who needs per-value audit and + refusal and a customer who needs throughput would each have no option on + some vendors. +- **Vendor-side PRF for search terms** (AWS `GenerateMac`, GCP MAC keys, + Vault `hmac`), so that the index key is never in the client. Rejected. The + vendor would receive the search input, or a hash of it that it can + brute-force for low-entropy values. One call per term would make a match + index cost about 20 calls per value. Azure has no MAC operation. "No key + outlives one settle" gives the same refusal property without these costs. +- **A hybrid mode** (per-value data keys and a cached index key). Rejected. + Its trade-off is not clear enough to justify a third mode in the test + matrix. +- **Vendor-side encryption of plaintext.** Rejected. It moves the trust + boundary to the vendor and has size limits (4 KiB on AWS). + +## Consequences + +- The cipher enforces key lifetime, not each backend. `load_index_key` + returns a lifetime (*this settle only*, *cache for TTL*), and the per-keyset + cache obeys it. One test then covers every backend. +- Under wrapped data keys, each separate `encryptQuery` costs one vendor + round trip. A caller who wants one round trip composes query terms into one + settle. +- The static key backend offers derived data keys only, because a static key + is always in memory. +- A vendor backend refuses ZeroKMS-only features (decryption policy, lock + context, `UnverifiedContext`) with a typed error. It never ignores them. +- A cipher holds ZeroKMS and at most one key vendor backend. v1 has no + migration between backends in either direction. diff --git a/packages/stack-encrypt/docs/adr/0010-a-v2-leaf-names-its-key-backend-and-mode.md b/packages/stack-encrypt/docs/adr/0010-a-v2-leaf-names-its-key-backend-and-mode.md new file mode 100644 index 0000000000..9892e7cd69 --- /dev/null +++ b/packages/stack-encrypt/docs/adr/0010-a-v2-leaf-names-its-key-backend-and-mode.md @@ -0,0 +1,41 @@ +--- +status: proposed +date: 2026-11-04 +--- + +# A v2 leaf names its key backend and key mode + +A v1 leaf (`SealedValue`, `cipher.rs`) is +`0x01 ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ ct`. It has no field that names a +key backend, because ZeroKMS was the only one. ADR-0009 lets one cipher hold +ZeroKMS and one key vendor backend, with two key modes. A reader must +therefore know which backend and which mode sealed a leaf before it asks for +a key. + +We keep v1 unchanged, byte for byte, and ZeroKMS writers continue to write +it. Every value sealed under ZeroKMS stays decryptable with no migration. +Leaves sealed by a key vendor backend use v2: + +``` +0x02 ‖ backend ‖ mode ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ ct +``` + +Under wrapped data keys the tag holds the wrapped data key. Under derived +data keys the tag is empty, and the iv is the derivation salt. `leaf_aad` +already binds the version and the header into the AAD, so the new bytes are +authenticated with no new mechanism. + +A v1-only reader refuses a v2 leaf with `UnknownVersion`. A cipher that does +not hold the backend a v2 leaf names refuses it before any key call, as it +refuses a foreign keyset today. EQL carries the leaf as opaque `SealedValue` +bytes, so the EQL payload shape does not change. The TypeScript FFI does not +use this leaf format (protect-ffi goes through `cipherstash-client`), so it is +not affected. + +## Considered options + +- **A v2 for every leaf, ZeroKMS included.** Rejected. It changes stored data + and every writer for no gain, and it puts existing data at risk. +- **Infer the backend from the keyset id.** Rejected. A reader would need + every keyset record before it could even route a leaf, and a wrong record + would produce a failed unwrap rather than a clear refusal. diff --git a/packages/stack-encrypt/docs/adr/0011-hand-written-vendor-clients-signed-in-the-guest.md b/packages/stack-encrypt/docs/adr/0011-hand-written-vendor-clients-signed-in-the-guest.md new file mode 100644 index 0000000000..2520d1586e --- /dev/null +++ b/packages/stack-encrypt/docs/adr/0011-hand-written-vendor-clients-signed-in-the-guest.md @@ -0,0 +1,47 @@ +--- +status: proposed +date: 2026-11-04 +--- + +# Hand-written vendor clients, signed in the guest + +The Go SDK reaches Stack Encrypt through a WASI guest. The guest builds the +ZeroKMS requests and derives the keys. The Go host is only an HTTP pipe +(`transport_send`) and a token source, and no key material crosses back to the +host (ADR-0005). Key vendor backends (ADR-0009) must keep that property. + +The official Rust SDKs (`aws-sdk-kms`, the Google Cloud and Azure crates) +depend on tokio and hyper and do not build for wasm32-wasip1. We therefore +write a small REST client for each vendor, over one transport trait of the +same shape as `ZeroKMSConnection`. Native Rust and the guest use the same +code. Each vendor is a separate crate (`stack-kms-aws` and so on), so that no +user compiles all four. + +The guest signs each vendor request itself (SigV4, the Google OAuth JWT, the +Azure AD token, the Vault token). It gets short-lived signing credentials from +the Go host through a new credential import, of the same shape as `token_get`. +The host fills the import from the vendor's own Go credential chain, so we do +not implement IMDS, the GCP metadata server or Azure managed identity in the +guest. Credentials cross the boundary. Key material does not. + +The Go module ships one guest that contains all vendor clients, not one guest +for each vendor. This decision depends on a measurement of the guest size. + +## Considered options + +- **Official SDKs on native Rust and hand-written clients in the guest.** + Rejected. There would be two code paths for each vendor, and the guest path + would be the less tested one. +- **The Go host calls the vendor's Go SDK and passes the unwrapped key to the + guest.** Rejected. It breaks the rule that key material never crosses to the + host. +- **One guest for each vendor.** Rejected for now. It multiplies the embedded + `.wasm` files, the deterministic test builds and the release workflow by up + to five. + +## Consequences + +We own the request signing code for four vendors, and its tests. Recorded +HTTP fixtures cover every vendor. LocalStack and `vault server -dev` run on +every PR. Live Google Cloud and Azure tests run in an opt-in, credentialed +workflow. diff --git a/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-aws-wrapped b/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-aws-wrapped new file mode 100644 index 0000000000..e9778634e5 Binary files /dev/null and b/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-aws-wrapped differ diff --git a/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-static-derived-empty-tag b/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-static-derived-empty-tag new file mode 100644 index 0000000000..e0d3f3010c Binary files /dev/null and b/packages/stack-encrypt/fuzz/corpus/sealed_value_decode/valid-leaf-v2-static-derived-empty-tag differ diff --git a/packages/stack-encrypt/fuzz/fuzz_targets/sealed_value_decode.rs b/packages/stack-encrypt/fuzz/fuzz_targets/sealed_value_decode.rs index a38487378f..5c8ae321d0 100644 --- a/packages/stack-encrypt/fuzz/fuzz_targets/sealed_value_decode.rs +++ b/packages/stack-encrypt/fuzz/fuzz_targets/sealed_value_decode.rs @@ -3,9 +3,10 @@ use libfuzzer_sys::fuzz_target; use stack_encrypt::SealedValue; -// Fuzz the frozen v1 leaf byte decoder. A stored ciphertext column comes back -// through `SealedValue::from_bytes` before anything is authenticated, so the -// bytes are attacker-controlled: decoding must never panic, only return `Err`. +// Fuzz the frozen leaf byte decoder, v1 (ZeroKMS) and v2 (a key vendor +// backend: `0x02 ‖ backend ‖ mode ‖ …`); the seed corpus holds both. A +// stored ciphertext column comes back through `SealedValue::from_bytes` +// before anything is authenticated, so the bytes are attacker-controlled: decoding must never panic, only return `Err`. // // The format is documented as lossless, so a second invariant is checked on // every accepted input: re-encoding gives back exactly the bytes decoded. diff --git a/packages/stack-encrypt/src/cipher.rs b/packages/stack-encrypt/src/cipher.rs index 0efa88ef09..6e73048acd 100644 --- a/packages/stack-encrypt/src/cipher.rs +++ b/packages/stack-encrypt/src/cipher.rs @@ -68,10 +68,12 @@ //! `aws-lc-rs` on native, RustCrypto on wasm32; vitaminc's own random nonce //! and versioned leaf layout). The leaf AAD is the labelled derivation //! `leaf_aad` (private): -//! `PAE("stack-encrypt/leaf", version, keyset_id, derived_aad, tag)`, with -//! [`SealedValue::FORMAT_VERSION`] — the version byte that prefixes the -//! leaf's frozen byte encoding ([`SealedValue::to_bytes`]) — and the keyset -//! id bound under the tag, so a stored leaf relabelled with a different +//! `PAE("stack-encrypt/leaf", header, keyset_id, derived_aad, tag)`, with +//! the leaf's envelope header — the [`SealedValue::FORMAT_VERSION`] byte +//! of a v1 (ZeroKMS) leaf, or `version ‖ backend ‖ mode` of a v2 leaf of a +//! key vendor backend, the bytes that prefix the leaf's frozen byte +//! encoding ([`SealedValue::to_bytes`]) — and the keyset id bound under +//! the tag, so a stored leaf relabelled with a different //! version byte fails verification instead of selecting different parsing //! rules, and one re-pointed at another keyset fails instead of asking that //! keyset for a key it never minted. The `tag` is always bound, so the @@ -97,7 +99,10 @@ use std::sync::{Arc, Mutex, PoisonError}; use std::time::Duration; use serde::{Deserialize, Serialize}; -use stack_kms::{DataKey, DataKeySource, DataKeyWithTag, IdentifiedBy, IndexKeySource}; +use stack_kms::{ + DataKey, DataKeySource, DataKeyWithTag, IdentifiedBy, IndexKeySource, KeyBackend, KeyMode, + KeyVendor, +}; #[cfg(feature = "http")] use stack_kms::{EnvKeyProvider, StackKms, StackKmsBuilder}; #[cfg(all(feature = "http", not(target_arch = "wasm32")))] @@ -111,7 +116,7 @@ use vitaminc_aead::{ use vitaminc_encrypt::{Aes256Cipher, Key as AesKey}; use vitaminc_protected::Protected; -use crate::keyset::{KeysetCache, KeysetCipher, KeysetState, Lookup, DEFAULT_NAME_TTL}; +use crate::keyset::{KeysetCache, KeysetCipher, KeysetState, Lookup, Retention, DEFAULT_NAME_TTL}; use crate::Descriptor; /// The recursive ciphertext container produced by [`StackCipher`]: vitaminc's @@ -135,7 +140,7 @@ pub type StackCipherText = Node; #[derive(Debug, thiserror::Error, miette::Diagnostic)] #[non_exhaustive] pub enum Error { - /// A ZeroKMS data-key or keyset operation failed. Transparent: the + /// A key backend's data-key or keyset operation failed. Transparent: the /// message, code and help are the [`stack_kms::Error`]'s, so a caller /// across a binding sees `stack_kms::keyset_not_found` itself. #[error(transparent)] @@ -236,21 +241,34 @@ pub enum Error { #[error("expected {expected} data keys from ZeroKMS but received {received}")] #[diagnostic(code(stack_encrypt::key_count_mismatch))] KeyCountMismatch { expected: usize, received: usize }, - /// A context rendered to a descriptor longer than ZeroKMS can bind - /// ([`Descriptor::MAX_LEN`]). Raised before any request is sent, so no - /// key is minted or retrieved for the batch. + /// A context rendered to a descriptor longer than a key backend can + /// bind ([`Descriptor::MAX_LEN`]). Raised before any request is sent, so + /// no key is minted or retrieved for the batch. #[error( - "context renders to a {len}-byte ZeroKMS descriptor; the limit is {} bytes", + "context renders to a {len}-byte key descriptor; the limit is {} bytes", Descriptor::MAX_LEN )] #[diagnostic( code(stack_encrypt::descriptor_too_long), help( - "ZeroKMS binds a descriptor of at most {} bytes. Shorten the context: fewer parts, shorter labels, or an id in place of a long name.", + "A key backend binds a descriptor of at most {} bytes. Shorten the context: fewer parts, shorter labels, or an id in place of a long name.", Descriptor::MAX_LEN ) )] DescriptorTooLong { len: usize }, + /// The key backend gave the default keyset's index key a lifetime + /// shorter than the cipher's own ([`stack_kms::KeyLifetime::Settle`] or + /// [`stack_kms::KeyLifetime::Ttl`]). The cipher holds the default keyset + /// for its whole life, so it refuses to start rather than keep a key + /// longer than its backend allows. + #[error( + "the key backend does not let the cipher keep the index key of the default keyset {keyset}" + )] + #[diagnostic( + code(stack_encrypt::default_keyset_lifetime), + help("Use a key backend whose default keyset the cipher may keep, and select keysets with a shorter key lifetime with `StackCipher::keyset`.") + )] + DefaultKeysetLifetime { keyset: Uuid }, /// Building a ZeroKMS client from the environment failed: credentials or /// client key missing or malformed. /// @@ -371,6 +389,33 @@ pub enum Error { ) )] ForeignKeyset { expected: Uuid, found: Uuid }, + /// A leaf names another key backend than the cipher's. A v1 leaf names + /// ZeroKMS; a v2 leaf names its key vendor backend + /// ([`SealedValue::key_backend`]). The cipher refuses the leaf before + /// any key call, as it refuses a [`ForeignKeyset`](Self::ForeignKeyset), + /// so the leaf never reaches a backend that did not seal it. The leaf's + /// backend byte is read before the leaf is authenticated: read this as + /// "not this cipher's leaf", never as "an untampered leaf". + #[error("leaf names key backend {found}, but this cipher's key backend is {expected}")] + #[diagnostic( + code(stack_encrypt::foreign_backend), + help("Open the value with a cipher that holds the key backend that sealed it.") + )] + ForeignBackend { + expected: KeyBackend, + found: KeyBackend, + }, + /// A data key was requested from a key vendor backend. This build seals + /// v1 leaves only, and a v1 leaf names ZeroKMS, so it cannot seal under + /// another key backend. Refused before any key call. + #[error("this build cannot seal a value under key backend {backend}")] + #[diagnostic( + code(stack_encrypt::seal_unsupported), + help( + "Seal values under ZeroKMS. Sealing under a key vendor backend is not available yet." + ) + )] + SealUnsupported { backend: KeyBackend }, /// A data key was requested through a [`StackCipher`] rather than a /// [`KeysetCipher`]: a [`Request::generate_data_key`] needs a keyset /// to mint under, and only a keyset-scoped pending has one. Always a @@ -450,10 +495,18 @@ impl crate::ErrorPayload for Error { fields.extend(source.payload()); fields } + Self::DefaultKeysetLifetime { keyset } => { + payload([("keyset", keyset.to_string().into())]) + } Self::ForeignKeyset { expected, found } => payload([ ("expected", expected.to_string().into()), ("found", found.to_string().into()), ]), + Self::ForeignBackend { expected, found } => payload([ + ("expected", expected.name().into()), + ("found", found.name().into()), + ]), + Self::SealUnsupported { backend } => payload([("backend", backend.name().into())]), #[cfg(feature = "http")] Self::Config(error) => error .downcast_ref::() @@ -692,6 +745,14 @@ impl StackCipher { /// ([`keyset_name_ttl`](StackCipherBuilder::keyset_name_ttl)) before it /// is asked again — ZeroKMS allows renames, and a running process /// notices one within that window. Selecting by id never asks twice. + /// + /// The key backend says how long the cipher may keep a keyset's index + /// key ([`KeyLifetime`](crate::kms::KeyLifetime)), and the cache obeys + /// it. ZeroKMS keys stay until evicted. A key with a TTL is loaded again + /// once the TTL ends. A key for one settle only is never cached: each + /// selection loads it again, and the returned handle (with what is built + /// from it) holds the only reference to it, so it drops with them. Hold + /// such a handle for one unit of work, not for the life of the process. pub async fn keyset( &self, keyset: impl Into, @@ -727,6 +788,11 @@ impl StackCipher { return Err(error); } }; + // An index key the cache may not keep goes to the caller alone: the + // handle, and what is built from it, hold the only references. + if state.retention == Retention::Never { + return Ok(KeysetCipher::new(self, state)); + } let state = self.keysets().insert(state, resolution); Ok(KeysetCipher::new(self, state)) } @@ -765,14 +831,19 @@ impl StackCipher { /// costs, shared by eager loading at /// [`init`](StackCipherBuilder::init) and lazy loading in /// [`StackCipher::keyset`] so both hold a keyset in exactly the same shape. -/// ZeroKMS answered a load with "no such keyset" — as opposed to not -/// answering at all. +/// The key backend answered a load with "no such keyset" — as opposed to +/// not answering at all. ZeroKMS says so as a keyset-not-found; a key vendor +/// backend as a [`NotFound`](stack_kms::BackendErrorKind::NotFound) failure. fn is_keyset_not_found(error: &Error) -> bool { matches!( error, - Error::Kms(stack_kms::Error::LoadKeyset( - stack_kms::LoadKeysetError::KeysetNotFound(_) - )) + Error::Kms( + stack_kms::Error::LoadKeyset(stack_kms::LoadKeysetError::KeysetNotFound(_)) + | stack_kms::Error::Backend { + kind: stack_kms::BackendErrorKind::NotFound, + .. + } + ) ) } @@ -784,22 +855,18 @@ async fn load_keyset( IdentifiedBy::Name(name) => Some(name.to_string()), IdentifiedBy::Uuid(_) => None, }; - let (id, index_key) = kms.load_index_key(Some(keyset)).await?; - Ok(keyset_state_of(id, name, &index_key)) + let loaded = kms.load_index_key_with_lifetime(Some(keyset)).await?; + Ok(keyset_state(loaded, name)) } -/// The state the cipher holds for a loaded keyset: its id, the name it was -/// selected by, and the PRF keyed by its index key. One construction for -/// every way a keyset is loaded. -fn keyset_state_of( - id: Uuid, - name: Option, - index_key: &stack_kms::IndexKey, -) -> Arc { +/// The state the cipher holds for a loaded index key: its PRF, and how long +/// the cache may keep it, from the lifetime the key backend gave it. +fn keyset_state(loaded: stack_kms::LoadedIndexKey, name: Option) -> Arc { Arc::new(KeysetState { - id, + id: loaded.keyset_id, name, - prf: hmac_prf_from_index_key(index_key), + prf: hmac_prf_from_index_key(&loaded.key), + retention: Retention::of(loaded.lifetime, std::time::Instant::now()), }) } @@ -807,9 +874,19 @@ fn keyset_state_of( /// this client — asked for by naming nothing. Loaded once, by /// [`init`](StackCipherBuilder::init); it is not the caller's to choose, so /// there is no id or name to carry. +/// +/// The cipher holds the default keyset for its whole life +/// ([`StackCipher::default_keyset`] is synchronous), so a key backend that +/// does not let the cipher keep its index key is refused here +/// ([`Error::DefaultKeysetLifetime`]), not obeyed in part. async fn load_default_keyset(kms: &K) -> Result, Error> { - let (id, index_key) = kms.load_index_key(None).await?; - Ok(keyset_state_of(id, None, &index_key)) + let loaded = kms.load_index_key_with_lifetime(None).await?; + if loaded.lifetime != stack_kms::KeyLifetime::UntilEvicted { + return Err(Error::DefaultKeysetLifetime { + keyset: loaded.keyset_id, + }); + } + Ok(keyset_state(loaded, None)) } /// The state of a [`StackCipherBuilder`] that has not been given a data-key @@ -1122,8 +1199,9 @@ impl StackCipher { } } -/// A single sealed leaf: the ZeroKMS metadata needed to retrieve its data key -/// (`iv`, `tag`) plus the vitaminc `LocalCipherText` sealed under that key. +/// A single sealed leaf: the key backend metadata needed to retrieve its +/// data key (`iv`, `tag`) plus the vitaminc `LocalCipherText` sealed under +/// that key. /// /// This is the only byte-format commitment the crate makes for *ciphertext* — /// the container tree ([`StackCipherText`]) has no canonical encoding, so @@ -1137,7 +1215,8 @@ impl StackCipher { /// [`to_bytes`](Self::to_bytes) / [`from_bytes`](Self::from_bytes) are the /// canonical encoding — the one storage format every consumer (this crate, /// the language bindings, anything reading a database column) agrees on. -/// The v1 layout: +/// There are two versions. ZeroKMS writes v1, and only v1. A key vendor +/// backend writes v2 (ADR-0010). The v1 layout: /// /// | offset | field | size | value | /// |-----------------|--------------------|-----------------|-------| @@ -1148,10 +1227,30 @@ impl StackCipher { /// | 35 | ZeroKMS key `tag` | `tag_len` | required to retrieve the key | /// | 35 + `tag_len` | local ciphertext | rest of buffer | the vitaminc `LocalCipherText` | /// +/// The v2 layout adds two bytes after the version, and is otherwise the +/// same: +/// +/// | offset | field | size | value | +/// |-----------------|--------------------|-----------------|-------| +/// | 0 | envelope version | 1 | [`VENDOR_FORMAT_VERSION`](Self::VENDOR_FORMAT_VERSION) (`0x02`) | +/// | 1 | key backend | 1 | [`KeyVendor::byte`](stack_kms::KeyVendor::byte): AWS KMS `1`, Google Cloud KMS `2`, Azure Key Vault `3`, Vault `4`, static `5` | +/// | 2 | key mode | 1 | [`KeyMode::byte`](stack_kms::KeyMode::byte): wrapped `1`, derived `2` | +/// | 3 | keyset id | 16 | the keyset the data key belongs to, raw UUID bytes | +/// | 19 | `iv` | 16 | identifies the data key; under derived data keys, the derivation salt | +/// | 35 | `tag_len` | 2 | length of `tag`, `u16` little-endian | +/// | 37 | `tag` | `tag_len` | under wrapped data keys, the wrapped data key; under derived data keys, empty | +/// | 37 + `tag_len` | local ciphertext | rest of buffer | the vitaminc `LocalCipherText` | +/// +/// The value `0` of the backend byte and of the mode byte is invalid, and +/// [`from_bytes`](Self::from_bytes) refuses a byte that names no backend or +/// no mode. ZeroKMS has no v2 backend byte, so a v2 leaf never names it. +/// /// The keyset id is what lets a leaf be opened without the caller saying /// which keyset it belongs to: retrieving the data key needs the keyset, /// and the leaf is self-describing so that a leaf lifted from a tree — what -/// a database column holds — is too. +/// a database column holds — is too. The backend byte does the same for +/// the key backend: a cipher refuses a leaf that names another key backend +/// than its own ([`Error::ForeignBackend`]) before any key call. /// /// The local ciphertext is itself a framed value — vitaminc's leaf wire /// format, versioned and owned by vitaminc — so the full stored byte string @@ -1165,22 +1264,26 @@ impl StackCipher { /// └─────────────────────────────────────────────────────────────────────────────────────┘ /// ``` /// -/// Both version bytes and the keyset id are authenticated under the one GCM -/// tag, each bound by the layer that owns its framing: the envelope version -/// and the keyset id through this crate's leaf-AAD derivation, -/// `PAE("stack-encrypt/leaf", version, keyset_id, derived_aad, tag)`, and -/// the inner version through vitaminc's `Context::for_leaf`, applied inside -/// `Aes256Cipher` to the AAD this crate hands it. Relabel either version -/// byte, or re-point the leaf at another keyset, in storage and the leaf -/// fails authentication rather than parsing under the wrong rules. Parsing -/// is structural only — nothing about a decoded leaf is trusted until it -/// decrypts. +/// Both version bytes, the v2 backend and mode bytes, and the keyset id are +/// authenticated under the one GCM tag, each bound by the layer that owns +/// its framing: the envelope header and the keyset id through this crate's +/// leaf-AAD derivation, +/// `PAE("stack-encrypt/leaf", header, keyset_id, derived_aad, tag)`, where +/// `header` is the version byte of a v1 leaf and `version ‖ backend ‖ mode` +/// of a v2 leaf, and the inner version through vitaminc's +/// `Context::for_leaf`, applied inside `Aes256Cipher` to the AAD this crate +/// hands it. Relabel a header byte, or re-point the leaf at another keyset, +/// in storage and the leaf fails authentication rather than parsing under +/// the wrong rules. Parsing is structural only — nothing about a decoded +/// leaf is trusted until it decrypts. /// /// The `serde` `Serialize`/`Deserialize` derives and /// [`into_parts`](Self::into_parts) / [`from_parts`](Self::from_parts) /// remain for callers that manage their own storage format; they carry the /// same fields, but their wire form is the serialiser's, not a commitment -/// of this crate. +/// of this crate. The serde form of a v1 leaf has no field for the backend, +/// and a v2 leaf adds one (`vendor`). `into_parts` drops the backend and +/// the mode, so persist a v2 leaf with [`to_bytes`](Self::to_bytes). #[derive(Debug, Serialize, Deserialize)] #[serde(try_from = "SealedValueRepr")] pub struct SealedValue { @@ -1194,12 +1297,79 @@ pub struct SealedValue { /// The leaf sealed by [`vitaminc_encrypt::Aes256Cipher`] under the data /// key: `version ‖ nonce ‖ ciphertext ‖ gcm_tag`. ciphertext: LocalCipherText, + /// The key vendor backend and key mode of a v2 leaf; `None` for a v1 + /// (ZeroKMS) leaf. Last, and absent from a v1 leaf's serde form, so + /// that form does not change. + #[serde(default, skip_serializing_if = "Option::is_none")] + vendor: Option, +} + +/// The two v2 header bytes: the key vendor backend and the key mode. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Serialize, Deserialize)] +#[serde(into = "[u8; 2]", try_from = "[u8; 2]")] +struct VendorHeader { + vendor: KeyVendor, + mode: KeyMode, +} + +impl VendorHeader { + /// Decode the backend byte and the mode byte. + fn from_bytes(vendor: u8, mode: u8) -> Result { + Ok(Self { + vendor: KeyVendor::from_byte(vendor).ok_or(LeafBytesError::UnknownBackend(vendor))?, + mode: KeyMode::from_byte(mode).ok_or(LeafBytesError::UnknownMode(mode))?, + }) + } +} + +impl From for [u8; 2] { + fn from(header: VendorHeader) -> Self { + [header.vendor.byte(), header.mode.byte()] + } +} + +impl TryFrom<[u8; 2]> for VendorHeader { + type Error = LeafBytesError; + + fn try_from([vendor, mode]: [u8; 2]) -> Result { + Self::from_bytes(vendor, mode) + } +} + +/// The envelope header bytes of a leaf, as written and as bound into the +/// leaf AAD: `version` for v1, `version ‖ backend ‖ mode` for v2. +struct LeafHeader { + bytes: [u8; 3], + len: usize, +} + +impl LeafHeader { + fn of(vendor: Option) -> Self { + match vendor { + None => Self { + bytes: [SealedValue::FORMAT_VERSION, 0, 0], + len: 1, + }, + Some(header) => Self { + bytes: [ + SealedValue::VENDOR_FORMAT_VERSION, + header.vendor.byte(), + header.mode.byte(), + ], + len: 3, + }, + } + } + + fn as_bytes(&self) -> &[u8] { + &self.bytes[..self.len] + } } /// A [`SealedValue`] byte encoding failed to encode or decode. Purely /// structural — a leaf that *decodes* has proven nothing about integrity /// (that is the AEAD open's job); a leaf that fails here was never a valid -/// v1 encoding at all. +/// v1 or v2 encoding at all. #[derive(Debug, PartialEq, Eq, thiserror::Error, miette::Diagnostic)] #[non_exhaustive] pub enum LeafBytesError { @@ -1227,6 +1397,21 @@ pub enum LeafBytesError { #[error("key tag of {0} bytes exceeds the format's u16 length field")] #[diagnostic(code(stack_encrypt::leaf_tag_too_long))] TagTooLong(usize), + /// The backend byte of a v2 leaf names no key vendor backend this build + /// knows. `0` is never valid. + #[error("unknown sealed-leaf key backend {0}")] + #[diagnostic( + code(stack_encrypt::leaf_backend), + help("The value was sealed by a newer build of stack-encrypt, or the bytes are not a sealed value.") + )] + UnknownBackend(u8), + /// The mode byte of a v2 leaf names no key mode. `0` is never valid. + #[error("unknown sealed-leaf key mode {0}")] + #[diagnostic( + code(stack_encrypt::leaf_mode), + help("The value was sealed by a newer build of stack-encrypt, or the bytes are not a sealed value.") + )] + UnknownMode(u8), } impl crate::ErrorPayload for LeafBytesError { @@ -1235,21 +1420,40 @@ impl crate::ErrorPayload for LeafBytesError { match self { Self::UnknownVersion(version) => payload([("version", (*version).into())]), Self::TagTooLong(len) => payload([("len", (*len).into())]), + Self::UnknownBackend(byte) => payload([("backend", (*byte).into())]), + Self::UnknownMode(byte) => payload([("mode", (*byte).into())]), Self::Truncated => serde_json::Map::new(), } } } impl SealedValue { - /// The version byte prefixing the frozen byte encoding - /// ([`to_bytes`](Self::to_bytes)). Also bound into every leaf's AAD (the + /// The version byte of the v1 encoding ([`to_bytes`](Self::to_bytes)), + /// the one ZeroKMS writes. Also bound into every leaf's AAD (the /// private `leaf_aad` derivation): bumping it re-keys authentication, so /// old leaves can never be relabelled as the new version (nor new as /// old). pub const FORMAT_VERSION: u8 = 1; - /// Encode into the frozen v1 byte layout — see the type-level docs for - /// the format. The inverse of [`from_bytes`](Self::from_bytes). + /// The version byte of the v2 encoding, the one a key vendor backend + /// writes. It is followed by the backend byte and the mode byte, and + /// all three are bound into the leaf AAD. + pub const VENDOR_FORMAT_VERSION: u8 = 2; + + /// The version byte this leaf encodes with: + /// [`FORMAT_VERSION`](Self::FORMAT_VERSION) for a ZeroKMS leaf, + /// [`VENDOR_FORMAT_VERSION`](Self::VENDOR_FORMAT_VERSION) for a leaf of a + /// key vendor backend. + pub fn format_version(&self) -> u8 { + match self.vendor { + None => Self::FORMAT_VERSION, + Some(_) => Self::VENDOR_FORMAT_VERSION, + } + } + + /// Encode into the frozen byte layout — v1 for a ZeroKMS leaf, v2 for a + /// leaf of a key vendor backend; see the type-level docs for the + /// format. The inverse of [`from_bytes`](Self::from_bytes). /// /// Infallible: every way of building a `SealedValue` rejects a tag too /// long for the `u16` length field ([`LeafBytesError::TagTooLong`]), so a @@ -1261,10 +1465,13 @@ impl SealedValue { // The saturating fallback is unreachable, and asserted so in tests. let tag_len = u16::try_from(self.tag.len()).unwrap_or(u16::MAX); debug_assert_eq!(usize::from(tag_len), self.tag.len()); + let header = LeafHeader::of(self.vendor); + let header = header.as_bytes(); let ciphertext = self.ciphertext.as_ref(); - let mut out = - Vec::with_capacity(1 + 16 + self.iv.len() + 2 + self.tag.len() + ciphertext.len()); - out.push(Self::FORMAT_VERSION); + let mut out = Vec::with_capacity( + header.len() + 16 + self.iv.len() + 2 + self.tag.len() + ciphertext.len(), + ); + out.extend_from_slice(header); out.extend_from_slice(self.keyset_id.as_bytes()); out.extend_from_slice(&self.iv); out.extend_from_slice(&tag_len.to_le_bytes()); @@ -1273,16 +1480,23 @@ impl SealedValue { out } - /// Decode the frozen v1 byte layout — the inverse of + /// Decode the frozen v1 or v2 byte layout — the inverse of /// [`to_bytes`](Self::to_bytes). Structural only: a decoded leaf is /// untrusted bytes until it decrypts. pub fn from_bytes(bytes: &[u8]) -> Result { const IV_LEN: usize = 16; let (&version, rest) = bytes.split_first().ok_or(LeafBytesError::Truncated)?; - if version != Self::FORMAT_VERSION { - return Err(LeafBytesError::UnknownVersion(version)); - } + let (vendor, rest) = match version { + Self::FORMAT_VERSION => (None, rest), + Self::VENDOR_FORMAT_VERSION => match rest { + [vendor, mode, rest @ ..] => { + (Some(VendorHeader::from_bytes(*vendor, *mode)?), rest) + } + _ => return Err(LeafBytesError::Truncated), + }, + _ => return Err(LeafBytesError::UnknownVersion(version)), + }; if rest.len() < 16 + IV_LEN + 2 { return Err(LeafBytesError::Truncated); } @@ -1302,6 +1516,7 @@ impl SealedValue { iv, tag: tag.to_vec(), ciphertext: LocalCipherText::from(ciphertext.to_vec()), + vendor, }) } @@ -1314,8 +1529,8 @@ impl SealedValue { Ok(()) } - /// Rebuild a leaf from its persisted parts — the inverse of - /// [`into_parts`](Self::into_parts). + /// Rebuild a v1 (ZeroKMS) leaf from its persisted parts — the inverse + /// of [`into_parts`](Self::into_parts). /// /// Fails with [`LeafBytesError::TagTooLong`] if `tag` does not fit the /// byte format's `u16` length field. Structural only: nothing about the @@ -1332,10 +1547,31 @@ impl SealedValue { iv, tag, ciphertext: LocalCipherText::from(ciphertext), + vendor: None, }) } + /// Rebuild a v2 leaf of the key vendor backend `vendor`, in key mode + /// `mode`, from its parts. As [`from_parts`](Self::from_parts) in every + /// other way. + pub fn from_vendor_parts( + vendor: KeyVendor, + mode: KeyMode, + keyset_id: Uuid, + iv: stack_kms::Iv, + tag: Vec, + ciphertext: Vec, + ) -> Result { + let mut leaf = Self::from_parts(keyset_id, iv, tag, ciphertext)?; + leaf.vendor = Some(VendorHeader { vendor, mode }); + Ok(leaf) + } + /// Decompose into `(keyset_id, iv, tag, ciphertext)` for persistence. + /// + /// The parts do not include the key backend or the key mode. A v2 leaf + /// rebuilt with [`from_parts`](Self::from_parts) is a v1 leaf, which + /// fails to open; persist a v2 leaf with [`to_bytes`](Self::to_bytes). pub fn into_parts(self) -> (Uuid, stack_kms::Iv, Vec, Vec) { ( self.keyset_id, @@ -1352,12 +1588,33 @@ impl SealedValue { self.keyset_id } - /// The ZeroKMS IV identifying this leaf's data key. + /// The key backend this leaf names: ZeroKMS for a v1 leaf, the key + /// vendor backend for a v2 leaf. Authenticated: a leaf relabelled with + /// another backend fails to open. A cipher refuses a leaf that names + /// another backend than its own before any key call + /// ([`Error::ForeignBackend`]). + pub fn key_backend(&self) -> KeyBackend { + match self.vendor { + None => KeyBackend::ZeroKms, + Some(header) => KeyBackend::Vendor(header.vendor), + } + } + + /// The key mode of a v2 leaf; `None` for a v1 (ZeroKMS) leaf, since + /// ZeroKMS has no key mode. Authenticated as + /// [`key_backend`](Self::key_backend) is. + pub fn key_mode(&self) -> Option { + self.vendor.map(|header| header.mode) + } + + /// The IV identifying this leaf's data key. Under derived data keys, it + /// is the derivation salt. pub fn iv(&self) -> &stack_kms::Iv { &self.iv } - /// The ZeroKMS key tag. + /// The key tag. Under wrapped data keys, it is the wrapped data key; + /// under derived data keys, it is empty. pub fn tag(&self) -> &[u8] { &self.tag } @@ -1375,6 +1632,7 @@ impl Clone for SealedValue { iv: self.iv, tag: self.tag.clone(), ciphertext: LocalCipherText::from(self.ciphertext.as_ref().to_vec()), + vendor: self.vendor, } } } @@ -1400,6 +1658,8 @@ struct SealedValueRepr { iv: stack_kms::Iv, tag: Vec, ciphertext: LocalCipherText, + #[serde(default)] + vendor: Option, } impl TryFrom for SealedValue { @@ -1411,6 +1671,7 @@ impl TryFrom for SealedValue { iv, tag, ciphertext, + vendor, } = repr; Self::tag_fits_length_field(&tag)?; Ok(Self { @@ -1418,6 +1679,7 @@ impl TryFrom for SealedValue { iv, tag, ciphertext, + vendor, }) } } @@ -1624,11 +1886,24 @@ fn leaf_cipher(key: &DataKey) -> Result { /// dev-persisted data existed. A leaf sealed under an earlier form fails /// authentication in `open_leaf` with a plain AEAD error, indistinguishable /// from tampering; re-encrypt anything that matters. -fn leaf_aad(aad: &Context<'_>, keyset_id: Uuid, tag: &[u8]) -> Context<'static> { +/// +/// # The v2 header +/// +/// The second piece is the leaf's envelope header: the version byte alone +/// for a v1 leaf, so a v1 leaf's AAD is byte for byte what it always was, +/// and `version ‖ backend ‖ mode` for a v2 leaf. PAE frames each piece with +/// its length, so a v2 header can never encode as a v1 one, and a v2 leaf +/// relabelled with another backend or mode fails verification. +fn leaf_aad( + aad: &Context<'_>, + vendor: Option, + keyset_id: Uuid, + tag: &[u8], +) -> Context<'static> { const LEAF_AAD_DOMAIN: &[u8] = b"stack-encrypt/leaf"; Context::pae(&[ LEAF_AAD_DOMAIN, - &[SealedValue::FORMAT_VERSION], + LeafHeader::of(vendor).as_bytes(), keyset_id.as_bytes(), aad.as_bytes(), tag, @@ -1646,6 +1921,21 @@ fn seal_leaf( aad: &Context<'_>, keyset_id: Uuid, key: DataKeyWithTag, +) -> Result { + // Only ZeroKMS seals in this build, and a ZeroKMS leaf is v1. The settle + // refuses a data key from a key vendor backend before this point + // (`Error::SealUnsupported`). + seal_leaf_as(None, plaintext, aad, keyset_id, key) +} + +/// [`seal_leaf`] in the v1 format (`vendor` is `None`) or the v2 format of +/// a key vendor backend. +fn seal_leaf_as( + vendor: Option, + plaintext: Protected>, + aad: &Context<'_>, + keyset_id: Uuid, + key: DataKeyWithTag, ) -> Result { // The `DataKeySource` is caller-supplied, so the key tag is not trusted // to fit the frozen byte format's `u16` length field: an oversized tag @@ -1655,12 +1945,13 @@ fn seal_leaf( SealedValue::tag_fits_length_field(&key.tag).map_err(|_| Unspecified)?; let iv = key.key.iv; let cipher = leaf_cipher(&key.key)?; - let ciphertext = cipher.seal(plaintext, &leaf_aad(aad, keyset_id, &key.tag))?; + let ciphertext = cipher.seal(plaintext, &leaf_aad(aad, vendor, keyset_id, &key.tag))?; Ok(SealedValue { keyset_id, iv, tag: key.tag, ciphertext, + vendor, }) } @@ -1668,7 +1959,10 @@ fn seal_leaf( fn open_leaf(keyed: KeyedLeaf, aad: &Context<'_>) -> Result>, Unspecified> { let KeyedLeaf { leaf, key } = keyed; let cipher = leaf_cipher(&key)?; - cipher.open(leaf.ciphertext, &leaf_aad(aad, leaf.keyset_id, &leaf.tag)) + cipher.open( + leaf.ciphertext, + &leaf_aad(aad, leaf.vendor, leaf.keyset_id, &leaf.tag), + ) } // ============================================================================= @@ -1873,6 +2167,43 @@ mod tests { ); } + /// "No such keyset" is an answer about the name from either kind of key + /// backend: ZeroKMS's keyset-not-found, or a key vendor backend's + /// `NotFound`. Every other failure is no answer. + #[test] + fn a_keyset_is_not_found_by_zerokms_or_by_a_key_vendor_backend() { + use stack_kms::{BackendErrorKind, LoadKeysetError}; + use zerokms_protocol::{ViturRequestError, ViturRequestErrorKind}; + let vitur = |kind| ViturRequestError::new(kind, "stubbed", std::io::Error::other("detail")); + let backend = |kind| { + Error::Kms(stack_kms::Error::Backend { + vendor: "test-vendor", + kind, + source: Box::new(std::io::Error::other("detail")), + }) + }; + assert!(is_keyset_not_found(&Error::Kms( + stack_kms::Error::LoadKeyset(LoadKeysetError::KeysetNotFound(vitur( + ViturRequestErrorKind::NotFound + ))) + ))); + assert!(is_keyset_not_found(&backend(BackendErrorKind::NotFound))); + for kind in [ + BackendErrorKind::AccessDenied, + BackendErrorKind::Throttled, + BackendErrorKind::Unavailable, + BackendErrorKind::InvalidCiphertext, + ] { + assert!(!is_keyset_not_found(&backend(kind)), "{kind:?}"); + } + assert!(!is_keyset_not_found(&Error::Kms( + stack_kms::Error::LoadKeyset(LoadKeysetError::Forbidden(vitur( + ViturRequestErrorKind::Forbidden + ))) + ))); + assert!(!is_keyset_not_found(&Error::Aead)); + } + /// Byte-level pin for the [`leaf_aad`] derivation. This is part of the /// frozen leaf format: a change to the domain label, the version byte, /// the keyset id's place, the piece order, or the PAE framing makes @@ -1882,7 +2213,12 @@ mod tests { #[test] fn leaf_aad_bytes_are_pinned() { let keyset = Uuid::from_bytes(*b"keyset-fixture16"); - let aad = leaf_aad(&Context::from_encoded(b"caller-aad"), keyset, b"key-tag"); + let aad = leaf_aad( + &Context::from_encoded(b"caller-aad"), + None, + keyset, + b"key-tag", + ); let hex: String = aad.as_bytes().iter().map(|b| format!("{b:02x}")).collect(); // PAE: LE64 count (5) ‖ per piece LE64 length ‖ piece, the pieces // being "stack-encrypt/leaf", [FORMAT_VERSION], the keyset id's 16 @@ -1892,4 +2228,182 @@ mod tests { "05000000000000001200000000000000737461636b2d656e63727970742f6c65616601000000000000000110000000000000006b65797365742d6669787475726531360a0000000000000063616c6c65722d61616407000000000000006b65792d746167" ); } + + /// The leaf-AAD pin for a v2 leaf: the header piece is + /// `version ‖ backend ‖ mode`, three bytes, where a v1 leaf's is one. + #[test] + fn v2_leaf_aad_bytes_are_pinned() { + let keyset = Uuid::from_bytes(*b"keyset-fixture16"); + let header = VendorHeader { + vendor: KeyVendor::Aws, + mode: KeyMode::Wrapped, + }; + let aad = leaf_aad( + &Context::from_encoded(b"caller-aad"), + Some(header), + keyset, + b"key-tag", + ); + let hex: String = aad.as_bytes().iter().map(|b| format!("{b:02x}")).collect(); + assert_eq!( + hex, + "05000000000000001200000000000000737461636b2d656e63727970742f6c656166030000000000000002010110000000000000006b65797365742d6669787475726531360a0000000000000063616c6c65722d61616407000000000000006b65792d746167" + ); + } + + const V2_HEADER: VendorHeader = VendorHeader { + vendor: KeyVendor::Static, + mode: KeyMode::Derived, + }; + + fn test_key() -> DataKeyWithTag { + DataKeyWithTag { + key: DataKey { + iv: *b"v2-leaf-iv-salt!", + key: [0x42; 32], + }, + tag: Vec::new(), + decryption_policy: None, + } + } + + fn open_with_test_key(leaf: SealedValue, aad: &Context<'_>) -> Result, Unspecified> { + use vitaminc_protected::Controlled; + let key = test_key().key; + open_leaf(KeyedLeaf { leaf, key }, aad).map(|plaintext| plaintext.risky_unwrap()) + } + + fn sealed_v2_bytes(aad: &Context<'_>) -> Vec { + let keyset = Uuid::from_bytes(*b"keyset-fixture16"); + let leaf = seal_leaf_as( + Some(V2_HEADER), + Protected::new(b"v2 plaintext".to_vec()), + aad, + keyset, + test_key(), + ) + .expect("seal a v2 leaf"); + assert_eq!(leaf.format_version(), SealedValue::VENDOR_FORMAT_VERSION); + assert_eq!( + leaf.key_backend(), + KeyBackend::Vendor(KeyVendor::Static), + "the leaf names its key vendor backend" + ); + assert_eq!(leaf.key_mode(), Some(KeyMode::Derived)); + leaf.to_bytes() + } + + /// A v2 leaf seals, survives its byte encoding, and opens under the + /// same key and AAD. + #[test] + fn a_v2_leaf_round_trips_through_its_bytes() { + let aad = Context::from_encoded(b"ctx"); + let bytes = sealed_v2_bytes(&aad); + assert_eq!( + &bytes[..3], + &[2, KeyVendor::Static.byte(), KeyMode::Derived.byte()] + ); + let leaf = SealedValue::from_bytes(&bytes).expect("decode v2 leaf"); + assert_eq!(leaf.to_bytes(), bytes, "the v2 encoding is lossless"); + assert_eq!( + open_with_test_key(leaf, &aad).expect("open v2 leaf"), + b"v2 plaintext" + ); + } + + /// The leaf AAD binds each v2 header byte: every other value of the + /// version, the backend or the mode byte is refused, by the decoder or + /// by the open. None of them opens. + #[test] + fn changing_any_v2_header_byte_makes_the_open_fail() { + let aad = Context::from_encoded(b"ctx"); + let bytes = sealed_v2_bytes(&aad); + let mut opened_by_decoder = 0; + for position in 0..3 { + for value in 0..=u8::MAX { + if value == bytes[position] { + continue; + } + let mut tampered = bytes.clone(); + tampered[position] = value; + match SealedValue::from_bytes(&tampered) { + Err(_) => {} + Ok(leaf) => { + opened_by_decoder += 1; + assert!( + open_with_test_key(leaf, &aad).is_err(), + "byte {position} changed to {value} must not open" + ); + } + } + } + } + // Every other valid backend and mode decodes, so the AEAD is what + // refuses them; the count proves the AEAD was reached. (Version 1 + // reads the following bytes at v1 offsets, and this fixture's + // `tag_len` then overruns the buffer, so the decoder refuses it. + // `a_v2_leaf_relabelled_as_v1_does_not_open` covers the AEAD case.) + assert_eq!( + opened_by_decoder, + (KeyVendor::ALL.len() - 1) + (KeyMode::ALL.len() - 1) + ); + } + + /// The same header refusal holds when a v2 leaf is relabelled as v1 + /// without a shift in the following bytes: a v1 leaf built from the v2 + /// leaf's parts does not open. + #[test] + fn a_v2_leaf_relabelled_as_v1_does_not_open() { + let aad = Context::from_encoded(b"ctx"); + let leaf = SealedValue::from_bytes(&sealed_v2_bytes(&aad)).expect("decode v2 leaf"); + let (keyset, iv, tag, ciphertext) = leaf.into_parts(); + let relabelled = SealedValue::from_parts(keyset, iv, tag, ciphertext).expect("v1 parts"); + assert_eq!(relabelled.key_backend(), KeyBackend::ZeroKms); + assert!(open_with_test_key(relabelled, &aad).is_err()); + } + + /// A v1 leaf opens as it always did: the v1 AAD header is the version + /// byte alone. + #[test] + fn a_v1_leaf_still_opens_under_the_v1_aad() { + let aad = Context::from_encoded(b"ctx"); + let keyset = Uuid::from_bytes(*b"keyset-fixture16"); + let leaf = seal_leaf( + Protected::new(b"v1 plaintext".to_vec()), + &aad, + keyset, + test_key(), + ) + .expect("seal a v1 leaf"); + let bytes = leaf.to_bytes(); + assert_eq!(bytes[0], SealedValue::FORMAT_VERSION); + assert_eq!(leaf.key_mode(), None); + let leaf = SealedValue::from_bytes(&bytes).expect("decode v1 leaf"); + assert_eq!( + open_with_test_key(leaf, &aad).expect("open v1 leaf"), + b"v1 plaintext" + ); + } + + /// The serde form keeps the backend and the mode of a v2 leaf, and a v1 + /// leaf's serde form has no field for them. + #[test] + fn serde_keeps_the_v2_header_and_leaves_v1_unchanged() { + let aad = Context::from_encoded(b"ctx"); + let v2 = SealedValue::from_bytes(&sealed_v2_bytes(&aad)).expect("decode v2 leaf"); + let json = serde_json::to_value(&v2).expect("serialise v2"); + assert_eq!(json["vendor"], serde_json::json!([5, 2])); + let back: SealedValue = serde_json::from_value(json).expect("deserialise v2"); + assert_eq!(back.to_bytes(), v2.to_bytes()); + + let v1 = SealedValue::from_parts(Uuid::nil(), [0; 16], vec![1], vec![2]).expect("v1"); + let json = serde_json::to_value(&v1).expect("serialise v1"); + assert!(json.get("vendor").is_none(), "{json}"); + let back: SealedValue = serde_json::from_value(json).expect("deserialise v1"); + assert_eq!(back.to_bytes(), v1.to_bytes()); + + let mut bad = serde_json::to_value(&v2).expect("serialise v2"); + bad["vendor"] = serde_json::json!([0, 2]); + assert!(serde_json::from_value::(bad).is_err()); + } } diff --git a/packages/stack-encrypt/src/codes.rs b/packages/stack-encrypt/src/codes.rs index f2af81fd84..0b6e21386d 100644 --- a/packages/stack-encrypt/src/codes.rs +++ b/packages/stack-encrypt/src/codes.rs @@ -82,9 +82,17 @@ mod tests { expected: a, found: b, }, + Error::ForeignBackend { .. } => Error::ForeignBackend { + expected: crate::kms::KeyBackend::ZeroKms, + found: crate::kms::KeyBackend::Vendor(crate::kms::KeyVendor::Aws), + }, + Error::SealUnsupported { .. } => Error::SealUnsupported { + backend: crate::kms::KeyBackend::Vendor(crate::kms::KeyVendor::Static), + }, Error::NoKeyset => Error::NoKeyset, Error::NotOpened => Error::NotOpened, Error::Plan(_) => Error::Plan(PlanError::NoContext), + Error::DefaultKeysetLifetime { .. } => Error::DefaultKeysetLifetime { keyset: a }, Error::Blob(_) => Error::Blob(BlobError::NotABlob), ])); errors.extend(boxed(variants![ @@ -98,6 +106,8 @@ mod tests { LeafBytesError::UnknownVersion(_) => LeafBytesError::UnknownVersion(9), LeafBytesError::Truncated => LeafBytesError::Truncated, LeafBytesError::TagTooLong(_) => LeafBytesError::TagTooLong(70_000), + LeafBytesError::UnknownBackend(_) => LeafBytesError::UnknownBackend(0), + LeafBytesError::UnknownMode(_) => LeafBytesError::UnknownMode(0), ])); errors.extend(boxed(variants![ TermError::Prf(_) => TermError::Prf(cause()), @@ -369,6 +379,40 @@ mod tests { assert_eq!(error.to_string(), "Unexpected error: kms"); } + /// A key vendor backend's failure shows through the encryption error as + /// itself: its code, and its vendor and kind as fields. Its refusal of a + /// ZeroKMS-only feature does the same. + #[test] + fn a_key_backend_error_shows_through() { + use crate::ErrorPayload; + use serde_json::json; + let failure = Error::Kms(crate::kms::Error::Backend { + vendor: "test-vendor", + kind: crate::kms::BackendErrorKind::AccessDenied, + source: Box::new(std::io::Error::other("vendor detail")), + }); + assert_eq!( + failure.code().map(|code| code.to_string()).as_deref(), + Some("stack_kms::backend") + ); + assert_eq!( + serde_json::Value::Object(failure.payload()), + json!({ "vendor": "test-vendor", "kind": "AccessDenied" }) + ); + let refusal = Error::Kms(crate::kms::Error::ZeroKmsOnly { + vendor: "test-vendor", + feature: crate::kms::ZeroKmsFeature::UnverifiedContext, + }); + assert_eq!( + refusal.code().map(|code| code.to_string()).as_deref(), + Some("stack_kms::zerokms_only") + ); + assert_eq!( + serde_json::Value::Object(refusal.payload()), + json!({ "vendor": "test-vendor", "feature": "unverified_context" }) + ); + } + /// Every error type's structured fields, one variant per arm. A binding /// hands these over as they are, so a field that goes missing or changes /// name is a break for every caller that reads it. @@ -426,6 +470,10 @@ mod tests { Box::new(Error::KeysetMismatch { left: a, right: b }), json!({ "left": a.to_string(), "right": b.to_string() }), ), + ( + Box::new(Error::DefaultKeysetLifetime { keyset: a }), + json!({ "keyset": a.to_string() }), + ), (Box::new(Error::Aead), json!({})), ( Box::new(Error::OpenedType { expected: "i32" }), @@ -450,6 +498,27 @@ mod tests { json!({ "len": 70_000 }), ), (Box::new(LeafBytesError::Truncated), json!({})), + ( + Box::new(LeafBytesError::UnknownBackend(9)), + json!({ "backend": 9 }), + ), + ( + Box::new(LeafBytesError::UnknownMode(7)), + json!({ "mode": 7 }), + ), + ( + Box::new(Error::ForeignBackend { + expected: crate::kms::KeyBackend::ZeroKms, + found: crate::kms::KeyBackend::Vendor(crate::kms::KeyVendor::Vault), + }), + json!({ "expected": "zerokms", "found": "vault-transit" }), + ), + ( + Box::new(Error::SealUnsupported { + backend: crate::kms::KeyBackend::Vendor(crate::kms::KeyVendor::Gcp), + }), + json!({ "backend": "gcp-kms" }), + ), ( Box::new(TermError::Bytes(TermBytesError::OddMatchTermsLength(3))), json!({ "len": 3 }), diff --git a/packages/stack-encrypt/src/descriptor.rs b/packages/stack-encrypt/src/descriptor.rs index bf61d81abc..2aed78b936 100644 --- a/packages/stack-encrypt/src/descriptor.rs +++ b/packages/stack-encrypt/src/descriptor.rs @@ -84,16 +84,18 @@ impl Descriptor { /// The separator between the parts of a list. pub const SEPARATOR: char = '/'; - /// The longest descriptor ZeroKMS accepts, in bytes of the rendered - /// string: the protocol's [`MAX_DESCRIPTOR_LEN`](crate::kms::MAX_DESCRIPTOR_LEN). - /// ZeroKMS derives key material over a fixed block of that size holding - /// the descriptor, so a longer one cannot be bound. Every data-key + /// The longest descriptor a key backend accepts, in bytes of the + /// rendered string. The limit is the same for every key backend, so + /// that a context which fits one fits all. It is ZeroKMS's + /// [`MAX_DESCRIPTOR_LEN`](crate::kms::MAX_DESCRIPTOR_LEN): ZeroKMS + /// derives key material over a fixed block of that size holding the + /// descriptor, so a longer one cannot be bound. Every data-key /// request checks its descriptors against this before anything is sent /// ([`Error::DescriptorTooLong`](crate::Error::DescriptorTooLong)); a /// context is free to be long, but what it renders to must fit — and /// the base64 escape grows a part by a third, so an escaped part fits /// less than a plain one. - pub const MAX_LEN: usize = stack_kms::MAX_DESCRIPTOR_LEN; + pub const MAX_LEN: usize = 512; /// Render `context` — any [`IntoContext`] type — from its parts. /// @@ -924,6 +926,16 @@ mod tests { assert!(!escaped.fits()); } + /// The limit is stated here, not taken from ZeroKMS, so that a key + /// vendor backend does not reach a ZeroKMS constant. It must stay equal + /// to ZeroKMS's, or a context that fits one key backend would not fit + /// another. + #[test] + fn the_limit_is_zerokms_limit_for_every_key_backend() { + assert_eq!(Descriptor::MAX_LEN, 512); + assert_eq!(Descriptor::MAX_LEN, crate::kms::MAX_DESCRIPTOR_LEN); + } + #[test] fn a_clone_shares_the_rendering() { let descriptor = Descriptor::of("a".repeat(Descriptor::MAX_LEN)); diff --git a/packages/stack-encrypt/src/json/derive.rs b/packages/stack-encrypt/src/json/derive.rs index 73bf163f0f..ef7289ed47 100644 --- a/packages/stack-encrypt/src/json/derive.rs +++ b/packages/stack-encrypt/src/json/derive.rs @@ -350,6 +350,7 @@ impl Selectors { id: Uuid::from_u128(1), name: None, prf: HmacSha256Prf::new(Protected::new([7; 32])), + retention: crate::keyset::Retention::UntilEvicted, }), Context::from_encoded(b"users/profile").into_owned(), ) @@ -3051,6 +3052,7 @@ mod tests { id: Uuid::from_u128(2), name: None, prf: prf(), + retention: crate::keyset::Retention::UntilEvicted, }), selectors.context.clone(), ); diff --git a/packages/stack-encrypt/src/keyset.rs b/packages/stack-encrypt/src/keyset.rs index a4473d13b7..a5e24b191f 100644 --- a/packages/stack-encrypt/src/keyset.rs +++ b/packages/stack-encrypt/src/keyset.rs @@ -20,6 +20,12 @@ //! [`init`](crate::StackCipherBuilder::init), so a misconfigured client //! fails at startup, and never evicts. //! +//! A key backend says how long the cipher may keep a keyset's index key +//! ([`KeyLifetime`]): until evicted (ZeroKMS), for a TTL, or for one settle +//! only. The cache obeys it, so each backend does not have to. The default +//! keyset must be one the cipher may keep until evicted, because the cipher +//! holds it for its whole life. +//! //! # Ids are identity; names are looked up //! //! A keyset's id is its identity: a sealed leaf carries it, and an id @@ -51,7 +57,7 @@ use std::num::NonZeroUsize; use std::sync::Arc; use std::time::{Duration, Instant}; -use stack_kms::IdentifiedBy; +use stack_kms::{IdentifiedBy, KeyLifetime}; use uuid::Uuid; use vitaminc_hmac::HmacSha256Prf; @@ -64,11 +70,58 @@ use crate::StackCipher; pub const DEFAULT_NAME_TTL: Duration = Duration::from_secs(5 * 60); /// What the cipher holds per loaded keyset: its resolved id, the name it was -/// loaded under if any, and the PRF keyed by its index key. +/// loaded under if any, the PRF keyed by its index key, and how long the +/// cache may keep that PRF. pub(crate) struct KeysetState { pub(crate) id: Uuid, pub(crate) name: Option, pub(crate) prf: HmacSha256Prf, + pub(crate) retention: Retention, +} + +/// How long the [`KeysetCache`] may keep a keyset's state: the +/// [`KeyLifetime`] its key backend gave the index key, fixed to a point in +/// time when it was loaded. +/// +/// The cipher enforces this, not each backend. A state the cache may not +/// keep at all is handed to the caller and never stored, so the only +/// references to it are the [`KeysetCipher`] handles and the pendings +/// built from them. When those drop, the PRF drops with them. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub(crate) enum Retention { + /// Never cached: the index key is for the settle that needed it. + Never, + /// Cached until this instant, then loaded again. + Until(Instant), + /// Cached until the bound evicts it (the ZeroKMS behaviour). + UntilEvicted, +} + +impl Retention { + /// The retention of an index key with `lifetime`, loaded at `now`. + /// + /// A zero TTL is never cached, the same as a settle. A TTL too long for + /// the clock to represent is kept until evicted. A lifetime this crate + /// does not know yet is never cached: the short answer is the safe one. + pub(crate) fn of(lifetime: KeyLifetime, now: Instant) -> Self { + match lifetime { + KeyLifetime::UntilEvicted => Self::UntilEvicted, + KeyLifetime::Ttl(ttl) if !ttl.is_zero() => { + now.checked_add(ttl).map_or(Self::UntilEvicted, Self::Until) + } + _ => Self::Never, + } + } + + /// Whether the cache may still hand this state out at `now`. The end of + /// a TTL is exclusive: at the instant it ends, the key is expired. + pub(crate) fn is_live_at(self, now: Instant) -> bool { + match self { + Self::Never => false, + Self::Until(end) => now < end, + Self::UntilEvicted => true, + } + } } /// A loaded keyset in the cache, with the name it is currently bound under @@ -288,13 +341,28 @@ impl KeysetCache { /// Look a keyset up by id or name, marking it most recently used. pub(crate) fn get(&mut self, by: &IdentifiedBy) -> Lookup { + self.get_at(by, Instant::now()) + } + + /// [`get`](Self::get), at `now`. A keyset whose index key has passed the + /// end of its lifetime is evicted here, as if the bound had evicted it, + /// and the lookup is a miss: the caller loads it again. + fn get_at(&mut self, by: &IdentifiedBy, now: Instant) -> Lookup { let (id, fresh) = match by { IdentifiedBy::Uuid(id) => (*id, true), IdentifiedBy::Name(name) => match self.by_name.get::(name) { - Some(alias) => (alias.id, alias.is_fresh_at(Instant::now(), self.name_ttl)), + Some(alias) => (alias.id, alias.is_fresh_at(now, self.name_ttl)), None => return Lookup::Miss(self.resolution()), }, }; + if self + .by_id + .get(&id) + .is_some_and(|entry| !entry.state.retention.is_live_at(now)) + { + self.evict(id); + return Lookup::Miss(self.resolution()); + } let Some(state) = self.touch(id) else { return Lookup::Miss(self.resolution()); }; @@ -341,6 +409,19 @@ impl KeysetCache { state: Arc, resolution: Resolution, ) -> Arc { + self.insert_at(state, resolution, Instant::now()) + } + + /// [`insert`](Self::insert), at `now`. Every entry whose index key has + /// passed the end of its lifetime is evicted first, so expired key + /// material does not wait in memory for a lookup that may never come. + fn insert_at( + &mut self, + state: Arc, + resolution: Resolution, + now: Instant, + ) -> Arc { + self.evict_expired(now); // A name lookup is answered with whatever the name means now: when // a later lookup has already bound it — to this keyset or another — // the caller gets that keyset (held, since no binding outlives its @@ -492,7 +573,26 @@ impl KeysetCache { else { return; }; - if let Some(entry) = self.by_id.remove(&oldest) { + self.evict(oldest); + } + + /// Evict every entry whose index key is past the end of its lifetime at + /// `now`. + fn evict_expired(&mut self, now: Instant) { + let expired: Vec = self + .by_id + .iter() + .filter(|(_, entry)| !entry.state.retention.is_live_at(now)) + .map(|(id, _)| *id) + .collect(); + for id in expired { + self.evict(id); + } + } + + /// Evict the bounded entry for `oldest`, with the names bound to it. + fn evict(&mut self, id: Uuid) { + if let Some(entry) = self.by_id.remove(&id) { // The entry's place in the order outlives it as a watermark: // once it is gone there is nothing left to order an older answer // for this keyset against. It is taken whether or not the entry @@ -504,11 +604,7 @@ impl KeysetCache { if let Some(name) = entry.name { // A name that has since moved to another id keeps its // binding: only this id's binding goes with it. - if self - .by_name - .get(&name) - .is_some_and(|alias| alias.id == oldest) - { + if self.by_name.get(&name).is_some_and(|alias| alias.id == id) { let _ = self.by_name.remove(&name); } } @@ -641,6 +737,7 @@ mod tests { id: Uuid::from_u128(id), name: name.map(str::to_owned), prf: HmacSha256Prf::new(Protected::new([id as u8; 32])), + retention: Retention::UntilEvicted, }) } @@ -1444,4 +1541,286 @@ mod tests { "a wide window is fresh" ); } + + // ========================================================================= + // Key lifetime + // ========================================================================= + + fn expiring(id: u128, retention: Retention) -> Arc { + Arc::new(KeysetState { + id: Uuid::from_u128(id), + name: None, + prf: HmacSha256Prf::new(Protected::new([id as u8; 32])), + retention, + }) + } + + #[test] + fn a_key_lifetime_is_fixed_to_a_retention_when_it_is_loaded() { + let now = Instant::now(); + let hour = Duration::from_secs(3600); + assert_eq!( + Retention::of(KeyLifetime::UntilEvicted, now), + Retention::UntilEvicted + ); + assert_eq!(Retention::of(KeyLifetime::Settle, now), Retention::Never); + assert_eq!( + Retention::of(KeyLifetime::Ttl(Duration::ZERO), now), + Retention::Never, + "a zero TTL is a settle" + ); + assert_eq!( + Retention::of(KeyLifetime::Ttl(hour), now), + Retention::Until(now + hour) + ); + assert_eq!( + Retention::of(KeyLifetime::Ttl(Duration::MAX), now), + Retention::UntilEvicted, + "a TTL past the clock's range is kept until evicted" + ); + } + + #[test] + fn a_ttl_ends_exclusively() { + let now = Instant::now(); + let end = now + Duration::from_secs(1); + assert!(Retention::Until(end).is_live_at(now)); + assert!(!Retention::Until(end).is_live_at(end), "expired at its end"); + assert!(!Retention::Never.is_live_at(now)); + assert!(Retention::UntilEvicted.is_live_at(end)); + } + + /// A keyset whose TTL has ended is a miss, and is gone from the cache: + /// its PRF does not wait there for the next load. + #[test] + fn a_keyset_past_its_ttl_is_evicted_on_lookup() { + let mut cache = cache(4); + let now = Instant::now(); + let end = now + Duration::from_secs(60); + let loaded = expiring(1, Retention::Until(end)); + let watch = Arc::downgrade(&loaded); + let resolution = cache.resolution(); + let _ = cache.insert_at(loaded, resolution, now); + + assert!( + matches!( + cache.get_at(&id(1), end - Duration::from_nanos(1)), + Lookup::Hit(_) + ), + "within its TTL the keyset is cached" + ); + assert!( + matches!(cache.get_at(&id(1), end), Lookup::Miss(_)), + "at the end of its TTL the keyset is loaded again" + ); + assert_eq!(cache.len(), 0, "the expired entry is evicted"); + assert!(watch.upgrade().is_none(), "and its PRF is dropped"); + } + + /// An insert sweeps out every expired entry, not only the one it is + /// about: an entry nobody asks for again is not kept past its TTL. + #[test] + fn an_insert_evicts_every_expired_keyset() { + let mut cache = cache(4); + let now = Instant::now(); + let end = now + Duration::from_secs(60); + for (n, retention) in [(1, Retention::Until(end)), (2, Retention::UntilEvicted)] { + let resolution = cache.resolution(); + let _ = cache.insert_at(expiring(n, retention), resolution, now); + } + assert_eq!(cache.len(), 2); + + let resolution = cache.resolution(); + let _ = cache.insert_at(expiring(3, Retention::UntilEvicted), resolution, end); + + assert_eq!(cache.len(), 2, "keyset 1 expired, 2 and 3 remain"); + assert!(matches!(cache.get_at(&id(1), end), Lookup::Miss(_))); + assert!(matches!(cache.get_at(&id(2), end), Lookup::Hit(_))); + assert!(matches!(cache.get_at(&id(3), end), Lookup::Hit(_))); + } + + /// An expired keyset's name goes with it, as on eviction. + #[test] + fn an_expired_keyset_takes_its_name_with_it() { + let mut cache = cache(4); + cache.name_ttl = Duration::MAX; + let now = Instant::now(); + let end = now + Duration::from_secs(60); + let named = Arc::new(KeysetState { + id: Uuid::from_u128(1), + name: Some("acme".into()), + prf: HmacSha256Prf::new(Protected::new([1u8; 32])), + retention: Retention::Until(end), + }); + let resolution = cache.resolution(); + let _ = cache.insert_at(named, resolution, now); + assert!(matches!(cache.get_at(&name("acme"), now), Lookup::Hit(_))); + + assert!(matches!(cache.get_at(&name("acme"), end), Lookup::Miss(_))); + assert_eq!(cache.names(), 0); + assert!(matches!(cache.get_at(&name("acme"), end), Lookup::Miss(_))); + } + + mod through_the_cipher { + use super::*; + use std::borrow::Cow; + use std::sync::atomic::{AtomicUsize, Ordering}; + + use stack_kms::{ + DataKey, DataKeySource, DataKeyWithTag, FakeDataKeySource, GenerateKeyPayload, + IndexKey, IndexKeySource, LoadedIndexKey, RetrieveKeyPayload, UnverifiedContext, + }; + + use crate::{nonempty, Error}; + + /// A key backend that gives each keyset it loads a fixed lifetime, + /// and the default keyset its own. Keys are the fake's. + struct Lifetimed { + inner: FakeDataKeySource, + default: KeyLifetime, + selected: KeyLifetime, + loads: AtomicUsize, + } + + impl Lifetimed { + fn new(default: KeyLifetime, selected: KeyLifetime) -> Self { + Self { + inner: FakeDataKeySource::new(), + default, + selected, + loads: AtomicUsize::new(0), + } + } + + fn loads(&self) -> usize { + self.loads.load(Ordering::SeqCst) + } + } + + impl DataKeySource for Lifetimed { + async fn generate_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option>, + ) -> Result, stack_kms::Error> { + self.inner + .generate_keys(payloads, keyset_id, unverified_context) + .await + } + + async fn retrieve_keys( + &self, + payloads: Vec>, + keyset_id: Option, + unverified_context: Option<&UnverifiedContext>, + ) -> Result, stack_kms::Error> { + self.inner + .retrieve_keys(payloads, keyset_id, unverified_context) + .await + } + } + + impl IndexKeySource for Lifetimed { + async fn load_index_key( + &self, + keyset_id: Option, + ) -> Result<(Uuid, IndexKey), stack_kms::Error> { + self.inner.load_index_key(keyset_id).await + } + + async fn load_index_key_with_lifetime( + &self, + keyset_id: Option, + ) -> Result { + let _ = self.loads.fetch_add(1, Ordering::SeqCst); + let lifetime = match keyset_id { + None => self.default, + Some(_) => self.selected, + }; + let (id, key) = self.inner.load_index_key(keyset_id).await?; + Ok(LoadedIndexKey::new(id, key, lifetime)) + } + } + + /// The one test that proves key lifetime for every key backend: the + /// cipher, not the backend, obeys the lifetime. Two selections of + /// one keyset cost one load when the key may be cached, and two when + /// it may not; a key that may not be cached is gone once the handle + /// that used it drops. Under every lifetime the keyset still seals + /// and opens. + #[tokio::test] + async fn the_cipher_obeys_every_key_lifetime() { + let hour = Duration::from_secs(3600); + for (lifetime, cached) in [ + (KeyLifetime::UntilEvicted, true), + (KeyLifetime::Ttl(hour), true), + (KeyLifetime::Ttl(Duration::ZERO), false), + (KeyLifetime::Settle, false), + ] { + let cipher = StackCipher::builder() + .kms(Lifetimed::new(KeyLifetime::UntilEvicted, lifetime)) + .init() + .await + .unwrap(); + assert_eq!( + cipher.kms().loads(), + 1, + "{lifetime:?}: init loads the default" + ); + let tenant = Uuid::from_u128(42); + + let first = cipher.keyset(tenant).await.unwrap(); + let watch = Arc::downgrade(&first.state); + let sealed = first + .encrypt_tree("hello".to_string(), nonempty!("greeting")) + .await + .unwrap(); + drop(first); + + assert_eq!( + watch.upgrade().is_some(), + cached, + "{lifetime:?}: whether the cipher kept the index key" + ); + + let second = cipher.keyset(tenant).await.unwrap(); + let expected = if cached { 2 } else { 3 }; + assert_eq!( + cipher.kms().loads(), + expected, + "{lifetime:?}: a second selection reloads only an uncached key" + ); + let opened: String = second + .decrypt_tree(sealed, nonempty!("greeting")) + .await + .unwrap(); + assert_eq!(opened, "hello", "{lifetime:?}"); + } + } + + /// The cipher holds its default keyset for its whole life, so a + /// backend that will not let it keep that index key is refused at + /// init, not obeyed in part. + #[tokio::test] + async fn a_default_keyset_the_cipher_may_not_keep_is_refused_at_init() { + for lifetime in [ + KeyLifetime::Settle, + KeyLifetime::Ttl(Duration::from_secs(60)), + ] { + let refused = StackCipher::builder() + .kms(Lifetimed::new(lifetime, KeyLifetime::UntilEvicted)) + .init() + .await + .err(); + assert!( + matches!( + refused, + Some(Error::DefaultKeysetLifetime { keyset }) if keyset == Uuid::nil() + ), + "{lifetime:?}: {refused:?}" + ); + } + } + } } diff --git a/packages/stack-encrypt/src/target/core.rs b/packages/stack-encrypt/src/target/core.rs index cbc024d876..ef380e6068 100644 --- a/packages/stack-encrypt/src/target/core.rs +++ b/packages/stack-encrypt/src/target/core.rs @@ -150,12 +150,7 @@ fn collect_retrieve_requests( ) { match ciphertext { CipherText::Single(leaf) | CipherText::EmptySequence(leaf) | CipherText::EmptyMap(leaf) => { - out.push(Request::retrieve_under( - *leaf.iv(), - leaf.tag().to_vec(), - descriptor.clone(), - leaf.keyset_id(), - )); + out.push(Request::retrieve_leaf(leaf, descriptor.clone())); } CipherText::Sequence(items) => { for item in items { diff --git a/packages/stack-encrypt/src/target/pending.rs b/packages/stack-encrypt/src/target/pending.rs index 6b2867b457..7c22bf5d3d 100644 --- a/packages/stack-encrypt/src/target/pending.rs +++ b/packages/stack-encrypt/src/target/pending.rs @@ -17,7 +17,8 @@ use std::pin::Pin; use std::sync::Arc; use stack_kms::{ - DataKey, DataKeySource, GenerateKeyPayload, IndexKeySource, Iv, MaybeSend, RetrieveKeyPayload, + DataKey, DataKeySource, GenerateKeyPayload, IndexKeySource, Iv, KeyBackend, MaybeSend, + RetrieveKeyPayload, }; use uuid::Uuid; @@ -590,6 +591,7 @@ struct Retrieve { tag: Vec, descriptor: Descriptor, keyset_id: Uuid, + backend: KeyBackend, } /// Issue the batched ZeroKMS calls for `requests`: at most one @@ -616,11 +618,13 @@ async fn dispatch( tag, descriptor, keyset_id, + backend, } => retrieves.push(Retrieve { iv, tag, descriptor, keyset_id, + backend, }), } } @@ -636,6 +640,27 @@ async fn dispatch( .chain(retrieves.iter().map(|retrieve| &retrieve.descriptor)) .try_for_each(Descriptor::check)?; + // A leaf names the key backend that sealed it. One that names another + // backend than the cipher's is refused here, before any key call, as a + // foreign keyset is refused at construction: the leaf never reaches a + // backend that did not seal it. + let backend = cipher.kms().key_backend(); + if let Some(retrieve) = retrieves + .iter() + .find(|retrieve| retrieve.backend != backend) + { + return Err(Error::ForeignBackend { + expected: backend, + found: retrieve.backend, + }); + } + // Only ZeroKMS has a seal path in this build: a v1 leaf names ZeroKMS, + // so a key vendor backend must not write one. CIP-5306 adds the v2 seal + // path, which needs the keyset's key mode. + if !generates.is_empty() && backend != KeyBackend::ZeroKms { + return Err(Error::SealUnsupported { backend }); + } + // The retrieves and the keyset loads run beside each other, after the // generates. Both sides are built here, before any key has been through // the stack, and boxed: a future is built on the stack and moved with diff --git a/packages/stack-encrypt/src/target/request.rs b/packages/stack-encrypt/src/target/request.rs index 792929c8b4..69a52fc1a6 100644 --- a/packages/stack-encrypt/src/target/request.rs +++ b/packages/stack-encrypt/src/target/request.rs @@ -14,7 +14,7 @@ use super::context::AeadContext; use std::sync::Arc; -use stack_kms::{DataKey, DataKeyWithTag, Iv}; +use stack_kms::{DataKey, DataKeyWithTag, Iv, KeyBackend}; use uuid::Uuid; use crate::keyset::KeysetState; @@ -40,6 +40,9 @@ pub(super) enum RequestKind { tag: Vec, descriptor: Descriptor, keyset_id: Uuid, + /// The key backend the leaf names. The settle refuses a request + /// whose backend is not the cipher's, before any key call. + backend: KeyBackend, }, /// The loaded state of the keyset `keyset_id`, its index-key PRF: from /// the cipher's keyset cache, or loaded from ZeroKMS and cached on a @@ -91,6 +94,36 @@ impl Request { Self::retrieve_under(iv, tag, Descriptor::of(context.into()), keyset_id) } + /// Request re-derivation of the data key of `leaf`, under `context` — + /// which must be the one the key was generated under. + /// + /// The same request as [`retrieve_data_key`](Self::retrieve_data_key) + /// with the leaf's iv, tag and keyset id, and it also carries the key + /// backend the leaf names ([`SealedValue::key_backend`]). The settle + /// refuses the request with [`Error::ForeignBackend`] before any key + /// call when that is not the cipher's key backend. + /// [`retrieve_data_key`](Self::retrieve_data_key) always names ZeroKMS, + /// so prefer this constructor when a [`SealedValue`] is in hand. + /// + /// [`SealedValue`]: crate::SealedValue + /// [`SealedValue::key_backend`]: crate::SealedValue::key_backend + pub fn retrieve_leaf_key(leaf: &crate::SealedValue, context: impl Into) -> Self { + Self::retrieve_leaf(leaf, Descriptor::of(context.into())) + } + + /// [`retrieve_leaf_key`](Self::retrieve_leaf_key) over an already + /// rendered descriptor. Crate-internal, as + /// [`generate_under`](Self::generate_under). + pub(crate) fn retrieve_leaf(leaf: &crate::SealedValue, descriptor: Descriptor) -> Self { + Self(RequestKind::RetrieveDataKey { + iv: *leaf.iv(), + tag: leaf.tag().to_vec(), + descriptor, + keyset_id: leaf.keyset_id(), + backend: leaf.key_backend(), + }) + } + /// [`generate_data_key`](Self::generate_data_key) over a descriptor that /// has already been rendered. /// @@ -117,6 +150,7 @@ impl Request { tag, descriptor, keyset_id, + backend: KeyBackend::ZeroKms, }) } @@ -437,7 +471,9 @@ mod tests { tag, descriptor, keyset_id, + backend, } => { + assert_eq!(backend, stack_kms::KeyBackend::ZeroKms); assert_eq!(keyset_id, ks()); assert_eq!(iv, Iv::default()); assert_eq!(tag, vec![7, 8, 9]); @@ -604,6 +640,7 @@ mod tests { id: Uuid::from_u128(id), name: None, prf: vitaminc_hmac::HmacSha256Prf::new(vitaminc_protected::Protected::new([1; 32])), + retention: crate::keyset::Retention::UntilEvicted, })), } } diff --git a/packages/stack-encrypt/tests/common/mod.rs b/packages/stack-encrypt/tests/common/mod.rs index 9fe1842226..2d030ae05f 100644 --- a/packages/stack-encrypt/tests/common/mod.rs +++ b/packages/stack-encrypt/tests/common/mod.rs @@ -15,7 +15,7 @@ use std::sync::{Arc, Mutex}; use stack_encrypt::StackCipher; use stack_kms::{ DataKey, DataKeySource, DataKeyWithTag, FakeDataKeySource, GenerateKeyPayload, IdentifiedBy, - IndexKey, IndexKeySource, RetrieveKeyPayload, UnverifiedContext, + IndexKey, IndexKeySource, KeyBackend, RetrieveKeyPayload, UnverifiedContext, }; use uuid::Uuid; @@ -92,12 +92,23 @@ pub struct CountingSource { generate_calls: Arc, retrieve_calls: Arc, load_calls: Arc, + backend: KeyBackend, } impl CountingSource { pub fn new() -> Self { Self::over(FakeDataKeySource::new()) } + + /// The same source, reporting `backend` as its key backend. The keys + /// still come from the fake, so this stands in for a key vendor backend + /// only in what the cipher routes on. + pub fn with_backend(backend: KeyBackend) -> Self { + Self { + backend, + ..Self::new() + } + } } impl CountingSource { @@ -107,6 +118,7 @@ impl CountingSource { generate_calls: Arc::new(AtomicUsize::new(0)), retrieve_calls: Arc::new(AtomicUsize::new(0)), load_calls: Arc::new(AtomicUsize::new(0)), + backend: KeyBackend::ZeroKms, } } @@ -145,6 +157,10 @@ impl DataKeySource for CountingSource { .retrieve_keys(payloads, keyset_id, unverified_context) .await } + + fn key_backend(&self) -> KeyBackend { + self.backend + } } impl IndexKeySource for CountingSource { diff --git a/packages/stack-encrypt/tests/frozen_bytes.rs b/packages/stack-encrypt/tests/frozen_bytes.rs index 9cbcec7e71..06cc3a69b9 100644 --- a/packages/stack-encrypt/tests/frozen_bytes.rs +++ b/packages/stack-encrypt/tests/frozen_bytes.rs @@ -1,9 +1,11 @@ //! Byte-level pins for the frozen encodings stack-encrypt commits to across //! languages: //! -//! * the [`SealedValue`] leaf layout -//! (`version ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ local_ciphertext`) — the -//! storage +//! * the [`SealedValue`] leaf layouts — v1 +//! (`version ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ local_ciphertext`, which +//! ZeroKMS writes) and v2 +//! (`version ‖ backend ‖ mode ‖ keyset_id ‖ iv ‖ tag_len ‖ tag ‖ +//! local_ciphertext`, which a key vendor backend writes) — the storage //! format a database column holds, and //! * the index-term encodings (equality: raw 32 bytes; match: LE `u16` //! positions; ORE/OPE: raw CLLW ciphertext bytes). @@ -29,8 +31,9 @@ use stack_encrypt::sem::{ use stack_encrypt::target::legacy::EncryptInto; use stack_encrypt::{CipherText, Error, LeafBytesError, SealedValue, StackCipher}; use stack_kms::{ - DataKey, DataKeySource, DataKeyWithTag, FakeDataKeySource, GenerateKeyPayload, IdentifiedBy, - IndexKey, IndexKeySource, RetrieveKeyPayload, UnverifiedContext, + DataKey, DataKeySource, DataKeyWithTag, DeterministicSource, FakeDataKeySource, + GenerateKeyPayload, IdentifiedBy, IndexKey, IndexKeySource, KeyBackend, KeyMode, KeyVendor, + RetrieveKeyPayload, UnverifiedContext, }; use uuid::Uuid; @@ -46,6 +49,13 @@ fn hex(bytes: &[u8]) -> String { bytes.iter().map(|b| format!("{b:02x}")).collect() } +fn unhex(hex: &str) -> Vec { + (0..hex.len()) + .step_by(2) + .map(|i| u8::from_str_radix(&hex[i..i + 2], 16).expect("hex")) + .collect() +} + // ============================================================================= // SealedValue leaf // ============================================================================= @@ -98,11 +108,168 @@ fn sealed_value_from_bytes_inverts_to_bytes() { #[test] fn sealed_value_rejects_unknown_version() { let mut bytes = fixture_leaf().to_bytes(); - bytes[0] = 2; - assert!(matches!( - SealedValue::from_bytes(&bytes), - Err(LeafBytesError::UnknownVersion(2)) - )); + for version in [0, 3, u8::MAX] { + bytes[0] = version; + assert_eq!( + SealedValue::from_bytes(&bytes).map(|_| ()), + Err(LeafBytesError::UnknownVersion(version)) + ); + } +} + +/// A known-answer test for a real v1 leaf: these bytes were sealed once by +/// [`DeterministicSource`] (seed `[7; 32]`) under the context +/// `users/email`, and must open, unchanged, for as long as v1 is read. A +/// change to the v1 framing, the v1 leaf AAD or the v1 decoder fails here. +#[tokio::test] +async fn a_pinned_v1_leaf_still_opens() { + const V1_LEAF: &str = "010000000000000000000000000000000045608626b2b6c6de1bfc90dd87c420ac2000d702c5b01099cca8d52801ec7a9ee26928d0f8c71e120a2a62cc9b7f2e96579a01c9186cd1c3d5e396fe5a727d374ab5cddf8abae2b708de5ceed1925f75508b9c4b271f1bff53234b"; + let bytes = unhex(V1_LEAF); + let leaf = SealedValue::from_bytes(&bytes).expect("decode the pinned v1 leaf"); + assert_eq!(leaf.format_version(), SealedValue::FORMAT_VERSION); + assert_eq!(leaf.key_backend(), KeyBackend::ZeroKms); + assert_eq!(leaf.key_mode(), None); + assert_eq!( + hex(&leaf.to_bytes()), + V1_LEAF, + "v1 re-encodes byte for byte" + ); + + let cipher = StackCipher::builder() + .kms(DeterministicSource::new([7; 32])) + .init() + .await + .expect("build cipher"); + let plaintext: String = cipher + .decrypt(CipherText::Single(leaf), b"users/email".as_slice()) + .await + .expect("the pinned v1 leaf opens"); + assert_eq!(plaintext, "known answer"); +} + +// ============================================================================= +// SealedValue v2 leaf +// ============================================================================= + +/// [`fixture_leaf`]'s parts as a v2 leaf of `vendor` in `mode`. +fn fixture_v2_leaf(vendor: KeyVendor, mode: KeyMode) -> SealedValue { + let (keyset_id, iv, tag, ciphertext) = fixture_leaf().into_parts(); + SealedValue::from_vendor_parts(vendor, mode, keyset_id, iv, tag, ciphertext) + .expect("fixture tag fits the length field") +} + +#[test] +fn v2_sealed_value_layout_is_pinned() { + let bytes = fixture_v2_leaf(KeyVendor::Aws, KeyMode::Wrapped).to_bytes(); + + // version(02) ‖ backend(01 — AWS KMS) ‖ mode(01 — wrapped) ‖ then the + // v1 fields: keyset_id ‖ iv ‖ tag_len ‖ tag ‖ local_ciphertext. + assert_eq!( + hex(&bytes), + "0201016b65797365742d666978747572653136303132333435363738396162636465660300aabbccdeadbeef" + ); + // Past the two extra header bytes, v2 is the v1 layout. + assert_eq!(&bytes[3..], &fixture_leaf().to_bytes()[1..]); +} + +#[test] +fn v2_sealed_value_from_bytes_inverts_to_bytes_for_every_backend_and_mode() { + for vendor in KeyVendor::ALL { + for mode in KeyMode::ALL { + let original = fixture_v2_leaf(vendor, mode); + let bytes = original.to_bytes(); + assert_eq!(&bytes[..3], &[2, vendor.byte(), mode.byte()]); + let decoded = SealedValue::from_bytes(&bytes).expect("decode v2 leaf"); + assert_eq!(decoded.format_version(), SealedValue::VENDOR_FORMAT_VERSION); + assert_eq!(decoded.key_backend(), KeyBackend::Vendor(vendor)); + assert_eq!(decoded.key_mode(), Some(mode)); + assert_eq!(decoded.keyset_id(), original.keyset_id()); + assert_eq!(decoded.iv(), original.iv()); + assert_eq!(decoded.tag(), original.tag()); + assert_eq!(decoded.ciphertext(), original.ciphertext()); + assert_eq!(decoded.to_bytes(), bytes); + assert_eq!( + decoded.clone().to_bytes(), + bytes, + "a clone keeps the v2 header" + ); + } + } +} + +#[test] +fn v2_sealed_value_rejects_an_unknown_backend_or_mode() { + let bytes = fixture_v2_leaf(KeyVendor::Static, KeyMode::Derived).to_bytes(); + for byte in [0, 6, u8::MAX] { + let mut backend = bytes.clone(); + backend[1] = byte; + assert_eq!( + SealedValue::from_bytes(&backend).map(|_| ()), + Err(LeafBytesError::UnknownBackend(byte)) + ); + } + for byte in [0, 3, u8::MAX] { + let mut mode = bytes.clone(); + mode[2] = byte; + assert_eq!( + SealedValue::from_bytes(&mode).map(|_| ()), + Err(LeafBytesError::UnknownMode(byte)) + ); + } +} + +#[test] +fn v2_sealed_value_rejects_truncation() { + let bytes = fixture_v2_leaf(KeyVendor::Vault, KeyMode::Wrapped).to_bytes(); + let tag_end = 3 + 16 + 16 + 2 + 3; + for len in 0..tag_end { + assert_eq!( + SealedValue::from_bytes(&bytes[..len]).map(|_| ()), + Err(LeafBytesError::Truncated), + "prefix of {len} bytes must be rejected" + ); + } + assert!(SealedValue::from_bytes(&bytes[..tag_end]).is_ok()); +} + +/// Under derived data keys the tag is empty: the shortest v2 leaf is the +/// header and the fixed-width fields. +#[test] +fn v2_sealed_value_decodes_the_shortest_leaf_the_layout_allows() { + let leaf = SealedValue::from_vendor_parts( + KeyVendor::Static, + KeyMode::Derived, + Uuid::nil(), + [7; 16], + Vec::new(), + Vec::new(), + ) + .expect("empty tag fits"); + let bytes = leaf.to_bytes(); + assert_eq!(bytes.len(), 3 + 16 + 16 + 2); + let decoded = SealedValue::from_bytes(&bytes).expect("the fixed fields alone are a leaf"); + assert!(decoded.tag().is_empty()); + assert!(decoded.ciphertext().is_empty()); + assert_eq!( + SealedValue::from_bytes(&bytes[..bytes.len() - 1]).map(|_| ()), + Err(LeafBytesError::Truncated) + ); +} + +#[test] +fn v2_sealed_value_rejects_oversized_tag_on_construction() { + let result = SealedValue::from_vendor_parts( + KeyVendor::Aws, + KeyMode::Wrapped, + Uuid::nil(), + [0; 16], + vec![0; usize::from(u16::MAX) + 1], + Vec::new(), + ); + assert_eq!( + result.map(|_| ()), + Err(LeafBytesError::TagTooLong(usize::from(u16::MAX) + 1)) + ); } #[test] diff --git a/packages/stack-encrypt/tests/key_backend.rs b/packages/stack-encrypt/tests/key_backend.rs new file mode 100644 index 0000000000..ec6f462e63 --- /dev/null +++ b/packages/stack-encrypt/tests/key_backend.rs @@ -0,0 +1,208 @@ +//! Routing by key backend: a sealed leaf names the key backend that sealed +//! it (a v1 leaf names ZeroKMS, a v2 leaf its key vendor backend), and a +//! cipher refuses a leaf that names another backend than its own before +//! any key call, as it refuses a leaf from a foreign keyset. +//! +//! No key vendor backend exists yet, so the "vendor" cipher here is the +//! fake source reporting a vendor backend: what the cipher routes on, and +//! nothing else. + +mod common; + +use std::sync::atomic::Ordering; + +use common::CountingSource; +use stack_encrypt::kms::{KeyBackend, KeyMode, KeyVendor}; +use stack_encrypt::{nonempty, CipherText, Error, SealedValue, StackCipher, StackCipherText}; +use uuid::Uuid; + +const AAD: &[u8] = b"ctx"; + +async fn cipher_over( + source: CountingSource, +) -> ( + StackCipher, + std::sync::Arc, + std::sync::Arc, +) { + let (generates, retrieves) = source.counters(); + let cipher = StackCipher::builder() + .kms(source) + .init() + .await + .expect("build cipher"); + (cipher, generates, retrieves) +} + +fn v2_leaf(vendor: KeyVendor, keyset: Uuid) -> SealedValue { + SealedValue::from_vendor_parts( + vendor, + KeyMode::Wrapped, + keyset, + [3; 16], + vec![9; 8], + vec![0xDE, 0xAD], + ) + .expect("v2 leaf") +} + +fn v1_leaf(keyset: Uuid) -> SealedValue { + SealedValue::from_parts(keyset, [3; 16], vec![9; 8], vec![0xDE, 0xAD]).expect("v1 leaf") +} + +fn assert_foreign(result: Result, expected: KeyBackend, found: KeyBackend) { + match result { + Err(Error::ForeignBackend { + expected: e, + found: f, + }) => { + assert_eq!((e, f), (expected, found)); + } + other => panic!("expected ForeignBackend, got {other:?}"), + } +} + +/// A ZeroKMS cipher refuses a v2 leaf before any retrieve, through the +/// client and through a keyset cipher. +#[tokio::test] +async fn a_zerokms_cipher_refuses_a_v2_leaf_before_any_key_call() { + let (cipher, _, retrieves) = cipher_over(CountingSource::new()).await; + let keyset = cipher.default_keyset(); + let vendor = KeyBackend::Vendor(KeyVendor::Aws); + + let result = cipher + .decrypt::( + CipherText::Single(v2_leaf(KeyVendor::Aws, keyset.keyset_id())), + AAD, + ) + .await; + assert_foreign(result, KeyBackend::ZeroKms, vendor); + + let result = keyset + .decrypt_tree::( + CipherText::Single(v2_leaf(KeyVendor::Aws, keyset.keyset_id())), + AAD, + ) + .await; + assert_foreign(result, KeyBackend::ZeroKms, vendor); + + assert_eq!(retrieves.load(Ordering::SeqCst), 0, "no key call"); +} + +/// One v2 leaf in a tree of v1 leaves refuses the whole tree, before any +/// retrieve: the v1 leaves are not retrieved first. +#[tokio::test] +async fn one_v2_leaf_refuses_a_whole_tree_before_any_key_call() { + let (cipher, _, retrieves) = cipher_over(CountingSource::new()).await; + let keyset = cipher.default_keyset(); + let v1 = keyset + .encrypt_tree("first".to_string(), AAD) + .await + .expect("encrypt"); + let tree: StackCipherText = CipherText::Sequence(vec![ + v1, + CipherText::Single(v2_leaf(KeyVendor::Vault, keyset.keyset_id())), + ]); + + let result = cipher.decrypt::, _>(tree, AAD).await; + assert!( + matches!( + result, + Err(Error::ForeignBackend { + found: KeyBackend::Vendor(KeyVendor::Vault), + .. + }) + ), + "{result:?}" + ); + assert_eq!(retrieves.load(Ordering::SeqCst), 0, "no key call"); +} + +/// A cipher over a key vendor backend refuses a v1 leaf (which names +/// ZeroKMS) and a v2 leaf of another vendor, before any key call. A v2 leaf +/// of its own vendor passes the routing check and reaches the key source. +#[tokio::test] +async fn a_vendor_cipher_routes_only_its_own_leaves_to_its_key_source() { + let own = KeyBackend::Vendor(KeyVendor::Static); + let (cipher, _, retrieves) = cipher_over(CountingSource::with_backend(own)).await; + let keyset = cipher.default_keyset().keyset_id(); + + let result = cipher + .decrypt::(CipherText::Single(v1_leaf(keyset)), AAD) + .await; + assert_foreign(result, own, KeyBackend::ZeroKms); + + let result = cipher + .decrypt::(CipherText::Single(v2_leaf(KeyVendor::Gcp, keyset)), AAD) + .await; + assert_foreign(result, own, KeyBackend::Vendor(KeyVendor::Gcp)); + assert_eq!(retrieves.load(Ordering::SeqCst), 0, "no key call"); + + // The fake holds no key for this made-up leaf, so the retrieve fails; + // what matters is that the leaf reached the key source. + let result = cipher + .decrypt::(CipherText::Single(v2_leaf(KeyVendor::Static, keyset)), AAD) + .await; + assert!(matches!(result, Err(Error::Kms(_))), "{result:?}"); + assert_eq!(retrieves.load(Ordering::SeqCst), 1, "one key call"); +} + +/// This build seals v1 leaves only, and a v1 leaf names ZeroKMS: a cipher +/// over a key vendor backend refuses to seal, before any key call, rather +/// than write a leaf that names the wrong backend. +#[tokio::test] +async fn a_vendor_cipher_refuses_to_seal_before_any_key_call() { + let own = KeyBackend::Vendor(KeyVendor::Azure); + let (cipher, generates, _) = cipher_over(CountingSource::with_backend(own)).await; + + let result = cipher + .default_keyset() + .encrypt_tree("value".to_string(), AAD) + .await; + assert!( + matches!(result, Err(Error::SealUnsupported { backend }) if backend == own), + "{result:?}" + ); + assert_eq!(generates.load(Ordering::SeqCst), 0, "no key call"); +} + +/// A ZeroKMS cipher still seals v1 leaves, and they still open. +#[tokio::test] +async fn a_zerokms_cipher_still_writes_and_opens_v1() { + let (cipher, _, _) = cipher_over(CountingSource::new()).await; + let ct = cipher + .default_keyset() + .encrypt_tree("value".to_string(), AAD) + .await + .expect("encrypt"); + let CipherText::Single(leaf) = &ct else { + panic!("expected a Single leaf, got {ct:?}"); + }; + assert_eq!(leaf.format_version(), SealedValue::FORMAT_VERSION); + assert_eq!(leaf.key_backend(), KeyBackend::ZeroKms); + assert_eq!(leaf.key_mode(), None); + let pt: String = cipher.decrypt(ct, AAD).await.expect("decrypt"); + assert_eq!(pt, "value"); +} + +/// `Request::retrieve_leaf_key` carries the leaf's backend, so a +/// hand-built request for a v2 leaf is refused as a tree's would be. +#[tokio::test] +async fn a_hand_built_request_for_a_v2_leaf_is_refused() { + use stack_encrypt::target::{Pending, Request}; + + let (cipher, _, retrieves) = cipher_over(CountingSource::new()).await; + let keyset = cipher.default_keyset(); + let leaf = v2_leaf(KeyVendor::Aws, keyset.keyset_id()); + let pending: Pending<'_, (), _> = Pending::request( + &keyset, + vec![Request::retrieve_leaf_key(&leaf, nonempty!("ctx"))], + |_| Ok(()), + ); + let result = pending.await; + assert!( + matches!(result, Err(Error::ForeignBackend { .. })), + "{result:?}" + ); + assert_eq!(retrieves.load(Ordering::SeqCst), 0, "no key call"); +} diff --git a/packages/stack-kms/src/backend.rs b/packages/stack-kms/src/backend.rs new file mode 100644 index 0000000000..8efd91be3f --- /dev/null +++ b/packages/stack-kms/src/backend.rs @@ -0,0 +1,232 @@ +//! Which key backend a key source is, and the key mode of a key vendor +//! backend's keyset. +//! +//! A sealed leaf names its key backend so that a reader can route it, or +//! refuse it, before it asks for a key. A ZeroKMS leaf names its backend by +//! its format version alone (the v1 leaf). A leaf sealed by a key vendor +//! backend is a v2 leaf, and carries one byte for the vendor +//! ([`KeyVendor::byte`]) and one byte for the key mode ([`KeyMode::byte`]). +//! ADR-0010 has the layout. +//! +//! # Stable byte values +//! +//! The byte values are a storage commitment: a database column holds them. +//! A value is never reused and never renumbered. The value `0` is reserved +//! as invalid for both fields, so a zeroed header never decodes. +//! +//! | vendor | byte | +//! |----------------------|------| +//! | (invalid) | 0 | +//! | AWS KMS | 1 | +//! | Google Cloud KMS | 2 | +//! | Azure Key Vault | 3 | +//! | HashiCorp Vault | 4 | +//! | static key | 5 | +//! +//! | key mode | byte | +//! |----------------------|------| +//! | (invalid) | 0 | +//! | wrapped data key | 1 | +//! | derived data key | 2 | + +use std::fmt; + +/// The service a keyset's data keys and index key come from: ZeroKMS, or +/// one key vendor backend. +/// +/// [`DataKeySource::key_backend`](crate::DataKeySource::key_backend) +/// returns it, and a cipher compares it with the backend a sealed leaf +/// names before it asks for a key. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +pub enum KeyBackend { + /// ZeroKMS. Its leaves are v1 leaves, which have no backend byte. + ZeroKms, + /// A key vendor backend. Its leaves are v2 leaves. + Vendor(KeyVendor), +} + +impl KeyBackend { + /// A stable, lower-case name for logs and error payloads. + pub const fn name(self) -> &'static str { + match self { + Self::ZeroKms => "zerokms", + Self::Vendor(vendor) => vendor.name(), + } + } +} + +impl fmt::Display for KeyBackend { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(self.name()) + } +} + +/// A key vendor backend: a key backend that is not ZeroKMS. +/// +/// Each variant has a stable byte ([`byte`](Self::byte)), which a v2 sealed +/// leaf stores. See the module docs for the table. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[non_exhaustive] +#[repr(u8)] +pub enum KeyVendor { + /// AWS KMS. + Aws = 1, + /// Google Cloud KMS. + Gcp = 2, + /// Azure Key Vault. + Azure = 3, + /// HashiCorp Vault Transit. + Vault = 4, + /// A static key, held in memory. + Static = 5, +} + +impl KeyVendor { + /// Every vendor, in byte order. + pub const ALL: [Self; 5] = [Self::Aws, Self::Gcp, Self::Azure, Self::Vault, Self::Static]; + + /// The byte a v2 sealed leaf stores for this vendor. + pub const fn byte(self) -> u8 { + self as u8 + } + + /// The vendor a v2 leaf's backend byte names, or `None` for a byte that + /// names no vendor (`0` included). + pub const fn from_byte(byte: u8) -> Option { + match byte { + 1 => Some(Self::Aws), + 2 => Some(Self::Gcp), + 3 => Some(Self::Azure), + 4 => Some(Self::Vault), + 5 => Some(Self::Static), + _ => None, + } + } + + /// A stable, lower-case name. A key vendor backend uses it as the + /// `vendor` of its [`Error::Backend`](crate::Error::Backend) and + /// [`Error::ZeroKmsOnly`](crate::Error::ZeroKmsOnly). + pub const fn name(self) -> &'static str { + match self { + Self::Aws => "aws-kms", + Self::Gcp => "gcp-kms", + Self::Azure => "azure-key-vault", + Self::Vault => "vault-transit", + Self::Static => "static", + } + } +} + +impl fmt::Display for KeyVendor { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(self.name()) + } +} + +/// The key mode of a keyset in a key vendor backend (ADR-0009). +/// +/// Each variant has a stable byte ([`byte`](Self::byte)), which a v2 sealed +/// leaf stores. ZeroKMS has no key mode. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[repr(u8)] +pub enum KeyMode { + /// Wrapped data key: each value has its own data key, wrapped by the + /// vendor key. The leaf's tag holds the wrapped data key. + Wrapped = 1, + /// Derived data key: each data key is derived from the keyset's root + /// key. The leaf's tag is empty, and its iv is the derivation salt. + Derived = 2, +} + +impl KeyMode { + /// Both modes, in byte order. + pub const ALL: [Self; 2] = [Self::Wrapped, Self::Derived]; + + /// The byte a v2 sealed leaf stores for this mode. + pub const fn byte(self) -> u8 { + self as u8 + } + + /// The mode a v2 leaf's mode byte names, or `None` for a byte that + /// names no mode (`0` included). + pub const fn from_byte(byte: u8) -> Option { + match byte { + 1 => Some(Self::Wrapped), + 2 => Some(Self::Derived), + _ => None, + } + } + + /// A stable, lower-case name for logs and error payloads. + pub const fn name(self) -> &'static str { + match self { + Self::Wrapped => "wrapped", + Self::Derived => "derived", + } + } +} + +impl fmt::Display for KeyMode { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(self.name()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// The byte values are a storage commitment. This pin fails if one + /// moves. + #[test] + fn the_byte_values_are_pinned() { + let vendors: Vec<(u8, &str)> = KeyVendor::ALL + .iter() + .map(|vendor| (vendor.byte(), vendor.name())) + .collect(); + assert_eq!( + vendors, + [ + (1, "aws-kms"), + (2, "gcp-kms"), + (3, "azure-key-vault"), + (4, "vault-transit"), + (5, "static"), + ] + ); + let modes: Vec<(u8, &str)> = KeyMode::ALL + .iter() + .map(|mode| (mode.byte(), mode.name())) + .collect(); + assert_eq!(modes, [(1, "wrapped"), (2, "derived")]); + } + + #[test] + fn from_byte_inverts_byte_and_refuses_every_other_value() { + for byte in 0..=u8::MAX { + match KeyVendor::from_byte(byte) { + Some(vendor) => assert_eq!(vendor.byte(), byte), + None => assert!(!KeyVendor::ALL.iter().any(|v| v.byte() == byte), "{byte}"), + } + match KeyMode::from_byte(byte) { + Some(mode) => assert_eq!(mode.byte(), byte), + None => assert!(!KeyMode::ALL.iter().any(|m| m.byte() == byte), "{byte}"), + } + } + assert_eq!(KeyVendor::from_byte(0), None); + assert_eq!(KeyMode::from_byte(0), None); + } + + #[test] + fn names_display_as_themselves() { + assert_eq!(KeyBackend::ZeroKms.to_string(), "zerokms"); + assert_eq!(KeyBackend::ZeroKms.name(), "zerokms"); + for vendor in KeyVendor::ALL { + assert_eq!(vendor.to_string(), vendor.name()); + assert_eq!(KeyBackend::Vendor(vendor).to_string(), vendor.name()); + } + for mode in KeyMode::ALL { + assert_eq!(mode.to_string(), mode.name()); + } + } +} diff --git a/packages/stack-kms/src/errors.rs b/packages/stack-kms/src/errors.rs index 9287e6f18e..6f11bef6cc 100644 --- a/packages/stack-kms/src/errors.rs +++ b/packages/stack-kms/src/errors.rs @@ -340,6 +340,15 @@ mod codes { }, Error::InvalidEndpoint(_) => Error::InvalidEndpoint(InvalidEndpoint::Userinfo), Error::Unexpected(_) => Error::Unexpected("unexpected".into()), + Error::Backend { .. } => Error::Backend { + vendor: "test-vendor", + kind: BackendErrorKind::NotFound, + source: Box::new(std::io::Error::other("no key")), + }, + Error::ZeroKmsOnly { .. } => Error::ZeroKmsOnly { + vendor: "test-vendor", + feature: crate::payload::ZeroKmsFeature::Context, + }, ])); errors.extend(boxed(variants![ InvalidEndpoint::Parse(_) => InvalidEndpoint::Parse(url::ParseError::EmptyHost), @@ -500,6 +509,53 @@ mod codes { } } + /// A backend failure gives its vendor and kind, never the vendor's own + /// error, which stays behind `source()`. + #[test] + fn a_backend_failure_gives_the_vendor_and_kind_and_not_the_source() { + use serde_json::json; + for kind in [ + BackendErrorKind::NotFound, + BackendErrorKind::AccessDenied, + BackendErrorKind::Throttled, + BackendErrorKind::Unavailable, + BackendErrorKind::InvalidCiphertext, + ] { + let error = Error::Backend { + vendor: "test-vendor", + kind, + source: Box::new(std::io::Error::other("marker-detail")), + }; + assert_eq!( + serde_json::Value::Object(error.payload()), + json!({ "vendor": "test-vendor", "kind": format!("{kind:?}") }) + ); + assert_eq!(kind.to_string(), format!("{kind:?}")); + let shown = error.to_string(); + assert_eq!(shown, format!("Key backend test-vendor failed: {kind:?}")); + assert!(!shown.contains("marker-detail"), "{shown}"); + assert!(std::error::Error::source(&error) + .is_some_and(|source| source.to_string() == "marker-detail")); + } + } + + #[test] + fn a_zerokms_only_refusal_names_the_vendor_and_the_feature() { + use serde_json::json; + let error = Error::ZeroKmsOnly { + vendor: "test-vendor", + feature: crate::payload::ZeroKmsFeature::DecryptionPolicy, + }; + assert_eq!( + serde_json::Value::Object(error.payload()), + json!({ "vendor": "test-vendor", "feature": "decryption_policy" }) + ); + assert_eq!( + error.to_string(), + "Key backend test-vendor does not support the ZeroKMS feature decryption_policy" + ); + } + /// The builder's help names the variable that is wrong. #[cfg(feature = "http")] #[test] @@ -751,6 +807,75 @@ pub enum Error { #[error("Unexpected error: {0}")] #[diagnostic(code(stack_kms::unexpected))] Unexpected(String), + + /// A key vendor backend failed. `vendor` names the backend (for example + /// `"aws-kms"`) and `kind` says what failed, in terms that hold for + /// every vendor. The vendor's own error is behind `source()`; the + /// message carries only the vendor and the kind. + #[error("Key backend {vendor} failed: {kind}")] + #[diagnostic( + code(stack_kms::backend), + help("Read the `kind` field. NotFound and AccessDenied are configuration; Throttled and Unavailable can be retried; InvalidCiphertext means the stored key or value is damaged, or belongs to another key.") + )] + Backend { + vendor: &'static str, + kind: BackendErrorKind, + #[source] + source: Box, + }, + + /// A key vendor backend was asked for an input that only ZeroKMS can + /// honour (a key context, a decryption policy, an unverified context). + /// The backend refuses it rather than ignore it. + #[error("Key backend {vendor} does not support the ZeroKMS feature {feature}")] + #[diagnostic( + code(stack_kms::zerokms_only), + help("Lock contexts, decryption policies and unverified contexts need ZeroKMS. Use a ZeroKMS keyset, or remove the feature from the request.") + )] + ZeroKmsOnly { + vendor: &'static str, + feature: crate::payload::ZeroKmsFeature, + }, +} + +/// What failed in a key vendor backend, in terms that hold for every vendor. +/// +/// Each vendor client maps its own errors to one of these, so that a caller +/// can act on a failure without knowing the vendor. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[non_exhaustive] +pub enum BackendErrorKind { + /// The vendor key, or the key material it protects, does not exist. + NotFound, + /// The vendor refused the credential, or the credential has no grant + /// for this key or operation. + AccessDenied, + /// The vendor rate-limited the request. Retry later. + Throttled, + /// The vendor could not be reached, or did not answer. Retry later. + Unavailable, + /// The vendor could not unwrap the key material: it is damaged, or it + /// was wrapped by another key. + InvalidCiphertext, +} + +impl BackendErrorKind { + /// The kind as error payloads spell it. + pub const fn as_str(self) -> &'static str { + match self { + Self::NotFound => "NotFound", + Self::AccessDenied => "AccessDenied", + Self::Throttled => "Throttled", + Self::Unavailable => "Unavailable", + Self::InvalidCiphertext => "InvalidCiphertext", + } + } +} + +impl std::fmt::Display for BackendErrorKind { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(self.as_str()) + } } impl ErrorPayload for Error { @@ -760,6 +885,13 @@ impl ErrorPayload for Error { Self::RetrieveKey(error) => error.payload(), Self::LoadKeyset(error) => error.payload(), Self::Auth(error) => error.payload(), + Self::Backend { vendor, kind, .. } => { + payload([("vendor", (*vendor).into()), ("kind", kind.as_str().into())]) + } + Self::ZeroKmsOnly { vendor, feature } => payload([ + ("vendor", (*vendor).into()), + ("feature", feature.as_str().into()), + ]), Self::ConnectionInit(_) | Self::InvalidEndpoint(_) | Self::Unexpected(_) => { serde_json::Map::new() } diff --git a/packages/stack-kms/src/key.rs b/packages/stack-kms/src/key.rs index c919ba9a17..33657a739f 100644 --- a/packages/stack-kms/src/key.rs +++ b/packages/stack-kms/src/key.rs @@ -8,7 +8,7 @@ use serde::{Deserialize, Deserializer, Serialize, Serializer}; use sha2::{Digest, Sha256}; use std::ops::Deref; use uuid::Uuid; -use vitaminc::protected::{OpaqueDebug, TimingSafeEq}; +use vitaminc::protected::{Controlled, OpaqueDebug, Protected, TimingSafeEq}; use zeroize::{Zeroize, ZeroizeOnDrop, Zeroizing}; use zerokms_protocol::{DecryptionPolicy, ViturKeyMaterial}; @@ -226,6 +226,19 @@ impl IndexKey { Ok(Self(key)) } + /// Build the index key from raw key material that a key vendor backend + /// unwrapped or derived. + /// + /// The bytes must be the keyset's index key exactly as every writer and + /// every reader of that keyset obtains it: a different key builds terms + /// that match nothing. A ZeroKMS keyset never needs this; use + /// [`from_key_material`](Self::from_key_material). The input is + /// [`Protected`], so the caller's copy is wiped when it drops, and the + /// index key wipes its own copy on drop. + pub fn from_key_bytes(key: Protected) -> Self { + Self(*key.risky_ref()) + } + pub fn key(&self) -> &Key { &self.0 } @@ -460,6 +473,17 @@ mod tests { ); } + #[test] + fn from_key_bytes_keeps_the_bytes_it_is_given() { + let index_key = + IndexKey::from_key_bytes(vitaminc::protected::Protected::new([9u8; 32])); + assert_eq!(index_key.key(), &[9u8; 32]); + assert!( + !format!("{index_key:?}").contains("9"), + "the key must not print" + ); + } + #[test] fn from_key_material_is_deterministic() { let a = IndexKey::from_key_material(&kat_client_key(), &kat_material()).unwrap(); diff --git a/packages/stack-kms/src/key_source.rs b/packages/stack-kms/src/key_source.rs index 844301eadb..07eb78896c 100644 --- a/packages/stack-kms/src/key_source.rs +++ b/packages/stack-kms/src/key_source.rs @@ -1,7 +1,18 @@ -//! An abstraction over the ZeroKMS data-key operations that higher-level -//! encryption layers (e.g. `stack-encrypt`) depend on. +//! An abstraction over the data-key operations of a **key backend** that +//! higher-level encryption layers (e.g. `stack-encrypt`) depend on. //! -//! [`StackKms`](crate::StackKms) is the production implementation. Downstream +//! A key backend is ZeroKMS or one key vendor backend (AWS KMS, Azure Key +//! Vault, HashiCorp Vault Transit, Google Cloud KMS, or a static key). The +//! vendor-neutral part of each request is its descriptor, its iv and tag, +//! and its keyset id. The key context, the decryption policy and the +//! [`UnverifiedContext`] are ZeroKMS-only extensions: a key vendor backend +//! refuses them with [`Error::ZeroKmsOnly`] (see +//! [`refuse_zerokms_extensions`](crate::refuse_zerokms_extensions)) and +//! never ignores them. A key vendor backend reports its own failures as +//! [`Error::Backend`], and says how long its index key may be kept with +//! [`IndexKeySource::load_index_key_with_lifetime`]. +//! +//! [`StackKms`](crate::StackKms) is the ZeroKMS implementation. Downstream //! crates take a `K: DataKeySource` rather than a concrete client so their //! encrypt/decrypt logic can be unit-tested against a deterministic in-memory //! fake (see [`FakeDataKeySource`], enabled by the `test-support` feature) @@ -13,6 +24,7 @@ use std::future::Future; use uuid::Uuid; use zerokms_protocol::{IdentifiedBy, UnverifiedContext}; +use crate::backend::KeyBackend; use crate::errors::Error; use crate::key::{DataKey, DataKeyWithTag, IndexKey}; use crate::maybe_send::MaybeSend; @@ -46,6 +58,19 @@ pub trait DataKeySource { keyset_id: Option, unverified_context: Option<&UnverifiedContext>, ) -> impl Future, Error>> + MaybeSend; + + /// The key backend this source is. + /// + /// A cipher reads it before any key call. It refuses a sealed leaf that + /// names another key backend, as it refuses a leaf from a foreign + /// keyset, so a leaf never reaches a backend that did not seal it. The + /// default is [`KeyBackend::ZeroKms`], so every existing source keeps + /// the ZeroKMS behaviour. A key vendor backend overrides it to return + /// [`KeyBackend::Vendor`]. A wrapper that forwards to another source + /// must forward this method too. + fn key_backend(&self) -> KeyBackend { + KeyBackend::ZeroKms + } } /// The slice of ZeroKMS functionality required to *index* encrypted data: @@ -70,6 +95,77 @@ pub trait IndexKeySource { &self, keyset_id: Option, ) -> impl Future> + MaybeSend; + + /// Load the index key as [`load_index_key`](Self::load_index_key) does, + /// and say how long the caller may keep it ([`KeyLifetime`]). + /// + /// The cipher calls this method, not `load_index_key`, and its + /// per-keyset cache obeys the lifetime: the cipher enforces key + /// lifetime, so each backend does not have to. The default + /// implementation returns [`KeyLifetime::UntilEvicted`], which is the + /// ZeroKMS behaviour. A key vendor backend overrides it to return + /// [`KeyLifetime::Settle`] (wrapped data keys) or + /// [`KeyLifetime::Ttl`] (derived data keys). + fn load_index_key_with_lifetime( + &self, + keyset_id: Option, + ) -> impl Future> + MaybeSend { + let load = self.load_index_key(keyset_id); + async move { + let (keyset_id, key) = load.await?; + Ok(LoadedIndexKey { + keyset_id, + key, + lifetime: KeyLifetime::UntilEvicted, + }) + } + } +} + +/// How long a caller may keep an index key after its key backend returned +/// it. +/// +/// The unit of the shortest lifetime is the **settle**: one batched round +/// trip of a pending's key requests to the key backend. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash)] +#[non_exhaustive] +pub enum KeyLifetime { + /// Keep the key only for the settle that needed it. The cipher never + /// caches it, and loads it again for the next selection of the keyset. + /// Under wrapped data keys the vendor sees, and can refuse, each one. + Settle, + /// Keep the key for this long after it was loaded, then load it again. + /// A zero duration is the same as [`Settle`](Self::Settle). Under + /// derived data keys the vendor sees, and can refuse, each reload. + Ttl(std::time::Duration), + /// Keep the key until the cipher's bounded cache evicts it. This is the + /// ZeroKMS behaviour, and the default. + #[default] + UntilEvicted, +} + +/// An index key, the keyset it belongs to, and how long it may be kept: +/// what [`IndexKeySource::load_index_key_with_lifetime`] returns. +#[derive(Debug)] +#[non_exhaustive] +pub struct LoadedIndexKey { + /// The resolved keyset id. + pub keyset_id: Uuid, + /// The keyset's index key. + pub key: IndexKey, + /// How long the caller may keep `key`. + pub lifetime: KeyLifetime, +} + +impl LoadedIndexKey { + /// An index key for `keyset_id` that may be kept for `lifetime`. + pub fn new(keyset_id: Uuid, key: IndexKey, lifetime: KeyLifetime) -> Self { + Self { + keyset_id, + key, + lifetime, + } + } } impl DataKeySource for crate::StackKms @@ -722,6 +818,46 @@ mod tests { assert_eq!(dk.key.key(), retrieved.key()); } + /// The default lifetime is the ZeroKMS one, so a source that does not + /// say otherwise keeps today's caching. + #[tokio::test] + async fn a_source_that_names_no_lifetime_is_kept_until_evicted() { + let src = FakeDataKeySource::new(); + let (id, key) = src.load_index_key(None).await.unwrap(); + let loaded = src.load_index_key_with_lifetime(None).await.unwrap(); + assert_eq!(loaded.keyset_id, id); + assert_eq!(loaded.key.key(), key.key()); + assert_eq!(loaded.lifetime, KeyLifetime::UntilEvicted); + assert_eq!(KeyLifetime::default(), KeyLifetime::UntilEvicted); + } + + #[tokio::test] + async fn a_source_can_name_its_own_lifetime() { + struct SettleOnly; + impl IndexKeySource for SettleOnly { + async fn load_index_key( + &self, + _keyset_id: Option, + ) -> Result<(Uuid, IndexKey), Error> { + Ok(( + Uuid::nil(), + IndexKey::from_key_bytes(vitaminc::protected::Protected::new([1u8; 32])), + )) + } + + async fn load_index_key_with_lifetime( + &self, + keyset_id: Option, + ) -> Result { + let (id, key) = self.load_index_key(keyset_id).await?; + Ok(LoadedIndexKey::new(id, key, KeyLifetime::Settle)) + } + } + let loaded = SettleOnly.load_index_key_with_lifetime(None).await.unwrap(); + assert_eq!(loaded.lifetime, KeyLifetime::Settle); + assert_eq!(loaded.key.key(), &[1u8; 32]); + } + #[test] fn the_stub_is_send_and_sync() { fn assert_send_sync() {} diff --git a/packages/stack-kms/src/lib.rs b/packages/stack-kms/src/lib.rs index ee484f39e1..583f970fe3 100644 --- a/packages/stack-kms/src/lib.rs +++ b/packages/stack-kms/src/lib.rs @@ -76,6 +76,7 @@ assert_eq!(keys.len(), 1); #![cfg_attr(test, allow(clippy::panic))] #![cfg_attr(test, allow(unused_results))] +mod backend; #[cfg(feature = "http")] mod builder; mod client; @@ -120,7 +121,8 @@ pub use maybe_send::MaybeSend; // Errors pub use errors::{ - Error, GenerateKeyError, InvalidKeyMaterialError, LoadKeysetError, RetrieveKeyError, + BackendErrorKind, Error, GenerateKeyError, InvalidKeyMaterialError, LoadKeysetError, + RetrieveKeyError, }; /// The trait every error from this crate implements to hand over its /// structured fields, with the rule for what an error may contain, and the @@ -131,12 +133,15 @@ pub use stack_auth::{diagnostic, ErrorPayload}; // Key material pub use key::{ClientKey, DataKey, DataKeyWithTag, IndexKey, V1KeySet}; +// Key backends and key modes (the v2 leaf's header bytes) +pub use backend::{KeyBackend, KeyMode, KeyVendor}; + // Key source abstractions (production = `StackKms`; tests = `FakeDataKeySource`) #[cfg(feature = "test-support")] pub use key_source::DeterministicSource; #[cfg(feature = "test-support")] pub use key_source::FakeDataKeySource; -pub use key_source::{DataKeySource, IndexKeySource}; +pub use key_source::{DataKeySource, IndexKeySource, KeyLifetime, LoadedIndexKey}; // Key providers pub use key_provider::{ @@ -149,7 +154,10 @@ pub use secret_key::SecretKey; pub use stack_profile::ProfileStore; // Operation payloads -pub use payload::{GenerateKeyPayload, RetrieveKeyPayload}; +pub use payload::{ + refuse_zerokms_extensions, GenerateKeyPayload, RetrieveKeyPayload, ZeroKmsExtensions, + ZeroKmsFeature, +}; // Commonly needed re-exports from the protocol / crypto layers pub use recipher::key::{GenRandom, Iv}; diff --git a/packages/stack-kms/src/payload.rs b/packages/stack-kms/src/payload.rs index 0c4ada8225..54a3823ce8 100644 --- a/packages/stack-kms/src/payload.rs +++ b/packages/stack-kms/src/payload.rs @@ -1,8 +1,86 @@ use recipher::key::Iv; use std::borrow::Cow; -use zerokms_protocol::{Context, DecryptionPolicy, KeyId, RetrieveKeySpec}; +use zerokms_protocol::{Context, DecryptionPolicy, KeyId, RetrieveKeySpec, UnverifiedContext}; -/// The requirements for generating a data key from ZeroKMS. +/// An input on a data-key request that only ZeroKMS can honour. +/// +/// The vendor-neutral part of a request is its descriptor, its iv and tag, +/// and its keyset id. A key vendor backend works from these alone. The +/// inputs named here have no meaning there, so a key vendor backend refuses +/// a request that carries one ([`Error::ZeroKmsOnly`](crate::Error::ZeroKmsOnly)) +/// and never ignores it. Find them with [`ZeroKmsExtensions`], or refuse them +/// in one call with [`refuse_zerokms_extensions`]. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash)] +#[non_exhaustive] +pub enum ZeroKmsFeature { + /// A non-empty key [`Context`]: tags and identity claims bound into the + /// data key. + Context, + /// A [`DecryptionPolicy`] that ZeroKMS checks on each retrieve. + DecryptionPolicy, + /// A non-empty [`UnverifiedContext`] sent with the request. + UnverifiedContext, +} + +impl ZeroKmsFeature { + /// The feature as error payloads spell it. + pub const fn as_str(self) -> &'static str { + match self { + Self::Context => "context", + Self::DecryptionPolicy => "decryption_policy", + Self::UnverifiedContext => "unverified_context", + } + } +} + +impl std::fmt::Display for ZeroKmsFeature { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.write_str(self.as_str()) + } +} + +/// A data-key payload that can carry ZeroKMS-only inputs. +/// +/// Implemented by [`GenerateKeyPayload`] and [`RetrieveKeyPayload`]. A key +/// vendor backend calls [`refuse_zerokms_extensions`] (or this trait +/// directly) before it sends anything. +pub trait ZeroKmsExtensions { + /// The first ZeroKMS-only input this payload carries, if any. An empty + /// context is not an input: it binds nothing. + fn zerokms_feature(&self) -> Option; +} + +/// Refuse a request that carries a ZeroKMS-only input, for the key vendor +/// backend `vendor`. +/// +/// Returns [`Error::ZeroKmsOnly`](crate::Error::ZeroKmsOnly) for the first +/// such input: a payload's context or decryption policy, else a non-empty +/// `unverified_context`. A key vendor backend calls this first in each +/// [`DataKeySource`](crate::DataKeySource) method, so that a lock context +/// or a policy is refused, not silently dropped. +pub fn refuse_zerokms_extensions( + vendor: &'static str, + payloads: &[P], + unverified_context: Option<&UnverifiedContext>, +) -> Result<(), crate::Error> { + let feature = payloads + .iter() + .find_map(ZeroKmsExtensions::zerokms_feature) + .or_else(|| { + unverified_context + .filter(|context| !context.0.is_empty()) + .map(|_| ZeroKmsFeature::UnverifiedContext) + }); + match feature { + Some(feature) => Err(crate::Error::ZeroKmsOnly { vendor, feature }), + None => Ok(()), + } +} + +/// The requirements for generating a data key. +/// +/// [`descriptor`](Self::descriptor) is vendor-neutral. The context and the +/// decryption policy are ZeroKMS-only extensions ([`ZeroKmsFeature`]). #[derive(Clone)] pub struct GenerateKeyPayload<'a> { pub descriptor: &'a str, @@ -24,9 +102,38 @@ impl<'a> GenerateKeyPayload<'a> { self.decryption_policy = Some(policy); self } + + /// The descriptor the data key is bound to. + pub fn descriptor(&self) -> &'a str { + self.descriptor + } } -/// The requirements for retrieving a data key from ZeroKMS. +impl ZeroKmsExtensions for GenerateKeyPayload<'_> { + fn zerokms_feature(&self) -> Option { + feature_of(&self.context, self.decryption_policy.as_ref()) + } +} + +/// The ZeroKMS-only input a context and a policy carry, context first. +fn feature_of( + context: &[Context], + decryption_policy: Option<&DecryptionPolicy>, +) -> Option { + if !context.is_empty() { + Some(ZeroKmsFeature::Context) + } else if decryption_policy.is_some() { + Some(ZeroKmsFeature::DecryptionPolicy) + } else { + None + } +} + +/// The requirements for retrieving a data key. +/// +/// The iv, the descriptor and the tag are vendor-neutral (see +/// [`iv_bytes`](Self::iv_bytes)). The context and the decryption policy are +/// ZeroKMS-only extensions ([`ZeroKmsFeature`]). pub struct RetrieveKeyPayload<'a> { pub iv: KeyId, pub descriptor: &'a str, @@ -56,6 +163,17 @@ impl<'a> RetrieveKeyPayload<'a> { self.decryption_policy = Some(policy); self } + + /// The iv the data key was generated with, as plain bytes. + pub fn iv_bytes(&self) -> Iv { + *self.iv.as_ref() + } +} + +impl ZeroKmsExtensions for RetrieveKeyPayload<'_> { + fn zerokms_feature(&self) -> Option { + feature_of(&self.context, self.decryption_policy.as_ref()) + } } impl<'a> From> for RetrieveKeySpec<'a> { @@ -120,6 +238,122 @@ mod tests { } } + mod zerokms_extensions { + use super::*; + use crate::Error; + use std::collections::HashMap; + + fn tag() -> Vec { + vec![Context::Tag("tenant-1".into())] + } + + #[test] + fn a_plain_payload_carries_no_zerokms_feature() { + let generate = GenerateKeyPayload::new("d", Cow::Owned(vec![])); + let retrieve = RetrieveKeyPayload::new([1u8; 16], "d", b"tag"); + assert_eq!(generate.zerokms_feature(), None); + assert_eq!(retrieve.zerokms_feature(), None); + assert_eq!(generate.descriptor(), "d"); + assert_eq!(retrieve.iv_bytes(), [1u8; 16]); + } + + #[test] + fn a_context_or_a_policy_is_a_zerokms_feature() { + let ctx = tag(); + assert_eq!( + GenerateKeyPayload::new("d", Cow::Borrowed(&ctx)).zerokms_feature(), + Some(ZeroKmsFeature::Context) + ); + assert_eq!( + GenerateKeyPayload::new("d", Cow::Owned(vec![])) + .with_decryption_policy(policy()) + .zerokms_feature(), + Some(ZeroKmsFeature::DecryptionPolicy) + ); + assert_eq!( + RetrieveKeyPayload::new([0u8; 16], "d", b"t") + .with_context(Cow::Borrowed(&ctx)) + .zerokms_feature(), + Some(ZeroKmsFeature::Context) + ); + assert_eq!( + RetrieveKeyPayload::new([0u8; 16], "d", b"t") + .with_decryption_policy(policy()) + .zerokms_feature(), + Some(ZeroKmsFeature::DecryptionPolicy) + ); + } + + fn refused(result: Result<(), Error>) -> ZeroKmsFeature { + match result { + Err(Error::ZeroKmsOnly { vendor, feature }) => { + assert_eq!(vendor, "test-vendor"); + feature + } + other => panic!("expected a ZeroKmsOnly refusal, got {other:?}"), + } + } + + #[test] + fn a_vendor_refuses_every_zerokms_feature() { + let ctx = tag(); + let plain = [GenerateKeyPayload::new("d", Cow::Owned(vec![]))]; + assert!(refuse_zerokms_extensions("test-vendor", &plain, None).is_ok()); + + let with_context = [ + GenerateKeyPayload::new("d", Cow::Owned(vec![])), + GenerateKeyPayload::new("d", Cow::Borrowed(&ctx)), + ]; + assert_eq!( + refused(refuse_zerokms_extensions( + "test-vendor", + &with_context, + None + )), + ZeroKmsFeature::Context + ); + + let with_policy = + [RetrieveKeyPayload::new([0u8; 16], "d", b"t").with_decryption_policy(policy())]; + assert_eq!( + refused(refuse_zerokms_extensions("test-vendor", &with_policy, None)), + ZeroKmsFeature::DecryptionPolicy + ); + + let mut claims = HashMap::new(); + let _ = claims.insert("sub".to_string(), None); + let unverified = UnverifiedContext(claims); + assert_eq!( + refused(refuse_zerokms_extensions( + "test-vendor", + &plain, + Some(&unverified) + )), + ZeroKmsFeature::UnverifiedContext + ); + } + + /// An empty unverified context carries nothing, so it is no input. + #[test] + fn an_empty_unverified_context_is_not_refused() { + let plain = [GenerateKeyPayload::new("d", Cow::Owned(vec![]))]; + let empty = UnverifiedContext(HashMap::new()); + assert!(refuse_zerokms_extensions("test-vendor", &plain, Some(&empty)).is_ok()); + } + + #[test] + fn every_feature_keeps_its_spelling() { + for (feature, text) in [ + (ZeroKmsFeature::Context, "context"), + (ZeroKmsFeature::DecryptionPolicy, "decryption_policy"), + (ZeroKmsFeature::UnverifiedContext, "unverified_context"), + ] { + assert_eq!(feature.as_str(), text); + assert_eq!(feature.to_string(), text); + } + } + } + #[test] fn generate_key_payload_with_decryption_policy_sets_the_policy() { let payload = GenerateKeyPayload::new("d", Cow::Owned(vec![]));