feat: KMS-enforced launch tickets

docs/deployment.md

@@ -561,6 +561,10 @@ curl http://<new-kms>:9203/finish
 
 > **Admin authentication.** The KMS *admin* RPCs (for example `ClearImageCache`) are served on a dedicated `[core.admin]` listener behind the shared HTTP authenticator, just like the VMM and gateway: set `[core.admin] enabled = true` with an `auth_token` (or the `DSTACK_KMS_ADMIN_TOKEN` / `ADMIN_API_TOKEN` env vars), and clients send `Authorization: Bearer <token>` or `X-Admin-Token`. Enabled with neither `auth_token` nor `htpasswd_file` (and `insecure_no_auth = false`) fails closed — the KMS refuses to start. See [dstack-kms admin authentication](../dstack/kms/README.md#admin-api-authentication).
 >
+> **Source pinning.** Before sending anything secret, the new KMS requires the source's RPC certificate, at the TLS handshake, to be a `kms:rpc` leaf signed by the current root CA its own auth API reports (`rootCaPubkey` in `GET /`), so `source_url` must be `https://`. A source without the registered roots never sees the launch ticket or `source_token`; onboarding fails if the auth API reports no root CA key. Right after a root epoch is activated, a source still serving the RPC certificate of the previous one fails the handshake until it restarts; onboarding attempts do not trigger that restart, so restart the source.
+>
+> **Launch ticket.** When the KMS contract requires one (`setRequireKmsLaunchTicket`), pass this instance's ticket as `launch_ticket` in the onboarding request; the source checks its hash against `kmsLaunchTickets()` before handing over the keys. Remove the ticket from the contract once the instance has joined. See [launch tickets](../dstack/kms/README.md#launch-tickets).
+>
 > **Key handover.** It is served as `Admin.GetKmsKey` on the `[core.admin]` listener, which needs `[core.admin.tls]` so it can verify the onboarding KMS's RA-TLS certificate. Pass the source admin URL (`https://`) and its token (`source_token`) in the onboarding request; the token is required unless the source enables the public handover. The legacy public `KMS.GetKmsKey` is off by default; set `core.onboard.public_key_handover = true` on the source only when an onboarding KMS cannot use the admin listener, such as a 0.5.x KMS or an onboarding request without `source_token`.
 
 #### Rotating KMS Root Keys

docs/security/security-best-practices.md

@@ -52,6 +52,8 @@ Because `launch_token_hash` is public, a guessable token can be recovered offlin
 
 Also understand the protection boundary of this mechanism: the guest verifies the token before any keys are provisioned, which means the token must reach the guest through `user_config` — a channel the host can read. The requirement therefore stops parties who only know the public app-compose.json from launching the app, but once a host has hosted a deployment it learns the token and can later relaunch instances of that compose with substituted encrypted envs. Mitigations: generate a fresh token per deployment and remove stale compose hashes from the on-chain whitelist; if the token must stay secret from the host, use the app-layer `APP_LAUNCH_TOKEN` encrypted-env pattern above instead (its check necessarily runs after key provisioning).
 
+A KMS-level launch ticket closes that gap for apps governed by `DstackApp`: the contract owner requires a ticket (`setRequireLaunchTicket`) and allows the hashes of the tickets they issue (`addLaunchTicket`), and the KMS refuses keys and certificates to a CVM without an allowed ticket. The check runs in the KMS, so it does not depend on the guest OS, and a ticket is revoked on chain without changing the compose hash. It is orthogonal to `requirements.launch_token_hash`. The host still holds the ticket, so issue one per deployment and remove it when the host stops serving the app. See [launch tickets](../../dstack/kms/README.md#launch-tickets) and [vmm-cli](../vmm-cli-user-guide.md#launch-tickets).
+
 If you use dstack-vmm's built-in UI, the prelaunch script has already been automatically filled in for you:
 
 ![Prelaunch Script](../assets/prelaunch-script.png)

docs/vmm-cli-user-guide.md

@@ -293,6 +293,27 @@ Deploy your application with the compose file:
 - `--kms-url`: KMS server URL
 - `--gateway-url`: Gateway server URL
 - `--stopped`: Create VM in stopped state (requires dstack-vmm >= 0.5.4)
+- `--launch-ticket-file`: File holding the launch ticket the guest presents to the KMS (see below)
+
+#### Launch Tickets
+
+When the app's `DstackApp` contract requires a launch ticket, the KMS releases
+keys only to a CVM presenting a ticket whose hash the contract allows. Create a
+fresh ticket per deployment, allow its hash on chain, and deploy with it:
+
+```bash
+./vmm-cli.py launch-ticket new ./prod.ticket   # writes the ticket (mode 0600), prints its hash
+# addLaunchTicket(<hash>, "prod") on the DstackApp contract, then:
+./vmm-cli.py deploy ... --launch-ticket-file ./prod.ticket
+```
+
+`launch-ticket hash FILE` prints the hash of an existing ticket, for
+`addLaunchTicket` or, for a KMS onboarding ticket, `addKmsLaunchTicket`. `update
+--launch-ticket-file FILE` replaces the ticket of a VM and `update
+--no-launch-ticket` removes it. The VMM keeps the ticket and writes it to the
+guest's sys-config at every boot, but never returns it. The host holds the
+ticket, so remove its hash from the contract once that host stops serving the
+app. See [launch tickets](../dstack/kms/README.md#launch-tickets).
 
 #### Port Mapping
 

dstack/cert-client/src/lib.rs

@@ -6,7 +6,7 @@ use std::sync::{Arc, Mutex};
 
 use anyhow::{bail, Context, Result};
 use dstack_kms_rpc::{kms_client::KmsClient, AppKeyResponse, GetAppKeyRequest, SignCertRequest};
-use dstack_types::{AppKeys, KeyProvider};
+use dstack_types::{AppKeys, KeyProvider, SysConfig};
 use ra_rpc::client::{CertInfo, RaClient, RaClientConfig};
 use ra_tls::{
     attestation::{AppInfo, AttestationVerifier, VersionedAttestation},
@@ -20,6 +20,7 @@ pub enum CertRequestClient {
     Kms {
         client: KmsClient<RaClient>,
         vm_config: String,
+        launch_ticket: String,
     },
 }
 
@@ -158,13 +159,18 @@ impl CertRequestClient {
                     .context("Failed to sign certificate")?;
                 Ok(vec![cert.pem(), ca.pem_cert.clone()])
             }
-            CertRequestClient::Kms { client, vm_config } => {
+            CertRequestClient::Kms {
+                client,
+                vm_config,
+                launch_ticket,
+            } => {
                 let response = client
                     .sign_cert(SignCertRequest {
                         api_version: 2,
                         csr: csr.to_vec(),
                         signature: signature.to_vec(),
                         vm_config: vm_config.clone(),
+                        launch_ticket: launch_ticket.clone(),
                     })
                     .await?;
                 Ok(response.certificate_chain)
@@ -178,7 +184,12 @@ impl CertRequestClient {
         root_epoch: Option<u32>,
         app_epoch: Option<u32>,
     ) -> Result<AppKeyResponse> {
-        let CertRequestClient::Kms { client, vm_config } = self else {
+        let CertRequestClient::Kms {
+            client,
+            vm_config,
+            launch_ticket,
+        } = self
+        else {
             bail!("only KMS-provided keys have older root epochs");
         };
         let response = client
@@ -187,6 +198,7 @@ impl CertRequestClient {
                 vm_config: vm_config.clone(),
                 root_epoch,
                 app_epoch,
+                launch_ticket: launch_ticket.clone(),
             })
             .await?;
         if root_epoch.is_some_and(|epoch| epoch != response.root_epoch) {
@@ -208,7 +220,7 @@ impl CertRequestClient {
     pub async fn create(
         keys: &AppKeys,
         attestation_verifier: Arc<AttestationVerifier>,
-        vm_config: String,
+        sys_config: &SysConfig,
     ) -> Result<CertRequestClient> {
         match &keys.key_provider {
             KeyProvider::None { key }
@@ -231,7 +243,11 @@ impl CertRequestClient {
                     client_cert.key_pem,
                     attestation_verifier,
                 )?;
-                Ok(CertRequestClient::Kms { client, vm_config })
+                Ok(CertRequestClient::Kms {
+                    client,
+                    vm_config: sys_config.vm_config.clone(),
+                    launch_ticket: sys_config.launch_ticket.clone().unwrap_or_default(),
+                })
             }
         }
     }
@@ -643,9 +659,7 @@ mod tests {
         };
         let request = || GetAppKeyRequest {
             api_version: 1,
-            vm_config: String::new(),
-            root_epoch: None,
-            app_epoch: None,
+            ..Default::default()
         };
         let result = request_app_keys(
             url.clone(),
@@ -665,9 +679,8 @@ mod tests {
         for requested_root_epoch in [None, Some(0), Some(1)] {
             let request = || GetAppKeyRequest {
                 api_version: 1,
-                vm_config: String::new(),
                 root_epoch: requested_root_epoch,
-                app_epoch: None,
+                ..Default::default()
             };
             let selected = AppKeyResponse {
                 root_epoch: requested_root_epoch.unwrap_or(1),
@@ -735,6 +748,7 @@ mod tests {
             )
             .unwrap(),
             vm_config: "selector-test".into(),
+            launch_ticket: String::new(),
         };
         for root_epoch in [None, Some(0), Some(1)] {
             for app_epoch in [None, Some(0), Some(2)] {

dstack/crates/dstack-auth/src/main.rs

@@ -75,6 +75,9 @@ struct BootInfo {
     /// KMS that predates root epochs, and from KMS checks made while onboarding.
     root_epoch: Option<usize>,
     root_epoch_hash: Option<String>,
+    /// hash of the launch ticket the attested subject presented; empty when it
+    /// presented none, and from the legacy routes.
+    launch_ticket_hash: String,
 }
 
 #[derive(Serialize)]
@@ -201,6 +204,10 @@ struct KmsRules {
     mr_aggregated: Vec<String>,
     devices: Vec<String>,
     allow_any_device: bool,
+    /// when set, a KMS importing the root keys must present one of these
+    /// launch ticket hashes; an empty list denies every import. Absent: the
+    /// import needs no ticket.
+    launch_tickets: Option<Vec<String>>,
 }
 
 #[derive(Deserialize, Default)]
@@ -210,6 +217,9 @@ struct AppRules {
     compose_hashes: Vec<String>,
     devices: Vec<String>,
     allow_any_device: bool,
+    /// when set, the app must present one of these launch ticket hashes; an
+    /// empty list denies everything. Absent: no ticket is needed.
+    launch_tickets: Option<Vec<String>>,
 }
 
 /// normalize a hex string for comparison: trim, drop a `0x`/`0X` prefix,
@@ -230,6 +240,13 @@ fn contains(list: &[String], value: &str) -> bool {
     list.iter().any(|x| norm(x) == v)
 }
 
+/// No allowlist admits any ticket; an empty one admits none.
+fn launch_ticket_ok(tickets: &Option<Vec<String>>, ticket_hash: &str) -> bool {
+    tickets
+        .as_ref()
+        .is_none_or(|tickets| !ticket_hash.is_empty() && contains(tickets, ticket_hash))
+}
+
 /// An empty `devices` list allows no device; any device needs `allowAnyDevice`.
 fn device_ok(allow_any: bool, devices: &[String], device_id: &str) -> bool {
     allow_any || contains(devices, device_id)
@@ -295,6 +312,12 @@ fn tcb_denial(info: &BootInfo, al: &Allowlist) -> Option<&'static str> {
 }
 
 fn app_policy(info: &BootInfo, al: &Allowlist) -> BootResponse {
+    if al
+        .app(&info.app_id)
+        .is_some_and(|app| !launch_ticket_ok(&app.launch_tickets, &info.launch_ticket_hash))
+    {
+        return deny(al, "invalid launch ticket");
+    }
     if let Some(reason) = tcb_denial(info, al) {
         return deny(al, reason);
     }
@@ -353,6 +376,10 @@ fn check_app(info: &BootInfo, al: &Allowlist) -> BootResponse {
 }
 
 fn check_kms(info: &BootInfo, al: &Allowlist) -> BootResponse {
+    // legacy callers present no ticket and cannot say which operation they authorize
+    if al.kms.launch_tickets.is_some() {
+        return deny(al, "invalid launch ticket");
+    }
     let result = kms_policy(info, al);
     if result.is_allowed {
         allow(info, al)
@@ -395,6 +422,11 @@ fn authorize(
     );
     result.rejection_reason = if request.operation.is_kms() != is_kms {
         "invalid authorization operation".into()
+    } else if request.operation == Operation::KmsImportKeys
+        && !launch_ticket_ok(&al.kms.launch_tickets, &info.launch_ticket_hash)
+    {
+        // only the import hands a joining KMS the root keys
+        "invalid launch ticket".into()
     } else if current.is_none() && request.operation != Operation::KmsBootstrap {
         "root epoch not registered".into()
     } else if request
@@ -506,6 +538,8 @@ fn load(path: &PathBuf) -> Allowlist {
 
 #[post("/bootAuth/app", data = "<info>")]
 fn boot_app(info: Json<BootInfo>, cli: &State<Cli>) -> Json<BootResponse> {
+    let mut info = info.into_inner();
+    info.launch_ticket_hash.clear();
     let r = check_app(&info, &load(&cli.config));
     rocket::info!(
         "bootAuth/app app={} compose={} os={} device={} tcb={} -> allowed={} ({})",
@@ -555,6 +589,10 @@ fn info_json(al: &Allowlist) -> Result<serde_json::Value> {
         "appImplementation": al.app_implementation,
     });
     if let Some(epoch) = al.current_root_epoch()? {
+        // the root CA key of the current epoch signs every serving KMS's RPC
+        // certificate
+        info["rootCaPubkey"] =
+            format!("0x{}", norm(&al.root_epochs[epoch.number].ca_pubkey)).into();
         info["rootEpoch"] = epoch.number.into();
         info["rootEpochHash"] = epoch.hash.into();
     }
@@ -1185,6 +1223,74 @@ mod tests {
         assert_eq!(serve.rejection_reason, "requested root epoch is not active");
     }
 
+    #[test]
+    fn launch_tickets_are_required_only_when_configured() {
+        let mut al: Allowlist = serde_json::from_value(json!({
+            "kmsContractAddr": "0x1111111111111111111111111111111111111111",
+            "rootEpochs": [{"caPubkey": "aa", "k256Pubkey": "bb"}, {"caPubkey": "cc", "k256Pubkey": "dd"}],
+            "osImages": ["aa"],
+            "kms": {"mrAggregated": ["bb"], "allowAnyDevice": true, "launchTickets": ["0xEE"]},
+            "apps": {"0x2222222222222222222222222222222222222222": {
+                "composeHashes": ["cc"], "allowAnyDevice": true, "launchTickets": ["0xFF"]
+            }}
+        }))
+        .unwrap();
+        let ask = |al: &Allowlist, is_kms: bool, operation: &str, ticket: Option<&str>| {
+            let mut request = if is_kms {
+                json!({"operation": operation, "appId": "11".repeat(20), "mrAggregated": "bb"})
+            } else {
+                json!({"operation": operation, "appId": "22".repeat(20), "composeHash": "cc", "osImageHash": "aa"})
+            };
+            if let Some(ticket) = ticket {
+                request["launchTicketHash"] = ticket.into();
+            }
+            let request: AuthorizationRequest = serde_json::from_value(request).unwrap();
+            authorize(&request, is_kms, al).unwrap()
+        };
+        for operation in ["app_get_key", "app_get_cert"] {
+            assert!(ask(&al, false, operation, Some("ff")).authorized);
+            assert_eq!(
+                ask(&al, false, operation, Some("ee")).rejection_reason,
+                "invalid launch ticket"
+            );
+            assert!(!ask(&al, false, operation, None).authorized);
+        }
+        assert!(ask(&al, true, "kms_import_keys", Some("0xee")).authorized);
+        assert_eq!(
+            ask(&al, true, "kms_import_keys", None).rejection_reason,
+            "invalid launch ticket"
+        );
+        // Only the import hands out the root keys.
+        assert!(ask(&al, true, "kms_serve", None).authorized);
+        // Legacy callers cannot present a ticket.
+        let legacy = boot("0x2222222222222222222222222222222222222222", "cc", "aa");
+        assert!(!check_app(&legacy, &al).is_allowed);
+        assert!(
+            !check_kms(
+                &BootInfo {
+                    mr_aggregated: "bb".into(),
+                    ..Default::default()
+                },
+                &al
+            )
+            .is_allowed
+        );
+
+        // An empty allowlist admits no ticket; none configured admits any.
+        al.kms.launch_tickets = Some(vec![]);
+        assert!(!ask(&al, true, "kms_import_keys", Some("ee")).authorized);
+        al.kms.launch_tickets = None;
+        assert!(ask(&al, true, "kms_import_keys", None).authorized);
+
+        assert_eq!(info_json(&al).unwrap()["rootCaPubkey"], "0xaa");
+        al.current_root_epoch = 1;
+        assert_eq!(info_json(&al).unwrap()["rootCaPubkey"], "0xcc");
+        assert!(info_json(&Allowlist::default())
+            .unwrap()
+            .get("rootCaPubkey")
+            .is_none());
+    }
+
     #[test]
     fn allows_only_a_kms_serving_the_current_root_epoch() {
         // Shared with the DstackKms contract tests.

dstack/dstack-types/src/authorization.rs

@@ -25,3 +25,47 @@ impl AuthorizationOperation {
         )
     }
 }
+
+/// The `launchTicketHash` of `ticket`: `sha256("dstack-launch-ticket/v1:" || ticket)`.
+///
+/// Shell equivalent: `printf 'dstack-launch-ticket/v1:%s' "$TICKET" | sha256sum`.
+pub fn launch_ticket_hash(ticket: &str) -> [u8; 32] {
+    crate::sha256(format!("dstack-launch-ticket/v1:{ticket}").as_bytes())
+}
+
+/// Shortest launch ticket a KMS accepts. Its hash is public, so a short ticket
+/// can be recovered offline; length alone cannot prove entropy, so generate
+/// tickets randomly.
+pub const LAUNCH_TICKET_MIN_LEN: usize = 32;
+/// Longest launch ticket a KMS accepts; sys-config must stay small.
+pub const LAUNCH_TICKET_MAX_LEN: usize = 1024;
+
+/// Refuse a ticket the KMS would refuse. Surrounding whitespace is refused
+/// rather than trimmed: it changes the hash, so a pasted newline would
+/// otherwise fail as an unknown ticket.
+pub fn check_launch_ticket(ticket: &str) -> Result<(), String> {
+    if !(LAUNCH_TICKET_MIN_LEN..=LAUNCH_TICKET_MAX_LEN).contains(&ticket.len()) {
+        return Err(format!(
+            "launch ticket must be {LAUNCH_TICKET_MIN_LEN} to {LAUNCH_TICKET_MAX_LEN} bytes, got {}",
+            ticket.len()
+        ));
+    }
+    if ticket.trim() != ticket {
+        return Err("launch ticket has surrounding whitespace".into());
+    }
+    Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    /// Shared with vmm-cli's tests, which compute the hash the contracts allow.
+    #[test]
+    fn launch_ticket_hash_is_domain_separated() {
+        assert_eq!(
+            hex::encode(launch_ticket_hash(&"t".repeat(32))),
+            "8185a84114cfd5c0fb70e2233dee133cfa6467b58537ece8b27137828baabecc"
+        );
+    }
+}

dstack/dstack-types/src/lib.rs

@@ -1303,6 +1303,12 @@ pub struct SysConfig {
     /// switches in app-compose, which a token here cannot widen.
     #[serde(default, skip_serializing_if = "Option::is_none")]
     pub guest_agent_api: Option<GuestAgentApiConfig>,
+    /// Launch ticket the guest presents to the KMS, which checks its hash
+    /// against the app's on-chain allowlist before releasing keys. A secret
+    /// the host holds: unlike `vm_config`, it is never embedded in an
+    /// attestation or returned by the guest agent.
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub launch_ticket: Option<String>,
     // JSON serialized VmConfig
     pub vm_config: String,
 }