Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 77 additions & 7 deletions docs/app-key-epochs.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,21 @@ keys or other apps. This extends SealedEnv (#1528); the PR dependency order is

## Authorization and rotation

`checkAppAuthorization` returns `currentAppEpoch` together with the current root
`authorizeApp` returns `currentAppEpoch` together with the current root
epoch and its chain hash. `requestedRootEpoch` and `requestedAppEpoch` are
independent `OptionalEpoch` values: omitted selects current, explicit zero selects
independent `OptionalUint256` values: omitted selects current, explicit zero selects
historical epoch zero. Future epochs are denied through `rejectionReason`.
`checkKmsAuthorization` remains separate and uses app epoch zero.
`authorizeKms` remains separate and uses app epoch zero.

`DstackApp.currentAppEpoch` starts at zero. Its owner calls
`advanceAppEpoch(expectedCurrentEpoch)` to increment it. The expected value
prevents a stale or retried transaction from advancing twice. Epochs cannot be
rolled back and are limited to uint32. `AdvanceAppEpoch` in `Manage.s.sol` exposes
this operation; `GetAppEpoch` in `Query.s.sol` reads it. Upgrade the app policy
implementation before advancing, and upgrade the KMS/auth backend and guests
before using nonzero epochs. Legacy policies without the ERC-165 `IAppEpoch`
capability remain at zero; failures of an advertised getter propagate.
before using nonzero epochs. The ERC-165 `IAppEpochAuth` capability combines
`currentAppEpoch()` and `authorizeApp(...)` in one interface. Legacy policies
without this capability remain at zero; failures of an advertised getter propagate.

The Ethereum auth backends query the policy through the KMS contract. The simple
backend reads `apps[appId].currentAppEpoch` (default zero); operators must preserve
Expand All @@ -28,6 +29,74 @@ value. Legacy boot authorization and legacy environment public-key issuance are
rejected once the app epoch is nonzero, preventing old clients from silently
continuing with epoch-zero keys.

## Epoch-scoped allowlists

Material approvals are an exact `epoch × material` matrix, not a maximum epoch.
Authorization checks only the selected row, with O(1) add, removal and lookup:

- `DstackKms`: OS images, KMS aggregated measurements and KMS device IDs are
indexed by **root epoch**. OS image limits also apply to app key requests.
- `DstackApp`: compose hashes, device IDs and `allowAnyDevice` are indexed by
**app epoch**. Each required material must be allowed in the selected row.

There is no implicit access in either direction. Rotating keys starts an empty
row; approving a new row neither copies old approvals nor grants historical keys.
Root and app selectors remain independent. Omitted selectors select the current
epochs. For an image handover, explicitly approve the new image in the old row so
it can migrate old keys, approve it in the new row, and remove the old image from
the old row after the handover. The old image never needs access to the new row.

Existing add/remove methods operate on the highest epoch: the latest **registered**
root epoch (including a pending epoch), or the current app epoch. Before any root
epoch is registered, they operate on epoch zero. The `*AtEpoch(epoch, material)`
methods select a particular registered root/current-or-historical app epoch;
removal affects only that row. `setAllowAnyDeviceAtEpoch(epoch, allowed)` is also
row-specific. Single-material add/remove scripts in `Manage.s.sol` accept optional
`ROOT_EPOCH` or `APP_EPOCH` variables, respectively, as does `SetAllowAnyDevice`.

The `*AtEpoch` getters read the selected row, including legacy epoch zero.
`EpochPolicyChanged` events include the row and the resulting permission. Legacy
bool getters and events describe **epoch zero only**; changing a nonzero row does
not change its legacy bool or emit a misleading legacy material event.
Pending root epoch numbers may be popped and reused; policy rows belong to epoch
numbers, not key hashes. Review or clear a pending row before activating a
replacement epoch with the same number.

### KMS keyring handover

Keyring handover is deliberately different from app key access: approval in the
**highest exported root epoch** authorizes the entire keyring, including all older
roots. The KMS passes that epoch explicitly to `authorizeKms`, so approval in the
current row cannot accidentally authorize a newer pending keyring. A registered
pending root may be authorized for KMS handover before activation; app key requests
still cannot select it. KMS self-authorization without a selector continues to
check the active root epoch. The source checks `kms_import_keys` for the recipient, and the recipient
checks `kms_export_keys` for the source, both at the highest transferred epoch.
This permits staged key distribution without changing which epoch the KMS serves.

### Upgrading existing proxies

The original bool mappings and `allowAnyDevice` retain their storage locations and
are used directly as row zero. New nested mappings consume three storage-gap slots
in each contract; no entries are copied and no reinitializer is required. Existing
owners, pending owners, policy flags, root registries and app epoch counters are
preserved. Even if the pre-upgrade counter is nonzero, old global approvals grant
only row zero after upgrading: explicitly populate the intended nonzero rows.

Upgrade the KMS policy before app policies, then approve the intended rows before
restarting guests or activating a new root. `DstackApp` advertises the optional
ERC-165 `IAppEpochAuth` capability so the KMS can discover the current epoch and
pass the selected app epoch and operation to `authorizeApp`. Its interface ID is
the XOR of both method selectors; policies advertising only either former
single-method interface must upgrade to the combined interface. Policies without
that capability can only authorize app epoch zero; failures of an advertised
method propagate instead of falling back.
Direct legacy app checks use the current app epoch; legacy KMS app authorization
still rejects nonzero app epochs.

These material matrices are implemented by the Solidity policies. Mock/simple auth
backends retain their own policy model and may reject handover of pending epochs.

## Derivation and lifetime

Every key is identified by `(root_epoch, app_epoch)`. App epoch zero keeps existing derived keys and app-key signatures. Nonzero app epochs append
Expand All @@ -42,8 +111,9 @@ returned root epoch. Epoch zero keeps the original `dstack-kms-issued` format.
Restart CVMs after advancing the policy. A running CVM continues to use its boot
key pair; omitted guest-agent GetKey selectors mean that CVM's boot epochs, not a
fresh chain lookup. Explicit historical requests select each dimension
independently and cache by the complete pair. Historical access still requires
current app authorization. Supplied `trusted_ca` always pins the KMS connection,
independently and cache by the complete pair. Historical requests must be approved
in the selected root/app rows. Revoking one row does not revoke other rows or
keys already delivered. Supplied `trusted_ca` always pins the KMS connection,
regardless of epoch selection; historical responses cannot replace that pin.

The guest migrates disk headers and environment ciphertext if either dimension
Expand Down
15 changes: 9 additions & 6 deletions dstack/kms/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,10 +267,13 @@ Rejection responses still carry the current epochs and root chain hash. See
`requestedRootEpoch` is optional in the JSON request. Omit it to use the epoch in
this authorization result; an explicit `0` selects epoch 0. Solidity represents
it as `OptionalUint256 { isSet, value }` alongside `AppInfo` in the request. The
contract rejects a requested epoch newer than the current one; historical
epochs remain available for migration. No serving epoch or serving hash is sent.
KMS-to-KMS handover transfers the complete keyring. The receiver explicitly
authorizes the source to export the highest received epoch before importing it.
contract rejects future root epochs for application requests; historical epochs
remain available subject to the selected policy row. No serving epoch or serving
hash is sent. KMS-to-KMS handover explicitly selects the highest exported root
epoch, including a registered pending epoch. Approval in that row authorizes the
complete keyring, including older roots. KMS self-authorization omits the selector
and checks the current root epoch. The recipient separately authorizes the
source to `kms_export_keys` at the highest received epoch before importing it.

Every request includes a required `operation` describing what the attested
subject is authorized to do. JSON uses `snake_case`; the contract receives
Expand Down Expand Up @@ -307,8 +310,8 @@ The response is an `AuthorizationResult`:
```

Both successful and rejected decisions include the current epoch and chain hash
from the same contract call or auth-simple configuration snapshot. With no
registered epoch, the result is `isAllowed: false`,
from the same contract call or auth-simple configuration snapshot. Except for `kms_bootstrap`, with no
registered epoch the result is `isAllowed: false`,
`rejectionReason: "Root epoch not registered"`, epoch 0 and a zero hash. A zero hash
never authorizes key release or requests an epoch switch.

Expand Down
13 changes: 13 additions & 0 deletions dstack/kms/auth-eth/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,3 +192,16 @@ remain available.
[KMS authorization protocol](../README.md#authorization-protocol) for the names
and subject semantics. Bootstrap alone may authorize before root registration;
unknown operations are rejected. Legacy `/bootAuth/*` routes are unchanged.


### Epoch-scoped material approvals

Material approvals use exact root/app epoch rows; neither older nor newer rows
are implicitly authorized. Existing add/remove methods default to the highest
registered root epoch or current app epoch. The `*AtEpoch` methods and optional
`ROOT_EPOCH` / `APP_EPOCH` variables on single-material management scripts select
an explicit row. Original bool mappings and events remain epoch-zero views after
upgrading; epoch-specific getters/events describe the new rows. KMS keyring
handover separately checks its highest exported root epoch, which authorizes the
whole keyring. See [epoch-scoped allowlists](../../../docs/app-key-epochs.md#epoch-scoped-allowlists)
for migration, upgrade order and compatibility.
149 changes: 117 additions & 32 deletions dstack/kms/auth-eth/contracts/DstackApp.sol
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
pragma solidity ^0.8.24;

import "./IAppAuth.sol";
import "./IAppEpoch.sol";
import "./AuthorizationOperation.sol";
import "./IAppEpochAuth.sol";
import "./IAppAuthBasicManagement.sol";
import "@openzeppelin/contracts-upgradeable/proxy/utils/Initializable.sol";
import "@openzeppelin/contracts-upgradeable/access/Ownable2StepUpgradeable.sol";
Expand All @@ -21,7 +22,7 @@ contract DstackApp is
ERC165Upgradeable,
IAppAuth,
IAppAuthBasicManagement,
IAppEpoch
IAppEpochAuth
{
// Mapping of allowed compose hashes for this app
mapping(bytes32 => bool) public allowedComposeHashes;
Expand All @@ -41,6 +42,13 @@ contract DstackApp is
uint256 public override currentAppEpoch;
event AppEpochAdvanced(uint256 indexed epoch);

// Epoch zero continues to use the original fields above after an upgrade.
mapping(uint256 => mapping(bytes32 => bool)) private _composeHashesByEpoch;
mapping(uint256 => mapping(bytes32 => bool)) private _deviceIdsByEpoch;
mapping(uint256 => bool) private _allowAnyDeviceByEpoch;

event EpochPolicyChanged(bytes32 indexed policy, bytes32 indexed value, uint256 indexed epoch, bool enabled);

// Additional events specific to DstackApp
event UpgradesDisabled();
event AllowAnyDeviceSet(bool allowAny);
Expand Down Expand Up @@ -133,7 +141,7 @@ contract DstackApp is
override(ERC165Upgradeable, IERC165)
returns (bool)
{
return interfaceId == type(IAppEpoch).interfaceId || interfaceId == 0x1e079198 // IAppAuth
return interfaceId == type(IAppEpochAuth).interfaceId || interfaceId == 0x1e079198 // IAppAuth
|| interfaceId == 0x8fd37527 // IAppAuthBasicManagement
|| super.supportsInterface(interfaceId);
}
Expand All @@ -148,26 +156,62 @@ contract DstackApp is
emit PolicyChanged(msg.sender, keccak256(bytes(policy)), value, enabled);
}

// Add a compose hash to allowed list
function addComposeHash(bytes32 composeHash) external onlyOwner {
function allowedComposeHashesAtEpoch(uint256 epoch, bytes32 composeHash) public view returns (bool) {
return epoch == 0 ? allowedComposeHashes[composeHash] : _composeHashesByEpoch[epoch][composeHash];
}

function allowedDeviceIdsAtEpoch(uint256 epoch, bytes32 deviceId) public view returns (bool) {
return epoch == 0 ? allowedDeviceIds[deviceId] : _deviceIdsByEpoch[epoch][deviceId];
}

function allowAnyDeviceAtEpoch(uint256 epoch) public view returns (bool) {
return epoch == 0 ? allowAnyDevice : _allowAnyDeviceByEpoch[epoch];
}

function addComposeHash(bytes32 composeHash) external {
addComposeHashAtEpoch(currentAppEpoch, composeHash);
}

function addComposeHashAtEpoch(uint256 epoch, bytes32 composeHash) public onlyOwner {
require(composeHash != bytes32(0), "invalid compose hash");
allowedComposeHashes[composeHash] = true;
emit ComposeHashAdded(composeHash);
_emitPolicy("compose-hash", composeHash, true);
_setComposeHash(epoch, composeHash, true);
}

// Remove a compose hash from allowed list
function removeComposeHash(bytes32 composeHash) external onlyOwner {
allowedComposeHashes[composeHash] = false;
emit ComposeHashRemoved(composeHash);
_emitPolicy("compose-hash", composeHash, false);
function removeComposeHash(bytes32 composeHash) external {
removeComposeHashAtEpoch(currentAppEpoch, composeHash);
}

// Set whether any device is allowed to boot this app
function setAllowAnyDevice(bool _allowAnyDevice) external onlyOwner {
allowAnyDevice = _allowAnyDevice;
emit AllowAnyDeviceSet(_allowAnyDevice);
_emitPolicy("allow-any-device", bytes32(0), _allowAnyDevice);
function removeComposeHashAtEpoch(uint256 epoch, bytes32 composeHash) public onlyOwner {
_setComposeHash(epoch, composeHash, false);
}

function _setComposeHash(uint256 epoch, bytes32 composeHash, bool allowed) private {
require(epoch <= currentAppEpoch, "Unknown app epoch");
if (epoch == 0) {
allowedComposeHashes[composeHash] = allowed;
if (allowed) emit ComposeHashAdded(composeHash);
else emit ComposeHashRemoved(composeHash);
_emitPolicy("compose-hash", composeHash, allowed);
} else {
_composeHashesByEpoch[epoch][composeHash] = allowed;
}
emit EpochPolicyChanged(keccak256("compose-hash"), composeHash, epoch, allowed);
}

function setAllowAnyDevice(bool allowed) external {
setAllowAnyDeviceAtEpoch(currentAppEpoch, allowed);
}

function setAllowAnyDeviceAtEpoch(uint256 epoch, bool allowed) public onlyOwner {
require(epoch <= currentAppEpoch, "Unknown app epoch");
if (epoch == 0) {
allowAnyDevice = allowed;
emit AllowAnyDeviceSet(allowed);
_emitPolicy("allow-any-device", bytes32(0), allowed);
} else {
_allowAnyDeviceByEpoch[epoch] = allowed;
}
emit EpochPolicyChanged(keccak256("allow-any-device"), bytes32(0), epoch, allowed);
}

// Set whether TCB status must be UpToDate to boot this app
Expand All @@ -177,28 +221,69 @@ contract DstackApp is
_emitPolicy("require-tcb-up-to-date", bytes32(0), _requireUpToDate);
}

// Add a device ID to allowed list
function addDevice(bytes32 deviceId) external onlyOwner {
function addDevice(bytes32 deviceId) external {
addDeviceAtEpoch(currentAppEpoch, deviceId);
}

function addDeviceAtEpoch(uint256 epoch, bytes32 deviceId) public onlyOwner {
require(deviceId != bytes32(0), "invalid device ID");
allowedDeviceIds[deviceId] = true;
emit DeviceAdded(deviceId);
_emitPolicy("device", deviceId, true);
_setDevice(epoch, deviceId, true);
}

function removeDevice(bytes32 deviceId) external {
removeDeviceAtEpoch(currentAppEpoch, deviceId);
}

function removeDeviceAtEpoch(uint256 epoch, bytes32 deviceId) public onlyOwner {
_setDevice(epoch, deviceId, false);
}

// Remove a device ID from allowed list
function removeDevice(bytes32 deviceId) external onlyOwner {
allowedDeviceIds[deviceId] = false;
emit DeviceRemoved(deviceId);
_emitPolicy("device", deviceId, false);
function _setDevice(uint256 epoch, bytes32 deviceId, bool allowed) private {
require(epoch <= currentAppEpoch, "Unknown app epoch");
if (epoch == 0) {
allowedDeviceIds[deviceId] = allowed;
if (allowed) emit DeviceAdded(deviceId);
else emit DeviceRemoved(deviceId);
_emitPolicy("device", deviceId, allowed);
} else {
_deviceIdsByEpoch[epoch][deviceId] = allowed;
}
emit EpochPolicyChanged(keccak256("device"), deviceId, epoch, allowed);
}

// Check if an app is allowed to boot
// Legacy callers authorize the current epoch, never an older policy implicitly.
function isAppAllowed(IAppAuth.AppInfo calldata bootInfo)
external
view
override
returns (bool isAllowed, string memory reason)
{
return _authorizeApp(bootInfo, currentAppEpoch);
}

function authorizeApp(
IAppAuth.AppInfo calldata bootInfo,
uint256 appEpoch,
bytes32 operation
)
external
view
override
returns (bool isAllowed, string memory reason)
{
if (!AuthorizationOperation.isApp(operation)) return (false, "Invalid authorization operation");
return _authorizeApp(bootInfo, appEpoch);
}

function _authorizeApp(
IAppAuth.AppInfo calldata bootInfo,
uint256 appEpoch
)
private
view
returns (bool, string memory)
{
if (appEpoch > currentAppEpoch) return (false, "Requested app epoch is not active");
// Optionally require TCB status to be up to date
if (
requireTcbUpToDate
Expand All @@ -208,12 +293,12 @@ contract DstackApp is
}

// Check if compose hash is allowed
if (!allowedComposeHashes[bootInfo.composeHash]) {
if (!allowedComposeHashesAtEpoch(appEpoch, bootInfo.composeHash)) {
return (false, "Compose hash not allowed");
}

// Check if device is allowed (when device restriction is enabled)
if (!allowAnyDevice && !allowedDeviceIds[bootInfo.deviceId]) {
if (!allowAnyDeviceAtEpoch(appEpoch) && !allowedDeviceIdsAtEpoch(appEpoch, bootInfo.deviceId)) {
return (false, "Device not allowed");
}

Expand All @@ -237,5 +322,5 @@ contract DstackApp is
}

// Add storage gap for upgradeable contracts
uint256[48] private __gap;
uint256[45] private __gap;
}
Loading
Loading