Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 87 additions & 4 deletions BENCHMARKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

<!-- Generated by `admin/bench-report render` from benchmarks/data, benchmarks/verified.toml, and benchmarks/gaps.toml. -->

How long rind and the usual Rust crypto libraries take for the same operations, on five machines, and what each costs to build. Every library runs its native path on each host. A ✓ marks an operation whose whole computation a published formal proof covers, as [Verified code](#verified-code) lists.
How long rind and the usual Rust crypto libraries take for the same operations, on six machines, and what each costs to build. Every library runs its native path on each host. A ✓ marks an operation whose whole computation a published formal proof covers, as [Verified code](#verified-code) lists.

## Hosts

Expand All @@ -11,12 +11,11 @@ How long rind and the usual Rust crypto libraries take for the same operations,
| `nixos` | AMD Ryzen 9 5900XT 16-Core Processor | NixOS 26.11 (Zokor) (Linux 6.18.56) | 2026-11-06 at [`f95b91da`](https://github.com/moq-dev/rind/commit/f95b91da886bee265dbd836192a7ade2f9158c47), noisy |
| `desktop` | Intel(R) Core(TM) Ultra 7 270K Plus | Ubuntu 26.04.1 LTS (Linux 7.0.0-42-generic) | 2026-11-06 at [`f95b91da`](https://github.com/moq-dev/rind/commit/f95b91da886bee265dbd836192a7ade2f9158c47), noisy |
| `nixos2` | Intel(R) Core(TM) i7-8700 CPU @ 3.20GHz | NixOS 26.11 (Zokor) (Linux 6.18.56) | 2026-11-06 at [`f95b91da`](https://github.com/moq-dev/rind/commit/f95b91da886bee265dbd836192a7ade2f9158c47) |
| `kixtop` | Apple M4 | macOS 26.6.2 (25.6.0) | 2026-11-07 at [`93f91f48`](https://github.com/moq-dev/rind/commit/93f91f48c9b742890a34bb3485071da6f2d241ab) |
| `ovh-iad` | Intel Core Processor (Haswell, no TSX) | NixOS 26.11 (Zokor) (Linux 6.18.40) | 2026-11-06 at [`6b185382`](https://github.com/moq-dev/rind/commit/6b185382a621d6ffe630db4118adc464bc4d158a) |
| `ovh-sea` | Intel Core Processor (Haswell, no TSX) | NixOS 26.11 (Zokor) (Linux 6.18.40) | 2026-11-06 at [`6b185382`](https://github.com/moq-dev/rind/commit/6b185382a621d6ffe630db4118adc464bc4d158a) |

Not measured yet: `kixtop`.

Each host builds with the pinned toolchain, in the Nix dev shell on Linux, and runs `admin/bench-report` only when idle. Times are criterion's estimates.
Each host builds with the pinned toolchain in the Nix dev shell, and runs `admin/bench-report` only when idle. Times are criterion's estimates.

## Speed

Expand Down Expand Up @@ -255,6 +254,84 @@ Measured 2026-11-06 at [`f95b91da`](https://github.com/moq-dev/rind/commit/f95b9

</details>

### `kixtop`: Apple M4

Measured 2026-11-07 at [`93f91f48`](https://github.com/moq-dev/rind/commit/93f91f48c9b742890a34bb3485071da6f2d241ab), on macOS 26.6.2 (25.6.0) with rustc 1.98.1 (48a229cea 2026-09-01).

The maintainer's MacBook, in daily use. Its cores have DIT, which rind sets around each call that handles a secret and the other libraries never set; Known gaps gives its cost.

<picture>
<source media="(prefers-color-scheme: dark)" srcset="benchmarks/kixtop-dark.svg">
<img alt="Times on kixtop" src="benchmarks/kixtop.svg">
</picture>

<details>
<summary>Times on kixtop</summary>

| Operation | rind | aws-lc-rs | ring | RustCrypto + dalek | graviola | libcrux | OpenSSL |
|---|---|---|---|---|---|---|---|
| full handshake, X25519MLKEM768 | 177.98 µs | 214.67 µs | - | - | 180.42 µs | - | 459.07 µs |
| full handshake, X25519 | 155.39 µs | 184.86 µs | 184.65 µs | - | 159.19 µs | - | 374.65 µs |
| full handshake, P-256 | 176.20 µs | 206.73 µs | 201.44 µs | - | 191.72 µs | - | 400.48 µs |
| full handshake, Ed25519 certificate | 127.23 µs | 150.93 µs | 155.04 µs | - | 124.12 µs | - | 383.83 µs |
| full handshake, RSA-2048 certificate | 371.35 µs | 406.71 µs | 446.94 µs | - | 683.15 µs | - | 643.30 µs |
| resumed handshake, X25519 | 66.29 µs | 92.46 µs | 88.29 µs | - | 61.75 µs | - | 255.12 µs |
| 1 MB transfer, AES-128-GCM | 308.57 µs | 310.78 µs | 372.59 µs | - | 327.11 µs | - | 338.35 µs |
| 1 MB transfer, AES-256-GCM | 343.96 µs | 341.91 µs | 380.26 µs | - | 397.41 µs | - | 371.49 µs |
| 1 MB transfer, ChaCha20-Poly1305 | 1.12 ms | 1.12 ms | 1.11 ms | - | 1.79 ms | - | 1.11 ms |
| AES-128-GCM seal, 32 B | 57 ns | 50 ns | 42 ns | 24 ns | 80 ns | 53 ns | 69 ns |
| AES-128-GCM seal, 1350 B | 212 ns | 180 ns | 212 ns | 213 ns | 215 ns | 619 ns | 197 ns |
| AES-128-GCM seal, 16 KB | 1.49 µs | 1.45 µs | 1.97 µs | 2.36 µs | 1.60 µs | 7.15 µs | 1.52 µs |
| AES-128-GCM open, 32 B | 47 ns | 47 ns | 39 ns | 28 ns | 85 ns | 64 ns | 65 ns |
| AES-128-GCM open, 1350 B | 214 ns | 193 ns | 247 ns | 237 ns | 226 ns | 647 ns | 199 ns |
| AES-128-GCM open, 16 KB | 1.61 µs | 1.57 µs | 2.18 µs | 2.46 µs | 1.71 µs | 6.97 µs | 1.66 µs |
| AES-256-GCM seal, 32 B | 61 ns | 54 ns | 45 ns | 25 ns | 79 ns | 53 ns | 67 ns |
| AES-256-GCM seal, 1350 B | 237 ns | 205 ns | 220 ns | 246 ns | 262 ns | 650 ns | 222 ns |
| AES-256-GCM seal, 16 KB | 1.75 µs | 1.72 µs | 2.04 µs | 2.72 µs | 2.16 µs | 7.20 µs | 1.79 µs |
| AES-256-GCM open, 32 B | 49 ns | 49 ns | 40 ns | 41 ns | 85 ns | 67 ns | 68 ns |
| AES-256-GCM open, 1350 B | 233 ns | 218 ns | 252 ns | 267 ns | 263 ns | 674 ns | 225 ns |
| AES-256-GCM open, 16 KB | 1.86 µs | 1.83 µs | 2.25 µs | 2.89 µs | 2.18 µs | 7.35 µs | 1.94 µs |
| ChaCha20-Poly1305 seal, 32 B | 185 ns | 163 ns | 148 ns | 344 ns | 196 ns | 171 ns ✓ | 215 ns |
| ChaCha20-Poly1305 seal, 1350 B | 792 ns | 771 ns | 755 ns | 1.78 µs | 1.25 µs | 1.98 µs ✓ | 842 ns |
| ChaCha20-Poly1305 seal, 16 KB | 7.70 µs | 7.65 µs | 7.67 µs | 15.21 µs | 12.85 µs | 22.07 µs ✓ | 7.45 µs |
| XChaCha20-Poly1305 seal, 32 B | 1.19 µs | 943 ns | - | 1.53 µs | 1.11 µs | 1.11 µs | - |
| XChaCha20-Poly1305 seal, 1350 B | 2.02 µs | 1.89 µs | - | 3.19 µs | 2.85 µs | 3.80 µs | - |
| XChaCha20-Poly1305 seal, 16 KB | 11.36 µs | 11.43 µs | - | 22.76 µs | 19.24 µs | 34.20 µs | - |
| AES-128 header protection | 35 ns | 3 ns | 3 ns | - | - | - | - |
| AES-256 header protection | 38 ns | 3 ns | 4 ns | - | - | - | - |
| ChaCha20 header protection | 89 ns ✓ | 73 ns | 68 ns | - | - | - | - |
| X25519 | 16.71 µs ✓ | 17.85 µs | 24.48 µs | 29.71 µs | 17.74 µs | 37.77 µs ✓ | 35.77 µs |
| P-256 ECDH | 26.27 µs ✓ | 28.51 µs | 32.31 µs | 180.62 µs | 31.56 µs | 183.72 µs ✓ | 49.96 µs |
| Ed25519 sign | 3.95 µs | 4.06 µs | 7.07 µs | 8.66 µs | 4.17 µs | 22.12 µs ✓ | 16.42 µs |
| Ed25519 verify | 18.66 µs | 18.52 µs | 19.94 µs | 18.42 µs | 18.69 µs | 29.57 µs ✓ | 41.08 µs |
| P-256 ECDSA sign | 7.11 µs ✓ | 11.27 µs | 11.01 µs | 44.33 µs | 8.83 µs | 67.38 µs ✓ | 12.62 µs |
| P-256 ECDSA verify | 27.15 µs ✓ | 27.15 µs | 29.35 µs | 157.87 µs | 28.60 µs | 156.54 µs ✓ | 41.90 µs |
| SHA-256, 64 B | 93 ns | 71 ns | 45 ns | 37 ns | 36 ns | 232 ns ✓ | 162 ns |
| SHA-256, 1350 B | 484 ns | 440 ns | 414 ns | 475 ns | 472 ns | 2.52 µs ✓ | 531 ns |
| SHA-256, 16 KB | 4.94 µs | 4.87 µs | 4.90 µs | 5.69 µs | 5.76 µs | 29.13 µs ✓ | 5.17 µs |
| SHA-512, 64 B | 113 ns | 108 ns | 88 ns | 79 ns | 177 ns | 141 ns ✓ | 211 ns |
| SHA-512, 1350 B | 827 ns | 775 ns | 737 ns | 722 ns | 1.88 µs | 1.58 µs ✓ | 878 ns |
| SHA-512, 16 KB | 8.77 µs | 8.70 µs | 8.69 µs | 8.50 µs | 22.11 µs | 18.41 µs ✓ | 8.88 µs |
| HMAC-SHA256, 64 B | 191 ns | 248 ns | 137 ns | 102 ns | 115 ns | 568 ns ✓ | 2.06 µs |
| HMAC-SHA256, 1350 B | 559 ns | 620 ns | 502 ns | 542 ns | 557 ns | 2.86 µs ✓ | 2.43 µs |
| HKDF-SHA256 extract and expand | 343 ns | 330 ns | 274 ns | 218 ns | - | 942 ns ✓ | 2.14 µs |
| HKDF-SHA256 expand | 109 ns | 154 ns | 94 ns | 70 ns | - | 481 ns ✓ | 1.54 µs |
| RSA-2048 PKCS#1 sign | 275.12 µs | 285.95 µs | 322.84 µs | 586.34 µs | 556.70 µs | - | 341.31 µs |
| RSA-2048 PKCS#1 verify | 8.38 µs | 9.48 µs | 11.05 µs | 30.77 µs | 18.51 µs | - | 16.66 µs |
| RSA-2048 PSS sign | 275.29 µs | 288.58 µs | 324.86 µs | 588.38 µs | 558.57 µs | 4.48 ms ✓ | 342.61 µs |
| RSA-2048 PSS verify | 8.71 µs | 9.84 µs | 11.48 µs | 39.26 µs | 18.79 µs | 299.96 µs ✓ | 17.44 µs |
| ML-KEM-768 key generation | 6.20 µs | 7.46 µs | - | 26.57 µs | 6.22 µs | 12.01 µs | 25.06 µs |
| ML-KEM-768 encapsulate | 6.84 µs | 8.01 µs | - | 27.77 µs | 6.40 µs | 9.39 µs | 24.07 µs |
| ML-KEM-768 decapsulate | 4.17 µs | 7.75 µs | - | 34.06 µs | 4.05 µs | 9.58 µs | 24.43 µs |
| P-384 ECDH | 91.49 µs | 105.75 µs | 412.48 µs | 494.20 µs | 153.84 µs | - | 279.51 µs |
| P-384 ECDSA sign | 22.93 µs | 49.16 µs | 223.72 µs | 112.64 µs | 81.12 µs | - | 104.84 µs |
| P-384 ECDSA verify | 88.12 µs | 111.27 µs | 396.57 µs | 426.43 µs | 151.78 µs | - | 238.33 µs |
| P-521 ECDH | 130.92 µs | 143.20 µs | - | 397.10 µs | - | - | 247.22 µs |
| P-521 ECDSA sign | 36.32 µs | 84.33 µs | - | 79.73 µs | - | - | 132.13 µs |
| P-521 ECDSA verify | 121.06 µs | 167.21 µs | - | 253.06 µs | - | - | 254.80 µs |

</details>

### `ovh-iad`: Intel Core Processor (Haswell, no TSX)

Measured 2026-11-06 at [`6b185382`](https://github.com/moq-dev/rind/commit/6b185382a621d6ffe630db4118adc464bc4d158a), on NixOS 26.11 (Zokor) (Linux 6.18.40) with rustc 1.98.1 (48a229cea 2026-09-01).
Expand Down Expand Up @@ -446,6 +523,10 @@ Measured 2026-11-06 at [`f95b91da`](https://github.com/moq-dev/rind/commit/f95b9
| rustls + aws-lc-rs | 36.6 (36.0 to 38.2) | 89 | 4.11 | cc | 20 |
| rustls + ring | 16.4 (16.2 to 16.4) | 31 | 1.94 | cc | 16 |

### `kixtop`

Not measured: error: host is busy (6.6 of 10 CPUs busy)

### `ovh-iad`

Measured 2026-11-06 at [`6b185382`](https://github.com/moq-dev/rind/commit/6b185382a621d6ffe630db4118adc464bc4d158a), with rustc 1.98.1 (48a229cea 2026-09-01), gcc (GCC) 16.2.0, and 4 jobs.
Expand Down Expand Up @@ -713,6 +794,8 @@ Each operation where another library is more than 10% faster than rind, on a hos
| Ed25519 sign | `ovh-iad` | 1.13× (aws-lc-rs) | Noise from the VPS's other tenants: `ovh-sea`, the same virtual CPU, measured 0.92× aws-lc-rs's time, and no dedicated host shows it. The next refresh rechecks it. |
| `p256-ecdsa-verify/*` | no ADX tier | 1.14× (aws-lc-rs) | CPUs older than Broadwell and Zen lack ADX, and so do virtual CPUs that hide it, as `ovh-iad`'s and `ovh-sea`'s do. There rind runs s2n-bignum's verified `_alt` routines, which trail by more than 10% only on the nightly runners: on the VPSes P-256 verification measured 1.05× aws-lc-rs's time. |
| `chacha20poly1305/*/16KB`, `xchacha20poly1305/*/16KB` | no AES-NI tier | 1.17× (aws-lc-rs) | Without SSE4.1 rind runs Graviola's scalar Poly1305 rather than port BoringSSL's SSE2 `poly1305_vec.c` for CPUs older than Westmere, as ALGORITHMS.md records. |
| AES-128-GCM seal, 32 B, AES-128-GCM seal, 1350 B, AES-128-GCM open, 32 B, AES-128-GCM open, 1350 B, AES-256-GCM seal, 32 B, AES-256-GCM seal, 1350 B, AES-256-GCM open, 32 B, ChaCha20-Poly1305 seal, 32 B, AES-128 header protection, AES-256 header protection, ChaCha20 header protection, SHA-256, 64 B, SHA-256, 1350 B, SHA-512, 64 B, SHA-512, 1350 B, HMAC-SHA256, 64 B, HMAC-SHA256, 1350 B, HKDF-SHA256 extract and expand, HKDF-SHA256 expand | `kixtop` | 1.11× to 12.62× (RustCrypto + dalek, aws-lc-rs, graviola, ring) | DIT: the M4 has it, and rind turns it on and off around every call that handles a secret, which costs about 35 to 55 ns a call there; the other libraries never set it. Keeping it per call was decided with the maintainer from [#387](https://github.com/moq-dev/rind/pull/387)'s findings, and `rind::cpu::Dit::hold()` ([#431](https://github.com/moq-dev/rind/pull/431)) pays it once across a batch of calls. With DIT denied (`--cfg rind_cpu_disable` and `RIND_CPU_DISABLE_dit`), the same commit measured each of these within 10% of the fastest library but AES header protection (1.2× aws-lc-rs's 2.9 ns) and HMAC-SHA-256 at 64 B (1.15× RustCrypto's 101 ns). |
| XChaCha20-Poly1305 seal, 32 B | `kixtop` | 1.26× (aws-lc-rs) | Noise: a rerun of the row on the same commit measured 1.04× aws-lc-rs's time, as did the run before this one, on a commit that changed only x86_64 code. The next refresh rechecks it. |

## Verified code

Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,10 @@
- **As fast as aws-lc-rs and ring.** rind matches or beats both on nearly
every operation TLS and QUIC run, from the handshake to each packet's
AEAD. Where another library is ahead, as ring and aws-lc-rs are at
RSA-2048 signing on x86_64, [Known gaps][gaps] says so, with the quest that
closes the gap or the reason it stays.
RSA-2048 signing on x86_64, and aws-lc-rs is at AES-GCM and short messages
on Apple silicon, where rind alone sets DIT around each call,
[Known gaps][gaps] says so, with the quest that closes the gap or the
reason it stays.

<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/moq-dev/rind/main/benchmarks/headline-dark.svg">
Expand Down
8 changes: 6 additions & 2 deletions admin/bench-report
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,11 @@ HOSTS = {
"nixos": {"jobs": 16},
"desktop": {"jobs": 12},
"nixos2": {"jobs": 3},
"kixtop": {"jobs": 4},
"kixtop": {
"jobs": 4,
"note": "The maintainer's MacBook, in daily use. Its cores have DIT, which rind sets around each call "
"that handles a secret and the other libraries never set; Known gaps gives its cost.",
},
"ovh-iad": {
"jobs": 4,
"ssh": "root@vps-bcb7229a.vps.ovh.us",
Expand Down Expand Up @@ -1486,7 +1490,7 @@ def report(hosts, tiers, verified, gaps):
if missing:
md += [f"Not measured yet: {', '.join(missing)}.", ""]
md += [
"Each host builds with the pinned toolchain, in the Nix dev shell on Linux, and runs "
"Each host builds with the pinned toolchain in the Nix dev shell, and runs "
"`admin/bench-report` only when idle. Times are criterion's estimates.",
"",
]
Expand Down
Loading