feat(crypto)!: bump to new leanVM version - #606
Draft
MegaRedHand wants to merge 43 commits into
Draft
Conversation
…ation)
Point the leanVM dependency at `main`. leanVM's `main` ("Xmss api rework")
removed the `lean-multisig` and `leansig_wrapper` crates ethlambda imported,
internalized XMSS in its own `xmss` crate (dropping the external leanSig
dependency), and reworked the recursive-aggregation API. Tracking main is
therefore a migration of the whole signature stack, not a version bump.
Changes:
- Cargo: replace `leansig`/`lean-multisig`/`leansig_wrapper` with `xmss` +
`rec_aggregation` (git, branch=main), plus `ssz` (ethereum_ssz, used by the
xmss SSZ impls) and `postcard` (secret-key format).
- types::signature: rebuild ValidatorPublicKey/Signature/SecretKey on leanVM's
`xmss` crate. Keys/signatures (de)serialize via SSZ (`PUB_KEY_SSZ_LEN` /
`SIGNATURE_SSZ_LEN`); secret keys via postcard. `SIGNATURE_SIZE` and the new
`PUBLIC_KEY_SIZE` are sourced from the xmss scheme constants. The sliding
preparation-window API becomes a fixed activation range (`advance_preparation`
is now a no-op; keys warm their signing cache on demand).
- crypto: rewrite aggregation/verification against `rec_aggregation`. Proofs now
serialize with `to_bytes()`/`from_bytes()` and embed participant pubkeys, so
the verify paths cross-check the embedded set against the expected validators
instead of attaching them at decode time. `setup_prover`/`setup_verifier` are
replaced by the idempotent `init_aggregation_bytecode`.
- Validator public keys are now 32 bytes (was 52); genesis/state layouts and
fixtures updated accordingly.
BREAKING: this changes the on-wire signature/proof formats and the genesis key
format. It is interop-incompatible with clients still on the previous scheme and
requires regenerated genesis keys; the fixture-driven spec tests need
regenerated fixtures. Landing this needs ecosystem coordination.
…e branch Tracking `branch = main` re-resolves on every fetch, so the build could change under us. Pin the current main HEAD for reproducibility; the comment notes it is main's tip and the rev can be bumped to move forward.
Moves the pin two commits forward from a73ab11 (`Xmss api rework`, #262): - `105c5c69` rec_aggregation: optional pubkey-less aggregate serialization - `c83b40f0` fully single-threaded verifier (`Vec` instead of `ArenaVec`), plus a `forbid-parallelism` feature that asserts it `105c5c69` split `SingleMessageInfo` into a `SingleMessageCore` (message, slot, bytecode claim) plus the pubkey set, so the binding checks in the Type-1 and Type-2 verify paths reach through `.core`. The `xmss` crate is byte-identical between the two revs, so key and pubkey formats are unchanged: the genesis keys generated for a73ab11 verify as-is. `c83b40f0` makes proof verification sequential rather than pool-dispatched. The `forbid_parallelism()` guards it adds are inert unless leanVM's `forbid-parallelism` feature is enabled, which we do not enable. Validated on a 3-node all-ethlambda devnet with fresh new-format genesis: real leanVM proving on every block, Type-2 aggregates verified from gossip in ~50ms, finality advancing one slot per slot, no errors.
leanVM `105c5c69` restored optional pubkey-less aggregate serialization, so the wire format no longer has to change relative to `main`. Proofs serialize via `to_bytes_without_pubkeys()` and the caller attaches the resolved signer set at decode with `from_bytes_without_pubkeys()`, exactly as `compress_without_pubkeys()`/`decompress_without_pubkeys()` did before the migration. Every public signature in the crypto crate now matches `main` again, including `split_type_2_by_message`, which regains its `pubkeys_per_component` argument (`reaggregate.rs` already resolves that layout for its merge step, so the call site just passes it through). Attaching the caller's set is what binds a proof to its participants: leanVM sorts and de-duplicates it, then the SNARK checks it against the hash the proof commits to. So the explicit `PublicKeySetMismatch` comparison the embedded-pubkey form needed is gone, and a wrong set now fails inside the verifier — covered by a new `test_verify_wrong_pubkey_set_fails`. Validated on a 3-node all-ethlambda devnet: 114 block Type-2 aggregates decoded and verified through the pubkey-less path (111 carrying attestation components), p50 43.7ms, finality tracking head at 3 slots, no errors. Reaggregation never triggered there, so `split_type_2_by_message` is covered by the Type-2 merge/verify/split round-trip test instead.
`ensure_prover_ready` only called `init_aggregation_bytecode`, so the arena was never engaged and every prover buffer used the system allocator. `setup_prover` lives in the `lean-multisig` facade, not in `rec_aggregation`, so importing crate by crate hid it. Engaging the arena arms a panic: only one proof may run at a time. Take a permit around each proving entry point; without it 5 of the 7 crypto tests panic. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Resolves the overlap with #541, which moved the signature primitives out of `ethlambda-types` and into `ethlambda-crypto` while this branch was rewriting those same primitives for leanVM's internalized XMSS. The two wire-size constants cannot follow `signature.rs` into `ethlambda-crypto`: `types::attestation::XmssSignature` and `types::state::ValidatorPubkeyBytes` are defined in terms of them, and `ethlambda-crypto` already depends on `ethlambda-types`. #541 hit the same constraint and hardcoded `SIGNATURE_SIZE` in `types`. Keep that placement, but source both from leanVM's xmss crate so they track the scheme parameters: main's literal 2536 is the old leanSig size and is wrong for leanVM's wire format. `ethlambda-types` therefore keeps a narrow `xmss` dependency for the two constants only, and `ssz`/`postcard` move to `ethlambda-crypto` along with the signing code that needed them.
`ethlambda-types` pulled in leanVM's `xmss` crate solely to source two integers, dragging the whole signing backend into the dependency graph of a crate that only describes wire formats. Hardcode `SIGNATURE_SIZE` and `PUBLIC_KEY_SIZE` there instead, and pin them to the scheme in `ethlambda-crypto` -- the one crate that sees both sides -- with const assertions. A leanVM bump that changes the XMSS parameters now fails to compile with a named error rather than silently resizing every signature and pubkey on the wire.
The const assertions named the mismatched constants but not their values: a
const panic message must be a string literal, so it cannot interpolate the two
numbers a reader needs in order to fix the constant.
Express the check as an array-length mismatch instead. rustc then evaluates both
sides and prints them ("expected an array with a size of 2536, found one with a
size of 1208"), so the corrected value is right there in the error.
A named constant would read better than `const _` but trips `dead_code`, which
is denied in CI, so the intent lives in the comment -- including a warning not
to take rustc's suggestion to edit the array length, which would silence the
guard rather than fix the constant.
leanVM's `setup_prover` engages a bump arena whose slabs never go back to the OS: `free` is a no-op and `begin_phase` only rewinds the per-thread bump pointers, so a node's RSS ratchets up to its allocation high-water mark and stays there whether or not it keeps proving. The new rev adds `setup_prover_without_arena`, which initializes the same prover on the system allocator. Proving is slower; memory is bounded and proofs are identical, which is the trade a long-lived node wants.
Every prover entry point paired `ensure_prover_ready()` with `prover_permit()`, and neither is useful without the other: setup is a no-op after the first call, and proving without the permit is exactly the concurrent-proof case leanVM panics on. Fold them into `acquire_prover()`, which returns the guard, so a caller cannot reach the prover half-armed. Setup now runs under the permit, which serializes the one-time bytecode compile instead of racing every waiting caller into the same `OnceLock`. The idempotency test drops each guard before taking the next: the permit is not reentrant, so holding two at once in one thread would deadlock.
The permit was taken at the top of each entry point, so a caller sat on the process-wide prover lock while it converted pubkeys, zipped signatures and postcard-decoded child proofs: work that touches no prover state and that grows with the number of aggregated signatures. Take it immediately before the aggregate/merge/split call instead. Decoding does need one thing setup provided: a Type-2 decode rebuilds the bytecode claim, and without the aggregation bytecode it returns `None`, which surfaces as a bogus `DeserializationFailed`. The decoding paths now call `ensure_verifier_ready()`, its documented prerequisite, which is a `OnceLock` init rather than a lock.
The previous commit had the decoding entry points call `ensure_verifier_ready()` and then `acquire_prover()`, which is the two-call pattern the merge was meant to remove: it splits one setup across two names and invites a caller to assume decoding is verifier-only work. Since those paths need setup before they can decode, they take the permit from the start and let it cover the decode. `aggregate_signatures` keeps the narrow scope: it only reshapes its inputs and never decodes, so nothing there needs setup.
The system allocator is the right default (bounded RSS), but it costs proving throughput, and a host with memory to spare should be able to buy that back. A `OnceLock<bool>` carries the choice: `enable_prover_arena()` sets it, and the first `acquire_prover()` latches it via `get_or_init(|| false)`, so the allocator cannot change under a prover that already ran. The setter reports whether it won the latch instead of failing silently, since being called too late is the likely mistake and it is invisible otherwise. Covered by an integration test rather than a lib test: the latch is process-wide, so enabling the arena in the lib binary would change the allocator under every other test and make the result ordering-dependent.
Gives operators the memory/throughput trade at boot: off keeps RSS bounded on the system allocator, on buys proving speed with memory that is never returned to the OS. Applied immediately after parsing, ahead of every other init, since the allocator choice latches on the first proof. The too-late branch warns rather than passing silently; it should be unreachable from here, but a future init that proves earlier would otherwise flip the node to the wrong allocator with no trace.
Setup was reached through `acquire_prover`, which coupled two unrelated things: a caller wanting the prove lock also triggered backend init, so the decoding paths had to hold the lock across work that needs no lock purely to get the bytecode compiled first. `init_leanvm(use_arena)` now does both halves in one place and binaries call it right after parsing arguments. `acquire_prover` is just the permit again, so every entry point takes it immediately before proving, and `ensure_verifier_ready` is gone: nothing needs a lazy hook anymore. The arena `OnceLock` goes too, since the choice is a parameter rather than latched global state. Ahead of the Hive test-driver branch in main, which verifies signatures. Tests that touch a proof call `init_leanvm` themselves, standing in for the startup call. This moves ~2.3s of bytecode compilation from the first proof or verification into startup. Nodes paid it either way; it is now off the first duty's critical path, at the cost of a slower boot for nodes that never prove.
The latest merge from main brought in two more test helpers (#554, #556) that build validators with hardcoded 52-byte pubkeys, which do not compile on this branch: tracking leanVM main shrinks XMSS pubkeys to 32 bytes. Reference the constant so the size follows the type, same as 8f6a6b6 did for the helpers the previous merge carried over.
# Conflicts: # Cargo.lock
…er the main merge Merging main brought in the offline block-building benchmark (#595, #596), which was written against the old 52-byte pubkey. It is not a textual conflict, so the merge landed silently and only broke at compile time: error[E0308]: mismatched types --> bin/ethlambda/src/benchmark/corpus.rs:127:5 | expected an array with a size of 32, found one with a size of 52 `synthetic_pubkey` now sizes its buffer from `PUBLIC_KEY_SIZE`, so the next scheme change moves it rather than breaking the build again. The same merge left three references to a dependency this branch removed: - The benchmark report embedded a resolved `leansig` revision read from Cargo.lock. leanVM internalized XMSS, so no `leansig` package resolves any more and every report would have printed `leansig=unknown`. The header now carries the single leanVM revision that pins the whole signature stack. - `rand` was a dependency of `ethlambda-crypto` and a dev-dependency of `ethlambda-blockchain` and `ethlambda-storage` only for leanSig keygen in tests this branch rewrote. Dropped from all three; `ethlambda-types` keeps its own, which was already unused before this branch. - CLAUDE.md still described 52-byte keys, 2536-byte signatures, and a `leansig` dependency. Also corrects the `SingleMessageAggregate` / `MultiMessageAggregate` doc comments, which claimed the proof bytes are leanVM's `to_bytes()` form with participant pubkeys embedded. Every producer and consumer uses `to_bytes_without_pubkeys()`: pubkeys stay off the wire and the verifier rebuilds the set from the aggregation bits, which is the property that makes the framing match main. Fixture-driven spec tests still fail (122 forkchoice, 73 stf, 7 ssz, 3 signature), all at deserialization: leanSpec's latest released fixtures are still on the 52-byte scheme. Every other test in the workspace passes.
leanVM `main` moved from a5909d18 (2026-07-30) to 5a4f55c1 (2026-09-03), 718 commits, and reshaped both halves of the crypto surface. The whole stack now arrives through leanVM's own facade crate, `leanvm`, which re-exports the aggregation API, the `xmss` module and the `rand` signing draws from: one dependency at one revision, in place of `xmss` + `lean-multisig` + a direct `ethereum_ssz`. ## One aggregate type, grouped by epoch `SingleMessageAggregateSignature` and `MultiMessageAggregateSignature` are gone. There is one `AggregateSignature`, whose XMSS claims are grouped by epoch: one `XmssGroup` per slot, carrying the one message signed at it and that group's sorted, deduplicated keys. Type-1 and Type-2 are now the same object, one group versus several, so the wrappers here differ only in how many groups they build. | before | after | | --- | --- | | `aggregate_single_message_signatures(children, raw, msg, slot, rate)` | `aggregate(children, raw_xmss, raw_sphincs, declare, rate)` | | `merge_single_message_aggregates(type_1s, rate)` | the same `aggregate`, children only | | `split_multi_message_aggregate(t2, index, rate)` | the same `aggregate`, `declare` naming the group to keep | | `verify_single_message_aggregate` / `verify_multi_message_aggregate` | `sig.verify()` | ## The binding left the wire `to_bytes_without_pubkeys()` used to carry `(message, slot)` inside the proof, so `verify_*` could compare it against what the caller expected and `split_*` could search the decoded components for a message. Neither is possible now: the epoch and the message live in the `XmssGroup`, which is exactly the part the without-pubkeys form omits. Every decode has to rebuild the whole signer set from the caller's own view of it, which is what the new `SignerSet` carries. Message, slot and keys travel together because none of the three can be supplied without the others. Three wrapper signatures change as a result: - `merge_type_1s_into_type_2` takes `(SignerSet, bytes)` per child. - `verify_type_2_signature` takes `&[SignerSet]` in place of parallel pubkey and binding lists. - `split_type_2_by_message` takes `&[SignerSet]` for the whole parent, since decoding it needs every claim even though one survives. `VerificationError::BindingMismatch` goes with them. A wrong message or slot changes the digest the proof is checked against, so it now fails inside the SNARK verifier rather than at a field compare ahead of it: the check is not weaker, it just costs a full verification to reach. ## A slot carries one message `ConflictingMessages` is new upstream: within one aggregate the message is a function of the epoch. Two distinct `AttestationData` at one slot have no representation in a single proof, and `on_block` rejects only duplicate `AttestationData`, not two entries at one slot, so a fork that splits attesters at slot N yields a block this leanVM cannot prove. Surfaced as `ethlambda_crypto::ConflictingMessages` rather than worked around: leanSpec still models a block's components as parallel `(root, slot)` lists, so the protocol question is open upstream. ## XMSS leanVM's `xmss` is BLAKE2s over binary fields now, not Poseidon over KoalaBear. The wire sizes are unchanged by coincidence (32-byte keys, 1208-byte signatures, both still static-asserted), but every genesis key and every stored proof from before the rewrite is invalid. The call surface moved too: `key_gen_from_seed` returns `(secret, public)` over an inclusive epoch range, `sign` draws its randomness from a caller-supplied rng, and `verify` reorders its arguments. The preparation window is gone, a key's signable range being fixed at generation, and `XmssSecretKey::prepare` only warms the cached bottom subtree. `KeyManager::advance_keys_to`, which looped an `advance_preparation()` that had already become a no-op, is now `prepare_keys_for`; an unsignable slot is named as exhaustion where it is reached. ## Testing Every `ethlambda-crypto` test passes against the new backend under `--include-ignored`: the Type-2 round trip (aggregate two Type-1s, merge, verify, narrow back to one claim, verify that), the three negative binding tests, the arena path, and the XMSS sign/verify round trip. Two new unit tests pin the signer-set construction every decode depends on. Fixture-driven spec tests still fail exactly as they did before this commit (122 forkchoice, 73 stf, 7 ssz, 3 signature), all at deserialization: leanSpec's latest released fixtures are still on the 52-byte scheme.
MegaRedHand
changed the title
leanVM main proves over a binary field tower, where multiplication is a carryless multiply. The x86 entry has enabled its instruction all along (`+pclmulqdq`); there was no aarch64 entry, because the devnet hosts are x86. That gap only became load-bearing with this leanVM: the old Poseidon over KoalaBear prover worked in a 31-bit prime field and never issued a carryless multiply at all. `aarch64-unknown-linux-gnu` defaults to `neon` alone, and LLVM gates PMULL behind the `aes` feature, so a Linux container on an Apple-silicon host built the one configuration with no hardware support for the prover's inner loop. `aarch64-apple-darwin` defaults to a CPU that already has `aes`, which is why a native `cargo test` never showed it. Measured on an M4 Max, one interval-2 aggregation of three XMSS signatures in a three-node local devnet: | build | aggregation | | --- | --- | | before | 27 s, then 56 s, then 83 s as sessions queued | | after | 214-343 ms | Before, no node ever finished a block: a single uncontended node took 60.6 s to build a zero-attestation block, against a 4 s slot. After, the same devnet imports blocks and the head advances every slot. `neoverse-n1` is the target CPU rather than `native` so the build stays reproducible across hosts; it is armv8.2, which both Apple silicon and the Neoverse N1/V1 hosts clear.
`ethlambda keygen` writes the key set and manifest `--hash-sig-keys-dir` reads, in the layout `hash-sig-cli generate` produces, so `generate-genesis.sh` can call the client instead of a separately pinned image. ## Why the client should own this An XMSS key file is only usable by a client built against the same scheme, and the scheme lives in leanVM, which ethlambda pins. The trap is that nothing in the file layout changes when the scheme does: leanVM's move from Poseidon over KoalaBear to BLAKE2s over binary fields kept the public key at 32 SSZ bytes and the secret key in postcard, so a key set from the wrong revision satisfies every check a genesis generator makes and fails only once a signature is verified. That reads as a consensus bug rather than a provisioning one. Generating here removes the second pin. The keys come from the same `ethlambda-crypto` types the node loads them with, through two new constructors on `ValidatorSecretKey`, so the generator and the loader cannot disagree about the format. `generate` takes the signable slots as an INCLUSIVE range, matching leanVM and making the off-by-one that gives every key one epoch too many hard to write. ## The manifest gains `leanvm_rev` Every other field is either derived from parameters that outlived the last scheme change or hardcoded. `key_scheme` is built from the lifetime, `V` and `CHAIN_LENGTH`, all three unchanged across the BLAKE2s rewrite, so it labels the two formats identically; `hash_function` does separate them but is a string, leanVM's facade exporting no name for its hash and neither of the other two parameters. The revision is resolved from `Cargo.lock` at build time and is the one field that cannot drift from what the binary links, so it is what a reader should trust. It also already existed, embedded for the benchmark report. ## Choices worth naming - `--force` gates overwriting an existing key set. A key is one-time-use material: replacing a set validators are still signing with makes each of them sign twice at one slot, under a key someone else now holds. Silence is the wrong default there. - No `--export-format`. `hash-sig-cli` took one to choose between `ssz` and a `both` that additionally wrote serde-JSON dumps its own help called legacy; nothing reads those, so there is one output form and no flag to pick it. A caller carrying `--export-format ssz` has to drop the argument. - `--num-validators` defaults to 1, leaving `--output-dir` the only required flag: one validator's pair is the shape for inspecting a key or replacing a single node's, and a genesis passes the count anyway. - Serial generation, as `hash-sig-cli` does. leanVM already fans one key out over the bottom subtrees and saturates the cores. - No `--seed`. Deriving 2N keys from one seed needs a KDF this has no reason to invent; `ValidatorSecretKey::generate_from_seed` is there for when it does. ## Verified Generated three validators at 2^18 epochs (9.8 s), then built a devnet genesis from them with `generate-genesis.sh`: keygen skipped as already present, scheme reported, and the 32-byte pubkeys landed in `config.yaml`. Tests cover the key set being loadable by `ValidatorSecretKey::from_bytes`, its public keys matching both the paired secret and the manifest, every generated key being distinct, the refusal to overwrite, and that `keygen` is not swallowed by the default sub-command.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Brings ethlambda up to leanVM
mainat5a4f55c1(2026-09-03) and adapts to theAPI rework that landed there.
This PR is standalone against
main: it branches off thebuild/leanvm-track-mainline, so the diff here is the completeethlambda-on-leanVM-main migration, not only the latest bump. The last commit,
build(deps): track leanVM main's unified aggregation API, is the new work;everything before it is the existing tracking branch (#539), which this
subsumes.
Two commits follow the bump: one enabling carryless multiply on aarch64, and one
adding
ethlambda keygen. Both are described at the end.leanVM
mainmoveda5909d18(2026-07-30) →5a4f55c1, 718 commits. The wholecrypto stack now arrives through leanVM's own facade crate,
leanvm, whichre-exports the aggregation API, the
xmssmodule and therandthat signingdraws from: one dependency at one revision, in place of
xmss+lean-multisig+ a directethereum_ssz.One aggregate type, grouped by epoch
SingleMessageAggregateSignatureandMultiMessageAggregateSignatureare gone.There is one
AggregateSignature, whose XMSS claims are grouped by epoch: oneXmssGroupper slot, carrying the one message signed at it and that group'ssorted, deduplicated keys. Type-1 and Type-2 are now the same object, one group
versus several, so the wrappers in
ethlambda-cryptodiffer only in how manygroups they build.
aggregate_single_message_signatures(children, raw, msg, slot, rate)aggregate(children, raw_xmss, raw_sphincs, declare, rate)merge_single_message_aggregates(type_1s, rate)aggregate, children onlysplit_multi_message_aggregate(t2, index, rate)aggregate,declarenaming the group to keepverify_single_message_aggregate/verify_multi_message_aggregatesig.verify()The binding left the wire
to_bytes_without_pubkeys()used to carry(message, slot)inside the proof, soverify_*could compare it against what the caller expected andsplit_*couldsearch the decoded components for a message. Neither is possible now: the epoch
and the message live in the
XmssGroup, which is exactly the part thewithout-pubkeys form omits. Every decode has to rebuild the whole signer set from
the caller's own view of it, which is what the new
SignerSetcarries. Message,slot and keys travel together because none of the three can be supplied without
the others.
Three wrapper signatures change as a result:
merge_type_1s_into_type_2takes(SignerSet, bytes)per child.verify_type_2_signaturetakes&[SignerSet]in place of parallel pubkey andbinding lists.
split_type_2_by_messagetakes&[SignerSet]for the whole parent, sincedecoding it needs every claim even though one survives.
VerificationError::BindingMismatchgoes with them. A wrong message or slotchanges the digest the proof is checked against, so it now fails inside the SNARK
verifier rather than at a field compare ahead of it. The check is not weaker, it
just costs a full verification to reach.
Worth a look: a slot carries one message
ConflictingMessagesis new upstream: within one aggregate the message is afunction of the epoch (
bind_messagein leanVM'splan_coveragekeys purely onepoch → message; signer sets never enter it). Two distinctAttestationDataatone slot therefore have no representation in a single proof, and
on_blockrejects only duplicate
AttestationData, not two entries at one slot. So a forkthat splits attesters at slot N yields a block this leanVM cannot prove, and
disjoint signers do not help.
The normal aggregator path is unaffected: same slot plus same message is the same
AttestationData, and partial coverage for it merges into one group with thekeys unioned, which is what
aggregate_proofs/aggregate_mixeddo.Surfaced as
ethlambda_crypto::ConflictingMessagesrather than worked around,since narrowing what a block may pack is a protocol decision. leanSpec
mainstill models a block's components as parallel
(root, slot)lists, so thequestion is open upstream too.
XMSS
leanVM's
xmssis BLAKE2s over binary fields now, not Poseidon over KoalaBear.The wire sizes are unchanged by coincidence (32-byte keys, 1208-byte signatures,
both still static-asserted in
ethlambda-crypto), but every genesis key andevery stored proof from before the rewrite is invalid: a devnet crossing this
boundary needs fresh genesis and a wiped DB.
The call surface moved too:
key_gen_from_seedreturns(secret, public)over aninclusive epoch range,
signdraws its randomness from a caller-supplied rng, andverifyreorders its arguments. The preparation window is gone, a key's signablerange being fixed at generation, and
XmssSecretKey::prepareonly warms thecached bottom subtree.
KeyManager::advance_keys_to, which looped anadvance_preparation()that had already become a no-op, is nowprepare_keys_for; an unsignable slot is named as exhaustion where it is reached.Testing
Every
ethlambda-cryptotest passes against the new backend under--include-ignored(13 tests): the Type-2 round trip (aggregate two Type-1s,merge, verify, narrow back to one claim, verify that), the three negative binding
tests, the arena path, and the XMSS sign/verify round trip. Two new unit tests pin
the signer-set construction every decode depends on.
make lintis clean. Fixture-driven spec tests still fail exactly as they did onthe tracking branch before this bump (122 forkchoice, 73 stf, 7 ssz, 3 signature),
all at deserialization: leanSpec's latest released fixtures are still on the
52-byte scheme, so they cannot load until leanSpec migrates and republishes.
Not in scope
passes an empty SPHINCS list.
DeferredClaim::recompute, andverify()repeats it to compare, sodecode-then-verify pays that pass twice. Left upstream.
mainfirst.Carryless multiply on aarch64
aarch64-unknown-linux-gnudefaults toneonalone, and LLVM gates PMULL behindthe
aesfeature. leanVM main's binary-field tower multiplies via carrylessmultiply, so a Linux container on an Apple-silicon host was the one build with no
hardware support for the prover's inner loop, and fell back to scalar code. The
x86 entry in
.cargo/config.tomlhas had+pclmulqdqall along; there was noaarch64 entry, because the devnet hosts are x86.
aarch64-apple-darwindefaultsto a CPU that already has
aes, which is why a nativecargo testnever showedit.
The old Poseidon over KoalaBear prover worked in a 31-bit prime field and never
issued a carryless multiply at all, so this only became load-bearing here.
Measured on an M4 Max, one interval-2 aggregation of three XMSS signatures in a
three-node local devnet:
Before, no node ever finished a block: a single uncontended node took 60.6 s to
build a zero-attestation block, against a 4 s slot.
ethlambda keygenWrites the key set and manifest
--hash-sig-keys-dirreads, in the layouthash-sig-cli generateproduces, sogenerate-genesis.shcan call the clientinstead of a separately pinned image.
This is the same trap as the section above, one level up. An XMSS key file is
only usable by a client built against the same scheme, and no file size changes
when the scheme does: the BLAKE2s rewrite kept the public key at 32 SSZ bytes and
the secret key in postcard, so a key set from the wrong revision satisfies every
check a genesis generator makes and fails only once a signature is verified.
That reads as a consensus bug rather than a provisioning one, and it is what made
this devnet's first genesis unusable.
Generating in the client removes the second pin: the keys come from the same
ethlambda-cryptotypes the node loads them with, through two new constructorson
ValidatorSecretKey, so the generator and the loader cannot disagree aboutthe format. The manifest gains
leanvm_rev, resolved fromCargo.lockat buildtime; every other field is hardcoded or derived from parameters that came through
the rewrite unchanged, so
key_schemelabels both formats identically.--forcegates overwriting an existing key set, because a key is one-time-usematerial: replacing a set validators are still signing with makes each of them
sign twice at one slot, under a key someone else now holds.
One caller-visible break: there is no
--export-format.hash-sig-clitookone to choose between
sszand aboththat additionally wrote serde-JSON dumpsits own help called legacy. Nothing reads those, so there is one output form and
no flag to pick it, and
generate-genesis.shhas to drop the--export-format sszargument to call this.Devnet verification
Three ethlambda nodes, one subnet, one aggregator, on a genesis generated by
ethlambda keygen:That exercises the whole migrated path with real proofs: keygen, XMSS sign,
interval-2 aggregate, proposer Type-1 wrap, Type-2 merge, gossip, decode against
rebuilt
SignerSets, verify.