Skip to content

refactor: reject legacy BLS storage formats - #5323

Draft
lklimek wants to merge 31 commits into
v5.1-devfrom
refactor/retire-legacy-bls-storage
Draft

lklimek wants to merge 31 commits into
v5.1-devfrom
refactor/retire-legacy-bls-storage

Conversation

@lklimek

@lklimek lklimek commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Basic explanation

What this does: Proposes retiring old on-disk Platform state and quorum formats. Loading one returns a migration instruction instead of decoding it.

Value: Removes the C++ bls-signatures dependency from the default production node. Tests and explicit mocks builds still use it.

Risks: This intentionally prevents direct startup from older databases and snapshots. The proposed migration is to run Platform v4.0.0 on the database and successfully commit at least one block before upgrading. The complete upgrade path has not been tested.

Draft for developer consultation — not ready to merge. Decide whether to require that intermediate migration or retain compatibility readers/adapters. This proposal removes the whole Platform-state V0 reader, including its non-BLS fields; retaining the historical schema with an explicit failing decoder is also a discussion point.

Issue being fixed or feature implemented

Targets v5.1-dev, which includes the BLS backend migration from #5320 and retains historical storage support. This PR isolates the compatibility-policy decision from that migration.

Production uses of the separate C++ BLS crate are in old storage readers. v4.0.0 writes Platform-state V1 and quorum-storage V2, while still being able to read older records.

What was done?

  • Remove Platform-state V0 and quorum-storage V0/V1 readers and conversions.
  • Keep their enum positions reserved with an uninhabited UnsupportedLegacyBlsStorage payload. Its decoder immediately returns an error directing the operator to v4.0.0; it does not parse the old payload or panic.
  • Preserve Platform-state V1/V2 and quorum-storage V2 tags, layouts and serialized bytes.
  • Remove bls-signatures from default features; retain it for tests and the explicit mocks feature.
  • Test rejection of old testnet/devnet records and owned/borrowed quorum decoding. Keep a supported-format fixture serialized before reader removal and check the existing serialization hash.

In-place changes to shipped generations

This changes local database compatibility, not block execution rules. Serialization already writes supported formats for all protocol versions. The proposed rejection applies when loading old records, including outer Platform-state V0 records; no version table or protocol activation changes are introduced.

How Has This Been Tested?

  • cargo test -p drive-abci --lib --all-features --locked platform_types: 133 tests, covering legacy errors, preserved supported tags and byte round trips.
  • cargo clippy -p drive-abci --all-targets --all-features --locked -- --no-deps -D warnings.
  • cargo clippy -p drive-abci --lib --locked -- --no-deps -D warnings verifies the default production configuration.
  • Historical quorum fixtures retain coverage of supported V2 records and fixed 48-byte key encoding; V1 fixtures assert the intended upgrade error.
  • Formatting and whitespace checks.
  • The normal/build dependency tree for default drive-abci excludes bls-signatures and bls-dash-sys.

No full replay or end-to-end v1.0 → v4.0.0 → this branch upgrade was run. Tests prove decoder behavior and supported serialization compatibility, not the complete operational migration procedure.

Breaking Changes

Direct loading of Platform-state V0 and quorum-storage V0/V1 databases/snapshots becomes unsupported. Operators must first have v4.0.0 successfully commit a block on that database; merely launching the binary is insufficient.

The title omits ! because the repository reserves it for consensus-breaking changes.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed
  • If I added or changed GroveDB structure, I described it in the area's structure.rs, regenerated grovedb-structure.json, and checked the structure viewer link posted on this pull request

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

🤖 Co-authored by Claudius the Magnificent AI Agent

PastaPastaPasta and others added 24 commits September 23, 2026 11:37
Bump every rust-dashcore git dependency from e4208c90 to 719de34b (dashpay/rust-dashcore dev, the merge of #1049). The breaking change in range is rust-dashcore #1042: secp256k1 0.30 -> 0.33, rand 0.9, getrandom 0.4, and the secp256k1 context argument dropped API-wide. The range also carries key-wallet fixes #1035 (DIP-15 contact pools extend as payments arrive), #1009 (provider transactions consult every fund-bearing account), #1048, and test-only #920/#1038.

Mechanical call-site migration: drop Secp256k1 contexts (derive_priv/from_priv/public_key/from_secret_key/Keypair::new take no context); secp.sign_ecdsa/recover_ecdsa/verify_ecdsa become methods on the key/signature; SecretKey::from_slice/from_byte_array become from_secret_bytes (slices go through <[u8; 32]>::try_from mapped to Error::InvalidSecretKey, the error from_slice returned); secret_bytes -> to_secret_bytes; SecretKey/SharedSecret lost AsRef<[u8]>, so use as_secret_bytes; thread_rng -> rng; StdRng::from_entropy -> from_os_rng. The secp256k1::hashes re-export is gone, so hex rendering uses the hex crate (same output).

Seeded RNG bridging (rand 0.8 caller -> secp's rand 0.9 StdRng) now fills a 32-byte seed and calls from_seed, which is what rand_core 0.6 from_rng did, so seeded keys are unchanged; drive-abci tests that feed seed_from_u64 into Keypair::new use the secp re-exported StdRng (seed_from_u64 and ChaCha12 are identical across rand_core 0.6/0.9).

rs-platform-encryption moves its own secp256k1 to 0.33.1 so only one secp256k1 is in the graph. simple-signer takes a direct rand 0.8 dep for the RngCore its callers pass. rs-dpp enables getrandom 0.3/0.4 wasm_js on wasm32-unknown-unknown, which the new transitive getrandom versions require there. Rust toolchain already at 1.98.1, matching rust-dashcore.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
infraclaw: install the reviewed publisher prerequisite only; no application or runner changes.
Conflicts resolved:
- Cargo.toml: v5.0-dev's grovedb dce8252f, this branch's rust-dashcore 719de34b.
- Cargo.lock: regenerated from v5.0-dev's lock; the only resolved change
  is secp256k1 0.30.0 -> 0.33.1 (sys 0.10.1 -> 0.14.1, bitcoin-io dropped),
  the same delta the bump produced before.
- rs-platform-encryption tests: v5.0-dev's seeded StdRng, with secp 0.33's
  context-free generate_keypair.
- rs-sdk put_document: v5.0-dev's restructured transition, with rand 0.9's
  from_os_rng / random.
- runner-image-candidate.yml: v5.0-dev's version (controller 1be03edb); the
  feature-branch trigger is no longer needed now that this branch carries
  v5.0-dev's runner-image CI.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
secp256k1 0.33 draws randomness through rand 0.9 / getrandom 0.3, and
key-wallet through getrandom 0.4. Their `wasm_js` backend reads only
`globalThis.crypto.getRandomValues`; getrandom 0.2's `js` feature also fell
back to Node's `require("crypto")`. Node.js exposes the WebCrypto global by
default from v19, so on Node 18 even a deterministic call such as deriving a
public key (secp256k1 rerandomizes its context afterwards) traps.

Node 18 has been end-of-life since April 2025. Raise `engines.node` of
wasm-sdk, wasm-dpp2 and js-evo-sdk to >=20, matching dashmate, and say why
in the evo-sdk README and next to the getrandom features in rs-dpp.

BREAKING CHANGE: @dashevo/wasm-sdk, @dashevo/wasm-dpp2 and @dashevo/evo-sdk
require Node.js >= 20.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…o secp256k1 0.33

Code that landed on v5.0-dev after this bump was first written (encrypted_for
in rs-sdk and wasm-sdk, moderation charter requests) still used the
secp256k1 0.30 / rand 0.8 API. Migrate it the same way as the rest of the
bump: drop the `Secp256k1` context from `PublicKey::from_secret_key`,
`SecretKey::from_slice` -> `from_secret_bytes`, `StdRng::from_entropy` ->
`from_os_rng`, and `Bytes32::random_with_rng` -> `Bytes32::new(rng.random())`
since platform-value's helper takes a rand 0.8 `StdRng`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Merge PR #4932 into v5.1-dev and carry the compatibility adaptations for
rust-dashcore PR #1112 at 870af146. Preserve the existing cryptography
implementations, legacy RPC projection, and FFI ABI.

Keep SQLite replay and accounting repairs, SwiftData accounting changes,
and wallet accounting changes in the dependent PR #5150.

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>
Keep the Core v24 masternode handling and persistence from #5227/#5228,
replacing the migration's superseded strict legacy conversions. Retain
rust-dashcore #1112, the WASM blst patch, and existing rand versions.

Validated formatting, scoped all-target/all-feature Clippy, 90 Core-update
tests, 3 masternode persistence tests and the DPP BLS protocol-boundary test.

Co-Authored-By: Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>
Declare the minimum Node version in wasm-dpp and dash and align their
README requirements with the existing WASM SDK packages.

Validated the production WASM-DPP loader and ECDSA signing on Node
20.20.2 without the test bootstrap's WebCrypto shim.

Co-Authored-By: Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>
Restore package metadata and documentation from v5.1-dev so the Node
minimum can be raised directly to 22 in a separately reviewable PR.
The WASM entropy backend remains part of the rust-dashcore migration.

Co-Authored-By: Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>
Use 314f1060 across all workspace dependencies and lockfile sources. This merged dev revision excludes the separate unmerged spent-claim restoration API needed by PR #5150.

Co-Authored-By: Codex <noreply@openai.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>
Record signatures, secure aggregates, scalar boundary behavior, key generation
and persisted validator bytes before replacing the BLS implementation.
All eight compatibility tests pass on the original backend.

Co-authored-by: Codex <noreply@openai.com>
Preserve historical scalar reduction, infinity parsing, signature bytes and
validator serialization with the previously committed compatibility vectors.
Adapt repository consumers and replace the blst git patch with registry 0.3.17.

Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: Codex <noreply@openai.com>
…ions

Reserve legacy enum tags while rejecting their payloads at deserialization.
Require v4.0.0 to commit a block before opening older databases. Remove
legacy readers and the default production dependency on bls-signatures;
retain the dependency only for tests and explicit mocks.

Test legacy rejection, supported storage tags, persisted-state round trips,
and the existing serialization hash using a pre-removal fixture.

Co-Authored-By: Codex <noreply@openai.com>
Move the signature implementation under bls and export PublicKey, SecretKey,
Signature and BlsError from dpp::bls. Update all repository consumers and
remove the former dpp::bls_signatures path.

Co-Authored-By: Codex <noreply@openai.com>
Move BLS Serde support under bls/serde.rs. Share key parsing between
Deserialize and field adapters while retaining tuple decoding for bincode
and deserialize_any for buffered tagged fields. Deprecate the old adapter
re-export and retain its optional-key path.

Co-Authored-By: Codex <noreply@openai.com>
Restore Platform-state V0 and quorum-storage V0/V1 readers and their
production dependency. Keep their removal separate from the backend
migration so the compatibility policy can be discussed independently.
Adapt restored imports to dpp::bls.

Co-Authored-By: Codex <noreply@openai.com>
Reject Platform-state V0 and quorum-storage V0/V1 with an instruction to
run v4.0.0 and commit a block before upgrading. Preserve format tags and
supported serialized bytes. Restrict the C++ BLS dependency to tests and
explicit mocks.

Isolated from #5320 for developer consultation on storage compatibility
and the required intermediate upgrade.

Co-Authored-By: Codex <noreply@openai.com>
@coderabbitai

coderabbitai Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@thepastaclaw

thepastaclaw commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

🕓 Review not started yet because this PR is a draft.

  • Request normal review — click when the PR is ready for review.
  • Request priority review — click to move this review to the front of the queue.

Commit ad2252a. Normal review starts when eligible; priority review starts as soon as a slot is available.

@lklimek
lklimek marked this pull request as ready for review October 7, 2026 11:24
@lklimek
lklimek marked this pull request as draft October 7, 2026 11:26
lklimek and others added 4 commits October 7, 2026 11:34
…siblings

The supported-format fixture lived in a separate `.hex` file loaded
through `include_str!` at two call sites, unlike the other platform
state fixtures, which are named statics in `test/fixture/platform_state.rs`.

Move it there as `PLATFORM_STATE_V8_DEVNET_SUPPORTED`, with its
provenance on the static instead of on one of its two uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>
Cover DPP legacy-key conversion with frozen vectors and preserve infinity rejection across basic and aggregate verification. Document the Rust API migration in the changelog.

Co-Authored-By: Codex <noreply@openai.com>
Keep PublicKey opaque while representing infinity and validated backend points with a private enum. Preserve encodings and validation behavior.

Co-Authored-By: Codex <noreply@openai.com>
@lklimek
lklimek marked this pull request as ready for review October 7, 2026 13:10
@lklimek
lklimek marked this pull request as draft October 7, 2026 13:11
Base automatically changed from test/bls-backend-compatibility to chore/rust-dashcore-1112-v5.1 October 7, 2026 15:27
@PastaPastaPasta
PastaPastaPasta force-pushed the chore/rust-dashcore-1112-v5.1 branch from 5d8438f to 1bc426d Compare October 13, 2026 20:10
Base automatically changed from chore/rust-dashcore-1112-v5.1 to v5.1-dev October 14, 2026 07:38
@github-actions github-actions Bot added this to the v5.1.0 milestone Oct 14, 2026
Retain current base dependency, wallet, FFI and BLS API updates while
preserving the retirement of legacy Platform state and quorum readers.
Keep the base's fixed-width public key storage encoding and historical
storage coverage; expect the retired V1 quorum format to be rejected.

Validated 133 platform_types tests, focused all-target/all-feature and
default-production Clippy, formatting, and the production dependency graph.

<sub>🤖 Co-authored by [Claudius the Magnificent](https://github.com/lklimek/claudius) AI Agent</sub>

Co-Authored-By: OpenAI Codex <noreply@openai.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants