Skip to content

wycheproof: test OAEP decryption - #740

Merged
tarcieri merged 1 commit into
RustCrypto:masterfrom
wihansaem1991:test/wycheproof-oaep-decrypt
Nov 1, 2026
Merged

tarcieri merged 1 commit into
RustCrypto:masterfrom
wihansaem1991:test/wycheproof-oaep-decrypt

Conversation

@wihansaem1991

Copy link
Copy Markdown
Contributor

tests/wycheproof.rs runs the PKCS#1 v1.5 and PSS signature vectors, but no OAEP decryption vectors. As a result, the padding checks in src/algorithms/oaep.rs are not pinned down by the test suite: removing any one of first_byte_is_zero, !nonzero_before_one or !looking_for_index from the valid computation leaves cargo test --release green.

This adds test_rsa_oaep_decrypt, which runs the 22 rsa_oaep_*_test.json files from the existing Wycheproof submodule (no submodule bump): 1098 vectors, with SHA-1, SHA-2 and SHA-512/224, SHA-512/256 used for the label hash and for MGF1. Valid vectors must decrypt to the expected message and invalid ones must be rejected.

Results:

valid computation in oaep.rs existing tests test_rsa_oaep_decrypt
unchanged pass 22/22 files pass
without first_byte_is_zero pass 21/22 files fail
without !nonzero_before_one pass 21/22 files fail
without !looking_for_index pass 21/22 files fail

(rsa_oaep_misc_test.json holds only valid vectors, so it still passes under those changes.)

The rsa_three_primes_oaep_* files are not included because multi-prime keys are not supported. Summary::fail now takes any Debug value so it can report the decryption result.

Checked with cargo test --release --test wycheproof (default features and --no-default-features --features encoding), cargo clippy --all -- -D warnings and cargo fmt --all -- --check. The new cases add about 9 seconds in release mode.

This test was written with an AI assistant (Claude Code), which also ran the tests and the mutations above.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Sjtw7YxFWFKBH5rRZfwi9T

tests/wycheproof.rs runs the PKCS#1 v1.5 and PSS signature vectors but
no OAEP decryption vectors. Dropping `first_byte_is_zero`,
`!nonzero_before_one` or `!looking_for_index` from the `valid`
computation in src/algorithms/oaep.rs leaves the whole test suite
green.

Run the 22 rsa_oaep_*_test.json files from the Wycheproof submodule:
1098 vectors with SHA-1, SHA-2 and SHA-512/224, SHA-512/256 for the
label hash and MGF1. Each of the three changes above makes 21 of the 22
files fail. The rsa_three_primes_oaep_* files are left out because
multi-prime keys are not supported.

Assisted-by: Claude Code:claude-opus-5-5

@tarcieri tarcieri left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@tarcieri
tarcieri merged commit 8f823d4 into RustCrypto:master Nov 1, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants