Skip to content

Validate header fields and sizes when deserializing CPC, Count-Min, Theta (v4) and VarOpt - #538

Merged
leerho merged 1 commit into
masterfrom
deserialize-validation
Oct 4, 2026
Merged

leerho merged 1 commit into
masterfrom
deserialize-validation

Conversation

@leerho

@leerho leerho commented Oct 3, 2026

Copy link
Copy Markdown
Member

Following #536, this applies the same kind of input validation to more deserializers, so that malformed or truncated serialized sketches are rejected with an exception before any field is read or used:

  • VarOpt union: require the full non-empty preamble before reading it.
  • Count-Min: include the preamble in the size check, and compute the table size in 64 bits.
  • Compact Theta, serial version 4: compute the packed data size in 64 bits, and validate the entry-bits and num-entries-bytes header fields, for both byte and stream deserialization.
  • CPC: validate lg_k, the coupon count and the number of table entries; bound decoding by the size of the compressed data; check decoded row and column indices; check the input size before allocating.

Each change has regression tests. All test suites pass, including under AddressSanitizer.

This is the last change planned for 5.3.0.

🤖 Generated with Claude Code

…heta and VarOpt

- var_opt_union: require the full non-empty preamble before reading it.
- count_min_sketch: include the preamble in the deserialized size check;
  compute num_buckets * num_hashes in 64 bits.
- compact theta, serial version 4: compute the packed data size in 64
  bits, and require entry bits in [1, 63] and num entries bytes in
  [1, 4], for both byte and stream deserialization.
- cpc_sketch: validate lg_k, num_coupons and the number of table entries;
  bound decoding by the number of compressed words; require decoded rows
  and columns to be in range; check the input size before allocating.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 37163065438

Coverage increased (+0.03%) to 82.375%

Details

  • Coverage increased (+0.03%) from the base build.
  • Patch coverage: 2 uncovered changes across 2 files (39 of 41 lines covered, 95.12%).
  • No coverage regressions found.

Uncovered Changes

File Changed Covered %
cpc/include/cpc_sketch_impl.hpp 12 11 91.67%
theta/include/compact_theta_sketch_parser_impl.hpp 12 11 91.67%
Total (6 files) 41 39 95.12%

Coverage Regressions

No coverage regressions found.


Coverage Stats

Coverage Status
Relevant Lines: 21430
Covered Lines: 17653
Line Coverage: 82.38%
Coverage Strength: 1372976.55 hits per line

💛 - Coveralls

@leerho
leerho merged commit 412c47e into master Oct 4, 2026
32 checks passed
@leerho
leerho deleted the deserialize-validation branch October 4, 2026 00:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants