Repository navigation
perf(tdigest): simplify rank anchor lookup - #290
Merged
Merged
Conversation
Select adjacent centroids directly when a query falls between means and the full equal-mean run for an exact match. Only exact matches need a second partition search. Remove the custom binary-search comparators and insertion-point corrections while retaining the existing weight accumulation and interpolation. Cover weighted repeated means in integration tests and query benchmarks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Rank queries used two custom binary-search comparators followed by insertion-point corrections to select their interpolation anchors. Use
partition_pointto express the two cases directly: adjacent centroids between distinct means, or the first and last centroids in an equal-mean run. Only an exact match needs the second search.The existing tail checks establish the index bounds. Weight accumulation and interpolation remain unchanged. This removes two comparator helpers and 24 lines from the implementation, adds one weighted-repeat boundary test and repeated-mean query benchmarks, and updates the changelog.
Validation
cargo x check,cargo x test(746 tests), andcargo x lintpass.cargo bench --package benchmarks --bench benchmarks -- tdigest::query::rank_repeated_meanspasses for all four query values.48165edcover 4,800 digest states and 6,220,288 query points. Mutable and frozen rank, CDF, PMF, batch quantile results, and serialized bytes match bit-for-bit. Cases include weighted ties, signed zero, extreme finite values, weights nearu64::MAX, updates, borrowed merges, and owned collection.Performance
Release-mode external probes link the base and candidate libraries into the same executable on Apple ARM64 (
rustc 1.99.0-nightly,3d6c19bb9). Each result is the median of nine rounds with alternating execution order. Setup is outside timing; scalar-query rounds use 500,000 calls, and CDF rounds use 50,000 calls.Unless noted, digests use
k = 200and 100,000 input values. Uniform inputs retain 264 centroids; repeated inputs cycle through values 0 through 7 and retain 248. Mixed queries use a deterministic mix of centroid means and interval midpoints.k = 2000, mixed rank queriesThe smaller-query and larger-
kregressions are retained explicitly: this is a simpler anchor lookup with workload-dependent performance, not a universal speedup. Scalar rank still sums preceding weights, so its worst-case complexity remains linear in the retained centroid count. These measurements do not imply an application-wide improvement.