Skip to content

perf: compare decimal and string bounds in the Arrow data buffer when gathering cache statistics - #7237

Open
andygrove wants to merge 23 commits into
apache:mainfrom
andygrove:perf/cache-stats-strings
Open

andygrove wants to merge 23 commits into
apache:mainfrom
andygrove:perf/cache-stats-strings

Conversation

@andygrove

@andygrove andygrove commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Which issue does this PR close?

Part of #6834.

This PR now also holds #7235 (decimal bounds), which is closed in favor of it, so that the two statistics changes are reviewed together. sunchao's review of #7235 at 158e105a71 still applies to the decimal commits.

It is stacked on #6988 and holds its commits as well. The change here is the decimal commits from #7235 (e30bc831b9, 8a9394fdaa, f20e6e123e, 158e105a71, plus 4b2656ee46, which registers ArrowCachedBatchStatsSuite in the CI suite lists) and the string commits (011718272e, 0f01c13981, 70eb4dcaeb). The rest are #6988's, and the diff shrinks to these once #6988 lands.

Rationale for this change

ArrowCachedBatchSerializer.gatherColumnStats records a lower and an upper bound for each column of every cached batch. For decimal and string columns this allocated per row, and #6834 lists both as the next steps after the null counts of #6853:

  • Decimals. gatherDecimalStats called CometVector.getDecimal, which for a precision past 18 digits builds a BigInteger and a BigDecimal from the 16 bytes in Arrow's data buffer, and then a Decimal around them. Most of the statistics time of a decimal column was that allocation.
  • Strings. gatherStringStats called getUTF8String, which allocates a UTF8String, and compared it through a boxed Ordering[Any]. After the decimal change, that left strings as the most expensive column type in the statistics loop.

What changes are included in this PR?

Decimal bounds

  • gatherDecimalStats reads a plain Arrow decimal column's 128-bit unscaled values straight from its data buffer and compares them without building anything per row. A value is a signed high word over an unsigned low word, little-endian. A narrow decimal is the sign extension of its low word, so one comparison covers every precision. All the values of a column share one scale, so their unscaled order is their numeric order. Only the rows of the two bounds become a Decimal, through the same getDecimal as before, so the bounds keep the type they had.
  • A decimal column that is not a plain Arrow vector (a dictionary-encoded one, for instance) takes the earlier by-value loop, which is kept as gatherDecimalStatsByValue.

String bounds

  • For a plain 32-bit Arrow string vector under the default StringType, gatherStringStats compares the bytes in the vector's data buffer and builds no UTF8String per row. The comparison is that of UTF8String.compareTo: eight bytes at a time as big-endian unsigned words, then byte by byte as unsigned values, and a shorter string sorts before a longer one it prefixes. A row is compared with the lower bound first, and with the upper bound only when it is not below it. Only the rows of the two bounds become UTF8Strings, as owned copies.
  • A collated string, a large-offset vector and a column that is not a plain Arrow vector keep the earlier loop, as gatherStringStatsByValue, and so keep comparing through the ordering boundsOrderings resolved for the column. The default collation is recognised by dataType == StringType, which a collated StringType is not equal to, so no version shim is needed.

Benchmark

CometInMemoryCacheBenchmark gets a decimal case in its statistics benchmark (precisions 10, 18 and 38 at scale 2, one row in ten null, values on both sides of zero) and string cases.

How are these changes tested?

New ArrowCachedBatchStatsSuite, registered in both pr_build_linux.yml and pr_build_macos.yml.

Decimals: it builds Arrow DecimalVectors at precisions 5, 10, 18, 19, 20, 30 and 38 with a range of scales, and compares the bounds and null count with a BigDecimal minimum and maximum:

  • values on either side of each word boundary of a 128-bit value (the low word's sign bit, a carry into the high word, both signs), in three orders;
  • the same values without the largest magnitudes, in positive-only and negative-only sets, because at the full range the high word alone decides the bounds and the low word's comparison never matters;
  • random values of random widths, with and without nulls, and single-value columns.

Comparing the low word as signed instead of unsigned fails the suite at 19, 20, 30 and 38 digits.

Strings: four tests compare the bounds with the minimum and maximum of the strings' UTF-8 bytes under an unsigned byte-wise order:

  • lengths on both sides of the 8-byte words, a shared prefix of every length, a value that is a prefix of another, and one- to four-byte characters whose bytes are at or above 0x80, in three orders, with duplicates and nulls at the start, between and at the end, and single-value columns;
  • the values that sort after and before a long shared prefix, so that the byte after it decides the bound;
  • random strings over a small alphabet of one- and two-byte characters, so that many share long prefixes, with nulls;
  • a large-offset vector, which takes the fallback loop.

Each of these fails if the comparison is changed: treating bytes as signed, comparing the words in native byte order, or dropping the tie-break on length.

CometInMemoryCacheSuite, CacheBatchCoalescerSuite and ArrowCachedBatchStatsSuite pass on Spark 4.1 (108 tests), and the stats and cache suites pass on 3.5. Spotless and the strict-warnings build pass, and dev/ci/check-suites.py finds every suite in both workflow files.

Benchmark (CometInMemoryCacheBenchmark --stats-only, Apple M3 Max, local[1], 5M rows, Arrow vectors). The "before" arm in each case is the by-value loop that this PR keeps as the fallback.

Decimals, run old, new, old in one session so that drift would show:

Collector Time for 5M rows
before (by value) 153 ms
after (data buffer) 21 ms
before, again 154 ms

Strings, before and after alternated three times in one session, medians of the best time:

Case before after
3 long and 3 string columns 171 ms 74 ms
3 string columns, keys of 55 to 75 bytes sharing their first 40 to 50 (URLs, paths) 227 ms 163 ms
3 string columns, keys of 1 to 6 bytes that differ in the first word 130 ms 53 ms
3 decimal columns, as a control 20 ms 20 ms

The string gain is smaller where a comparison reads five or six words before it finds a difference, and it is larger on short keys. The absolute times differ from one session to the next with the load on the machine.

gatherColumnStats visited every row to count nulls, including for binary,
array, map and struct columns, which record a null count and no bounds.
A CometDecodedVector already holds that count, taken from Arrow when the
vector is built. Use it when the vector holds exactly the batch's rows: a
column without bounds records it without a pass over its rows, and an
all-null column of any type skips its bounds loop.

Spark's own vectors are still counted row by row. Its vectorized Parquet
reader marks a column missing from a file all-null without counting any
nulls, so numNulls is 0 there while every row isNullAt.

The statistics benchmark now measures Comet's Arrow vectors as well as
Spark's, and binary and nested columns.

Part of apache#6834.
Spark's DefaultCachedBatch records the same null counts and no bounds
for binary, nested and void columns. nullCountsWithoutBounds now checks
that the relation is stored as CometCachedBatch before reading its
statistics, so the row write path and native scan legs check the format
as the Spark vectorized reader leg already did.
encodeBatches now converts a batch that is not written directly before it
gathers the statistics, so every column the collector reads is a
CometDecodedVector holding the null count Arrow took when it was built. The
row-by-row null count for Spark's vectors and the check that a vector holds
exactly the batch's rows go, and so do the test of that check and its helper.
The statistics benchmark measures only Arrow vectors, which is all the
collector reads now.
The columnar cache write stored each input batch as one cached batch,
and nothing bounds how small those batches are. A shuffle read returns
at least one batch per map output block, so a relation cached straight
from a shuffle over 200 map tasks into 50 partitions held 10000 batches
of about 500 rows, each with its own message metadata, compression
framing and statistics row.

CacheBatchCoalescer now sits in front of the columnar write. An
Arrow-backed batch laid out the way the reader reads it is passed
through when it holds at least half of
spark.sql.inMemoryColumnarStorage.batchSize, so full native batches are
still written without a copy. Every other batch is copied through
ArrowWriter.writeColumns into batches of batchSize rows, the size the
row path cuts at: a small batch is worth the copy, and a batch of Spark
column vectors was already copied into Arrow. Row order is kept, and
each batch is copied before the input advances, because a native
producer releases the batch it returned last when asked for the next.

Closes apache#6950.
The cache statistics now come from the Arrow batch that is written, so a
vector longer than its batch would count nulls past the batch's last row
and could record the batch all null. Require each vector to hold exactly
the batch's rows to take the direct path, and convert otherwise.

@sunchao sunchao left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 0f01c13981bb126541b6c3a7881dc9d4baf88a97 across correctness, compatibility, performance, design and test coverage, including its prerequisites. I found no new runtime defect in the string optimization. Unsigned byte ordering, prefix handling, collation dispatch and ownership of the final bounds match the existing contracts. I did not independently reproduce the benchmark timings.

Approval is pending validation. This head inherits the missing ArrowCachedBatchStatsSuite registration from #7235, where I left the registration comment. I reproduced dev/ci/check-suites.py exiting with code 255 here, and the current-head Preflight job reports the same missing suite. The normal build/test jobs were skipped. Please carry the registration fix into this stack and get those checks running.

The coalescer queued every fragment of a large Spark-vector batch before returning the first, so a small spark.sql.inMemoryColumnarStorage.batchSize multiplied the Arrow allocation of one input. It now keeps the batch it is copying and a row offset, and copies only as far as one output batch each time it is asked for one.
# Conflicts:
#	spark/src/main/scala/org/apache/spark/sql/comet/execution/arrow/ArrowCachedBatchSerializer.scala
#	spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala
#	spark/src/test/scala/org/apache/spark/sql/benchmark/CometInMemoryCacheBenchmark.scala
@andygrove

Copy link
Copy Markdown
Member Author

The registration of ArrowCachedBatchStatsSuite is in the suite lists of both pr_build_linux.yml and pr_build_macos.yml now. It comes in through #7235 (4b2656e), and dev/ci/check-suites.py exits 0 on this head (d8ea75a). I also merged apache/main and the updated #6988 in, which cleared the conflict left by #6853 being squash-merged.

@comphead comphead left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing 8-byte words byte-reversed and unsigned, then the tail bytes and the length, gives the same order as UTF8String.compareTo, absolute offsets keep sliced vectors correct, and the dt == StringType guard sends collated and char/varchar types to the by-value path on 4.x. One question on the evidence, outside the diff. The 161 to 63 ms figure comes from the existing mixed case in CometInMemoryCacheBenchmark, and the description says the strings' share is not measured separately. Those strings are 9 to 12 bytes with a 6-byte shared prefix (str_a_ plus digits), so compareBytes never gets past its first word, and the longer shared prefixes the word loop is meant for are not exercised. #7235 added a dedicated decimal case to that benchmark. Could a string-only case with long shared prefixes, such as URL or path keys, and one with short distinct keys back the claim for those shapes?

@andygrove

Copy link
Copy Markdown
Member Author

Added both cases to CometInMemoryCacheBenchmark --stats-only (70eb4dcaeb): three string columns with 55 to 75 byte keys that share their first 40 to 50 bytes (URLs, file paths and S3 keys), and three with 1 to 6 byte keys that differ in the first word. Before, after, before, after, before, after in one session, medians of the best time over 5M rows.

The long-prefix case goes from 227 ms to 163 ms (28% less) and the short-key case from 130 ms to 53 ms (59% less). The gain is smaller where the comparison has to read five or six words before it finds a difference, which is what you expected, but it still pays. The decimal control stayed at 20 ms in every run. The description has the table.

@andygrove andygrove changed the title perf: compare string bounds in the Arrow data buffer when gathering cache statistics perf: compare decimal and string bounds in the Arrow data buffer when gathering cache statistics Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants