From e08216d776f11b53abdfaba4eb85b1b107cfbc39 Mon Sep 17 00:00:00 2001 From: dustin Date: Fri, 23 Oct 2026 10:17:58 +0000 Subject: [PATCH 1/3] fix: fall back when an outer hash join's streamed order cannot be checked (#7126) * fix: fall back when an outer hash join's streamed order cannot be checked An outer hash join that runs as DataFusion's right outer join keeps its unmatched probe rows in place only when the probe input declares an order. When the streamed side's reported order could not be checked, the serde declared none but still converted the join, so a sort Spark had already dropped above it was lost. This happened over a sorted localCheckpoint and over some caches written by an outer hash join. The join now falls back to Spark when that order reaches it through an input from the JVM or through a nested loop join. Over native operators in the same plan it stays native, since DataFusion keeps the order it can see there. * test: cover a right outer join over a cache of a right outer join The cache test now runs both shapes that run as DataFusion's right outer join, LeftOuter built on the right and RightOuter built on the left, in both cache formats. The nested loop join case notes that it only gives up the native join for nothing until #6898 is fixed. * fix: say that an outer hash join with an unchecked order stays in Spark The fallback reason still described the old behavior, where the join stayed native and declared no order. It now says the join stays in Spark, and the tests share it through one constant. The compatibility guide leads with when the join falls back and notes that a checkpoint reaches a native join only with spark.comet.convert.rdd.enabled. The two checkpoint tests share their setup. * docs: keep only the comments that explain when an outer join's order is trusted Drop comments that restate the code or a test's name, and shorten the rest to why an order from the JVM or a nested loop join can't be used. (cherry picked from commit c56f6a0209dba0600103a37fac7ca9b006639857) --- .../adding_a_new_operator.md | 5 +- .../latest/compatibility/operators.md | 19 +- .../apache/spark/sql/comet/operators.scala | 58 ++++-- .../comet/exec/CometInMemoryCacheSuite.scala | 98 +++++++++- .../apache/comet/exec/CometJoinSuite.scala | 183 +++++++++++++++++- 5 files changed, 336 insertions(+), 27 deletions(-) diff --git a/docs/source/contributor-guide/adding_a_new_operator.md b/docs/source/contributor-guide/adding_a_new_operator.md index 0706e71f7de..c54b47504b4 100644 --- a/docs/source/contributor-guide/adding_a_new_operator.md +++ b/docs/source/contributor-guide/adding_a_new_operator.md @@ -372,7 +372,10 @@ ordering must keep it natively, and the native operator must say so by returning `maintains_input_order` for that child, as DataFusion's filter and projection do. The planner declares the order through an operator only when the operator says it keeps it. An operator that cannot keep the order natively must not report it, or has to be added to `mayReportFalseOrder` in -`operators.scala`, which makes the hash join serde send no order over it. +`operators.scala`. The hash join serde then sends no order over it, and an outer hash join above it +that has to keep that order falls back to Spark. Comet's hash joins are the exception within one +native plan: an outer hash join over one of them stays native, even over one that declared no +order, since DataFusion keeps the order it can see there. A leaf that reports an ordering is where the order starts, so the planner has to know that the native operator produces its rows in that order. It knows that of an input from the JVM and of a diff --git a/docs/source/user-guide/latest/compatibility/operators.md b/docs/source/user-guide/latest/compatibility/operators.md index 01d2a2ef675..8785874c6f6 100644 --- a/docs/source/user-guide/latest/compatibility/operators.md +++ b/docs/source/user-guide/latest/compatibility/operators.md @@ -132,12 +132,19 @@ express falls back to Spark and records a fallback reason (see [Understanding Comet Plans](../understanding-comet-plans.md)), because Comet cannot tell whether an operator above the join relies on the order. -Where the order Spark reports might not hold, Comet declares nothing and an outer join can return -its unmatched rows out of order. The output of a nested loop join -([#6898](https://github.com/apache/datafusion-comet/issues/6898)) is one such case, and so is a -cache or checkpoint that one wrote. Comet also declares nothing when it cannot follow the -order back to its source, as with an RDD checkpoint, and over some caches written by an outer hash -join whose order does hold ([#6922](https://github.com/apache/datafusion-comet/issues/6922)). +An outer join falls back to Spark when Comet cannot check the order Spark reports and that order +reaches the join through an input from the JVM or through a nested loop join. Spark keeps the rows +in the order they arrive. One such input is an RDD checkpoint. It reaches a native join only with +`spark.comet.convert.rdd.enabled=true`, which defaults to `false`. Without that setting the +checkpoint stays a Spark input and the join stays in Spark anyway. Another is a cache written by an +outer hash join whose own probe input was not a scan +([#6922](https://github.com/apache/datafusion-comet/issues/6922)). The output of a nested loop +join can already be out of order when it reaches the join +([#6898](https://github.com/apache/datafusion-comet/issues/6898)). + +When the order Comet cannot check comes only from another native hash join in the same plan, the +join does not fall back. It runs natively and declares no order, since DataFusion keeps the order +it can see there. ## Window Functions diff --git a/spark/src/main/scala/org/apache/spark/sql/comet/operators.scala b/spark/src/main/scala/org/apache/spark/sql/comet/operators.scala index b17441d76f5..a05ca87d2e6 100644 --- a/spark/src/main/scala/org/apache/spark/sql/comet/operators.scala +++ b/spark/src/main/scala/org/apache/spark/sql/comet/operators.scala @@ -2821,6 +2821,13 @@ case class CometSortAggregateExec( this.copy(child = newChild) } +object CometHashJoin { + + private[apache] val uncheckedStreamedOrderReason: String = + "the streamed side's sort order, which this outer join has to keep, cannot be checked, " + + "so the join stays in Spark" +} + trait CometHashJoin { // Only BroadcastHashJoinExec can be null-aware (NOT IN subqueries). @@ -2948,7 +2955,8 @@ trait CometHashJoin { // Declaring the order there would let an operator above rely on it, an // aggregate in sorted mode for instance, which then splits groups. So nothing is declared // when such a join feeds the streamed side with no sort in between, or when a leaf reports - // an order that cannot be checked; see mayReportFalseOrder. + // an order that cannot be checked; see mayReportFalseOrder. When it comes from the JVM or a + // nested loop join, the join stays in Spark, as Spark may have dropped a sort above it. // // Declaring any order makes DataFusion keep the probe rows in place, so the longest prefix // of the order that Comet can express is enough. A key that exprToProto takes counts as @@ -2957,9 +2965,14 @@ trait CometHashJoin { // check `supportedSortType` makes for a sort. Nothing here can tell whether an operator above // relies on the order, so a join whose first key Comet cannot express stays in Spark. val (streamed, _) = streamedSide(join.buildSide, join.left, join.right) + val keepsStreamedOrder = + runsAsRightOuterJoin(join.joinType, join.buildSide) && join.outputOrdering.nonEmpty + if (keepsStreamedOrder && mayReportFalseOrder(streamed, countInPlan = false)) { + withFallbackReason(join, CometHashJoin.uncheckedStreamedOrderReason) + return None + } val streamedSortOrders = - if (runsAsRightOuterJoin(join.joinType, join.buildSide) && - join.outputOrdering.nonEmpty && !mayReportFalseOrder(streamed)) { + if (keepsStreamedOrder && !mayReportFalseOrder(streamed, countInPlan = true)) { val orders = join.outputOrdering.iterator .takeWhile(order => !hasNonDefaultStringCollation(order.dataType)) .map(exprToProto(_, streamed.output)) @@ -3012,11 +3025,16 @@ trait CometHashJoin { // order, but the scan reads the final plan behind it once there is one. While CometExecRule // converts a plan, a native plan's JVM inputs sit behind a CometScanWrapper, a leaf, so the // walk looks through that too. - private def mayReportFalseOrder(plan: SparkPlan): Boolean = plan match { + // + // With `countInPlan` off, a Comet hash join in the same native plan does not count: DataFusion + // sees its probe input's order itself. Past an input from the JVM, every join counts again. + private def mayReportFalseOrder(plan: SparkPlan, countInPlan: Boolean): Boolean = plan match { case adaptive: AdaptiveSparkPlanExec if adaptive.isFinalPlan => - mayReportFalseOrder(adaptive.executedPlan) + mayReportFalseOrder(adaptive.executedPlan, countInPlan) case _ if plan.outputOrdering.isEmpty => false case _: SortExec | _: CometSortExec => false + // Its rows stay in order only over a one-row build side, and a native join above would move + // them. Over more build rows they are already out of order (#6898); drop this once fixed. case _: CometBroadcastNestedLoopJoinExec => true case join: CometBroadcastHashJoinExec => hashJoinMayReportFalseOrder( @@ -3024,40 +3042,46 @@ trait CometHashJoin { join.joinType, join.buildSide, join.left, - join.right) + join.right, + countInPlan) case join: CometHashJoinExec => hashJoinMayReportFalseOrder( join.nativeOp, join.joinType, join.buildSide, join.left, - join.right) + join.right, + countInPlan) case join: HashJoin => - mayReportFalseOrder(streamedSide(join.buildSide, join.left, join.right)._1) - case wrapper: CometScanWrapper => mayReportFalseOrder(wrapper.originalPlan) + mayReportFalseOrder(streamedSide(join.buildSide, join.left, join.right)._1, countInPlan) + case wrapper: CometScanWrapper => + mayReportFalseOrder(wrapper.originalPlan, countInPlan = true) case scan: CometInMemoryTableScanExec => - mayReportFalseOrder(scan.originalPlan.relation.cachedPlan) - case scan: InMemoryTableScanExec => mayReportFalseOrder(scan.relation.cachedPlan) - case stage: QueryStageExec => mayReportFalseOrder(stage.plan) + mayReportFalseOrder(scan.originalPlan.relation.cachedPlan, countInPlan = true) + case scan: InMemoryTableScanExec => + mayReportFalseOrder(scan.relation.cachedPlan, countInPlan = true) + case stage: QueryStageExec => mayReportFalseOrder(stage.plan, countInPlan) // A scan reports an order that Spark already relies on. case _: DataSourceScanExec | _: DataSourceV2ScanExecBase | _: RangeExec | _: CometLeafExec => false case _ if plan.children.isEmpty => true - case _ => plan.children.exists(mayReportFalseOrder) + case _ => plan.children.exists(mayReportFalseOrder(_, countInPlan)) } // Whether a Comet hash join that reports an order may not keep it. Only the streamed side's // order reaches the join's output, so a false order on the build side is of no concern, and an - // outer join of the two shapes has to have declared the order on its probe input as well. + // outer join of the two shapes has to have declared the order on its probe input if counted. private def hashJoinMayReportFalseOrder( nativeOp: Operator, joinType: JoinType, buildSide: BuildSide, left: SparkPlan, - right: SparkPlan): Boolean = { + right: SparkPlan, + countInPlan: Boolean): Boolean = { val (streamed, streamedIndex) = streamedSide(buildSide, left, right) - mayReportFalseOrder(streamed) || - (runsAsRightOuterJoin(joinType, buildSide) && !declaresStreamedOrder(nativeOp, streamedIndex)) + mayReportFalseOrder(streamed, countInPlan) || + (countInPlan && runsAsRightOuterJoin(joinType, buildSide) && + !declaresStreamedOrder(nativeOp, streamedIndex)) } // Whether the native planner declares the streamed order on the probe input of the join that diff --git a/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala b/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala index 1019d88f2ce..ccc5e5e6c20 100644 --- a/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala +++ b/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala @@ -37,9 +37,9 @@ import org.apache.spark.CometDriverPlugin import org.apache.spark.sql.{CometTestBase, DataFrame, Observation, QueryTest, Row} import org.apache.spark.sql.catalyst.expressions.{And, Ascending, Attribute, AttributeReference, EqualTo, Expression, GreaterThan, GreaterThanOrEqual, LessThan, Literal} import org.apache.spark.sql.catalyst.optimizer.{BuildLeft, BuildRight, BuildSide} -import org.apache.spark.sql.catalyst.plans.LeftOuter +import org.apache.spark.sql.catalyst.plans.{LeftOuter, RightOuter} import org.apache.spark.sql.columnar.{CachedBatch, SimpleMetricsCachedBatch} -import org.apache.spark.sql.comet.{CometBroadcastHashJoinExec, CometHashAggregateExec, CometHashJoinExec, CometInMemoryTableScanExec, CometSortAggregateExec, CometSortExec, CometSortMergeJoinExec} +import org.apache.spark.sql.comet.{CometBroadcastHashJoinExec, CometHashAggregateExec, CometHashJoin, CometHashJoinExec, CometInMemoryTableScanExec, CometSortAggregateExec, CometSortExec, CometSortMergeJoinExec} import org.apache.spark.sql.comet.execution.arrow.{ArrowCachedBatchSerializer, CometCachedBatchHelper} import org.apache.spark.sql.comet.execution.shuffle.CometCelebornShuffleManager import org.apache.spark.sql.comet.util.Utils @@ -3546,4 +3546,98 @@ class CometInMemoryCacheSuite extends CometTestBase { } } } + + test("an outer hash join over a cache of an outer hash join over a native filter keeps order") { + // The filter keeps the serde from seeing that the first join declared its order, so the + // second join, reading that join's cache from the JVM, cannot check the order. + for { + rightOuter <- Seq(false, true) + nativeCache <- Seq(true, false) + adaptive <- Seq(false, true) + } { + val (joinType, buildSide) = + if (rightOuter) (RightOuter, BuildLeft) else (LeftOuter, BuildRight) + def outerJoin(streamed: String, small: String, on: String): String = + if (rightOuter) s"$small RIGHT JOIN $streamed ON $on" + else s"$streamed LEFT JOIN $small ON $on" + withSQLConf( + SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> adaptive.toString, + SQLConf.SHUFFLE_PARTITIONS.key -> "2", + SQLConf.COLUMN_BATCH_SIZE.key -> "8", + CometConf.COMET_BATCH_SIZE.key -> "8", + CometConf.COMET_EXEC_IN_MEMORY_CACHE_ENABLED.key -> nativeCache.toString, + CometConf.COMET_CONVERT_FROM_IN_MEMORY_CACHE_ENABLED.key -> (!nativeCache).toString) { + withParquetTable((0 until 2000).map(i => ((i % 40) * 1000, i)), "big") { + withParquetTable((0 until 40 by 2).map(m => (m * 1000, m)), "small") { + withTempView("cached", "joined") { + try { + spark + .table("big") + .toDF("k", "v") + .repartition(2, $"k") + .sortWithinPartitions("k") + .cache() + .createOrReplaceTempView("cached") + spark.table("cached").count() + sql("SELECT /*+ BROADCAST(b) */ a.k, a.v, b._2 AS w FROM " + + outerJoin("(SELECT * FROM cached WHERE v >= 0) a", "small b", "a.k = b._1")) + .cache() + .createOrReplaceTempView("joined") + spark.table("joined").count() + val cachedPlan = cachedPlanOf(spark.table("joined")) + val context = s"$joinType nativeCache=$nativeCache AQE=$adaptive" + assert( + find(cachedPlan) { + case join: CometBroadcastHashJoinExec => + join.joinType == joinType && join.buildSide == buildSide && + join.nativeOp.getHashJoin.getStreamedSortOrdersCount > 0 + case _ => false + }.isDefined, + s"$context: the cache was not written by a native outer join that sent the " + + s"order:\n$cachedPlan") + assert( + partitionsInOrder(spark.table("joined"), keyOrder(Ascending)), + s"$context: the cached outer join rows are out of order") + + def df: DataFrame = sql("SELECT /*+ BROADCAST(c) */ j.k, j.v, c._2 FROM " + + outerJoin("joined j", "small c", "j.k = c._1")) + .sortWithinPartitions("k") + val (_, plan) = + checkSparkAnswerAndFallbackReason( + df, + CometHashJoin.uncheckedStreamedOrderReason) + val planContext = s"$context\n$plan" + if (nativeCache) { + assert( + collect(plan) { case s: CometInMemoryTableScanExec => s }.size == 1, + planContext) + } else { + assert( + collect(plan) { case s: InMemoryTableScanExec => s }.size == 1, + planContext) + assert( + collect(plan) { case s: CometInMemoryTableScanExec => s }.isEmpty, + planContext) + } + assert( + collect(plan) { case j: BroadcastHashJoinExec => + (j.joinType, j.buildSide) + } == Seq((joinType, buildSide)), + planContext) + assert( + collect(plan) { case j: CometBroadcastHashJoinExec => j }.isEmpty, + planContext) + assert( + collect(plan) { case s @ (_: CometSortExec | _: SortExec) => s }.isEmpty, + planContext) + assert(partitionsInOrder(df, keyOrder(Ascending)), planContext) + } finally { + spark.catalog.clearCache() + } + } + } + } + } + } + } } diff --git a/spark/src/test/scala/org/apache/comet/exec/CometJoinSuite.scala b/spark/src/test/scala/org/apache/comet/exec/CometJoinSuite.scala index 2f54f1b4991..d54ac56d5bc 100644 --- a/spark/src/test/scala/org/apache/comet/exec/CometJoinSuite.scala +++ b/spark/src/test/scala/org/apache/comet/exec/CometJoinSuite.scala @@ -35,7 +35,7 @@ import org.apache.spark.sql.catalyst.expressions.{And, Ascending, AttributeRefer import org.apache.spark.sql.catalyst.optimizer.{BuildLeft, BuildRight, BuildSide} import org.apache.spark.sql.catalyst.plans.{Inner, LeftOuter, LeftSemi} import org.apache.spark.sql.catalyst.plans.logical.Join -import org.apache.spark.sql.comet.{CometBroadcastExchangeExec, CometBroadcastHashJoinExec, CometBroadcastNestedLoopJoinExec, CometFilterExec, CometHashJoinExec, CometNativeScanExec, CometProjectExec, CometRangeExec, CometSampleExec, CometSortExec, CometSortMergeJoinExec, CometSparkToColumnarExec, CometUnionExec, CometWindowExec} +import org.apache.spark.sql.comet.{CometBroadcastExchangeExec, CometBroadcastHashJoinExec, CometBroadcastNestedLoopJoinExec, CometFilterExec, CometHashJoin, CometHashJoinExec, CometNativeScanExec, CometProjectExec, CometRangeExec, CometSampleExec, CometSortExec, CometSortMergeJoinExec, CometSparkToColumnarExec, CometUnionExec, CometWindowExec} import org.apache.spark.sql.execution.{ColumnarToRowTransition, InputAdapter, LocalTableScanExec, SortExec, SparkPlan, WholeStageCodegenExec} import org.apache.spark.sql.execution.adaptive.{AdaptiveSparkPlanExec, AQEShuffleReadExec, QueryStageExec} import org.apache.spark.sql.execution.aggregate.SortAggregateExec @@ -2709,6 +2709,127 @@ class CometJoinSuite extends CometTestBase { } } + // Takes the checkpoint with AQE off, as an adaptive plan reports no order. + private def withOrderedCheckpoint(f: DataFrame => Unit): Unit = + withSQLConf( + SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "false", + SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "-1", + SQLConf.SHUFFLE_PARTITIONS.key -> "2", + CometConf.COMET_CONVERT_FROM_RDD_ENABLED.key -> "true", + CometConf.COMET_EXEC_RANGE_ENABLED.key -> "true") { + f( + spark + .range(2000) + .selectExpr("id % 100 AS k", "id AS v") + .repartition(2, $"k") + .sortWithinPartitions("k", "v") + .localCheckpoint(eager = true)) + } + + test("outer hash joins over a checkpointed streamed side fall back and keep its order") { + // Spark drops the sort above the join for the checkpoint's order, which Comet cannot check. + withOrderedCheckpoint { streamed => + val build = spark.range(10).selectExpr("id * 10 AS k", "id AS w") + val order = + (0 to 1).map(i => SortOrder(BoundReference(i, LongType, nullable = true), Ascending)) + for (adaptive <- Seq(false, true)) + withSQLConf(SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> adaptive.toString) { + for { + (hint, sparkJoin) <- Seq( + "shuffle_hash" -> classOf[ShuffledHashJoinExec], + "broadcast" -> classOf[BroadcastHashJoinExec]) + joinType <- Seq("left_outer", "right_outer") + } { + def df: DataFrame = { + val on = streamed("k") === build("k") + val joined = if (joinType == "left_outer") { + streamed.join(build.hint(hint), on, joinType) + } else { + build.hint(hint).join(streamed, on, joinType) + } + joined + .select(streamed("k"), streamed("v"), build("w")) + .sortWithinPartitions("k", "v") + } + val (_, plan) = + checkSparkAnswerAndFallbackReason(df, CometHashJoin.uncheckedStreamedOrderReason) + val context = s"$hint $joinType AQE=$adaptive\n$plan" + assert(collect(plan) { case j if j.getClass == sparkJoin => j }.size == 1, context) + assert( + collect(plan) { + case j: CometHashJoinExec => j + case j: CometBroadcastHashJoinExec => j + }.isEmpty, + context) + assert( + collect(plan) { case s @ (_: CometSortExec | _: SortExec) => s }.isEmpty, + context) + assert(partitionsInOrder(df, order: _*), context) + } + } + } + } + + test("an outer hash join over a native inner join over a checkpoint falls back") { + // The order walk reaches the checkpoint through the inner join's streamed side. + withOrderedCheckpoint { streamed => + val d1 = spark.range(100).where("id % 4 != 3").selectExpr("id AS k", "id AS a") + val d2 = spark.range(10).selectExpr("id * 10 AS k", "id AS w") + val order = + (0 to 1).map(i => SortOrder(BoundReference(i, LongType, nullable = true), Ascending)) + def df: DataFrame = streamed + .join(d1.hint("broadcast"), streamed("k") === d1("k"), "inner") + .join(d2.hint("broadcast"), streamed("k") === d2("k"), "left_outer") + .select(streamed("k"), streamed("v"), d1("a"), d2("w")) + .sortWithinPartitions("k", "v") + val (_, plan) = + checkSparkAnswerAndFallbackReason(df, CometHashJoin.uncheckedStreamedOrderReason) + val inner = collect(plan) { case j: CometBroadcastHashJoinExec => j.joinType } + assert(inner == Seq(Inner), plan) + val outer = collect(plan) { case j: BroadcastHashJoinExec => j.joinType } + assert(outer == Seq(LeftOuter), plan) + assert(collect(plan) { case s @ (_: CometSortExec | _: SortExec) => s }.isEmpty, plan) + assert(partitionsInOrder(df, order: _*), plan) + } + } + + test("chained outer hash joins over a native window keep its order natively") { + // The window and both joins share one native plan, where DataFusion sees the order itself. + withParquetTable((0 until 2000).map(i => (i % 40, i)), "t") { + withParquetTable((0 until 40 by 2).map(k => (k, k * 10)), "d1") { + withParquetTable((0 until 40 by 3).map(k => (k, k * 100)), "d2") { + val order = (0 to 1).map { i => + SortOrder(BoundReference(i, IntegerType, nullable = true), Ascending) + } + for (adaptive <- Seq(false, true)) { + withSQLConf(SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> adaptive.toString) { + def df: DataFrame = sql("""SELECT w.k, w.v, w.rn, d1._2 AS x, d2._2 AS y + |FROM (SELECT k, v, row_number() OVER (PARTITION BY k ORDER BY v) AS rn + | FROM (SELECT _1 AS k, _2 AS v FROM t)) w + |LEFT JOIN d1 ON w.k = d1._1 + |LEFT JOIN d2 ON w.k = d2._1""".stripMargin) + .sortWithinPartitions("k", "v") + val (_, plan) = checkSparkAnswer(df) + val context = s"AQE=$adaptive\n$plan" + assert(new ExtendedExplainInfo().getFallbackReasons(plan).isEmpty, context) + val joins = collect(plan) { case j: CometBroadcastHashJoinExec => j } + assert(joins.size == 2 && joins.forall(_.joinType == LeftOuter), context) + assert(collect(plan) { case w: CometWindowExec => w }.size == 1, context) + assert( + collect(plan) { + case s @ (_: CometSortExec | _: SortExec) + if s.find(_.isInstanceOf[CometBroadcastHashJoinExec]).isDefined => + s + }.isEmpty, + context) + assert(partitionsInOrder(df, order: _*), context) + } + } + } + } + } + } + // Compares the rows of a fresh `df` with the ones Spark returns for it, as a multiset unless // `isSorted`. The rows of a key tie under a sort, so their order within it can differ between // the two. @@ -3183,6 +3304,66 @@ class CometJoinSuite extends CometTestBase { } } + test("chained outer hash joins over a native filter over a sorted cache stay native") { + // DataFusion sees the first join's order, in the same native plan, so the second sends none. + withSortedCache() { + def df: DataFrame = sql("""SELECT /*+ BROADCAST(b), BROADCAST(c) */ a.k, a.v, b._2, c._2 + |FROM (SELECT * FROM cached WHERE v >= 0) a + |LEFT JOIN small b ON a.k = b._1 + |LEFT JOIN small c ON a.k = c._1""".stripMargin) + .sortWithinPartitions("k") + val (_, plan) = checkSparkAnswer(df) + // Spark's cache format adds a reason of its own to the cache scan. + assert( + !new ExtendedExplainInfo().getFallbackReasons(plan).exists(_.contains("sort order")), + plan) + val joins = collect(plan) { case j: CometBroadcastHashJoinExec => j } + assert(joins.size == 2 && joins.forall(_.joinType == LeftOuter), plan) + val (first, second) = + joins.partition(_.left.find(_.isInstanceOf[CometBroadcastHashJoinExec]).isEmpty) + assert(first.size == 1 && second.size == 1, plan) + assert(collect(first.head.left) { case f: CometFilterExec => f }.size == 1, plan) + assert(first.head.nativeOp.getHashJoin.getStreamedSortOrdersCount > 0, plan) + assert(second.head.nativeOp.getHashJoin.getStreamedSortOrdersCount == 0, plan) + assert(collect(plan) { case s @ (_: CometSortExec | _: SortExec) => s }.isEmpty, plan) + assert(partitionsInOrder(df, keyOrder(Ascending)), plan) + } + } + + test("an outer hash join over a native nested loop join falls back") { + // Over two build rows the nested loop join emits rows out of order (#6898), so they are + // compared as a multiset. Over one build row they stay in order, and Spark's join keeps it. + withSortedCache() { + withParquetTable(Seq(Tuple1(0), Tuple1(1)), "two") { + withParquetTable(Seq(Tuple1(1)), "one") { + def query(nlBuild: String): String = + s"""SELECT /*+ BROADCAST(n), BROADCAST(b) */ a.k, a.v, n._1 AS z, b._2 AS w + |FROM cached a JOIN $nlBuild n ON a.v % 2 <= n._1 + |LEFT JOIN small b ON a.k = b._1""".stripMargin + val reason = CometHashJoin.uncheckedStreamedOrderReason + def checkFallback(df: => DataFrame): SparkPlan = { + sameRowsAsSpark(df) + val plan = df.queryExecution.executedPlan + val reasons = new ExtendedExplainInfo().getFallbackReasons(plan) + assert(reasons.exists(_.contains(reason)), plan) + val nestedLoopJoins = collect(plan) { case j: CometBroadcastNestedLoopJoinExec => j } + assert(nestedLoopJoins.size == 1, plan) + val outer = collect(plan) { case j: BroadcastHashJoinExec => j.joinType } + assert(outer == Seq(LeftOuter), plan) + plan + } + + checkFallback(sql(query("two"))) + + def ordered: DataFrame = sql(query("one")).sortWithinPartitions("k") + val plan = checkFallback(ordered) + assert(collect(plan) { case s @ (_: CometSortExec | _: SortExec) => s }.isEmpty, plan) + assert(partitionsInOrder(ordered, keyOrder(Ascending)), plan) + } + } + } + } + test("an outer hash join declares the streamed order over a semi join") { // `k IN (SELECT ...)` runs as a left semi join, which emits its probe rows in ascending // order whether or not the probe input declares an order. The outer join above it therefore From 5467f8b01738ba6c557e537c285f042415004a37 Mon Sep 17 00:00:00 2001 From: Oleks V Date: Fri, 6 Nov 2026 20:07:48 +0000 Subject: [PATCH 2/3] fix: declare the input order of a native window over a sorted cache (#7256) * fix: declare the input order of a native window over a sorted cache Spark puts no sort below a window when its input reports the order the window needs, a relation sorted before it was cached for instance. When those rows enter the native plan from the JVM, the ScanExec they arrive through declares no order, and the native window failed the task with "Expects PARTITION BY expression to be ordered". Send the window's required ordering in a new input_sort_orders field on the Window proto when the child reports an order that satisfies it and mayReportFalseOrder trusts that order. The planner declares it on the window's input with with_declared_order, as the hash join does for its probe input. A key that Comet cannot express, or that holds a collated string, sends the window back to Spark. Closes #7223. * refactor: build the window's input order only where it is used - Send no input order when a native sort below the window declares its own. - Build the order only once the window's own expressions have serialized, so a window that falls back for another reason does not walk the plan or record an extra reason. - Map a collated key straight to None instead of signalling it with an empty list. - Drop the empty-list branch in WindowBuilder, as with_declared_order already leaves the plan unchanged for no orders. * chore: trim comments and drop a window test the hash join test already covers * test: cover WindowAggExec, nulls, descending keys and the collated fallback From review: - The cache test also runs a frame up to UNBOUNDED FOLLOWING, which runs in WindowAggExec, over a nullable partition key and a descending order key, so the declared order has to match the cache's. - The planner test covers both window operators through the partition order lookup they share, since WindowAggExec fails only when its stream starts, not in partition_by_sort_keys. - A Spark 4.x test checks that a window over a cache sorted by a collated key stays in Spark for the new reason and returns Spark's rows. - The debugging guide's window snippet builds the operators over `input`. * fix: keep a stacked window's longer input order and declare it on empty buckets The window now sends the rest of the order its child reports past the required keys, up to the first that cannot be expressed, so a window above it in the same native plan sees the longer order. An empty bucket of a sorted bucketed scan is planned as an EmptyExec, which now counts as a source whose order holds. The cache test checks the declared columns, directions and null placements. * fix: declare a window's order next to a partial one its input declares Two aliases of one sorted column share an order in Spark, so a window over the window for one alias may need the other alias's order with no sort in between. with_declared_order now returns early only when the input's declared order already satisfies the one sent, and otherwise declares it next to the input's when keeps_source_order trusts the input. The serde checks only a suffix key's child against the child's output, since its same-order expressions are not sent and may name pruned columns. * refactor: send only a window's required order and merge overlapping tests with_declared_order extends a partial order at the upper window, so the serde no longer sends the keys a window's child reports past the ones the window requires. The two stacked-window tests become one over a single cache, the EmptyExec unit test folds into the extension test, and redundant assertions and comments are trimmed. * fix: declare only a window's partition keys and fall back on an unchecked order The native window checks only its partition keys' order, so the serde sends only those, and a float order key that Spark sorts with -0.0 and 0.0 as equal is never declared. When the order the child reports may be false past the native plan, the window stays in Spark, as the outer hash join does, instead of failing the task. The SortedInputExec doc and the operator guide now cover the window. * fix: keep a window's doubted order native, fall back on collated keys Falling back when the window's input order may be false handed the same rows to Spark's window, which trusts that order too and grouped them wrongly. The window again sends no order there, so the native window rejects its input as on main. A window whose partition or order key holds a collated string now stays in Spark, as WindowGroupLimit does, since the native window compares keys byte by byte. keeps_source_order accepts a right outer hash join whose probe declares an order, which is when DataFusion keeps its unmatched probe rows in place. * fix: fall back on a collated window key only over a trusted order Falling back hands the rows to Spark's window, which trusts the order the input reports. Over an order mayReportFalseOrder doubts, such as a native nested loop join's, that grouped the rows wrongly, so the window now stays native there, as on main, and the native window rejects the input. The outer join regression test partitions by k, which Spark reports through the alias a, so it needs the order declared next to the join's. * fix: sort the input of a Spark window over a doubted order Spark plans no sort below a window whose child reports the order it needs, and its window trusts that order. Over a Comet operator that may not keep it, such as a native nested loop join (#6898), a window that stays in Spark grouped rows out of order. CometExecRule now sorts that input again. With that, a window with a collated key falls back in every case, and so does a window with ORDER BY but no PARTITION BY over a doubted order, since the native window checks no order there. The compatibility guide lists collated keys among the window's fallbacks. * address review: run the nested loop join window test with AQE on, sort Python windows too (cherry picked from commit cd19831a7ea9130aae7b95882810e9b050a13aa7) --- .../adding_a_new_operator.md | 23 +- docs/source/contributor-guide/debugging.md | 7 +- .../latest/compatibility/operators.md | 3 + .../src/execution/operators/sorted_input.rs | 15 +- native/core/src/execution/planner.rs | 118 ++++++++- native/core/src/execution/planner/join.rs | 79 +++--- native/core/src/execution/planner/window.rs | 8 +- native/proto/src/proto/operator.proto | 3 + .../apache/comet/rules/CometExecRule.scala | 19 +- .../spark/sql/comet/CometWindowExec.scala | 54 +++- .../apache/spark/sql/comet/operators.scala | 5 + .../comet/exec/CometInMemoryCacheSuite.scala | 240 +++++++++++++++++- .../comet/exec/CometWindowExecSuite.scala | 38 ++- 13 files changed, 537 insertions(+), 75 deletions(-) diff --git a/docs/source/contributor-guide/adding_a_new_operator.md b/docs/source/contributor-guide/adding_a_new_operator.md index c54b47504b4..6317780def6 100644 --- a/docs/source/contributor-guide/adding_a_new_operator.md +++ b/docs/source/contributor-guide/adding_a_new_operator.md @@ -365,17 +365,18 @@ the order satisfies. Report what the Spark operator reports and nothing more. `S example, reports no order, so `CometSampleExec` must not report its child's. The claim has to hold at run time as well. A native hash join declares the order Spark reports for -its streamed side to DataFusion, and an aggregate above the join may then run in sorted mode, or a -top-k sort may stop reading early, on the strength of that order. If the order is false, the -result is wrong groups or wrong rows, not only rows out of order. So an operator that reports an -ordering must keep it natively, and the native operator must say so by returning `true` from -`maintains_input_order` for that child, as DataFusion's filter and projection do. The planner -declares the order through an operator only when the operator says it keeps it. An operator that -cannot keep the order natively must not report it, or has to be added to `mayReportFalseOrder` in -`operators.scala`. The hash join serde then sends no order over it, and an outer hash join above it -that has to keep that order falls back to Spark. Comet's hash joins are the exception within one -native plan: an outer hash join over one of them stays native, even over one that declared no -order, since DataFusion keeps the order it can see there. +its streamed side to DataFusion, and a native window the order of its partition keys. An aggregate +above may then run in sorted mode, or a top-k sort may stop reading early, on the strength of that +order. If the order is false, the result is wrong groups or wrong rows, not only rows out of order. +So an operator that reports an ordering must keep it natively, and the native operator must say so +by returning `true` from `maintains_input_order` for that child, as DataFusion's filter and +projection do. The planner declares the order through an operator only when the operator says it +keeps it. An operator that cannot keep the order natively must not report it, or has to be added to +`mayReportFalseOrder` in `operators.scala`. The hash join and window serdes then send no order over +it, an outer hash join above it that has to keep that order falls back to Spark, and a window that +runs in Spark above it gets its input sorted again. Comet's hash joins are the exception within one +native plan: an outer hash join over one of them stays native, even over one that declared no order, +since DataFusion keeps the order it can see there. A leaf that reports an ordering is where the order starts, so the planner has to know that the native operator produces its rows in that order. It knows that of an input from the JVM and of a diff --git a/docs/source/contributor-guide/debugging.md b/docs/source/contributor-guide/debugging.md index 8795854193d..51d74a1ea94 100644 --- a/docs/source/contributor-guide/debugging.md +++ b/docs/source/contributor-guide/debugging.md @@ -260,7 +260,8 @@ so `spark.comet.debug.enabled` and `COMET_LOG_LEVEL` do not affect it. #### Example: dumping window operator output In `WindowBuilder::build` in `native/core/src/execution/planner/window.rs`, -the window operator builds `window_agg`, either a `BoundedWindowAggExec` or a +the window operator builds `window_agg` over `input`, the child plan with +Spark's input order declared, either a `BoundedWindowAggExec` or a `WindowAggExec` depending on whether every window expression runs with bounded memory. It then optionally wraps `window_agg` in a casting `ProjectionExec` and builds the `SparkPlan` from either the projection or `window_agg`. To see the raw @@ -271,14 +272,14 @@ projection: let window_agg: Arc = if all_bounded { Arc::new(BoundedWindowAggExec::try_new( window_expr, - Arc::clone(&child.native_plan), + input, InputOrderMode::Sorted, !partition_exprs.is_empty(), )?) } else { Arc::new(WindowAggExec::try_new( window_expr, - Arc::clone(&child.native_plan), + input, !partition_exprs.is_empty(), )?) }; diff --git a/docs/source/user-guide/latest/compatibility/operators.md b/docs/source/user-guide/latest/compatibility/operators.md index 8785874c6f6..f10d01fe3ad 100644 --- a/docs/source/user-guide/latest/compatibility/operators.md +++ b/docs/source/user-guide/latest/compatibility/operators.md @@ -197,6 +197,9 @@ incorrect result. When any single window expression in a `WindowExec` falls back - `lag` / `lead` with a non-literal default value ([#4268](https://github.com/apache/datafusion-comet/issues/4268)). - A `ROWS` offset that is not an integer or long, or a `RANGE` offset that is not numeric. - Any `PARTITION BY` or `ORDER BY` expression that Comet cannot serialize. +- Any `PARTITION BY` or `ORDER BY` key whose type carries a non-default `StringType` collation (e.g. + `UTF8_LCASE`). The native window finds partitions and peers by comparing keys byte by byte, which splits values + that the collation ties. A `RANGE` frame with an explicit offset on a `DATE` `ORDER BY` runs natively, but over a date outside Spark's supported range of 0001-01-01 to 9999-12-31, such as one produced by `date_add` with a very large day count, the diff --git a/native/core/src/execution/operators/sorted_input.rs b/native/core/src/execution/operators/sorted_input.rs index a31053bd6e8..ad561cf49e3 100644 --- a/native/core/src/execution/operators/sorted_input.rs +++ b/native/core/src/execution/operators/sorted_input.rs @@ -37,13 +37,14 @@ use datafusion::physical_plan::{ /// sorted before it was cached for instance, but the `ScanExec` it arrives through declares none, /// and DataFusion's hash join keeps the unmatched probe rows of a right outer join in place only /// when the probe input declares an order. Otherwise it moves them after the matched rows of each -/// batch, which loses an order Spark has already dropped a sort for. The planner puts this only -/// on that join's probe input, with the order Spark reports for it, and only when the rows reach -/// that input from the JVM through operators that keep their order. The join keeps the order in -/// its own output, so the operators above it in the native plan see it too: an aggregate grouped -/// by the key runs in sorted, streaming mode, a sort aggregate takes the order as already -/// satisfied, and a top-k sort whose keys start with it stops reading early. A wrong order would -/// therefore give wrong groups or wrong rows, not only rows out of order. +/// batch, which loses an order Spark has already dropped a sort for. A window rejects an input +/// that does not declare its partition keys' order. The planner puts this only on that join's +/// probe input or that window's input, with the order Spark reports for it, and only when the +/// rows reach that input from the JVM through operators that keep their order. Both keep the +/// order in their own output, so the operators above them in the native plan see it too: an +/// aggregate grouped by the key runs in sorted, streaming mode, a sort aggregate takes the order +/// as already satisfied, and a top-k sort whose keys start with it stops reading early. A wrong +/// order would therefore give wrong groups or wrong rows, not only rows out of order. #[derive(Debug)] pub struct SortedInputExec { input: Arc, diff --git a/native/core/src/execution/planner.rs b/native/core/src/execution/planner.rs index 268f35570a6..e91c6a8f25f 100644 --- a/native/core/src/execution/planner.rs +++ b/native/core/src/execution/planner.rs @@ -4110,8 +4110,10 @@ mod tests { "col_0@0 ASC", probe.properties().output_ordering().unwrap().to_string() ); - // The join keeps the order in its own output, so the operators above it see it too. + // The join keeps the order in its own output, so the operators above it see it too, + // and an order declared above it holds, such as a window's for an alias of the key. assert!(planned.native_plan.properties().output_ordering().is_some()); + assert!(PhysicalPlanner::keeps_source_order(&planned.native_plan)); } } @@ -4279,6 +4281,48 @@ mod tests { assert!(!PhysicalPlanner::keeps_source_order(&scan(2))); } + #[test] + fn declared_order_extends_an_order_the_input_declares() { + // An order is declared over an empty bucket, and next to the order the input declares, + // unless an operator in between may have moved rows. + let schema = Arc::new(Schema::new(vec![ + Field::new("col_0", DataType::Int32, true), + Field::new("col_1", DataType::Int32, true), + ])); + let planner = PhysicalPlanner::default(); + let declare = |plan: &Arc, index: i32| { + planner + .with_declared_order( + Arc::clone(plan), + &[create_sort_order(index, 3, false, true)], + ) + .unwrap() + }; + let empty: Arc = + Arc::new(datafusion::physical_plan::empty::EmptyExec::new(schema)); + let by_col_0 = declare(&empty, 0); + assert_eq!("SortedInputExec", by_col_0.name()); + assert!(Arc::ptr_eq(&declare(&by_col_0, 0), &by_col_0)); + + let both = declare(&by_col_0, 1); + assert!(Arc::ptr_eq(both.children()[0], &by_col_0)); + for index in 0..2 { + let partition_by: Vec> = + vec![Arc::new(Column::new(&format!("col_{index}"), index))]; + let ordered = get_ordered_partition_by_indices(&partition_by, &both).unwrap(); + assert_eq!(1, ordered.len(), "col_{index}"); + } + + let sort: Arc = Arc::new(SortExec::new( + LexOrdering::new(vec![PhysicalSortExpr::new_default(Arc::new(Column::new( + "col_0", 0, + )))]) + .unwrap(), + empty, + )); + assert!(Arc::ptr_eq(&declare(&sort, 1), &sort)); + } + #[tokio::test] async fn spark_plan_dynamic_filter_preserves_metrics() { use crate::execution::metrics::utils::to_native_metric_node; @@ -4346,24 +4390,22 @@ mod tests { } } - #[tokio::test] - async fn window_cast_projection_reports_window_time() { - use crate::execution::metrics::utils::to_native_metric_node; + /// `func() OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW)`, returning + /// `result_type`. + fn window_function_expr( + func: &str, + result_type: Option, + ) -> spark_operator::WindowExpr { use spark_operator::{ lower_window_frame_bound::LowerFrameBoundStruct, upper_window_frame_bound::UpperFrameBoundStruct, CurrentRow, LowerWindowFrameBound, UnboundedPreceding, UpperWindowFrameBound, WindowExpr, WindowFrame, WindowFrameType, WindowSpecDefinition, }; - - let session = Arc::new(SessionContext::new()); - let planner = PhysicalPlanner::new(Arc::clone(&session), 0); - // `row_number` returns UInt64 natively and Spark expects Int32, so the planner adds a - // casting projection above the window. - let row_number = WindowExpr { + WindowExpr { built_in_window_function: Some(Expr { expr_struct: Some(ScalarFunc(spark_expression::ScalarFunc { - func: "row_number".to_string(), + func: func.to_string(), ..Default::default() })), query_context: None, @@ -4385,9 +4427,20 @@ mod tests { }), ..Default::default() }), - result_type: Some(create_proto_datatype()), + result_type, ..Default::default() - }; + } + } + + #[tokio::test] + async fn window_cast_projection_reports_window_time() { + use crate::execution::metrics::utils::to_native_metric_node; + + let session = Arc::new(SessionContext::new()); + let planner = PhysicalPlanner::new(Arc::clone(&session), 0); + // `row_number` returns UInt64 natively and Spark expects Int32, so the planner adds a + // casting projection above the window. + let row_number = window_function_expr("row_number", Some(create_proto_datatype())); let operator = Operator { children: vec![create_scan()], op_struct: Some(OpStruct::Window(Box::new(spark_operator::Window { @@ -4442,6 +4495,45 @@ mod tests { ); } + #[test] + fn window_declares_input_order_on_a_scan() { + // Both window operators need their partition keys' order, which a `ScanExec` declares + // only through the order Spark sent. `percent_rank` runs in `WindowAggExec`. + let partition_by: Vec> = vec![Arc::new(Column::new("col_0", 0))]; + for (func, operator) in [ + ("row_number", "BoundedWindowAggExec"), + ("percent_rank", "WindowAggExec"), + ] { + let window_input = |input_sort_orders| { + let window = Operator { + children: vec![create_scan()], + op_struct: Some(OpStruct::Window(Box::new(spark_operator::Window { + window_expr: vec![window_function_expr(func, None)], + partition_by_list: vec![create_bound_reference(0)], + order_by_list: vec![create_sort_order(0, 3, false, true)], + input_sort_orders, + ..Default::default() + }))), + ..Default::default() + }; + let (_, _, planned) = PhysicalPlanner::default() + .create_plan(&window, &mut vec![], 1) + .unwrap(); + assert_eq!(operator, planned.native_plan.name()); + Arc::clone(planned.native_plan.children()[0]) + }; + let input = window_input(vec![create_sort_order(0, 3, false, true)]); + assert_eq!("SortedInputExec", input.name()); + let ordered = get_ordered_partition_by_indices(&partition_by, &input).unwrap(); + assert_eq!(1, ordered.len(), "{func}"); + + let input = window_input(vec![]); + assert_eq!("ScanExec", input.name()); + let ordered = get_ordered_partition_by_indices(&partition_by, &input).unwrap(); + assert!(ordered.is_empty(), "{func}"); + } + } + #[test] fn projection_rolls_up_same_plan_id_child_metrics() { let op_scan = Operator { diff --git a/native/core/src/execution/planner/join.rs b/native/core/src/execution/planner/join.rs index 0b91f27187d..fb2949b8207 100644 --- a/native/core/src/execution/planner/join.rs +++ b/native/core/src/execution/planner/join.rs @@ -35,6 +35,7 @@ use datafusion::{ LexOrdering, PhysicalExpr, }, physical_plan::{ + empty::EmptyExec, filter::FilterExec, joins::{ utils::JoinFilter, HashJoinExec, NestedLoopJoinExec, PartitionMode, SortMergeJoinExec, @@ -402,27 +403,31 @@ impl PhysicalPlanner { Ok(Arc::new(prepared)) } - /// `plan`, declaring `sort_orders` as the order its rows arrive in, unless it already - /// declares one. Spark's plan knows the order of an input that enters the native plan from - /// the JVM, and of a scan of sorted files, but the `ScanExec` or the file scan the rows - /// arrive through declares none. Only that order is declared: when an operator in between may - /// have moved rows, `plan` is left as it is. + /// `plan`, declaring `sort_orders` as the order its rows arrive in, unless the order it + /// declares already satisfies them. Spark's plan knows the order of an input that enters the + /// native plan from the JVM, and of a scan of sorted files, but the `ScanExec` or the file + /// scan the rows arrive through declares none. Only that order is declared: when an operator + /// in between may have moved rows, `plan` is left as it is. /// - /// The serde sends an order only when the streamed side's Comet operators report it, and it - /// checks that nothing under them moves rows (`mayReportFalseOrder`), while this walk asks - /// the native operators the same thing through `maintains_input_order`. The two agree as - /// long as a Comet operator that reports an order keeps it natively and says so, and a Comet - /// leaf that reports one is a source this walk knows or declares its order itself, as the - /// native range does. One that does not lands here with `plan` unchanged, and the join then - /// moves its unmatched rows after Spark has dropped the sort above it. Nothing here can fall - /// back at that point, so the invariant is kept at the operator: see "Adding a New Operator" - /// in the contributor guide. - fn with_declared_order( + /// An input that declares only part of `sort_orders`, such as a window over another window, + /// gets them declared next to its own order. + /// + /// The serde sends an order only when the Comet operators of a join's streamed side or of a + /// window's input report it, and it checks that nothing under them moves rows + /// (`mayReportFalseOrder`), while this walk asks the native operators the same thing through + /// `maintains_input_order`. The two agree as long as a Comet operator that reports an order + /// keeps it natively and says so, and a Comet leaf that reports one is a source this walk + /// knows or declares its order itself, as the native range does. One that does not lands + /// here with `plan` unchanged, and the join then moves its unmatched rows after Spark has + /// dropped the sort above it, or the window rejects its input. Nothing here can fall back at + /// that point, so the invariant is kept at the operator: see "Adding a New Operator" in the + /// contributor guide. + pub(super) fn with_declared_order( &self, plan: Arc, sort_orders: &[Expr], ) -> Result, ExecutionError> { - if plan.output_ordering().is_some() || !Self::keeps_source_order(&plan) { + if sort_orders.is_empty() || !Self::keeps_source_order(&plan) { return Ok(plan); } let exprs = sort_orders @@ -430,8 +435,14 @@ impl PhysicalPlanner { .map(|expr| self.create_sort_expr(expr, plan.schema())) .collect::, _>>()?; match LexOrdering::new(exprs) { - Some(ordering) => Ok(Arc::new(SortedInputExec::new(plan, ordering))), - None => Ok(plan), + Some(ordering) + if !plan + .equivalence_properties() + .ordering_satisfy(ordering.iter().cloned())? => + { + Ok(Arc::new(SortedInputExec::new(plan, ordering))) + } + _ => Ok(plan), } } @@ -439,11 +450,12 @@ impl PhysicalPlanner { /// keep their input's order, so that the order Spark reports for them still holds. A source is /// an input from the JVM, or a scan of files, which reads the files of its one file group one /// after the other: Spark reports an order for a file scan only when every partition reads one - /// sorted file, as for a sorted bucketed table with one file per bucket. Any other operator may - /// have moved rows, such as the null-aware anti join, whose streamed side Comet builds on, and - /// declaring Spark's order over it would be wrong. + /// sorted file, as for a sorted bucketed table with one file per bucket. An `EmptyExec`, a + /// bucket with no file, has no rows to be out of order. Any other operator may have moved + /// rows, such as the null-aware anti join, whose streamed side Comet builds on, and declaring + /// Spark's order over it would be wrong. pub(super) fn keeps_source_order(plan: &Arc) -> bool { - if plan.is::() || plan.is::() { + if plan.is::() || plan.is::() || plan.is::() { return true; } if let Some(scan) = plan.downcast_ref::() { @@ -453,18 +465,21 @@ impl PhysicalPlanner { .is_some_and(|config| config.file_groups.len() == 1); } // DataFusion reports every right hash join as keeping its probe order, but a right outer - // join keeps it only when the probe declares one, and no input on this path declares an - // order. An inner join emits its matches probe row by probe row, and a right semi, anti or - // mark join builds its probe indices in ascending order, so none of them depends on a - // declaration. Any other join type keeps no probe order at all. + // join keeps its unmatched probe rows in place only when the probe declares an order, as + // the planner does for the join's streamed side. An inner join emits its matches probe + // row by probe row, and a right semi, anti or mark join builds its probe indices in + // ascending order, so none of them depends on a declaration. Any other join type keeps no + // probe order at all. if let Some(join) = plan.downcast_ref::() { - if !matches!( - join.join_type(), + let keeps_probe_order = match join.join_type() { DFJoinType::Inner - | DFJoinType::RightSemi - | DFJoinType::RightAnti - | DFJoinType::RightMark - ) { + | DFJoinType::RightSemi + | DFJoinType::RightAnti + | DFJoinType::RightMark => true, + DFJoinType::Right => join.right().output_ordering().is_some(), + _ => false, + }; + if !keeps_probe_order { return false; } } diff --git a/native/core/src/execution/planner/window.rs b/native/core/src/execution/planner/window.rs index 85499fac4bd..d7aded1f710 100644 --- a/native/core/src/execution/planner/window.rs +++ b/native/core/src/execution/planner/window.rs @@ -132,17 +132,21 @@ impl OperatorBuilder for WindowBuilder { // trigger a retract call. let window_expr = window_expr?; let all_bounded = window_expr.iter().all(|e| e.uses_bounded_memory()); + + // Both window operators reject an input that does not declare its partition keys' order. + let input = + planner.with_declared_order(Arc::clone(&child.native_plan), &wnd.input_sort_orders)?; let window_agg: Arc = if all_bounded { Arc::new(BoundedWindowAggExec::try_new( window_expr, - Arc::clone(&child.native_plan), + input, InputOrderMode::Sorted, !partition_exprs.is_empty(), )?) } else { Arc::new(WindowAggExec::try_new( window_expr, - Arc::clone(&child.native_plan), + input, !partition_exprs.is_empty(), )?) }; diff --git a/native/proto/src/proto/operator.proto b/native/proto/src/proto/operator.proto index 5bc1ac12aa7..ffa67f5fc3f 100644 --- a/native/proto/src/proto/operator.proto +++ b/native/proto/src/proto/operator.proto @@ -1240,6 +1240,9 @@ message Window { repeated spark.spark_expression.Expr order_by_list = 2; repeated spark.spark_expression.Expr partition_by_list = 3; Operator child = 4; + // The partition keys ascending, when Spark planned no sort below the window because its input + // reports that order. Empty otherwise. + repeated spark.spark_expression.Expr input_sort_orders = 5; } // Top-K rows per partition group. Corresponds to Spark's WindowGroupLimitExec diff --git a/spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala b/spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala index 37323e3dce7..79b39d511b8 100644 --- a/spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala +++ b/spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala @@ -53,7 +53,7 @@ import org.apache.spark.sql.execution.datasources.v2.parquet.ParquetScan import org.apache.spark.sql.execution.datasources.v2.text.TextScan import org.apache.spark.sql.execution.exchange.{BroadcastExchangeExec, BroadcastExchangeLike, Exchange, ReusedExchangeExec, ShuffleExchangeExec, ShuffleExchangeLike} import org.apache.spark.sql.execution.joins.{BroadcastHashJoinExec, BroadcastNestedLoopJoinExec, ShuffledHashJoinExec, SortMergeJoinExec} -import org.apache.spark.sql.execution.window.WindowExec +import org.apache.spark.sql.execution.window.{WindowExec, WindowExecBase} import org.apache.spark.sql.internal.SQLConf import org.apache.spark.sql.types._ @@ -963,6 +963,21 @@ case class CometExecRule(session: SparkSession, queryStagePrep: Boolean = false) newPlan } + /** + * Sorts the input of a window that runs in Spark, a Python one included, when its order may be + * false. Spark plans no sort below a window whose child reports the order it needs, and its + * window trusts that order. A Comet operator that may not keep the order it reports, such as a + * native nested loop join (#6898), then hands the window rows out of order, and it groups them + * wrongly. + */ + private def sortDoubtedWindowInputs(plan: SparkPlan): SparkPlan = plan.transformUp { + case window: WindowExecBase + if window.requiredChildOrdering.head.nonEmpty && + CometHashJoinExec.mayReportFalseInputOrder(window.child) => + window.withNewChildren( + Seq(SortExec(window.requiredChildOrdering.head, global = false, child = window.child))) + } + private def _apply(plan: SparkPlan): SparkPlan = { // We shouldn't transform Spark query plan if Comet is not loaded. if (!isCometLoaded(conf)) { @@ -1012,7 +1027,7 @@ case class CometExecRule(session: SparkSession, queryStagePrep: Boolean = false) if (clearWastedBuildSideBridges(transformed)) { transformed = transform(planWithJoinRewritten) } - var newPlan = revertUnsafePartialAggregates(transformed) + var newPlan = sortDoubtedWindowInputs(revertUnsafePartialAggregates(transformed)) // if the plan cannot be run fully natively then explain why (when appropriate // config is enabled) diff --git a/spark/src/main/scala/org/apache/spark/sql/comet/CometWindowExec.scala b/spark/src/main/scala/org/apache/spark/sql/comet/CometWindowExec.scala index 69fa6e371ae..cc049292327 100644 --- a/spark/src/main/scala/org/apache/spark/sql/comet/CometWindowExec.scala +++ b/spark/src/main/scala/org/apache/spark/sql/comet/CometWindowExec.scala @@ -36,7 +36,7 @@ import org.apache.comet.{CometConf, ConfigEntry} import org.apache.comet.CometSparkSessionExtensions.withFallbackReason import org.apache.comet.serde.{AggSerde, CometOperatorSerde, CometSortOrder, LiteralOuterClass, OperatorOuterClass} import org.apache.comet.serde.OperatorOuterClass.Operator -import org.apache.comet.serde.QueryPlanSerde.{aggExprToProto, exprToProto, liftFallbackReasons, scalarFunctionExprToProto, serializeDataType} +import org.apache.comet.serde.QueryPlanSerde.{aggExprToProto, exprToProto, hasNonDefaultStringCollation, liftFallbackReasons, scalarFunctionExprToProto, serializeDataType} object CometWindowExec extends CometOperatorSerde[WindowExec] { @@ -75,16 +75,68 @@ object CometWindowExec extends CometOperatorSerde[WindowExec] { } } + // Spark plans no sort below the window when its input already reports the order the window + // needs. The native window relies on that order only when mayReportFalseOrder trusts it. A + // window that runs in Spark over a doubted order gets its input sorted again by + // CometExecRule, since Spark's window would trust that order too. + val requiredOrdering = op.requiredChildOrdering.head + val trustsInputOrder = + SortOrder.orderingSatisfies(op.child.outputOrdering, requiredOrdering) && + !CometHashJoinExec.mayReportFalseInputOrder(op.child) + + // The native window finds partitions and peers by comparing keys byte by byte, so it would + // tell apart collated strings that Spark treats as equal, as CometWindowGroupLimitExec notes. + val collated = (op.partitionSpec ++ op.orderSpec.map(_.child)) + .filter(key => hasNonDefaultStringCollation(key.dataType)) + if (collated.nonEmpty) { + withFallbackReason( + op, + collated + .map(_.sql) + .mkString("Window: non-default string collation on key(s): ", ", ", "")) + return None + } + + // Without partition keys the native window checks no order at all, so over a doubted order it + // would compute over rows that may be out of order. With partition keys it rejects an input + // whose order it cannot see. + if (op.partitionSpec.isEmpty && op.orderSpec.nonEmpty && !trustsInputOrder) { + withFallbackReason( + op, + "the order of the window's input, which Spark relies on, may be false") + return None + } + val partitionExprs = op.partitionSpec.map(exprToProto(_, op.child.output)) val sortOrders = op.orderSpec.map(exprToProto(_, op.child.output)) if (windowExprProto.forall(_.isDefined) && partitionExprs.forall(_.isDefined) && sortOrders.forall(_.isDefined)) { + // Rows from the JVM arrive through a ScanExec that declares no order. Send the partition + // keys' order, the only one the native window checks, for the planner to declare (see + // SortedInputExec). + val inputSortOrders = + if (op.partitionSpec.nonEmpty && trustsInputOrder) { + val partitionKeys = requiredOrdering.take(op.partitionSpec.length) + val orders = partitionKeys.flatMap(exprToProto(_, op.child.output)) + if (orders.length < partitionKeys.length) { + withFallbackReason( + op, + "the order of the window's input, which Spark relies on, cannot be expressed " + + "natively") + return None + } + orders + } else { + Nil + } + val windowBuilder = OperatorOuterClass.Window.newBuilder() windowBuilder.addAllWindowExpr(windowExprProto.map(_.get).asJava) windowBuilder.addAllPartitionByList(partitionExprs.map(_.get).asJava) windowBuilder.addAllOrderByList(sortOrders.map(_.get).asJava) + windowBuilder.addAllInputSortOrders(inputSortOrders.asJava) Some(builder.setWindow(windowBuilder).build()) } else { // Whichever of the window / partition / order expressions failed has already recorded its diff --git a/spark/src/main/scala/org/apache/spark/sql/comet/operators.scala b/spark/src/main/scala/org/apache/spark/sql/comet/operators.scala index a05ca87d2e6..85fe4a879b9 100644 --- a/spark/src/main/scala/org/apache/spark/sql/comet/operators.scala +++ b/spark/src/main/scala/org/apache/spark/sql/comet/operators.scala @@ -3011,6 +3011,11 @@ trait CometHashJoin { } } + // mayReportFalseOrder for a window's input, checked by CometWindowExec before it relies on that + // order and by CometExecRule before a window that stays in Spark does. + private[apache] def mayReportFalseInputOrder(plan: SparkPlan): Boolean = + mayReportFalseOrder(plan, countInPlan = true) + // Whether the order that `plan` reports may be false, because something that feeds it with no // sort in between cannot be relied on to keep it. Comet's hash joins report the order of their // streamed side, which Spark's own joins keep, but two Comet joins do not always: an outer join diff --git a/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala b/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala index ccc5e5e6c20..e11cdfe3275 100644 --- a/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala +++ b/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala @@ -39,16 +39,17 @@ import org.apache.spark.sql.catalyst.expressions.{And, Ascending, Attribute, Att import org.apache.spark.sql.catalyst.optimizer.{BuildLeft, BuildRight, BuildSide} import org.apache.spark.sql.catalyst.plans.{LeftOuter, RightOuter} import org.apache.spark.sql.columnar.{CachedBatch, SimpleMetricsCachedBatch} -import org.apache.spark.sql.comet.{CometBroadcastHashJoinExec, CometHashAggregateExec, CometHashJoin, CometHashJoinExec, CometInMemoryTableScanExec, CometSortAggregateExec, CometSortExec, CometSortMergeJoinExec} +import org.apache.spark.sql.comet.{CometBroadcastHashJoinExec, CometBroadcastNestedLoopJoinExec, CometHashAggregateExec, CometHashJoin, CometHashJoinExec, CometInMemoryTableScanExec, CometSortAggregateExec, CometSortExec, CometSortMergeJoinExec, CometWindowExec} import org.apache.spark.sql.comet.execution.arrow.{ArrowCachedBatchSerializer, CometCachedBatchHelper} import org.apache.spark.sql.comet.execution.shuffle.CometCelebornShuffleManager import org.apache.spark.sql.comet.util.Utils -import org.apache.spark.sql.execution.{ColumnarToRowExec, CometSparkPlanInfoHelper, FilterExec, FormattedMode, RowToColumnarExec, SortExec, SparkPlan, SparkPlanInfo} +import org.apache.spark.sql.execution.{ColumnarToRowExec, CometSparkPlanInfoHelper, FilterExec, FormattedMode, RowToColumnarExec, SortExec, SparkPlan, SparkPlanInfo, WholeStageCodegenExec} import org.apache.spark.sql.execution.adaptive.{AdaptiveSparkPlanExec, AQEShuffleReadExec, QueryStageExec, ShuffleQueryStageExec} import org.apache.spark.sql.execution.columnar.{CometInMemoryRelationHelper, InMemoryRelation, InMemoryTableScanExec} import org.apache.spark.sql.execution.exchange.{Exchange, ReusedExchangeExec, ShuffleExchangeLike} import org.apache.spark.sql.execution.joins.{BroadcastHashJoinExec, SortMergeJoinExec} -import org.apache.spark.sql.functions.{count, lit, max, min, sum} +import org.apache.spark.sql.execution.window.WindowExec +import org.apache.spark.sql.functions.{col, count, lit, max, min, sum} import org.apache.spark.sql.internal.{SQLConf, StaticSQLConf} import org.apache.spark.sql.types._ import org.apache.spark.sql.vectorized.{ColumnarBatch, ColumnVector} @@ -3456,6 +3457,239 @@ class CometInMemoryCacheSuite extends CometTestBase { } } + // The order a native window declares on its input, one "index direction nulls" per key. + private def windowInputOrder(window: CometWindowExec): Seq[String] = + window.nativeOp.getWindow.getInputSortOrdersList.asScala.toSeq.map { expr => + val order = expr.getSortOrder + s"${order.getChild.getBound.getIndex} ${order.getDirection} ${order.getNullOrdering}" + } + + // Checks `df`'s rows, the order each native window declares, the top one first, and that no + // sort is planned. + private def checkWindows(df: DataFrame, rows: Seq[Row], orders: Seq[Seq[String]]) = { + QueryTest.sameRows(rows, df.collect().toSeq).foreach(fail(_)) + val plan = df.queryExecution.executedPlan + withClue(s"$plan\n") { + assert(collect(plan) { case w: CometWindowExec => w }.map(windowInputOrder) == orders) + assert(collect(plan) { case s @ (_: CometSortExec | _: SortExec) => s }.isEmpty) + } + } + + test("a native window over a sorted cache declares the cache's order") { + // Spark plans no sort below a window over the cache's sort keys, so the scan has to declare + // the partition key's order (#7223). The key is nullable, so the null placement has to match, + // and the descending order key is not declared. dense_rank runs in BoundedWindowAggExec and + // the UNBOUNDED FOLLOWING frame in WindowAggExec. Within a key, v = k + 20 * j for j in + // 0 until 100, and a NULL key holds the multiples of 20. + val key = (v: Long) => if (v % 20 == 0) null else v % 20 + val queries = Seq( + "dense_rank() OVER (PARTITION BY k ORDER BY v DESC)" -> + ((v: Long) => (100 - v / 20).toInt), + "sum(v) OVER (PARTITION BY k ORDER BY v DESC " + + "ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING)" -> { (v: Long) => + val j = v / 20 + (j + 1) * (v % 20) + 10 * j * (j + 1) + }) + Seq(true, false).foreach { nativeCache => + withConversions( + SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "false", + SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "-1", + SQLConf.SHUFFLE_PARTITIONS.key -> "2", + CometConf.COMET_EXEC_IN_MEMORY_CACHE_ENABLED.key -> nativeCache.toString) { + try { + val cached = spark + .range(2000) + .selectExpr("CASE WHEN id % 20 = 0 THEN NULL ELSE id % 20 END AS k", "id AS v") + .repartition(2, col("k")) + .sortWithinPartitions(col("k"), col("v").desc) + .cache() + cached.count() + + queries.foreach { case (window, expected) => + withClue(s"nativeCache=$nativeCache, $window: ") { + checkWindows( + cached.selectExpr("k", "v", s"$window AS w"), + (0L until 2000L).map(v => Row(key(v), v, expected(v))), + Seq(Seq("0 Ascending NullsFirst"))) + } + } + } finally { + spark.catalog.clearCache() + } + } + } + } + + test("a native window declares its order next to the order its input declares") { + // Spark plans no sort above an operator that keeps the cache's order, so a window's input may + // declare only the order that operator needed: an upper window's input the lower window's, + // and an outer join's output its streamed side's. The window's order is declared next to it + // (#7223). Alias a of sort key k shares its order. One partition, so no exchange comes in + // between. + val ids = 0L until 24L + val key = (id: Long) => if (id % 4 == 0) None else Some(id % 3) + val nullable = (k: Option[Long]) => k.map(Long.box).orNull + val keySums = ids.groupBy(key).map { case (k, group) => k -> group.sum } + val keyCounts = ids.groupBy(key).map { case (k, group) => k -> group.size } + val pairCounts = + ids.groupBy(id => (key(id), id % 2)).map { case (p, group) => p -> group.size } + withConversions( + SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "false", + SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "-1") { + try { + val cached = spark + .range(0, 24, 1, 1) + .selectExpr( + "CASE WHEN id % 4 = 0 THEN NULL ELSE id % 3 END AS k", + "id % 2 AS x", + "id AS v") + .sortWithinPartitions("k", "x") + .selectExpr("k", "k AS a", "x", "v") + .cache() + cached.count() + + checkWindows( + cached + .selectExpr("k", "x", "v", "sum(v) OVER (PARTITION BY k) AS s") + .selectExpr("k", "x", "v", "sum(s) OVER (PARTITION BY k, x) AS t"), + ids.map { id => + val k = key(id) + Row(nullable(k), id % 2, id, keySums(k) * pairCounts((k, id % 2))) + }, + Seq( + Seq("0 Ascending NullsFirst", "1 Ascending NullsFirst"), + Seq("0 Ascending NullsFirst"))) + checkWindows( + cached + .selectExpr("k", "a", "v", "sum(v) OVER (PARTITION BY k) AS s") + .selectExpr("k", "a", "v", "sum(s) OVER (PARTITION BY a) AS t"), + ids.map { id => + val k = key(id) + Row(nullable(k), nullable(k), id, keySums(k) * keyCounts(k)) + }, + Seq(Seq("1 Ascending NullsFirst"), Seq("0 Ascending NullsFirst"))) + // The join declares the order of a, which Spark reports in place of k, so the window by k + // needs its order declared next to the join's. The residual condition leaves rows of key 0 + // unmatched among matched ones, so the key stays together only if the join keeps its + // unmatched probe rows in place. + val one = spark.range(1).toDF("j").hint("broadcast") + checkWindows( + cached + .join(one, col("k") === col("j") && col("v") % 2 === 0, "left_outer") + .selectExpr("k", "a", "v", "sum(v) OVER (PARTITION BY k) AS s"), + ids.map { id => + val k = key(id) + Row(nullable(k), nullable(k), id, keySums(k)) + }, + Seq(Seq("0 Ascending NullsFirst"))) + } finally { + spark.catalog.clearCache() + } + } + } + + test("a window over a cache sorted by a collated string key stays in Spark") { + assume(isSpark40Plus, "collated string types require Spark 4.0+") + // The native window compares keys byte by byte, so it would split the UTF8_LCASE partition + // of '0a' and '0A' and rank them apart. + val key = (v: Long) => s"${v % 4}${if (v % 8 < 4) "a" else "A"}" + withConversions( + SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "false", + SQLConf.SHUFFLE_PARTITIONS.key -> "2") { + try { + val source = spark + .range(200) + .selectExpr( + "CAST(id % 4 AS STRING) || CASE WHEN id % 8 < 4 THEN 'a' ELSE 'A' END " + + "COLLATE UTF8_LCASE AS k", + "id % 2 AS n", + "id AS v") + val byKey = source.repartition(2, col("k")).sortWithinPartitions("k", "v").cache() + val byGroup = source.repartition(2, col("n")).sortWithinPartitions("n", "k").cache() + byKey.count() + byGroup.count() + + Seq( + // Each of the four keys holds 50 rows. + byKey.selectExpr("k", "v", "dense_rank() OVER (PARTITION BY k ORDER BY v) AS r") -> + ((v: Long) => Row(key(v), v, (v / 4 + 1).toInt)), + // Each group holds two keys, '0' and '2' or '1' and '3'. + byGroup.selectExpr("n", "k", "dense_rank() OVER (PARTITION BY n ORDER BY k) AS r") -> + ((v: Long) => Row(v % 2, key(v), (v % 4 / 2 + 1).toInt))).foreach { case (df, row) => + QueryTest.sameRows((0L until 200L).map(row), df.collect().toSeq).foreach(fail(_)) + val plan = df.queryExecution.executedPlan + assert(collect(plan) { case w: CometWindowExec => w }.isEmpty, plan) + val reasons = new ExtendedExplainInfo().getFallbackReasons(plan) + assert(reasons.exists(_.startsWith("Window: non-default string collation")), reasons) + } + } finally { + spark.catalog.clearCache() + } + } + } + + test("a window that stays in Spark over a nested loop join sorts the join's rows again") { + // A native nested loop join emits its rows build row by build row, out of the streamed side's + // order that it reports (#6898), and Spark plans no sort above it. A window that runs in Spark + // there, one without partition keys or one with a collated key, gets its input sorted again. + // One partition, so no exchange comes between the join and the window, with or without AQE. + // With AQE, the stage's own pass can make the added sort native and then the window above it. + Seq(false, true).foreach { aqe => + withConversions(SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> aqe.toString) { + try { + val pairs = spark.range(2).toDF("z").hint("broadcast") + val sorted = (columns: Seq[String], keys: Seq[String]) => + spark + .range(0, 12, 1, 1) + .selectExpr(columns: _*) + .sortWithinPartitions(keys.map(col): _*) + .cache() + .join(pairs, col("v") % 2 <= col("z")) + val queries = Seq( + sorted(Seq("id AS v"), Seq("v")) + .selectExpr("v", "dense_rank() OVER (ORDER BY v) AS r")) ++ + (if (isSpark40Plus) { + val collated = Seq( + "id % 3 AS k", + "id AS v", + "CASE id % 4 WHEN 0 THEN 'A' WHEN 1 THEN 'a' WHEN 2 THEN 'B' ELSE 'b' END " + + "COLLATE UTF8_LCASE AS s") + Seq( + // lower(s) leaves the case-insensitive peers equal in the output, whose order + // among themselves the comparison would otherwise see. + sorted(collated, Seq("s")) + .selectExpr("lower(s) AS s", "dense_rank() OVER (ORDER BY s) AS r"), + sorted(collated, Seq("k", "s")).selectExpr( + "k", + "count(*) OVER (PARTITION BY k ORDER BY s " + + "ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING) AS c")) + } else { + Nil + }) + queries.foreach { df => + val (_, plan) = checkSparkAnswer(df) + withClue(s"$plan\n") { + assert(plan.isInstanceOf[AdaptiveSparkPlanExec] == aqe) + val finalPlan = stripAQEPlan(plan) + val joins = collect(finalPlan) { case j: CometBroadcastNestedLoopJoinExec => j } + assert(joins.nonEmpty) + val windowInputs = collect(finalPlan) { + case w: WindowExec => w.child + case w: CometWindowExec => w.child + } + assert(windowInputs.nonEmpty && windowInputs.forall { + case _: SortExec | _: CometSortExec | WholeStageCodegenExec(_: SortExec) => true + case _ => false + }) + } + } + } finally { + spark.catalog.clearCache() + } + } + } + } + test("an aggregate on the key above a hash join over a sorted cache groups each key once") { // The order the join declares on its probe input reaches past the join, so the aggregate // above it in the same native plan groups in sorted, streaming mode. That is right while diff --git a/spark/src/test/scala/org/apache/comet/exec/CometWindowExecSuite.scala b/spark/src/test/scala/org/apache/comet/exec/CometWindowExecSuite.scala index 76b39c34a5b..3aaa45ffefb 100644 --- a/spark/src/test/scala/org/apache/comet/exec/CometWindowExecSuite.scala +++ b/spark/src/test/scala/org/apache/comet/exec/CometWindowExecSuite.scala @@ -27,7 +27,7 @@ import org.scalatest.Tag import org.apache.hadoop.fs.Path import org.apache.spark.sql.{CometTestBase, DataFrame, Row} import org.apache.spark.sql.catalyst.expressions.{Alias, Cast, Divide, Expression, MakeDecimal, WindowExpression} -import org.apache.spark.sql.comet.{CometSortExec, CometWindowExec, CometWindowGroupLimitExec} +import org.apache.spark.sql.comet.{CometNativeScanExec, CometSortExec, CometWindowExec, CometWindowGroupLimitExec} import org.apache.spark.sql.execution.SparkPlan import org.apache.spark.sql.execution.window.{WindowExec => SparkWindowExec} import org.apache.spark.sql.expressions.Window @@ -1957,4 +1957,40 @@ class CometWindowExecSuite extends CometTestBase { } } } + + test("window over a sorted bucketed table with empty buckets runs natively") { + // With legacy bucketed output ordering on, Spark plans no sort below the window. Two keys fill + // at most two of the eight buckets, and each empty one has to declare the order too (#7223). + withTable("sorted_bucketed") { + // A single writer task writes one file per bucket. + (0 until 40) + .map(i => (i % 2, i)) + .toDF("k", "v") + .coalesce(1) + .write + .format("parquet") + .bucketBy(8, "k") + .sortBy("k", "v") + .saveAsTable("sorted_bucketed") + withSQLConf( + SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "false", + SQLConf.BUCKETING_ENABLED.key -> "true", + SQLConf.AUTO_BUCKETED_SCAN_ENABLED.key -> "false", + SQLConf.LEGACY_BUCKETED_TABLE_SCAN_OUTPUT_ORDERING.key -> "true") { + val df = sql( + "SELECT k, v, dense_rank() OVER (PARTITION BY k ORDER BY v) AS r " + + "FROM sorted_bucketed") + val (_, plan) = checkSparkAnswerAndOperator(df) + val windows = collect(plan) { case w: CometWindowExec => w } + assert(windows.size == 1, plan) + assert( + windows.head.child match { + case scan: CometNativeScanExec => scan.bucketedScan + case _ => false + }, + plan) + assert(windows.head.nativeOp.getWindow.getInputSortOrdersCount == 1, plan) + } + } + } } From 9eb616bca3d1b0cdd93f408afaa1f2cd8399e1b1 Mon Sep 17 00:00:00 2001 From: dustin Date: Sat, 14 Nov 2026 04:49:54 +0000 Subject: [PATCH 3/3] fix: sort a Spark window's doubted input by the whole order it reports (#7555) sortDoubtedWindowInputs sorted the input of a window that stays in Spark by the window's own keys only. Spark had already planned the operators above around the longer order the input reported, so a second window ordered by more keys, or a sortWithinPartitions it had removed, saw rows out of order. The sort now uses the whole reported order when it covers the window's keys. Closes #7553. (cherry picked from commit fd82542adf69b64ed20ecb82c73493b36273c8e4) --- .../adding_a_new_operator.md | 8 +- .../apache/comet/rules/CometExecRule.scala | 11 ++- .../comet/exec/CometInMemoryCacheSuite.scala | 76 ++++++++++++++++++- 3 files changed, 89 insertions(+), 6 deletions(-) diff --git a/docs/source/contributor-guide/adding_a_new_operator.md b/docs/source/contributor-guide/adding_a_new_operator.md index 6317780def6..f8c35919749 100644 --- a/docs/source/contributor-guide/adding_a_new_operator.md +++ b/docs/source/contributor-guide/adding_a_new_operator.md @@ -374,9 +374,11 @@ projection do. The planner declares the order through an operator only when the keeps it. An operator that cannot keep the order natively must not report it, or has to be added to `mayReportFalseOrder` in `operators.scala`. The hash join and window serdes then send no order over it, an outer hash join above it that has to keep that order falls back to Spark, and a window that -runs in Spark above it gets its input sorted again. Comet's hash joins are the exception within one -native plan: an outer hash join over one of them stays native, even over one that declared no order, -since DataFusion keeps the order it can see there. +runs in Spark above it gets its input sorted again, by the whole order the input reports, since the +operators above the window may rely on more of it than the window needs. When that order does not +cover the window's keys, the sort is by those keys instead. Comet's hash joins are the exception +within one native plan: an outer hash join over one of them stays native, even over one that +declared no order, since DataFusion keeps the order it can see there. A leaf that reports an ordering is where the order starts, so the planner has to know that the native operator produces its rows in that order. It knows that of an input from the JVM and of a diff --git a/spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala b/spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala index 79b39d511b8..076e8064aa3 100644 --- a/spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala +++ b/spark/src/main/scala/org/apache/comet/rules/CometExecRule.scala @@ -969,13 +969,20 @@ case class CometExecRule(session: SparkSession, queryStagePrep: Boolean = false) * window trusts that order. A Comet operator that may not keep the order it reports, such as a * native nested loop join (#6898), then hands the window rows out of order, and it groups them * wrongly. + * + * The sort is by the whole order the child reports, not only by the part the window needs, or + * by the window's own keys when the reported order does not cover them. The window passes that + * order on as its own, and Spark planned the operators above around it: a window or a sort + * above that needs a longer order got no sort of its own. */ private def sortDoubtedWindowInputs(plan: SparkPlan): SparkPlan = plan.transformUp { case window: WindowExecBase if window.requiredChildOrdering.head.nonEmpty && CometHashJoinExec.mayReportFalseInputOrder(window.child) => - window.withNewChildren( - Seq(SortExec(window.requiredChildOrdering.head, global = false, child = window.child))) + val required = window.requiredChildOrdering.head + val reported = window.child.outputOrdering + val keys = if (SortOrder.orderingSatisfies(reported, required)) reported else required + window.withNewChildren(Seq(SortExec(keys, global = false, child = window.child))) } private def _apply(plan: SparkPlan): SparkPlan = { diff --git a/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala b/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala index e11cdfe3275..5d4fa7e328f 100644 --- a/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala +++ b/spark/src/test/scala/org/apache/comet/exec/CometInMemoryCacheSuite.scala @@ -32,7 +32,7 @@ import org.apache.arrow.memory.{ArrowBuf, BufferAllocator} import org.apache.arrow.vector.{BitVector, FixedSizeBinaryVector, IntVector, LargeVarCharVector, VarBinaryVector, VarCharVector} import org.apache.arrow.vector.compression.{CompressionCodec, CompressionUtil, NoCompressionCodec} import org.apache.arrow.vector.types.pojo.ArrowType -import org.apache.spark.{SparkConf, SparkException} +import org.apache.spark.{SparkConf, SparkEnv, SparkException} import org.apache.spark.CometDriverPlugin import org.apache.spark.sql.{CometTestBase, DataFrame, Observation, QueryTest, Row} import org.apache.spark.sql.catalyst.expressions.{And, Ascending, Attribute, AttributeReference, EqualTo, Expression, GreaterThan, GreaterThanOrEqual, LessThan, Literal} @@ -3690,6 +3690,80 @@ class CometInMemoryCacheSuite extends CometTestBase { } } + // The lower window has no PARTITION BY, so it stays in Spark, and needs only v's order. Its + // input reports (v, w), so Spark plans no sort of its own for the window ordered by (v, w) + // above, or for a sortWithinPartitions by (v, w). + private def checkOrderAboveSparkWindow(rowCount: Long, buildRows: Long): Unit = { + withConversions( + SQLConf.ADAPTIVE_EXECUTION_ENABLED.key -> "false", + SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "-1") { + try { + val pairs = spark.range(buildRows).toDF("z").hint("broadcast") + val lower = spark + .range(0, rowCount, 1, 1) + .selectExpr("id % 4 AS v", "id AS w") + .sortWithinPartitions("v", "w") + .cache() + .join(pairs, col("w") % 2 <= col("z")) + .selectExpr("v", "w", "sum(w) OVER (ORDER BY v) AS s") + val upper = lower.selectExpr("*", "row_number() OVER (ORDER BY v, w) AS r") + val upperPlan = upper.queryExecution.executedPlan + assert(collect(upperPlan) { case w: WindowExec => w }.size == 2, upperPlan) + assertOnlySortUnderLowerWindow(upperPlan) + checkSparkAnswer(upper) + val sorted = lower.sortWithinPartitions("v", "w") + val plan = sorted.queryExecution.executedPlan + assert(collect(plan) { case w: WindowExec => w }.size == 1, plan) + assertOnlySortUnderLowerWindow(plan) + assert(collect(plan) { case j: CometBroadcastNestedLoopJoinExec => j }.size == 1, plan) + val rows = sorted.collect().map(r => (r.getLong(0), r.getLong(1))) + val outOfOrder = rows.sliding(2).collectFirst { + case Array(a, b) if Ordering[(Long, Long)].gt(a, b) => s"$a comes before $b" + } + assert(outOfOrder.isEmpty, outOfOrder) + } finally { + spark.catalog.clearCache() + } + } + } + + // checkOrderAboveSparkWindow rests on Spark having planned no sort of its own above the lower + // window, so the one sort in the plan is the one under that window, by v then w. + private def assertOnlySortUnderLowerWindow(plan: SparkPlan): Unit = { + val sorts = collect(plan) { case s @ (_: SortExec | _: CometSortExec) => s } + val lowerWindows = collect(plan) { + case w: WindowExec if collect(w.child) { case c: WindowExec => c }.isEmpty => w + } + assert(sorts.size == 1 && lowerWindows.size == 1, plan) + val windowInput = lowerWindows.head.child match { + case stage: WholeStageCodegenExec => stage.child + case child => child + } + assert(windowInput eq sorts.head, plan) + val keys = sorts.head match { + case s: SortExec => s.sortOrder.map(_.child) + case s: CometSortExec => s.sortOrder.map(_.child) + } + assert(keys.map { case a: Attribute => a.name; case e => e.toString } == Seq("v", "w"), plan) + } + + test("a window that stays in Spark over a nested loop join keeps the order its input reports") { + // Over two build rows the native join emits its rows build row by build row, out of (v, w) + // order. A sort by v alone under the lower window would leave w out of order. + checkOrderAboveSparkWindow(4000, 2) + } + + test("a spilling sort under a window that stays in Spark keeps the order its input reports") { + // Over one build row the join keeps (v, w) order. A sort by v alone under the lower window + // may reorder the rows that tie on v when it merges its spill files. + val key = "spark.shuffle.spill.numElementsForceSpillThreshold" + val conf = SparkEnv.get.conf + val previous = conf.getOption(key) + conf.set(key, "10000") + try checkOrderAboveSparkWindow(200000, 1) + finally previous.fold(conf.remove(key))(conf.set(key, _)) + } + test("an aggregate on the key above a hash join over a sorted cache groups each key once") { // The order the join declares on its probe input reaches past the join, so the aggregate // above it in the same native plan groups in sorted, streaming mode. That is right while