Skip to content

[AURON #2539] Support residual conditions in native broadcast hash joins - #2551

Open
Deegue wants to merge 2 commits into
apache:masterfrom
Deegue:auron_#2539_support_rc_in_bhj
Open

Deegue wants to merge 2 commits into
apache:masterfrom
Deegue:auron_#2539_support_rc_in_bhj

Conversation

@Deegue

@Deegue Deegue commented Sep 28, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #2539

Rationale for this change

Native broadcast hash joins currently support residual conditions only for inner joins. Other join types, such as outer, semi, anti, and existence joins, fall back to Spark when the join includes an additional condition such as l.k = r.k AND l.v < r.v.
These conditions must participate in determining whether rows match. Applying a filter after the join cannot preserve the required semantics for these join types.

What changes are included in this PR?

  • Extend native broadcast hash joins to evaluate residual conditions for supported outer, semi, anti, and existence joins using the shared join-condition evaluator.
  • Update match tracking only for candidate pairs that satisfy the condition, preserving unmatched rows and existence results.
  • Bound candidate batches when processing duplicate keys and gather only the columns required by the condition.
  • Preserve output projections and row counts, including when execution falls back to native sort-merge join.
  • Add Rust and Spark regression tests covering join types, build sides, nulls, projections, spill behavior, and the existing configuration switch.

Are there any user-facing changes?

No.

How was this patch tested?

UTs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support and extend residual conditions in native broadcast hash joins

1 participant