Repository navigation
Open
Labels
Description
Bug triage pass for 2026-11-16.
- Issues processed: 17 (8 bugs, 9 enhancements)
- Bugs by priority: critical 2, high 0, medium 5, low 1
- Skipped: 1
Criteria: Bug Triage Guide.
Labels have already been applied and requires-triage removed. Please spot-check the calls and close this issue when satisfied. Many issues arrived already carrying a type label; those were kept as they were.
Bugs
priority:critical
- Native MergeRows can panic or produce wrong masks - arrow 59
and_notmishandles sliced arrays (#7596)- Area labels: none
- Rationale: Besides the panic,
and_noton a sliced array can return wrong masks with no error, so inserts and deletes land on the wrong rows, a silent wrong result.
- Native string to decimal cast fails on a string with more than 38 digits: NULL or
CAST_INVALID_INPUTwhere Spark returns a value (#7581)- Area labels:
area:expressions - Rationale: Comet returns NULL where Spark returns a value (for example
1.00) with ANSI off, a silent wrong result.
- Area labels:
priority:medium
- fair_unified: consumers that have not started yet take an equal share, so the running sort or aggregate spills early (#7628)
- Area labels: none
- Rationale: Results are correct, but the extra spilling is a significant performance degradation (56 GB spilled against 19.7 GB for Spark in the report).
- make_date can throw on unconsumed rows through PySpark and raw row iterators (#7616)
- Area labels:
area:expressions - Rationale: Comet raises an error on a row Spark never evaluates, only for partial consumers of raw row iterators; a visible failure, not a wrong result.
- Area labels:
- A failed native
growis only logged when the task makes no latertry_grow(#7615)- Area labels: none
- Rationale: A memory request Spark rejected is not surfaced, so the task can run past its grant; it needs a failed
growand no latertry_grow, and does not return wrong data.
- Native shuffle write fails when the local dir path contains
.data; failed tasks also leave temp files behind (#7594)- Area labels:
area:shuffle - Rationale: Native shuffle write fails for local dirs whose path contains
.data, which can be worked around by renaming the directory, and failed tasks leave temp files behind.
- Area labels:
- lpad/rpad do not use withNullShortCircuit, so ANSI errors are raised on rows Spark skips (#7583)
- Area labels:
area:expressions - Rationale: Under ANSI, Comet raises an error on a NULL-input row where Spark returns NULL; a visible failure that needs an ANSI cast in a child expression.
- Area labels:
priority:low
- Native string to decimal cast raises
CAST_INVALID_INPUTinstead ofNUMERIC_VALUE_OUT_OF_RANGEunder ANSI when the scale adjustment is more than 38 (#7576)- Area labels:
area:expressions - Rationale: Both engines fail under ANSI and return NULL otherwise; only the error class differs.
- Area labels:
Enhancements
- Native Iceberg writes: assert manifest metrics parity with iceberg-java for every data type (#7639)
- Area labels:
area:writer - Rationale: Asks for more parity tests before native Iceberg writes are enabled by default.
- Area labels:
- Native Iceberg writes: support
hdfsdata locations (#7637)- Area labels:
area:writer - Rationale: Native Iceberg writes fall back to iceberg-java for
hdfsby design; this asks for new support.
- Area labels:
- Support struct and array map keys in native MapSort (#7632)
- Area labels:
area:expressions - Rationale: Requests native evaluation for key types that already fall back.
- Area labels:
- Comet Native Executors: raise Spark's exceptions for cluster failures (#7621)
- Area labels: none
- Rationale: Requests a new structured error path; the limitation is already documented.
- Migrate to the new apache/datafusion-spark repository (#7619)
- Area labels: none
- Rationale: Docs, skills and dependency housekeeping after an upstream move;
mainis unaffected.
- Research using the Rust Allocator API (stabilized in 1.100) for memory-pool accounting (#7613)
- Area labels: none
- Rationale: A research task, not a defect.
- Restore native row-group pruning for FLOAT and DOUBLE range filters in Iceberg scans (#7588)
- Area labels:
area:scan - Rationale: Results are correct today; this asks to recover pruning that fix: keep Iceberg residual pushdown within Spark semantics #7156 turned off.
- Area labels:
- Speculate string functions such as upper in NullShortCircuit by adding them to is_cheap (#7587)
- Area labels:
area:expressions - Rationale: A performance optimization request.
- Area labels:
- Consolidate the Scala "cannot fail" checks: arrays.cannotThrow and QueryPlanSerde.cannotFail (#7585)
- Area labels:
area:expressions - Rationale: A refactor to remove duplicated lists.
- Area labels:
Escalations to consider
- lpad/rpad do not use withNullShortCircuit, so ANSI errors are raised on rows Spark skips (#7583)
- Same family as make_date can throw on unconsumed rows through PySpark and raw row iterators #7616 and the fix: skip arithmetic and concat operands that Spark skips under ANSI mode #7456 work: Comet raises errors on rows Spark skips. If more reports arrive under ANSI, consider
priority:high.
- Same family as make_date can throw on unconsumed rows through PySpark and raw row iterators #7616 and the fix: skip arithmetic and concat operands that Spark skips under ANSI mode #7456 work: Comet raises errors on rows Spark skips. If more reports arrive under ANSI, consider
Skipped — needs more info
- TPC-DS q54/q68: local SF100 results and distributed reproduction limits (#6946)
- A reproduction report for Investigate TPC-DS q54 and q68 slowdowns in the 1.1.0 benchmark on Spark 4.2 #6418 that says it is not a claim of a defect; unclear whether it is a bug or an enhancement.
requires-triageis left in place.
- A reproduction report for Investigate TPC-DS q54 and q68 slowdowns in the 1.1.0 benchmark on Spark 4.2 #6418 that says it is not a claim of a defect; unclear whether it is a bug or an enhancement.
Activity