Skip to content

fix(core): keep the cause of plan serialization errors on the client - #2911

Open
bharadwaj-pendyala wants to merge 1 commit into
apache:mainfrom
bharadwaj-pendyala:fix/keep-plan-serialize-error-cause
Open

bharadwaj-pendyala wants to merge 1 commit into
apache:mainfrom
bharadwaj-pendyala:fix/keep-plan-serialize-error-cause

Conversation

@bharadwaj-pendyala

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #2885.

Rationale for this change

NoahKusaba reported in #2885 that when the client can't serialize a logical plan, the cause gets flattened into an Internal string. find_root can't reach it, quotes and newlines come out Debug-escaped, and DataFusion adds its "likely caused by a bug in DataFusion's code" footer even when the cause is the user's, like a dropped Iceberg table.

What changes are included in this PR?

  • execute_query_params adds context to the serialize and encode errors instead of replacing them, as the issue suggests.
  • execute_physical_plan in the same file had the same closures for physical plans, so it gets the same change.

With the default codec and a MemTable scan, the error used to be Internal("failed to serialize logical plan: Context(\"Error serializing custom table ...\", NotImplemented(...))"). Now it's Context("failed to serialize logical plan", Context(..., NotImplemented(...))), and find_root returns the NotImplemented.

For physical plans, DataFusion's own codec already reports an unsupported node as Internal, so that message still has the footer. It's no longer wrapped in a second Internal, though.

The encode closures changed for consistency. Encoding into a Vec<u8> doesn't fail in practice, so no test reaches them.

Left out: python/src/codec.rs wraps DistributedQueryExec encode and decode errors the same way. The Python client submits through execute_query_params, so it gets this fix, but those codec paths are separate and could be a follow-up.

Test plan:

  • New logical_plan_serialization_error_keeps_its_cause and physical_plan_serialization_error_keeps_its_cause. Both fail on main and pass with the fix.
  • cargo test -p ballista-core --lib: 587 passed. cargo test -p ballista --test context_setup: 7 passed.
  • cargo fmt --all -- --check, and clippy on ballista-core with --all-targets --all-features -D warnings.

Are there any user-facing changes?

The error changes from Internal (or Execution for encode) to Context. Its text now reads "failed to serialize logical plan" with each cause on its own "caused by" line, without the Internal error: prefix or the bug-report footer. Nothing in the repo matches on the old text.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Client hides plan-encoding errors behind an Internal error

1 participant