Repository navigation
fix(core): keep the cause of plan serialization errors on the client - #2911
Open
bharadwaj-pendyala wants to merge 1 commit into
Open
bharadwaj-pendyala wants to merge 1 commit into
bharadwaj-pendyala wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #2885.
Rationale for this change
NoahKusaba reported in #2885 that when the client can't serialize a logical plan, the cause gets flattened into an
Internalstring.find_rootcan't reach it, quotes and newlines come outDebug-escaped, and DataFusion adds its "likely caused by a bug in DataFusion's code" footer even when the cause is the user's, like a dropped Iceberg table.What changes are included in this PR?
execute_query_paramsadds context to the serialize and encode errors instead of replacing them, as the issue suggests.execute_physical_planin the same file had the same closures for physical plans, so it gets the same change.With the default codec and a
MemTablescan, the error used to beInternal("failed to serialize logical plan: Context(\"Error serializing custom table ...\", NotImplemented(...))"). Now it'sContext("failed to serialize logical plan", Context(..., NotImplemented(...))), andfind_rootreturns theNotImplemented.For physical plans, DataFusion's own codec already reports an unsupported node as
Internal, so that message still has the footer. It's no longer wrapped in a secondInternal, though.The encode closures changed for consistency. Encoding into a
Vec<u8>doesn't fail in practice, so no test reaches them.Left out:
python/src/codec.rswrapsDistributedQueryExecencode and decode errors the same way. The Python client submits throughexecute_query_params, so it gets this fix, but those codec paths are separate and could be a follow-up.Test plan:
logical_plan_serialization_error_keeps_its_causeandphysical_plan_serialization_error_keeps_its_cause. Both fail on main and pass with the fix.cargo test -p ballista-core --lib: 587 passed.cargo test -p ballista --test context_setup: 7 passed.cargo fmt --all -- --check, and clippy onballista-corewith--all-targets --all-features -D warnings.Are there any user-facing changes?
The error changes from
Internal(orExecutionfor encode) toContext. Its text now reads "failed to serialize logical plan" with each cause on its own "caused by" line, without theInternal error:prefix or the bug-report footer. Nothing in the repo matches on the old text.