Skip to content

fix: Handle absolute paths in Delta log file listing (#907) - #932

Merged
vinishjail97 merged 3 commits into
apache:mainfrom
zhang-arvin:fix/delta-absolute-path-handling
Sep 26, 2026
Merged

vinishjail97 merged 3 commits into
apache:mainfrom
zhang-arvin:fix/delta-absolute-path-handling

Conversation

@zhang-arvin

Copy link
Copy Markdown
Contributor

Closes #907

Description

The Delta protocol (PROTOCOL.md, Add File and Remove File actions) allows a data file path to be relative to the table root or absolute. XTable's delta sources always assume relative paths and concatenate the table base path onto the logged path when it does not lexically start with the base, producing locations like s3a://bucket/tab/s3://bucket/tab/part-0.parquet. The resulting manifest entries carry record_count=0 and are pruned during scan planning, so the sync reports success while silently dropping rows from the converted table.

Root cause

DeltaActionsConverter.getFullPathToFile and DeltaKernelActionsConverter.getFullPathToFile use String.startsWith prefix matching to decide whether the logged path is already resolved. Any absolute path not lexically under the table base (a scheme alias such as s3:// against an s3a:// base, a same-scheme different-bucket/container location, or a file:// URI) fails the prefix test and is treated as relative, mangling the location.

Changes

  • DeltaActionsConverter.getFullPathToFile: use the logged path as-is when it is absolute. A path is absolute when it has a URI scheme (e.g. s3://, s3a://, hdfs://, file://, abfss://, gs://) or starts with /.
  • DeltaKernelActionsConverter.getFullPathToFile: same fix for the Delta Kernel source path.
  • Added unit tests for relative paths, absolute paths under the table base (unchanged), scheme-alias paths, absolute paths outside the table base, and file:// URIs.

With the fix, all of the failure cases from the issue resolve correctly:

tableBasePath dataFilePath result
s3a://bucket/tab part-0.parquet s3a://bucket/tab/part-0.parquet
s3a://bucket/tab s3a://bucket/tab/part-0.parquet unchanged
s3a://bucket/tab s3://bucket/tab/part-0.parquet unchanged
s3a://bucket/tab s3://other-bucket/x/part-0.parquet unchanged
s3a://bucket/tab file:///local/t/part-0.parquet unchanged

Validation

  • ./mvnw -pl xtable-core -am -Dtest=TestDeltaActionsConverter -DfailIfNoTests=false -DskipITs test — 6 tests, all passing
  • ./mvnw spotless:check -pl xtable-core — passing

@vinishjail97 vinishjail97 left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for picking this up and your first set of contributions @zhang-arvin, added a minor comment. Please fix the PR title check to include the prefix "fix:"

@zhang-arvin zhang-arvin changed the title [Bug] Handle absolute paths in Delta log file listing (#907) fix: Handle absolute paths in Delta log file listing (#907) Sep 10, 2026
@vinishjail97
vinishjail97 merged commit 3353cb5 into apache:main Sep 26, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Absolute paths in the Delta log are treated as relative, silently dropping rows from the converted table

2 participants