Skip to content

fix: handle upserts after adding a column - #4444

Draft
ViktorBystrov72 wants to merge 1 commit into
apache:mainfrom
ViktorBystrov72:fix/3105-upsert-schema-evolution
Draft

ViktorBystrov72 wants to merge 1 commit into
apache:mainfrom
ViktorBystrov72:fix/3105-upsert-schema-evolution

Conversation

@ViktorBystrov72

@ViktorBystrov72 ViktorBystrov72 commented Nov 20, 2026 •

Copy link
Copy Markdown

Which issue does this PR close?

Closes #3105

What changes are included in this PR?

During upsert, read the selected snapshot using the current field types and match incoming fields by Iceberg field ID, including nested fields. This supports adding nullable fields, renaming fields and dropping and re-adding a field with the same name. Writes use the original incoming rows.

Are these changes tested?

35 tests in test_upsert.py and 25 schema compatibility and projection checks passed. The regression cases cover the main and dev branches, key type widening, partial schemas and field ID changes. Ruff, git diff --check and the GitHub CI checks passed.

AI Disclosure

Codex (gpt-6.2-sol) assisted with the implementation and tests. A separate local review checked snapshot projection and ID-based comparisons. This PR is a draft pending the author's review of nested-value comparisons and default handling.

@Fokko

Fokko commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

@ViktorBystrov72 Thanks for raising this PR, could you update the PR description te English?

@ViktorBystrov72 ViktorBystrov72 changed the title fix: поддержать upsert после добавления столбца fix: handle upserts after adding a column Nov 21, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upsert fails after update_schema().union_by_name() due to schema mismatch

2 participants