Skip to content

Add ManageSnapshots.fast_forward_branch - #3900

Open
Ferdinanddb wants to merge 3 commits into
apache:mainfrom
Ferdinanddb:fast-forward-branch
Open

Add ManageSnapshots.fast_forward_branch#3900
Ferdinanddb wants to merge 3 commits into
apache:mainfrom
Ferdinanddb:fast-forward-branch

Conversation

@Ferdinanddb

Copy link
Copy Markdown

Adds ManageSnapshots.fast_forward_branch, following Java's ManageSnapshots.fastForwardBranch and Spark's system.fast_forward procedure.

with table.manage_snapshots() as ms:
    ms.fast_forward_branch("main", "audit")
  • the ref to fast-forward to can be a branch or a tag and is left untouched
  • a branch that does not exist yet is created at that snapshot with default retention properties, an existing branch keeps its retention properties
  • fast-forwarding to the snapshot the branch already points to is a no-op, a branch that is not an ancestor raises ValueError
  • operations chained in one manage_snapshots() see each other, and a ref set twice in a chain results in a single set-snapshot-ref update

Supersedes #3649 by @JeroenSchmidt, which has been conflicting since August. It keeps the same API and intra-chain behaviour and addresses the open review comments from @rambleraptor (refs injected directly in the tests, default retention of a created branch documented, tag target and no-op covered, shorter docs). Differences: plain ValueError instead of new exception classes, the pending refs come from update_table_metadata instead of a separate overlay, and chained updates on the same ref are collapsed.

The second commit is separable: it adds the gt=0 bounds that SnapshotRef already has to SetSnapshotRefUpdate, so an invalid retention value fails when it is passed instead of at apply time, like SnapshotRef.Builder in Java. Happy to split it out if preferred.

Fast-forward a branch to the snapshot of another ref, following Java's
ManageSnapshots.fastForwardBranch: the ref to fast-forward to can be a
branch or a tag, a missing branch is created with default retention, and
a branch that is not an ancestor raises ValueError.

Operations chained in one manage_snapshots() see each other, and a ref
set twice in a chain results in a single set-snapshot-ref update.
rollback_to_snapshot and rollback_to_timestamp stage pending ref updates
before validating, like commitIfRefUpdatesExist in Java.

Supersedes apache#3649.
SnapshotRef requires min-snapshots-to-keep, max-snapshot-age-ms and
max-ref-age-ms to be greater than 0, but SetSnapshotRefUpdate did not,
so an invalid value only failed when the ref was built at apply time.
Java validates these in SnapshotRef.Builder when they are set.

@rambleraptor rambleraptor left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We already went through reviews, so this looks good to me. Thanks for doing this!

Comment thread tests/table/test_manage_snapshots.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants