Docs: document the Transaction write API (#1008) - #3809
Conversation
|
@kevinjqliu this picks up #1008 after the handoff from @pramila-bishnoi. It's a small docs-only change: adds the Transaction write API (which was missing from this section) and a Delete subsection so each operation has its own heading. Would appreciate a review when you get a chance. Thanks! |
|
Gentle bump on this one. CI is green and it's a small docs-only change picking up #1008 after @pramila-bishnoi's handoff. @kevinjqliu @Fokko whenever one of you has a moment for a quick review, would really appreciate it. Happy to adjust anything. |
|
|
||
| Every write operation is available through two APIs. | ||
|
|
||
| The **`Table` API** exposes each operation directly on the table object: `tbl.append(...)`, `tbl.overwrite(...)`, `tbl.delete(...)`, `tbl.dynamic_partition_overwrite(...)` and `tbl.upsert(...)`. Each call opens a transaction, applies the single operation, and commits it as one atomic snapshot. This is the simplest mode and the right default when you only need a single write. |
There was a problem hiding this comment.
A small nit in the wording. The tbl.upsert(...) could potentially generate more than one snapshot, but it will be written atomically. Maybe we should be less strict in the wording:
| The **`Table` API** exposes each operation directly on the table object: `tbl.append(...)`, `tbl.overwrite(...)`, `tbl.delete(...)`, `tbl.dynamic_partition_overwrite(...)` and `tbl.upsert(...)`. Each call opens a transaction, applies the single operation, and commits it as one atomic snapshot. This is the simplest mode and the right default when you only need a single write. | |
| The **`Table` API** exposes each operation directly on the table object: `tbl.append(...)`, `tbl.overwrite(...)`, `tbl.delete(...)`, `tbl.dynamic_partition_overwrite(...)` and `tbl.upsert(...)`. Each call opens a transaction, applies the operation, and commits atomically. This is the simplest mode and the right default when you only need a single write. |
|
@iSparshP I left one comment, apart from that, this looks good. Thanks for picking this up 👍 |
upsert may produce more than one snapshot while still committing atomically, so describe the Table API as committing atomically rather than as a single snapshot.
|
thanks @Fokko, good catch. you are right that upsert can end up as more than one snapshot even though its still atomic, so the old wording was too strict. i took your suggestion and reworded it to just say it commits atomicaly instead of as one snapshot. pushed the change, let me know if you want anything else tweaked. |
rambleraptor
left a comment
rambleraptor
left a comment
There was a problem hiding this comment.
Thanks so much for doing this!
|
|
||
| The **`Table` API** exposes each operation directly on the table object: `tbl.append(...)`, `tbl.overwrite(...)`, `tbl.delete(...)`, `tbl.dynamic_partition_overwrite(...)` and `tbl.upsert(...)`. Each call opens a transaction, applies the operation, and commits atomically. This is the simplest mode and the right default when you only need a single write. | ||
|
|
||
| The **`Transaction` API** exposes the same operations on a transaction object obtained from `tbl.transaction()`. It batches multiple operations into a single atomic commit: either every operation becomes visible together, or, on failure, none of them do, and readers never observe an intermediate state. |
There was a problem hiding this comment.
| The **`Transaction` API** exposes the same operations on a transaction object obtained from `tbl.transaction()`. It batches multiple operations into a single atomic commit: either every operation becomes visible together, or, on failure, none of them do, and readers never observe an intermediate state. | |
| The **`Transaction` API** exposes the same operations on a transaction object obtained from `tbl.transaction()`. It batches multiple operations into a single atomic commit. On success, every operation becomes visible together. On failure, none of the operations occur and readers never observe an immediate state |
Quick change to make this easier to read
|
thanks @rambleraptor for the suggestion, applied it. i kept the wording "intermediate state" instead of "immediate" since the point is that readers never see a half applied state, but reworded into three short sentences like you suggested for readability. @Fokko your earlier nit about the atomicity wording is addressed too, and @rambleraptor has approved. would you be able to give this a quick look / approve when you get a chance? it's docs only and CI is green. happy to tweak anything else. thanks! |
|
@iSparshP yep, that was a typo on my part. Thanks for that! |
3 participants
Addresses #1008. The write docs already cover append / overwrite /
overwrite_filter/ delete /dynamic_partition_overwrite/ upsert, but the Transaction API was undocumented in this section. This adds a "Write API modes:TableandTransaction" subsection explaining both modes, the single-atomic-commit guarantee, and combining schema + data changes in one commit. It also adds a### Deleteheading so each operation is its own subsection, per the issue.Builds on the direction from the earlier #2770 and #3647 (both closed by the stale bot). Handed off by @pramila-bishnoi on the issue thread.
Closes #1008