You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
REST catalog: send Idempotency-Key on mutations and retry safely when the server advertises it #4290
The REST catalog spec defines an optional Idempotency-Key header for mutating requests. A server signals support by returning idempotency-key-lifetime (an ISO-8601 duration such as PT30M) in the GET /v1/config response. PyIceberg does not send the header yet.
Java iceberg-core already implements this. The reference PRs are apache/iceberg#14740(send the header), #17947 (retry keyed requests on 5xx), and #18070 (stop retrying after the lifetime window).
Lifetime: idempotency-key-lifetime in the config response, same file
Requested behavior
Read idempotency-key-lifetime from the config response. If it is absent, send no header.
When it is present, send Idempotency-Key with a new UUIDv7 on mutating calls. The spec declares the parameter on createNamespace, dropNamespace, updateProperties, createTable, and registerTable, among others. Do not send it on reads.
Reuse the same key when retrying the same logical operation. Use a new key for a different operation.
Retry only with the same key. Stop retrying once the lifetime window has passed.
Test idea: a mock server returns 503 on the first commit and 200 on the second. Assert that both requests carry the same key and that a read request has no key.
* Drafted with AI assistance. I reviewed the spec links and reference PRs.
Thanks for @YuangGao@Fokko for landing #4310. One follow-up on the lifetime bound, since the PR description says it isn't needed.
The bound matters most when the server is overloaded. An overloaded server answers with 503 plus Retry-After. urllib3 honors that header by default and sleeps for the full value with no cap (backoff_max only applies to backoff). So with idempotency-key-lifetime: PT5M, a single Retry-After: 600 sends the retry after the server may have dropped the key.
A keyed retry outside the window is dangerous:
The first commit lands, but the response is lost.
The client retries after the window with the same key.
The server no longer has the key, so it treats the retry as a new request and applies the mutation a second time.
That's a duplicate commit and corrupted table state. The key exists to prevent exactly that. The spec covers it too: "Clients SHOULD NOT reuse an Idempotency-Key after this window elapses." Java enforces it in ExponentialHttpRequestRetryStrategy (apache/iceberg#18070).
I'll file a follow-up issue to bound keyed retries, including Retry-After sleeps, to the advertised lifetime.
Activity
HotSushi commentedon Nov 19, 2026
Thanks for @YuangGao @Fokko for landing #4310. One follow-up on the lifetime bound, since the PR description says it isn't needed.
The bound matters most when the server is overloaded. An overloaded server answers with 503 plus Retry-After. urllib3 honors that header by default and sleeps for the full value with no cap (backoff_max only applies to backoff). So with idempotency-key-lifetime: PT5M, a single Retry-After: 600 sends the retry after the server may have dropped the key.
A keyed retry outside the window is dangerous:
That's a duplicate commit and corrupted table state. The key exists to prevent exactly that. The spec covers it too: "Clients SHOULD NOT reuse an Idempotency-Key after this window elapses." Java enforces it in ExponentialHttpRequestRetryStrategy (apache/iceberg#18070).
I'll file a follow-up issue to bound keyed retries, including Retry-After sleeps, to the advertised lifetime.