Repository navigation
Description
Feature Request / Improvement
Follow-up to #4290 and #4310 cc: @YuangGao @Fokko
#4310 sends Idempotency-Key on mutations and retries keyed POSTs under rest.client.*, but it doesn't enforce the lifetime the server advertises in GET /v1/config. The PR describes this as intentional because urllib3 retries are bounded. They aren't bounded by the key lifetime, though, and a retry outside that window can apply a mutation twice.
The REST catalog spec says:
Clients SHOULD NOT reuse an Idempotency-Key after this window elapses.
Java enforces this in ExponentialHttpRequestRetryStrategy (apache/iceberg#18070, building on apache/iceberg#17947).
Why it matters
An overloaded server responds with 503 and Retry-After. urllib3 honors Retry-After by default (respect_retry_after_header=True) and sleeps for the full value. backoff_max caps backoff only, not Retry-After.
Example with idempotency-key-lifetime: PT5M:
- The client sends a commit. The server applies it, but the response is lost (timeout, proxy 502, dropped connection).
- The retry gets
503withRetry-After: 600. - urllib3 sleeps 600s and retries with the same key.
- The server has evicted the key, treats the request as new, and applies the mutation again.
The result is a duplicate commit and corrupted table state, which is exactly what the key is supposed to prevent.
Backoff alone can also exceed short windows. For example, max-retries=5, backoff-factor=10 and a 30s socket timeout add up to about 450s of retrying.
Current behavior
- In
pyiceberg/catalog/rest/__init__.py,ConfigResponse.idempotency_key_lifetimeis read as a string and only checked for presence. _create_connection_adapter(..., retry_post=True)addsPOSTtoallowed_methods. Nothing parses the lifetime or compares it with elapsed time.PT0Sor a negative duration still turns on keyed retries. In Java, a non-positive lifetime means keyed requests are never retried.- A malformed value is accepted silently.
Requested behavior
- Parse
idempotency-key-lifetimeas an ISO-8601 duration, e.g. withpydantic.TypeAdapter(timedelta).- Malformed: log a warning and disable the bound. The key is still sent, matching Java's
HTTPClient.parseKeyLifetime. - Zero or negative: never retry keyed requests.
- Malformed: log a warning and disable the bound. The key is still sent, matching Java's
- Record when the first attempt of a keyed request started.
- Before each retry of a keyed request, decline the retry if
elapsed + next_sleep >= lifetime.next_sleepis theRetry-Aftervalue when present, otherwise the backoff time including jitter. When a retry is declined, return the last response so_handle_non_200_responseraises the typed error, as it does today when retries run out. - Leave non-keyed requests and idempotent methods (
GET,HEAD,PUT,DELETE) without a key as they are.
Activity