Skip to content

Give the S3 filesystem its own default retry codes #1375

Open

Description

@laughingman7743

Part of #1342 (item 4).

Decision

The filesystem gets its own default S3 retry codes, separate from Athena's. When the retry policy of a connection is still the default, the S3 codes are added. A policy that the user configured is used as given.

Current state (29cc1d3, offline)

The default RetryConfig retries THROTTLING_ERROR_CODES (util.py:56), which are Athena's codes. With one injected error each, PyAthena's layer made 2 attempts on ThrottlingException, and 1 attempt on S3's SlowDown (OSError(EBUSY)), ServiceUnavailable (OSError(EBUSY)), InternalError (OSError(EIO)), and RequestTimeout (TimeoutError). Only botocore retries S3 errors, so stop_retries_on and unstoppable_retries govern no S3 retry by default. docs/filesystem.md:482 says only that requests are "sent with the retry policy".

To decide in the PR

The exact codes, and how the two retry layers (botocore and PyAthena) compose in attempts and elapsed time; the docs state both.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions