[laughingman7743]

WHAT

The shared converter for timestamp with time zone raises DataError when the text does not identify one instant, instead of returning a naive or arbitrarily resolved datetime. This applies to the fetch methods of every cursor that converts values (all but SparkCursor), to values inside type-hinted arrays, maps, and rows, and to PandasCursor, which uses the same converter for these columns. The asynchronous cursors share the converter.

Text	Before	After
Zone name unknown to Python's time zone database	naive datetime, one warning per name	DataError: Unknown time zone 'X'. Update the tzdata package or the system time zone database.
Local time that a DST transition repeats (2020-11-01 01:30:00.000 America/New_York)	aware datetime with fold=0 (the earlier instant)	DataError: '...' is ambiguous: the local time occurs twice in America/New_York. Convert the value to UTC in SQL, for example with AT TIME ZONE 'UTC'.
Local time that a DST transition skips (2020-03-08 02:30:00.000 America/New_York)	aware datetime with fold=0	DataError: '...' does not exist: the local time is skipped in America/New_York. ...
Offsets, UTC, other zone names	unchanged	unchanged

The dependency tzdata now has the floor >=2024.2 (IANA 2024b).

Exception classes per cursor for the ambiguous value, measured on live Athena:

• Cursor (raised by execute(), which converts the first page), S3FSCursor, ArrowCursor and PolarsCursor fetch methods, and PandasCursor chunked reads: DataError.
• PandasCursor whole reads: OperationalError caused by the DataError, because the reader wraps read failures. Name the column in DataError for time zone values in every cursor #1217 unwraps it and adds the column name to every temporal DataError.

Arrow and Polars tables keep these columns as text, so only their fetch methods are affected.

Release note (4.2.0, breaking): values of timestamp with time zone with an unknown zone name, or with a local time that a DST transition repeats or skips, raise DataError instead of returning a naive datetime or the earlier instant. Convert such values to UTC in SQL (AT TIME ZONE 'UTC') or read to_unixtime(). tzdata>=2024.2 is required.

WHY

Closes #1220. Part of #1212. The decision and live measurements are in #1220 (comment):

• Athena writes two instants in the DST fall-back hour as the same text, so neither instant can be chosen without risking a wrong value.
• Athena never writes a skipped local time. One appears only when Python's tz rules differ from Athena's.
• Iceberg time zone columns read back as UTC, and Hive tables have no time zone type, so named zones come from the query (at_timezone(), AT TIME ZONE, with_timezone()), where converting them gives the exact instant.
• Every zone key in Trino's zone index is in tzdata 2024b except America/Coyhaique, which Athena rejects.

TEST

Tested at 6b07242 (Python 3.15, pandas 3.0.6, pyarrow 26.0.0, Polars 2.0.0). The later commits change only pyathena/converter.py (docstring, early return for fixed offsets) and tests/pyathena/test_converter.py; at cabcb4e, lint and tests/pyathena/test_converter.py (190 passed) passed again:

• UV_PYTHON=3.13 just lint: passed.
• uv run --env-file .env pytest -n 4 tests/pyathena/test_converter.py tests/pyathena/test_cursor.py tests/pyathena/s3fs/test_cursor.py tests/pyathena/arrow/test_cursor.py tests/pyathena/polars/test_cursor.py tests/pyathena/pandas/test_cursor.py: 741 passed, 1 skipped (s3fs/test_cursor.py:253, an unconditional skip on master). This run preceded the type-hinted array test; tests/pyathena/test_converter.py passed afterwards (190 passed).
• The new converter tests for unknown, unreadable, ambiguous, and nonexistent zones fail on master's converter.
• Live tests run the ambiguous value through Cursor (result file and managed results), S3FSCursor, ArrowCursor, PolarsCursor, and PandasCursor (whole and chunked).
• Per-value cost of _to_datetime_with_tz at cabcb4e (same interpreter, 200,000 calls, best of 7): UTC 3.71 → 4.09 µs (+10%), America/New_York 3.76 → 4.17 µs (+11%), +05:30 3.71 → 3.82 µs (+3%; fixed offsets skip the transition check). Make PandasCursor's temporal dtypes and fetched types depend only on the Athena type #1219 replaces PandasCursor's per-value conversion with a vectorized one.
• uv.lock changes only the tzdata specifier; uv lock --check passes with uv 0.12.5 (lock revision 3) and 0.13.1.

Not run: async cursors (they share the result sets and converter) and the SQLAlchemy suites. The full AWS suite runs in CI when the PR is marked Ready.

🤖 Generated with Claude Code

[laughingman7743]

Self-review round one (behavior and implementation)
Base b2d7aa24b00a204e9e150952c708e3bb1126f9e2, head 6b072428e15b7413b849d13fdfcc4582945a9cc4; repair head 0ebd096b.

Covered:

• Callers of _to_datetime_with_tz: the default, pandas, Arrow, and Polars converter maps, and type-hinted nested values (DataError propagates, checked).
• Exception handlers on fetch paths: result_set.py:455, s3fs/result_set.py:210, polars/result_set.py:124, and PandasCursor's whole-read wrapping into OperationalError (documented in the PR, unwrapped in Name the column in DataError for time zone values in every cursor #1217).
• NULL and empty text, UTC, offsets (+14:00, -14:00), years 1 and 9999 including ambiguous and skipped times in 9999 (no OverflowError from timestamp() in the error branch).
• _parse_time_zone caching (failures are not cached); unknown and unreadable names.
• Tests: expected instants around transitions are independent of the implementation; the 11 new converter cases fail on master.

Result: FINDINGS (two minor issues), repaired in 0ebd096b:

1. The docstring said a DST transition, but the check covers any change of the zone's UTC offset; reworded.
2. test_to_datetime_with_tz_unknown_zone_name cleared the cache although failures are not cached; removed.

Out of scope: the column name in the message (#1217), vectorized pandas checks (#1219), and async cursors, which share the converter and the result sets and were not run.

[laughingman7743]

Repair record (extra /code-review and /simplify passes, by the author; head cabcb4e3)

Applied:

• 0ebd096b..cabcb4e3: bind folded = value.replace(fold=1) once.
• Return early for a fixed-offset timezone, which has no transitions. Offsets now cost +3% per value instead of +9%.
• Merge the ambiguous and skipped local time tests into one parametrized test.
• Drop the cache_clear() calls, which no longer guard anything because failed lookups are not cached.
• Import DataError from pyathena.error.

Rejected or deferred:

• A stale system tz database (searched before the tzdata package) can turn a valid value into DataError. This is intended over a silently wrong instant, and the limitation is documented in Document the temporal type contract and gate the release on it #1221.
• PandasCursor whole reads wrap the error in OperationalError (Name the column in DataError for time zone values in every cursor #1217).
• Not caching failed lookups costs only on the error path, which aborts the read.
• No cache can leak from a mock that always raises.
• The UTC and -04:00 cases are kept on purpose: they show that the repeated local time converts when the zone has no transition. Their docstring now says so.
• Comparing utcoffset() directly would need an Optional workaround for mypy.

Validation at cabcb4e3: UV_PYTHON=3.13 just lint passed, and tests/pyathena/test_converter.py passed (190). Both self-review perspectives applied to the repair: the change of behavior is limited to the early return for datetime.timezone (ZoneInfo is not a subclass), and the PR body's performance and test claims were updated.

[laughingman7743]

Self-review round two (claims, callers, and operations)
Base b2d7aa24b00a204e9e150952c708e3bb1126f9e2, head 0ebd096bdd (full pass over the PR body, commit messages, docstrings, and the issue premises they repeat).

Claims checked:

• Before/after table: master's naive fallback with one warning and the fold=0 result were reproduced; the new messages are taken from the code.
• Per-cursor exception classes: measured on live Athena with this branch; Cursor raises in execute() (the tests assert it, including managed results).
• Type-hinted arrays, maps, and rows raise DataError (checked for each).
• tzdata 2024.2 is IANA 2024b (tzdata.IANA_VERSION); uv lock --check passes with uv 0.12.5 and 0.13.1.
• Benchmark: same interpreter (Python 3.15), master's module loaded from origin/master.
• Docs: no page describes the naive fallback; docs/s3fs.md (aware datetime) and docs/pandas.md stay correct.

Callers and operations:

• SQLAlchemy receives the DB API DataError.
• Installations pinning tzdata<2024.2 will fail to resolve; this is in the release note.
• No AWS requests are added.
• A false DataError is possible when Python's tz rules have a transition that Athena's do not; that skew limitation is documented under Document the temporal type contract and gate the release on it #1221.

Result: FINDINGS, PR body corrected:

1. "Every cursor" was wrong; SparkCursor does not convert values, so the claim is narrowed.
2. The claim that named zones come only from queries is limited to what was measured (Iceberg reads back as UTC; Hive has no time zone type).
3. The tzdata 2024b coverage claim is stated as the Trino zone-index comparison it rests on.
4. The test record names the repair head.

No code changes in this round.

[laughingman7743]

Independent review (relayed result from Codex CLI, codex exec -s read-only, model gpt-6-astra, session 01a1470d-dcec-76f0-ac96-44fd17980d97)

• Base: b2d7aa24b00a204e9e150952c708e3bb1126f9e2. Head: cabcb4e3.
• Snapshot: a detached worktree of the head. It was unchanged afterwards, and so was the PR worktree.
• The prompt contained the intended behavior and the conventions only: no PR number, description, commit messages, or prior findings.

Reviewer's coverage:

• The shared converter: unknown zones, repeated and skipped local times, transition boundaries, offsets, UTC, and NULL or empty values. Fold handling was checked against PEP 495.
• Parser and typed nested conversions; synchronous and asynchronous result sets; the pandas, Arrow, Polars, and S3FS callers.
• The changed unit and integration tests, independent expected values, and the wiring of the real Athena fixtures.
• The tzdata>=2024.2 floor and the consistency of the lock file (revision 3, resolved 2024.2).

Result: CLEAN. No actionable findings against the stated behavior, including the accepted PandasCursor wrapping and the absence of the column name.

Limits: static review only. No builds, tests, or AWS execution.