Repository navigation
Remove the Python 3.15 awswrangler and pyarrow markers (GH-1180) - #1379
Conversation
awswrangler 3.18.0 allows pyarrow 26, the first release with Python 3.15 wheels, so the benchmarks install awswrangler on every supported Python version and the lock no longer needs a pyarrow constraint for 3.15. The lock now resolves one pyarrow version (26.0.1) for every Python version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| # Released awswrangler caps pyarrow below 26.0.0, the first release with | ||
| # Python 3.15 wheels; the marker keeps the workspace lock installable on 3.15. | ||
| "awswrangler>=3.17,<4; python_version < '3.15'", | ||
| "awswrangler>=3.18,<4", |
There was a problem hiding this comment.
Self-review round one (implementation behavior) — b66b045d9c18da66d9ed8ca5db9ae8c22eb8cc77..b6ee461e73ee8320c97b93c23f67db817c1974b0
Covered: benchmarks/pyproject.toml, pyproject.toml [tool.uv], uv.lock (only awswrangler and pyarrow change; pyarrow now has a single entry with cp315 wheels), the benchmark adapters that import awswrangler.athena._read (benchmarks/tests/test_adapters.py:19), and the package metadata (pyarrow>=22.0.0 extras unchanged, so installed users are unaffected).
Result: CLEAN. The >=3.18 floor is required for the marker-free dependency to resolve on 3.15 (3.17.x caps pyarrow below 26).
Limitation: the new combination Polars 1.x + pyarrow 26 on Python 3.11–3.13 and pyarrow 26 on 3.14 is only exercised by the AWS suites; a local offline run is not possible because the test session needs Athena. Covered by dispatching the Test workflow for every supported Python version before Ready.
| [tool.uv] | ||
| # pyarrow 26.0.0 is the first release with Python 3.15 wheels; without this, every | ||
| # resolution of the lock keeps the version that the benchmarks' awswrangler allows. | ||
| constraint-dependencies = ["pyarrow>=26.0.0; python_version >= '3.15'"] |
There was a problem hiding this comment.
Self-review round two (compatibility, operations, claims) — b66b045d9c18da66d9ed8ca5db9ae8c22eb8cc77..b6ee461e73ee8320c97b93c23f67db817c1974b0
Claims audited against history: the constraint was moved to [tool.uv] by #1292 (33bf9e8), not #1164 — PR body corrected. Lock revision 5 dates from #1292 (8a95f9f); the benchmark stack's default UvVersion 0.12.5 (benchmarks/cloudformation/benchmark.yaml:56) synced it in the 2026-10-19 and 2026-10-23 measurements, so no uv bump is needed. awswrangler 3.18.0 was released 2026-10-27; pyarrow 26.0.1 on 2026-11-03 (the repository sets no release-age cooldown).
Operational: CI on 3.11–3.14 moves from pyarrow 25.0.1 to 26.0.1; no CloudFormation or workflow change.
Result: FINDINGS (1, PR text only) — repaired in the PR body; no code change.
| version-file = "pyathena/_version.py" | ||
|
|
||
| [tool.uv] | ||
| # pyarrow 26.0.0 is the first release with Python 3.15 wheels; without this, every |
There was a problem hiding this comment.
Independent review (relayed) — Codex CLI (codex exec --sandbox read-only, model gpt-6-astra, session 01a1b4cf-ab32-7a12-b9bd-bc1aec9b1e42), static review of a detached snapshot at b6ee461e73ee8320c97b93c23f67db817c1974b0, diff b66b045d9c18da66d9ed8ca5db9ae8c22eb8cc77..b6ee461e73ee8320c97b93c23f67db817c1974b0. The prompt omitted the PR number, description, commit message and self-review results. The snapshot was unchanged afterwards.
Covered (reviewer): the diff, both manifests, lock markers and PyPI artifact metadata, CI/tox install paths, uv 0.12.5 benchmark setup, the three benchmark groups, awswrangler adapters/tests, Arrow/pandas/Polars conversion paths, remaining marker references. Lock check: pyarrow 26.0.1 with matching wheels on 3.11–3.15; Polars 1.44.2 on 3.11–3.13, 2.0.0 on 3.14–3.15. No resolution, install, build or test was executed by the reviewer.
Result: FINDINGS (1)
- [P2] Removing the constraint leaves
pyarrow>=22.0.0as the 3.15 floor; a minimum-version re-lock (uv lock --resolution lowest-direct) could select pyarrow 22.0.0, which has no cp315 wheels, and fall back to a source build.
Disposition: rejected (verified). No workflow, recipe or config resolves the lock with lowest/lowest-direct (searched .github, justfile, pyproject.toml, tox.ini), and the committed lock is what CI installs. The same floor-versus-wheel gap exists for pandas>=3.0.0 and numpy (pandas 3.0.0 and numpy 2.0.0 have no cp315 wheels either), so a 3.15-only pyarrow constraint would not make a lowest resolution installable. Removing the constraint is the change agreed in #1180.
WHAT
python_version < '3.15'marker from the benchmarks'awswranglerdependency and raise its floor to 3.18 (awswrangler>=3.18,<4).[tool.uv] constraint-dependenciesentrypyarrow>=26.0.0; python_version >= '3.15'and its comment. Measure PolarsCursor on both Polars 1.x and 2.x in the benchmark harness #1292 moved it there from the dev group that Support Python 3.15 #1164 added it to.uv lock --upgrade-package awswrangler --upgrade-package pyarrow: awswrangler 3.17.1 → 3.18.0, and pyarrow 25.0.1 (3.11–3.14) + 26.0.0 (3.15) → 26.0.1 for every supported Python version. No other package changes. The lock stays at revision 5, which the benchmark stack's default uv 0.12.5 has read since Measure PolarsCursor on both Polars 1.x and 2.x in the benchmark harness #1292 (the 2026-10-19 and 2026-10-23 measurements).WHY
Closes #1180.
awswrangler 3.18.0 (released 2026-10-27) allows
pyarrow<27.0.0, so the markers that kept the workspace lock installable on Python 3.15 are no longer needed, and pyarrow is no longer locked at different versions for 3.11–3.14 and 3.15.TEST
At b6ee461:
uv lock --check: passed.uv sync --group dev --python 3.15in a fresh environment: Python 3.15.2, pyarrow 26.0.1, Polars 2.0.0.uv sync --locked --python 3.15inbenchmarks/installs awswrangler 3.18.0 and pyarrow 26.0.1.just lint: passed.just benchmark test: 340 passed, 1 skipped.🤖 Generated with Claude Code