Skip to content

Remove the Python 3.15 awswrangler and pyarrow markers (GH-1180) #1379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
laughingman7743 merged 1 commit into from
Nov 7, 2026
Merged

Remove the Python 3.15 awswrangler and pyarrow markers (GH-1180) #1379

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions benchmarks/pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,7 @@ requires-python = ">=3.11"
classifiers = ["Private :: Do Not Upload"]
dependencies = [
"PyAthena[pandas,arrow,polars,sqlalchemy]",
# Released awswrangler caps pyarrow below 26.0.0, the first release with
# Python 3.15 wheels; the marker keeps the workspace lock installable on 3.15.
"awswrangler>=3.17,<4; python_version < '3.15'",
"awswrangler>=3.18,<4",

@laughingman7743 laughingman7743 Nov 7, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review round one (implementation behavior) — b66b045d9c18da66d9ed8ca5db9ae8c22eb8cc77..b6ee461e73ee8320c97b93c23f67db817c1974b0

Covered: benchmarks/pyproject.toml, pyproject.toml [tool.uv], uv.lock (only awswrangler and pyarrow change; pyarrow now has a single entry with cp315 wheels), the benchmark adapters that import awswrangler.athena._read (benchmarks/tests/test_adapters.py:19), and the package metadata (pyarrow>=22.0.0 extras unchanged, so installed users are unaffected).

Result: CLEAN. The >=3.18 floor is required for the marker-free dependency to resolve on 3.15 (3.17.x caps pyarrow below 26).

Limitation: the new combination Polars 1.x + pyarrow 26 on Python 3.11–3.13 and pyarrow 26 on 3.14 is only exercised by the AWS suites; a local offline run is not possible because the test session needs Athena. Covered by dispatching the Test workflow for every supported Python version before Ready.

"polars",
"psutil>=7,<8",
]
Expand Down
3 changes: 0 additions & 3 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,9 +138,6 @@ source = "vcs"
version-file = "pyathena/_version.py"

[tool.uv]
# pyarrow 26.0.0 is the first release with Python 3.15 wheels; without this, every

@laughingman7743 laughingman7743 Nov 7, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent review (relayed) — Codex CLI (codex exec --sandbox read-only, model gpt-6-astra, session 01a1b4cf-ab32-7a12-b9bd-bc1aec9b1e42), static review of a detached snapshot at b6ee461e73ee8320c97b93c23f67db817c1974b0, diff b66b045d9c18da66d9ed8ca5db9ae8c22eb8cc77..b6ee461e73ee8320c97b93c23f67db817c1974b0. The prompt omitted the PR number, description, commit message and self-review results. The snapshot was unchanged afterwards.

Covered (reviewer): the diff, both manifests, lock markers and PyPI artifact metadata, CI/tox install paths, uv 0.12.5 benchmark setup, the three benchmark groups, awswrangler adapters/tests, Arrow/pandas/Polars conversion paths, remaining marker references. Lock check: pyarrow 26.0.1 with matching wheels on 3.11–3.15; Polars 1.44.2 on 3.11–3.13, 2.0.0 on 3.14–3.15. No resolution, install, build or test was executed by the reviewer.

Result: FINDINGS (1)

  • [P2] Removing the constraint leaves pyarrow>=22.0.0 as the 3.15 floor; a minimum-version re-lock (uv lock --resolution lowest-direct) could select pyarrow 22.0.0, which has no cp315 wheels, and fall back to a source build.

Disposition: rejected (verified). No workflow, recipe or config resolves the lock with lowest/lowest-direct (searched .github, justfile, pyproject.toml, tox.ini), and the committed lock is what CI installs. The same floor-versus-wheel gap exists for pandas>=3.0.0 and numpy (pandas 3.0.0 and numpy 2.0.0 have no cp315 wheels either), so a 3.15-only pyarrow constraint would not make a lowest resolution installable. Removing the constraint is the change agreed in #1180.

# resolution of the lock keeps the version that the benchmarks' awswrangler allows.
constraint-dependencies = ["pyarrow>=26.0.0; python_version >= '3.15'"]

@laughingman7743 laughingman7743 Nov 7, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review round two (compatibility, operations, claims) — b66b045d9c18da66d9ed8ca5db9ae8c22eb8cc77..b6ee461e73ee8320c97b93c23f67db817c1974b0

Claims audited against history: the constraint was moved to [tool.uv] by #1292 (33bf9e8), not #1164 — PR body corrected. Lock revision 5 dates from #1292 (8a95f9f); the benchmark stack's default UvVersion 0.12.5 (benchmarks/cloudformation/benchmark.yaml:56) synced it in the 2026-10-19 and 2026-10-23 measurements, so no uv bump is needed. awswrangler 3.18.0 was released 2026-10-27; pyarrow 26.0.1 on 2026-11-03 (the repository sets no release-age cooldown).

Operational: CI on 3.11–3.14 moves from pyarrow 25.0.1 to 26.0.1; no CloudFormation or workflow change.

Result: FINDINGS (1, PR text only) — repaired in the PR body; no code change.

# The benchmark measures PolarsCursor in one environment per Polars major version;
# the root development group pins Polars by Python version instead. The benchmark's
# s3fs group brings aiobotocore, which pins botocore to a few patch releases;
Expand Down
Loading
Loading