[laughingman7743]

WHAT

Lets the benchmark harness measure PolarsCursor on both Polars 1.x and 2.x, in one queue on one stack.

• Dependency groups.
  • benchmarks/pyproject.toml gets polars1 = ["polars>=1.39.0,<2"] and polars2 = ["polars>=2,<3"], with default-groups = ["test", "polars1"].
  • The root pyproject.toml declares these two groups and the root dev group as one [tool.uv] conflicts set, so uv.lock resolves each separately.
  • The dev group's Python 3.15 pin pyarrow>=26.0.0; python_version >= '3.15' moves to the root [tool.uv] constraint-dependencies. Inside the conflict set, the pin would no longer reach the other resolutions, and a root sync without dev would select PyArrow 25.0.1, which has no Python 3.15 wheel, on Python 3.15.
  • Lock changes: the selected package versions are unchanged except Polars in the root resolution without dev, which is now 2.0.0 on every Python. Master also selected 1.44.2 there for Python < 3.14; the polars<2 pin is in the dev group. No workflow installs the polars extra without dev.
  • The benchmark's own dev group is renamed to test, because uv matched --group dev from benchmarks/ with the root dev group and rejected it with polars1.
• Environments. EC2 bootstrap syncs polars1 into /opt/pyathena/.venv (as before) and polars2 into /opt/pyathena/.venv-polars2. Locally, just benchmark test runs the tests in benchmarks/.venv and benchmarks/.venv-polars2.
• Jobs.
  • jobs --environment polars2 adds "environment": "polars2" to each job and a -polars2 suffix to its ID. One queue can then hold the same cases for both Polars versions: claims, done markers, results, and resume are all keyed by job ID.
  • run_job() runs such a job with the interpreter of .venv-<environment> beside the worker's own environment (fleet.interpreter()).
  • work() refuses to start, before claiming, if a job's environment is missing on the host.
  • merge_jobs() (the queue command) rejects an unknown environment before publishing.
  • Jobs without the field run as before.
• README documents the groups, the environments, local use of the second environment, and jobs --environment. .gitignore and the benchmark's ruff exclude cover .venv-*.

WHY

Closes #1292 (part of #1288). The benchmark runs on Python 3.12, where the root dev group's polars<2.0.0; python_version < '3.14' made the shared lock resolve Polars 1.44.2 for the benchmark too. So the combined run could not measure the Polars 2.0 UNLOAD path that #1287 names, and PyAthena supports both lines (polars>=1.39.0).

The maintainer decided the following in #1292:

• the harness tests run in both environments;
• uv 0.12.5 on EC2 stays;
• the combined run starts after the open issues are resolved.

The design keeps one stack, and so one prepared snapshot, rather than a stack per environment. Preparation selects rows with LIMIT and no ordering, so a second stack's snapshot could hold different rows.

TEST

Tested commit: 33bf9e8

• just benchmark test: lint and mypy in benchmarks/.venv, then pytest in both environments. 256 passed, 1 skipped in benchmarks/.venv (Polars 1.44.2), and 256 passed, 1 skipped in benchmarks/.venv-polars2 (Polars 2.0.0).

• just lint, just docs lint, uv lock --check: passed.

• New tests:

  • test_jobs_of_another_environment_are_named_and_marked_for_it
  • test_jobs_command_selects_the_environment
  • test_a_job_of_another_environment_runs_its_interpreter
  • test_jobs_of_a_missing_environment_stop_the_worker_before_claiming
  • test_unknown_environments_are_rejected_before_publishing
  • a user-data assertion in test_hosts_start_the_queue_worker_and_expose_their_tags

• Mutation checks: each of the following makes one of the new tests fail:

  • removing the ID suffix;
  • running sys.executable instead of the environment's interpreter;
  • removing the missing-environment check;
  • dropping --environment from the CLI wiring;
  • removing the unknown-environment check in merge_jobs().

• Dependency resolution, checked locally with uv 0.13.1 on Python 3.12 unless noted:

  Command	Polars
  uv run --locked from benchmarks/ (default groups)	1.44.2
  --no-group polars1 --group polars2	2.0.0
  --group polars2 alone	rejected as conflicting with the default polars1
  uv sync --package pyathena-benchmarks --locked (the CI step)	1.44.2
  --no-default-groups without a Polars group	2.0.0 (the README says to add a group)
  Root uv sync --locked, Python 3.15	2.0.0, unchanged
  Root uv sync --locked, Python 3.12	1.44.2, unchanged
  uv 0.12.5 (the EC2 default), --no-default-groups --group polars1	1.44.2
  uv 0.12.5, --no-default-groups --group polars2	2.0.0

  Root uv run --locked --no-dev and uv sync --group dev (used by other workflows) also work.

  Compared with master, uv export --locked --all-extras, with and without --no-dev, on Python 3.11-3.15, and the benchmark's default groups differ only in the Polars entries above. PyArrow is 25.0.1 below Python 3.15 and 26.0.0 on 3.15 in every resolution, as on master.

Not run:

• A stack deployment, whose bootstrap now runs two syncs.
• A fleet run with --environment polars2 jobs.

Both belong to the combined run of #1288.

🤖 Generated with Claude Code

[laughingman7743]

Self-review round one (behavior and implementation), base d8526ef, initial head 8a95f9f, repaired head 81b5aa6.

Covered:

• the dependency groups and the conflict set, with each sync path checked locally;
• expand_jobs(), merge_jobs(), interpreter(), run_job(), and work();
• the CLI wiring, the EC2 user data, the justfile, .gitignore, the ruff exclude, the README, and the new tests.

Passes: my own read, /code-review (high), and /simplify (four angles).

Result: FINDINGS, repaired.

1. queue accepted any environment value, so a typo would surface only on the hosts. Every worker would then refuse to start and keep its host. Repaired in 17c635b: merge_jobs() rejects an unknown environment before publishing (test_unknown_environments_are_rejected_before_publishing).
2. The environment was applied in two places: a field spread into both job literals, and the ID suffix on base. Repaired in 17c635b: both are set in one loop after expansion.
3. Two operator pitfalls are now documented in the README (17c635b):
   • a sync without either group gets whatever Polars version the lock resolves;
   • a worker started from .venv-polars2 would run the default jobs on Polars 2.
4. Cleanups in 81b5aa6:
   • a shared BENCHMARK_RUN in the justfile;
   • a processes fixture for the two run_job() tests;
   • j.get("environment") not in (None, *ENVIRONMENTS).

[laughingman7743]

Round one, deferrals with reasons:

• mypy in .venv-polars2: added in 17c635b, then removed in 81b5aa6. The harness uses only pl.__version__, and follow_imports = "silent" hides errors in the PyAthena modules it follows. The two runs also share one .mypy_cache, so each invalidated the other's entries. The pytest run in both environments, which the maintainer asked for, stays.
• Summary rows without the environment or Polars version (report.summarize() keys on (scale, case_id)): each job's summary is in a job directory whose ID ends with -polars2, and environment.json records the installed versions. Adding a column changes the report format, which is outside this PR.
• Enforcing polars/sqla_polars for --environment polars2: the README recommends them. A non-Polars control can still be queued on purpose, because every trial imports Polars.
• Running only tests/test_adapters.py in the second environment: Measure PolarsCursor on both Polars 1.x and 2.x in the benchmark harness #1292 decided on the full suite in both.
• Parallel syncs in bootstrap: the second sync links from the uv cache, and background jobs would escape set -e.
• Unifying --no-default-groups --group polars2 (bootstrap) and --no-group polars1 --group polars2 (justfile, README): the bootstrap excludes the test group on purpose.

[laughingman7743]

Round one, evidence:

• Mutation checks: each of the following makes one of the new tests fail:
  • removing the ID suffix;
  • sys.executable in run_job();
  • removing the missing-environment check;
  • dropping --environment from the CLI;
  • removing the unknown-environment check in merge_jobs().
• just benchmark test at 81b5aa6: 256 passed, 1 skipped in each environment (Polars 1.44.2 and 2.0.0).
• uv lock --check: passed.
• The lock diff changes forks and markers only; package versions are unchanged.

[laughingman7743]

Self-review round two (claims, callers, operations), base d8526ef, initial head 81b5aa6, repaired head 6d3f219.

Claims checked: the PR body, README, the comments in the root pyproject.toml and fleet.py, the justfile comment, and the commit messages.

Result: FINDINGS, repaired.

1. The README said to select one of the two groups in every sync. That was too broad: the default groups include polars1, so a plain sync is correct. The real pitfall is a sync with --no-default-groups alone, which installed Polars 2.0.0 when checked locally. Repaired in 6d3f219: the sentence now names that case.
2. The PR body's test counts, test list, and mutation checks predated round one. They are updated, and the description now mentions the merge_jobs() check.

[laughingman7743]

Round two, evidence for the claims:

Commands checked:

• The README's local Polars 2 example (UV_PROJECT_ENVIRONMENT="$PWD/.venv-polars2" uv run --locked --no-group polars1 --group polars2 python -m pyathena_bench plan ... --family polars) runs and prints 30 cases.
• uv 0.12.5 (the EC2 UvVersion default) syncs this lock with each bootstrap command, installing Polars 1.44.2 and 2.0.0.
• The CI step uv sync --package pyathena-benchmarks --locked gives 1.44.2 with the test group.
• Root uv sync --locked (Python 3.15 → 2.0.0, Python 3.12 → 1.44.2), uv run --locked --no-dev, and uv sync --group dev are unchanged.

Callers: jobs without environment produce the same IDs and dictionaries as before, so existing job files and queues keep working. Claims, done markers, results, and resume use the job ID, which the suffix separates (fleet.py run_attempt() and work()).

Operator view:

• Bootstrap runs a second uv sync, which links from the uv cache.
• chown -R /opt/pyathena covers the new environment.
• A worker refuses a queue whose environment is missing before claiming, as it already does for jobs wider than --api-slots. Such a worker keeps its host, so queue validates the names first.
• Because the root pyproject.toml and uv.lock change, the AWS test jobs will run when this PR is marked Ready. The root environments' package versions are unchanged.

Limits: no stack deployment or fleet run, which belong to #1288.

[laughingman7743]

Independent review (relayed), static only.

Reviewer: Codex CLI 0.163.0, model gpt-6-astra, reasoning effort max, codex exec --sandbox read-only, session 01a14f6b-c5bd-7831-b83c-4cb134f5d1ec. Base d8526ef, head 6d3f219, on a detached snapshot (unchanged afterwards). The reviewer got the literal diff, the measured uv facts, the conventions, and the testing guide. It got no PR text, commit messages, or prior findings.

Covered: job expansion, CLI forwarding, queue publication, and legacy jobs without environment. Worker startup, interpreter selection, attempt failures, checkpoints, resume, status, reports, and adapters. Dependency groups and lock markers, local/CI commands, EC2 bootstrap/systemd, packaging, ignore rules, documentation, and the changed tests' distinct contracts.

Result: FINDINGS (1).

• P2 — Python 3.15 Arrow installs without dev lose compatible wheels (uv.lock:1364). A root uv sync --locked --no-dev --extra arrow --python 3.15 now selects PyArrow 25.0.1, because extra != 'group-8-pyathena-dev' is true; the base lock selected 26.0.0. The locked 25.0.1 artifacts have no CPython 3.15 wheel, so this installation now needs a native source build and fails when PyArrow source builds are disabled. The supplied root-sync checks kept dev and therefore did not cover this regression.

[laughingman7743]

Repaired in 33bf9e8. The Python 3.15 pin pyarrow>=26.0.0; python_version >= '3.15' moves from the root dev group to the root [tool.uv] constraint-dependencies, which apply to every resolution of the lock, and uv.lock is regenerated.

Verified locally:

• uv export --locked --no-dev --extra arrow gives PyArrow 25.0.1 below Python 3.15 and 26.0.0 on 3.15, as on master.
• Root uv sync --locked: Python 3.15 → polars 2.0.0 / pyarrow 26.0.0; Python 3.12 → 1.44.2 / 25.0.1.
• The EC2 syncs with uv 0.12.5 still give Polars 1.44.2 / 2.0.0.
• Comparison with master: uv export --all-extras, with and without --no-dev, on Python 3.11-3.15, plus the benchmark's default groups. Only Polars entries differ.
  • The root resolution without dev now has Polars 2.0.0 only, where master also listed 1.44.2 for Python < 3.14. This is intended, because the polars<2 pin is in the dev group. No workflow installs the polars extra without dev; the database sweep's --no-dev installs no extras.
• just lint: passed.
• just benchmark test: 256 passed, 1 skipped in each environment.

Self-review of the repair:

• Round one, behavior: the constraint changes only the PyArrow selection on Python 3.15 outside dev. The benchmark's awswrangler marker (python_version < '3.15') keeps its cap from conflicting.
• Round two, claims: the moved comment now says "every resolution". The PR description records the lock change.

Independent follow-up: Codex CLI 0.163.0, gpt-6-astra, effort max, read-only, session 01a14f74-91ed-7172-bb4b-16eb95d5f079, on the detached snapshot of 33bf9e8 and the diff 6d3f219..33bf9e8. Result: CLEAN. "The repair resolves the earlier P2 finding ... No unintended resolution effects found. Python 3.11–3.14 remains unaffected; benchmark groups retain their Polars selections." Static inspection only.