Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions .github/workflows/test-select-runners.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Test runner selection

on:
pull_request:
paths:
- select-runners/**
- plan-release-signing/**
- .github/workflows/test-select-runners.yml
workflow_dispatch:

permissions:
contents: read

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
test:
runs-on: ubuntu-24.04
timeout-minutes: 5
outputs:
runners: ${{ steps.runners.outputs.runners }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/setup-node@249970729cb0ef3589644e2896645e5dc5ba9c38 # v6.5.0
with:
node-version: "24"
package-manager-cache: false
- run: node --test select-runners/main.test.cjs
- run: python3 -m unittest discover -s plan-release-signing
- uses: $/select-runners
id: runners
with:
free-only: "true"
roles: |
{"linux-smoke": ["namespace-profile-ubuntu-24-04-8", "ubuntu-24.04"]}
required: linux-smoke ubuntu-x86_64-16 ubuntu-aarch64-4 windows-x86_64-16

selected-runner:
needs: test
runs-on: ${{ fromJSON(needs.test.outputs.runners).linux-smoke }}
timeout-minutes: 5
steps:
- run: echo "Scheduled on the selected free runner"

unavailable-runner:
needs: test
if: ${{ fromJSON(needs.test.outputs.runners).ubuntu-x86_64-codspeed != '' }}
runs-on: ${{ fromJSON(needs.test.outputs.runners).ubuntu-x86_64-codspeed }}
steps:
- run: exit 1
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

Shared GitHub Actions for Astral projects.

- [Select runners](select-runners/) chooses GitHub, Depot, and Namespace runners
for named roles and selects GitHub-hosted alternatives for forks.
- [Simple token service](simple-token-service/): exchange GitHub Actions OIDC tokens
for scoped GitHub App installation tokens.
- [Key-value store](key-value-store/): share workflow state within a GitHub Actions
Expand Down
9 changes: 6 additions & 3 deletions plan-release-signing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ The action provides four JSON outputs:
- `macos`: macOS artifact declarations with native runner and Python architecture.
- `windows`: Windows artifact declarations with native runner and Python architecture.

The native matrices use Astral's macOS runners, Astral's x86 Windows runner,
and GitHub's ARM64 Windows runner. An unknown native target fails planning so a
new release target cannot bypass signing or verification.
Pass the `runners` JSON output from [Select runners](../select-runners/) to use
the same selection as the rest of the workflow. The native matrices require
`macos-aarch64-8`, `windows-x86_64-16`, and `windows-aarch64-8` for the corresponding
targets. If `runners` is omitted, the action uses the preferred candidates from
the shared catalog. Missing required runners and unknown native targets fail
planning so a new release target cannot bypass signing or verification.
4 changes: 4 additions & 0 deletions plan-release-signing/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,9 @@ inputs:
description: "Directory containing cargo-dist GitHub release archives."
required: false
default: "github-archives"
runners:
description: "Resolved JSON runner map from select-runners. Defaults to the catalog's preferred candidates."
required: false

outputs:
targets:
Expand Down Expand Up @@ -54,4 +57,5 @@ runs:
RELEASE_WORKSPACE: ${{ inputs.workspace }}
RELEASE_WHEELS_DIRECTORY: ${{ inputs.wheels-directory }}
RELEASE_GITHUB_ARCHIVES_DIRECTORY: ${{ inputs.github-archives-directory }}
RELEASE_RUNNERS: ${{ inputs.runners }}
run: uv run --locked "$ACTION_PATH/plan.py" >> "$GITHUB_OUTPUT"
44 changes: 38 additions & 6 deletions plan-release-signing/plan.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,20 @@

PLATFORMS = {
"macos": {
"aarch64-apple-darwin": ("namespace-profile-macos-15", "arm64"),
"x86_64-apple-darwin": ("namespace-profile-macos-15", "x64"),
"aarch64-apple-darwin": ("macos-aarch64-8", "arm64"),
"x86_64-apple-darwin": ("macos-aarch64-8", "x64"),
},
"windows": {
"aarch64-pc-windows-msvc": (
"github-windows-11-aarch64-8",
"windows-aarch64-8",
"arm64",
),
"i686-pc-windows-msvc": (
"namespace-profile-windows-2022-x86-64-16x32",
"windows-x86_64-16",
"x86",
),
"x86_64-pc-windows-msvc": (
"namespace-profile-windows-2022-x86-64-16x32",
"windows-x86_64-16",
"x64",
),
},
Expand All @@ -37,6 +37,26 @@
SIGNED_TARGET_SUFFIXES = ("-apple-darwin", "-pc-windows-msvc")


def runner_labels(value: str = "") -> dict[str, str]:
"""Read resolved labels, or the preferred catalog candidates for callers without a map."""
if value:
runners = json.loads(value)
else:
roles_path = Path(__file__).parents[1] / "select-runners" / "roles.json"
roles = json.loads(roles_path.read_text(encoding="utf-8"))
runners = {role: candidates[0] for role, candidates in roles.items()}
if not isinstance(runners, dict) or not all(
isinstance(role, str)
and isinstance(label, str)
and label
and label.strip() == label
and not any(ord(char) < 32 or ord(char) == 127 for char in label)
for role, label in runners.items()
):
raise ValueError("runners must map roles to non-empty runner labels")
return runners


def release_targets(workspace: Path) -> list[str]:
"""Read and validate the target inventory used by cargo-dist."""
targets = tomllib.loads(workspace.read_text(encoding="utf-8"))["dist"]["targets"]
Expand Down Expand Up @@ -122,12 +142,23 @@ def signing_plan(
archive_package: str,
wheels_directory: PurePosixPath,
github_archives_directory: PurePosixPath,
runners: dict[str, str] | None = None,
) -> dict[str, list[dict]]:
"""Require every macOS and Windows target to have a native verifier."""
expected = {target for target in targets if target.endswith(SIGNED_TARGET_SUFFIXES)}
configured = {target for platforms in PLATFORMS.values() for target in platforms}
if unsupported := expected - configured:
raise ValueError(f"Unsupported release signing targets: {sorted(unsupported)}")
if runners is None:
runners = runner_labels()
required_roles = {
role
for platforms in PLATFORMS.values()
for target, (role, _) in platforms.items()
if target in expected
}
if missing := required_roles - runners.keys():
raise ValueError(f"Missing release signing runners: {sorted(missing)}")

plan = {}
for system, platforms in PLATFORMS.items():
Expand All @@ -137,7 +168,7 @@ def signing_plan(
plan[system] = [
{
"target": target,
"runner": platforms[target][0],
"runner": runners[platforms[target][0]],
"python-architecture": platforms[target][1],
**artifact_inputs(
system,
Expand All @@ -163,6 +194,7 @@ def main() -> None:
os.environ["RELEASE_ARCHIVE_PACKAGE"],
PurePosixPath(os.environ["RELEASE_WHEELS_DIRECTORY"]),
PurePosixPath(os.environ["RELEASE_GITHUB_ARCHIVES_DIRECTORY"]),
runner_labels(os.environ.get("RELEASE_RUNNERS", "")),
)

print(f"targets={json.dumps(targets, separators=(',', ':'))}")
Expand Down
44 changes: 43 additions & 1 deletion plan-release-signing/test_plan.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

sys.path.insert(0, str(Path(__file__).parent))

from plan import package_inventories, signing_plan
from plan import package_inventories, runner_labels, signing_plan

TARGETS = [
"aarch64-apple-darwin",
Expand Down Expand Up @@ -112,6 +112,48 @@ def test_rejects_duplicate_executables(self) -> None:
with self.assertRaisesRegex(ValueError, "executables must be distinct"):
package_inventories(json.dumps(packages))

def test_uses_resolved_free_runners(self) -> None:
runners = {
"macos-aarch64-8": "macos-15",
"windows-x86_64-16": "windows-2025",
"windows-aarch64-8": "windows-11-arm",
}
plan = signing_plan(
TARGETS,
package_inventories(json.dumps(UV_PACKAGES)),
"uv",
PurePosixPath("wheels"),
PurePosixPath("github-archives"),
runner_labels(json.dumps(runners)),
)
self.assertEqual(
[target["runner"] for target in plan["macos"]], ["macos-15"] * 2
)
self.assertEqual(
[target["runner"] for target in plan["windows"]],
["windows-11-arm", "windows-2025", "windows-2025"],
)

def test_rejects_missing_runner_in_supplied_map(self) -> None:
with self.assertRaisesRegex(ValueError, "Missing release signing runners"):
signing_plan(
TARGETS,
package_inventories(json.dumps(UV_PACKAGES)),
"uv",
PurePosixPath("wheels"),
PurePosixPath("github-archives"),
{},
)

def test_preferred_catalog_candidates(self) -> None:
plan = self.plan(UV_PACKAGES, "uv")
self.assertEqual(plan["macos"][0]["runner"], "namespace-profile-macos-15")
self.assertEqual(plan["windows"][0]["runner"], "github-windows-11-aarch64-8")
self.assertEqual(
plan["windows"][1]["runner"],
"namespace-profile-windows-2022-x86-64-16x32",
)


if __name__ == "__main__":
unittest.main()
92 changes: 92 additions & 0 deletions select-runners/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Select runners

Choose runners for CI jobs from the [runner inventory](runners.json), which
lists exact labels for Namespace, Depot, GitHub, and CodSpeed. The
[default roles](roles.json) map job requirements to ordered lists of those
labels. Both files ship with the action.

For each role, the action selects the first candidate. With `free-only`
enabled, it selects the first candidate in the `github-standard` group.
By default, workflows outside the `astral-sh` organization use only free
candidates. A pull request targeting an Astral repository runs on Astral's
runners. A workflow running in the contributor's fork uses GitHub-hosted
alternatives. Astral repositories must have
access to the configured runner pools.

Pin the action to a full commit SHA so the resolver, inventory, and roles
change together. Selection uses the configured labels and takes place before
GitHub schedules the dependent jobs. A queued job stays on its selected label.

## Usage

Run selection in an existing setup or planning job, then pass the `runners` JSON
output to dependent jobs or reusable workflows. The initial job needs a fixed
runner because its steps execute after GitHub schedules it.

```yaml
jobs:
plan:
runs-on: ubuntu-24.04
outputs:
runners: ${{ steps.runners.outputs.runners }}
free-only: ${{ steps.runners.outputs.free-only }}
steps:
- uses: astral-sh/github-actions/select-runners@<commit-sha>
id: runners
with:
required: ubuntu-x86_64-16 windows-x86_64-16

test:
needs: plan
runs-on: ${{ fromJSON(needs.plan.outputs.runners).ubuntu-x86_64-16 }}
steps:
- run: echo "Run tests"
```

Reusable workflows can accept a single required `runners` string input and use
`${{ fromJSON(inputs.runners).ubuntu-x86_64-16 }}`. The caller remains responsible
for job permissions, conditions, timeouts, and workload settings.

To select other machines from the inventory, use `roles` to add a role or
replace the candidates for a default role. This example uses a 32-core
Namespace runner for Windows builds and a 32-core Depot runner for Linux
builds. Each role names a standard GitHub runner for forks.

```yaml
with:
roles: |
{
"windows-build": ["namespace-profile-windows-2022-x86-64-32x64", "windows-2025"],
"linux-build": ["depot-ubuntu-24.04-32", "ubuntu-24.04"]
}
required: windows-build linux-build
```

## Inputs and outputs

| Input | Meaning |
| ----------- | ------------------------------------------------------------------------------------------ |
| `free-only` | Optional `true` or `false` override. Defaults to `true` outside `astral-sh`. |
| `roles` | Optional JSON object mapping role names to ordered lists of inventory labels. |
| `required` | Whitespace-separated roles that must resolve. Unknown or unavailable roles fail selection. |

| Output | Meaning |
| ----------- | ------------------------------------------------------------------ |
| `runners` | Compact JSON object mapping available roles to runner labels. |
| `free-only` | `true` when only free candidates were eligible; otherwise `false`. |

Sizes in ordinary role names describe the preferred runner. Free fallbacks may
have fewer CPUs and less memory, so callers may need longer timeouts or less
parallelism. GitHub provides the `github-standard` runners free of charge for
public repositories. Private repositories follow their GitHub billing plan.

Specialized roles are omitted when no eligible candidate exists:

- `ubuntu-aarch64-pgo` requires the native 64 GB ARM runner used for release PGO.
A caller may instead use `ubuntu-aarch64-4` with PGO disabled.
- `ubuntu-x86_64-codspeed` requires CodSpeed's walltime benchmark runner. Guard
the benchmark job with an `if` checking that this role is present.

The `-github` variants use the GitHub runner image. The corresponding Linux
build roles use Depot's runner and cache.
The `-slim` role permits the constrained `ubuntu-slim` image as its free fallback.
23 changes: 23 additions & 0 deletions select-runners/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: "Select runners"
description: "Resolve shared runner roles, with GitHub-hosted fallbacks outside Astral."

inputs:
free-only:
description: "Select only free candidates. Defaults to true outside the astral-sh organization."
required: false
roles:
description: "JSON object adding or replacing roles with ordered lists of catalog labels."
required: false
required:
description: "Runner roles that must be available, separated by whitespace."
required: false

outputs:
runners:
description: "JSON object mapping available runner roles to labels."
free-only:
description: "Whether selection was restricted to free candidates."

runs:
using: node24
main: main.cjs
Loading