Check that dependencies come from PyPI on pull requests to main

.github/workflows/check_pypi_dependencies.yml

@@ -0,0 +1,24 @@
+name: PyPI dependencies
+
+# PyPI rejects an upload whose metadata has a direct reference ("name @ git+https://..."), and the
+# Release workflow only finds out after the merge to main. Check the dependencies on the pull
+# request instead.
+
+on:
+  pull_request:
+    branches:
+      - main
+  workflow_dispatch:
+
+jobs:
+  pypi-dependencies:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+
+      - uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.12"
+
+      - name: Check that every dependency can be installed from PyPI
+        run: uv run --no-project --with packaging python scripts/check_pypi_dependencies.py

CHANGELOG.md

@@ -5,6 +5,12 @@ format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Added
+
+- `PyPI dependencies` workflow and `scripts/check_pypi_dependencies.py`: on every pull request to
+  `main`, fail if a dependency is a direct reference (`name @ git+https://...`), is not on PyPI, or
+  has a version range that no release satisfies, instead of failing the release after the merge.
+
 ### Changed
 
 - `__version__` is read from the installed package metadata (`importlib.metadata`), not hardcoded,

docs/src/content/docs/development/publishing.md

@@ -33,6 +33,24 @@ Before merging a release to `main`, update the version in `pyproject.toml` and `
 3. Optionally, in the GitHub repository settings create an environment named `pypi` whose
    deployment branches are limited to `main`.
 
+### Dependencies must come from PyPI
+
+PyPI rejects an upload whose metadata has a direct reference, such as
+`cunumpy @ git+https://github.com/org/cunumpy.git@devel` ("400 Can't have direct dependency"),
+and the `Release` workflow would only fail after the merge to `main`. The `PyPI dependencies`
+workflow checks this on every pull request to `main`, with `scripts/check_pypi_dependencies.py`.
+It fails when a dependency, or an optional dependency, is
+
+- a direct reference (`name @ git+https://...` or `name @ file:///...`),
+- a project that PyPI does not know, or
+- a version range that no release on PyPI satisfies (for example `cunumpy>=0.7` before 0.7.0 is
+  published), so a dependency has to be released before the package that needs it.
+
+During development a dependency can come from git without touching the published metadata, with
+`[tool.uv.sources]` (`cunumpy = { git = "https://github.com/org/cunumpy", branch = "devel" }`)
+next to a plain `"cunumpy>=0.6.3"` requirement. Run the check locally with
+`uv run --no-project --with packaging python scripts/check_pypi_dependencies.py`.
+
 ### Coverage
 
 The tests upload coverage to [Codecov](https://codecov.io/). Public repositories need no

scripts/check_pypi_dependencies.py

@@ -0,0 +1,176 @@
+"""Check that every dependency in pyproject.toml can be installed from PyPI.
+
+PyPI rejects an upload whose metadata has a direct reference such as
+``cunumpy @ git+https://github.com/org/cunumpy.git@devel`` ("400 Can't have direct
+dependency"), and the release fails after the version is merged. This check catches that,
+and its sibling, in a pull request instead:
+
+* a requirement with a URL (``name @ git+https://...``, ``name @ file:///...``);
+* a project that PyPI does not know;
+* a version range that no release on PyPI satisfies, e.g. ``cunumpy>=0.7`` before 0.7.0 is
+  published, or only yanked releases do.
+
+The project's own name is skipped, so the ``name[docs,test]`` entry of a ``dev`` extra is
+fine. Requirement markers (``; python_version < "3.11"``) are not evaluated: a dependency is
+checked whether or not the marker applies. ``[dependency-groups]`` are not part of the
+published metadata and are not checked.
+
+Usage::
+
+    python scripts/check_pypi_dependencies.py [pyproject.toml]
+
+Needs ``packaging`` (``uv run --no-project --with packaging python scripts/...``). Exits with
+status 1 and prints GitHub Actions annotations when something fails.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+import time
+import urllib.error
+import urllib.request
+from collections.abc import Callable
+from pathlib import Path
+
+import tomllib
+from packaging.requirements import InvalidRequirement, Requirement
+from packaging.utils import canonicalize_name
+from packaging.version import InvalidVersion, Version
+
+PYPI_URL = "https://pypi.org/pypi/{name}/json"
+
+#: ``{version: [file entries]}`` of a project on PyPI, or ``None`` if PyPI does not know it.
+Releases = dict[str, list[dict]]
+Fetch = Callable[[str], Releases | None]
+
+
+class PyPIUnreachableError(RuntimeError):
+    """PyPI could not be queried, so the check can say nothing."""
+
+
+def fetch_releases(name: str, attempts: int = 3) -> Releases | None:
+    """Return the releases of ``name`` on PyPI, or ``None`` if there is no such project."""
+    url = PYPI_URL.format(name=name)
+    for attempt in range(1, attempts + 1):
+        try:
+            with urllib.request.urlopen(url, timeout=20) as response:  # noqa: S310
+                return json.load(response)["releases"]
+        except urllib.error.HTTPError as error:
+            if error.code == 404:
+                return None
+            failure = f"HTTP {error.code}"
+        except (urllib.error.URLError, TimeoutError, json.JSONDecodeError) as error:
+            failure = str(error)
+        if attempt < attempts:
+            time.sleep(2**attempt)
+    raise PyPIUnreachableError(f"could not query {url}: {failure}")
+
+
+def requirement_strings(pyproject: dict) -> list[tuple[str, str]]:
+    """Return ``(where, requirement)`` for the dependencies and every optional dependency."""
+    project = pyproject.get("project", {})
+    found = [("dependencies", text) for text in project.get("dependencies", [])]
+    for extra, texts in project.get("optional-dependencies", {}).items():
+        found += [(f"optional-dependencies.{extra}", text) for text in texts]
+    return found
+
+
+def is_installable(releases: Releases, requirement: Requirement) -> bool:
+    """Whether some release on PyPI that is not yanked satisfies the requirement."""
+    for text, files in releases.items():
+        if not files or all(entry.get("yanked", False) for entry in files):
+            continue
+        try:
+            version = Version(text)
+        except InvalidVersion:
+            continue
+        if requirement.specifier.contains(version, prereleases=None):
+            return True
+    return False
+
+
+def check(pyproject: dict, fetch: Fetch = fetch_releases) -> list[str]:
+    """Return one message per dependency that PyPI would reject or that cannot be installed."""
+    project = pyproject.get("project", {})
+    own_name = canonicalize_name(project.get("name", ""))
+    problems = []
+    if "dependencies" in project.get("dynamic", []):
+        problems.append(
+            "dependencies are dynamic, so they cannot be read from pyproject.toml: "
+            "check the built metadata (Requires-Dist) instead"
+        )
+    cache: dict[str, Releases | None] = {}
+    for where, text in requirement_strings(pyproject):
+        try:
+            requirement = Requirement(text)
+        except InvalidRequirement as error:
+            problems.append(f"[{where}] {text!r} is not a valid requirement: {error}")
+            continue
+        if requirement.url:
+            problems.append(
+                f"[{where}] {text!r} is a direct reference ({requirement.url}); PyPI rejects "
+                f"uploads with one. Use a version range of a release on PyPI, e.g. "
+                f'"{requirement.name}>=<version>", and keep the git URL for local development in '
+                f"[tool.uv.sources] or a dependency group."
+            )
+            continue
+        name = canonicalize_name(requirement.name)
+        if name == own_name:
+            continue
+        if name not in cache:
+            cache[name] = fetch(name)
+        releases = cache[name]
+        if releases is None:
+            problems.append(
+                f"[{where}] {text!r}: no project {requirement.name!r} on PyPI"
+            )
+        elif not is_installable(releases, requirement):
+            newest = max(
+                (v for v in releases if releases[v]),
+                key=_version_key,
+                default="none",
+            )
+            problems.append(
+                f"[{where}] {text!r}: no release on PyPI satisfies it "
+                f"(newest: {newest}); publish the dependency first, or loosen the range"
+            )
+    return problems
+
+
+def _version_key(text: str) -> Version:
+    try:
+        return Version(text)
+    except InvalidVersion:
+        return Version("0")
+
+
+def main(argv: list[str] | None = None) -> int:
+    """Run the check on a pyproject.toml; return the exit status."""
+    parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])
+    parser.add_argument("pyproject", nargs="?", default="pyproject.toml", type=Path)
+    args = parser.parse_args(argv)
+
+    with args.pyproject.open("rb") as handle:
+        pyproject = tomllib.load(handle)
+    try:
+        problems = check(pyproject)
+    except PyPIUnreachableError as error:
+        print(f"::error file={args.pyproject}::{error}")
+        return 1
+
+    total = len(requirement_strings(pyproject))
+    if problems:
+        for problem in problems:
+            print(f"::error file={args.pyproject}::{problem}")
+        print(
+            f"\n{len(problems)} of {total} dependencies cannot be installed from PyPI."
+        )
+        return 1
+    print(f"All {total} dependencies in {args.pyproject} can be installed from PyPI.")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

src/app/tests/unit/test_check_pypi_dependencies.py

@@ -0,0 +1,158 @@
+"""The PyPI dependency check (scripts/check_pypi_dependencies.py), against a fake PyPI."""
+
+import importlib.util
+import sys
+from pathlib import Path
+
+import pytest
+
+# the script needs packaging, and tomllib (Python 3.11+); the workflow runs it on 3.12
+pytest.importorskip("packaging")
+tomllib = pytest.importorskip("tomllib")
+
+ROOT = Path(__file__).resolve().parents[4]
+SCRIPT = ROOT / "scripts" / "check_pypi_dependencies.py"
+
+spec = importlib.util.spec_from_file_location("check_pypi_dependencies", SCRIPT)
+checker = importlib.util.module_from_spec(spec)
+sys.modules[spec.name] = checker
+spec.loader.exec_module(checker)
+
+#: what the fake PyPI knows: project -> {version: files}
+RELEASES = {
+    "numpy": {"1.26.0": [{}], "2.0.0": [{}], "2.1.0rc1": [{}]},
+    "cunumpy": {"0.6.2": [{}], "0.6.3": [{}]},
+    "yankedonly": {"1.0": [{"yanked": True}]},
+    "empty": {"1.0": []},
+}
+
+
+def fake_fetch(name):
+    return RELEASES.get(name)
+
+
+def run(project, **tables):
+    return checker.check({"project": {"name": "demo", **project}, **tables}, fake_fetch)
+
+
+def test_plain_dependencies_pass():
+    assert run({"dependencies": ["numpy", "cunumpy>=0.6.3"]}) == []
+
+
+@pytest.mark.parametrize(
+    "requirement",
+    [
+        "cunumpy @ git+https://github.com/max-models/cunumpy.git@devel",
+        "cunumpy @ https://example.org/cunumpy-0.6.3.tar.gz",
+        "cunumpy @ file:///tmp/cunumpy",
+    ],
+)
+def test_direct_references_are_rejected(requirement):
+    problems = run({"dependencies": [requirement]})
+    assert len(problems) == 1
+    assert "direct reference" in problems[0]
+    assert "cunumpy" in problems[0]
+
+
+def test_direct_references_in_extras_are_rejected_too():
+    problems = run(
+        {"optional-dependencies": {"dev": ["cunumpy @ git+https://x/y.git"]}}
+    )
+    assert len(problems) == 1
+    assert problems[0].startswith("[optional-dependencies.dev]")
+
+
+def test_unknown_project():
+    problems = run({"dependencies": ["not-on-pypi"]})
+    assert problems == [
+        "[dependencies] 'not-on-pypi': no project 'not-on-pypi' on PyPI"
+    ]
+
+
+def test_range_no_release_satisfies():
+    problems = run({"dependencies": ["cunumpy>=0.7"]})
+    assert len(problems) == 1
+    assert "no release on PyPI satisfies" in problems[0]
+    assert "newest: 0.6.3" in problems[0]
+
+
+def test_yanked_and_file_less_releases_do_not_count():
+    assert len(run({"dependencies": ["yankedonly"]})) == 1
+    assert len(run({"dependencies": ["empty"]})) == 1
+
+
+def test_prereleases_count_only_when_asked_for():
+    assert run({"dependencies": ["numpy>=2.1"]}) != []
+    assert run({"dependencies": ["numpy>=2.1.0rc1"]}) == []
+
+
+def test_the_projects_own_name_is_skipped_with_extras():
+    assert (
+        run({"optional-dependencies": {"dev": ["demo[docs,test]"], "docs": ["numpy"]}})
+        == []
+    )
+    assert run({"optional-dependencies": {"dev": ["Demo[docs]"]}}) == []
+
+
+def test_own_name_with_a_direct_reference_is_still_rejected():
+    assert len(run({"dependencies": ["demo @ git+https://x/y.git"]})) == 1
+
+
+def test_markers_are_checked_regardless_of_the_environment():
+    assert run({"dependencies": ['numpy; python_version < "3.9"']}) == []
+    assert len(run({"dependencies": ['not-on-pypi; python_version < "3.9"']})) == 1
+
+
+def test_invalid_requirement():
+    problems = run({"dependencies": ["numpy>>=1"]})
+    assert len(problems) == 1
+    assert "not a valid requirement" in problems[0]
+
+
+def test_dynamic_dependencies_are_reported():
+    problems = run({"dynamic": ["dependencies"]})
+    assert len(problems) == 1
+    assert "dynamic" in problems[0]
+
+
+def test_each_project_is_looked_up_once():
+    calls = []
+
+    def counting_fetch(name):
+        calls.append(name)
+        return RELEASES.get(name)
+
+    pyproject = {
+        "project": {
+            "name": "demo",
+            "dependencies": ["numpy"],
+            "optional-dependencies": {"a": ["numpy>=1"], "b": ["NumPy"]},
+        }
+    }
+    assert checker.check(pyproject, counting_fetch) == []
+    assert calls == ["numpy"]
+
+
+def test_main_exit_status(tmp_path, capsys):
+    good = tmp_path / "good.toml"
+    good.write_text('[project]\nname = "demo"\ndependencies = []\n')
+    assert checker.main([str(good)]) == 0
+    assert "All 0 dependencies" in capsys.readouterr().out
+
+    bad = tmp_path / "bad.toml"
+    bad.write_text(
+        '[project]\nname = "demo"\ndependencies = ["x @ git+https://y/z.git"]\n'
+    )
+    assert checker.main([str(bad)]) == 1
+    out = capsys.readouterr().out
+    assert out.startswith("::error file=")
+    assert "1 of 1 dependencies" in out
+
+
+def test_this_projects_pyproject_has_only_pypi_dependencies():
+    """No network: the requirements are valid and none is a direct reference."""
+    with (ROOT / "pyproject.toml").open("rb") as handle:
+        pyproject = tomllib.load(handle)
+    for where, text in checker.requirement_strings(pyproject):
+        requirement = checker.Requirement(text)
+        assert requirement.url is None, f"[{where}] {text} is a direct reference"