Skip to content

fix(lookup): date every proposable release and admit prereleases only on purpose - #342

Open
sweepies wants to merge 2 commits into
mainfrom
fix/lookup-release-age
Open

sweepies wants to merge 2 commits into
mainfrom
fix/lookup-release-age

Conversation

@sweepies

@sweepies sweepies commented Nov 15, 2026 •

Copy link
Copy Markdown
Owner

Fixes from the lookup and datasource scan. Each one was letting through a release that policy should have held back.

Invariant

Every version the lookup could propose is either dated and flagged by its source, or marked unchecked. A positive waitHours holds an unchecked release instead of treating it as undated. It shows up in the report as a notice, and a later lookup asks about it again. A missing answer can no longer skip the release-age wait.

Fixes

  1. Release dates cover every version, not the newest N (Go, Maven, NuGet).
    • New Version::unchecked (catalog.rs) and a release-dates state (dates.rs). The state keeps each registry answer per version: a date, or "cannot be served". It is keyed by package and, for Maven, by the operator registry entry's scope.
    • Go .info and Maven POM HEAD requests go only to versions with no stored answer, wherever they sort, with a bounded buffer_unordered fan-out of 8.
    • A cold lookup dates every version once. After that, a lookup asks only about versions it hasn't seen. This makes guide/sources.md's ".info for new versions only" true.
    • A failed request stores nothing and leaves the version unchecked. This replaces Maven's fail-open .ok()?.
    • A 404 from the Go proxy or a missing Maven POM drops the version.
    • The old top-N cut (GO_DATED_RELEASES, Maven's DATED) is gone, and so are reached_known and the head-stopping merge().
    • NuGet reads every registration page (nuget-registration v2 lists them all), up to 4 at once.
      • The flat-container listing and the registration entries are matched without build metadata.
      • A version the registration doesn't describe yet is unchecked.
      • A page link off nuget.org fails the lookup instead of being ignored.
    • I didn't plumb each repository's baseline into the shared catalog fetch. Catalogs are shared across repositories and reused for up to 23 h, so a floor recorded at fetch time could be stale for another repository's lower baseline, which fails open. Dating everything once and storing the answers is simpler, and it fails closed by construction.
  2. A permitted filter admits prereleases only when it asks for them (lookup/src/lib.rs, config/src/matching.rs).
    • Filter::asks_for_prerelease implements three rules:
      • A non-negated regex: asks when it matches the prerelease but not the same version without its prerelease part (regex:-rc\. and regex:nightly ask; regex:^2\. does not).
      • A semver: range asks when it names a prerelease version itself and matches (semver:>=3.0.0-rc.1 and semver:[1.0-beta1,2) ask; semver:[1,2) and semver:<3 do not).
      • A negated filter never asks.
    • Documented in guide/lookup-policy.md and configuration.md.
    • The existing nightly-pins example (regex:-nightly\.) keeps working.
  3. mise prerelease flag. mise() now carries prerelease: true into Version::prerelease, and the lookup treats it like a prerelease version, including for the current version, latest, and dist-tag targets. This matches github-releases.
  4. mise-versions tag dates. When the mirror's tags.json has a tag that releases.json doesn't date, the github-tags lookup falls back to GitHub's own tag catalog. That path dates annotated tags by their tagger and is cached by TagCatalog/TagObject. guide/lookup-policy.md already promised this; guide/sources.md now says how.
  5. Go .info fetches run in parallel with a bounded fan-out, as part of fix 1.

Cache and version bookkeeping:

  • catalog.rs and config/src are hashed sources, so every derived representation and state that lists them got a version bump, and its row in tests/derived-versions.tsv / tests/state-versions.tsv was updated.
  • release-dates is in tests/state-exempt.tsv, since it holds facts about immutable releases.
  • internals/cache.md describes the new state.

Tests

  • datasource/tests/go_proxy.rs:
    • every version is dated once, and a warm lookup makes no .info requests;
    • a new v0.29.4 below a known v0.30.0 is dated;
    • a failed .info leaves the version unchecked and is retried.
  • datasource/tests/sources.rs:
    • NuGet reads an older linked page holding an unlisted 6.0.5+build (yanked, dated);
    • an undescribed version is unchecked;
    • an off-site page fails the lookup;
    • Maven dates all 13 versions, retries a 503, dates new older-line releases on the next lookup and drops a 404 POM.
  • datasource/tests/versions_host.rs: an annotated tag the mirror leaves undated gets its tagger date through GitHub.
  • datasource/tests/mise_lookup.rs: prereleases_stay_in_the_catalog_for_the_policy_to_decide now asserts that the flag is carried (jq 1.7rc2, prettier alphas), and that it is absent on stable releases.
  • lookup/tests/policy.rs:
    • npm !regex:^3\., regex:^2\. and !semver:>=3 admit no rc; regex:-rc\. and semver:>=2.2.0-rc.1 do;
    • Maven semver:[1,2) admits no beta; semver:[1.5-beta1,2) and regex:-beta do;
    • PyPI semver:<3 admits no rc; semver:>=2.2.0rc1,<3 does;
    • a flagged stable-looking prerelease is held, unless the current version is flagged;
    • an unchecked release waits and is reported, and is proposed with waitHours = 0.
  • Ran on the VM: cargo test -p patchula-config -p patchula-datasource -p patchula-lookup -p patchula-engine, and clippy -D warnings on those crates plus patchula-tools.
  • The corpus snapshots are unchanged: the corpus test passes without regenerating them.

Dropped

Nothing.

Follow-ups

  • A cold lookup of a Go module or Maven artifact with hundreds of versions now makes one date request per version, once. Watch cold-run timings on large monorepos.
  • Rows the old Go code wrote to the releases table (source = 'go') are no longer read. A later cache migration can drop them. I didn't add one here, to avoid colliding with other open cache migrations.
  • Repositories in mise's registry whose tags are mostly lightweight with no releases now always take the GitHub API path for github-tags.

View guided diff Turn on auto-fix

… on purpose

- Go, Maven and NuGet date and flag every listed version, not the newest
  few: Go `.info` and Maven POM dates are asked once per version (bounded
  fan-out) and kept as `release-dates` state, so a warm lookup asks only
  about versions it has not seen, wherever they sort; NuGet reads every
  registration page. A failed date request, or a NuGet version the
  registration does not describe yet, marks the version unchecked, and
  policy holds an unchecked release under a positive wait instead of
  reading it as undated.
- A `permitted` filter admits prereleases only when it asks for them: a
  regex that matches the prerelease part, or a range that names a
  prerelease. Negated and merely narrowing filters no longer do, which
  stops Maven `[1,2)` or PyPI `<3` letting betas through.
- mise's `prerelease: true` is carried into the catalog and treated like
  a prerelease version, matching `github-releases`.
- A `github-tags` lookup through mise-versions falls back to GitHub's own
  tag catalog when the mirror leaves a tag undated, so annotated tags are
  dated by their tagger as the docs promise.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
count-patchula df0ca6d Commit Preview URL

Branch Preview URL
Nov 15 2026, 11:02 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant