Skip to content

mirror: stop re-downloading packages indexed by SHA512 only - #1665

Open
somaz94 wants to merge 1 commit into
aptly-dev:masterfrom
somaz94:fix/sha512-only-mirror-redownload
Open

somaz94 wants to merge 1 commit into
aptly-dev:masterfrom
somaz94:fix/sha512-only-mirror-redownload

Conversation

@somaz94

@somaz94 somaz94 commented Nov 11, 2026

Copy link
Copy Markdown

Fixes #1664

Requirements

All new code should be covered with tests, documentation should be updated. CI should pass.

Also, to speed up things, if you could kindly "Allow edits and access to secrets by maintainers" in the
PR settings, as this allows us to rebase the PR on master, fix conflicts, run coverage and help with
implementing code and tests.

Description of the Change

Ubuntu 26.10 indexes list packages with SHA512 only. The package pool derives its paths from SHA256 (or MD5 for legacy paths), so Verify never finds a file that was already downloaded, and every aptly mirror update fetches the whole mirror again. -skip-existing-packages does not help either: the key built from the index hashes the MD5, SHA1 and SHA256 sums, which are all empty here, so it never matches the stored key, which was computed after the download filled in the full checksums.

BuildDownloadQueue now looks up the package with the same architecture, name and version from the previous update. When a file listed by SHA512 alone has a match there with the same name, size and SHA512, its pool path and full checksums are reused and the files hash is recomputed. Verify then finds the file in the pool the same way it does for SHA256 indexes, and the key matches again for -skip-existing-packages. A file whose SHA512 changed is still downloaded.

The new TestDownloadSHA512OnlyFlat fails on master (the second update queues the package again) and passes with this change. It covers the default path, -skip-existing-packages, and an index whose SHA512 differs. go test ./deb/ ./files/ ./http/ and golangci-lint run pass locally. I did not add a system test since the change is in the queue building logic, but I can add one under t04_mirror if you prefer.

I used Claude Code to help trace the code path and write the test, and reviewed and ran everything myself.

Checklist

  • allow Maintainers to edit PR (rebase, run coverage, help with tests, ...)
  • unit-test added (if change is algorithm)
  • functional test added/updated (if change is functional)
  • man page updated (if applicable)
  • bash completion updated (if applicable)
  • documentation updated
  • author name in AUTHORS

somaz94 added a commit to somaz94/somaz94 that referenced this pull request Nov 11, 2026
@codecov

codecov Bot commented Nov 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 74.35897% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.60%. Comparing base (e35afce) to head (19cca50).

Files with missing lines Patch % Lines
deb/remote.go 72.97% 5 Missing and 5 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1665      +/-   ##
==========================================
- Coverage   77.91%   77.60%   -0.32%     
==========================================
  Files         165      165              
  Lines       16141    16180      +39     
==========================================
- Hits        12577    12557      -20     
- Misses       2350     2408      +58     
- Partials     1214     1215       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Updating mirrors for Ubuntu stonking redownloads existing packages

1 participant