Skip to content

[low] Use PyPI API filename field for duplicate-distribution check - #49

Open
elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/43-use-api-filename
Open

[low] Use PyPI API filename field for duplicate-distribution check#49
elhoim wants to merge 1 commit into
MISP:mainfrom
elhoim:fix/43-use-api-filename

Conversation

@elhoim

@elhoim elhoim commented Aug 31, 2026

Copy link
Copy Markdown
Member

BLUF — The PyPI duplicate check compares a URL-derived filename, so published files get re-uploaded.

  • Problem — The duplicate-distribution check in .github/workflows/pypi-publish.yml rebuilds the comparison filename by unquoting the basename of the PyPI JSON API's url field, even though the same response already carries a canonical filename field; under PEP 625 sdist name normalisation the URL basename can diverge from it, so an already-published file fails to match.
  • Fix — Uses file_info['filename'] directly and drops the now-unused unquote and urlparse import.
  • Effect — Release runs skip already-published distributions reliably instead of attempting a redundant re-upload.

Finding 43 (Low) — .github/workflows/pypi-publish.yml:78-82

Problem

The comparison filename is reconstructed via unquote(Path(urlparse(file_info['url']).path).name) when PyPI's JSON API already returns file_info['filename'] directly; sdist name normalization drift (PEP 625) would defeat the duplicate check.

Fix

The duplicate-distribution check in .github/workflows/pypi-publish.yml reconstructed the comparison filename by unquoting the basename of file_info['url'] instead of using file_info['filename'], which PyPI's JSON API already provides directly. Under PEP 625 sdist name normalization, the URL path's basename can diverge from the canonical distribution filename (e.g. case/separator normalization differences), defeating the duplicate check so an already-published file could fail to be skipped, causing a spurious re-upload attempt. Fixed by using file_info['filename'] directly and dropping the now-unused unquote/urlparse import.

Verification

Reproduced against the unmodified code at 9b8c605, then re-checked after the change.

Before
Simulated PyPI JSON API file_info with url basename differing from filename field (PEP 625-style normalization drift):
  file_info = {"filename": "misp_modules_cli-1.2.0.tar.gz", "url": ".../Misp-Modules-CLI-1.2.0.tar.gz"}
  old_way = unquote(Path(urlparse(file_info["url"]).path).name)  -> "Misp-Modules-CLI-1.2.0.tar.gz"
  Comparing against actual dist/ artifact name "misp_modules_cli-1.2.0.tar.gz": old_way == artifact -> False (duplicate check fails to match, would re-attempt upload of an already-published file)
After
Same file_info; new_way = file_info["filename"] -> "misp_modules_cli-1.2.0.tar.gz"
  new_way == "misp_modules_cli-1.2.0.tar.gz" -> True (duplicate check now correctly matches the actual artifact name)
Also verified: full embedded Python block still `compile()`s cleanly after the edit.

python bin/cli.py --help exits 0 and the module still imports cleanly. Verification was performed offline against the pure functions — no running misp-modules instance is required.

Branched from 9b8c605. This PR addresses only this finding; the other findings from the same review are in separate PRs, so they will need rebasing against each other as they merge.

🤖 Generated with Claude Code

The duplicate-distribution filter reconstructed the comparison filename by unquoting the basename of file_info['url'], instead of using file_info['filename'] which PyPI's JSON API already provides directly. Under PEP 625 sdist name normalization, the URL path and the actual filename can diverge, defeating the duplicate check and causing re-uploads or upload failures. Use file_info['filename'] directly.
@elhoim elhoim changed the title Use PyPI API filename field for duplicate-distribution check [low] Use PyPI API filename field for duplicate-distribution check Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant