Repository navigation
Fix Adler-32 for non-ASCII text #15665
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -14,7 +14,7 @@ | |
| def adler32(plain_text: str) -> int: | ||
| """ | ||
| Function implements adler-32 hash. | ||
| Iterates and evaluates a new value for each character | ||
| Iterates over the UTF-8 bytes of the input text. | ||
|
|
||
| >>> adler32('Algorithms') | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The fix is correct: I compared it with zlib.adler32(text.encode()) on "€100", "héllo", "日本語" and an emoji, and all matched. But the doctests only cover ASCII, so nothing would catch this bug again. Please add non-ASCII doctests with the literal expected values, e.g.: You could also compare against zlib: |
||
| 363791387 | ||
|
|
@@ -24,7 +24,7 @@ def adler32(plain_text: str) -> int: | |
| """ | ||
| a = 1 | ||
| b = 0 | ||
| for plain_chr in plain_text: | ||
| a = (a + ord(plain_chr)) % MOD_ADLER | ||
| for plain_byte in plain_text.encode("utf-8"): | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Looks good: encode() is evaluated once before the loop, and UTF-8 matches what zlib.adler32 gets when callers pass text.encode(). One suggestion: say in the docstring that the checksum is over the UTF-8 bytes, so non-ASCII results match zlib. That is the behaviour change, so documenting it helps. Line 17 already says "UTF-8 bytes", which is a good start. |
||
| a = (a + plain_byte) % MOD_ADLER | ||
| b = (b + a) % MOD_ADLER | ||
| return (b << 16) | a | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Optional: there is another open PR for the same issue that changes the signature to bytes (this one keeps str and is backward compatible). The two overlap, so it may be worth coordinating with the maintainers, so only one is merged. This approach avoids breaking callers that pass strings, which I'd consider a plus.