Repository navigation
feat(strings): add suffix array algorithm #15688
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
40e7bfd
8be06b2
cacfbac
1e02eda
4bd43e4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| """ | ||
| https://en.wikipedia.org/wiki/Suffix_array | ||
| https://cp-algorithms.com/string/suffix-array.html | ||
|
|
||
| A suffix array lists the starting positions of every suffix of a string in | ||
| lexicographic order. It needs one integer per suffix, while a suffix tree | ||
| needs up to 2n nodes, so it is the compact structure used for pattern search | ||
| and for the LCP-based algorithms built on top of it. | ||
|
|
||
| Construction here uses prefix doubling (Manber-Myers). It is simpler and much | ||
| lighter in memory than the linear-time constructions (Ukkonen for suffix | ||
| trees, SA-IS for suffix arrays), but runs slower: O(n log² n) instead of O(n). | ||
|
|
||
| Time Complexity: O(n log² n) - where n is the length of the text | ||
| Space Complexity: O(n) | ||
| """ | ||
|
|
||
|
|
||
| def suffix_array(text: str) -> list[int]: | ||
| """ | ||
| Return the starting indices of all suffixes of `text`, in lexicographic | ||
| order. | ||
|
|
||
| >>> suffix_array("banana") | ||
| [5, 3, 1, 0, 4, 2] | ||
| >>> suffix_array("mississippi") | ||
| [10, 7, 4, 1, 0, 9, 8, 6, 3, 5, 2] | ||
| >>> suffix_array("abababab") | ||
| [6, 4, 2, 0, 7, 5, 3, 1] | ||
| >>> suffix_array("aaaa") | ||
| [3, 2, 1, 0] | ||
| >>> suffix_array("aça") | ||
| [2, 0, 1] | ||
| >>> suffix_array("a") | ||
| [0] | ||
| >>> suffix_array("") | ||
| [] | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correctness looks good: I compared this against the naive sorted(range(n), key=lambda i: text[i:]) on 3,000 random strings over small alphabets, including non-ASCII characters, with no mismatches. Could you add a doctest that does the same cross-check, e.g.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added, using |
||
| >>> all( | ||
| ... suffix_array(s) == sorted(range(len(s)), key=lambda i: s[i:]) | ||
| ... for s in ("aaaa", "abab", "banana", "mississippi", "abracadabra") | ||
| ... ) | ||
| True | ||
| """ | ||
| n = len(text) | ||
| if n == 0: | ||
| return [] | ||
|
|
||
| suffixes = list(range(n)) | ||
| # Ranking by code point is enough to start: Python orders strings by code | ||
| # point too, so one comparison of the first character is already right. | ||
| rank = list(map(ord, text)) | ||
| k = 1 | ||
|
|
||
| while k < n: | ||
| # Sort by the first 2k characters of each suffix, which are exactly the | ||
| # pair (rank of the first k, rank of the next k). A second half that | ||
| # runs past the end gets -1: ranks are never negative, so a shorter | ||
| # suffix sorts before a longer one that shares its prefix. The index i | ||
| # only breaks ties so the order stays deterministic; tied suffixes are | ||
| # given the same rank in the renumbering below. | ||
| keys = [(rank[i], rank[i + k] if i + k < n else -1, i) for i in range(n)] | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Minor: the third tuple element i is only a tie-breaker, and the renumbering step makes tied entries share a rank. A short comment saying so would save readers a moment.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added a comment noting the index only makes the sort deterministic and that tied suffixes share a rank in the renumbering step. |
||
| keys.sort() | ||
| suffixes = [i for _, _, i in keys] | ||
|
|
||
| # Renumber so that equal 2k-prefixes share a rank and every distinct | ||
| # prefix gets the next one. This keeps ranks dense in range(n). | ||
| # `keys` is already in the same order as `suffixes`, so the first two | ||
| # entries of neighbouring keys are the pairs to compare. | ||
| new_rank = [0] * n | ||
| for position in range(1, n): | ||
| previous, current = suffixes[position - 1], suffixes[position] | ||
| same_prefix = keys[position - 1][:2] == keys[position][:2] | ||
| new_rank[current] = new_rank[previous] + int(not same_prefix) | ||
|
|
||
| rank = new_rank | ||
| # Dense ranks ending at n - 1 means every suffix has a unique rank, so | ||
| # the 2k-prefixes already order them completely; doubling cannot change | ||
| # the order, and the next round always breaks too, since 2k >= n. | ||
| if rank[suffixes[-1]] == n - 1: | ||
| break | ||
| k *= 2 | ||
|
|
||
| return suffixes | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Optional: most files in this repo end with an
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added the |
||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| import doctest | ||
|
|
||
| doctest.testmod() | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This answers cclauss's question well. Since the advantage over suffix trees is memory and simplicity, please say plainly that construction here is slower than the O(n) approaches (Ukkonen for suffix trees, SA-IS for suffix arrays). Also, "a node per character" (lines 6 to 7) is slightly loose: a suffix tree has up to 2n nodes, so "up to 2n nodes" is more precise.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both are in the module docstring now: "up to 2n nodes" for suffix trees, and an explicit sentence that prefix doubling is O(n log² n) versus the O(n) constructions (Ukkonen, SA-IS).