Scan a hugepage bitmap in HugeRegion::Release instead of a linear walk. - #1009
Draft
copybara-service[bot] wants to merge 1 commit into
Draft
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
HugeRegion::Release selected free-but-backed hugepages by walking all kNumHugePages (512) entries of backed_ and pages_used_, forwards or backwards depending on adaptive_release, until enough candidates were found. Track releasability directly in a Bitmap<kNumHugePages> (unreleasable_hugepages_): bit i is set iff hugepage i cannot be released right now (unbacked, in use, or mid-unback in UnbackHugepages), so the clear bits are exactly the free-but-backed hugepages. Inc/Dec/UnbackHugepages maintain it alongside free_backed_count_, and a debug assert checks CountBits() == kNumHugePages - free_backed_count_. Release now walks free ranges of that bitmap with word-at-a-time scans: NextFreeRange for the forward (non-adaptive) order and the new Bitmap::PrevFreeRange for the reverse (adaptive) order, taking the highest hugepages of a run first so the selected set is identical to the old reverse scan. Release also returns early when nothing is free-but-backed, skipping the UnbackHugepages pass over an all-false should_unback array. Bitmap::PrevFreeRange mirrors NextFreeRange: it clamps end to N and is built on FindClearBackwards/FindSetBackwards, so it never inspects individual bits. Add tests for PrevFreeRange boundary cases (end == 0, end > N, all-set, all-clear, ranges at index 0 and ending at N, word-boundary ranges) and for Release across fragmented free runs in both orders. PiperOrigin-RevId: 983926156
copybara-service
Bot
force-pushed
the
test_983926156
branch
from
September 21, 2026 02:33
cca1559 to
1e4dfbc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scan a hugepage bitmap in HugeRegion::Release instead of a linear walk.
HugeRegion::Release selected free-but-backed hugepages by walking all
kNumHugePages (512) entries of backed_ and pages_used_, forwards or backwards
depending on adaptive_release, until enough candidates were found.
Track releasability directly in a Bitmap
(unreleasable_hugepages_): bit i is set iff hugepage i cannot be released
right now (unbacked, in use, or mid-unback in UnbackHugepages), so the clear
bits are exactly the free-but-backed hugepages. Inc/Dec/UnbackHugepages
maintain it alongside free_backed_count_, and a debug assert checks
CountBits() == kNumHugePages - free_backed_count_.
Release now walks free ranges of that bitmap with word-at-a-time scans:
NextFreeRange for the forward (non-adaptive) order and the new
Bitmap::PrevFreeRange for the reverse (adaptive) order, taking the highest
hugepages of a run first so the selected set is identical to the old reverse
scan. Release also returns early when nothing is free-but-backed, skipping
the UnbackHugepages pass over an all-false should_unback array.
Bitmap::PrevFreeRange mirrors NextFreeRange: it clamps end to N and is built
on FindClearBackwards/FindSetBackwards, so it never inspects individual bits.
Add tests for PrevFreeRange boundary cases (end == 0, end > N, all-set,
all-clear, ranges at index 0 and ending at N, word-boundary ranges) and for
Release across fragmented free runs in both orders.