Fold the partition select into the sized-free tag compare. - #1006
Draft
copybara-service[bot] wants to merge 1 commit into
Draft
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
do_free_with_size previously tested (uptr & kNormalOrBadDeallocationMask) != kNormalMask (accepting both kNormalP0 and kNormalP1 via their shared bit) and then had fast_free_with_size re-derive the partition from the pointer with PartitionFromPointerFast (x86: btq $43; jb / aarch64: tbnz). Heap partitioning defaults to off, so essentially every sized free is P0. Widen the mask to kTagOrBadDeallocationMask and compare against the exact kNormalP0 tag value: the not-taken fallthrough now is P0, and the partition is known without a second test. On the taken side, test the exact kNormalP1 value first (a partitioned heap still pays two compares, as before), then nullptr (tag 0 fails both compares), cold, and the sampled/illformed handler. fast_free_with_size takes the partition as a template parameter, which also removes the duplicated GetSizeClass/FreeSmall arms (b/470136917). The fold is sound because the tag field of kTagOrBadDeallocationMask is disjoint from the high-bits and alignment fields and the exact kNormalP0 / kNormalP1 values lie entirely within kTagMask; static_asserts check both, plus that neither tag value is 0 so nullptr is rejected. kNormalPartitions == 1 builds compile the P1 test out. A new test covers nullptr through every sized deallocation entry point. x86 TCMallocInternalDeleteSized drops btq/jb (-2 instructions, -1 cmp/-1 jcc in the fast_path goldens for delete(size) and delete(size,align)); aarch64 drops the tbnz (-1 jcc). The cold-hint path gains one compare (cycles unchanged in the benchmark). The only other golden delta is new(cold,token_1)->size in the two release_base_opt goldens (jcc -1, cmov +1). That row already differs from the checked-in golden when built from unmodified HEAD sources (FDO drift), so it is picked up by the regeneration but is not caused by this change. PiperOrigin-RevId: 983910800
copybara-service
Bot
force-pushed
the
test_983910800
branch
from
September 21, 2026 02:38
10e2b09 to
23b886a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fold the partition select into the sized-free tag compare.
do_free_with_size previously tested (uptr & kNormalOrBadDeallocationMask) !=
kNormalMask (accepting both kNormalP0 and kNormalP1 via their shared bit) and
then had fast_free_with_size re-derive the partition from the pointer with
PartitionFromPointerFast (x86: btq $43; jb / aarch64: tbnz).
Heap partitioning defaults to off, so essentially every sized free is P0.
Widen the mask to kTagOrBadDeallocationMask and compare against the exact
kNormalP0 tag value: the not-taken fallthrough now is P0, and the partition is
known without a second test. On the taken side, test the exact kNormalP1 value
first (a partitioned heap still pays two compares, as before), then nullptr
(tag 0 fails both compares), cold, and the sampled/illformed handler.
fast_free_with_size takes the partition as a template parameter, which also
removes the duplicated GetSizeClass/FreeSmall arms (b/470136917).
The fold is sound because the tag field of kTagOrBadDeallocationMask is
disjoint from the high-bits and alignment fields and the exact kNormalP0 /
kNormalP1 values lie entirely within kTagMask; static_asserts check both, plus
that neither tag value is 0 so nullptr is rejected. kNormalPartitions == 1
builds compile the P1 test out. A new test covers nullptr through every sized
deallocation entry point.
x86 TCMallocInternalDeleteSized drops btq/jb (-2 instructions, -1 cmp/-1 jcc in
the fast_path goldens for delete(size) and delete(size,align)); aarch64 drops
the tbnz (-1 jcc). The cold-hint path gains one compare (cycles unchanged in
the benchmark).
The only other golden delta is new(cold,token_1)->size in the two
release_base_opt goldens (jcc -1, cmov +1). That row already differs from the
checked-in golden when built from unmodified HEAD sources (FDO drift), so it is
picked up by the regeneration but is not caused by this change.