Skip to content

Fold the partition select into the sized-free tag compare. - #1006

Draft
copybara-service[bot] wants to merge 1 commit into
masterfrom
test_983910800
Draft

copybara-service[bot] wants to merge 1 commit into
masterfrom
test_983910800

Conversation

@copybara-service

Copy link
Copy Markdown

Fold the partition select into the sized-free tag compare.

do_free_with_size previously tested (uptr & kNormalOrBadDeallocationMask) !=
kNormalMask (accepting both kNormalP0 and kNormalP1 via their shared bit) and
then had fast_free_with_size re-derive the partition from the pointer with
PartitionFromPointerFast (x86: btq $43; jb / aarch64: tbnz).

Heap partitioning defaults to off, so essentially every sized free is P0.
Widen the mask to kTagOrBadDeallocationMask and compare against the exact
kNormalP0 tag value: the not-taken fallthrough now is P0, and the partition is
known without a second test. On the taken side, test the exact kNormalP1 value
first (a partitioned heap still pays two compares, as before), then nullptr
(tag 0 fails both compares), cold, and the sampled/illformed handler.
fast_free_with_size takes the partition as a template parameter, which also
removes the duplicated GetSizeClass/FreeSmall arms (b/470136917).

The fold is sound because the tag field of kTagOrBadDeallocationMask is
disjoint from the high-bits and alignment fields and the exact kNormalP0 /
kNormalP1 values lie entirely within kTagMask; static_asserts check both, plus
that neither tag value is 0 so nullptr is rejected. kNormalPartitions == 1
builds compile the P1 test out. A new test covers nullptr through every sized
deallocation entry point.

x86 TCMallocInternalDeleteSized drops btq/jb (-2 instructions, -1 cmp/-1 jcc in
the fast_path goldens for delete(size) and delete(size,align)); aarch64 drops
the tbnz (-1 jcc). The cold-hint path gains one compare (cycles unchanged in
the benchmark).

The only other golden delta is new(cold,token_1)->size in the two
release_base_opt goldens (jcc -1, cmov +1). That row already differs from the
checked-in golden when built from unmodified HEAD sources (FDO drift), so it is
picked up by the regeneration but is not caused by this change.

do_free_with_size previously tested (uptr & kNormalOrBadDeallocationMask) !=
kNormalMask (accepting both kNormalP0 and kNormalP1 via their shared bit) and
then had fast_free_with_size re-derive the partition from the pointer with
PartitionFromPointerFast (x86: btq $43; jb / aarch64: tbnz).

Heap partitioning defaults to off, so essentially every sized free is P0.
Widen the mask to kTagOrBadDeallocationMask and compare against the exact
kNormalP0 tag value: the not-taken fallthrough now is P0, and the partition is
known without a second test.  On the taken side, test the exact kNormalP1 value
first (a partitioned heap still pays two compares, as before), then nullptr
(tag 0 fails both compares), cold, and the sampled/illformed handler.
fast_free_with_size takes the partition as a template parameter, which also
removes the duplicated GetSizeClass/FreeSmall arms (b/470136917).

The fold is sound because the tag field of kTagOrBadDeallocationMask is
disjoint from the high-bits and alignment fields and the exact kNormalP0 /
kNormalP1 values lie entirely within kTagMask; static_asserts check both, plus
that neither tag value is 0 so nullptr is rejected.  kNormalPartitions == 1
builds compile the P1 test out.  A new test covers nullptr through every sized
deallocation entry point.

x86 TCMallocInternalDeleteSized drops btq/jb (-2 instructions, -1 cmp/-1 jcc in
the fast_path goldens for delete(size) and delete(size,align)); aarch64 drops
the tbnz (-1 jcc).  The cold-hint path gains one compare (cycles unchanged in
the benchmark).

The only other golden delta is new(cold,token_1)->size in the two
release_base_opt goldens (jcc -1, cmov +1).  That row already differs from the
checked-in golden when built from unmodified HEAD sources (FDO drift), so it is
picked up by the regeneration but is not caused by this change.

PiperOrigin-RevId: 983910800
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant