Skip to content

don't inhibit random field reordering on repr(packed(1)) - #125360

Merged
bors merged 1 commit into
rust-lang:masterfrom
RalfJung:packed-field-reorder
May 29, 2024
Merged

bors merged 1 commit into
rust-lang:masterfrom
RalfJung:packed-field-reorder

Conversation

@RalfJung

@RalfJung RalfJung commented May 21, 2024 •

Copy link
Copy Markdown
Member

inhibit_struct_field_reordering_opt being false means we exclude this type from random field shuffling. However, packed(1) types can still be shuffled! The logic was added in #48528 since it's pointless to reorder fields in packed(1) types (there's no padding that could be saved) -- but that shouldn't inhibit -Zrandomize-layout (which did not exist at the time).

We could add an optimization elsewhere to not bother sorting the fields for repr(packed) types, but I don't think that's worth the effort.

This does change the behavior in that we may now reorder fields of packed(1) structs (e.g. if there are niches, we'll try to move them to the start/end, according to NicheBias). We were always allowed to do that but so far we didn't. Quoting the reference:

On their own, align and packed do not provide guarantees about the order of fields in the layout of a struct or the layout of an enum variant, although they may be combined with representations (such as C) which do provide such guarantees.

@rustbot

ghost commented May 21, 2024

Copy link
Copy Markdown
Collaborator

r? @fmease

rustbot has assigned @fmease.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 21, 2024
@RalfJung
RalfJung force-pushed the packed-field-reorder branch from ea7545b to 1fcce90 Compare May 21, 2024 12:12
pub fn can_randomize_type_layout(&self) -> bool {
!self.inhibit_struct_field_reordering_opt()
&& self.flags.contains(ReprFlags::RANDOMIZE_LAYOUT)
!self.inhibit_struct_field_reordering() && self.flags.contains(ReprFlags::RANDOMIZE_LAYOUT)

ghost May 21, 2024 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still a bit silly, the only place where can_randomize_type_layout is called is effectively inside an if !self.inhibit_struct_field_reordering()... but I didn't want to do a larger refactoring here, not sure what even would be the better structure.

@the8472

ghost commented May 21, 2024

Copy link
Copy Markdown
Member

Note that this does not only affect Zrandomize-layout builds. Some structs will also be reordered in regular builds.

@rust-log-analyzer

This comment has been minimized.

@RalfJung
RalfJung force-pushed the packed-field-reorder branch from 1fcce90 to a53f3ed Compare May 21, 2024 12:52
@rustbot

ghost commented May 21, 2024

Copy link
Copy Markdown
Collaborator

Some changes occurred in src/tools/clippy

cc @rust-lang/clippy

continue;
}
if def.repr().inhibit_struct_field_reordering_opt() {
if def.repr().inhibit_struct_field_reordering() {

ghost May 21, 2024

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW clippy already used to assume inhibit_struct_field_reordering_opt means that the field order is guaranteed, which was not the case before this PR.

@rust-log-analyzer

This comment has been minimized.

@RalfJung
RalfJung force-pushed the packed-field-reorder branch from a53f3ed to 823f5bb Compare May 21, 2024 14:32
@rust-log-analyzer

This comment has been minimized.

@RalfJung
RalfJung force-pushed the packed-field-reorder branch from 823f5bb to f707e61 Compare May 21, 2024 16:38
@rust-log-analyzer

This comment has been minimized.

@RalfJung
RalfJung force-pushed the packed-field-reorder branch from f707e61 to 37aeb75 Compare May 21, 2024 17:22
@rustbot

ghost commented May 21, 2024

Copy link
Copy Markdown
Collaborator

The Miri subtree was changed

cc @rust-lang/miri

@fmease

ghost commented May 29, 2024

Copy link
Copy Markdown
Member

@bors r+

@bors

ghost commented May 29, 2024

Copy link
Copy Markdown
Collaborator

📌 Commit 37aeb75 has been approved by fmease

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 29, 2024
@bors

ghost commented May 29, 2024

Copy link
Copy Markdown
Collaborator

⌛ Testing commit 37aeb75 with merge f2e1a3a...

@the8472 the8472 added the relnotes Marks issues that should be documented in the release notes of the next release. label May 29, 2024
@bors

ghost commented May 29, 2024

Copy link
Copy Markdown
Collaborator

☀️ Test successful - checks-actions
Approved by: fmease
Pushing f2e1a3a to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label May 29, 2024
@bors
bors merged commit f2e1a3a into rust-lang:master May 29, 2024
@rustbot rustbot added this to the 1.80.0 milestone May 29, 2024
@pnkfelix

ghost commented Jun 4, 2024

Copy link
Copy Markdown
Contributor

@rustbot label: +perf-regression-triaged

@rustbot rustbot added the perf-regression-triaged The performance regression has been triaged. label Jun 4, 2024
@Manishearth

ghost commented Jun 13, 2024 •

Copy link
Copy Markdown
Member

This broke ICU4X. It's our fault: we assumed repr(packed) meant repr(C, packed), but this is an assumption I have seen quite commonly in the Rust ecosystem around networking and FFI code, since there has never really been a concept of "Rust packed" distinct from "C packed" till now, and 99% of code using packed will either be using it for networking (where they need C, packed, or for zero-copy stuff (where they need stability, which you get from C, packed)

So while I wouldn't go as far as to say it's a breaking change, I wonder if this should be done more carefully, perhaps requiring packed to be paired with Rust or C.

Or even having repr(packed) default to C, packed unless explicitly specified? Rust, packed does seem to be somewhat niche to me until repr(Rust) stabilizes.

(The fix for ICU4X is unicode-org/icu4x#5049)

@the8472

ghost commented Jun 13, 2024

Copy link
Copy Markdown
Member

Some crates getting broken is expected, it also happened the last time we changed field ordering (#102750) which is why i added the relnotes label.

Also note that packed(2) and up already did reorder fields.

If the beta crater run shows a bigger impact then a more careful approach would make sense.

@Manishearth

ghost commented Jun 13, 2024

Copy link
Copy Markdown
Member

Some crates getting broken is expected, it also happened the last time we changed field ordering (#102750) which is why i added the relnotes label.

Right, but I think in this case, while the nomicon and reference seem to be clear that repr(packed) is repr(Rust, packed) by default, my experience with the ecosystem is that most people assume repr(packed) to mean repr(C, packed), because repr(Rust, packed) doesn't really make sense for 99% of the use cases of packed.

So while the docs are clear, I wonder if the ecosystem's impression of how things work is going to be a problem for this change. That's not as much the case for packed(2).

I'm also not sure if this will show up much on crater since it may require specific kinds of test failures.

@ChrisDenton

ghost commented Jun 18, 2024

Copy link
Copy Markdown
Member

If this is a common source of confusion, maybe there should be a lint that encourages people to use an explicit representation with packed?

@cuviper

ghost commented Jun 19, 2024

Copy link
Copy Markdown
Member

maybe there should be a lint that encourages people to use an explicit representation with packed?

The missing-abi lint is similar in spirit, "No declared ABI for extern declaration." That one is allowed by default, but in this case the implicit "Rust" ABI may be more surprising, justifying a warning by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merged-by-bors This PR was explicitly merged by bors. perf-regression Performance regression. perf-regression-triaged The performance regression has been triaged. relnotes Marks issues that should be documented in the release notes of the next release. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.