Repository navigation
fn_addr_eq: we actually can guarantee basically nothing - #162140
Conversation
|
r? @joboet rustbot has assigned @joboet. Use Why was this reviewer chosen?The reviewer was selected based on:
|
| /// (From an implementation perspective, this is possible because functions may sometimes be | ||
| /// processed more than once by the compiler, resulting in duplicate machine code.) | ||
| /// | ||
| /// Despite these false positives and false negatives, this comparison can still be useful. |
There was a problem hiding this comment.
One potentially remaining use we were able to come up with is a set of callbacks.
If deduplicating additions to the set by "the runtime behavior should happen at least once" works for you then this should work fine.
There was a problem hiding this comment.
That seems incredibly niche.
Would you like to see it spelled out in the docs?
There was a problem hiding this comment.
Well, the more general idea was that it might be helpful to also have a list of the few things that it can be used for, it might help users to figure out which side they fall on.
That's just the only practical thing that came to mind.
There was a problem hiding this comment.
Wouldn't cases like the "different unsafe preconditions" example above cause incorrect results for this kind of use case? Equality returning true does not guarantee that they have the same behavior, so deduplicating in that scenario is not correct.
There was a problem hiding this comment.
If the callbacks are all required to be safe to call, then calling fewer of them due to deduplication cannot introduce UB.
There is some sense in which equal functions have the same behavior -- they have the same asm code, after all. We might be able to give a guarantee of the sort "if neither function causes UB for the given arguments, then calling f is equivalent to calling g"?
Cc @rust-lang/opsem
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
Suppose there is a bunch of function pointers. They are compared and called.
All function called are identified at compile time. All calls are inlined.
Functions are never called, so their body is optimized out.
Functions are merged together. All function have the same address.
Ok, having reread that... I guess that depends on whether the comparison lets the function addresses escape like black_box would? Then the optimizer can't remove their bodies. I think even whole-program optimization wouldn't allow it since something could temporarily encrypt the bits of the pointers in some way.
There was a problem hiding this comment.
Not fully arbitrarily, no. It's still the case that it must be source code that can compile to the same asm as other source code. The docs have a "guaranteed not equal" example of two functions that print different strings; that guarantee sounds legit to me.
The example perfectly demonstrates my argument from #162140 (comment). Those functions are never called. Their bodies could be optimized out as unreachable, function merged, and function pointer would compare equal.
There was a problem hiding this comment.
Hm... I don't think LLVM considers this a legal transformation today, but I'm also not sure how to forbid it in the spec.
I have removed the example.
There was a problem hiding this comment.
Hm... I don't think LLVM considers this a legal transformation today, but I'm also not sure how to forbid it in the spec.
LLVM won't do that transform today, but I think it would be legal? Comparison only escapes the address, not the provenance, so the function cannot be called, and optimizing it to unreachable is allowed.
550db73 to
a54d730
Compare
| /// never introduce undefined behavior because it only skips executing some of the registered | ||
| /// callbacks. | ||
| /// | ||
| /// # Examples |
There was a problem hiding this comment.
Would also just add here: in general, while there may be niche cases that this can be legitimately useful, in pretty much all cases you're also encouraged to refactor your code anyway to avoid the pitfalls. For example, if you make your functions instead be ZSTs that implement CustomTrait and use Arc<dyn CustomTrait>, now all of a sudden you can use Arc::ptr_eq and expect it to mean what you want. In general avoiding function pointers is just more likely to always do the correct thing and not be noticeably worse in performance in ways that matter.
There was a problem hiding this comment.
I think you mean Arc::ptr_eq.
that said, comparing &dyn CustomTrait with std::ptr::addr_eq also has pitfalls, so I wouldn't necessarily recommend it. e.g. two different &dyn CustomTrait can compare equal with addr_eq, but have different behavior:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2024&gist=9c9028b37168b6e402b51eb4996b2eef
There was a problem hiding this comment.
Right, I forgot that vtables are genuinely different even though the pointers may be the same, especially for ZSTs.
That said, if you use Arc::ptr_eq, you should refer to the same reference-counted version of a dyn CustomTrait, even if the unsized pointers may compare identically. And really, my point was that there are ways of achieving infallible equality in ways that make more sense for your program that don't rely on function pointer equality.
| /// These limitations imply that comparing function pointers (via this function or via `==`) is only | ||
| /// useful in extremely niche circumstances. For instance, if a library maintains a set of | ||
| /// callbacks, then it can be legimiate to deduplicate callbacks based on `fn_addr_eq`. This can | ||
| /// never introduce undefined behavior because it only skips executing some of the registered | ||
| /// callbacks. | ||
| #[stable(feature = "ptr_fn_addr_eq", since = "1.85.0")] |
There was a problem hiding this comment.
I believe that this is not legitimate. Suppose that one of the callbacks exposes the provenance of one pointer, while the other callback doesn't, and they have the same machine code. This deduplication can cause a pointer's provenance to not be exposed, even though it would have been exposed without the deduplication. This can technically increase the amount of UB.
There was a problem hiding this comment.
If the library just says "I will call some of these callbacks but not all of them", that's legitimate.
There was a problem hiding this comment.
Similar issues can happen without exposed provenance if one of the callbacks mutates some byte by adding provenance to it.
There was a problem hiding this comment.
I'm trying to understand how that might cause problems where you run fewer copies of a callback than otherwise expected, though. Like, if the provenance is needed and isn't provided.
There was a problem hiding this comment.
expose_provenance is a side-effect. If something else relies on the side-effect having happened, then they're toast.
So this amounts to the library basically saying "callbacks may spuriously just not be run for arbitrary reasons". I don't know when that is ever useful, but it's a spec one can write and deduplication with fn_addr_eq does satisfy that spec.
I have edited the text to make this more clear. I'm still not sure this example is worth actually spelling out, but t-libs seems to prefer having an extremely artificial example over not having any example.
There was a problem hiding this comment.
To be clear, I'm not 100% sure that's a blocking opinion from libs, just, we really struggled to find examples and wanted to justify this function's existence somehow for a reason other than just documenting how bad function equality is. It does still feel worth having some kind of attribute like #[diagnostic::discouraged] to force people to allow a lint that explains why you almost always do not want to use this function.
Since, well, the main issue is that we know that if we don't offer this function, people are going to try and find a way to hack in its functionality anyway, e.g. casting to non-function pointers. So, we might as well offer a better way if it's already possible, but still heavily discourage it.
There was a problem hiding this comment.
It does still feel worth having some kind of attribute like #[diagnostic::discouraged] to force people to allow a lint that explains why you almost always do not want to use this function.
I don't disagree, but I'm not signing up for being the one to implement that.
There was a problem hiding this comment.
Maybe just keep:
These limitations imply that comparing function pointers (via this function or via
==) is only useful in extremely niche circumstances.
...and cut the rest?
c207ce5 to
72b5658
Compare
|
I'm pretty torn here because this feels alot to me like the LLVM bugs we know about around ptr2int+int2ptr, which we've generally filed more under "that's a bug we hope to get fixed" not "we should change the documented behaviour". Deduping on LLVM-IR is fine, right? So this is just about whether things are also deduping on the machine code? How often are we emitting things where the former isn't doing most of the work? (Certainly having no deduping would be really bad for lots of small functions, but things like deduping |
This comment was marked as outdated.
This comment was marked as outdated.
|
I wonder if it makes sense for Miri to always, or randomly, return |
|
Miri cannot know when returning true would be acceptable, and it does not have a concept of "machine code". Returning true when the pointers are the same value, and conservatively returning false in all other cases is the only thing that Miri can implement. |
|
Every function has an integer address in Miri. There's nothing to be "conservative" about. In fact generic functions get more than one address (we pick a new one on every fn-item-to-fn-ptr cast), so Miri already sometimes returns Miri could give all functions the same address. Or bucket all functions onto 16 different addresses, or so. Though we probably should only do that with |
|
I feel like giving all functions the same address should only be enabled via a |
It's a good way to test code but the problem is that it could also hide the other path. |
Exactly what an evil tool would do, yes. |
|
The final comment period, with a disposition to merge, as per the review above, is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. |
|
@joboet this is ready for your review. :) Or maybe someone else who's reading along wants to take this? |
|
I can take over if needed, although I'll let @joboet decide for now |
|
FCP has completed and this reads fine, so |
fn_addr_eq: we actually can guarantee basically nothing Fixes rust-lang#160202. See that issue for context and discussion.
fn_addr_eq: we actually can guarantee basically nothing Fixes rust-lang#160202. See that issue for context and discussion.
…uwer Rollup of 8 pull requests Successful merges: - #164196 (`rust-analyzer` subtree update) - #164037 (Restrict application of optimization attributes from instance) - #164201 (Improve `--bless` output for `tests/debuginfo`) - #164202 ([rustdoc] Move the creation of `IndividualTestOptions` into the test runner closure to ensure the temporary folder is removed right after the test is done running) - #162140 (fn_addr_eq: we actually can guarantee basically nothing) - #163946 (Fix transmutability type mismatch bug) - #163978 (Copy over some methods from At to InferCtxt) - #164045 (-Zassumptions-on-binders: replace region vars with bound vars when handling alias outlives')
View all comments
Fixes #160202.
See that issue for context and discussion.