[RalfJung]

I am fairly sure we discussed at some point whether it's okay to access memory beyond the declared size of an extern static, and @nikic confirmed that this is fine. But I couldn't find any official place where we document this. So, let's add it to the Reference.

I wondered if we also need the converse on the UB page: it is UB to even start the program if an extern static ends up being backed by memory that is smaller or less-aligned than the declared type. What do you think?

Cc @rust-lang/opsem
Fixes rust-lang/unsafe-code-guidelines#259
Fixes rust-lang/unsafe-code-guidelines#554

[Jules-Bertholet]

Perhaps be more explicit about the need to use &raw (and avoid an intermediate reference)?

View changes since the review

[RalfJung]

I have removed the &raw requirement, anticipating #2338. I don't think we want subobject provenance to get in the way here, so this should be fine (if the static is big enough to read element N):

extern { static mut X: (); }

let ptr = &X as *const () as *const i32;
ptr.add(N).read();

This is a bad idea because of aliasing rules (the pointer gets invalidated on writes), but we already lint against that.

[DianaNites]

This would formally resolve/answer rust-lang/unsafe-code-guidelines#259 and maybe some of rust-lang/unsafe-code-guidelines#546 right?

[RalfJung]

The latter has already been resolved by #1657.

And this does resolve the former, yes. Thanks for digging that up.

[RalfJung]

This new kind of UB is kind of remarkable in that it does not require any code to trigger. But I don't see an alternative...

View changes since the review

[workingjubilee]

seems we sure did declare extern "abi" to be unsafe for a good reason.

[bjorn3]

For dynamic libraries that may be used by a PIE executable, the size given by the symbol must be exact given that the PIE executable will emit a copy relocation that copies a block with exactly the size the symbol had at link time to memory the executable image has reserved for this and redirect all accesses to the static to this copy. This way the executable can avoid GOT indirection, which is a slight perf win. And yes, this means adding elements to a static array in a dylib (or otherwise changing the size) is an ABI breaking change on Linux.

View changes since the review

[RalfJung]

Uh... I understand like maybe half of those words. (Can I have some 🥧 please? :D )
"Copying" sounds wrong, statics are places and if you copy them, well, you have two copies so that can't be right?

But it sounds like you are saying linkme and inventory are unsound? IIRC they rely on extern statics that are bigger than declared, filled in by the linker.

[bjorn3]

Basically when linking the executable, space is reserved for each static referenced from a dylib inside the executable based on symbol size. Then at runtime the dynamic linker will copy the contents of the static in the dylib to the reserved space in the executable and redirect all references to this static to the version in the executable. So there are two copies in memory, but only the one inside the executable is observable by the user. The copy inside the dylib is unused and would be safe to unmap from memory after the dynamic linker has copied it.

linkme and inventory don't work across dylib boundaries anyway as they need the section they emit to be contiguous in memory. They would only see items defined inside the same dylib/executable as the one where the access code happened to be codegened.

[RalfJung]

Thanks!

And what does that mean for crates like linkme/inventory?

[bjorn3]

linkme and inventory only see items defined in the same dylib/executable as the access code is codegened. So if you depend on them to have a global view, you will get misbehavior.

[bjorn3]

Only ELF has copy relocations for PIE executables. The other object file formats don't implement this "optimization".

[RalfJung]

Ah, I see.

Not sure if we should make our UB-rules format-dependent, though.

[CAD97]

There are two cases here, IIUC:

• Rust is a dylib using the static. Access to the static goes through GOT lookup, independent of whether Rust is the definition site of the static or not. When linking to the dylib, the linker fixups must supply a memory region at least the declared size.
• Rust is the binary using a static from a dylib. Rust supplies the memory region for the static matching the declared size. At dynamic linking time (before main), the dynamic linker writes the initialization value to the provided memory region for the static.

The memory region of the static always matches the declaration exactly if Rustc is the one allocating it; it's only when Rust is linking to an external memory region that it may potentially be larger. The case where Rust allocates the static and dynamic linking initializes it could be declared morally equivalent to life-before-main w.r.t. the abstract machine, and thus the initialization going out of bounds of the static is an out-of-bounds access unsoundness. Notably, if the Rust code declares a larger static that doesn't require the entire static value to be initialized, it's perfectly fine for the linker to copy initialize a smaller value in that memory region.

In conclusion, I think ELF copy relocation doesn't preclude allowing Rust extern static from linking to a memory region that ends up being larger than declared. But it is another weird case of where the wrong signature in an unsafe extern block can lead to UB even if the symbols go unused by Rust. Which isn't pretty, but it is the reality of today's compiler/linker plumbing.

[RalfJung]

So... do you think the wording in the PR right now is accurate?
Note that this is about extern blocks. If you define a static in a Rust crate, compile that as a dylib, build a binary against that, and then load a different dylib into the binary, then there are all sorts of limitations that we have not even begun to document. That's not what I am trying to do here.

[bjorn3]

Maybe change r[items.extern.attributes.link.kind-raw-dylib.size] to clarify that the size needs to match exactly, but that is effectively already implied when combined with r[items.extern.static.size], so fine either way.

[joshlf]

What is the ground truth with respect to which "less mutable" is evaluated? I understand what "declaring an extern static with some... mutability" means, but what does the mutability of an "actual symbol" refer to?

View changes since the review

[RalfJung]

It refers to whether that symbol is in actually mutable memory. My understanding is that in ELF there are flags indicating which sections are mutable and which are not, and I presume other binary formats have something similar.

[CAD97]

Possible alternative wording instead of symbol: the linking-resolved memory region.

[RalfJung]

I went for "The actual memory that the extern static resolves to ", which is also what I used in the other file.

[RalfJung]

I guess this does make new commitments, we are officially allowing symbols-bigger-than-the-static-they-are-imported-with. So this should probably be FCP'd. I guess t-opsem is enough?

[CAD97]

We probably want to rope the linker-format experts in to the FCP as well at least, given potential limiting factors like ELF copy relocation could exist in other object formats. (Although I don't think ELF copy relocation precludes allowing this.)

[RalfJung]

I think this is ready to go now... what's the best way to make progress @traviscross ? Some sort of FCP for opsem or opsem+lang?

[traviscross]

I think this is ready to go now... what's the best way to make progress @traviscross ? Some sort of FCP for opsem or opsem+lang?

Sounds right. I'll check this over editorially, then propose dual FCP merge.

[joshtriplett]

@rfcbot concern document-pattern-for-section-iteration

(Filing this about the ongoing conversation iterating on how to correctly do section iteration, without requiring any form of asm. I expect to resolve it and check a box the moment we have a documented incantation for that.)

[RalfJung]

@digama0 @joshlf friendly opsem checkbox reminder :)

[joshtriplett]

Ongoing conversation in https://rust-lang.zulipchat.com/#narrow/channel/136281-t-opsem/topic/linker.20section.20slice.20iteration , and the general trend seems to be "yeah, this should work", with some thought going in to making sure that's the case.

[joshlf]

This example is pretty subtle. It'd be good to, before this example, precisely document the conditions under which a static can be used to access bytes. Currently the only prose that exists says when it can't be:

In particular, an extern static that marks the start of a linker section cannot be used to access statics with different symbols placed in that section.

View changes since the review

[joshtriplett]

We talked about this in today's lang meeting. I'm hoping that this is a relatively straightforward thing to confirm with LLVM. If it is indeed straightforward, I'm hoping we can document it rather than explicitly restricting it (rather than before, when we implicitly restricted it). If it turns out to not be straightforward, then given that people already do it anyway in C, and need to be able to interoperate with that pattern from Rust, then I can drop the concern and we can proceed with what we have. That said, it'd be helpful to have a non-normative note saying essentially "we know this pattern exists, this currently makes it UB but we're trying to document an interoperable way to interface with it that isn't UB", so that this doesn't get lost. (For calibration on importance, interoperating with this pattern will be essential for writing some types of Linux kernel code.)

[RalfJung]

I'm hoping that this is a relatively straightforward thing to confirm with LLVM.

I hope so too. I'm waiting for confirmation from @nikic.

[traviscross]

I'm hoping that this is a relatively straightforward thing to confirm with LLVM.

I hope so too. I'm waiting for confirmation from @nikic.

I wonder about other backends. E.g., GCC closed a similar issue, involving Linux kernel code, as INVALID:¹

And given gcc 7 is to be released yet, can we have a switch to disable this optimization?

This is nothing new in GCC 7, you've most likely just been extremely lucky in the past that it happened to work as you expected. Other projects had to change similar UB code years ago. It isn't just a single optimization, but lots of them that rely on pointer arithmetics being defined only within the same object.

...

In C

extern struct builtin_fw __start_builtin_fw[];
extern struct builtin_fw __end_builtin_fw[];

declares two external arrays, thus they are two independent objects. It is like if you have:

int a[10];
int b[10];

in your program, although they might be allocated adjacent, such that

int *p = &a[10]; int *q = &b[0]; memcmp (&p, &q, sizeof (p)) == 0;
&b[0] - &a[0]

is still UB. What you do with __start_*/__end_* symbols is nothing you can define in C, you need linker support or asm for that, and to use it without UB you also need to use an optimization barrier that has been suggested.

The kernel side discussion was here. Linus wrote:

This is the best I could come up with: assuming gcc is not allowed to reason about what's inside the asm(), this is the only way I could think of to lose the array information without incurring unnecessary overheads. It should also be relatively safe as there is no way to accidentally use the underlying arrays without explicitly declaring them.

Ugh. I worry about the other places where we do things like this, depending on the linker just assigning the addresses and us being able to compare them.

If there is a compiler option to disable this optimization, I would almost prefer that.. Because we really do have a whole slew of these things.

And:

You could define a wrapper if you like, something like

/*
 * Assign dst = src, but prevent the compiler from inferring anything
 * about the assigned value, so it can't do any unwanted optimization.
 */
#define blind_assign(dst,src) asm("" : "=X" (dst) : "0" (src))

We already have RELOC_HIDE() and OPTIMIZER_HIDE_VAR() that basically do this.

They both use r. I guess they could use X.

Nor does it seem that GCC's position has shifted here.² When the matter was raised on 2026-08-13, a GCC maintainer responded:

No this is NOT a false positive.

And:

The subtraction is well-defined in practice

NO It is not. See PR 77964 for why it shows up as being invalid thing to do.

GCC has an issue for documenting the correct way to iterate a "section array"; the proposal is to document the pattern of using inline assembly as an optimization barrier, as they suggested to the kernel developers.

cc @rust-lang/wg-gcc-backend

Footnotes

1. Note that the example of defined behavior in this PR is written carefully, and does not use offset_from, to avoid this same issue. ↩

2. Though that particular fold does not happen, currently, for reasons that are unrelated to section walking and are similar to the environ case I had mentioned. ↩

[traviscross]

cc rust-lang/rust#125119

    /// Allows registering static items globally, possibly across crates, to iterate over at runtime.
    (unstable, global_registration, "CURRENT_RUSTC_VERSION", Some(125119)),

[RalfJung]

IMO it is the responsibility of GCC to properly model Rust's semantics, not the other way around. This is assuming that the semantics work fine in LLVM, which demonstrates that the semantics can reasonably be implemented.

[traviscross]

IMO it is the responsibility of GCC to properly model Rust's semantics, not the other way around.

I don't disagree.

[RalfJung]

Looking at the issue, it seems like GCC folks claim that comparing unrelated pointers, such as pointers to two different statics, is UB in C. In think they are wrong even in C, but they are definitely wrong in Rust. This isn't just about statics, or overlapping statics. Pointer comparison is always well-defined in Rust and the GCC backend will have to find a way to deal with this.

(Subtracting unrelated pointers is indeed UB, and on the Rust side that would be offset_from which is quite clear about this.)

[Jules-Bertholet]

comparing unrelated pointers, such as pointers to two different statics, is UB in C.

IIUC, == and != between such pointers is not UB, but </<=/>/>= are. https://www.open-std.org/jtc1/sc22/wg14/www/docs/n3220.pdf#subsection.6.5.9

[RalfJung]

The issue is about == / !=, so the other operators are not relevant here.

[traviscross]

The guarantees we're making here support writing the decentralized registration pattern. I've pushed a guide-like example section that demonstrates how to soundly encode this, including the optimized form, with place-relative offsets, that's used by the Linux kernel. This more-optimized form saves up to 28 bytes per table entry and allows the pages backing the table to be marked read-only, which lets them be shared across processes.

Please have a look.

My feeling is that even if we were to make the additional guarantee that we're discussing, people would be better off not using it, since there's no way to use statics with unsafe(link_section = …) to write this better encoding. The assembler directives needed for this optimized form are shared across architectures, so there's no loss of portability on ELF targets. Regardless of how we write this, it relies on (documented) linker behavior, not something the language provides.

[RalfJung]

saves up to 28 bytes per table entry

A table entry is usually ptr-sized, i.e., 4/8 bytes. How can one save 28 bytes over that...?

[traviscross]

saves up to 28 bytes per table entry

A table entry is usually ptr-sized, i.e., 4/8 bytes. How can one save 28 bytes over that...?

See the text in the commit. The relocation record is up to 24 bytes.

[RalfJung]

Ah so this is about on-disk binary size, not runtime RAM usage?

TBH I feel like that example has way too much detail for a section of the Rust reference.^^ Going into so much detail distracts from the main content. But that's a separate question.

[traviscross]

Ah so this is about on-disk binary size, not runtime RAM usage?

There are three aspects of this form that save RAM:

• On 64-bit targets, there's a 2x savings from using 4-byte entries rather than pointer-sized entries.
• Because there are no writes to the table at load time, the pages stay file-backed and can be shared between processes; with pointer entries, the relocations make the pages of the table private to each process. This is particularly relevant for shared libraries.
• (Minor:) The on-disk representation (where we save up to 24 bytes) will sit in the page cache.

TBH I feel like that example has way too much detail for a section of the Rust reference.^^ Going into so much detail distracts from the main content. But that's a separate question.

Yes, there's some tension there. I have ideas about how to better organize or present this long term. I do think this sort of material adds value to the Reference.

(Also, even today, looking at the rendered form rather than the Markdown may affect one's sense of this: it's set off visually from the main body of the section in a kind of admonition designed for these asides.)

[traviscross]

As an update, it would surprise me if we wanted to make the additional guarantee today, given that it would make the following program nominally sound, but we would then miscompile it:

use core::sync::atomic::{AtomicBool, Ordering::Relaxed};

#[used] #[unsafe(link_section = "tbl")]
static E1: u32 = 1;
#[used] #[unsafe(link_section = "tbl")]
static E2: u32 = 2;
unsafe extern "C" {
    static __start_tbl: [u32; 0];
    static __stop_tbl: [u32; 0];
}

static TAKEN: AtomicBool = AtomicBool::new(false);

#[inline(never)]
fn f(p: *const u32) -> u32 {
    if p == &raw const E1 {
        TAKEN.fetch_or(true, Relaxed);
        let a = unsafe { *p }; //~ UB today and under the PR.
        let b = unsafe { *p.add(1) }; //~ UB today and under the PR.
        a + b
    } else { 0 }
}

fn main() {
    let p = core::hint::black_box(&raw const __start_tbl as *const u32);
    let p_end = core::hint::black_box(&raw const __stop_tbl as *const u32);
    assert!(p.wrapping_add(2) == p_end); //~ Proves the table has two entries.
    let v = f(p);
    assert!(TAKEN.load(Relaxed)); //~ Proves the branch was taken.
    assert_eq!(v, 0); //~ Above `-O0`, this doesn't panic.
    //            ^
    // Since the branch was taken, there's no logical way that this
    // value could be zero.
}

Playground link

(Thanks to @RalfJung for cleaning up my earlier version of this example.)

@RalfJung noted the relationship of this to llvm/llvm-project#220206 and that we have consequent existing miscompilations. The most similar one I could come up with is:

use core::sync::atomic::{AtomicU8, Ordering::Relaxed};

static A: [u32; 1] = [1];
static B: [u32; 1] = [2];
static TAKEN: AtomicU8 = AtomicU8::new(0);

#[inline(never)]
fn last_if_ends_at_a(s: &[u32]) -> u32 {
    if s.as_ptr_range().end == A.as_ptr() {
        TAKEN.fetch_or(1, Relaxed);
        *s.iter().next_back().unwrap()
    } else { 0 }
}

#[inline(never)]
fn last_if_ends_at_b(s: &[u32]) -> u32 {
    if s.as_ptr_range().end == B.as_ptr() {
        TAKEN.fetch_or(2, Relaxed);
        *s.iter().next_back().unwrap()
    } else { 0 }
}

fn main() {
    let vb = last_if_ends_at_a(core::hint::black_box(&B));
    let va = last_if_ends_at_b(core::hint::black_box(&A));
    let taken = TAKEN.load(Relaxed);
    assert!(taken != 0, "the statics are not adjacent in this build");
    if taken & 1 != 0 { assert_ne!(vb, 2); } //~ OK above `-O0`.
    if taken & 2 != 0 { assert_ne!(va, 1); } //~ OK above `-O0`.
    //                                 ^
    // The relevant branch was taken, so there's no logical way for
    // these inequalities to hold.
}

Playground link

(Let me know if there's a better example.)

Comparing these, the first still seems more concerning to me for these reasons:

One, the second example is an existing bug; the first example isn't a bug today. We'd make it into a bug with the additional guarantee. It seems generally worth avoiding making new guarantees that would create new known miscompilations in nominally sound code.

Two, in the second example, the equal addresses are an accident of layout. There's not a particularly good reason for someone to compare these. When they are compared, the wrong read then goes backward from an unrelated static. In the first example, conversely, the equal addresses are part of the design, and it doesn't seem ridiculous that someone would compare them in this kind of way. Here, the mistaken read is in the forward walk.

Three, speculatively, the two might not share a fix. Setting aside unnamed_addr merging, when a pointer equals the address of a global and is known to be dereferenceable there, does anything else today prevent us from assuming that it is the global? The additional guarantee would violate this, though. Particularly if the optimization relying on this is important (vtables were mentioned), I could imagine this needing more thought.

---

It remains my sense, therefore, that making the proposed additional guarantee is nontrivial. Meanwhile, the decentralized registration pattern cannot be written soundly in Rust today. But with the other guarantees we're making in this PR, it can be written soundly and in a portable way, across ELF targets.

This PR now includes full documentation on how to write this pattern correctly. The documented optimized form, in use by the Linux kernel, is more efficient than, and arguably superior overall to, what can be achieved by leaning on the additional guarantee (i.e., using statics with unsafe(link_section = …) to add entries).

[joshtriplett]

Based on the new documentation showing two non-UB ways to iterate over function pointers in a section, and the comment in the lang meeting that this interoperates with how the Linux kernel does it (so C and Rust code can interoperate using this same mechanism), this seems fine.

@rfcbot resolve document-pattern-for-section-iteration
@rfcbot reviewed

[rust-rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔