memory clobber, barriers, fences #697

In 0.7.8 and 0.7.9 many of the asm!() calls that we expose as public API are marked as non memory clobbering via nomem (since #372). That allows the compiler to move these accesses around and especially through other memory accesses.

At face value, e.g. a cpsid, cpsie, or basepri writes indeed do not touch any memory or perform inter-thread synchronization and nomem appears completely adequate (see the language reference).
But in practice leaving it at that appears highly dangerous and a massive footgun. In all the cases that I can come up with these writes specifically want to disable/enable access to memory before/after some non-atomic op is performed on that memory. In these cases the user currently has to manually prevent the compiler from moving the write into or even across the non-atomic op. Not doing that demonstrably leads to issues like in rtic.

That burden alone is arguably already a bad idea and e.g. CMSIS tries very hard to not leave it to the user. But it gets worse: the right tool for that manual optimization protection isn't readily identified. Some other asm!() writes have compiler fences of varying shape (mostly SeqCst). Yet others are marked as memory clobbering (#526/#502 removed nomem again from isb, dsb, dmb). The third option would be to place traditional "barriers", in the form of empty memory clobbers asm!("", options(nostack, preserves_flags)) (e.g. like linux arm64 or interestingly also linux rust).

We should remove nomem from several asm!() calls for the following reasons:

• Mask instructions do not themselves access memory, but the context predominantly needs to prevent memory accesses from moving across the masking/unmasking op
• control::write changes processor state and does ISB. Surrounding memory accesses must execute in the intended privilege context
• The code entered by bootstrap may read the initialized stack
• During the semihosting bkpt the host may read or write memory through the argument pointer
• Coprocessor transfers access state that may also be accessible through memory loads and stores (see e.g. Cortex-M33 TRM B6.6)

Downstream crates that want to be compatible with cortex-m 0.7.[89] should consider a hotfix with a asm!("", options(nostack, preserves_flags)) barrier pair.

A compiler fence is strictly speaking not the right tool here. It only "establishes synchronization with other atomic operations and fences". And neither the ops above nor the memory acess that we want to protect are atomic operations. That's the on-paper pessimistic view. In practice LLVM is more defensive and "AA (Alias Analysis) will return ModRef (potentially touching memory) for anything Acquire or Release". That's then surfaced for most outcomes of getSyncEffects() and typically leads to a memory barrier. But we should not rely on it. For more detail on this see rust-lang/unsafe-code-guidelines#347.

But then again the memory clobber barrier (be it !nomem on the workhorse asm!() or a separate empty asm barrier) alone is currently also demonstrably imperfect at least in some situations due to a compiler limitation. I tried this with a BASEPRI example that publishes a pointer after leaving the critical section, similar to that issue. With memory clobbers alone, the compiler still elides initialization.

Therefore we should also add fences to work around the compiler bug:

• Add Release before BASEPRI writes and Acquire afterwards
• BASEPRI_MAX cannot weaken the mask and only needs Acquire afterwards

Comments welcome.