There are so many things we approximate to make life simple. Wires, for example, have no resistance or other strange effects. Crystal oscillators output their exact frequency. But surely our model of how a computer stores and loads memory is accurate, right? You put data in a particular location and, later, you take it out. The [FEX-Emu] developers have a different perspective. Once you have caches and, perhaps, multiple CPUs, it isn’t that easy.
The basic problem is this: if one CPU (or, more accurately, bus master) writes to a location, will another CPU have access to the new value? X86’s Total Store Ordering model gives programmers strong guarantees about when loads and stores become visible, while ARM deliberately uses a weaker memory model that permits considerably more reordering for performance and efficiency.
An emulator can, in theory, compensate by translating ordinary x86 memory operations into ARM acquire/release operations, but doing that for nearly every memory reference can be expensive. Newer ARM extensions such as LRCPC help considerably, while Apple took a more direct approach by adding an x86-compatible TSO mode to Apple Silicon. That lets ordinary loads and stores behave the way translated x86 code expects with comparatively little overhead.
Things get much uglier with unaligned accesses and atomic operations. X86 software routinely performs accesses that ARM would consider badly aligned, and x86 provides surprisingly strong atomicity guarantees within a cache line. FEX sometimes has to catch alignment faults and dynamically patch translated code with barriers. Split-lock operations are worse still: some require excursions through the kernel and signal handlers and can be hundreds or thousands of times slower than the normal case. Qualcomm’s newer Oryon cores improve matters by supporting coherent cache-line atomics, while Valve has shipped a Linux kernel optimization that handles some troublesome unaligned atomics directly.
There’s another particularly nasty corner involving write-combined GPU memory. PC games frequently expect x86 ordering semantics while writing uncached buffers destined for a discrete GPU. ARM currently lacks a clean equivalent for some of these stores, and FEX measured worst-case bandwidth more than 800× slower, enough to reduce some games to below 1 FPS. UMA systems fare much better because drivers can often substitute ordinary cache-coherent memory.
It’s a long article, but a good illustration of why modern emulators are less about translating instructions and more about reproducing decades of architectural assumptions that software quietly depends upon. Of course, not all emulators or processor recreations are this accurate, and often that’s good enough. But sometimes you need a recreation that is truly cycle-accurate and behaves exactly like the original.
as into the weeds as this summary is, i can’t even imagine what i will see if i click on the hyperlink with the ominous text of “isn’t that easy”
Yeah, this article lacks about two levels of introductions. It helps to know that it is about FEX, which is an software emulation layer to run x86 code on ARM.
It is mainly developed by Valve to run Steam and all its games on ARM machines, and it’s kinda shocking how fast it goes for how different the architectures are. (10-20% performance penalty)
But it’s good to have in general, outside of gaming, as it might actually put a dent in the x86 monopoly.
Hey Al, it was a pleasure to read such well prepared and carefully-worded article.
The issue is that Intel (and AMD) never officially explicitly documented the actual x86 memory model. Which doesn’t mean that the CPUs themselves aren’t working properly, but the actual contract (ISA & ABI specs) with system and application developers is unclear and and not fully defined, especially in corner cases.
So, TSO is an memory model that’s expensive (in terms of logic gates and power) to implement, but Intel & AMD have the cash to throw at it. TSO is a nice assumption that started to became broken for the last 20 years with the mass adoption of multicore super-scalar out-of-order cpus – if you want to have high performance, you need multi-level memory hierarchy, and TSO sucks big time with it. Intel and AMD can afford to pay tons of money to manage the inevitable short-comings of TSO, but ARM and others RISC-based design teams decided to work smarter and to define memory models which are much more scalable for multi-level memory hierarchies – aka WMO (weak memory ordering). From practical perspective, it’s all the same for the end user – the software must work correctly all the time. The difference is that the contract with the system and app developers is cleaner, explicit and it demands explicit management of which data is core-local and which must be globally visible (and when).
Apple got inventive and developed interesting solution trying to get advantages of both worlds, as using WMO as default memory model and allowing specific processes to use hardware-assisted TSO (which usually means Rosetta2-emulated apps). I think that as IA32 became legacy when x86-64 arrived (it just sucks less), I hope to see WMO more and more widespread as time and platforms continue to develop.
Regards!
There are some serious downsides to the x86 approach, of course, in that (within each CPU) meeting ordering requirements via the store buffer costs power and die area and can hobble any code with a non-sequential access pattern from leveraging the full superscalar capabilities of the core on account of things like false 4K aliasing. Between multiple cores it (again) consumes die area, power, and latency syncing between the caches.
It’s a bummer if you’re writing code with a well defined access pattern such that you know there is no need for the ordering constraints because you’ve taken care of that via data structure design and modulo scheduling loops, but end up store bound anyway because the chip is assuming worst case when you’ve designed your code never to generate that worst case.
I’ve often wished there was a way (analogous to how you can set NaN handling modes for the FPU) to opt out of the stalls it otherwise inserts to ensure ordering if your application doesn’t need it =:-/