Understanding the Itanium Memory Model and C++ Atomics

The C++11 memory model was designed to be a hardware-independent abstraction, but it was heavily influenced by real-world CPU architectures. One of the most fascinating historical anomalies in this space is the Intel Itanium (IA-64) processor family. Up until the C++20 revision (via proposal P0668R5), the standard had specific, complex rules to accommodate Itanium's highly relaxed memory ordering.

Specifically, many developers wonder: Why couldn't Itanium implement memory_order_relaxed using plain, unadorned loads and stores? And did the Itanium mf (memory fence) instruction only provide local thread ordering?

In this article, we'll dive deep into the microarchitectural quirks of Itanium, explain Multi-Copy Atomicity (MCA), and dissect the exact code paths that forced compilers to use stronger instructions for relaxed atomics.

The Core Issue: Non-Multi-Copy Atomicity (Non-MCA)

To understand Itanium, we must first understand Multi-Copy Atomicity (MCA). In an MCA architecture (like x86 or ARMv8), once a store instruction becomes visible to any other core in the system, it simultaneously becomes visible to all cores. There is a single, globally agreed-upon moment when a write "happens."

Itanium is non-MCA for plain loads (ld) and stores (st). On Itanium, a plain store can become visible to a neighboring core (for example, via local cache-forwarding or a shared cluster bus) before it propagates to more distant cores in the system. This lack of global write atomicity creates bizarre causal violations that C++'s memory model cannot allow.

The WRC (Write-to-Read Causality) Violation

Consider the classic Write-to-Read Causality (WRC) scenario, which motivated the C++11 mapping fix for Itanium:

// Thread 1
r3 = y.load(memory_order_acquire); // Sees 1
r1 = x.load(memory_order_relaxed); // Sees 0

// Thread 2
x.store(1, memory_order_relaxed);

// Thread 3
r2 = x.load(memory_order_relaxed); // Sees 1
y.store(1, memory_order_release);

In the C++ memory model, the outcome r2 = 1, r3 = 1, and r1 = 0 is strictly forbidden. Why? Because:

  • Thread 3 reads x = 1, then stores y = 1 with release semantics.
  • Thread 1 reads y = 1 with acquire semantics, establishing a "synchronizes-with" relationship. This means Thread 1 must see everything that happened before Thread 3's store, which includes the write to x.
  • Therefore, Thread 1's load of x must return 1. Returning 0 violates basic causality.

If Itanium compiled memory_order_relaxed to plain ld and st instructions, this forbidden outcome could easily happen. Thread 2's plain store to x might become visible to Thread 3 early, but because Itanium lacks MCA for plain stores, that write might not propagate to Thread 1 until much later. Thus, Thread 1 would see y = 1 but still read the old value x = 0.

The Fix: Why Relaxed Atomics Used Acquire/Release on Itanium

To prevent these causality violations, compiler implementers realized they could not map memory_order_relaxed to plain ld and st. Instead, the official C++11 mapping recommended:

  • Relaxed Loads: Compiled to ld.acq (Load-Acquire)
  • Relaxed Stores: Compiled to st.rel (Store-Release)

According to the Intel Itanium Architecture Software Developer's Manual, store-releases (st.rel) to writeback memory possess a special property: remote write atomicity. When a store-release is executed, it is guaranteed to become visible to all processors in the same order. By upgrading relaxed stores to st.rel, Itanium behaves like a multi-copy atomic machine for these operations, preserving the expected causal chains of C++11.

Did the mf Fence Only Provide Local Ordering?

Yes. The Itanium memory fence instruction, mf, is fundamentally a local-only ordering barrier. It ensures that all memory accesses initiated by the local processor before the mf are made visible before any subsequent accesses initiated by the same processor.

However, mf does not provide cumulativity (transitivity across multiple threads). It cannot force a write from another processor to globally propagate to all other nodes. Because mf lacks this global synchronizing capability, pairing plain loads and stores with seq_cst fences was insufficient to guarantee sequential consistency. This is why the C++20 committee ultimately decided to revise the memory model in P0668R5, acknowledging that on architectures like Itanium, relaxed operations must be promoted to stronger primitives anyway, making the theoretical distinction between seq_cst fences and seq_cst operations moot in practice.

Summary

Itanium's unique, highly relaxed memory architecture forced C++ compiler writers to make a compromise. Because plain loads and stores lacked Multi-Copy Atomicity, mapping memory_order_relaxed directly to them would break the causal guarantees of the C++ memory model. By using ld.acq and st.rel for relaxed accesses, and acknowledging that the local-only mf fence couldn't patch plain accesses into sequentially consistent ones, the industry moved toward the cleaner, more robust memory model definitions we enjoy in C++20 today.