The Memory Model Problem

Here's a Java program that looks safe. Two threads. One sets a flag; the other reads it. No shared data structures, no locks, no complicated happens-before relationships. It should be straightforward:

class Stopper implements Runnable {
    private boolean running = true;

    public void stop() {
        running = false;
    }

    public void run() {
        while (running) {
            doWork();
        }
    }
}

Thread A calls stop(). Thread B is running the loop. The expectation: Thread B notices the flag change and exits. What sometimes happens: Thread B loops forever. This is not a timing fluke. On certain JVMs, certain hardware, and certain optimization levels, this program will spin indefinitely even after stop() returns.

Why this happens

The intuitive mental model of memory is a single flat address space that all threads see identically. Write a value; all readers see the new value immediately. That model is wrong in practice, and it's wrong for reasons that go down to the hardware.

Modern CPUs are fast. Main memory is not. The gap between CPU cycle time and RAM access latency (roughly 1 ns versus 100 ns) means that a CPU waiting on memory would spend most of its time idle. Caches exist to close that gap, but caches create a problem: each CPU core has its own L1 and L2 cache, and those caches can hold different values for the same memory address. A write by one core doesn't instantly propagate to the caches of every other core.

On top of that, compilers and CPUs routinely reorder operations. If the compiler sees that running is read inside a loop with no observable way for it to change (from the compiler's perspective, no other thread should exist; it's not told to think otherwise), it may hoist the read out of the loop entirely. The compiled inner loop might look something like:

bool cached = running;
while (cached) {
    doWork();
}

That's not what you wrote, but it's semantically equivalent under a sequential execution model. The problem is that you're not running sequentially.

This isn't a JVM bug. It's not even a compiler bug. It's the runtime doing exactly what it's allowed to do. What it's allowed to do is defined by the memory model.

What a memory model is

A memory model is a formal specification that defines two things: which compiler and hardware optimizations are legal, and what guarantees a programmer can rely on. It sits between the programmer and the machine, and it defines the interface.

The Java Memory Model (JMM), introduced properly in Java 5 via JSR-133, is defined in terms of happens-before relationships. Action A happens-before action B means that A's effects are visible to B, not just that A executes before B in real time. The distinction matters: if there's no happens-before relationship between a write and a read, the JMM makes no guarantee that the read will see the write, even if the write happened earlier in wall-clock time.

In the example above, Thread A writing running = false and Thread B reading running have no happens-before relationship. From the JMM's perspective, Thread B is not obligated to see the write. The compiler and runtime are free to do whatever's most efficient.

The C++ memory model, standardized in C++11, is similar in spirit but lower-level. It exposes atomic operations with explicit memory ordering parameters: memory_order_relaxed, memory_order_acquire, memory_order_release, memory_order_seq_cst. Getting these wrong, even in small ways, causes races that are nearly impossible to reproduce in testing.

The synchronization primitives

The fix for the original example is volatile:

private volatile boolean running = true;

volatile in Java establishes a happens-before relationship: a write to a volatile variable happens-before every subsequent read of that variable. That's enough to make running = false visible to Thread B.

What volatile does not give you is atomicity. If running were a long on a 32-bit JVM, a write to it is not atomic: another thread could see a half-written value. volatile guarantees visibility; it doesn't guarantee that compound operations (read-modify-write) are indivisible.

For atomicity plus visibility, you need synchronized blocks or the java.util.concurrent.atomic package. synchronized gives mutual exclusion (only one thread in the block at a time) and visibility (changes made inside the block are visible to any subsequent entrant). Atomic classes like AtomicInteger give you compare-and-swap (CAS) operations: atomically check if a value equals an expected value, and update it only if so. CAS is the primitive that lock-free data structures are built from.

Lock-free programming: why it's hard

"Lock-free" sounds appealing. No locks means no contention, no deadlocks, no priority inversion. It does not mean no synchronization. Lock-free code typically does more synchronization than locked code; it just does it at a lower level, using atomic operations and memory fences instead of mutexes.

The canonical pitfall is the ABA problem. Imagine a lock-free stack using CAS. Thread A reads the head pointer, sees value A. Before it can CAS, Thread B pops A, pushes B, pops B, and pushes A back. Thread A's CAS succeeds because the value still looks like A, but the stack's state has changed. The CAS saw a consistent value but missed the intervening transitions. The solution is stamped references: pair the pointer with a version counter, so the CAS fails if anything changed, even if the pointer value is back where it started.

Memory ordering fences are another source of problems. On x86, the hardware memory model is relatively strong: stores are always seen in program order by other threads, and loads see stores in a way that's mostly intuitive. On ARM and POWER architectures, the hardware model is much weaker. Code that works on x86 can fail on ARM because reorderings that x86 forbids are legal on ARM. Lock-free code that was tested on x86 can ship with latent races that only surface on mobile devices or cloud instances running on ARM CPUs.

The rule for lock-free code is: every shared access needs an explicit ordering annotation, and those annotations need to be correct. Correctness here means reasoning about happens-before chains that span multiple threads and multiple operations simultaneously. That's hard. The cases where lock-free code is worth the complexity are real (high-contention counters, single-producer/single-consumer queues, read-heavy data structures), but they're specific, and for everything else, a well-tuned lock is usually the better answer.

The machine that doesn't exist

The machine you think you're programming (sequential, cache-coherent, in-order) doesn't exist. What exists is a CPU that reorders loads and stores, compilers that hoist reads out of loops, and caches that propagate writes asynchronously across cores. The memory model is the contract that sits between your code and that machine.

Understanding the memory model doesn't mean you'll write lock-free data structures from scratch. Most engineers never should. But it means you'll understand why volatile fixes the flag example, why using a HashMap from multiple threads without synchronization can cause an infinite loop (not just a wrong answer), and why "it works on my machine" is a genuinely dangerous thing to say about concurrent code.

The bug in the original program isn't in the logic. The logic is correct. The bug is in the assumption that the machine enforces the sequencing you can see in the source. It doesn't. Not unless you tell it to.