There's a table of latency numbers that has circulated the internet for about fifteen years. It was originally compiled by Jeff Dean and Peter Norvig at Google around 2010, updated periodically since, and by now it's been turned into posters, coffee mugs, and slide decks at companies that would benefit far more from actually reasoning with it.
The numbers themselves aren't the interesting part. The interesting part is what they imply about the decisions you make while designing systems: which cache to use, whether to batch a database call, whether to put two services in the same datacenter. Most engineers who can recite the numbers couldn't tell you how they inform any of those choices.
The numbers
The approximate 2024 figures, rounded to useful magnitudes:
- L1 cache access: ~1 ns
- L2 cache access: ~4 ns
- L3 cache access: ~10–40 ns (depends heavily on cache size and NUMA topology)
- Main memory (RAM): ~100 ns
- NVMe SSD random read: ~100 µs (100,000 ns)
- Network round-trip within the same datacenter: ~500 µs
- Network round-trip cross-region (US East to US West): ~50 ms (50,000 µs)
- Traditional spinning-disk seek: ~10 ms, though HDDs have mostly left the hot-path in modern systems
The units shift three times as you scan that list: nanoseconds, then microseconds, then milliseconds. Each shift is a factor of 1,000. By the time you reach a cross-region network call, you are seven orders of magnitude away from an L1 cache hit.
The exact figures are less important than the orders of magnitude. L1 cache is roughly 1 ns. RAM is roughly 100 ns. An NVMe SSD is roughly 100 µs. A same-DC network hop is roughly 500 µs. If you know those four numbers, you have the skeleton of the whole table.
The ratios matter more than the absolutes
An L1 cache hit takes about 1 ns. A RAM access takes about 100 ns. That's a 100x difference. The physical reason: L1 cache sits a few millimeters from the CPU core. RAM sits on a separate chip across the memory bus. Even at the speed of light, that distance costs time, and the actual cost includes bus arbitration, DRAM row activation, and other overhead that makes "100x slower than L1" a conservative estimate for some workloads.
A same-datacenter network round-trip at 500 µs is about 5,000x slower than a RAM access. The reasons compound: you leave the CPU, traverse the kernel network stack, cross a NIC, travel through switches, hit another NIC, traverse another kernel stack, and return. Every one of those transitions adds latency. A cross-region call adds another 100x on top, mostly because the speed of light across a continent is not negotiable.
These ratios are the point. When you're deciding whether to add an in-process cache for something currently computed from RAM, the question becomes: is RAM access so cheap that the cache isn't worth the complexity? At 100 ns, if you can compute the value in under a microsecond, probably yes. When you're deciding whether to cache a result that currently requires a network call, the break-even is completely different: a cache hit is in RAM (100 ns), a miss is a network call (500 µs), so the cache is worth it if any non-trivial fraction of requests are repeated.
Applying them to real decisions
"Should we cache this in Redis?" is a question about whether the cost of a Redis round-trip (a network call, roughly 500 µs in the same DC) is cheaper than the cost of recomputing or re-fetching the value. If the value comes from a database query that itself requires a network call, you're paying 500 µs either way, so caching only wins if it has a high hit rate. If the value is computed in memory, caching is almost certainly wrong. You're paying 500 µs to avoid paying less than 1 µs.
"Should we batch these database writes?" asks how cost scales with the number of writes. Each individual write costs a network round-trip (~500 µs) plus whatever the database charges on top. If you can batch 100 writes into one network call, you save 99 round-trips. At 500 µs each, that's 49.5 ms saved per batch cycle. Whether that matters depends on your throughput, but the arithmetic is there to check.
"Should we put these two services in the same region?" is a question about the 100x difference between a same-DC call (500 µs) and a cross-region call (50 ms). If the two services call each other frequently and those calls are on the critical path, the answer is usually yes. Unless there's a strong reason for geo-distribution, the latency cost is hard to justify.
None of these decisions are made purely by the numbers, but the numbers narrow the space of reasonable positions considerably. When someone argues for an architecture that requires ten cross-region calls on the critical path, the latency table alone can close that argument. Ten cross-region calls at 50 ms each is 500 ms minimum, before you've done a single byte of useful computation. That's the kind of thing that should be obvious in design review but often isn't, because people haven't internalized what 50 ms actually means relative to everything else.
The hidden one: tail latency
None of the numbers above are about tail latency, and they should be. p99 (the latency at the 99th percentile, meaning the worst request out of 100) is often 10x the median (p50). Sometimes more. This isn't random noise; it has causes: lock contention under load, JVM garbage collection pauses, OS scheduling jitter when a thread is descheduled at an inconvenient moment, connection pool exhaustion, disk flush stalls.
The critical insight about tail latency isn't that individual services are slow on p99. It's what happens when you compose them. If your service makes five independent downstream calls and each is fast on average but has a slow p99, your combined latency is dominated by the slowest of the five. Concretely: if each service has a p99 of 10 ms, the probability that all five complete within 10 ms is 0.99 raised to the fifth power, which is about 0.95. That means your combined p95 is the upstream p99. Your p99 will be worse than the upstream p99. You've amplified tail latency by fanning out.
This matters a lot in microservices architectures, where a single user-facing request may fan out to a dozen upstream services. Each service looks fine in isolation. The aggregate looks like it's on fire. Hedge requests (sending a duplicate request after a timeout and taking the first response) can help for read-only calls, but they add load, which can worsen the tail latency of services that are already struggling.
There's no simple fix for tail latency amplification. The numbers just tell you it's inevitable when you fan out, so you should design with that in mind: minimize the fan-out on critical paths, prefer parallel over serial calls, and set realistic p99 budgets that account for the math.
Slow compared to what?
The best thing these numbers give you isn't a lookup table. It's a calibrated sense of magnitude that makes the first question "slow compared to what?" instead of just "slow."
A function that takes 10 µs is fast in absolute terms. It's also 100x slower than a RAM access and 10x slower than an NVMe read. Whether that matters depends on how often it runs and what it's competing against. A function that takes 500 µs is slow for an in-process computation and fast for a database call.
When you've internalized these numbers, not just memorized them but built intuitions around them, you stop needing to benchmark obvious cases. You know a cross-region call on a hot path is wrong before you measure it. You know an in-process cache for a nanosecond-range operation is not worth the complexity before you prototype it. You reserve measurement for the cases that actually aren't obvious, which is where measurement does the most good.
Back to writing