Latency numbers worth memorising

Back to list2026-07-08
You

Rough numbers for what is fast and what is not?

ChatGPT

The orders of magnitude, not the digits:

OperationTime
L1 cache reference1 ns
Main memory reference100 ns
SSD random read100 us
Round trip in a datacenter500 us
Round trip across an ocean150 ms

The useful consequence: one network call costs as much as a million memory accesses, so batching calls beats optimising the code between them.

You

We added a cache and p99 got worse.

ChatGPT

Classic signature of a cache that misses under load: every miss now pays the lookup plus the original work, and the population step often serialises on a lock. Measure hit rate before and after, and add jitter to expiry so entries do not all die in the same second.