You
Rough numbers for what is fast and what is not?
ChatGPT
The orders of magnitude, not the digits:
| Operation | Time |
|---|---|
| L1 cache reference | 1 ns |
| Main memory reference | 100 ns |
| SSD random read | 100 us |
| Round trip in a datacenter | 500 us |
| Round trip across an ocean | 150 ms |
The useful consequence: one network call costs as much as a million memory accesses, so batching calls beats optimising the code between them.
You
We added a cache and p99 got worse.
ChatGPT
Classic signature of a cache that misses under load: every miss now pays the lookup plus the original work, and the population step often serialises on a lock. Measure hit rate before and after, and add jitter to expiry so entries do not all die in the same second.