Rough numbers for what is fast and what is not?
The orders of magnitude, not the digits:
| Operation | Time |
|---|---|
| L1 cache reference | 1 ns |
| Main memory reference | 100 ns |
| SSD random read | 100 us |
| Round trip in a datacenter | 500 us |
| Round trip across an ocean | 150 ms |
The useful consequence: one network call costs as much as a million memory accesses, so batching calls beats optimising the code between them.
We added a cache and p99 got worse.
Classic signature of a cache that misses under load: every miss now pays the lookup plus the original work, and the population step often serialises on a lock. Measure hit rate before and after, and add jitter to expiry so entries do not all die in the same second.
Where do I start when a script is slow and I have no idea why?
Profile before guessing. For Python:
python -m cProfile -s cumtime script.py 2>&1 | head -25
Read the cumulative column first, not the per-call one. Nine times out of ten the top entry is a function nobody suspected, and the one you were about to optimise is at 2%.
Any downside worth knowing before I commit?
It commits you to a format that is tedious to migrate away from later. The first weeks also look worse than doing nothing, which is when most people abandon it.