Retries that make an outage worse

Back to list2023-09-05
You

Our retries turned a blip into a full outage.

ChatGPT

Synchronised retries are an attack on your own service. Exponential backoff with jitter and a cap, plus a circuit breaker that stops trying entirely once failures dominate. Without jitter every client retries at the same instant, which is precisely the thundering herd.

You

How do I know when it stops being worth it?

ChatGPT

When you stop noticing it. If it has not surfaced a problem in six months, it is either solved or it was never the bottleneck you thought.