Small service, limited time. Which one first

Back to list2023-12-06
You

Small service, limited time. Which one first?

ChatGPT

Logs with structure, because they answer "what happened" for incidents you did not anticipate. Metrics come next for the handful of numbers you would page on. Traces are worth it once a request crosses more than two services.

You
architecture.png
You

Our alerts fire constantly and everyone mutes them.

ChatGPT

Alert on symptoms users feel, not on causes. Disk at 85% is a cause and often harmless; requests failing is a symptom and always matters. Every alert that cannot be acted on immediately should be a dashboard instead.

You

Our metrics cost exploded after adding a user id label.

ChatGPT

That is the mechanism: every distinct label value creates a series. User ids, request ids and full URLs never belong in labels. Put them in logs and traces, where high cardinality is the point rather than the problem.