Head sampling drops exactly the requests I want to see

Tail sampling: buffer the spans and decide after the outcome is known, keeping everything that errored or exceeded a latency threshold plus a small random slice of the rest. Costs more memory at the collector and is almost always worth it.

Logs with structure, because they answer "what happened" for incidents you did not anticipate. Metrics come next for the handful of numbers you would page on. Traces are worth it once a request crosses more than two services.