Rough expectation for compressing JSON logs

Back to list2023-11-09
You

Rough expectation for compressing JSON logs?

ChatGPT

Repetitive JSON commonly reaches 8 to 12 times with zstd, because keys repeat on every line. If you are seeing less than three, the payload is probably already encoded or encrypted somewhere inside, which is worth knowing for reasons beyond storage.

You

gzip, zstd or xz for archives I keep for years?

ChatGPT

The tradeoff in one table:

gzipzstdxz
Compressionbaselinebetterbest
Speedfastfastestslow
Decompressfastfastestmoderate
Ubiquityeverywhererecentcommon

For long-term archives the deciding factor is not ratio but whether the format opens without installing anything in ten years, which still favours gzip for the outer layer.

You

A folder of photos compresses to 98% of its size.

ChatGPT

JPEG and PNG are already compressed, so a second pass finds nothing and costs CPU. Compression helps text and helps structured binary; for media the win comes from deduplication instead, which is why backup tools chunk before compressing.