Compression is one of the largest levers in analytical storage. Because columnar warehouses store each column separately, and a column's values are far more similar to each other than the values across a row, they compress dramatically - often 5-10× - which cuts storage cost and, more importantly, reduces the bytes a query has to read from disk. In scan-bound analytical work, less data read translates almost directly into faster queries.

Lightweight encodings

The first layer is value-aware encoding applied per column before any general-purpose codec runs:

• Run-length encoding (RLE): stores a repeated value once with a count. A sorted, low-cardinality column (status, country) collapses to almost nothing.
• Dictionary encoding: replaces repeated strings with small integer codes into a dictionary - ideal for low-to-medium cardinality text, and it lets filters compare integers instead of strings.
• Delta encoding: stores differences between consecutive values. Sorted timestamps or monotonic IDs become tiny deltas.
• Bit-packing / frame-of-reference: uses only as many bits as the value range needs rather than a full 32/64-bit word.

These work because columnar layout groups like with like; the same encodings would be far less effective row by row.

General-purpose codecs

On top of the encodings, a byte-level compressor squeezes the rest. LZ4 is fast to decompress with moderate ratios - a good default for hot data where CPU time on read matters. ZSTD gives better ratios at tunable levels for colder data. Gzip/zlib compresses well but decompresses slowly, so it's fading from query-hot paths. The codec is usually configurable per column or per table, so you can match it to how often the data is read.

The CPU/IO trade-off

Compression trades CPU for IO: tighter compression means fewer bytes to read but more cycles to decompress. The right point depends on the bottleneck - on object storage or slow disks, IO dominates and aggressive compression wins; on already-cached hot data, lighter/faster codecs avoid making CPU the new limit. Sorting data before compressing it amplifies every encoding above, since RLE and delta both reward ordered, clustered values - which is why compression and partitioning and sort order are tuned together.

Frequently asked questions

Why do columnar stores compress so well? Values within a single column are highly similar (same type, often repeated or sorted), so per-column encodings like RLE, dictionary, and delta find far more redundancy than row-wise storage would.

LZ4 vs. ZSTD - which should I use? LZ4 for hot, frequently-read data where fast decompression matters; ZSTD (at a tuned level) for colder data where a better ratio saves more storage and IO than the extra CPU costs.

Does sorting data help compression? Yes, substantially. Sorting clusters similar values, making run-length and delta encoding far more effective - often the single biggest win.

Can compression slow queries down? Only when CPU is already the bottleneck (hot, cached data with a heavy codec). For scan-bound queries reading from disk or object storage, compression almost always speeds things up.