ZFS Compression LZ4 vs ZSTD: Which to Use per Dataset

For zfs compression lz4 vs zstd, the verdict is lz4 everywhere by default, then zstd (level 3, which is what plain zstd means) on datasets full of compressible, read-mostly data: logs, documents, source trees, backups. Leave VM zvols, databases and media on lz4. On OpenZFS 2.2 or newer, early abort keeps zstd from wasting CPU on incompressible blocks.

Compression is a per-dataset property, so this isn’t a bet you make once for the whole pool. Set a sane default at the top and override it where the data earns it. Everything below runs on your CPU, so if you’re still picking the box, ZFS hardware requirements comes first. Btrfs offers zstd too; ZFS vs Btrfs for a home NAS covers that bigger decision.

How they were compared

No new benchmarks were run for this comparison. It draws on published numbers and the OpenZFS documentation:

• Codec speed and ratio: the benchmark table in the Zstandard README, produced with lzbench on a Core i7-9700K at 4.9GHz under Ubuntu 24.04, using the Silesia corpus. It measures the codecs in memory, not through a filesystem.
• Early abort: the test runs in OpenZFS pull request #13244, on a Raspberry Pi 4, a Ryzen 5900X and an Intel 8700K, against datasets compressing about 1.05x and about 2.25x.
• Recordsize effects: one user’s synthetic 1 GB file in OpenZFS discussion #11293.
• Behavior and defaults: the OpenZFS zfsprops(7) and zfs(4) manuals, the Proxmox VE ZFS docs, and a TrueNAS blog post. TrueNAS sells storage appliances, so treat that post as vendor guidance; it publishes no figures.

ZFS compresses each record independently, as explained in that discussion, so your compressratio won’t match a whole-file codec benchmark. The ordering between codecs carries over; the absolute ratios don’t.

LZ4 vs zstd in OpenZFS: spec table

Sources: zfsprops(7), the LZ4 block format, RFC 8878, zfs(4) and zpool-features(7).

The codec numbers, from the Zstandard README:

zstd-1 buys a much better ratio for a modest drop in compression speed, but lz4 decompresses about two and a half times faster. zstd’s fast mode at level 4 essentially matches lz4’s ratio and compression speed while decompressing far slower. The README doesn’t list level 3, the ZFS default; TrueNAS says it improves on lz4’s ratio “with only a small throughput cost.”

Where each one wins

lz4 for VMs, databases and anything latency-bound. zvols and database files get read in small chunks, constantly, and much of that traffic pays for decompression, which is where lz4’s lead is biggest. Proxmox recommends lz4 “because it adds very little CPU overhead.”

lz4 for media. Video, photos and music are already compressed. ZFS only stores a block compressed when it clears a 12.5% default savings threshold, so most media blocks land uncompressed whatever you pick. Setting zstd-9 on a Jellyfin library is a rite of passage; so is finding out it changed nothing.

lz4 for boot pools. OpenZFS’s GRUB2 compatibility lists, both 2.12 and later and 2.06, include lz4_compress and omit zstd_compress. If GRUB reads the pool, keep it on lz4.

zstd for text, logs and backups. Logs, documents, source trees, database dumps and backup targets receiving uncompressed data compress well and are mostly written once. A better ratio means fewer bytes on disk and fewer to read back. It stretches RAM too: ARC stores blocks in compressed form by default, per zfs(4), so more logical data fits in cache. The ARC sizing guide shows how to check whether cache is actually your bottleneck.

zstd-fast: rarely. Per the table, it buys lz4’s compression speed with worse decompression. Skip it unless you have a measured reason.

gzip: not anymore. The OpenZFS 2.0 release notes describe zstd as providing “similar or better compression levels to GZIP, but with much better performance.” Existing gzip datasets read fine; new ones should use zstd.

Why does zstd compress better inside a ZFS record?

zstd compresses better because it can match repeated data anywhere inside a record and finishes with entropy coding, while lz4 can only refer back 64 KB. ZFS compresses each record independently, so the default 128K record already gives zstd twice lz4’s reach, and a 1M record widens the gap considerably.

The LZ4 block format caps a match offset at 65535 bytes: “65536 and beyond cannot be coded.” RFC 8878 lets a zstd frame use windows from 1 KB up to terabytes, recommending 8 MB as the interoperable ceiling, so a 1M record fits easily. zstd also entropy-codes its output with Huffman and FSE stages; lz4 has neither.

Discussion #11293 shows how much recordsize can matter. A user wrote one compressible, synthetic 1 GB file to zstd datasets: 1.03x at 64K recordsize, 1.72x at 256K, 2.24x at 512K and 2.63x at 1M. Under lz4 at 1M, the same file stayed at 1.03x. One artificial file predicts nothing about your photo backups, but the direction holds: zstd rewards big records and lz4 barely notices them.

Don’t raise recordsize everywhere to chase that. An OpenZFS developer in the same thread says large recordsize “is good for files that are written and read sequentially or in respectively large chunks” but hurts random access through read-modify-write. Use 1M on sequentially written archives and backups; leave VMs and databases alone.

Does zstd early abort fix incompressible data?

Mostly, on OpenZFS 2.2 and newer. For zstd levels 3 and above, on records of at least 128 KiB, ZFS first tries lz4; if lz4 gives up, it tries zstd-1; if that fails too, the block is stored uncompressed without ever running the expensive level you asked for.

The OpenZFS 2.2.0 release notes say incompressible data is “detected quickly, avoiding wasted work.” Two module parameters in zfs(4) control it: zstd_earlyabort_pass (default 1, enabled) and zstd_abort_size (default 131072 bytes). The PR explains the second pass: lz4 alone “was losing up to 8.5% of the compressed savings versus no early abort” on highly compressible data. The author’s summary of the trade: ”< 0.3% more space used to go from 2 hours to 15 minutes? Sold.”

There are two limits. Early abort only saves CPU on data that refuses to compress; data that does compress still pays the full cost of whatever level you set, and TrueNAS says throughput “roughly halves for every two levels of added compression.” It also skips records under 128 KiB, which covers small-block zvols.

# Needs 2.2 or newer; the parameter should print 1
zfs version
cat /sys/module/*/parameters/zstd_earlyabort_pass

Which one to pick

Pick lz4 as the pool-wide default and override it with plain zstd on the datasets holding text, logs and backups. That covers nearly every home NAS and mini-PC setup without touching anything else. Only go above zstd-3 for write-once archives you can afford to write slowly.

# What are you running, and what does it compress to today?
zfs version
zfs get -r compression,compressratio tank

# Pool-wide default, inherited by every child dataset
sudo zfs set compression=lz4 tank

# Compressible, read-mostly data gets zstd (level 3)
sudo zfs set compression=zstd tank/backups
sudo zfs set compression=zstd tank/logs
sudo zfs set compression=zstd tank/documents

# Confirm VMs and media still inherit lz4
zfs get -o name,value,source compression tank/vms tank/media

Swap tank and the dataset names for yours. The property only affects newly written data, so set it before the data lands; compressratio covers a dataset’s referenced space, so a dataset holding old lz4 blocks and new zstd ones shows a blend.

Tempted by a higher level for an archive? Create two scratch datasets, copy the same sample of real data into each, and compare compressratio and copy time before committing. Bigger numbers feel better, and you pay for them at write time.

One compatibility check before the first zstd: the zstd_compress feature isn’t read-only compatible, so once it’s active, a ZFS build without zstd support, such as anything before OpenZFS 2.0, can’t import the pool at all. It stays active until every dataset that ever used zstd is destroyed. Switching back to lz4 doesn’t undo it.

FAQ

does changing zfs compression recompress existing data

No, the property only applies to newly written data; existing blocks keep the codec they were written with. To convert them, copy the data into a fresh dataset or run zfs rewrite where your OpenZFS build includes it. zfs rewrite -P keeps incremental sends from re-sending rewritten data, but snapshot-held blocks still cost extra space.

what zstd level should i use for zfs

Use plain zstd, which is level 3, for almost everything that deserves zstd. TrueNAS describes zstd-3 as improving on lz4’s ratio at only a small throughput cost, while each step up costs real write speed. Reserve levels above 3 for write-once archives, and compare compressratio on a sample before committing.

does zfs compression use a lot of cpu on a mini pc

lz4 uses little CPU, which is why Proxmox recommends it, while zstd uses more and the cost climbs with the level. In the Zstandard README’s desktop-class lzbench run, zstd-1 compressed at 510 MB/s against lz4’s 675 MB/s; a low-power mini-PC will post lower numbers for both. If writes pin the CPU, step down a level.

can i boot from a zfs pool that uses zstd

Not with GRUB: OpenZFS’s GRUB2 compatibility feature lists include lz4_compress but not zstd_compress, so keep any GRUB-read boot pool on lz4. Setups that keep the kernel on a separate boot pool or partition sidestep this, because the restriction only applies to whatever the bootloader itself has to read.