Most filesystems are indifferent to the hardware underneath them. OpenZFS is not, because it takes on jobs that a RAID card or an array controller would normally do, and it needs unobstructed access to the disks to do them. The result is a short list of hardware decisions that genuinely matter, wrapped in a much longer list of folklore. This guide separates the two, and marks which choices you can revisit later and which ones are set the moment the pool is created.
How much RAM does ZFS need?
ZFS has no universal RAM requirement per TB of storage. Start with the operating system’s requirements, allow for peak application and VM memory, then leave room for ARC and other ZFS allocations. A dedicated file server and a host running virtual machines can need different memory budgets even with identical disks. More ARC helps when it can retain blocks the workload reads again; it cannot make every workload faster.
The zfs_arc_max module parameter defaults to zero, which selects automatic sizing. Upstream OpenZFS 2.3 and later use the larger of total memory minus 1 GiB or five eighths of total memory. Earlier Linux releases use half of system memory, and distributions can override the upstream setting. On a 32 GiB machine using the newer upstream default, the ceiling is 31 GiB; that is a limit, not memory reserved immediately. See the version-specific ARC defaults.
For an existing host, compare memory pressure and cache misses during the workload that matters before changing the budget. If applications compete with ARC, consider a lower ceiling. The ZFS ARC size and cache tuning guide covers checking the effective limit, trying a cap and rolling it back. Keep the purchasing decision here separate from the commands used to tune an installed system.
The 1 GB per TB myth
The rule that ZFS needs 1 GB of RAM per TB of storage is not an OpenZFS requirement for ordinary pools. It also cannot size deduplication reliably: the deduplication table depends on unique block count and block size, not just drive capacity. Treat a fixed RAM-to-capacity ratio as an unsupported shortcut, whether it produces a reassuringly small number or an expensive shopping list.
The OpenZFS workload tuning documentation explains the mechanism. Deduplication maintains an on-disk hash table, the DDT, with one entry per unique block in the pool, and each cached entry uses slightly more than 320 bytes of memory. The DDT has to be consulted for every dedup-able block written or freed. If it does not fit in memory, each miss becomes a random read from disk, and the documentation gives the worst case bluntly: on a single 7200 RPM drive doing roughly 100 IO/s, uncached DDT reads cap write throughput at about 100 blocks per second.
Run the arithmetic and the rule stops being mysterious. One TiB of unique data at the default 128 KiB record size is about 8.4 million blocks, so roughly 2.7 GB of DDT. The same TiB stored as 16 KiB blocks, which is typical for zvols backing virtual machines, is about 67 million blocks and roughly 21 GB. That is where “several GB of RAM per TB” comes from, and it applies only when deduplication is enabled. The documentation also describes how to stop guessing: zdb -S simulates a DDT against the data already in the pool and reports the ratio you would have achieved, and multiplying the entry count by 320 bytes gives the memory requirement directly.
With deduplication off, memory planning is primarily about the host’s workload and cache needs. The ZFS and RAIDZ capacity calculator estimates disk space separately; it does not infer a RAM requirement from the pool’s size. Budget application memory independently before deciding how much cache headroom is available.
ECC memory
The OpenZFS hardware documentation states the position plainly: no technique used in ZFS, or in any other filesystem, can protect against a bit flip in memory, so ECC memory is highly recommended.
The reasoning is worth understanding rather than repeating. A flipped bit has four possible outcomes. It can land in unused memory and do nothing. It can land in something read from disk and cause a crash or a transient failure that a reboot clears. It can land in data on its way to disk, and here the timing decides everything: if the flip happens before the checksum is calculated, ZFS faithfully stores corrupt data with a valid checksum and never knows. Or it can land in on-disk metadata, where the worst case is a pool that will not import.
None of that is a ZFS-specific weakness. Every filesystem has the same exposure, and ZFS is if anything better placed because its checksums catch corruption that happens after the calculation. But ZFS is often chosen precisely for end-to-end integrity, and non-ECC memory is the gap in that chain.
Controllers: pass the disks through
The ideal ZFS controller is dull. The documentation lists three attributes: driver support on the major OpenZFS platforms, high bandwidth per port, and low cost. Support for RAID, battery backup units and hardware write caches is described as unnecessary.
Hardware RAID controllers should not be used with ZFS, and the documented reasons are concrete rather than ideological:
- Self-healing is blocked. A checksum failure on one disk is normally repaired from parity or a mirrored copy. When the controller owns the redundancy, ZFS has nothing to reconstruct from.
- Sector size information is not necessarily passed correctly on RAID 1, and cannot be passed correctly on RAID 5 or 6, which breaks the alignment work described below.
- RAID 5 and 6 reintroduce the partial-stripe write hole that ZFS avoids by design through copy-on-write.
- A failed controller may need to be replaced with the same model, or at least the same manufacturer. Disks presented natively work with any controller.
- A write cache adds a failure point that battery or flash backup only partially mitigates.
- If one array on a multi-array controller fails, the identifiers exposed to the operating system can shift, which the documentation notes has faulted pools imported from a cachefile.
Presenting each disk as a single-drive RAID 0 volume is the usual workaround and is explicitly not recommended, because most of the objections above still apply. Use a host bus adapter.
Two more interface notes from the same page. SATA disks should never be placed behind a SAS expander without an interposer, and SAS controllers should not be expected to work with SATA port multipliers. USB drives and USB-to-SATA adapters have documented problems with sector size reporting, SMART passthrough and error recovery control, and the documentation says they should be considered unreliable.
Disks: the choices that stick
Sector size and ashift. A vdev’s ashift fixes the smallest possible I/O it will issue, it is set when the vdev is created, and it cannot be changed afterwards. ZFS normally derives it from what the drive reports, which is reliable on modern drives but was not on the first generation of Advanced Format disks, which misreported 4096-byte sectors as 512 bytes for Windows XP compatibility. The practical consequence outlives the bad drives: the documentation notes that replacing a 512-byte sector drive with a 4096-byte one, in a vdev that was created with 512-byte sectors, will adversely affect performance. The reverse causes no harm. Creating pools with ashift=12 is the conventional hedge, and it is a decision you make once.
Error recovery control. This is the most underrated line item on a disk spec sheet. When a drive hits an unreadable sector it retries, and how long it is willing to retry is configurable only on drives that implement the feature: TLER on Western Digital, ERC on Seagate, CCTL on Hitachi and Samsung. Drives with it typically default to about 7 seconds. Drives without it can retry for arbitrarily long, and the documentation notes that several minutes is not impossible, during which I/O to that drive stalls. ZFS does not currently set this value itself, so on drives that support it the documentation suggests scripting a low limit at every boot.
Rotational speed and queuing. High-RPM drives buy lower seek times at a real cost in price and density, and the documentation puts the gain at no more than roughly a factor of six over slower drives. For a bulk pool, more spindles across more vdevs is usually the better spend, since random IOPS scale with vdev count rather than with the speed of any one disk.
Flash. For a separate log device, power failure protection is the property that matters, because a SLOG exists to make synchronous writes durable and a drive that loses its volatile write cache on power loss defeats the purpose.
A cache device is a different tradeoff, and the cost is quantified in the module parameters. Every buffer held on an L2ARC device needs a header in ARC, and those headers are not evicted under memory pressure, so a large cache device on a small-memory system eats the very thing it is supposed to extend. l2arc_meta_percent exists to bound this: it caps L2ARC-only headers at 33% of ARC size by default, and the documentation warns that too many headers on a system with an oversized L2ARC can render it slow or unusable. The TrueNAS guidance draws the practical line at 32 GiB, advising against adding L2ARC below it. More RAM first, then flash.
A sane order of spend
If the budget is finite, this ordering follows from the documentation rather than from taste. Redundancy inside the pool and a real backup outside it come first, because no hardware choice substitutes for either. ECC memory and enough RAM come next, because they protect and accelerate everything. A plain HBA comes third. Special, log and cache vdevs come last, and only when a specific workload justifies them.
The layout and parity decisions those disks land in are covered in RAIDZ1 vs RAIDZ2 vs RAIDZ3. What happens when one of these disks eventually fails is in reading a DEGRADED zpool status.
