ZFS ARC Cache Size Tuning: Set Limits Safely

Here is zfs arc cache size tuning explained for a small homelab: let ZFS cache useful data, then cap it when applications need more memory. OpenZFS recommends reducing ARC when it competes with other applications. A full RAM graph alone is a poor reason to spend Saturday editing kernel parameters. OpenZFS tuning guidance.

Who this is for / who should skip

This guide covers an existing OpenZFS installation on Linux, including Proxmox. You need a host shell, sudo access, and a maintenance window for persistent changes. Inspection is quick; evaluating a setting means replaying your actual workload.

Run the commands on the machine importing the ZFS pool. If your mini-PC merely mounts an SMB/NFS share, investigate ARC on the NAS. For a managed appliance, use its release-specific administration instructions before changing host configuration.

What ARC does with your RAM

ARC, commonly expanded as Adaptive Replacement Cache, holds cached blocks in memory and balances recent access with repeated use. Its size adapts under memory pressure, but referenced buffers cannot always be evicted immediately. That makes it reclaimable cache with practical constraints. OpenZFS implementation notes.

zfs_arc_max sets the ARC ceiling in bytes; it does not limit every ZFS allocation. zfs_arc_min sets its lower bound. Avoid raising the minimum on a machine sharing RAM with applications: doing so reduces room for reclamation. A maximum setting of 0 selects automatic sizing; it does not disable ARC. OpenZFS parameter manual.

ARC defaults depend on your installation

The old advice that Linux ZFS always takes half your RAM is stale. These are default ceilings, not promises that the cache immediately allocates that memory.

Installation	Default ARC ceiling
Upstream Linux OpenZFS before 2.3.0	Half of system RAM
Upstream OpenZFS from 2.3.0	Larger of RAM minus 1 GiB and five-eighths of RAM
New Proxmox installations starting with 8.1	Installer sets 10% of RAM, capped at 16 GiB

The upstream values come from OpenZFS’s version-aware parameter documentation. The Proxmox override is an installation policy; older installations do not automatically inherit it. Proxmox documentation.

For a shared host, budget RAM for peak application demand, the operating system, other ZFS allocations, and breathing room before assigning ARC its share. For a dedicated NAS, start with the distribution default. Pool capacity alone does not describe the data your applications repeatedly access.

Measure pressure and useful cache hits first

Inspect the running configuration:

zfs --version
cat /sys/module/zfs/parameters/zfs_arc_max
awk '$1 ~ /^(size|c|c_min|c_max)$/ {print $1, $3}' \
  /proc/spl/kstat/zfs/arcstats

Here, size is actual ARC usage, c is its adaptive target, and c_min/c_max are effective bounds. Record c_max even when the module parameter prints 0. For interval sampling, run zarcstat 1 or arcstat 1 on older packages; stop with Ctrl-C. The tool’s -v option lists fields, including demand hits, demand misses, and metadata hits. OpenZFS statistics manual.

Also run cat /proc/pressure/memory where PSI is available. The some line measures time with tasks stalled on memory; full measures simultaneous stalls across all non-idle tasks. Sustained pressure alongside slow applications is stronger evidence than a busy RAM gauge. Linux PSI documentation.

Compare the same file browsing, VM activity, or application job before and after tuning, allowing repeated reads to warm the cache. Judge latency and memory pressure together. A low hit rate during a first read of an archive does not establish that buying RAM will help.

Keep a lower cap if application stalls improve and storage performance remains acceptable. Consider increasing it when repeated reads still miss cache, the ARC reaches its ceiling, and memory headroom exists. These are reasons to run a comparison, not guarantees of a speedup.

Test a limit, then make it persistent

The following uses Proxmox’s documented 8 GiB example, expressed as 8589934592 bytes. Use it only if your memory budget supports it and the reported c_min is lower. It is an example, not a recommendation for every mini-PC. Proxmox limit example.

Save the effective ceiling and change the live parameter in the same Bash session:

arc_previous_max=$(awk '$1 == "c_max" {print $3}' \
  /proc/spl/kstat/zfs/arcstats)
printf '%s\n' "$arc_previous_max"
printf '%s\n' 8589934592 | sudo tee /sys/module/zfs/parameters/zfs_arc_max

Write down the printed value. Recheck c_max using the inspection command. Lowering the ceiling need not immediately shrink the cache without memory pressure; do not manufacture an out-of-memory event to force it. Returning the live parameter to 0 is unsupported. Runtime caveats.

To restore the previous effective ceiling in that shell:

printf '%s\n' "$arc_previous_max" | sudo tee /sys/module/zfs/parameters/zfs_arc_max

Keep the minimum below the maximum. If your chosen cap is at or below c_min, resolve that first rather than blindly copying the example.

After a successful trial, back up /etc/modprobe.d/zfs.conf, then edit its existing ARC option or add it if absent:

options zfs zfs_arc_max=8589934592

Avoid conflicting definitions. On Proxmox with ZFS root, run sudo update-initramfs -u -k all, then reboot during maintenance and recheck the effective limits. For persistent rollback, restore the previous configuration, rebuild initramfs, and reboot again. Proxmox persistence instructions.

L2ARC and dataset settings: resist the shopping list

An SSD L2ARC is another read-cache tier, and its bookkeeping consumes RAM. It can help reusable random-read workloads that overflow ARC. Buying one to rescue a memory-starved host is a poor starting point. TrueNAS cache guidance.

A SLOG serves synchronous write logging; it does not add ARC capacity. Diagnose whether reads or synchronous writes are slow before buying either device. TrueNAS read and write cache explanation.

Leave primarycache=all unless you have evidence for a dataset-specific exception. primarycache=metadata excludes file data from ARC while retaining metadata caching. Consider a controlled trial for a streaming archive; do not apply it across the pool because an unrelated database tuning post suggested it. OpenZFS dataset properties.

Storage, backups, and operations

Keep independent backups on a separate NAS or disconnected external disk. Restore a sample dataset into a separate location, open its files, and verify application recovery. Snapshots on the source pool do not replace that backup. TrueNAS backup guidance.

After kernel or ZFS updates, check the effective limits again and review kernel logs for memory failures. Alert on sustained memory stalls and failed jobs. Schedule host updates through your distribution, then validate the result; sister site Tech Sentinel covers why confirming a fix matters.