In production, “disk full” is rarely a single event. It’s a chain reaction: a log file stops rotating, a database panics, a queue backs up, and suddenly the incident channel is full of people repeating “but df says there’s space.” ZFS gives you unusually sharp tools for avoiding that mess—if you understand the difference between space that exists and space your dataset can actually write.
This is a deep, practical guide to ZFS refreservation: what it actually guarantees, what it doesn’t, how it interacts with quotas and snapshots, and how to use it to keep critical apps alive while everything else fights over the remaining bytes. The goal isn’t academic correctness. The goal is to never again watch a perfectly “healthy” pool take down a tier-0 service because the last few GiB were eaten by something boring.
What refreservation is (and why it exists)
refreservation is a ZFS dataset property that reserves space based on referenced data for that dataset. In plain English: it’s ZFS saying, “No matter what else happens in this pool, this dataset will have at least N bytes available to write, as long as the pool itself hasn’t already crossed the point of physical impossibility.”
The “ref” part matters. ZFS is copy-on-write. A dataset’s “used” space is not just about what’s in its current filesystem state. Snapshots can keep old blocks alive. Clones can share blocks. Deletes don’t always free space when you think they do. That’s how you end up with the classic ZFS argument:
- Application: “I deleted 500GB. Why can’t I write 10GB?”
- ZFS: “Because those 500GB are still referenced by snapshots, clones, or other datasets. Nice try.”
refreservation is designed to protect a dataset’s ability to keep writing even when its own history (snapshots) or its relationship to other datasets (clones) makes space accounting non-obvious.
One operational way to describe it: refreservation is how you keep “important datasets” from being starved by “interesting datasets.” Important: databases, journaling queues, metadata stores, VM boot volumes. Interesting: anything that humans can dump into, especially logs, upload buckets, build artifacts, or “temporary” scratch areas that have been temporary since 2019.
Joke #1: Refreservation is like putting your lunch in the office fridge with your name on it—except it actually works.
Interesting facts & historical context
Storage engineering is full of “new” ideas that are really old ideas with better tooling. A few context points that help refreservation make sense:
- ZFS was built for end-to-end integrity, not just capacity management. Space controls like quotas and reservations exist because “correct but full” still breaks apps.
- Copy-on-write filesystems make deletes complicated. Traditional filesystems generally free blocks on delete; ZFS only frees blocks when nothing references them. Snapshots are the most common “something.”
- The “df paradox” predates ZFS. Even on older systems, reserved blocks, filesystem overhead, and delayed allocation caused “df shows free” moments. ZFS just makes the reasons more powerful—and more confusing.
- Thin provisioning became mainstream in virtualization and brought a new class of outages: guests think they have space; the backend disagrees at 3 a.m. ZFS reservations are one of the more honest ways to manage thin vs thick tradeoffs.
- Snapshots moved from “backup feature” to “operational default.” The moment teams began snapshotting frequently (hourly, every 15 minutes, sometimes every 5), space pressure became a day-to-day concern.
- “Keep 20% free” is not superstition. Many storage systems (including ZFS) degrade in performance and allocation flexibility as they approach full. Refreservation won’t fix physics.
- ZFS property inheritance is a feature and a footgun. Reservations and quotas can inherit unexpectedly; refreservation is frequently set at the dataset level precisely because “one size fits all” fails.
- Refquota/refreservation were shaped by real admin pain. The “ref” properties exist because “used” can include space that isn’t truly yours to free (snapshots/clones).
A mental model: referenced vs available space
If you operate ZFS long enough, you end up with two mental ledgers:
- Logical ownership: what a dataset appears to contain right now.
- Physical reality: which blocks are still referenced anywhere in the pool.
ZFS exposes this split via dataset space properties. The exact output varies by platform (OpenZFS on Linux vs illumos derivatives), but the ideas are stable:
used: total space consumed by dataset and descendants, including snapshots (depending on context) and overhead.referenced: space referenced by the dataset’s current state (excluding snapshots).usedbydataset,usedbysnapshots,usedbychildren: the best “where did it go?” breakdown.available: what ZFS believes can be written to the dataset, after quotas/reservations and pool constraints.
refreservation is tied to referenced. Think of it as: “I want to guarantee that this dataset can grow its referenced bytes by at least N, regardless of snapshot baggage elsewhere.” The mechanics are subtle, but operationally the effect is clear: it carves out pool space that other datasets can’t claim.
Reservation vs refreservation vs quotas
ZFS offers several levers that sound similar. They are not interchangeable. Here’s the practical mapping:
reservation
reservation reserves space for a dataset and its descendants (depending on how you structure things). It’s a “this subtree gets at least N bytes” guarantee. It’s useful when you treat a dataset as a container for many children and want a minimum slice of the pool protected.
refreservation
refreservation is about the dataset’s own referenced space (not snapshots). It’s the sharper tool when you care about a dataset’s ability to keep writing its current live state even when snapshots exist and deletions don’t immediately return space.
quota
quota limits how much space a dataset (and typically its descendants) can consume. It’s a “you can’t grow past this” ceiling. Good for multi-tenant control. Bad if you set it and forget it.
refquota
refquota limits referenced space (current live data), not snapshots. This is often what you want when you don’t want snapshots to be the reason a dataset refuses writes—though you’re still constrained by overall pool space.
A quick decision rule
- If you want to cap an app’s live data: use
refquota. - If you want to cap everything including snapshot sprawl: use
quota. - If you want to guarantee a minimum slice for a dataset’s live writes: use
refreservation. - If you want to guarantee a minimum slice for a whole subtree: use
reservation.
Joke #2: If quotas are a diet, refreservations are meal prep—less exciting, more effective, and everybody resents them until the crisis hits.
How ZFS enforces refreservation (what’s really guaranteed)
ZFS reservations are not magical extra space; they’re exclusion zones. When you set a refreservation, ZFS reduces what other datasets see as available because that space is earmarked.
Three important operational truths:
- Refreservation only helps if the pool has room to honor it. If the pool is truly full (or effectively full due to slop space and metadata needs), ZFS can’t allocate blocks it doesn’t have. Refreservation is a policy, not a law of thermodynamics.
- Refreservation does not reserve “snapshot space.” It’s about referenced/live data growth. Snapshots can still consume the pool and create pain, just not by stealing the guaranteed slice from that dataset’s live writes.
- Refreservation can make other datasets fail sooner—and that’s the point. It’s a prioritization mechanism. Your log archive dataset should fail before your database dataset does.
Also, ZFS is honest but not always intuitive: the available value on a dataset already reflects these policy constraints. If your refreservation is large, zfs list for other datasets will show less available, even if the pool has raw free space.
Three corporate-world mini-stories
Mini-story #1: The outage caused by a wrong assumption (“Snapshots are backups, right?”)
A mid-sized company ran a customer-facing API backed by PostgreSQL on ZFS. The storage layout looked clean: one pool, a dataset for the database, a dataset for WAL, and a dataset for logs. They took frequent snapshots because it felt like the responsible thing to do. And it was—until the night it wasn’t.
A developer ran a backfill job that created a large temporary table and then dropped it. The team watched the database’s filesystem usage spike and then fall. Everyone exhaled. But the pool didn’t. Because snapshots were retaining the blocks, the “freed” space wasn’t actually free. The pool drifted toward full over the next few hours as normal writes continued.
At about 95% pool utilization (effective), performance degraded. Latency climbed. Autovacuum started thrashing. Then the WAL dataset hit ENOSPC. PostgreSQL did what databases do when they can’t write WAL: it stopped being a database.
The wrong assumption was subtle: “If the database deletes data, the pool gets space back.” On ZFS with snapshots, that’s not guaranteed. They had no refreservation on WAL, no refquota to keep temp growth sane, and their snapshot retention policy was “forever until we remember to clean it up.” The fix wasn’t heroic: they added a WAL dataset refreservation sized to cover peak write bursts, capped the database dataset with refquota, and established snapshot lifetimes with enforcement.
The postmortem had a line I’ve seen in many forms: “We treated snapshots like backups.” Snapshots are a time machine attached to your allocation layer. They’re incredibly useful—just not free.
Mini-story #2: The optimization that backfired (“Thin everything!”)
A different org consolidated multiple services onto a single ZFS pool to simplify operations. They moved VM disks and container volumes into datasets and leaned into thin provisioning. It looked efficient: high utilization, low waste, everyone happy. Finance loved it.
Then someone “optimized” by removing a set of dataset reservations. The logic sounded solid in a meeting: “Reservations waste space. We can reclaim capacity and get better density.” They removed reservations from a few infrastructure datasets, including the one holding critical VM boot volumes.
Two weeks later, a build pipeline went rogue. It filled an artifact dataset with multi-gigabyte blobs. Because nothing was reserved, the artifacts dataset happily consumed the pool’s remaining headroom. When a hypervisor tried to write to a VM disk (a tiny metadata update, nothing dramatic), the write failed. Suddenly VMs that had plenty of apparent free space inside the guest couldn’t commit writes at the storage layer. The result was a flurry of filesystem remounts read-only and a lot of “how is this possible?”
The lesson wasn’t “thin provisioning is bad.” The lesson was: thin provisioning without priority rules is gambling. Reservations/refreservations are those rules. The team reintroduced guarantees for boot volumes and databases, and they put the artifact dataset behind a quota plus a separate “overflow” pool. The density went down a bit. The sleep improved a lot.
Mini-story #3: The boring but correct practice that saved the day (“Protect the journal”)
A payments team ran a message queue whose durability depended on a write-ahead log. The queue wasn’t glamorous, but it was the system of record for in-flight work. Someone on the team had been burned before by storage-full incidents, so they did something deeply unsexy: they created a dedicated dataset for the journal and set a refreservation based on worst-case ingestion rate and incident response time.
Months later, a “harmless” debug flag got enabled during a customer escalation. Logs exploded. Not in a cute way—in a “terabytes in a day” way. The log dataset ate everything it could, snapshots included, and the pool went from comfortable to tight in hours.
But the queue kept working. The journal dataset still had space to write because its refreservation carved out a slice that the log dataset couldn’t steal. The incident was noisy—alerts, cleanup, retention policy fixes—but it wasn’t catastrophic. No lost messages, no corrupted state, no midnight restore from backup.
That’s what refreservation is for: not making pools bigger, but making failures land on the right targets. When things go wrong, you want the least important dataset to be the first one to scream.
Practical tasks (commands + interpretation)
These are hands-on tasks you can run on an OpenZFS system (commonly Linux with ZFS-on-Linux/OpenZFS). Commands assume you have privileges. Replace pool/dataset names to match your environment.
Task 1: Get the pool’s health and capacity reality
cr0x@server:~$ zpool status -v
pool: tank
state: ONLINE
scan: scrub repaired 0B in 0 days 00:12:41 with 0 errors on Sun Dec 22 02:10:11 2025
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
sda ONLINE 0 0 0
sdb ONLINE 0 0 0
errors: No known data errorsInterpretation: Before you tune properties, confirm you’re not debugging around a degraded pool. Reservations won’t rescue you from failing disks or a pool that can’t allocate due to errors.
Task 2: See pool-level free space and fragmentation hints
cr0x@server:~$ zpool list -o name,size,alloc,free,cap,frag,health
NAME SIZE ALLOC FREE CAP FRAG HEALTH
tank 7.25T 6.10T 1.15T 84% 29% ONLINEInterpretation: High CAP and rising FRAG are the conditions where “should work” turns into “why is it slow and failing?” If you’re above ~85–90% consistently, you’re in the danger zone regardless of reservations.
Task 3: Inventory datasets and their available space
cr0x@server:~$ zfs list -o name,used,avail,refer,mountpoint
NAME USED AVAIL REFER MOUNTPOINT
tank 6.10T 1.02T 192K /tank
tank/db 2.40T 220G 2.10T /tank/db
tank/db-wal 180G 120G 175G /tank/db-wal
tank/logs 1.60T 140G 1.20T /tank/logs
tank/artifacts 1.90T 90G 1.50T /tank/artifactsInterpretation: avail here is what matters to applications. It’s already adjusted for reservations/quotas. If a critical dataset has low avail, you don’t have a theoretical problem—you have a near-term incident.
Task 4: Show reservation-related properties for all datasets
cr0x@server:~$ zfs get -r -o name,property,value,source reservation,refreservation,quota,refquota tank
NAME PROPERTY VALUE SOURCE
tank reservation none default
tank refreservation none default
tank quota none default
tank refquota none default
tank/db reservation none default
tank/db refreservation 300G local
tank/db quota none default
tank/db refquota 3T local
tank/db-wal reservation none default
tank/db-wal refreservation 200G local
tank/db-wal quota none default
tank/db-wal refquota none default
tank/logs reservation none default
tank/logs refreservation none default
tank/logs quota 2T local
tank/logs refquota none defaultInterpretation: If you can’t answer “which datasets are guaranteed space?” from one command, you’re running on vibes. This view makes the policy explicit.
Task 5: Set a refreservation for a critical dataset
cr0x@server:~$ sudo zfs set refreservation=200G tank/db-wal
cr0x@server:~$ zfs get refreservation tank/db-wal
NAME PROPERTY VALUE SOURCE
tank/db-wal refreservation 200G localInterpretation: This earmarks 200G of pool space for the WAL dataset’s referenced growth. If you overshoot, you can starve other datasets—so size it based on real write rates and incident response time.
Task 6: Confirm how refreservation affects other datasets’ available space
cr0x@server:~$ zfs list -o name,avail tank/logs tank/artifacts tank/db-wal
NAME AVAIL
tank/logs 90G
tank/artifacts 40G
tank/db-wal 120GInterpretation: If you just set a refreservation, expect other datasets’ AVAIL to shrink. That’s not a bug; it’s the policy doing its job.
Task 7: See where space is tied up (dataset vs snapshots)
cr0x@server:~$ zfs list -o name,used,usedbydataset,usedbysnapshots,usedbychildren,refer -t filesystem tank/db
NAME USED USEDBYDATASET USEDBYSNAPSHOTS USEDBYCHILDREN REFER
tank/db 2.40T 2.10T 260G 40G 2.10TInterpretation: If USEDBYSNAPSHOTS is large, deletes won’t free space. If your app “freed” data but your pool didn’t, this breakdown is where you start.
Task 8: List snapshots and identify big offenders
cr0x@server:~$ zfs list -o name,used,refer,creation -t snapshot tank/db | tail -n 5
tank/db@auto-2025-12-23-0000 12.4G 2.06T Tue Dec 23 00:00 2025
tank/db@auto-2025-12-23-0100 9.8G 2.07T Tue Dec 23 01:00 2025
tank/db@auto-2025-12-23-0200 15.1G 2.07T Tue Dec 23 02:00 2025
tank/db@auto-2025-12-23-0300 11.2G 2.08T Tue Dec 23 03:00 2025
tank/db@auto-2025-12-23-0400 10.6G 2.08T Tue Dec 23 04:00 2025Interpretation: Snapshot USED is space unique to that snapshot (relative to other snapshots). Big numbers often correlate with churny workloads (databases, CI artifacts) and can justify tightening retention or moving workloads.
Task 9: Check whether a dataset is blocked by a quota/refquota rather than pool fullness
cr0x@server:~$ zfs get -o name,property,value quota,refquota tank/logs
NAME PROPERTY VALUE
tank/logs quota 2T
tank/logs refquota noneInterpretation: If an app sees ENOSPC but the pool isn’t full, quotas are a prime suspect. Quotas are a “local full” even when the pool has capacity.
Task 10: Measure real referenced headroom vs refreservation size
cr0x@server:~$ zfs get -o name,property,value referenced,refreservation tank/db-wal
NAME PROPERTY VALUE
tank/db-wal referenced 175G
tank/db-wal refreservation 200GInterpretation: With referenced at 175G and refreservation at 200G, you’re essentially guaranteeing roughly 25G of additional referenced growth (subject to pool reality). If referenced is already above the refreservation, you’re not “owed” extra; the reservation is effectively fully consumed.
Task 11: Use a reservation instead when you want to protect a subtree
cr0x@server:~$ sudo zfs set reservation=500G tank/db
cr0x@server:~$ zfs get reservation tank/db
NAME PROPERTY VALUE SOURCE
tank/db reservation 500G localInterpretation: This is appropriate when tank/db contains multiple children (tablespaces, backups, temp areas) and you want the group protected. Use with care: subtree reservations can be surprisingly “sticky” in their effects on pool availability.
Task 12: Remove a refreservation safely (and verify impact)
cr0x@server:~$ sudo zfs set refreservation=none tank/db-wal
cr0x@server:~$ zfs get refreservation tank/db-wal
NAME PROPERTY VALUE SOURCE
tank/db-wal refreservation none defaultInterpretation: Removing guarantees should be treated like removing a circuit breaker: it might be fine, but you’re changing failure modes. Do it with an explicit reason and a rollback plan.
Task 13: Check pool “slop space” constraints (why the last few GiB don’t behave)
cr0x@server:~$ zfs get -o name,property,value special_small_blocks 2>/dev/null
cr0x@server:~$ zfs list -o name,avail tankInterpretation: Many operators get surprised by the practical limit where ZFS becomes conservative near full to preserve pool operability. If you’re designing guarantees, don’t plan to use “100% of pool size.” Leave headroom.
Task 14: Identify whether deletes are blocked by snapshots/clones
cr0x@server:~$ zfs get -o name,property,value used,referenced,usedbysnapshots tank/artifacts
NAME PROPERTY VALUE
tank/artifacts used 1.90T
tank/artifacts referenced 1.50T
tank/artifacts usedbysnapshots 380GInterpretation: If the “deleted” data was part of referenced state but snapshots retain it, usedbysnapshots grows. That’s often the explanation for “rm -rf didn’t fix it.”
Task 15: Create a “sacrificial” dataset with a quota to absorb garbage
cr0x@server:~$ sudo zfs create tank/scratch
cr0x@server:~$ sudo zfs set quota=200G tank/scratch
cr0x@server:~$ zfs get -o name,property,value quota tank/scratch
NAME PROPERTY VALUE
tank/scratch quota 200GInterpretation: This is the operational version of “put the messy stuff in one room with a lock.” You can point temporary jobs here so they hit a wall before production datasets do.
Fast diagnosis playbook
When something is failing with ENOSPC, slow writes, or “available space doesn’t make sense,” don’t spelunk randomly. Check these in order.
1) Pool reality: is the pool actually full or unhealthy?
cr0x@server:~$ zpool list -o name,size,alloc,free,cap,frag,health
cr0x@server:~$ zpool status -vWhat you’re looking for: CAP > ~90%, very high fragmentation, or degraded state. If the pool is on fire, dataset-level tweaks are secondary.
2) Dataset constraints: quota/refquota/reservation/refreservation
cr0x@server:~$ zfs get -r -o name,property,value,source quota,refquota,reservation,refreservation tankWhat you’re looking for: A dataset hitting a quota/refquota, or other datasets “losing” availability because a large reservation/refreservation exists somewhere else.
3) Space held by snapshots and children
cr0x@server:~$ zfs list -o name,used,usedbydataset,usedbysnapshots,usedbychildren,refer -t filesystem tank
cr0x@server:~$ zfs list -t snapshot -o name,used,creation | tailWhat you’re looking for: Big usedbysnapshots on churny datasets; a child dataset consuming the parent’s reservation; snapshots multiplying growth.
4) If performance is the symptom: check pool I/O saturation and latency
cr0x@server:~$ iostat -x 1 5
cr0x@server:~$ zpool iostat -v 1 5What you’re looking for: High disk utilization, long await times, and uneven device load. A pool near full can also amplify write amplification and metadata churn.
5) Confirm the app’s view (filesystem vs ZFS)
cr0x@server:~$ df -hT /tank/db /tank/db-wal
cr0x@server:~$ zfs list -o name,avail,used,refer tank/db tank/db-walWhat you’re looking for: Mismatch between df and zfs list generally means ZFS policy (quota/reservation) or snapshot reality is the deciding factor, not what the OS thinks in generic terms.
Common mistakes (symptoms + fixes)
Mistake 1: “We set refreservation and now the pool ‘lost’ space.”
Symptom: Other datasets show dramatically reduced AVAIL. Someone claims you “consumed” hundreds of gigabytes without writing data.
What’s happening: Reservations/refreservations reduce shared availability; that’s the guarantee being honored.
Fix: Right-size guarantees based on worst-case writes and response time. If you need to protect multiple services, distribute smaller guarantees rather than one giant carve-out. Confirm with:
cr0x@server:~$ zfs get -r -o name,property,value refreservation,reservation tankMistake 2: Assuming refreservation protects you from snapshots filling the pool
Symptom: You set refreservation on a critical dataset, but writes still fail when the pool is nearly full.
What’s happening: If the pool can’t allocate blocks due to overall fullness (plus ZFS’s need for metadata and operational headroom), the guarantee can’t conjure space.
Fix: Enforce snapshot retention and capacity headroom. Treat “pool > 90%” as an incident, not a metric. Investigate snapshot usage:
cr0x@server:~$ zfs list -o name,used,usedbysnapshots -t filesystem tank | sort -h -k3Mistake 3: Using reservation when you meant refreservation (or vice versa)
Symptom: You reserve space on a parent dataset, but a child dataset still hits ENOSPC; or you set refreservation on a parent and it doesn’t behave like “subtree guaranteed.”
What’s happening: reservation and refreservation apply differently in hierarchy and in accounting terms.
Fix: Decide whether you’re protecting a single dataset’s live writes (refreservation) or an entire subtree (reservation). Verify inheritance and where the reservation is applied:
cr0x@server:~$ zfs get -r -o name,property,value,source reservation,refreservation tank/dbMistake 4: Treating quotas as “safe defaults” without monitoring
Symptom: A service fails writes even though the pool has plenty of free space.
What’s happening: The dataset hit quota or refquota. From the app’s perspective, it’s out of disk.
Fix: Monitor dataset used vs quotas and alert early. Confirm quickly:
cr0x@server:~$ zfs get -o name,property,value quota,refquota tank/serviceMistake 5: Making guarantees bigger than your “oh no” runway
Symptom: Less-critical datasets hit ENOSPC constantly; teams start bypassing controls; you end up in a whack-a-mole cycle.
What’s happening: You over-allocated guaranteed space relative to pool size and real usage patterns. Guarantees become permanent contention.
Fix: Sum reservations and refreservations and compare to realistic pool free space targets. You want guarantees to protect key workloads, not strangle everything else.
cr0x@server:~$ zfs get -r -H -o value refreservation,reservation tank | awk '
{ if ($1 != "none") sum += $1 }
END { print sum }'Interpretation: This rough approach needs unit handling in real scripts, but the operational point stands: total guarantees must fit inside your capacity plan with headroom.
Checklists / step-by-step plan
Checklist A: Introducing refreservation for a critical service (without surprises)
- Measure the write burst. Look at the service’s worst 1–6 hour write volume, plus overhead and incident response time.
- Confirm pool headroom. If the pool lives above 85–90%, fix capacity first. Guarantees on a chronically full pool are just rearranging chairs.
- Create a dedicated dataset. Don’t share a dataset between “critical writes” and “everything else.”
- Set
refreservationon the dataset. Start modest, then adjust based on observed behavior. - Optionally cap the noisy neighbors. Add
quotaorrefquotato log/artifact datasets so they fail before tier-0 does. - Validate availability changes. Confirm other datasets’
availdidn’t drop below safe operational thresholds. - Simulate the failure mode. Fill a non-critical dataset until it hits quota and confirm critical dataset continues writing.
Checklist B: Capacity incident response when a pool is trending full
- Identify top consumers (
zfs listandusedbysnapshotsbreakdown). - Check snapshot retention and delete the right snapshots (not random ones) when safe.
- Confirm no runaway dataset is unconstrained (no quota) while critical datasets lack guarantees.
- Reduce churn: stop the job that’s generating the write storm.
- Only then consider emergency property changes (temporary quota decreases, temporary reservations) with explicit rollback steps.
Step-by-step: A sane “tiering” policy with refreservation
This is a practical pattern that works well in corporate multi-service pools:
- Tier 0 datasets (must keep writing): set refreservation; consider dedicated pool or special vdevs if applicable; alert on avail dropping below the guarantee runway.
- Tier 1 datasets (important but degradable): moderate quotas/refquotas; smaller reservations if needed.
- Tier 2 datasets (best effort): quotas are mandatory; snapshots retained short; accept ENOSPC as a controlled failure.
FAQ
1) Does refreservation guarantee free space even if the pool is 99% full?
No. It guarantees policy-level availability within the pool’s physical ability to allocate. Near-full pools can fail allocations due to metadata needs and ZFS’s operational safety margins.
2) What’s the simplest way to tell if ENOSPC is due to quota or pool fullness?
Check both the pool and the dataset:
cr0x@server:~$ zpool list -o name,cap,free
cr0x@server:~$ zfs get -o name,property,value quota,refquota tank/the-datasetIf the pool has free space but the dataset has a quota/refquota near its limit, it’s a dataset constraint.
3) Why does deleting files not free space on ZFS?
Because snapshots (and sometimes clones) can still reference the blocks. Deleting removes references from the live filesystem, but the blocks remain allocated until nothing references them.
4) Should I use refreservation for databases?
Often yes for write-ahead logs/journals and other durability-critical volumes, because they are the first to take the service down when writes fail. Pair with snapshot discipline and capacity monitoring.
5) Can refreservation help with “df shows space but writes fail”?
It can prevent one class of failure—being starved by other datasets—but it doesn’t solve quota misconfigurations or a pool that’s effectively full. Use zfs list as the source of truth for dataset availability.
6) Is refreservation the same as preallocating a file?
No. Preallocating a file (like fallocate) allocates blocks to that file. Refreservation is a dataset-level guarantee that reduces what other datasets can use. It’s policy, not a specific allocation layout.
7) How do snapshots interact with refreservation?
Snapshots consume pool space as the live dataset changes. Refreservation doesn’t “reserve snapshot space,” but it helps ensure the dataset’s live referenced data has a protected runway to grow.
8) How do I pick a refreservation size?
Base it on: peak write rate × (time-to-detect + time-to-mitigate) plus some buffer for operational slop. If your WAL can grow by 30G in an hour during a replay storm, reserving 10G is optimism, not engineering.
9) Can I set refreservation on a zvol?
Yes, and it can be especially relevant for zvol-backed VM disks where you want to guarantee backend space. But also consider volsize and whether you’re effectively doing thin provisioning.
Conclusion
ZFS refreservation is not a capacity cheat code. It’s a failure-mode design tool. In a shared pool, something will eventually try to eat all available space—logs, artifacts, a runaway query, a “temporary” export. Without guarantees, the wrong thing fails first, and the incident turns into a business problem.
Use refreservation to keep the lights on for the datasets that must keep writing. Pair it with quotas to contain the datasets that shouldn’t be allowed to ruin your day. And remember the most operationally useful ZFS truth: when the space math gets weird, it’s almost always snapshots, reservations, or both—and ZFS will tell you which, if you ask it the right way.