Every storage argument in a corporate chat room eventually turns into a screenshot of zfs list and a confident claim that “ZFS is lying.” It isn’t. It’s just doing accounting with rules that are different from the rules your brain learned from ext4, VMFS, or that one SAN UI that rounds everything to “about a lot.”
This piece is about two deceptively small fields—logicalused and used—and why they’re the difference between a calm on-call and an escalated incident. We’ll go deep enough to satisfy the person who reads zdb for fun, but practical enough that you can diagnose a space crisis while someone from Finance is asking why “we’re paying for disks we’re not using.”
Why these two numbers matter
In ZFS, “how much space is used?” is not one question. It’s at least three:
- How much space would my data occupy if it were stored without compression and without sharing blocks with snapshots/clones?
- How much physical space is actually allocated on disk right now?
- How much space can I free by deleting this dataset, file, snapshot, or clone?
logicalused and used are the first two questions. Confuse them and you will:
- Mis-size pools (buy too much, or worse, too little).
- Delete the wrong thing under pressure (classic “we deleted the dataset but nothing freed” moment).
- Blame ZFS for your own misunderstanding (a proud IT tradition).
One short joke, as tradition demands: ZFS storage reports are like restaurant bills—if you don’t understand the line items, you’ll accuse the waiter of theft. The waiter is just doing math you didn’t ask to see.
Definitions: used vs logicalused
What used means
used is the amount of space that dataset has allocated in the pool, measured in actual on-disk space. It includes the effects of compression, copies, parity overhead (indirectly, because allocation is in actual pool space), and it includes blocks that are only referenced by this dataset and blocks that are shared with snapshots/clones depending on which property you are reading.
In plain terms: used is physical pool allocation attributed to that dataset’s namespace, as ZFS accounts it.
What logicalused means
logicalused is the logical size of the data stored in that dataset—what you would get if you summed file sizes (or block logical sizes) before compression. It is designed to answer: “How big is the data, conceptually?” not “How much disk did it cost?”
In plain terms: logicalused is ‘apparent size’ from ZFS’s viewpoint.
Why they diverge
The delta between them is mostly explained by:
- Compression:
logicalusedstays “big,”usedshrinks. - Snapshots and clones: deletion may not free
usedif blocks are still referenced elsewhere. - Reservations: space can be “unavailable” without being “used by data.”
- Copies, recordsize/volblocksize, padding: overhead and allocation granularity can make
usedlarger than you expect.
The space accounting model (what ZFS is actually counting)
ZFS is a copy-on-write filesystem with pooled storage. That single sentence is why your intuition from traditional filesystems fails.
Blocks, references, and why deletion doesn’t always reclaim space
In ZFS, blocks are freed when the last reference goes away. A file in a dataset references blocks. A snapshot also references blocks (a point-in-time set of references). A clone references blocks too. Until every referencing object lets go, the blocks remain allocated.
So you can delete the file—and still not free a byte—because the snapshot holds the old blocks. That’s not “snapshot overhead,” it’s the snapshot doing its job: preserving old data.
“Used by dataset” vs “used by snapshots” is accounting, not physics
ZFS exposes a family of properties to break down usage. The key is to understand that some properties answer “what is attributable to this dataset” and others answer “what is exclusive to this dataset.”
If you remember one thing for on-call: used is not the same as “space freed if I destroy this dataset.” For that you want usedbydataset, usedbysnapshots, usedbychildren, usedbyrefreservation, and, often, logicalused plus logicalusedbydataset.
Compression is the cleanest reason logical and physical differ
Compression changes how many physical sectors are allocated, but it doesn’t change what applications think they wrote. Your database still wrote 1 TB of pages; ZFS might have stored 300 GB because it compressed well. That’s exactly why logicalused exists: it keeps the “what the app thinks it consumed” view alive, even when you store it cheaper.
Historical context and interesting facts
These aren’t trivia for trivia’s sake; they explain why ZFS reports what it reports.
- ZFS was designed to end filesystem repair as a ritual. Checksums, copy-on-write, and end-to-end integrity were core goals, and the accounting model followed: immutable snapshots and shared blocks are features, not edge cases.
- Early ZFS evangelism pushed “snapshots are cheap.” True for metadata and unchanged blocks. But changed blocks accrue, and space accounting had to explain that reality.
- “Pooled storage” was a philosophical shift. Traditional filesystems lived on fixed partitions/LUNs. ZFS moved the constraints to the pool, which is why datasets can have independent quotas/reservations but share the same physical allocator.
- The
logicalusedfamily arrived to make compression visible. Operators needed a way to separate “application footprint” from “disk footprint,” especially for capacity planning and chargeback. - Copy-on-write is why fragmentation and allocation overhead exist. ZFS avoids in-place overwrite; it writes new blocks and flips pointers. Space accounting therefore has to handle multiple versions of blocks and short-lived allocation churn.
- OpenZFS standardization matters. Properties like
usedby*andlogicalused*are widely available now, but older vendor builds and older illumos-era forks varied—leading to many “but my system doesn’t show that property” war stories. - Large record sizes were a deliberate performance choice. ZFS recordsize defaults (e.g., 128K) were chosen to make sequential IO efficient, but they also affect how much physical space is allocated for small writes and how compressible data becomes.
- Space accounting became a support battleground. Storage vendors and internal platform teams learned the hard way that a single “Used” column creates tickets. The detailed properties exist because operators demanded answers, not vibes.
How compression changes the story
Compression is the most common reason logicalused dwarfs used. It’s also the reason ZFS capacity planning can look like magic to people who haven’t lived through a pool filling up at 3 a.m.
Example: the database that “used 10 TB” on a 4 TB pool
I’ve seen the panicked message: “The dataset says 10 TB used but the pool is only 4 TB—ZFS is corrupt.” No, it’s compressed. logicalused tracks logical size; used tracks real allocation. If your data compresses 3:1, logical numbers can be larger than the pool capacity and still be perfectly healthy.
Compression ratio is not a promise
The moment your workload changes, your “free space” changes in a way that feels unfair. Logs compress well until they don’t. VM images compress well until someone turns on full-disk encryption inside the guest. Backups compress well until your backup software starts doing its own compression and encryption.
Capacity planning based on yesterday’s compression ratio is like budgeting based on yesterday’s weather: it works right up until you need it to work.
Second short joke: compression is like packing for a trip—everything fits until you add shoes.
Snapshots, clones, and the tax man
Snapshots are the reason ZFS can be your best friend and your most convincing liar. They don’t lie, but they do reveal how much you don’t know about “deleted” data.
Snapshots keep blocks alive
When you snapshot a dataset, you are freezing a set of block pointers. Later changes allocate new blocks; the old blocks remain referenced by the snapshot. This is why:
- Deleting a file might not reduce
usedmuch (or at all). - Destroying old snapshots can suddenly free huge amounts of space.
- “Used by snapshots” can grow quickly on churny workloads (VMs, databases, CI workspaces).
Clones are shared ownership with paperwork
A clone is a writable dataset that starts as a snapshot. It shares blocks until diverged. This complicates space reclamation because now you have multiple children referencing the same ancestry. If you destroy the origin thinking you’ll free space, you may discover the clone has become the primary owner of those blocks.
The practical consequence
In incident response, snapshots are the first place to look for “space that won’t go away.” In capacity reporting, snapshots are why you should never treat “dataset used” as equivalent to “cost.”
Reservations, quotas, refquota: the hidden contracts
Space accounting isn’t just physics; it’s policy. ZFS lets you make promises (reservations) and impose limits (quotas). Those promises and limits show up in ways that surprise people who only look at used.
Quota vs refquota (and why it matters)
quota limits a dataset and its descendants. refquota limits only the dataset’s own referenced space (not children). This distinction matters when your “project dataset” is a parent with many children.
If you want to prevent a single dataset from ballooning while allowing children to grow independently, refquota is the tool. If you want to cap an entire subtree, quota is the tool.
Reservation vs refreservation
reservation reserves space for a dataset and its descendants. refreservation reserves space only for the dataset itself. Reservations reduce what the pool reports as available even if the dataset hasn’t actually written that data yet.
This is where “why is the pool full?” incidents come from: the pool isn’t full of data; it’s full of promises.
Refreservation shows up as usedbyrefreservation
When someone sets refreservation for “performance safety” and forgets about it, it becomes a silent capacity killer. The space is allocated logically in accounting terms, making the pool look tight and causing allocation failures earlier than expected.
Practical tasks: commands + interpretation
These are the commands I actually reach for when someone says “ZFS space is wrong.” Each task includes what you’re looking for and how to interpret it.
Task 1: Compare used vs logicalused for a dataset
cr0x@server:~$ zfs list -o name,used,logicalused,avail,refer,logicalrefer -H tank/app
tank/app 320G 950G 1.2T 310G 920GInterpretation: The dataset holds ~950G of logical data but only costs ~320G of physical pool space—compression and/or block sharing is paying rent. If the pool is low, deleting this dataset might only free around used-ish (and possibly less if snapshots/clones share blocks).
Task 2: Get the usage breakdown (the argument-settler)
cr0x@server:~$ zfs list -o name,used,usedbydataset,usedbysnapshots,usedbychildren,usedbyrefreservation -H tank/app
tank/app 320G 180G 120G 0B 20GInterpretation: 120G is attributed to snapshots, 20G is tied up in refreservation. If you’re trying to reclaim space, deleting snapshots may be more effective than deleting files.
Task 3: List snapshots with space impact
cr0x@server:~$ zfs list -t snapshot -o name,used,refer,logicalused -s used -H tank/app
tank/app@daily-2025-12-20 40G 310G 920G
tank/app@daily-2025-12-21 35G 312G 925G
tank/app@daily-2025-12-22 25G 315G 930GInterpretation: Snapshot used is the space uniquely held because of that snapshot (roughly: blocks that would be freed if it were destroyed, assuming no clones). Sort by used to find the expensive ones.
Task 4: Check compression settings and observed ratio
cr0x@server:~$ zfs get -o name,property,value -H compression,compressratio,logicalused,used tank/app
tank/app compression zstd
tank/app compressratio 2.96x
tank/app logicalused 950G
tank/app used 320GInterpretation: A ~3x ratio explains the gap. If compressratio is near 1.00x but logicalused is still huge, look for snapshots/clones or allocation overhead.
Task 5: Confirm whether you’re dealing with a volume (zvol) vs filesystem
cr0x@server:~$ zfs get -H -o name,property,value type,volblocksize,recordsize tank/vm-001
tank/vm-001 type volume
tank/vm-001 volblocksize 16K
tank/vm-001 recordsize -Interpretation: Volumes have different behavior: small random writes, padding, and blocksize choices can change physical usage patterns and snapshot growth.
Task 6: Spot reservations and why “avail” is small
cr0x@server:~$ zfs get -H -o name,property,value reservation,refreservation,quota,refquota tank/app
tank/app reservation none
tank/app refreservation 50G
tank/app quota none
tank/app refquota noneInterpretation: That 50G refreservation will show up in usedbyrefreservation and reduce pool free space even if the dataset is mostly empty.
Task 7: Confirm pool health and real free space (don’t ignore slop)
cr0x@server:~$ zpool list -o name,size,alloc,free,frag,capacity,health
NAME SIZE ALLOC FREE FRAG CAPACITY HEALTH
tank 7.25T 6.70T 550G 48% 92% ONLINEInterpretation: 92% capacity is “start canceling meetings” territory. Even if datasets show “avail,” pool-level fragmentation and the allocator’s needs can make writes fail sooner than people expect.
Task 8: Show top datasets by physical used
cr0x@server:~$ zfs list -o name,used,logicalused,compressratio -S used | head -n 10
NAME USED LOGICALUSED COMPRESSRATIO
tank/vm 2.4T 2.6T 1.08x
tank/backups 1.9T 6.1T 3.21x
tank/home 980G 1.1T 1.12x
tank/app 320G 950G 2.96xInterpretation: Start with what costs physical space, not what looks big logically. The 6.1T logical backups are only 1.9T real; the VMs are expensive because they don’t compress much.
Task 9: Identify snapshot-heavy datasets quickly
cr0x@server:~$ zfs list -o name,used,usedbysnapshots,logicalused -S usedbysnapshots | head -n 10
NAME USED USEDBYSNAPSHOTS LOGICALUSED
tank/vm 2.4T 820G 2.6T
tank/app 320G 120G 950G
tank/home 980G 40G 1.1TInterpretation: usedbysnapshots is your “space pinned by history” signal. High values mean churn plus snapshots; tune retention or snapshot cadence, not just dataset size.
Task 10: Check whether a dataset has clones depending on snapshots
cr0x@server:~$ zfs get -H -o name,property,value clones tank/app@daily-2025-12-20
tank/app@daily-2025-12-20 clones tank/app-clone-qaInterpretation: You can’t destroy that snapshot without dealing with the clone. If someone says “we deleted snapshots but space didn’t free,” clones are a prime suspect.
Task 11: Estimate what you’d free by destroying a snapshot (dry run logic)
cr0x@server:~$ zfs list -t snapshot -o name,used -s used -H tank/app | tail -n 5
tank/app@daily-2025-11-30 2G
tank/app@daily-2025-12-01 3G
tank/app@daily-2025-12-02 4G
tank/app@daily-2025-12-03 9G
tank/app@daily-2025-12-04 15GInterpretation: Snapshot used is a decent first approximation of “what you might reclaim,” but remember: clones can prevent reclamation, and deleting a snapshot can shift accounting to other snapshots.
Task 12: Destroy snapshots safely with a pattern (and verify)
cr0x@server:~$ zfs destroy -n -v tank/app@daily-2025-12-0{1,2,3}
would destroy tank/app@daily-2025-12-01
would destroy tank/app@daily-2025-12-02
would destroy tank/app@daily-2025-12-03cr0x@server:~$ zfs destroy -v tank/app@daily-2025-12-0{1,2,3}
will destroy tank/app@daily-2025-12-01
will destroy tank/app@daily-2025-12-02
will destroy tank/app@daily-2025-12-03cr0x@server:~$ zfs list -o name,used,avail -H tank/app
tank/app 250G 1.3TInterpretation: Use -n first. Then execute. Then verify. In production, the difference between hero and headline is one dry run.
Task 13: Check for space “stuck” due to refreservation
cr0x@server:~$ zfs list -o name,usedbyrefreservation,refreservation -H tank/app
tank/app 20G 50Gcr0x@server:~$ zfs set refreservation=none tank/appcr0x@server:~$ zfs list -o name,used,avail -H tank/app
tank/app 230G 1.4TInterpretation: You just reclaimed “policy space.” Confirm with the owning team before removing reservations; sometimes they exist for legitimate burst tolerance.
Task 14: Compare file apparent sizes to ZFS logical numbers (sanity check)
cr0x@server:~$ df -hT /tank/app
Filesystem Type Size Used Avail Use% Mounted on
tank/app zfs 1.5T 310G 1.2T 21% /tank/appcr0x@server:~$ du -sh /tank/app
910G /tank/appInterpretation: du is a view through the filesystem namespace and can be skewed by sparse files, permissions, and live changes. Treat it as supporting evidence, not truth. ZFS properties are usually more consistent for accounting.
Task 15: If you suspect sparse files in VMs or databases
cr0x@server:~$ ls -lh /tank/vm/images/vm-001.img
-rw------- 1 root root 500G Dec 25 10:11 /tank/vm/images/vm-001.imgcr0x@server:~$ du -h /tank/vm/images/vm-001.img
120G /tank/vm/images/vm-001.imgInterpretation: That file is sparse: logical size is 500G, physical consumption is 120G. ZFS logicalused tends to align with “500G worldview,” while used aligns with “120G reality,” modulated by compression and snapshots.
Corporate-world mini-stories from production
Mini-story 1: The incident caused by a wrong assumption
The storage team got paged for “pool at 95%” on a Friday afternoon, the kind of timing that suggests the universe has a calendar and a sense of humor. The application owner pointed at logicalused: their dataset “held multiple terabytes,” therefore they were the villain. They volunteered to delete a batch of old exports. Everyone cheered; nobody checked snapshots.
Exports were deleted. du plummeted. Slack filled with thumbs-up. The pool didn’t budge. The next write-heavy job hit ENOSPC within an hour and took out an ingestion pipeline. Now it was an incident with executives, not just engineers.
We finally did the adult thing: ran zfs list -o usedby* and sorted snapshots by used. The dataset had an aggressive snapshot schedule—every 15 minutes—for “easy rollback,” and the workload was rewriting large files constantly. Snapshots weren’t cheap. They were hoarding yesterday’s blocks like a museum curator with a storage budget.
The fix wasn’t heroic. We reduced snapshot frequency, shortened retention, and destroyed a set of high-impact snapshots after confirming no clones depended on them. Space returned, ingestion resumed, and the postmortem’s key lesson was printed in bold: “Deleting files is not a space reclamation plan when snapshots exist.”
What hurt wasn’t the technical complexity; it was the wrong assumption that file deletion equals space reclamation. In ZFS, references matter more than intent.
Mini-story 2: The optimization that backfired
A platform team wanted better VM performance and decided to “optimize snapshots” by switching the VM dataset to a smaller block size and cranking snapshot frequency up, assuming that smaller blocks meant smaller deltas. The theory sounded plausible in a meeting. Theory often does.
In practice, the smaller blocks increased metadata overhead and turned writes into a confetti parade of allocations. The VM workload—lots of random writes—produced a steady stream of new blocks. Snapshots captured the old ones. Space usage from snapshots grew faster, not slower. Compression ratio also got worse because smaller, more random blocks didn’t compress as efficiently.
The first sign wasn’t a dashboard; it was latency. The pool got fragmented and busy. Then it was capacity: usedbysnapshots became the dominant term. The team tried deleting “old” snapshots, but clones from CI workflows held references, so the big frees never arrived.
The recovery plan was unglamorous: revert snapshot cadence to something sane, re-evaluate blocksize choices per workload, and stop cloning from long-retained snapshots. The true optimization was not “more knobs,” it was aligning policy (snapshots) with workload churn and understanding that logicalused is not a bill.
Mini-story 3: The boring but correct practice that saved the day
A different organization had a rule: every capacity alert had a mandatory “space accounting snapshot” attached—four commands pasted into the ticket: zpool list, zfs list -o usedby* on the top datasets, snapshot list sorted by used, and a check for reservations/quotas. It was so boring that engineers complained it was bureaucracy. It was also the reason their incidents ended quickly.
One night, a pool started climbing in usage unexpectedly. The on-call followed the checklist, discovered a single dataset with usedbyrefreservation spiking, and found that an automation job had applied a large refreservation to “protect performance” during a migration. The migration ended; the reservation stayed.
They removed the reservation after confirming it wasn’t still needed. Pool free space recovered instantly. No snapshot purges, no risky deletions, no drama. The fix was literally a property change and a note in the runbook.
The post-incident comment from the on-call was the kind you frame: “We didn’t fix storage; we fixed accounting.” That’s what most ZFS space incidents are.
Fast diagnosis playbook
This is the triage flow when someone says “we’re out of space” or “ZFS numbers don’t add up.” Run it in order; it’s designed to find the bottleneck quickly, not to satisfy curiosity.
First: is the pool actually tight?
cr0x@server:~$ zpool list -o name,size,alloc,free,capacity,frag,health
NAME SIZE ALLOC FREE CAPACITY FRAG HEALTH
tank 7.25T 6.70T 550G 92% 48% ONLINEDecision: If capacity is above ~80–85% on many pools (and especially above 90%), assume performance and allocation risk. Don’t debate dataset numbers yet; stabilize the pool.
Second: which datasets cost the most physical space?
cr0x@server:~$ zfs list -o name,used,avail,logicalused,compressratio -S used | head -n 15Decision: Focus on used for immediate relief, not logicalused. The pool can only allocate physical.
Third: is space pinned by snapshots/clones?
cr0x@server:~$ zfs list -o name,used,usedbysnapshots,usedbydataset,usedbychildren -S usedbysnapshots | head -n 15cr0x@server:~$ zfs list -t snapshot -o name,used -s used | tail -n 20Decision: If usedbysnapshots is large, look at snapshot retention/cadence and clones. Snapshot destruction is often the safest reclaim lever—after confirming dependencies.
Fourth: are reservations making “free” look smaller than it is?
cr0x@server:~$ zfs list -o name,usedbyrefreservation,refreservation -S usedbyrefreservation | head -n 15Decision: If reservations dominate, you’re not out of disk; you’re out of unpromised disk. That’s a policy discussion and often a quick fix.
Fifth: is compression masking growth risk?
cr0x@server:~$ zfs get -o name,property,value -H compressratio,compression tank | headDecision: If your plan relies on compression staying high, assume it will drop the moment the workload changes (encryption, already-compressed media, VM changes). Add headroom.
Common mistakes (symptoms and fixes)
Mistake 1: Treating logicalused as physical consumption
Symptom: “Dataset is bigger than the pool” panic, or internal chargeback that bills teams for terabytes they didn’t actually cost.
Fix: Use used (physical) for capacity and cost. Use logicalused for application footprint and growth trends. When in doubt, show both, plus compressratio.
Mistake 2: Deleting files to reclaim space when snapshots exist
Symptom: du drops, pool free space doesn’t. Users swear they deleted “hundreds of gigabytes.” They did. Snapshots kept it.
Fix: Check usedbysnapshots. List snapshots by used. Confirm clones. Destroy snapshots according to retention policy after validating restore requirements.
Mistake 3: Forgetting refreservations
Symptom: Pool “free” is low but datasets don’t seem large. usedbyrefreservation is non-trivial. New allocations fail earlier than expected.
Fix: Inventory refreservation/reservation and validate each one. Remove or right-size. Document why it exists.
Mistake 4: Ignoring clones when cleaning snapshots
Symptom: zfs destroy fails due to clones, or snapshot deletion succeeds but frees less than expected.
Fix: Check clones property on snapshots. Decide whether clones are still needed; if they are, accept the pinned space or migrate clone data elsewhere.
Mistake 5: Misreading refer vs used
Symptom: Someone says “dataset refer is small, so it’s small,” while used is huge due to children/snapshots/reservations.
Fix: Use zfs list -o usedby* and decide whether the cost is in the dataset itself, its snapshots, its children, or reservations.
Mistake 6: Running pools too full because “ZFS has snapshots and compression”
Symptom: Performance degrades, allocations become erratic, scrub/resilver times stretch, and then a small burst causes ENOSPC.
Fix: Keep headroom. Build alerting on pool capacity, not just dataset avail. Treat 90% as an emergency threshold for most production pools.
Mistake 7: Capacity planning based on a single week of compressratio
Symptom: The pool “suddenly” fills after a workload change (encryption, new data types), despite stable logical growth.
Fix: Track both logical and physical growth over time. Assume compression can regress. Use conservative ratios for forecasts.
Checklists / step-by-step plan
Space dispute checklist (end arguments, not friendships)
- Capture
zpool listfor pool-level truth: capacity, free, fragmentation. - Capture
zfs list -o used,logicalused,compressratiofor top datasets sorted byused. - For the top 3 datasets, capture
zfs list -o usedbydataset,usedbysnapshots,usedbychildren,usedbyrefreservation. - List snapshots on the top offender sorted by
used. - Check for clones on any high-
usedsnapshots. - Check reservations/quotas:
reservation,refreservation,quota,refquota. - Decide action: destroy snapshots, remove reservations, delete datasets, or expand pool.
- After changes, re-capture the same commands and attach to the ticket for closure.
Emergency reclaim plan (when you’re above 90%)
- Freeze non-essential churn: pause big CI runs, disable heavy test restores, defer batch jobs.
- Destroy the highest-impact snapshots first (after confirming no clones and restore requirements).
- Remove accidental reservations and stale quotas that are blocking writes.
- If you must delete data, target datasets with high
usedbydataset(exclusive-ish) rather than ones dominated by snapshot usage. - Once stable, adjust snapshot retention/cadence to match churn, and plan expansion with headroom.
Capacity reporting plan (stop billing wars)
- Report both physical and logical:
usedandlogicalused. - Include
compressratioand snapshot breakdown (usedbysnapshots). - For chargeback, bill on physical
used, but show logical as “demand signal.” - Review reservations quarterly; treat them as contracts with an owner.
FAQ
1) Which number should I use for capacity planning: used or logicalused?
Use used for “will the pool fill?” because it reflects physical allocation. Use logicalused for “how big is the application’s data footprint?” and for predicting how painful a compression regression might be.
2) Why does logicalused exceed the pool size?
Because it ignores compression and can reflect sparse files or data that compresses well. It’s reporting logical size, not the cost on disk. If used fits and the pool has free space, this is not an error.
3) I deleted 500 GB and nothing freed. What happened?
Snapshots (or clones) likely still reference the deleted blocks. Check usedbysnapshots, list snapshots sorted by used, and check the clones property on snapshots before you delete them.
4) Does destroying a snapshot always free its used amount?
Not always. Snapshot used is an estimate of unique space held due to that snapshot, but shared references and clones can change what is freed and how the accounting shifts to other snapshots.
5) What’s the difference between refer and used?
refer is the amount of data accessible through the dataset (roughly “what you see”). used includes space attributable to snapshots, children, and reservations depending on context. When in doubt, use the usedby* breakdown.
6) Why is avail on a dataset different from pool free space?
Dataset avail is constrained by quotas/reservations and parent limits. Pool free space is physical. Also, at high pool fullness, effective allocatable space can be less than “free” due to fragmentation and allocator needs.
7) Is a high compressratio always good?
It’s good for capacity, usually fine for performance, but it can create planning risk if you assume it will persist. A workload change can drop compression quickly, causing physical growth to accelerate.
8) How do reservations show up in “used” if no data was written?
Reservations reduce available space and appear in accounting via usedbyrefreservation (and related properties). They’re deliberate pre-allocation promises, not data blocks.
9) What’s the quickest way to find what’s pinning space?
Run zfs list -o usedbydataset,usedbysnapshots,usedbychildren,usedbyrefreservation on the biggest datasets by used. One of those line items will usually tell you where the space is stuck.
10) Can I “defragment” ZFS to get space back?
Fragmentation mostly affects performance and allocation efficiency, not accounting correctness. The way to “get space back” is to remove references (delete snapshots/clones/data) or add capacity, not to chase a mythical defrag button.
Conclusion
logicalused and used are not competing truths. They are two lenses: one tells you how big the data is in principle, the other tells you what it costs on disk. Most production fights happen when someone insists there can only be one truth.
When you’re diagnosing capacity, lead with pool reality (zpool list), then find who’s paying the physical bill (used), then ask who’s pinning history (usedbysnapshots) and who signed contracts (refreservation). Do that consistently and you’ll stop arguing about ZFS numbers—because you’ll finally be reading them.