You’re replicating ZFS like a grown-up. Snapshots, incrementals, maybe encryption, maybe a little compression.
Then a team does a “small cleanup”: a dataset rename, a subtree move, or a mountpoint reshuffle.
Replication that ran for months now dies with a cryptic “does not match incremental source” or receives into a weird place.
The good news: ZFS can handle renames and moves without drama. The bad news: it only does so if you design for it,
and if you stop assuming dataset names are identities. They are not. They’re labels.
The mental model: what breaks and why
ZFS replication is not “copy a folder over SSH.” It’s replaying filesystem transactions from one storage graph
into another, using a stream created by zfs send and applied by zfs receive.
A snapshot is a named point in time for a dataset. An incremental stream is “changes since snapshot A to snapshot B.”
If the receiver doesn’t have the exact snapshot A on the exact dataset you’re targeting, it can’t apply the diff.
Here’s the trap: people treat dataset names like stable identifiers. They script replication pipelines like:
“send pool/app to backup as backup/app.” Then someone renames pool/app to
pool/apps/app. On the source side, the snapshot lineage still exists, but it now lives under a new name.
On the target side, your pipeline still tries to apply incrementals to the old destination dataset. Boom.
ZFS does track datasets by internal GUIDs, but the send/receive workflow still depends on matching snapshot names and
selecting the right receive target. This is why “rename or move” isn’t inherently dangerous—what’s dangerous is
designing replication around names and mountpoints instead of around controlled dataset roots and predictable mapping.
Joke #1: A dataset rename is like changing your Slack display name and expecting payroll to follow it. It’s not that kind of system.
What “rename” and “move” mean in ZFS terms
- Rename changes the dataset name (e.g.,
tank/app→tank/apps/app), preserving snapshots and children. - Move is usually the same operation: you’re renaming a subtree under a new parent.
- Mountpoint change changes where it appears in the OS filesystem (
mountpointproperty), not the dataset identity. - Promotion changes clone ancestry. This can invalidate assumptions about which snapshot is “the base” for incrementals.
When replication actually breaks
Replication breaks when at least one of these happens:
- You send incrementals from snapshot X, but the receiver dataset doesn’t have snapshot X.
- You receive into a different dataset than last time (often due to scripts mapping names incorrectly after a rename).
- The receiver has diverged writes (someone mounted it read-write and changed it), so a fast-forward receive is impossible without rollback.
- You changed replication mode (e.g., from “plain” to “raw encrypted”) midstream without reseeding.
- You used
zfs recvwithout-uand mountpoint properties caused conflicts, so receive fails or mounts into production paths.
The theme: ZFS is consistent. Humans are creative.
Facts & historical context that matter in real life
- ZFS snapshots are cheap because they’re just metadata references to existing blocks; only changed blocks consume new space.
- Incremental send depends on common ancestry; it’s not “diff by name,” it’s “diff by snapshot lineage.” Missing the base snapshot is fatal.
- Dataset rename preserves snapshots and children; the snapshot objects don’t vanish just because the dataset label changed.
- Bookmarks exist to keep an incremental base without keeping a snapshot; they’re lightweight pointers to a transaction group.
- ZFS receive is transactional; a failed receive doesn’t half-apply data in the normal case. That’s why it’s such a strong primitive for automation.
- Properties can be included or excluded in replication (e.g., via
zfs send -p), which affects mountpoints and can surprise you. - Raw encrypted sends preserve encryption and do not expose plaintext to the sender host’s userspace, but they require consistent feature support.
- One-to-many replication scales better than many-to-one improvisation; ZFS streams are deterministic, but your ad-hoc mapping logic isn’t.
One operational quote that’s aged well: Hope is not a strategy.
— Gene Kranz.
Design rules: replication that survives renames
Rule 1: Pick a replication root and never repurpose it
Create a stable “replication root” dataset on both sides, and treat it like a namespace boundary.
Example: everything you replicate lives under tank/replica-src on the source and bkp/replica on the target.
Application datasets can move around in production, but replication always targets the stable root path on each system.
Practically, this means you replicate datasets into a dedicated subtree and expose them to consumers via clones, mountpoint overrides,
or separate “published” datasets. Don’t let your replication target path equal your production mountpoints.
If you do, you’re one typo away from mounting a backup dataset over a live directory. That’s not a backup strategy; it’s performance art.
Rule 2: Separate “what to replicate” from “where it lives today”
The source dataset name might change. What shouldn’t change is the intent: “this is the dataset that backs service X.”
Encode intent with a property or a mapping file, not by parsing dataset names.
A good pattern is tagging:
org.example:replicate=true and org.example:replica-name=svc-x.
Your automation discovers datasets by property, then replicates them to a fixed destination path like
bkp/replica/svc-x. Rename the source all you want; the replication identity stays stable.
Rule 3: Keep a consistent snapshot naming scheme across moves
Snapshot names are part of your contract. If you replicate incrementals, both sides need the same snapshot name
for the base and target. Pick a scheme and don’t get clever:
replica-YYYYMMDD-HHMM or replica-auto-###.
If you must change schemes, reseed with a full send once and restart incrementals. Don’t try to “bridge” with magic.
Rule 4: Use holds and bookmarks intentionally
If you rely on a base snapshot for incrementals, protect it. Use zfs hold to prevent pruning from deleting
it before the receiver has caught up. If you want to avoid keeping old snapshots around forever, use bookmarks:
they let you do incrementals from a “remembered point” without retaining the full snapshot.
Rule 5: Never let the target diverge (unless you mean it)
Your replication target should be treated as read-only. Enforce it socially (permissions), operationally (don’t mount it automatically),
and technically (consider readonly=on on received datasets). Divergence is the #1 reason you end up using
zfs recv -F under pressure, which is the ZFS equivalent of “I’ll just restart prod, what’s the worst that could happen?”
Rule 6: Decide your receive mapping strategy upfront
You have three common patterns:
- Mirror names:
tank/app→bkp/tank/app. Easy until you rename. Then you need rename-handling logic. - Stable destination names: tag source datasets and map to
bkp/replica/<service>. Best for rename/move tolerance. - Receive under a parent with
zfs recv -d: preserve subtree layout under a dedicated root. Good for bulk migration, but still name-coupled.
For “no-drama,” stable destination names win. They cost a little setup and save a lot of midnight.
Taskbook: commands, output, and decisions (12+)
These are the tasks I actually run when replication gets weird—plus what the output means and what decision it drives.
Hostnames are examples: src1 is source, bkp1 is backup.
Task 1: Identify what changed (rename/move evidence)
cr0x@server:~$ zfs list -t filesystem -o name,creation -r tank | head
NAME CREATION
tank Mon Jan 8 09:12 2024
tank/apps Tue Apr 2 13:41 2024
tank/apps/app1 Tue Apr 2 13:44 2024
tank/oldapp Wed Mar 6 10:18 2024
What it means: This gives you current names and rough timeline.
If a dataset “moved,” you’ll see a new parent path; creation time typically stays the same for the dataset itself.
Decision: If the dataset name changed, stop relying on name-based mapping. Switch to property-based mapping or explicitly rename the target.
Task 2: Confirm snapshot presence and naming on the source
cr0x@server:~$ zfs list -t snapshot -o name,creation -s creation -r tank/apps/app1 | tail -5
tank/apps/app1@replica-20240130-0200 Tue Jan 30 02:00 2024
tank/apps/app1@replica-20240131-0200 Wed Jan 31 02:00 2024
tank/apps/app1@replica-20240201-0200 Thu Feb 1 02:00 2024
tank/apps/app1@replica-20240202-0200 Fri Feb 2 02:00 2024
tank/apps/app1@replica-20240203-0200 Sat Feb 3 02:00 2024
What it means: You have a consistent snapshot series.
Decision: Pick the last successfully replicated snapshot as your incremental base. If you can’t identify it, inspect the target.
Task 3: Confirm snapshot presence on the target dataset you think is correct
cr0x@server:~$ ssh bkp1 zfs list -t snapshot -o name -r bkp/replica/app1 | tail -5
bkp/replica/app1@replica-20240130-0200
bkp/replica/app1@replica-20240131-0200
bkp/replica/app1@replica-20240201-0200
bkp/replica/app1@replica-20240202-0200
bkp/replica/app1@replica-20240203-0200
What it means: Target has the same snapshots, so incrementals should work.
Decision: If snapshots don’t match, either you’re receiving into the wrong dataset or you need a reseed/full send.
Task 4: Check for divergence (writes on target)
cr0x@server:~$ ssh bkp1 zfs get -H -o property,value,source readonly bkp/replica/app1
readonly off local
What it means: The target is writable. That’s not proof it diverged, but it’s an invitation.
Decision: Set readonly=on on replication targets unless you have a deliberate failover workflow.
Task 5: Find the last common snapshot quickly
cr0x@server:~$ comm -12 \
<(zfs list -H -t snapshot -o name -r tank/apps/app1 | sed 's/^.*@/@/' | sort) \
<(ssh bkp1 zfs list -H -t snapshot -o name -r bkp/replica/app1 | sed 's/^.*@/@/' | sort) | tail -3
@replica-20240201-0200
@replica-20240202-0200
@replica-20240203-0200
What it means: The snapshot names match across source/target. The last line is your incremental base candidate.
Decision: Use that base for zfs send -i (or -I if you’re sending a range including intermediate snapshots).
Task 6: Dry-run the send size estimate (sanity check)
cr0x@server:~$ zfs send -nPv -i tank/apps/app1@replica-20240203-0200 tank/apps/app1@replica-20240204-0200
send from @replica-20240203-0200 to tank/apps/app1@replica-20240204-0200 estimate: 1.42G
total estimated size is 1.42G
What it means: You’re about to ship ~1.4 GiB. If you expected 10 MiB, something’s wrong (or your app got busy).
Decision: If the estimate is huge unexpectedly, check for churn (logs, tmp, databases) and consider excluding or rethinking dataset layout.
Task 7: Perform the incremental replication with explicit receive behavior
cr0x@server:~$ zfs send -w -i tank/apps/app1@replica-20240203-0200 tank/apps/app1@replica-20240204-0200 | \
ssh bkp1 zfs recv -u bkp/replica/app1
What it means: -w sends raw (encrypted if the dataset is encrypted). -u prevents auto-mounting on the backup host.
If this fails, you’ll get a specific error; don’t “fix” it with -F until you understand the mismatch.
Decision: If receive fails due to missing base snapshot, stop and reconcile datasets. If it fails due to mountpoint conflicts, keep -u and fix properties.
Task 8: Verify the receive actually landed where you think
cr0x@server:~$ ssh bkp1 zfs list -o name,used,refer,mountpoint bkp/replica/app1
NAME USED REFER MOUNTPOINT
bkp/replica/app1 8.21G 8.21G /bkp/replica/app1
What it means: Dataset exists and mountpoint is what the target thinks it should be.
Decision: If mountpoint points into a production path, stop and set mountpoint=none or receive with -u and adjust before mounting.
Task 9: Inspect receive-side errors and partial state
cr0x@server:~$ ssh bkp1 zpool status -x
all pools are healthy
What it means: Pool isn’t degraded. If replication is slow or failing, the problem is likely not a vdev actively dying.
Decision: If zpool status shows resilvering, errors, or a degraded vdev, fix storage health before blaming ZFS send/recv.
Task 10: Find hidden blockers: holds that prevent pruning
cr0x@server:~$ zfs holds tank/apps/app1@replica-20240203-0200
NAME TAG TIMESTAMP
tank/apps/app1@replica-20240203-0200 repl-base Sat Feb 3 02:05 2024
What it means: A hold exists. Great when intentional; annoying when forgotten.
Decision: Keep holds on the last replicated base until the target confirms receipt. Release holds as part of a post-replication hook.
Task 11: Use a bookmark as an incremental anchor (when you want to prune snapshots)
cr0x@server:~$ zfs bookmark tank/apps/app1@replica-20240203-0200 tank/apps/app1#repl-anchor
cr0x@server:~$ zfs list -t bookmark -o name -r tank/apps/app1
tank/apps/app1#repl-anchor
What it means: The bookmark exists. It can serve as the “from” side of incrementals even if you later destroy the snapshot.
Decision: Use bookmarks when retention is aggressive but replication still needs a stable base.
Task 12: Check feature flags and encryption compatibility (avoid ugly surprises)
cr0x@server:~$ zpool get -H -o name,property,value feature@encryption tank
tank feature@encryption active
cr0x@server:~$ ssh bkp1 zpool get -H -o name,property,value feature@encryption bkp
bkp feature@encryption active
What it means: Both pools support encryption features. Raw sends won’t fail due to missing feature support.
Decision: If the target lacks needed features, either upgrade or avoid raw/encrypted workflows until you can align capabilities.
Task 13: Diagnose throughput limits during replication
cr0x@server:~$ iostat -xz 1 3
Linux 6.6.0 (src1) 02/04/2026 _x86_64_ (16 CPU)
avg-cpu: %user %nice %system %iowait %steal %idle
12.10 0.00 3.20 18.40 0.00 66.30
Device r/s w/s rkB/s wkB/s await svctm %util
nvme0n1 82.0 120.0 98000.0 62000.0 8.20 0.45 91.0
What it means: High %util and elevated await suggest storage is the bottleneck (or at least heavily exercised).
Decision: If disks are pegged, throttle replication, schedule it off-peak, or adjust recordsize/compression for the workload.
Task 14: Validate that you’re not accidentally changing properties via replication
cr0x@server:~$ zfs get -H -o property,value,source mountpoint,canmount tank/apps/app1
mountpoint /var/lib/app1 local
canmount on default
cr0x@server:~$ ssh bkp1 zfs get -H -o property,value,source mountpoint,canmount bkp/replica/app1
mountpoint /bkp/replica/app1 local
canmount noauto local
What it means: The backup dataset won’t auto-mount (noauto). That’s healthy for a replica.
Decision: Force backup targets to be inert by default: canmount=noauto, controlled mounts only during restore tests.
Handling renames and dataset moves (the safe patterns)
Pattern A: Property-based identity with stable destination names (recommended)
This is the “adult in the room” approach. You tag the dataset with an identity that does not change when renamed,
then replicate to a stable destination dataset named after that identity.
How it works:
- On the source dataset (wherever it currently lives), set a property like
org.example:replica-name=app1. - Your replication tool discovers datasets by property, not by path.
- All streams land in
bkp/replica/app1, regardless of whether the source istank/app1ortank/apps/app1.
Operational benefit: Renames don’t matter. Moves don’t matter. Only lineage matters, and you keep that stable.
What can still break: promotion/clone tricks, encryption mode changes, or letting the target diverge.
Pattern B: If you mirror names, treat rename as a replicated event
Some environments want the destination path to mirror the source path for browsing convenience.
Fine. But then you must replicate renames as a first-class change.
ZFS doesn’t ship “rename events” as separate things. Your automation must detect renames (or moves)
and apply matching renames on the target before sending incrementals.
Minimal approach:
- Maintain a mapping file of source dataset GUID → destination dataset path.
- On each run, compare current source name for that GUID. If changed, rename the target dataset accordingly.
- Then run incrementals as usual.
This works because renaming a dataset does not change its snapshots or GUID. Your job is just to keep the receiver
dataset path aligned with where your automation expects it.
Pattern C: Use zfs recv -d under a stable root for subtree moves
If you want to preserve relative names but not absolute names, receive into a stable root with -d.
Example: source sends tank/apps/app1, target receives under bkp/replica with -d,
creating bkp/replica/tank/apps/app1 or similar depending on the send stream and target options.
This can be good for “migration day” or bulk replication, but it’s still name-coupled. If the source path changes,
you’re now creating a new path on the target unless you also rename the old one.
What to do during a rename window
If a rename/move is planned, you can make it boring:
- Pause replication briefly (or at least stop automatic pruning).
- Take a snapshot right before rename:
@replica-pre-rename. - Rename/move dataset.
- Confirm snapshots still exist under new name.
- Update mapping/property if needed.
- Resume replication using the last common snapshot.
The point is not “avoid renames.” The point is “avoid renames plus automation that assumes names never change.”
Joke #2: If your replication script uses cut -d/ -f2 to identify datasets, it’s not automation—it’s a cry for help.
Three corporate mini-stories from the trenches
Mini-story 1: The incident caused by a wrong assumption
A mid-size company ran ZFS replication from a production filer to a backup filer every 15 minutes. The replication script was
straightforward: it listed datasets under tank/prod, then mirrored the same path under bkp/prod.
Everyone thought of the dataset path as the identity of the data. It had been stable for years, so why not?
A platform team reorganized the dataset tree to match a new org chart. Nothing malicious, nothing rushed:
tank/prod/finance became tank/prod/business/finance. The datasets were renamed properly with ZFS,
snapshots and children intact. They even sent out a change notice. Replication, however, had no idea that
finance had “moved.” It tried to send incrementals for tank/prod/business/finance into
bkp/prod/business/finance, which didn’t exist yet. So it created it, received a full baseline, and moved on.
The backup filer now had two copies: the old bkp/prod/finance stuck at yesterday’s last snapshot, and the new
bkp/prod/business/finance with today’s snapshots. Nobody noticed because the monitoring only checked
that “replication job succeeded” and that “latest snapshot exists somewhere.”
Then a restore was needed. The on-call grabbed the obvious dataset name—bkp/prod/finance—and found it stale.
Panic, escalation, war room, blame. The data was there, just under a new path. The restore succeeded after a delay,
and the real lesson landed hard: the dataset path was never the identity; it was a convenience label.
The fix was boring: property-based mapping to a stable destination name. They kept the mirrored browsing tree for humans,
but replication targeted a stable “service ID” dataset and published readable clones for convenience.
It removed an entire class of silent failure.
Mini-story 2: The optimization that backfired
A larger enterprise wanted replication to be “fast enough to ignore.” Someone tuned the pipeline:
parallel sends, heavy compression in userspace, and aggressive snapshot pruning to reduce dataset counts.
It looked good in the first few tests—benchmarks love empty disks and calm workloads.
The first backfire was CPU. Compression in the pipe fought with encryption, checksums, and the normal load.
Replication moved faster, but everything else moved slower. They also hit weird latency spikes because the replication
jobs aligned with application batch processing. Users complained. The fix should have been “schedule it,” but the team
doubled down: more parallelism, more tuning.
The second backfire was more subtle. Snapshot pruning became aggressive enough that the “last common snapshot”
sometimes disappeared before the receiver got the incrementals (network hiccup, maintenance window, whatever).
The system then fell back to full sends. Full sends were huge, so they took longer, so more incrementals were missed,
so more full sends happened. It was a self-inflicted positive feedback loop. The backup system wasn’t broken; it was
being asked to sprint while someone kicked its ankles.
The final backfire was operational complexity. When a dataset was renamed, the parallel pipeline tried to “auto-heal”
by creating new destination datasets and reseeding. That hid the real problem (a mapping change) behind “success.”
Restores became a scavenger hunt.
They ended up removing most of the “optimization” and replacing it with two things: holds/bookmarks to guarantee an
incremental base, and a hard rule that replication targets must be stable and inert (canmount=noauto, readonly=on).
Performance improved because the system stopped doing full sends unnecessarily. The cheapest optimization is often not
doing extra work.
Mini-story 3: The boring but correct practice that saved the day
A company with regulatory requirements ran weekly restore tests. Not “we mounted it once last year,” but a documented run:
receive dataset stays unmounted, they clone a snapshot into a temporary dataset, set a safe mountpoint, mount it,
run application-level integrity checks, then destroy the clone. Every week. Same steps. Same paper trail.
It was so boring that it became a standing joke. But it forced discipline: replication targets stayed read-only,
no one used them as scratch space, and the team constantly exercised the exact commands they would need during an incident.
They also noticed small breakages early: a snapshot naming mismatch, a prune policy that got too aggressive, a receive
that started failing due to a property change.
Then an actual incident hit: a ransomware event that forced rapid restore for a specific service.
The service dataset had been renamed two weeks prior during a cleanup. The production name changed, but the replication identity
property stayed the same. The backup dataset was exactly where the runbook expected it: bkp/replica/svc-payments.
Restore was not “easy,” but it was predictable. They cloned the last good snapshot, mounted it in an isolated path,
validated data, then promoted it into service. The rename never mattered. The weekly restore test had already proven
the mapping and the steps. Boring practices don’t get applause. They do get your weekend back.
Fast diagnosis playbook (find the bottleneck fast)
When replication slows down or fails, don’t start randomly toggling flags. You’ll end up with a mess that “works” but can’t restore cleanly.
Run this in order. The goal is to identify whether the bottleneck is lineage, mapping, storage health,
CPU, or network.
First: prove lineage and mapping (most failures)
- Confirm you’re receiving into the intended dataset: list target snapshots for the destination path and confirm it contains the expected series.
- Find the last common snapshot name using a quick intersection (see Task 5). If there is none, you’re reseeding.
- Check for divergence: is the target writable, mounted, or used by anything? If yes, assume divergence until proven otherwise.
Second: check pool health and disk pressure
- Pool health:
zpool status -xmust be clean. If not, stop treating replication as the primary issue. - Disk saturation:
iostat -xzand ZFS-specific stats (if available on your OS) to see if you’re I/O bound. - Space headroom: low free space breaks replication in dumb ways and makes fragmentation worse.
Third: check CPU and network
- CPU: if you’re using compression in the pipe, encryption, or heavy checksumming, watch CPU steal and user time.
- Network: verify throughput and packet loss. Replication streams are sensitive to flaky links because retries cost time and snapshot bases can age out.
- Latency spikes: align replication schedules away from application batch jobs, scrubs, and resilvers.
Decision outcomes
- If lineage/mapping is wrong: fix the dataset mapping (rename target dataset, change property mapping), don’t “force receive.”
- If storage is unhealthy: repair/replace, then retry replication; forcing receives onto unstable pools is how you create second incidents.
- If resources are saturated: throttle replication, reduce parallelism, or schedule it. “Faster” is not “better” if it breaks RPO.
Common mistakes: symptom → root cause → fix
1) “cannot receive incremental stream: most recent snapshot does not match”
Symptom: Receive fails when applying an incremental stream.
Root cause: The destination dataset does not have the exact base snapshot the stream expects (wrong target, pruned base, or divergent history).
Fix: Identify last common snapshot (Task 5). If none, reseed with a full send into the correct destination. Stop pruning bases before replication confirms success.
2) Incrementals suddenly became full sends after a rename
Symptom: After a dataset move, your system “helpfully” creates a new target dataset and runs full baselines.
Root cause: Name-based mapping created a new destination path. The old destination still contains the lineage.
Fix: Rename the destination dataset to match expected mapping, or—better—switch to stable destination names based on properties.
3) Backups are “successful,” but restores are stale or missing data
Symptom: Jobs report success; restore attempts find old snapshots.
Root cause: Replication is landing in a different dataset than your restore runbook expects (usually due to rename/move or multiple destinations).
Fix: Enforce a single authoritative destination per service identity. Validate “latest snapshot exists in dataset X,” not “exists somewhere.”
4) Receive fails with mountpoint or busy filesystem issues
Symptom: zfs recv errors about mounting, non-empty directories, or mountpoint conflicts.
Root cause: Receiving properties (or default mount behavior) tries to mount into paths that exist or are in use.
Fix: Receive with -u. On the target, set canmount=noauto and safe mountpoint (or mountpoint=none) for replica datasets.
5) Replication got slower over time, then started missing RPO
Symptom: Same data, same link, but replication windows creep longer.
Root cause: Churn increased (logs, temp, databases), snapshots retained longer than expected, or pool fragmentation and free-space pressure increased.
Fix: Move high-churn data into its own dataset with different snapshot schedule, keep free space headroom, and stop sending more often than you can finish.
6) Someone used the replica for “quick analytics” and now incrementals won’t apply
Symptom: Incrementals fail; receive suggests rollback/force.
Root cause: Target diverged due to local writes.
Fix: Policy: replica datasets are read-only and not mounted by default. If already diverged, decide: destroy and reseed, or zfs recv -F only if you accept losing target-side changes.
7) Snapshot pruning breaks replication intermittently
Symptom: Some runs work, some fail with missing base.
Root cause: Retention deletes the base snapshot before the remote side confirms it received it (especially after network interruptions).
Fix: Use holds on base snapshots until replication success. Consider bookmarks as anchors to avoid long snapshot retention.
Checklists / step-by-step plan
Checklist A: Make replication rename-proof (property-based mapping)
- Pick a stable destination root on backup:
bkp/replica. - For each service dataset, set identity properties on the source:
org.example:replicate=trueorg.example:replica-name=<service-id>
- On backup, create destination datasets matching service IDs (once):
bkp/replica/<service-id>. - Force safe properties on backup targets:
canmount=noauto,readonly=on,mountpoint=none(or a safe path). - Standardize snapshot names:
replica-YYYYMMDD-HHMM. - Implement holds for the last replicated base; release holds after successful receive.
- Add a restore test workflow using clones (not mounting the replica directly).
Checklist B: Planned rename or dataset move (do it without breaking incrementals)
- Confirm last successful replicated snapshot on both sides.
- Place a hold on that snapshot (source side) to prevent pruning during the change window.
- Take a pre-change snapshot (
@replica-pre-rename). - Rename/move dataset with
zfs rename(and ensure children follow as intended). - Verify snapshots exist under the new dataset name.
- If you mirror names on the target, rename the target dataset to match; otherwise, ensure properties still map to the same destination ID.
- Resume replication from the last common snapshot; verify with a dry-run size estimate.
- Release holds once the new snapshot is confirmed on the receiver.
Checklist C: When you must reseed (full send) safely
- Stop incrementals for that dataset.
- Create a fresh baseline snapshot on source.
- Receive into a new dataset path (temporary) to avoid clobbering known-good replicas.
- After success, switch mapping to the new dataset (or rename it into place), then destroy the old replica per policy.
- Restart incrementals from the baseline snapshot.
FAQ
1) Does renaming a dataset delete snapshots or break snapshot GUIDs?
No. A rename changes the dataset’s name in the namespace. Snapshots move with it. What breaks is your automation if it expects the old path.
2) Why does incremental replication care so much about the base snapshot?
Because an incremental stream is literally “apply changes from snapshot A to snapshot B.” If the receiver doesn’t have snapshot A in the target dataset, it can’t apply the delta safely.
3) Should I use zfs recv -F to fix mismatches?
Only if you accept that the receiver will roll back/destroy newer snapshots (and possibly local changes) to match the stream.
It’s a tool, not a default. Use it when you’re correcting a controlled replica, not when you’re unsure where the stream is landing.
4) Can I replicate a dataset that got promoted from a clone?
You can, but promotion changes ancestry. Your existing incremental base may no longer match.
Expect to reseed or carefully choose a new common snapshot chain.
5) How do bookmarks help with renames and pruning?
Bookmarks don’t care about mountpoints and they’re cheap. They let you keep an incremental anchor even if you delete old snapshots.
They don’t solve name-mapping problems by themselves, but they prevent “base snapshot vanished” outages.
6) Should replica datasets be mounted?
Not by default. Keep them inert: canmount=noauto and receive with -u. For restores, clone a snapshot and mount the clone in a safe location.
7) Do I replicate properties like mountpoint and readonly?
Be selective. Replicating mountpoint into a backup host is a classic foot-gun. Prefer setting safe local properties on the receiver and avoid inheriting production mount behavior.
8) If my source dataset moves under a different parent, do I need a new full send?
Not inherently. The lineage is still there. If your destination mapping stays stable (property-based ID), you can keep sending incrementals normally.
If your mapping is name-based, you’ll likely create a new destination path and accidentally force a reseed.
9) How do I keep “human-friendly browsing” on the backup without coupling replication to names?
Replicate into stable identity datasets (service IDs). Then create read-only clones or secondary datasets with friendly names for browsing.
Humans get a tree; automation gets stability.
10) What’s the single best habit to prevent replication surprises?
Weekly restore tests using clones, with a runbook that specifies exact datasets and snapshot names. It exposes mapping drift early—before you need it during an incident.
Next steps you can actually do this week
- Pick a replication identity scheme: property-based mapping to a stable destination name per service. Implement it for one dataset first.
- Make replica datasets inert: set
canmount=noauto, receive with-u, considerreadonly=on. - Standardize snapshot names and stop allowing multiple naming schemes in the same environment.
- Add holds or bookmarks so pruning can’t delete the incremental base before replication completes.
- Write a restore test runbook that mounts only clones, never the replica dataset directly, and run it on a schedule.
- Instrument the right signals: last common snapshot per service ID, replication lag, and “received into the expected dataset,” not just exit codes.
Renames and dataset moves are normal. Your job is to make them boring. ZFS will cooperate—as long as you stop treating dataset names like destiny.