Ports with Missing Features: When “It’s There” Doesn’t Mean It Works

What this problem really is (and why it keeps happening)

“Port” is an overloaded word. Sometimes it means “it compiles and passes unit tests.” Sometimes it means “it can be built in our distro.” Sometimes it means “we exposed the API, but the implementation is a no-op.” And sometimes it means “it works if you squint and avoid half the features.”

In production, a port with missing features is usually one of these:

• Build-time presence without runtime capability. The binary includes code, but the kernel, filesystem, hardware, or permissions block the path.
• Partial implementation. The “happy path” works, the edge conditions don’t, and those edges are exactly what production hits: failures, congestion, resync, rollback.
• Feature gate mismatch. The switch exists, but enabling it doesn’t activate because another prerequisite is missing (kernel config, module option, sysctl, firmware).
• Silent fallback. The system claims success, but drops back to a slower or less safe mode. You only notice after the first incident—or the first bill.
• Version and ABI drift. A port matches upstream headers but not upstream behavior; compatible enough to compile, different enough to break semantics.

Why it keeps happening is simpler than we’d like to admit: ports are often funded to reach “it runs,” not “it behaves.” Testing focuses on functional correctness, not negative-space correctness: what happens when the feature is missing, half-present, or present-but-incompatible.

And the worst category is “documentation correctness” without “operational correctness.” If your feature page ends at “set enable=true,” congratulations—you’ve written a press release, not an operations guide.

One dry operational truth: feature parity is rarely binary. It’s a matrix: kernel version × distro patches × libc × filesystem × firmware × security model × orchestration layer × workload profile. A port might be “there” in six of those dimensions and missing in the seventh. Guess which one production finds.

Joke #1: A feature that only works in the lab is called “a demo.” In production we call it “an incident rehearsal.”

The “ported” trap: compatibility claims are not performance claims

A common misread: “Supported” gets interpreted as “fast,” “safe,” or “equivalent.” Vendors and internal platform teams often mean a narrower promise: “won’t crash immediately” or “works with default settings.”

For storage and reliability work, that’s not enough. You need to answer:

• Does it preserve durability semantics (fsync, barriers, cache flush) under power loss?
• Does it handle degraded mode correctly (multipath failover, RAID rebuild, object store backfill)?
• Does it keep latency SLOs under compaction, GC, resync, and snapshotting?
• Does it expose observability (counters, tracepoints, logs) so you can prove it works?

If you can’t prove it under stress, you don’t have a feature. You have an opinion.

A few facts and historical context worth remembering

These aren’t trivia for trivia’s sake. They explain why “it’s there” so often means “not really.”

1. POSIX compatibility has always been a spectrum. Different UNIX flavors historically “supported POSIX” while diverging on edge cases like signals, I/O semantics, and file locking.
2. Linux NFS implementations evolved unevenly across versions. Features like NFSv4 delegations and idmapping have had long periods where they existed but were operationally brittle, especially across mixed client/server versions.
3. EXT4’s journaling options became a compatibility minefield. Modes like data=ordered, barriers, and writeback behavior changed with kernel improvements and device cache flush correctness.
4. NVMe and SCSI multipath histories are different. dm-multipath was built for SCSI-era behavior; NVMe introduced ANA and native multipath semantics, and “it’s multipath” doesn’t mean it fails over the way you expect.
5. ZFS feature flags were designed to avoid pool upgrade traps. That’s a good system, but it means “ZFS is available” still doesn’t guarantee “this pool is compatible everywhere you plan to import it.”
6. Containers made filesystem semantics user-visible again. Overlay filesystems and union mounts surface edge-case behavior (rename, xattrs, whiteouts) that many apps never tested outside CI.
7. glibc vs musl isn’t just size and licensing. Differences in DNS resolution behavior, thread stack defaults, locale, and error codes can change runtime behavior without changing your code.
8. Crypto stacks have a long tradition of “builds fine, fails weird.” Between OpenSSL versions, providers, FIPS modes, and kernel crypto, algorithms may appear present but be disabled by policy or missing acceleration.
9. Network offloads have shipped broken more than once. TSO/GSO/GRO, checksum offload, and segmentation features can be “supported” by a NIC but buggy in a specific driver+firmware combination.

Failure modes: how “ported” features fail in real systems

1) The no-op configuration: flag set, nothing changes

This is the classic: a config option exists because upstream has it, and your port carried the configuration schema. But the underlying module wasn’t built with the required dependency, or the runtime environment blocks it.

Examples: enabling TRIM/discard in a VM where the hypervisor doesn’t pass it; enabling asynchronous I/O in a libc/kernel combination where the code silently uses sync I/O; enabling “encryption” where only key management exists but the cipher layer is absent or policy-disabled.

2) “Supported” but only on one code path

Ports often implement the common case and skip the ugly parts: recovery, error handling, and concurrency. It looks fine until you hit a retry storm, a leader election, or a disk returns medium errors.

3) Feature present, semantics different

This one is dangerous because you won’t see an obvious failure. You get subtly wrong outcomes: different ordering guarantees, different fsync behavior, different timeouts, different locking.

In storage terms, semantics matter more than speed. A fast filesystem that lies about durability is not “fast.” It’s just “quietly optimistic.”

4) Silent fallback to “compat mode”

Some systems attempt a feature, fail to negotiate it, and keep going. That’s user-friendly—until it becomes an SLO issue. Your app continues to run, but now you’re on a slower algorithm, older protocol version, or safer-but-costlier mode.

5) Observability gap: you can’t prove it’s working

Ports sometimes miss tracepoints, counters, or structured logs. The feature might work, but you can’t confirm it. When things go wrong, you have no diagnostic breadcrumbs. That’s functionally equivalent to “not supported,” because you can’t operate it.

6) Performance cliffs under real workloads

The port might pass correctness tests and even basic benchmarks, then fall off a cliff with realistic I/O patterns: small random writes, mixed reads/writes, metadata-heavy workloads, bursty sync writes, or concurrent snapshots.

There’s a paraphrased idea often attributed to W. Edwards Deming: Without data, you’re just another person with an opinion. In ops terms: without measurement, you’re just another person with a pager.

Joke #2: The fastest storage system is the one that drops writes. It’s also the one your auditors will remember.

Fast diagnosis playbook: find the bottleneck before you argue

When a “ported feature” doesn’t behave, people immediately debate architecture. Don’t. First, establish reality in three passes: negotiate, observe, and validate.

First: prove negotiation (is the feature actually enabled end-to-end?)

• Check runtime flags (not config files): sysfs, procfs, driver info, filesystem mount options.
• Check versioned capability sets: ZFS feature flags, NFS protocol versions, TLS cipher lists, kernel config options.
• Check the “other side”: server vs client, hypervisor vs guest, controller vs initiator.

Second: observe behavior (is it taking the intended code path?)

• Use counters and tracing: iostat, perf, bpftrace, zpool iostat, nfsstat, ethtool stats.
• Look for fallback evidence: log messages, negotiated protocol downgrades, “using safe mode” banners, kernel dmesg warnings.
• Measure latency distribution: p95/p99 tells you about cliffs; average hides them.

Third: validate semantics (does it do what you thought it does?)

• Durability tests: fsync behavior under power-loss simulation is hard, but you can at least validate flush commands, barriers, and cache policy.
• Failure-mode tests: pull a path, kill a node, corrupt a block, fill the disk. If the port doesn’t behave under stress, it doesn’t behave.
• Compatibility tests: import/export pools, mount from mixed clients, upgrade/downgrade components.

If you do these three passes, you’ll stop arguing about feelings and start making decisions with evidence: “this port lacks feature X in kernel Y,” or “the feature is negotiated but falls back under load,” or “the semantics differ; we must gate it per workload.”

Practical tasks: 12+ commands that tell you what’s actually supported

These are the kinds of checks you can run during an incident, during a migration, or before you let a “ported” platform touch customer data. Each task includes what the output means and the decision it drives.

Task 1: Identify the real kernel and build flavor

cr0x@server:~$ uname -a
Linux server 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-01-23) x86_64 GNU/Linux

What it means: Kernel version and distro build matter because ports often target “Linux” broadly but rely on specific backports or configs.

Decision: If the feature depends on a specific kernel range, stop guessing. Pin the kernel or move to a known-good build.

Task 2: Check kernel config for a supposedly “supported” feature

cr0x@server:~$ zgrep -E 'DM_MULTIPATH|NVME_MULTIPATH|BTRFS_FS|OVERLAY_FS' /proc/config.gz
CONFIG_DM_MULTIPATH=m
CONFIG_NVME_MULTIPATH=y
CONFIG_BTRFS_FS=m
CONFIG_OVERLAY_FS=y

What it means: The kernel can only provide what it was built to provide. “Supported by distro” sometimes means “optional module you didn’t install.”

Decision: If a required option is missing, there is no point tuning user space. Fix the kernel/modules first.

Task 3: Confirm module presence and parameters (runtime, not theory)

cr0x@server:~$ lsmod | egrep 'nvme|dm_multipath|zfs|overlay'
nvme_fabrics           24576  0
nvme_core             200704  2 nvme_fabrics,nvme
overlay               155648  2

What it means: If the module isn’t loaded, the feature is not active. Simple. Brutal.

Decision: Load the module and validate behavior. If it won’t load, capture dmesg and stop claiming the feature exists.

Task 4: Verify filesystem mount options and actual filesystem type

cr0x@server:~$ findmnt -no SOURCE,FSTYPE,OPTIONS /var/lib/postgresql
/dev/nvme0n1p2 ext4 rw,relatime,errors=remount-ro,data=ordered

What it means: Many “features” are mount options (barriers, discard, noatime) and they can be missing or overridden.

Decision: If the expected option is absent (e.g., discard), either enable it explicitly or run periodic trim—don’t assume it happens.

Task 5: Check whether discard/TRIM is actually supported by the block device

cr0x@server:~$ lsblk -D -o NAME,ROTA,DISC-GRAN,DISC-MAX,DISC-ZERO
NAME        ROTA DISC-GRAN DISC-MAX DISC-ZERO
nvme0n1        0      512B       2G         0
nvme0n1p2      0      512B       2G         0

What it means: If DISC-MAX is 0B, discard isn’t supported through this stack. That can happen with some RAID controllers, hypervisors, or misconfigured devices.

Decision: Don’t enable discard options that do nothing; they can add overhead or false confidence. Use a storage path that propagates discard if you need it.

Task 6: Confirm write cache and flush behavior exposure

cr0x@server:~$ sudo hdparm -W /dev/sda
/dev/sda:
 write-caching =  1 (on)

What it means: Write cache being on isn’t inherently bad, but it changes your durability story. The important question is whether flushes are honored all the way down.

Decision: If you can’t guarantee proper cache protection (BBU, PLP), you must treat fsync semantics as suspect and adjust your storage design.

Task 7: Measure I/O latency and saturation quickly

cr0x@server:~$ iostat -xz 1 3
Linux 6.1.0-18-amd64 (server) 	01/22/2026 	_x86_64_	(16 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.21    0.00    1.10    8.42    0.00   87.27

Device            r/s     w/s   rkB/s   wkB/s  rrqm/s  wrqm/s  %util  await
nvme0n1        120.0   980.0  4800.0 64000.0     0.0     0.0   92.1   14.3

What it means: High %util and rising await indicate device saturation or queueing. If your “ported” feature was supposed to reduce latency, it isn’t.

Decision: If the device is saturated, stop blaming the application. Fix I/O path, queue depth, scheduler, or provisioning.

Task 8: Confirm I/O scheduler and queue settings (common port mismatch)

cr0x@server:~$ cat /sys/block/nvme0n1/queue/scheduler
[none] mq-deadline kyber bfq

What it means: Some schedulers behave better for certain workloads; some “ported performance tuning guides” assume a scheduler that isn’t in use.

Decision: If your workload is latency-sensitive and you’re on an ill-suited scheduler, test changes—but record them and gate them per device class.

Task 9: Check negotiated TLS capabilities (crypto ports love silent downgrades)

cr0x@server:~$ openssl version -a
OpenSSL 3.0.11 19 Sep 2023 (Library: OpenSSL 3.0.11 19 Sep 2023)
built on: Tue Oct 10 10:10:10 2023 UTC
platform: debian-amd64

What it means: OpenSSL major versions change provider behavior. Something “supported” in 1.1.1 might require configuration in 3.x.

Decision: If crypto behavior differs across environments, treat it as a port risk. Validate negotiated ciphers and protocols, don’t assume.

Task 10: Verify NFS protocol and client/server capabilities

cr0x@server:~$ nfsstat -m
/var/lib/app from nfs01:/export/app
 Flags: rw,vers=4.1,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=10.0.2.10

What it means: You can see the negotiated NFS version. If you thought you were on 4.2 with certain semantics and you’re on 4.1, your “feature” might not exist.

Decision: Force the intended version and test. If the server can’t do it, stop expecting client-side magic.

Task 11: Validate NVMe multipath and ANA state

cr0x@server:~$ sudo nvme list-subsys
nvme-subsys0 - NQN=nqn.2014-08.org.nvmexpress:uuid:2f2a...
\
 +- nvme0 fc traddr=nn-0x500a098... host_traddr=nn-0x500a098... live optimized
 +- nvme1 fc traddr=nn-0x500a098... host_traddr=nn-0x500a098... live non-optimized

What it means: Multipath “exists” only if paths are visible and states make sense. “Non-optimized” paths might be standby or used incorrectly depending on policy.

Decision: If paths aren’t present or states are wrong, fix fabric zoning, host configuration, or kernel multipath settings before tuning applications.

Task 12: Catch silent filesystem fallbacks in container storage

cr0x@server:~$ docker info | sed -n '/Storage Driver/,$p' | head -n 8
 Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
 userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd

What it means: Overlay behavior depends on backing filesystem features. If d_type support is missing, overlay can misbehave. If userxattr is false, certain security/metadata features may be unavailable.

Decision: If critical overlay features aren’t supported, move the runtime to a filesystem that supports them, or switch storage driver and accept the tradeoffs.

Task 13: ZFS pool feature flags and compatibility reality

cr0x@server:~$ zpool get -H -o name,property,value ashift,autotrim,feature@async_destroy rpool
rpool	ashift	12
rpool	autotrim	off
rpool	feature@async_destroy	active

What it means: ZFS feature flags are explicit. A pool can be imported but still have active features that older ports can’t handle safely.

Decision: If you need cross-host portability, control pool upgrades and feature activation. “It imports” is not the same as “it’s safe everywhere.”

Task 14: Check for runtime warnings that indicate stubs or disabled paths

cr0x@server:~$ dmesg -T | egrep -i 'fallback|disable|unsupported|unknown|deprecated' | tail -n 10
[Wed Jan 22 00:12:44 2026] nvme nvme0: missing or invalid ANA log, disabling ANA support
[Wed Jan 22 00:12:45 2026] overlayfs: upper fs does not support xattr, falling back to index=off

What it means: The kernel sometimes tells you directly that it disabled the feature. People ignore this because it’s “just a warning.”

Decision: Treat these lines as requirements. If the feature is disabled, stop designing around it.

Task 15: Confirm syscalls and behavior differences (glibc/musl, seccomp)

cr0x@server:~$ strace -f -e trace=io_uring_setup,openat,fsync -o /tmp/trace.log ./app --once
cr0x@server:~$ tail -n 6 /tmp/trace.log
12345 io_uring_setup(256, {flags=0, sq_thread_cpu=0, sq_thread_idle=0}) = -1 EPERM (Operation not permitted)
12345 openat(AT_FDCWD, "/var/lib/app/data", O_RDONLY|O_CLOEXEC) = 3
12345 fsync(3) = 0

What it means: Your app “supports io_uring,” but in a container with seccomp or insufficient privileges, it may be blocked and silently fall back to older I/O.

Decision: If the intended syscall is denied, either adjust sandbox policy (carefully) or accept the fallback and size capacity accordingly.

Three corporate-world mini-stories (anonymous, painfully real)

Mini-story 1: The incident caused by a wrong assumption

A platform team rolled out a new “hardened” base image for internal services. It was pitched as a drop-in replacement: same packages, slightly newer kernel, smaller attack surface. The migration plan was clean: bake AMIs, rolling deploy, watch dashboards.

One service—a write-heavy API with a local queue—started showing periodic latency spikes that looked like GC pauses. CPU was fine. Memory was fine. Network was fine. Yet every few minutes, p99 jumped and throughput dipped. Everyone stared at application traces because, of course, it must be the app.

It wasn’t. The base image changed the storage stack in a subtle way: the filesystem mount options no longer included discard, and the underlying virtual disk didn’t actually advertise discard anyway. The old image had been running a weekly trim job; the new one didn’t. Over time, the SSD-backed storage started behaving like a pessimistic historian: it remembered every write, and it resented you for it.

The “ported feature” was the promise that the new image was operationally equivalent. It was. Except for a boring maintenance behavior that wasn’t documented as a requirement, and therefore wasn’t tested. The fix was not heroic: restore trim behavior, validate discard support end-to-end, and add an explicit check in the image pipeline to fail builds when storage assumptions changed.

Mini-story 2: The optimization that backfired

A storage team migrated a fleet from iSCSI LUNs to NVMe over Fabrics. The vendor demo was excellent. Latency improved in controlled tests. Someone noticed the kernel had native NVMe multipath support and decided to remove dm-multipath to “reduce overhead.” Cleaner stack, fewer moving parts.

Within weeks, intermittent errors appeared during fabric maintenance events. Nothing catastrophic—just brief stalls that caused timeouts upstream. The workload was a distributed database. Distributed databases are emotionally sensitive. A few seconds of I/O uncertainty becomes a cascade of leader changes, retries, and compactions. Everything looked like an application problem because the app was the one screaming.

The root cause was feature parity assumptions: the kernel’s NVMe multipath existed, but the specific combination of firmware and fabric behavior produced ANA state transitions the port didn’t handle well. The system would keep paths “live” but misclassify optimized vs non-optimized during certain events. It wasn’t broken all the time, just enough to poison tail latency.

The fix was to stop optimizing for elegance and optimize for predictability: restore a validated multipath configuration, add health checks that assert path state correctness, and rehearse failover in staging with the same firmware. The lesson landed painfully: removing layers is not always simplification; sometimes it’s just removing guardrails.

Mini-story 3: The boring but correct practice that saved the day

A mid-sized company ran a private Kubernetes cluster with a CSI driver for block storage. They had two distributions available internally: “standard Linux” and a minimal container-host OS. The minimal OS was popular because it booted fast and was easier to lock down. It also had a habit of missing kernel modules that “everyone assumes.”

Before allowing it into production, one SRE insisted on a “capability contract” test suite. It wasn’t glamorous. It booted a node, attached a volume, ran a set of sysfs checks, verified mount options, performed fsync stress, and then performed a forced detach/reattach. It also checked for specific kernel warnings and refused to proceed if any “fallback” strings appeared in dmesg.

During the first run, the suite failed immediately: the node could mount volumes, but didn’t support a required filesystem feature for snapshots. The CSI driver claimed snapshots were supported because the API objects existed. Under the hood, it fell back to a slow full-copy behavior that worked, but would have melted their storage budget and destroyed restore times.

The practice that saved them was mundane: codifying assumptions as tests. They didn’t “trust the port.” They tested the port. The minimal OS still shipped, but with a clear label: no snapshots until the kernel module set and filesystem features matched the contract. Nobody got paged, and the CFO remained blissfully unaware of how close they came to funding a surprise data-copy festival.

Common mistakes: symptom → root cause → fix

1) “We enabled it, but performance didn’t change”

Symptom: Toggle a feature (discard, compression, multipath, async I/O). Metrics remain unchanged.

Root cause: The feature isn’t negotiated end-to-end or is disabled at runtime due to missing capability.

Fix: Validate from bottom to top: device support (lsblk -D), kernel warnings (dmesg), mount options (findmnt), and driver stats.

2) “It works until failover, then it stalls”

Symptom: Normal latency until a path failure or maintenance; then tail latency explodes.

Root cause: Partial implementation of failover semantics, wrong path policy, or a port that doesn’t handle transitional states correctly.

Fix: Rehearse failover deliberately. Check multipath state and health. Prefer known-good, conservative configurations over “clean stacks” when reliability matters.

3) “Snapshots exist, but restores are painfully slow”

Symptom: Snapshot API succeeds; restore takes forever and hammers storage.

Root cause: The port provides snapshot objects but lacks copy-on-write or native snapshot support; it falls back to full copy.

Fix: Verify implementation: filesystem capability, storage backend support, and real restore I/O patterns. Gate snapshot usage by backend type.

4) “Encryption is on, but CPU skyrockets”

Symptom: Enabling TLS or disk encryption causes throughput collapse.

Root cause: Missing hardware acceleration, different crypto provider behavior, or policy disabling certain algorithms so negotiation picks slow ones.

Fix: Inspect negotiated ciphers, CPU profiles, and crypto acceleration availability. Choose ciphers deliberately; don’t accept whatever the handshake gives you.

5) “Containers started failing with weird filesystem errors”

Symptom: Overlay storage errors, odd rename failures, permission quirks.

Root cause: Backing filesystem lacks required features (d_type, xattrs), or kernel overlay implementation differs from what your runtime expects.

Fix: Validate overlay prerequisites and switch backing filesystem or driver. Don’t run overlay on “whatever happened to be mounted.”

6) “Same app, different distro, different behavior”

Symptom: Timeouts, DNS issues, different error handling after a base image change.

Root cause: libc differences, resolver behavior, threading defaults, or kernel sysctl defaults changed.

Fix: Treat base image and libc as part of the platform API. Pin versions, run compatibility tests, and diff sysctls.

7) “The feature is present, but we can’t observe it”

Symptom: No metrics or logs to confirm behavior; only indirect inference.

Root cause: Port omitted counters/tracepoints or didn’t wire them into your telemetry.

Fix: Add explicit observability requirements to acceptance criteria. If you can’t observe it, you can’t operate it.

Checklists / step-by-step plan: how to ship ports safely

Step-by-step: build a “capability contract” for your platform

1. List non-negotiable semantics per workload. For databases: fsync correctness and latency; for object storage: durability and rebuild behavior; for streaming: tail latency under backpressure.
2. Translate semantics into verifiable checks. “Supports discard” becomes “DISC-MAX > 0 and mount option set or periodic trim verified.” “Supports snapshots” becomes “restore does not require full copy.”
3. Document prerequisites explicitly. Kernel options, required modules, minimum firmware, and required backing filesystem features.
4. Create an automated validation suite. Run it on every new kernel, base image, and storage backend. Fail fast when capabilities drift.
5. Include failure drills. Detach a path, reboot a node, fill disk to 95%, force resync, rotate certificates. Measure SLO impact.
6. Define your fallback policy. If a feature can’t be negotiated, should the system refuse to start, or run in degraded mode with loud alerts?
7. Gate rollout by capability, not by host class name. “prod-storage-02” is not a capability. “discard supported end-to-end” is.
8. Keep a compatibility matrix. Not a novel; a table: kernel versions, driver versions, firmware, feature flags, known caveats.

Operational checklist: before you accept “ported” as “production-ready”

• Can you prove the feature is negotiated end-to-end (client + server + kernel + device)?
• Can you prove the feature changes behavior under load (not just configuration state)?
• Do you have at least one negative test where the feature is missing and the system fails loudly?
• Do you have observability: counters, logs, and clear dashboards that show feature health?
• Have you tested the “ugly path”: failover, resync, full disk, high latency, packet loss?
• Is rollback safe (pool feature flags, protocol downgrades, config compatibility)?

Decision rules (opinionated, because you’re busy)

• If the feature affects durability or integrity and you can’t validate it, disable it and design around that reality.
• If the feature affects performance and can silently fall back, alert on the fallback or treat performance claims as marketing.
• If the feature is a dependency for your incident response (snapshots, restore speed, observability), do not ship without it.
• If your platform team says “it should work,” ask for: kernel config proof, negotiated capability proof, and a failure drill report.

FAQ

1) What’s the difference between “feature exists” and “feature works”?

“Exists” means you can see a knob, module, API, or config option. “Works” means the system takes the intended code path under realistic conditions, and you can prove it with observation and tests.

2) Why do ports ship with stubbed features?

Because delivering API parity is often easier than delivering semantic parity. Stubs reduce integration friction and buy time. The problem is when nobody labels the stub as a stub.

3) How do silent fallbacks happen?

Many systems prioritize availability. If negotiation fails, they choose an older protocol, a slower algorithm, or a safer mode and keep running. That’s great for demos and terrible for SLOs unless you detect and alert on the downgrade.

4) Are “compatibility matrices” worth the effort?

Yes, if you keep them short and tied to acceptance tests. The value is not the document; it’s the discipline of knowing which combinations are tested and which are wishful thinking.

5) What’s the fastest way to tell if a storage feature is real?

Check device capability (sysfs/lsblk), check kernel warnings (dmesg), and then measure behavior (iostat latency, rebuild/failover tests). If any layer disagrees, assume the feature is not real.

6) How do I handle “supported on Linux” claims from vendors?

Translate that claim into: which kernel versions, which modules, which filesystems, which firmware, and which failure scenarios are validated. If they can’t answer, you’re the test lab.

7) Should we prefer fewer layers (for example, removing dm-multipath)?

Prefer fewer layers only when you can prove the remaining layer handles failure and observability at least as well. “Less” is not automatically “simpler” in failure modes.

8) What if a missing feature is only a performance problem, not correctness?

Performance failures still become reliability failures when they trigger timeouts, retries, elections, or backpressure. Treat tail latency as an availability risk, not just a speed annoyance.

9) How do we make teams stop arguing about whether it’s the app or the platform?

Make the platform publish a capability contract with tests and evidence, and make services declare which capabilities they require. Then disagreements become diffs and test results, not meetings.

Conclusion: what to do next week, not next quarter

Ports with missing features aren’t rare bugs. They’re the natural outcome of shipping “presence” before “behavior.” The fix is not more optimism. It’s more verification, more explicit contracts, and fewer silent fallbacks.

Practical next steps:

1. Create a capability contract for your platform: the handful of features you truly depend on (durability, snapshots, multipath behavior, crypto negotiation, observability).
2. Automate the checks shown above into CI for base images, kernels, and node builds. Fail fast when prerequisites drift.
3. Run one failure drill per critical feature in staging: path loss, node reboot, resync, snapshot restore, high-latency injection.
4. Make fallbacks loud: if the system downgrades, alert. If it can’t provide a required feature, refuse to start.

When someone says “it’s there,” your job is to ask: “Can we prove it’s active, observable, and correct under failure?” If not, treat it as missing. Production will.