Docker “OCI runtime create failed”: decode it and fix the real cause

You type docker run. Docker thinks for half a second. Then it punts with:
OCI runtime create failed and a wall of vaguely accusatory text. Nothing starts. Your pager starts.
The root cause is usually not “OCI”. It’s almost always a very specific kernel, filesystem, security, or config failure
that Docker is merely reporting—badly.

This is a field guide for production: how to read the error like a stack trace, locate the failing layer, and fix the
actual problem without cargo-cult restarts. We’ll go from “what does OCI even do here?” to concrete commands,
expected outputs, and decisions you should make.

Decode the error: what “OCI runtime create failed” really means

The headline is misleading. “OCI runtime create failed” is not a singular error. It’s a bucket. Docker is telling you:
“I asked the OCI runtime to create a container, and something in the runtime’s setup phase exploded.”

That “setup phase” includes:

• creating namespaces (pid, mount, network, user)
• setting cgroups
• preparing the root filesystem (overlay mounts, bind mounts, tmpfs)
• applying security policy (seccomp, AppArmor/SELinux)
• setting up the init process and exec’ing your entrypoint

When any of those steps fails, Docker often prints a message that starts with:
docker: Error response from daemon: OCI runtime create failed: ...
and then a nested chain like container_linux.go, process_linux.go,
failed to create shim task, permission denied, invalid argument, or no space left on device.

Your job is to ignore the generic prefix and obsess over the last specific syscall-ish complaint.
That last bit is usually the truth.

One dry rule: treat this error like a kernel error wearing Docker’s coat. Fix the kernel-facing problem, not the coat.

Joke #1 (short, relevant): OCI errors are like “check engine” lights—accurate that something’s wrong, unhelpful about which cylinder is on fire.

Facts and history: why this error exists at all

A little context helps because OCI errors are shaped by how containers evolved: layers of standards, runtimes, and daemons
talking to each other across APIs. Here are some concrete facts that explain the weirdness.

1. Linux containers predate Docker. cgroups landed in Linux around 2007–2008; namespaces matured through the 2000s. Docker popularized the packaging.
2. Docker originally used LXC by default. Early Docker versions used LXC; later it moved to libcontainer (Go) and then standardized on runc.
3. OCI (Open Container Initiative) was created in 2015. The goal: standard image and runtime specs so “container” means something portable.
4. runc came out of Docker’s libcontainer. runc is the reference OCI runtime; many runtimes implement the same spec.
5. containerd was split out to reduce coupling. Docker delegated more to containerd; you’ll see errors mentioning “shim” because containerd uses shims to manage container processes.
6. The OCI “create” step is not “start your app.” “Create” means set up container state and environment; the app exec is late in the process and fails for totally different reasons.
7. cgroup v2 changed defaults and assumptions. Distros flipped the default hierarchy; some Docker setups broke until configured for systemd cgroups and compatible drivers.
8. Security modules changed behavior over time. SELinux and AppArmor policy evolutions can turn a previously-working container into a sudden “permission denied” failure.
9. OverlayFS has kernel feature gates. Certain overlay modes (like metacopy, redirect_dir, idmapped mounts) depend on kernel versions and can fail oddly when toggled.

Fast diagnosis playbook (do this in order)

When production is burning, you need a shortest-path algorithm, not a philosophy class. Here’s the order that tends to
find the bottleneck quickly.

1) Capture the full error chain

• Re-run the command with maximum daemon context (logs + events).
• Extract the final failure: permission denied, no space left on device, invalid argument, exec format error, operation not permitted.

2) Check host-level “obvious lies”: disk, inodes, memory, kernel logs

• Disk/inodes on Docker root (/var/lib/docker or custom path).
• dmesg/journal for AppArmor/SELinux denials or mount failures.
• Memory/cgroup pressure if the error chain hints at fork/exec failures.

3) Identify the failing subsystem based on the last error

• Mount/rootfs: overlay2 mount errors, “unknown filesystem,” “invalid argument,” “permission denied.”
• Security: AppArmor/SELinux AVC/denied logs; seccomp “operation not permitted.”
• cgroups: errors mentioning cgroup paths, controllers, or systemd.
• Entrypoint: “exec format error,” “no such file or directory” (often interpreter missing), “text file busy.”

4) Reproduce with the smallest possible container

• Try a known-good image and trivial command (like busybox true).
• If even that fails, it’s host/runtime. If only one image fails, it’s image config, entrypoint, or mounts.

5) Fix one thing, then retest

Don’t shotgun “restart Docker, reboot node, disable security” as a sequence. Make one change, validate, move on.
You’re debugging, not performing ritual.

Map the failure to the layer: Docker → containerd → runc → kernel

If you want to be fast, think in layers and pick the right log.

• Docker CLI prints the summarized error. Useful for the final string, not for details.
• dockerd logs show what Docker asked containerd to do and what came back.
• containerd logs show shim and runtime calls. “failed to create shim task” is usually here.
• runc does the kernel syscalls. Its errors are the ones that matter most.
• kernel logs (dmesg/journal) often contain the real denial or mount rejection, especially for LSMs.

“OCI runtime create failed” usually means runc returned non-zero during create.
The create step includes mounting your rootfs and bind mounts. So storage and security problems dominate.

Quote (paraphrased idea): Werner Vogels’ paraphrased idea: everything fails, all the time—your job is building systems that expect and handle it.

Practical tasks: 12+ commands with outputs and decisions

Below are practical tasks you can run on a Linux host running Docker Engine. Each one includes:
the command, what typical output means, and the decision to make from it. Run them in the order that matches your symptoms,
not necessarily top-to-bottom.

Task 1: Re-run the failing container with maximum clarity

cr0x@server:~$ docker run --rm --name oci-test alpine:3.20 true
docker: Error response from daemon: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /proc/self/fd: permission denied: unknown.

What it means: The last clause (open /proc/self/fd: permission denied) is your lead. That points to security policy (SELinux/AppArmor/seccomp) or a hardened /proc mount behavior.

Decision: Jump to kernel/LSM logs (Tasks 4–6) before touching storage.

Task 2: Verify whether the problem is global or image-specific

cr0x@server:~$ docker run --rm busybox:1.36 true
Unable to find image 'busybox:1.36' locally
1.36: Pulling from library/busybox
Digest: sha256:...
Status: Downloaded newer image for busybox:1.36
docker: Error response from daemon: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /proc/self/fd: permission denied: unknown.

What it means: If even BusyBox fails, your host runtime environment is broken, not your application image.

Decision: Focus on daemon config, kernel features, LSM policy, cgroups, and storage driver health.

Task 3: Read Docker daemon logs around the failure

cr0x@server:~$ sudo journalctl -u docker --since "10 min ago" --no-pager
Jan 03 09:41:18 server dockerd[1327]: time="2026-01-03T09:41:18.912345678Z" level=error msg="Handler for POST /v1.45/containers/create returned error: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /proc/self/fd: permission denied: unknown"
Jan 03 09:41:18 server dockerd[1327]: time="2026-01-03T09:41:18.913456789Z" level=info msg="Attempting next endpoint for containerd"

What it means: dockerd is repeating the runtime error, but it confirms the timeline and that it’s not a client-side glitch.

Decision: If dockerd logs show “permission denied,” “invalid argument,” “no space,” etc., chase that exact phrase downstream.

Task 4: Read containerd logs (where shim/runc errors show up)

cr0x@server:~$ sudo journalctl -u containerd --since "10 min ago" --no-pager
Jan 03 09:41:18 server containerd[1189]: time="2026-01-03T09:41:18.905432198Z" level=error msg="RunPodSandbox for "..." failed" error="failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /proc/self/fd: permission denied: unknown"

What it means: “failed to create shim task” is containerd’s way of saying the runtime couldn’t set up the container. The last clause is still the key.

Decision: If you see mount-related errors here, go to storage driver checks. If you see permission/denied, go to LSM checks.

Task 5: Check kernel logs for AppArmor denials

cr0x@server:~$ sudo dmesg -T | tail -n 20
[Fri Jan  3 09:41:18 2026] audit: type=1400 audit(1735897278.912:312): apparmor="DENIED" operation="open" profile="docker-default" name="/proc/self/fd/" pid=24310 comm="runc:[2:INIT]" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

What it means: That’s a smoking gun. AppArmor blocked runc’s init process from reading /proc/self/fd.

Decision: Fix the AppArmor profile or run containers with an appropriate profile override (see Common mistakes and checklist). Do not disable AppArmor globally unless you enjoy living dangerously.

Task 6: Check SELinux denials (RHEL/Fedora/CentOS derivatives)

cr0x@server:~$ sudo ausearch -m avc -ts recent | tail -n 5
type=AVC msg=audit(1735897278.912:312): avc:  denied  { mounton } for  pid=24310 comm="runc:[2:INIT]" path="/var/lib/docker/overlay2/..." scontext=system_u:system_r:container_runtime_t:s0 tcontext=system_u:object_r:default_t:s0 tclass=dir permissive=0

What it means: SELinux blocked a mount operation. The tcontext being default_t is suspicious; labels are wrong or the storage path isn’t labeled for containers.

Decision: Restore correct contexts on Docker storage paths, or adjust policy properly. Avoid setting SELinux to permissive as your “fix”; it’s a diagnostic step at most.

Task 7: Check Docker’s storage driver and root directory

cr0x@server:~$ docker info --format 'Driver={{.Driver}} RootDir={{.DockerRootDir}} CgroupDriver={{.CgroupDriver}}'
Driver=overlay2 RootDir=/var/lib/docker CgroupDriver=systemd

What it means: You now know where the bits live and which driver is involved. Overlay2 is normal; it also means mount and kernel overlay support matter.

Decision: If RootDir is on an odd filesystem (NFS, eCryptfs, old XFS without ftype), expect pain. Validate the backing filesystem next.

Task 8: Check disk space and inodes on Docker RootDir filesystem

cr0x@server:~$ df -h /var/lib/docker
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p3  200G  198G  2.0G  99% /

cr0x@server:~$ df -i /var/lib/docker
Filesystem       Inodes   IUsed    IFree IUse% Mounted on
/dev/nvme0n1p3  13107200 13090000  17200   100% /

What it means: “No space left on device” can be disk blocks or inodes. Overlay2 is inode-hungry.

Decision: If either is near full, stop debugging fancy things. Reclaim space (Task 9) or expand storage. Containers can’t mount layers if the filesystem can’t create metadata.

Task 9: Reclaim Docker space safely (and understand what you’re deleting)

cr0x@server:~$ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          54        12        38.6GB    22.4GB (58%)
Containers      17        2         1.1GB     1.0GB (91%)
Local Volumes   31        10        96.2GB    40.5GB (42%)
Build Cache     0         0         0B        0B

cr0x@server:~$ docker system prune -f
Deleted Containers:
...
Deleted Images:
...
Total reclaimed space: 12.3GB

What it means: docker system df shows where the space is. Prune removes stopped containers, dangling images, and unused networks. Volumes are only removed with --volumes.

Decision: In production, prune is acceptable if you understand your image caching strategy and you’re not relying on stopped containers as forensic evidence. For nodes that rebuild often, prune proactively with guardrails.

Task 10: Validate the backing filesystem for overlay2 (XFS ftype)

cr0x@server:~$ findmnt -no FSTYPE,OPTIONS /var/lib/docker
xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota

cr0x@server:~$ sudo xfs_info /dev/nvme0n1p3 | grep ftype
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1

What it means: Overlay2 on XFS needs ftype=1. If it’s ftype=0, you will eventually get weird overlay behavior and container start failures.

Decision: If ftype=0, the correct fix is migrating Docker RootDir to a correctly formatted filesystem. There is no safe in-place switch.

Task 11: Check mount propagation / shared mounts (common in Kubernetes hosts too)

cr0x@server:~$ findmnt -o TARGET,PROPAGATION /var/lib/docker
TARGET          PROPAGATION
/var/lib/docker shared

What it means: Some container workloads require shared mount propagation; others fail if the host mount layout is unusual. If it’s private in an environment expecting shared, you can see mount-related OCI create failures.

Decision: If you suspect mount propagation issues, compare with a known-good node and align systemd mount units. Avoid random mount --make-rshared / in production unless you understand the blast radius.

Task 12: Inspect seccomp status and test with an unconfined profile (diagnostic only)

cr0x@server:~$ docker info --format 'SecurityOptions={{json .SecurityOptions}}'
SecurityOptions=["name=seccomp,profile=default","name=apparmor"]

cr0x@server:~$ docker run --rm --security-opt seccomp=unconfined alpine:3.20 true
docker: Error response from daemon: OCI runtime create failed: runc create failed: unable to start container process: error during container init: open /proc/self/fd: permission denied: unknown.

What it means: Seccomp unconfined didn’t change anything. That suggests seccomp isn’t the cause (in this example); AppArmor likely is.

Decision: Use unconfined toggles as controlled experiments. If unconfined “fixes” it, then craft a proper profile rather than leaving it wide open.

Task 13: Validate cgroup mode and hierarchy (common with cgroup v2)

cr0x@server:~$ stat -fc %T /sys/fs/cgroup
cgroup2fs

cr0x@server:~$ docker info --format 'CgroupVersion={{.CgroupVersion}} CgroupDriver={{.CgroupDriver}}'
CgroupVersion=2 CgroupDriver=systemd

What it means: You’re on cgroup v2 and Docker is using systemd driver (generally correct for modern distros). If you see mismatch (cgroup v2 with cgroupfs driver in some setups), you can get create failures.

Decision: Align Docker’s cgroup driver with systemd on systemd-based hosts. If you’re in a legacy corner, test on a known-good node before changing fleet-wide.

Task 14: Confirm the entrypoint binary is executable and matches architecture

cr0x@server:~$ docker run --rm --entrypoint /bin/sh alpine:3.20 -c 'uname -m; file /bin/busybox'
x86_64
/bin/busybox: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, stripped

What it means: If your failing image has an ARM binary on an x86_64 host, you’ll see exec format error during create/start. This check proves what “normal” looks like.

Decision: If architecture mismatches, fix the image build or use multi-arch manifests correctly. Don’t “fix” it by installing qemu emulation on prod nodes unless you absolutely have to and understand performance implications.

Task 15: Validate bind mounts exist and have correct permissions

cr0x@server:~$ docker run --rm -v /does-not-exist:/data alpine:3.20 true
docker: Error response from daemon: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/does-not-exist" to rootfs at "/data": mount /does-not-exist:/data, flags: 0x5000: no such file or directory: unknown.

What it means: A missing host path causes an OCI create failure because the mount setup happens before the process starts.

Decision: Fix the host path, create it with correct ownership, or use Docker-managed volumes instead of brittle host paths.

Task 16: Spot overlay2 mount errors explicitly

cr0x@server:~$ sudo grep -R "overlay" -n /var/log/syslog | tail -n 5
Jan 03 09:41:18 server kernel: [Fri Jan  3 09:41:18 2026] overlayfs: upper fs does not support tmpfile.
Jan 03 09:41:18 server kernel: [Fri Jan  3 09:41:18 2026] overlayfs: failed to mount overlay: invalid argument

What it means: The backing filesystem lacks features overlay expects. That can come from exotic filesystems, certain mount options, or older kernels.

Decision: Move Docker RootDir to a supported local filesystem (ext4 or properly configured XFS), or upgrade kernel/storage configuration to meet overlay requirements.

Joke #2 (short, relevant): Turning off SELinux to “fix Docker” is like removing your smoke alarm because it’s loud.

Three corporate mini-stories from the trenches

Mini-story 1: The incident caused by a wrong assumption

A mid-sized company ran Docker on a fleet of general-purpose VMs. One team owned “the base image,” another team owned “the platform.”
The platform team rolled out a hardening change: new AppArmor policies and a kernel update. The assumption was simple and wrong:
“Containers are isolated; a stricter host profile won’t affect them.”

Monday morning, a subset of services stopped deploying. The symptom was identical everywhere:
OCI runtime create failed with open /proc/self/fd: permission denied.
Teams initially blamed the image because the failure appeared at container start. A few engineers wasted hours rebuilding images
and rolling tags, because of course it must be “the app.”

The breakthrough came from someone who read dmesg instead of Slack.
Kernel audit logs showed AppArmor denials against docker-default.
The policy update tightened access to /proc paths in ways that runc’s init process didn’t like.
Docker wasn’t broken. The host security posture was, relative to their runtime.

The fix was surgical: adjust the AppArmor profile used for those workloads, and roll it through the fleet with a canary.
They also wrote a one-page runbook: “If the error ends with permission denied, check LSM logs first.”
The total incident time could have been 20 minutes. It was closer to half a day because the first assumption was incorrect.

Mini-story 2: The optimization that backfired

Another org tried to optimize build and deploy speed on CI runners. They set Docker’s RootDir to a network-attached filesystem
because it was “fast storage” and made cleanup easy. Also, it avoided local disk upgrades. Everyone loves a project that
appears to save money by moving complexity somewhere else.

It worked for a while. Then random jobs began failing with OCI runtime create failed,
usually phrased as mount failures or “invalid argument.” It was intermittent. That was the worst part.
Intermittent failures are where engineering time goes to die quietly.

The real issue: overlay2 on a non-local filesystem is a compatibility minefield. Even if it “works,” you can hit edge cases:
file locking semantics, d_type support, tmpfile support, latency spikes during mount operations, and timing-sensitive races
when thousands of layers churn daily.

The rollback was unglamorous: move Docker RootDir back to local SSD, enforce pruning quotas, and accept that build cache is a
performance feature—not a data-retention strategy. They stopped seeing create failures immediately. The “optimized” design
had been optimizing the wrong thing: storage convenience, not runtime correctness.

Mini-story 3: The boring but correct practice that saved the day

A large enterprise ran a mixed fleet: different kernel versions, different distros, different compliance regimes.
They were allergic to outages, so they did something deeply unfashionable: they standardized the container host baseline.
Same filesystem type and options for Docker RootDir, same cgroup driver settings, same security module configuration,
and a narrow set of tested kernel versions.

One day a new service started failing on a single cluster with OCI runtime create failed and invalid argument.
The on-call engineer compared the node to their known-good baseline using a tiny checklist:
docker info, filesystem type, kernel version, and a quick scan of kernel logs.

The offender popped quickly: a small subset of nodes had been provisioned with an older XFS volume where ftype=0
due to an outdated imaging script. It didn’t break everything immediately; it broke just enough to be annoying, and only under certain layer patterns.

Because they had a baseline, the fix wasn’t a debate. They drained those nodes, rebuilt with the correct filesystem settings,
and brought capacity back. No heroics, no “disable security,” no late-night experimental flags. Boring won, again.

Common mistakes: symptoms → root cause → fix

OCI create failures look repetitive, but the root causes are not. Here are the patterns that show up most often in production,
mapped the way you actually debug them.

1) “permission denied” during container init

Symptom: Error ends with permission denied, often mentioning /proc, mounts, or files under /var/lib/docker.

Root cause: AppArmor/SELinux policy denial, wrong labels on Docker directories, or restrictive mount options.

Fix: Check dmesg and audit logs. Restore SELinux contexts on Docker RootDir; adjust AppArmor profile. Use --security-opt only as diagnosis, then implement a proper policy.

2) “no space left on device” but you have free GBs

Symptom: OCI create fails with ENOSPC while df -h shows space.

Root cause: Inodes exhausted (df -i) or Docker RootDir is on a smaller filesystem than you think (bind-mounted somewhere else).

Fix: Check df -i and docker info RootDir. Prune unused layers; expand filesystem; consider moving RootDir to a dedicated volume.

3) “invalid argument” around overlay mounts

Symptom: failed to mount overlay: invalid argument or similar in kernel logs.

Root cause: Unsupported backing filesystem (NFS, older XFS config, incompatible mount options) or kernel/overlay feature mismatch.

Fix: Put Docker RootDir on supported local FS; verify XFS ftype=1; upgrade kernel if required; stop trying to run overlay2 on “creative” storage.

4) “exec format error” or “no such file or directory” at start

Symptom: Container fails immediately; error chain ends with exec failures.

Root cause: Wrong architecture binary; missing interpreter (e.g., script with #!/bin/bash but no bash); CRLF line endings in entrypoint scripts can also contribute to confusion.

Fix: Verify image architecture and entrypoint. Ensure interpreters exist. Build multi-arch images properly.

5) Bind mount failures

Symptom: Error mentions “error mounting” host path, “no such file,” or “not a directory.”

Root cause: Missing host path, wrong type (file vs directory), or permissions/labels preventing mount.

Fix: Create the path, fix permissions and SELinux contexts, or switch to named volumes.

6) cgroup errors (especially after OS upgrades)

Symptom: OCI create fails mentioning cgroups, controllers, or systemd.

Root cause: cgroup v2 transition issues, mismatched Docker cgroup driver, or locked-down controllers.

Fix: Align to systemd cgroup driver on systemd hosts; ensure required controllers are enabled; validate with docker info and stat -fc %T /sys/fs/cgroup.

Checklists / step-by-step plan

Incident checklist: restore service first, then repair properly

1. Get the exact final error string.
   Re-run a minimal container and capture output.
   Decide if it smells like storage, security, cgroups, or entrypoint.
2. Confirm it’s not image-specific.
   Run busybox true.
   If that fails, stop blaming the app.
3. Check host capacity (blocks + inodes) on Docker RootDir.
   If full, prune or expand. Don’t overthink it.
4. Check kernel logs for denials and mount errors.
   If you see AppArmor/SELinux denies, you have your root cause category.
5. Compare one broken node to one good node.
   Same kernel? same filesystem? same Docker config? same security options?
   Differences are usually the answer.
6. Apply the smallest safe mitigation.
   Examples: free space, restore labels, adjust a profile, fix a mount path.
   Avoid “disable the entire security module” as a mitigation unless it’s a short diagnostic with a rollback plan.
7. Retest with minimal container, then real workload.
   If minimal works but app doesn’t, you’ve moved from host-level to app-level debugging (entrypoint, mounts, user permissions).

Prevention checklist: stop future you from meeting present you

1. Put Docker RootDir on a supported local filesystem. ext4 or correctly configured XFS. Avoid network storage for overlay2 runtime paths.
2. Monitor both disk usage and inode usage. Alert before 85–90% for both, not just GBs.
3. Standardize cgroup driver and hierarchy. If you’re on systemd, use systemd cgroups unless you have a compelling reason.
4. Log LSM denials centrally. AppArmor and SELinux denials should show up in your logging pipeline with filters, otherwise you’ll debug blind.
5. Baseline host configuration and diff it. Kernel version, storage driver, mount options, daemon.json. Make drift visible.
6. Keep an emergency “minimal container” test in the runbook. BusyBox/Alpine test tells you if the host is fundamentally broken.
7. Document approved security-opt exceptions. If some workloads need unconfined settings, treat that as a risk decision, not a developer whim.

FAQ

1) What is OCI in Docker errors?

OCI is the Open Container Initiative spec. In practice, Docker uses an OCI runtime (commonly runc) to set up namespaces,
cgroups, mounts, and then exec your process. The error means that setup failed.

2) Why does the error mention runc or containerd shim?

Docker delegates runtime operations to containerd, which uses a shim to manage container processes. runc performs the low-level
create/start operations. Failures bubble up through these layers, so you see all their names in the chain.

3) Is “OCI runtime create failed” ever fixed by restarting Docker?

Occasionally—if the daemon is wedged or containerd is unhealthy. But most of the time it’s a real host issue (space, mounts,
policy, cgroups). Restarting may clear symptoms temporarily while the underlying cause remains.

4) How do I tell if it’s an image problem or a host problem?

Run a tiny known-good container. If docker run --rm busybox true fails, it’s host/runtime. If BusyBox works but
your image fails, it’s likely entrypoint, architecture, missing interpreter, mounts, or permissions inside the image.

5) Why do I get “no such file or directory” when the file exists?

Often the entrypoint is a script with a shebang pointing to an interpreter that doesn’t exist in the image (e.g., /bin/bash).
Another common cause is architecture mismatch producing confusing exec errors.

6) How does disk inode exhaustion cause OCI create failures?

Overlay2 creates lots of small files and directories for layer metadata. If inodes hit 100%, the runtime can’t create required
directories during mount/setup, so create fails even if you still have free GBs.

7) Can SELinux/AppArmor really stop containers from starting?

Yes. The runtime itself (runc) is a process on the host subject to host security policy. If policy blocks mounts, file access,
or certain operations, container create fails before your application ever runs.

8) What’s the safest way to “test if security is the problem”?

Use targeted diagnostics: check kernel/audit logs first. If you must experiment, run a single test container with a specific
override (like an unconfined seccomp profile) and immediately revert. Don’t flip global settings as a casual test.

9) Does cgroup v2 cause OCI runtime create failed?

It can, especially with mismatched Docker configuration or older runtime components. Validate cgroup version and driver via
docker info and ensure the host enables necessary controllers.

10) If overlay2 is broken, can I switch storage drivers on the same RootDir?

Treat that as migration work, not a toggle. Switching drivers typically requires moving or rebuilding Docker state. Do it deliberately:
drain workloads, back up what matters (volumes), and rebuild nodes if you can.

Conclusion: what to do next time

“OCI runtime create failed” is Docker saying “the kernel said no,” with extra steps. The fastest path is:
capture the final error clause, classify it (security/storage/cgroups/entrypoint), and go to the logs that can actually confirm it.

Next steps that pay off immediately:

• Write a tiny runbook snippet: BusyBox test, docker info RootDir/Driver, df -h and df -i, then dmesg/journalctl.
• Baseline your container hosts: filesystem type/options, XFS ftype, cgroup mode, security modules. Drift is the silent killer.
• Stop storing Docker’s writable runtime layers on “clever” storage. Use local disks for overlay2; use proper storage systems for persistent data.
• When the error says “permission denied,” believe it. Go straight to AppArmor/SELinux/audit logs and fix policy or labels.