Docker Security: Stop Leaking Secrets With This One File Layout

Was this helpful?

Most secret leaks in Docker setups aren’t Hollywood hacks. They’re boring: a stray .env baked into an image, a CI artifact uploaded “for debugging,” or a developer running docker inspect and accidentally pasting the output into a ticket.

Fixing this doesn’t require a new platform team, a vault migration, or a twelve-week committee ritual. It requires a file layout that makes the secure path the easy path—and makes the insecure path annoying enough that people stop doing it.

The one file layout that changes everything

This is the layout. It’s not pretty. It’s not clever. It’s the kind of thing you adopt once and then never want to talk about again, which is exactly what you want for secrets.

cr0x@server:~$ tree -a -L 3 .
.
├── .dockerignore
├── .gitignore
├── Dockerfile
├── README.md
├── compose.yaml
├── scripts
│   ├── bootstrap-dev.sh
│   └── verify-no-secrets.sh
├── src
│   └── app.py
├── config
│   ├── app.example.yaml
│   └── logging.yaml
└── secrets
    ├── README.md
    ├── dev
    │   ├── app.env.example
    │   └── tls.example
    └── runtime
        ├── DO_NOT_COMMIT
        └── .keep

What each directory means (and what it forbids)

  • src/: application code only. Never secrets. If code needs a secret, it reads it at runtime from a file path or an injected environment variable. It does not ship the secret.
  • config/: non-secret configuration committed to git. Provide *.example* templates so engineers don’t invent ad-hoc names like prod.env.
  • secrets/: this is the trick. You create a place for secrets so people stop sprinkling them everywhere. But you also make it impossible to commit them by policy and tooling:
    • secrets/dev/ contains examples only for local development. Example files teach shape, not content.
    • secrets/runtime/ is where real secrets land on a machine or in a pipeline workspace. It is never committed. It is mounted at runtime.
  • .dockerignore: your first hard barrier. Docker can only copy what you send in the build context. Don’t send secrets.
  • compose.yaml: defines runtime mounts/secrets, not build-time copying.
  • scripts/verify-no-secrets.sh: automation that fails builds when someone tries to get creative.

The .dockerignore that does the heavy lifting

If you take only one thing from this piece, take this: your build context is an attack surface. The Docker daemon (local or remote) receives a tar of your context. If your .env is in there, you’ve already lost.

cr0x@server:~$ cat .dockerignore
# Never send secrets to the build context
secrets/
**/*.pem
**/*.key
**/*id_rsa*
**/*.p12
**/*.jks
**/.env
**/.env.*
**/*credentials*
**/*token*
**/*secret*
**/kubeconfig
**/.npmrc
**/.pypirc
**/.netrc

# Common junk that bloats builds and leaks internals
.git/
.gitignore
Dockerfile*
compose*.yml
compose*.yaml
node_modules/
__pycache__/
*.log

Yes, you can still leak a secret with a filename that doesn’t match those patterns. That’s why the layout also includes scanning and guardrails. Defense in depth is just a fancy way of saying “assume someone will eventually do the wrong thing.”

Dockerfile pattern: build without secrets, run with secrets

Your Dockerfile should not accept secrets as ARG. It should not copy .env. It should not RUN echo "$TOKEN". If you need secrets at build time (private dependency registries), use BuildKit secret mounts so the secret never becomes a layer.

cr0x@server:~$ sed -n '1,160p' Dockerfile
# syntax=docker/dockerfile:1.6
FROM python:3.12-slim AS runtime

WORKDIR /app

# Install deps first to leverage caching
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy only the application
COPY src/ ./src/
COPY config/ ./config/

# Run as non-root
RUN useradd -r -u 10001 appuser && chown -R appuser:appuser /app
USER appuser

# Secrets are loaded at runtime from /run/secrets or mounted files
ENV APP_CONFIG=/app/config/logging.yaml
CMD ["python", "-m", "src.app"]

Compose pattern: runtime injection, not build-time baking

Compose is where people get sloppy because “it’s just dev.” And then dev turns into staging, staging turns into prod, and prod turns into an incident review.

cr0x@server:~$ sed -n '1,200p' compose.yaml
services:
  app:
    build:
      context: .
    image: acme/app:dev
    environment:
      # Non-secret values only
      - LOG_LEVEL=info
    volumes:
      # Runtime config is fine if it is not secret
      - ./config:/app/config:ro
      # Real secrets: mounted from secrets/runtime (not in git)
      - ./secrets/runtime:/run/secrets:ro
    read_only: true
    tmpfs:
      - /tmp
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    ports:
      - "8080:8080"

Decision: if your team relies on .env files, keep them—but store them under secrets/runtime and mount them read-only. The point is to prevent them from entering git history or Docker layers.

Joke 1: If you put secrets in environment variables, they’ll eventually end up in a screenshot. Humans screenshot like it’s a backup strategy.

Why secrets leak in Docker (failure modes you can actually reproduce)

Docker leaks happen because Docker is good at moving bytes around and bad at understanding intent. It will happily tar up your build context, cache layers forever, and record metadata you didn’t think about. Meanwhile, engineers will happily optimize for “works now” because on-call is a character-building experience nobody asked for.

Leak path #1: build context includes secrets

docker build sends the entire context directory to the daemon. If the daemon is remote (common in CI or when using a shared builder), you’ve just shipped secrets over the wire. Even if the Dockerfile never copies them, the context transmission itself can be logged, cached, or inspected in unintended ways.

Leak path #2: Dockerfile layers preserve history

When you do RUN export TOKEN=... && some-command or RUN echo "$TOKEN" > /tmp/token, you create a layer. Even if you delete the file later, the layer can preserve it. Image history can also expose build args and command strings.

Leak path #3: environment variables are discoverable

Environment variables show up in docker inspect. They can appear in crash dumps, support bundles, process lists, and logs. They also tend to get copied into monitoring labels (“just for convenience”), which is how you end up with an API key in a metrics backend.

Leak path #4: bind mounts and permissions are mismatched

Mounting a secret file is fine. Mounting it writable and running as root is how you end up with mutated secrets, accidental commits, and “why did the container rewrite my config?” debugging sessions at 2 a.m.

Leak path #5: CI artifacts and debug logs

CI systems love artifacts. People love artifacts when debugging. If your pipeline uploads docker inspect, env output, or the entire workspace tarball, secrets will escape. Not because someone is malicious. Because someone is tired.

A few facts and historical context (because our mistakes are old)

  • Fact 1: Docker’s image layer model is based on union filesystems; deleting a file in a later layer doesn’t remove it from earlier layers. This is why “I removed the secret later” is not a fix.
  • Fact 2: Environment variables have been a standard configuration mechanism since early Unix. They’re convenient—and historically terrible at confidentiality.
  • Fact 3: The original Docker build process sent the whole build context as a tar stream to the daemon. That behavior shaped years of accidental secret exposure in CI builders.
  • Fact 4: BuildKit introduced secret mounts specifically because people kept stuffing credentials into build args and image layers to access private package registries.
  • Fact 5: docker history can reveal commands used to build an image. If a secret appears in a RUN command line, it can be visible even if the file isn’t.
  • Fact 6: Many container runtimes and orchestrators persist environment variables in metadata stores (and sometimes in logs), multiplying the blast radius of “just put it in ENV.”
  • Fact 7: Early container adoption encouraged “single artifact” thinking—bake everything into the image. Security practice moved the opposite direction: immutable images, mutable secrets injected at runtime.
  • Fact 8: Git’s design makes removing secrets hard: even if you delete a file, it remains in history unless you rewrite it. The best leak is the one you never commit.
  • Fact 9: Many high-profile breaches started with credential discovery in source repositories or artifacts, not with a novel exploit chain. Attackers love scavenger hunts.

Rules of the road: where secrets may live, and where they never may

What you do

  • Keep secrets out of the build context. Use .dockerignore aggressively and treat it as a security control, not a performance tweak.
  • Inject secrets at runtime via mounted files. Prefer /run/secrets (conventional path) or a read-only mount under a dedicated directory.
  • Use BuildKit secret mounts for build-time auth. This is the least-bad way to fetch private dependencies without leaking credentials into layers.
  • Make “secure” the default layout. Developers will follow the path of least resistance; your repo should make the safe path the shortest path.
  • Scan continuously. Don’t trust humans. Also don’t trust yourself from six months ago.

What you don’t do

  • No secrets in ENV or Compose environment. If it’s a secret, it should be in a file or a secret store integration, not in metadata.
  • No copying secret files in Dockerfile. Not even “for a second.” Layers are forever.
  • No “debug output” that prints env. If your debug script starts with env | sort, delete it and apologize to the future.
  • No committing real secrets, even “just for testing.” Testing is how leaks become permanent.

One quote to keep you honest

Paraphrased idea from Gene Kim (DevOps/operations author): “The best way to improve reliability is to make problems visible and fix them systematically.”

Practical tasks: 14 commands to audit and fix leaks

These are production-grade tasks. Each includes: the command, what the output means, and what decision you make. Run them on a developer machine, in CI, and on a build server. Different environments leak in different ways.

Task 1: Confirm your build context is not sending secrets

cr0x@server:~$ docker build --no-cache --progress=plain -t acme/app:check .
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 612B done
#2 [internal] load .dockerignore
#2 transferring context: 2B done
#3 [internal] load metadata for docker.io/library/python:3.12-slim
#4 [internal] load build context
#4 transferring context: 48.35kB 0.0s done
...

Output meaning: “transferring context” shows the size sent to the daemon. If you see megabytes you didn’t expect, you’re probably sending junk—or secrets.

Decision: If context is large or suspicious, tighten .dockerignore and re-run until the context size matches what you’d be comfortable emailing to a stranger.

Task 2: Verify .dockerignore is actually applied

cr0x@server:~$ docker build -t acme/app:ignore-test --no-cache --progress=plain .
#4 [internal] load build context
#4 transferring context: 48.35kB done
#4 DONE 0.1s

Output meaning: If you temporarily place a big file in secrets/runtime and the context size doesn’t change, your ignore patterns are working. If it grows, they’re not.

Decision: Fix patterns until the context stays stable even when secrets/ contains real files.

Task 3: Scan the repository for common secret file names

cr0x@server:~$ find . -maxdepth 4 -type f \( -name ".env" -o -name ".env.*" -o -name "*.pem" -o -name "*.key" -o -name "*kubeconfig*" \) -print
./secrets/dev/app.env.example

Output meaning: Anything outside secrets/dev that looks like a secret is a problem waiting for a commit.

Decision: Move real secret-like files into secrets/runtime and ensure they’re ignored by git and Docker.

Task 4: Ensure git won’t accept secrets under secrets/runtime

cr0x@server:~$ cat .gitignore
# runtime secrets must never be committed
secrets/runtime/*
!secrets/runtime/.keep

# local env files
.env
.env.*

Output meaning: The negation line keeps a placeholder file so the directory exists. Everything else is ignored.

Decision: If secrets/runtime is not ignored, fix it now; otherwise someone will accidentally commit during a rushed hotfix.

Task 5: Detect already-tracked secrets (the “ignored but committed” trap)

cr0x@server:~$ git ls-files | grep -E '(^|/)\.env(\.|$)|secrets/runtime|\.pem$|\.key$' || true

Output meaning: If anything prints, it’s already in git history or tracked in the index.

Decision: Remove from the index and rotate the credential. Ignoring doesn’t un-leak.

Task 6: Remove accidentally tracked secret files (without deleting local copies)

cr0x@server:~$ git rm --cached -r secrets/runtime
fatal: pathspec 'secrets/runtime' did not match any files

Output meaning: “did not match” is good: nothing tracked there. If it removes files, you had a problem.

Decision: If it removed files, commit the removal and immediately rotate any secrets that were present.

Task 7: Inspect image history for leaked build arguments or commands

cr0x@server:~$ docker history --no-trunc acme/app:check | head
IMAGE          CREATED          CREATED BY                                      SIZE      COMMENT
a1b2c3d4e5f6   2 minutes ago    CMD ["python" "-m" "src.app"]                   0B        buildkit.dockerfile.v0
<missing>      2 minutes ago    ENV APP_CONFIG=/app/config/logging.yaml        0B        buildkit.dockerfile.v0
<missing>      3 minutes ago    RUN /bin/sh -c useradd -r -u 10001 appuser...  1.2MB     buildkit.dockerfile.v0

Output meaning: You’re looking for tokens, passwords, private URLs, or echo statements that wrote secrets.

Decision: If anything secret-like appears, rebuild with a corrected Dockerfile and revoke/rotate credentials. Also purge old images from registries if possible.

Task 8: Inspect the image filesystem for “oops” files

cr0x@server:~$ docker run --rm acme/app:check sh -lc 'ls -la /run /run/secrets || true; find /app -maxdepth 3 -type f -name ".env" -o -name "*.pem" -o -name "*.key" 2>/dev/null || true'
ls: /run/secrets: No such file or directory

Output meaning: In an image, /run/secrets typically won’t exist unless created. That’s fine. What’s not fine is finding .env, keys, or certificates inside the image.

Decision: If secret files exist in the image, you must treat them as compromised and rebuild clean.

Task 9: Confirm containers aren’t running with secret env vars

cr0x@server:~$ docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}' $(docker ps -q --filter name=app) | grep -Ei 'pass|token|secret|key' || true

Output meaning: No output is the goal. If you see secret-ish variables, you’re exposing them via inspection and possibly logs.

Decision: Move those secrets to mounted files or orchestrator secrets and scrub them from environment configuration.

Task 10: Confirm mounted secret files exist and are read-only

cr0x@server:~$ docker exec -it $(docker ps -q --filter name=app) sh -lc 'mount | grep /run/secrets; ls -la /run/secrets'
/dev/sda1 on /run/secrets type ext4 (ro,relatime)
total 8
drwxr-xr-x 2 root root 4096 Feb  4 10:22 .
drwxr-xr-x 1 root root 4096 Feb  4 10:22 ..
-r--r----- 1 root root   64 Feb  4 10:22 db_password

Output meaning: The mount shows (ro,...) and the secret file is not world-readable.

Decision: If it’s rw or permissions are loose, fix Compose/Kubernetes manifests. Secrets should be readable only by the app user, not the whole container.

Task 11: Check file ownership vs. container user (permission mismatch diagnosis)

cr0x@server:~$ docker exec -it $(docker ps -q --filter name=app) sh -lc 'id; stat -c "%U %G %a %n" /run/secrets/db_password'
uid=10001(appuser) gid=10001(appuser) groups=10001(appuser)
root root 440 /run/secrets/db_password

Output meaning: If the container runs as appuser but the file is owned by root with 440, your app might not be able to read it unless group permissions align.

Decision: Either run secrets with appropriate group ownership, use 0444 for read-only where acceptable, or configure your orchestrator secret projection with correct UID/GID.

Task 12: Scan the image for high-entropy strings (cheap secret leak detector)

cr0x@server:~$ docker save acme/app:check | tar -xOf - | strings | grep -E '[A-Za-z0-9+/]{32,}={0,2}' | head

Output meaning: This is noisy, but it catches “base64-looking” blobs that sometimes indicate embedded tokens.

Decision: If you see anything suspicious, narrow it down by exporting the filesystem and grepping specific paths. Treat positives seriously.

Task 13: Use BuildKit secret mount correctly (private dependency example)

cr0x@server:~$ DOCKER_BUILDKIT=1 docker build \
  --secret id=pypi_token,src=./secrets/runtime/pypi_token \
  --progress=plain -t acme/app:with-private-deps .
#6 [runtime 2/6] RUN --mount=type=secret,id=pypi_token ...
#6 DONE 8.7s

Output meaning: You see a RUN --mount=type=secret step. That indicates you’re not copying the token into the image.

Decision: If the Dockerfile uses ARG instead, stop and refactor. Build args are not secret storage.

Task 14: Prove the secret didn’t end up in the resulting image

cr0x@server:~$ docker run --rm acme/app:with-private-deps sh -lc 'grep -R "pypi" -n / 2>/dev/null | head'

Output meaning: You’re looking for token strings or auth config files. Ideally this returns nothing meaningful (it may return irrelevant matches; investigate if it does).

Decision: If the token or auth config exists in the final filesystem, the build step leaked it. Rotate and fix the Dockerfile.

Three corporate mini-stories from the land of “it worked on my laptop”

Mini-story 1: The incident caused by a wrong assumption

A payments-adjacent team had a Docker image that built fine for months. They used a private package registry and assumed their build argument was “safe enough” because it was only passed in CI. The Dockerfile took ARG NPM_TOKEN, wrote it to ~/.npmrc, installed dependencies, and then deleted the file. Clean, right?

It wasn’t clean. The token had already lived in a layer. Someone later ran an internal image scanning job that saved images to a shared artifact store for analysis. Another team—trying to debug a build issue—pulled the artifact and unpacked the tar. That token wasn’t “in prod,” but it was still a credential that worked.

The blast radius was embarrassing, not catastrophic: access to internal packages, a minor supply-chain exposure, and a week of forced rotation across projects. The real damage was morale. Engineers who had been careful about “not committing secrets” learned that Docker layers are their own kind of history.

The fix was boring: BuildKit secret mounts, a strict .dockerignore, and a policy that tokens must not appear in Dockerfile instructions. The team also stopped uploading raw images as “debug artifacts.” They uploaded build logs with scrubbers and only the metadata they needed.

After the dust settled, someone asked why this wasn’t caught earlier. The answer was simple: everyone assumed “delete the file” means “it’s gone.” That assumption is how layers get you.

Mini-story 2: The optimization that backfired

A platform group optimized CI build times by enabling aggressive Docker layer caching on shared runners. Builds got faster, which meant everybody was happy for about two sprints. Then they started seeing “ghost” behavior: a feature branch could build successfully even after removing access to a secret, because the layer that used the secret was cached.

The worse part: a developer added a temporary debug line printing a configuration value that happened to include a token. The build logs were retained longer than they should have been. Meanwhile, cached layers containing dependency download configs were shared across projects. That’s not supposed to happen, but “supposed to” is not an access control mechanism.

The incident wasn’t a breach in the classic sense. It was an exposure. Security filed it as “credential handling failure.” The platform team filed it as “cache scope misconfiguration.” Everyone agreed the root cause was an optimization that reduced friction for builds and increased friction for containment.

They kept caching—but made it disciplined: per-project isolated caches, no cross-tenant sharing, and explicit rules that any build step using secrets must be non-cacheable or use secret mounts that don’t contaminate layers. They also shortened log retention and scrubbed known key patterns.

Performance is a feature. Security is also a feature. If you optimize one by silently borrowing from the other, you’ll pay interest later.

Mini-story 3: The boring but correct practice that saved the day

A healthcare-ish company had a simple standard: secrets/runtime exists in every service repo, is ignored by git and Docker, and pipelines mount it at runtime. No exceptions. Engineers complained for a month. Then they stopped noticing it.

One Friday, someone accidentally copied a production credential into a local file named prod.env in the repository root. The developer was about to commit. The pre-commit hook screamed. CI also screamed. The build context stayed small because .dockerignore ignored .env* patterns, and the repo-level secret scan flagged the high-entropy blob.

The credential never landed in git. It never landed in the image. It never landed in the registry. The person rotated it anyway, because they were trained to treat “near-miss” as “maybe compromised.” That rotation took minutes, not days, because they had a documented procedure and a consistent secret injection path.

Security didn’t have to play detective. SRE didn’t have to wake anyone up. The system didn’t save the day with fancy crypto; it saved the day with predictable file placement and multiple small guardrails.

Joke 2: The best secret is the one you never leak. The second-best secret is the one you rotate before anyone notices you leaked it.

Fast diagnosis playbook (check first/second/third)

When someone says “we leaked a secret” or “the scanner says the image contains credentials,” don’t debate theory. Triage fast. Here’s the playbook I use.

First: confirm whether the secret is in git history, the image, or runtime only

  • Check git tracking: git ls-files + grep patterns; if tracked, treat as compromised.
  • Check image history: docker history --no-trunc for visible command lines.
  • Check filesystem: run a container and find for common secret file types.

Why first: the remediation differs. Git history is permanent unless rewritten. Image registries replicate. Runtime-only exposure might be limited to a single host.

Second: identify the injection mechanism

  • Env vars: docker inspect and check orchestration manifests.
  • Mounted files: verify /run/secrets mounts and permissions.
  • Build-time secrets: find ARG, --build-arg, and package manager auth files (.npmrc, .netrc).

Why second: you need to prevent recurrence. Rotating the secret is step zero; fixing the path is the real work.

Third: scope the blast radius and rotate decisively

  • Search CI logs and artifacts for the secret string or key ID patterns.
  • Check registries for affected tags/digests; remove or quarantine them.
  • Rotate credentials; invalidate tokens; re-issue keys; update deployments using the new secret path.

Why third: secrets leak faster than you can schedule a meeting. Rotation beats analysis paralysis.

Common mistakes: symptoms → root cause → fix

Mistake 1: “We deleted the file in a later Dockerfile step”

Symptoms: secret scanner flags the image; you can’t find the file in the running container; security insists it’s still there.

Root cause: the secret existed in an earlier layer; deletion only masked it in the final filesystem view.

Fix: rebuild without ever writing the secret to a layer. Use BuildKit secret mounts or fetch dependencies outside the image build. Purge old image tags and rotate credentials.

Mistake 2: “It’s fine, it’s only in environment variables”

Symptoms: secrets appear in docker inspect; support bundles contain tokens; someone pastes config into chat and now it’s in searchable history.

Root cause: env vars are metadata; they get copied, logged, scraped, and inspected.

Fix: mount secrets as files under /run/secrets (or orchestrator-projected secrets). Keep env vars for non-secret toggles. If an app only supports env vars, wrap it: read from file and export in a tiny entrypoint, but accept the residual risk and restrict access to inspect/logs.

Mistake 3: “Our .dockerignore exists, so we’re safe”

Symptoms: CI build context transfer is huge; secrets are found in builder cache; different machines produce different results.

Root cause: .dockerignore is incomplete, or you’re building from a different context than you think (monorepo builds, wrong directory, CI path differences).

Fix: verify context size in build logs; explicitly set context: in Compose; add tests that fail if secrets/ is in the context; standardize build entry points (scripts) so developers don’t freehand builds.

Mistake 4: “We mount secrets, but the app can’t read them”

Symptoms: app crashes with permission denied; secret files exist but reads fail; engineers ‘fix’ it by running as root.

Root cause: UID/GID mismatch and overly strict file mode, or secrets projected with root-only permissions.

Fix: run the app with a stable UID; project secrets with correct ownership; use group-readable modes; avoid “run as root” as a patch.

Mistake 5: “We mounted ./secrets into the container and forgot it’s writable”

Symptoms: secret files change unexpectedly; local dev secrets get overwritten; someone commits modified files by accident.

Root cause: writable bind mount from the host; container processes write back to the host filesystem.

Fix: mount secrets read-only. Make the container filesystem read_only: true and allow writes only to tmpfs paths.

Mistake 6: “CI uploaded artifacts for debugging, including the workspace”

Symptoms: secret appears in CI artifact store; token string found in archived tarballs; security asks why your pipeline is a data exfiltration service.

Root cause: artifact upload rules too broad; no scrubber; no separation between build output and workspace state.

Fix: upload only explicit build outputs; never upload docker inspect or full workspace; scrub logs; keep secrets out of workspace in the first place with the layout described here.

Checklists / step-by-step plan

Step-by-step: adopt the file layout in an existing repo

  1. Create directories: config/, secrets/dev/, secrets/runtime/, scripts/.
  2. Move non-secret configs from repo root into config/. Keep committed configs non-secret.
  3. Create example secret templates in secrets/dev (e.g., app.env.example), and document expected keys.
  4. Add strict .gitignore for secrets/runtime and .env*.
  5. Add strict .dockerignore to exclude secrets, git, and CI junk.
  6. Refactor Dockerfile to copy only src/ and config/. Remove any COPY . . unless you enjoy audits.
  7. Switch runtime to mounts in Compose/Kubernetes. Standardize on /run/secrets.
  8. Stop using build args for secrets. Replace with BuildKit secret mounts where unavoidable.
  9. Add guardrails: a script that scans for secrets and fails CI. Make it fast enough to run on every PR.
  10. Rotate credentials if you find anything suspect in history, images, registries, or logs.

CI checklist (minimum viable safety)

  • Build with BuildKit enabled (DOCKER_BUILDKIT=1).
  • Do not print environment variables in logs.
  • Do not upload workspace archives as artifacts.
  • Do not share Docker layer cache across projects/tenants unless it’s designed and audited for that.
  • Run a secret scan against the git diff and against the built image.

Runtime checklist (containers in production)

  • Run as non-root with a stable UID.
  • Mount secrets read-only; project them with correct permissions.
  • Make the root filesystem read-only; use tmpfs for writable scratch.
  • Drop Linux capabilities and set no-new-privileges.
  • Ensure logs never include secret file contents or environment dumps.

FAQ

1) Is a .env file always a bad idea?

No. A .env file is a convenient format. The bad idea is letting it drift into git, Docker build contexts, images, or artifacts. Put real ones in secrets/runtime and ignore them.

2) Why mounted files instead of environment variables?

Mounted files are less likely to be logged, scraped, or exposed via metadata inspection. They also support tighter filesystem permissions. Env vars are easy; easy isn’t the same as safe.

3) What about Docker Compose “secrets” support?

Use it where supported, but don’t rely on it as magic. Many teams still end up bind-mounting secret files. The layout works for both: a stable place on the host or in CI where secrets are stored and mounted at runtime.

4) Can BuildKit secrets leak anyway?

Yes, if you print them, write them to disk, or copy them into the image. BuildKit gives you a safe injection mechanism; it doesn’t override bad scripting.

5) We need a private dependency registry during build. What’s the safest pattern?

Use BuildKit --secret mounts and configure the package manager to read credentials from the secret mount for the duration of the install step. Keep the step minimal and avoid caching if you can’t prove it’s clean.

6) If a secret is committed but the repo is private, do we still rotate?

Yes. “Private repo” is not a security boundary you can audit perfectly. Rotate, then clean history if required by policy. The rotation is the urgent part.

7) How do we prevent developers from bypassing the layout?

Make it the default in templates, add pre-commit and CI checks, and enforce .dockerignore and Dockerfile patterns in review. Also: keep the workflow fast so people don’t invent “temporary” hacks.

8) What about Kubernetes?

The same principle applies: secrets injected at runtime, ideally as files (secret volumes). Keep the container image free of secrets. Standardize on a path like /run/secrets so the app doesn’t care where the secret came from.

9) Are read-only filesystems worth the trouble?

Yes. They prevent a whole class of accidents: writing secrets into the container filesystem, mutating config, and leaving credential crumbs in writable layers. Pair it with tmpfs for /tmp and whatever your app needs.

10) What’s the fastest way to see if we’re leaking through build context?

Run docker build --progress=plain and watch the context transfer size. If it’s large or changes when you add a file under secrets/, your ignore rules are wrong.

Next steps (do these this week)

Secrets don’t leak because engineers are careless. They leak because systems are permissive and workflows reward speed. Fix the workflow.

  1. Adopt the layout: src/, config/, secrets/dev (examples), secrets/runtime (real, ignored), strict .dockerignore.
  2. Refactor Dockerfiles to copy only what they need. Delete COPY . . unless you can justify it in writing.
  3. Move secrets to runtime mounts under /run/secrets, read-only, with sane permissions and non-root containers.
  4. Enable BuildKit and use secret mounts for build-time auth. Ban ARG for credentials.
  5. Add two cheap guardrails: scan the repo for secret-shaped files and scan built images for secret-shaped content.
  6. If you find anything: rotate first, analyze second, and don’t negotiate with your own hindsight.
← Previous
Windows Says “Connected, No Internet”? Fix It Without Resetting Everything
Next →
RHEL 10 Install: The Enterprise Setup Checklist You Wish You Had Earlier

Leave a comment