If you’ve ever restarted a “stateless” Postgres container and found a “fresh” database waiting for you like an amnesiac goldfish, you’ve met the
biggest lie in container land: that the database cares about your YAML more than it cares about the filesystem.
Postgres is boring in the best way—until you run it in Docker with the wrong storage choices, sloppy backups, or casual upgrades. Then it becomes a
detective story where the culprit is usually you, yesterday.
What actually goes wrong (and why Docker makes it easy)
Most Postgres “data loss in Docker” incidents aren’t mystical. They’re the collision of two totally reasonable systems:
Docker assumes your container is disposable; Postgres assumes its data directory is sacred.
Docker gives you simple primitives—images, containers, volumes, bind mounts, networks. Postgres gives you strict durability rules—WAL,
fsync, checkpoints, and a violent dislike of corrupted files. When people get hurt, it’s usually from a bad assumption:
“The container is the database.” No. The container is a process wrapper. The database is the on-disk state plus WAL plus backups plus the rules you follow.
You can absolutely run Postgres in Docker. Plenty of production stacks do. But you need to be explicit about:
- Where the data directory lives and how it’s mounted.
- How backups work—and how you’ve proven they restore.
- How you do upgrades without accidentally re-initializing the cluster.
- What durability trade-offs you’ve made (sometimes unknowingly).
- How you detect storage and WAL pressure before it turns into downtime.
The pitfall pattern is consistent: a small convenience choice at day 1 becomes a week-12 outage because nobody revisited it once it “worked.”
Interesting facts & historical context
- Postgres started in the mid-1980s (as POSTGRES at Berkeley) and kept its “do the right thing with data” culture even as tooling evolved.
- Write-Ahead Logging (WAL) is the heart of Postgres durability. It’s not optional in spirit, even when configs try to pretend otherwise.
- Docker volumes are managed by Docker and live under Docker’s data directory by default; this makes them portable across container replacements, but not across host loss.
- Bind mounts predate Docker as a Unix concept. They’re powerful and transparent, which is why they’re also how you shoot your own foot with permissions and path mistakes.
- “docker system prune” has been around for years and remains one of the fastest ways to delete the wrong thing if you treat volumes as “cache.”
- Postgres major upgrades aren’t in-place by default; they usually require dump/restore or pg_upgrade, and both have sharp edges inside containers.
- Overlay filesystems became mainstream with containers; they’re great for images and layers, and a terrible place to store databases unless you enjoy I/O surprises.
- Kubernetes popularized the “pets vs cattle” mantra, which works great until you apply “cattle” logic to the database storage itself.
- Postgres has had strong logical replication for years (publication/subscription); it can be a practical migration path off a bad Docker setup without downtime.
Your mental model: container lifecycle vs. database lifecycle
Containers are replaceable. Database state is not.
A container is a running instance of an image. Kill it, recreate it, reschedule it—fine. That’s the point. Postgres doesn’t care about containers.
Postgres cares about PGDATA (the data directory), and it assumes:
- Files stay put.
- Ownership and permissions remain consistent.
- fsync means fsync.
- WAL reaches stable storage when it says it does.
Your job is to ensure Docker’s storage plumbing doesn’t violate those assumptions. Most issues reduce to:
- Wrong mount target: Postgres writes to container filesystem layers, not persistent storage.
- Wrong mount source: you mounted an empty directory and accidentally triggered initdb.
- Wrong permissions: Postgres can’t write, so it fails or behaves oddly under entrypoint logic.
- Wrong upgrade method: you created a new cluster and pointed apps at it.
- Wrong durability settings: performance “wins” that quietly accept data loss on crash.
Why “it worked on my laptop” is a trap
On a laptop, you might not notice that you’ve stored Postgres inside the container layer. You restart the container a few times; data stays there.
Then someone runs docker rm or CI recreates containers, and the data evaporates because it was never in a volume.
Containers are great at making the wrong thing look stable. They will happily preserve your mistakes until the day they don’t.
Data loss scenarios you can reproduce (and prevent)
Scenario 1: No volume mount → data lives in the container layer
The classic. You run Postgres without a persistent mount. Postgres writes to /var/lib/postgresql/data inside the container filesystem.
Restarting the container keeps data. Deleting/recreating the container destroys it.
Prevention: always mount a Docker volume or bind mount to the actual PGDATA directory, and prove it’s mounted with docker inspect.
Scenario 2: Mounting the wrong path → data goes somewhere else
The official image uses /var/lib/postgresql/data. People mount /var/lib/postgres or /data because they’ve done it elsewhere.
Postgres keeps writing to the default path. Your mounted storage sits unused, beautifully empty, like a spare parachute left on the plane.
Prevention: inspect the container mounts and confirm the database is writing into the mounted filesystem. Check SHOW data_directory; inside Postgres.
Scenario 3: Bind mounting an empty directory → initdb runs and “creates” a new cluster
Many entrypoints initialize a new cluster when PGDATA looks empty. If you accidentally bind mount a brand-new host directory over the real data directory,
the container sees emptiness and runs initdb. You’ve now created a brand-new database next to your real one, and your application connects to the wrong thing.
This is how “data loss” starts as “huh, why is my table missing?” and ends as “why did we write new data into the wrong cluster for six hours?”
Scenario 4: “docker compose down -v” and friends → you asked Docker to delete your database
Docker makes it easy to clean up. That includes volumes. If your Postgres storage is a named volume in Compose, down -v removes it.
If it’s an anonymous volume, you might remove it via prune without noticing.
Prevention: treat the Postgres volume as production state. Protect it with naming conventions, labels, and process. Avoid anonymous volumes for databases.
Scenario 5: Running Postgres on overlay storage → performance weirdness and higher corruption risk
Docker’s writable layer is typically overlay2 (or similar). It’s fine for application logs. It’s not where you want database random I/O and heavy fsync.
Performance becomes inconsistent; latency spikes. Under crash or disk pressure, corruption incidents become more plausible.
Prevention: use a volume or bind mount backed by a real filesystem on the host, not the container writable layer.
Scenario 6: Durability “tuning” that’s really data-loss mode
Turning off fsync or setting synchronous_commit=off can make benchmarks look heroic.
Then the host crashes, and the database is missing recent transactions. That wasn’t “unexpected.” That was the deal.
There are legitimate reasons to relax durability (e.g., ephemeral dev, certain analytics pipelines where losing seconds is fine).
But for anything user-facing: don’t. Postgres is fast enough when you place it on sane storage and configure it properly.
Joke #1: Disabling fsync to “speed up Postgres” is like removing the brakes to “improve commute time.” You’ll get there faster, briefly.
Scenario 7: WAL fills the disk → Postgres stops, and recovery gets messy
WAL is append-heavy. When disk fills, Postgres can’t write WAL, and it will halt. If you also have bad retention settings or no monitoring,
you can end up with a stuck instance and no clean recovery path.
Prevention: monitor disk usage where PGDATA and WAL live. Set sensible max_wal_size, configure archiving if you need PITR, and keep headroom.
Scenario 8: Container timezone/locale mismatch and encoding mistakes → “data loss” by misinterpretation
Not all “loss” is deletion. If you initialize a cluster with different locale/encoding and later compare data dumps, you can see mangled sorting,
collation issues, or broken text expectations. It’s not missing bytes, but it can look like corruption.
Prevention: explicitly set locale/encoding at init, document it, and keep it stable across rebuilds.
Scenario 9: Major version upgrade by swapping image tags → you created an incompatible cluster
Changing postgres:14 to postgres:16 and restarting with the same volume does not “upgrade” Postgres.
Postgres will refuse to start because the data directory format differs. Under pressure, people “fix” this by deleting the volume.
That’s not an upgrade. That’s arson.
Prevention: use pg_upgrade (often easiest with two containers and a shared volume strategy), or logical replication, or dump/restore—depending on size and downtime tolerance.
Scenario 10: Permissions drift (rootless Docker, host UID/GID changes) → Postgres won’t start, someone “recreates” the DB
Bind mounts inherit host permissions. Change the host user mapping, move directories, switch to rootless Docker, or restore from backup with different ownership,
and suddenly Postgres can’t access its own files. In a panic, teams often blow away the mount and “start fresh.”
Prevention: standardize ownership and use named volumes when possible. If you must bind mount, pin UID/GID expectations and test on the same OS family.
Fast diagnosis playbook
When Postgres-in-Docker is misbehaving, don’t wander. Start with the three questions that decide everything: “Where is the data, can it write, and is it durable?”
First: confirm you’re looking at the right cluster
- Check PGDATA mount: is the data directory backed by a volume/bind mount?
- Check data directory path: what does Postgres report as
data_directory? - Check cluster identity: look at
system_identifierand the timeline; compare to what you expect.
Second: check for obvious storage pressure
- Disk full: host filesystem usage where the volume lives.
- WAL bloat:
pg_walsize and replication slots. - I/O stalls: latency and fsync timing; container CPU can look fine while storage is dying.
Third: check durability and crash-recovery status
- Config sanity: ensure
fsyncandfull_page_writesaren’t disabled for production. - Logs: crash recovery loops, “invalid checkpoint record,” or permission errors.
- Kernel/filesystem events: dmesg for I/O errors; these often explain “random” corruption.
How to find the bottleneck quickly
If the symptom is “slow,” decide whether it’s CPU, memory, lock contention, or storage I/O. With Docker, storage is the usual suspect, and the logs often tell you early.
If the symptom is “missing data,” stop writing immediately and verify you didn’t start a new cluster by mistake.
Practical tasks: commands, outputs, and decisions
These are the tasks I actually run when things smell off. Each includes (1) the command, (2) what the output means, and (3) the decision you make.
Assume the container is named pg and you have shell access to the host.
Task 1: List containers and confirm which Postgres is running
cr0x@server:~$ docker ps --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}'
NAMES IMAGE STATUS PORTS
pg postgres:16 Up 2 hours 0.0.0.0:5432->5432/tcp
Meaning: Confirms which image tag is live and whether it recently restarted. A suspicious “Up 3 minutes” often correlates with data directory issues.
Decision: If the image tag changed recently, treat upgrades as the prime suspect. If it restarted unexpectedly, go straight to logs and mount checks.
Task 2: Inspect mounts and verify PGDATA is persisted
cr0x@server:~$ docker inspect pg --format '{{json .Mounts}}'
[{"Type":"volume","Name":"pgdata","Source":"/var/lib/docker/volumes/pgdata/_data","Destination":"/var/lib/postgresql/data","Driver":"local","Mode":"z","RW":true,"Propagation":""}]
Meaning: You want a mount that targets the real data directory. If you see no mount to /var/lib/postgresql/data, your data is in the container layer.
Decision: If the mount is missing or points elsewhere, stop and fix storage before you do anything else. Don’t “restart until it works.”
Task 3: Confirm Postgres thinks its data directory is where you mounted it
cr0x@server:~$ docker exec -it pg psql -U postgres -Atc "SHOW data_directory;"
/var/lib/postgresql/data
Meaning: This should match the mount destination. If it doesn’t, you’re writing somewhere you didn’t intend.
Decision: If mismatched, correct environment variables or command-line flags, and ensure the official image’s expected PGDATA path is used consistently.
Task 4: Check whether you accidentally initialized a new cluster (system identifier)
cr0x@server:~$ docker exec -it pg psql -U postgres -Atc "SELECT system_identifier FROM pg_control_system();"
7264851093812409912
Meaning: The system identifier is effectively the cluster identity. If it changed after a redeploy, you’re not on the same cluster.
Decision: If the identifier is unexpected, stop application writes, locate the original volume/bind mount, and restore connectivity to the correct data directory.
Task 5: Check container logs for initdb or permission errors
cr0x@server:~$ docker logs --since=2h pg | tail -n 30
PostgreSQL Database directory appears to contain a database; Skipping initialization
2026-01-03 10:41:07.123 UTC [1] LOG: starting PostgreSQL 16.1 on x86_64-pc-linux-gnu
2026-01-03 10:41:07.124 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
Meaning: “Skipping initialization” is good. If you see “initdb: warning” or “Database directory appears to be empty” unexpectedly, you mounted an empty directory.
If you see “permission denied,” it’s a bind mount ownership problem.
Decision: Initdb when you didn’t expect it is a red alarm. Don’t proceed until you explain why Postgres thought the directory was empty.
Task 6: Identify whether your volume is named or anonymous
cr0x@server:~$ docker volume ls
DRIVER VOLUME NAME
local pgdata
local 3f4c9b6a7c1b0b3e8b8d8af2c2e1d2f9d8e7c6b5a4f3e2d1c0b9a8f7e6d5
Meaning: Named volumes (pgdata) are easier to protect and reference. Anonymous volumes are easy to lose during cleanup.
Decision: For databases, use named volumes or explicit bind mounts. If you see anonymous volumes attached to Postgres, migrate before “cleanup day.”
Task 7: See what containers are using the volume (avoid deleting the wrong one)
cr0x@server:~$ docker ps -a --filter volume=pgdata --format 'table {{.Names}}\t{{.Status}}\t{{.Image}}'
NAMES STATUS IMAGE
pg Up 2 hours postgres:16
Meaning: If more than one container uses the same volume, you might have accidental multi-writer access. That’s a corruption risk.
Decision: Ensure only one Postgres instance writes a given data directory. If you need HA, use replication, not shared storage multi-writers.
Task 8: Check free space on the host filesystem backing Docker
cr0x@server:~$ df -h /var/lib/docker
Filesystem Size Used Avail Use% Mounted on
/dev/nvme0n1p2 450G 410G 40G 92% /
Meaning: 92% is danger territory for WAL growth and vacuum spikes. Databases don’t fail politely when disks fill.
Decision: If you’re above ~85–90% in production, plan immediate cleanup or expansion. Then set alerting and headroom targets.
Task 9: Check WAL directory size inside the container
cr0x@server:~$ docker exec -it pg bash -lc 'du -sh /var/lib/postgresql/data/pg_wal'
18G /var/lib/postgresql/data/pg_wal
Meaning: Large WAL can be normal under write load, but sudden growth often means stuck replication slots, too-large max_wal_size,
or archiving that isn’t draining.
Decision: If WAL is ballooning, check replication slots and archiver status immediately before disk fills.
Task 10: Check replication slots (common cause of unbounded WAL)
cr0x@server:~$ docker exec -it pg psql -U postgres -x -c "SELECT slot_name, active, restart_lsn FROM pg_replication_slots;"
-[ RECORD 1 ]----------------------------
slot_name | analytics_consumer
active | f
restart_lsn | 0/2A3F120
Meaning: An inactive slot can retain WAL forever if no consumer advances it.
Decision: If the slot is unused, drop it. If it’s needed, fix the consumer and confirm it’s advancing. Don’t just “increase the disk.”
Task 11: Check archiver health if you use WAL archiving
cr0x@server:~$ docker exec -it pg psql -U postgres -x -c "SELECT archived_count, failed_count, last_archived_wal, last_failed_wal FROM pg_stat_archiver;"
-[ RECORD 1 ]-----------------------
archived_count | 18241
failed_count | 12
last_archived_wal | 0000000100000000000001A3
last_failed_wal | 0000000100000000000001A1
Meaning: Failures mean your PITR chain may have gaps. It also means WAL might pile up if archiving is part of your retention plan.
Decision: Investigate the archiving command and storage. If failures are recent, assume recovery options are compromised until proven otherwise.
Task 12: Verify durability settings (catch accidental “benchmark mode”)
cr0x@server:~$ docker exec -it pg psql -U postgres -Atc "SHOW fsync; SHOW synchronous_commit; SHOW full_page_writes;"
on
on
on
Meaning: For production OLTP, this is the baseline you want. If fsync=off, you’ve explicitly accepted corruption risk on crash.
Decision: If these are off, flip them on and plan a controlled restart. Then explain to stakeholders why prior settings were unsafe.
Task 13: Confirm the container isn’t memory-starved (OOM kills look like random crashes)
cr0x@server:~$ docker stats --no-stream pg
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
a1b2c3d4e5f6 pg 85.3% 1.9GiB / 2.0GiB 95.0% 1.2GB/1.1GB 35GB/22GB 78
Meaning: 95% memory use with high CPU suggests pressure. If the host OOM killer intervenes, Postgres restarts and you risk longer recovery.
Decision: Increase memory limits, adjust Postgres memory settings (shared_buffers, work_mem), and verify host-level memory isn’t oversubscribed.
Task 14: Check for filesystem I/O errors on the host (the unglamorous truth)
cr0x@server:~$ sudo dmesg -T | tail -n 20
[Sat Jan 3 10:22:11 2026] nvme0n1: I/O 128 QID 7 timeout, aborting
[Sat Jan 3 10:22:11 2026] EXT4-fs error (device nvme0n1p2): ext4_find_entry:1459: inode #262401: comm postgres: reading directory lblock 0
Meaning: If the kernel reports I/O timeouts or filesystem errors, stop blaming Docker. Your storage is failing or misbehaving.
Decision: Treat as an incident. Reduce write load, take a backup if possible, and plan for failover or replacement storage.
Task 15: Confirm backups exist and are restorable (don’t confuse “files” with “backups”)
cr0x@server:~$ docker exec -it pg bash -lc 'ls -lh /backups | tail -n 5'
-rw-r--r-- 1 root root 1.2G Jan 3 02:00 pg_dumpall_2026-01-03.sql.gz
-rw-r--r-- 1 root root 1.1G Jan 2 02:00 pg_dumpall_2026-01-02.sql.gz
Meaning: You have artifacts. That’s not the same as a verified restore.
Decision: If you can’t prove restore, schedule a restore test. Production is not the time to learn your dumps are empty or truncated.
Task 16: Run a quick restore smoke test into a throwaway database
cr0x@server:~$ docker exec -it pg bash -lc 'createdb -U postgres restore_smoke && gunzip -c /backups/pg_dumpall_2026-01-03.sql.gz | psql -U postgres -d restore_smoke -v ON_ERROR_STOP=1'
SET
SET
CREATE TABLE
ALTER TABLE
Meaning: If it runs without errors and creates objects, your dump is at least syntactically usable. It’s not a full validation, but it’s better than vibes.
Decision: If errors occur, stop assuming you have backups. Fix the pipeline and rerun until it’s boring.
Three corporate-world mini-stories
Mini-story #1: The wrong assumption (the “containers are persistence” incident)
A mid-sized SaaS team moved a legacy app into Docker Compose to make local dev match staging. Postgres went into the same Compose file.
The engineer who did the migration had good intentions and a deadline, which is the combination that generates the most interesting outages.
They tested restarts with docker restart. Data was still there. They high-fived the universe and shipped the same Compose file to a small production VM.
The Postgres service had no volume configured—just the default container filesystem.
Weeks later, they rotated the host for a security patch. The new host started the Compose stack, and Postgres came up clean… because it was empty.
The application also came up clean… and started recreating tables, because the migration tool saw “no schema” and proceeded like a helpful toddler with permanent markers.
The incident response was messy because the team initially treated it like corruption. They chased WAL settings and kernel versions.
The real cause was simpler: they had never persisted the cluster. There was no “restore from disk.” There was only “restore from backup,” and the backups were partial.
The corrective action that stuck: they added a named volume, pinned PGDATA explicitly, and wrote a preflight script that refused to start production if the data directory looked freshly initialized.
That script annoyed people exactly once, then saved them from repeating the same mistake during a later redeploy.
Mini-story #2: The optimization that backfired (the “fast disk” that lied)
Another organization had a noisy Postgres container and a lot of write traffic. Latency spikes showed up during peak, and the usual suspects were blamed:
autovacuum, locks, query plans. They did some tuning, got modest wins, and still saw occasional stalls.
An infrastructure engineer suggested moving the Docker data directory to a “faster” network-backed filesystem used elsewhere for artifacts.
It benchmarked well for sequential writes and large files. Postgres uses fsync-heavy patterns, small random I/O, and metadata churn.
The move looked fine in synthetic tests and even in the first few days of production.
Then came the first real host hiccup. A brief storage stall caused Postgres to log I/O warnings and then crash. On restart it entered crash recovery.
Recovery took much longer than expected, and application timeouts turned into a user-visible outage.
The postmortem was blunt: the “fast” filesystem was optimized for throughput, not latency consistency and durability semantics.
Worse, its behavior under load didn’t match what Postgres expects when it calls fsync. They had traded short-term performance for brittle recovery behavior.
They rolled back to local SSD-backed storage and focused on the boring fixes: right-sizing the instance, tuning checkpoints, and adding replica capacity.
The performance improved, but the real win was stability—recovery became predictable again.
Mini-story #3: The boring but correct practice that saved the day (restore drills)
A large internal platform team ran Postgres in containers for dozens of small services. Nothing exotic: volumes, pinned versions, decent monitoring.
The part that felt overkill to newcomers was the quarterly restore drill. Every quarter, they restored a subset of databases into an isolated environment.
They verified schema, row counts for a few key tables, and application smoke tests.
One day, a host suffered a storage failure. A Postgres primary died hard. The replica was behind more than they liked, and there was uncertainty about WAL availability.
Nobody panicked, which is not a personality trait—it’s a procedure.
They failed over where they could. For one service, they had to restore from backup to a new volume because replication state was questionable.
The restore was slower than a failover but worked exactly as rehearsed. The service returned with acceptable data freshness loss, already agreed with the business.
In the debrief, the team’s “secret sauce” wasn’t a clever tool. It was repetition. They had practiced restoring so many times that the real incident felt like a slightly
more annoying drill.
Common mistakes: symptoms → root cause → fix
1) “My database reset after redeploy”
- Symptoms: Empty schema, default users only, app migrations run from scratch.
- Root cause: No persistent volume, or mounted an empty directory over PGDATA causing initdb.
- Fix: Stop writes, locate the original volume/bind mount, and reattach it. Add a named volume and a startup guard that checks for expected cluster identity.
2) “Data is there, but the app can’t find it”
- Symptoms: psql shows correct data; app sees missing rows/tables; or app connects but returns “relation does not exist.”
- Root cause: App connecting to a different Postgres instance/port, wrong database name, wrong network, or wrong volume attached to a similarly named container.
- Fix: Confirm connection string, container name resolution, port mappings, and
system_identifier. Label volumes and containers clearly.
3) “Postgres won’t start after upgrade”
- Symptoms: Fatal error about database files being incompatible with server.
- Root cause: Major version change without pg_upgrade or dump/restore.
- Fix: Roll back to the previous image tag to restore service. Plan a real upgrade: pg_upgrade in a controlled workflow or logical replication migration.
4) “WAL keeps growing until disk fills”
- Symptoms:
pg_walhuge; disk usage climbs; Postgres eventually stops. - Root cause: Inactive replication slot, archiving failures, or replication lag with retention constraints.
- Fix: Identify slots, drop unused ones, fix consumers, fix archiving, and add alerts on WAL directory growth and disk headroom.
5) “Random restarts, sometimes during load”
- Symptoms: Container restarts; logs show abrupt termination; queries fail intermittently.
- Root cause: OOM kills due to tight container memory limits, or host memory pressure.
- Fix: Increase container memory, tune Postgres memory settings, and avoid overcommitting the host. Confirm with
dmesgand container stats.
6) “We restored the volume from a snapshot and now Postgres complains”
- Symptoms: Crash recovery fails, missing WAL segments, inconsistent state errors.
- Root cause: Storage-level snapshot taken without filesystem/application coordination; snapshot captured an inconsistent point-in-time relative to WAL.
- Fix: Prefer logical backups or coordinated physical backups (pg_basebackup, archiving). If you snapshot, freeze I/O or use filesystem features designed for crash-consistent snapshots and test restore.
7) “Permissions denied on startup”
- Symptoms: Postgres logs mention permission denied in PGDATA; container exits immediately.
- Root cause: Bind mount owned by the wrong UID/GID; SELinux labeling issues; rootless Docker mismatch.
- Fix: Fix ownership to Postgres user, adjust mount options, consider named volumes to avoid host FS permission drift, and standardize UID/GID.
8) “Performance is unpredictable: great then awful”
- Symptoms: Latency spikes, slow checkpoints, autovacuum stalls, random I/O waits.
- Root cause: Overlay storage, network filesystems, noisy neighbors, mis-sized checkpoints, or WAL on slow storage.
- Fix: Put PGDATA on stable local storage, tune checkpoint parameters, monitor fsync/checkpoint timing, and isolate the database from competing disk workloads.
Checklists / step-by-step plan
Checklist A: The “I’m about to run Postgres in Docker for real” plan
- Choose storage on purpose. Use a named volume or a bind mount to dedicated host storage. Don’t rely on the container writable layer.
- Pin image tags. Use
postgres:16.1(example) rather thanpostgres:latest. “Latest” is not a strategy. - Lock down volume naming. Name it like a production asset. Add labels that indicate environment and service.
- Set explicit PGDATA. Keep it consistent across environments and scripts.
- Configure backups from day one. Decide: logical dumps, physical backups + WAL archiving, or both.
- Test restores. Run a restore smoke test regularly, not when you’re already in trouble.
- Alert on disk and WAL growth. You want to know at 70–80%, not at 99%.
- Keep durability defaults unless you can defend changes. If you change fsync-related settings, document the data-loss budget explicitly.
- Plan upgrades like migrations. Major upgrades require a procedure, not a tag flip.
Checklist B: “We suspect data loss” incident steps
- Stop writes. If the app is writing into a wrong or fresh cluster, every minute increases damage.
- Capture evidence. Container logs,
docker inspectoutput, Postgressystem_identifier, and mount details. - Identify the correct data directory. Find the volume/bind mount that contains the expected cluster (look for
PG_VERSION, relation files, and matching identifier). - Confirm backup status. What’s the newest restorable backup? Are WAL archives intact if you need PITR?
- Recover service safely. Prefer reattaching the correct volume. If you must restore, restore into a new volume and validate before switching.
- Prevent recurrence. Add startup guards, remove anonymous volumes, and protect production volumes from prune workflows.
Checklist C: Upgrade path that won’t ruin your weekend
- Inventory extensions. Ensure extensions exist and are compatible with the target Postgres version.
- Pick an upgrade approach: pg_upgrade (fast, needs coordination) vs dump/restore (simple, can be slow) vs logical replication (zero/low downtime, more moving parts).
- Clone production data. Use a staging environment with realistic data to rehearse the upgrade.
- Time the cutover. Have an explicit rollback plan: old image + old volume untouched.
- Validate. Run application smoke tests, check row counts on critical tables, and compare key query performance.
FAQ
1) Is it “safe” to run Postgres in Docker in production?
Yes, if you treat storage, backups, and upgrades as first-class. Docker doesn’t remove database responsibilities; it adds new ways to misconfigure them.
2) Docker volume or bind mount—what should I use?
Named Docker volumes are usually safer operationally: fewer permission surprises, easier to reference, and less coupling to host paths.
Bind mounts can be great when you need to control the filesystem and snapshots, but they demand disciplined ownership and path management.
3) Why did my database “reset” when I changed a Compose file?
Often because the service name, volume name, or mount path changed, causing Docker to create a new volume, or because you mounted a new empty host directory.
Postgres then initialized a fresh cluster.
4) Can I just upgrade by changing the image tag?
Minor versions: usually yes. Major versions: no. Major versions need pg_upgrade, dump/restore, or logical replication migration. Tag flips are how you discover incompatibility at runtime.
5) What’s the fastest way to confirm I’m on the right data?
Query system_identifier, check data_directory, and verify mounts with docker inspect. If those don’t line up, don’t trust anything else.
6) Why is WAL huge even though traffic is normal?
Common causes: an inactive replication slot, a lagging replica, or WAL archiving failures. WAL retention is not “cleanup.” It’s a contract with consumers.
7) Is “docker system prune” safe on a host running Postgres containers?
It can be, but only if your operational rules are strict and you understand what it will remove. If your database uses anonymous volumes or “unused” volumes,
prune can delete the wrong thing. Treat prune like a chainsaw: useful, not subtle.
8) What durability settings should I never change in production?
Don’t disable fsync or full_page_writes for OLTP systems. Be cautious with synchronous_commit.
If you relax durability, write down the exact data-loss window you accept and get buy-in.
9) How do I avoid accidental initdb when mounting storage?
Use a startup guard: check for an expected marker file, expected PG_VERSION, and (ideally) expected system_identifier.
Refuse to start if the directory looks brand new in an environment where it shouldn’t.
10) Why does performance get worse after moving to “better” storage?
Many storage systems optimize for throughput, not latency consistency and fsync semantics. Databases punish inconsistent latency. Measure fsync and checkpoint timings,
and choose storage that behaves well under pressure.
Quote (paraphrased idea) from Richard Cook: “In complex systems, failures are normal; success requires ongoing adaptation.” — Richard Cook, operations and safety researcher.
Joke #2: The only thing more persistent than a Docker volume is an engineer insisting they don’t need backups—right up until they do.
Conclusion: next steps you can do today
If you run Postgres in Docker, your job is not to make it “container-native.” Your job is to make it boring. Boring storage. Boring backups. Boring upgrades.
The exciting path is the one that ends with a blank schema and a long night.
- Audit mounts: verify PGDATA is on a named volume or a deliberate bind mount, and that Postgres reports the expected data_directory.
- Protect volumes: eliminate anonymous volumes for databases and stop using
down -vin any workflow that touches production. - Verify backups by restoring: run a restore smoke test this week, then schedule it regularly.
- Check WAL risks: inspect replication slots and archiver status, and add alerts for WAL growth and disk headroom.
- Write the upgrade runbook: pin versions and pick a real major-upgrade method before you need it.