PostgreSQL Startup Failed: could not create lock file "/var/run/postgresql/.s.PGSQL.5432.lock"#
Deep technical analysis, RCA (Root Cause Analysis), and definitive fix for production environments.
Prerequisites#
Before implementing any fixes, verify your system tools and access rights:
- Systemd: Version 211+ (required to support the
RuntimeDirectorydirective) - Verify version:
systemd --version | head -3 - Verify override capability:
systemctl edit --help | head -5 - PostgreSQL: Any active Linux version (e.g., 12, 13, 14, 15, 16)
- Systemd Tmpfiles: If using the
tmpfiles.dalternative configuration - Verify availability:
which systemd-tmpfiles - Root/Sudo Access: Administrative privileges required to alter systemd units and write config files.
This is the type of incident that seems simple but can open an operational hole if you only fix it with a quick patch. The error message mentions a "lock file," but the real problem lies in the volatile directory lifecycle of modern Linux and how the service was orchestrated.
1) Incident scenario#
You attempt to start the service:
sudo systemctl start postgresql
# Job for postgresql.service failed because the control process exited with error code.
In systemctl status, the cause often appears superficial. The root error usually surfaces in the instance log:
FATAL: could not create lock file "/var/run/postgresql/.s.PGSQL.5432.lock": No such file or directory
When this happens, PostgreSQL is not "broken." It simply cannot create the socket/lock because the expected parent directory does not exist in that boot cycle.
2) RCA (root cause analysis): where the environment failed#
2.1 /var/run is volatile (tmpfs)#
In many distributions, /var/run (or a symlink to /run) lives in memory. When the host restarts, the content evaporates.
2.2 binary expectation#
The Postgres process expects the runtime directory to already exist to bind the Unix socket and create the lock.
2.3 orchestration failure#
If the systemd unit does not correctly declare the runtime directory (or a manual installation left a gap), nothing recreates /var/run/postgresql before the service starts.
Result: Consistent startup failure, especially after a reboot.
3) Professional troubleshooting workflow#
In a real incident, I don't get stuck in the systemctl abstraction. I go directly to the binary to isolate the error.
Step 1 - bypass systemd#
sudo -u postgres /usr/bin/postgres -D /var/lib/pgsql/data
This exposes path/permission errors without orchestration noise.
Step 2 - confirm if 5432 is free (or zombie/concurrent processes)#
# 1. Verify if TCP port 5432 is currently active
ss -ltnp | grep 5432
# 2. List all active postgres processes to check for concurrency
ps aux | grep postgres | grep -v grep
# 3. Identify if multiple database instances exist in the system
pg_lsclusters
# 4. Search the system for postgres zombie processes
ps aux | awk '{if ($8=="Z") print}'
If a process is already listening, there might be an instance/parallel service collision. If nothing is there, it reinforces the directory/socket diagnosis.
Step 3 - validate runtime dir, disk space, and ownership#
# 1. Verify if the runtime folder exists and what its permissions are
ls -ld /var/run/postgresql /run/postgresql 2>/dev/null
# 2. Confirm if the postgres user has a valid UID/GID
id postgres
# 3. Check for available free space on the volatile /run (tmpfs) mount
df -h /run
# 4. Check available inodes on the tmpfs filesystem
df -i /run
# 5. Confirm if the /run path is mounted correctly as tmpfs
mount | grep -w "/run"
Here you confirm if the path exists, if resources are sufficient on the tmpfs drive, and if ownership/permissions are compatible.
4) Fix: from hotfix to permanent solution#
4.0 preventive systemd configuration backup#
Before applying any modifications to your process manager configuration, create a backup of your unit files:
# 1. Create a safe backup directory
sudo mkdir -p /root/systemd-backup-$(date +%Y%m%d)
# 2. Back up the original PostgreSQL service unit file
sudo cp /lib/systemd/system/postgresql.service /root/systemd-backup-$(date +%Y%m%d)/
# Note: On some RedHat/CentOS distributions, the path might be /usr/lib/systemd/system/postgresql.service
# 3. Check for any existing active systemd configuration overrides
ls -la /etc/systemd/system/postgresql.service.d/ 2>/dev/null
4.1 immediate hotfix (service up now)#
sudo mkdir -p /var/run/postgresql
sudo chown postgres:postgres /var/run/postgresql
sudo chmod 775 /var/run/postgresql
sudo systemctl start postgresql
This solves it instantly but is not permanent if you don't handle automatic recreation at boot.
4.2 definitive solution via systemd (reboot-safe)#
The mature fix is to declare the runtime directory in the service unit:
sudo systemctl edit postgresql
Override content:
[Service]
RuntimeDirectory=postgresql
RuntimeDirectoryMode=0775
Application:
sudo systemctl daemon-reload
sudo systemctl restart postgresql
With this, systemd creates the directory at the correct cycle, sets the mode, and removes it when necessary, maintaining operational idempotency.
5) Complementary alternative: tmpfiles.d#
In some scenarios (distro/packaging customizations), it might also be useful to declare it in tmpfiles:
# /etc/tmpfiles.d/postgresql.conf
d /var/run/postgresql 0775 postgres postgres -
Apply without reboot:
sudo systemd-tmpfiles --create /etc/tmpfiles.d/postgresql.conf
This approach ensures early creation during boot via systemd-tmpfiles.
6) Practical security: why 775 and never 777#
Incorrect permissions on a socket path are a local vector for abuse.
0775(drwxrwxr-x) maintains writing for the owner/group and controlled reading.0777opens the surface to untrusted local users, including potential interference in IPC/sockets.
To consistently validate active socket file access permissions:
# 1. List detailed permissions of the socket file and parent folder
ls -la /var/run/postgresql/
# 2. Check if the postgres user has read/write permissions in the directory
sudo -u postgres test -r /var/run/postgresql/ && echo "Postgres can read"
sudo -u postgres test -w /var/run/postgresql/ && echo "Postgres can write"
# 3. Verify if there are processes maintaining open lock files or sockets in the folder
sudo lsof +D /var/run/postgresql/
# Or targeting the socket directly
sudo lsof /var/run/postgresql/.s.PGSQL.5432 2>/dev/null
For environments with stricter requirements, validate if 0770 is possible based on legitimate socket consumers.
7) Post-incident and verification logs#
Checklist I use to close the RCA with evidence:
- Capture failure log with timestamp.
- Record
/runstate before/after fix. - Record applied unit override.
- Inspect detailed post-fix debug logs:
# Monitor real-time PostgreSQL log output
sudo tail -n 50 /var/log/postgresql/postgresql-*.log
# Query systemd unit logs from the last 10 minutes
sudo journalctl -u postgresql --since "10 min ago"
# Filter systemd logs for error or permission denied indicators
sudo journalctl -u postgresql | grep -iE "permission|denied|error|fatal" | tail -n 20
- Confirm healthcheck and post-reboot persistence (controlled system reboot):
# Verify if the service is active (should return 'active')
sudo systemctl is-active postgresql
# Check if the runtime directory was successfully recreated by systemd
ls -ld /var/run/postgresql
# Confirm the Unix socket file was created successfully
ls -la /var/run/postgresql/.s.PGSQL.5432*
# Test the local connection over the Unix socket
sudo -u postgres psql -c "SELECT version();"
# Verify if listening on the correct TCP port
ss -ltnp | grep 5432
Without post-reboot validation, the incident is not closed.
8) Infrastructure lessons learned#
- Internal service log > generic orchestrator message.
- Idempotency is a criterion for operational quality. If it only works after a manual command, it's still broken.
- Resilient infrastructure rebuilds itself. Temporary runtime directories must be part of the service design.
- Audit other volatile services: Scan if other active system daemons on the server share the same vulnerability of depending on
/var/runwithout explicit declarations in systemd:
# List services using the RuntimeDirectory directive
grep -r "RuntimeDirectory" /usr/lib/systemd/system/*.service 2>/dev/null | head -15
# Check for service units currently in a failed state
systemctl list-units --state=failed
Recovery checklist: PostgreSQL lock file error#
Use this operational checklist to resolve the issue and ensure stack reliability:
- [ ] Diagnosis: Executed systemd bypass and gathered error logs from
/var/log/postgresql/. - [ ] Port/Processes: Verified that port 5432 is free and filtered out concurrent or zombie processes.
- [ ] Resources: Checked free space and inodes on the
/run(tmpfs) mount point. - [ ] Backups: Made preventive backups of the original unit configuration files in
/root/systemd-backup-$(date +%Y%m%d). - [ ] Temporary Fix: Manually created the directory, changed ownership (
postgres:postgres), and adjusted permissions (775). - [ ] Permanent Fix: Created the systemd override (
RuntimeDirectory) and ran daemon-reload. - [ ] Post-Fix Logs: Audited systemd journalctl for permission or creation errors.
- [ ] Socket Security: Validated secure permissions (
0775) on the socket file and confirmed postgres user access. - [ ] System Scan: Checked for other systemd services vulnerable to reboots and volatile
/var/rundirectories. - [ ] Post-Reboot Validation: Restarted the VM or server to confirm automatic directory recreation and service health.
Production takeaways#
Resolving this error is not about "creating a folder." It is about aligning PostgreSQL with Linux's volatile runtime model via correct systemd orchestration.
When you close the root cause with RuntimeDirectory (or tmpfiles where appropriate), you stop fighting fires and start operating databases predictably in production.
Was this article helpful?
Leave a quick reaction to help prioritize future technical guides:
This post is licensed under CC BY-NC.



Comments
Join the discussion below.
0 comments