Stabilizing a VPS with full SWAP, Wazuh, remote editors, and recurrent SSH disconnects
Back to blog

Stabilizing a VPS with full SWAP, Wazuh, remote editors, and recurrent SSH disconnects

6/7/2026 · 10 min · Infrastructure

Stabilizing a VPS with Full Swap, Wazuh, Remote Editors, and Recurrent SSH Disconnects#

Hello everyone. Today was a demanding day at the "infra-lab." I was deep in the flow, developing a project I'll refer to as "Evolya," when my remote SSH sessions started dropping out of seemingly nowhere. It wasn't just a simple case of the editor disconnecting - the entire VPS felt like it was taking a sudden nap every few minutes.

I was utilizing two simultaneous remote editors via SSH: Kiro and Codex. Approximately every 10 minutes, I'd get hit with that same sinking feeling: "Connection lost," "Session frozen," and an unresponsive terminal. It brought back that classic infrastructure question that separates junior admins from senior SREs: "Is this a network latency issue, a misconfigured SSH keepalive, or is the server actively dying from the inside?"

Explanatory Note: Kiro and Codex are code editors with AI-assisted capabilities that execute backend tasks on the remote server via the SSH protocol. They spawn Language Server Protocol (LSP) servers, system file watchers, and background indexing helpers that consume significant CPU and RAM resources.

Having spent years in support and infrastructure, these symptoms triggered a very specific alert profile in my mind. Recurrent SSH drops during heavy remote development workloads are rarely just indicative of "bad internet." Most often, the culprit is memory pressure, saturated swap partitions, excessive I/O wait times, Java processes aggressively consuming heap, or Node.js instances indexing the same project multiple times. It's the kernel desperately spending its last cycles trying to stay alive rather than responding to your terminal requests.

Faced with this, I stopped treating it as an "editor problem" and performed a full forensic autopsy on the VPS.

Operational symptoms: the forensic profile#

The recurring pattern was consistent and predictable:

The project in question was complex - thousands of files, hundreds of Node.js dependencies, and heavy language server activity. This matters because remote editors are far more than just "pretty windows" on your local machine. They spawn intensive backend processes on the server: indexers, file watchers, Language Server Protocol (LSP) instances, TypeScript Servers (tsserver), and various extension-specific caches. With two editors open, these costs weren't just combined; they were often duplicated.

Forensic diagnosis: checking the OOM killer#

When a web server or VPS experiences intermittent hangs and extreme slowness under memory stress, the first suspect is the Linux kernel's OOM (Out Of Memory) Killer. It steps in to terminate resource-heavy processes to prevent a total system panic.

To audit whether the OOM Killer was activated and which services were sacrificed:

# Query the kernel message buffer for processes terminated due to memory exhaustion
dmesg | grep -i "oom\|out of memory"

# Search the main system syslog file
grep -i "oom\|out of memory" /var/log/syslog

# Audit process kills via systemd journalctl logs
journalctl -k | grep -i "oom\|killed process"

# Check if there are orphaned processes actively in a killed/zombie state
ps aux | grep -i "killed"

Disk i/o performance audit (i/o bottlenecks)#

SSH session disconnects are frequently caused by disk I/O contention (disk thrashing) during intensive swap operations. To confirm whether the SSD/NVMe drive on the VPS is saturated:

# Monitor extended disk read/write statistics (refreshed every 1 second)
iostat -x 1 5

# Monitor which processes are generating the highest disk reads and writes in real time
iotop -o

# List processes sorted by detailed disk I/O consumption
sudo iotop -oP

# Track virtual memory paging rates (si/so - swap in / swap out)
vmstat 1 10

# Check for processes currently blocked waiting for disk I/O operations to complete
cat /proc/stat | grep procs_blocked

Phase 1: The initial autopsy (memory and SWAP)#

The VPS was provisioned with 12GB of RAM - historically more than enough for development environments and auxiliary services. However, a quick look at the memory state told a different story.

When I executed the standard diagnostic:

free -g

The output was a smoking gun:

          total        used        free      shared  buff/cache   available
Mem:             11           7           2           0           1           3
Swap:             1           1           0

The Critical Failure Point: A 1GB total swap space that was 100% utilized (0 free).

Keeping a mere 1GB of swap for a server that simultaneously hosts a Wazuh/OpenSearch stack alongside heavy Node.js development processes is dangerously tight. When physical RAM approaches its limit and the swap is already saturated, the Linux kernel enters a state of extreme distress known as Disk Thrashing.

Technical deep-dive: what happens when SWAP fills up?#

When Linux runs out of comfortable physical memory and has no remaining swap space to offload "cold" pages, it enters a spiral of performance degradation:

  1. Kernel Paging Agony: The kernel attempts to move memory pages between RAM and disk, but with zero free swap, it has nowhere to put them.
  2. I/O Backlog: Every process competing for RAM triggers expensive disk reads and writes as the system tries to shuffle data.
  3. IO Wait Spike: The CPU begins to spend a massive percentage of its cycles simply waiting for the SSD to complete I/O operations.
  4. Interactive Process Starvation: High-priority interactive processes, such as sshd, are starved of CPU time. If sshd cannot respond to a TCP keepalive packet or a client request within the timeout window, the connection is forcibly closed by the client.
  5. Soft Hang: To the user, the server appears "frozen" even though the kernel is still technically running - it's just far too busy managing its own survival to talk to you.

Below is the visual flow of the memory pressure degradation cascade:

flowchart TD A["RAM Full (Memory Pressure)"] --> B["Swap Active (Paging)"] B --> C{"Swap Available?"} C -- "No" --> D["OOM Killer Triggered (Terminates Processes)"] C -- "Yes" --> E["Intense Disk I/O (Disk Thrashing)"] E --> F["High CPU Wait (I/O Wait)"] F --> G["sshd Process Starved of CPU"] G --> H["SSH Session Frozen / Disconnected"] D --> H style A fill:#1e3a5f,stroke:#fff,stroke-width:2px,color:#fff style B fill:#78350f,stroke:#fff,stroke-width:2px,color:#fff style C fill:#78350f,stroke:#fff,stroke-width:2px,color:#fff style D fill:#7f1d1d,stroke:#fff,stroke-width:2px,color:#fff style E fill:#78350f,stroke:#fff,stroke-width:2px,color:#fff style F fill:#78350f,stroke:#fff,stroke-width:2px,color:#fff style G fill:#7f1d1d,stroke:#fff,stroke-width:2px,color:#fff style H fill:#7f1d1d,stroke:#fff,stroke-width:2px,color:#fff

Identifying the "council of villains"#

After confirming the memory exhaustion, I needed to identify the exact processes responsible for the pressure.

I relied on a sorted ps command to list the top 20 memory consumers:

ps aux --sort=-%mem | head -20

The culprits were predictable, but their combination was lethal.

The heavyweight: Wazuh indexer (java/opensearch)#

The Wazuh Indexer, which utilizes an OpenSearch backend, was consuming nearly 2GB of RAM on its own. While its Java heap was configured at a fixed 1G, the total footprint included JVM overhead, thread stacks, direct buffers, and memory-mapped files. In a development server, Wazuh isn't just competing with production services; it's competing with your development tools.

Tuning the Wazuh indexer JVM heap size#

On shared development servers or low-resource virtual private servers, restricting the heap limit for the Java Virtual Machine (JVM) is essential to keep other critical processes running.

  1. Open the JVM configuration options file for the indexer:
sudo nano /etc/wazuh-indexer/jvm.options
  1. Reduce the minimum (-Xms) and maximum (-Xmx) allocated memory limits (for example, setting 512MB for smaller systems):
-Xms512m
-Xmx512m
  1. Optionally, check general memory limits inside the indexer config file:
sudo nano /etc/wazuh-indexer/opensearch.yml
  1. Restart the indexer service to apply the configuration adjustments:
sudo systemctl restart wazuh-indexer
  1. Confirm the memory footprint of the indexer process after restarting:
ps aux | grep wazuh | grep -v grep

The duplicators: kiro and codex (Node.js & tsserver)#

Both editors use SSH to spawn backend processes. For a TypeScript project, this typically involves node, the tsserver (TypeScript Language Service), and dozens of file watchers. Because I had two editors open on the same project path, I had two separate clusters of these processes performing almost identical indexing work, effectively doubling the overhead.

The Fatal Mix: Wazuh Indexer + Kiro + Codex + Node/TSServer + Undersized Swap.

Imposing memory limits on Node.js processes#

Because Language Server (LSP) and TypeScript Server (TSServer) processes running on Node.js can expand indefinitely and consume all available RAM, it is recommended to set a hard heap ceiling for the Node runtime.

  1. Restrict the maximum space globally in user shell profile variables:
export NODE_OPTIONS="--max-old-space-size=2048" # Restrict to 2GB
  1. Or apply limits individually when starting scripts or background services:
node --max-old-space-size=1024 app.js # Restrict to 1GB
  1. For multi-tenant environments requiring strict limit boundaries, utilize Linux kernel control groups (cgroups):
# Create a dedicated memory control group
sudo cgcreate -g memory:/node-limit

# Set the group memory threshold limit to 2GB
sudo cgset -r memory.limit_in_bytes=2G node-limit

# Run the target process sandboxed under the control group
sudo cgexec -g memory:node-limit node app.js

Solution 1: Emergency SWAP expansion (horizontal scaling)#

The first priority was giving the kernel immediate room to breathe. I decided to increase the swap space to 8GB, providing a much larger buffer for memory pikes.

Safeguarding system configuration files#

Before editing storage configurations or critical system mounts, always create verified backups. A configuration error in the /etc/fstab file will prevent the operating system from booting, locking you out of the remote server.

# Backup the current fstab file with a timestamp
sudo cp /etc/fstab /etc/fstab.bak.$(date +%Y%m%d)

# Create a compressed tarball backup of all modified system config files
sudo tar czf /root/system-backup-$(date +%Y%m%d).tar.gz /etc/fstab /etc/sysctl.conf /etc/ssh/sshd_config

The Workflow:

  1. Disable Old Swap: sudo swapoff -a (This might take a moment as it moves data back to RAM).
  2. Allocate Large File: sudo fallocate -l 8G /swapfile_new (Faster than dd for modern filesystems).
  3. Strict Permissions: sudo chmod 600 /swapfile_new (Critical for security; swap can contain sensitive in-memory data).
  4. Format and Activate:
sudo mkswap /swapfile_new
sudo swapon /swapfile_new

Verification: Always validate with free -h and swapon --show to ensure the new file is being utilized. The goal here isn't to rely on swap for performance, but to prevent the "zero-free-memory" cliff that triggers freezes.

Persistence: I updated /etc/fstab to ensure this survives a reboot:

/swapfile_new none swap sw 0 0

Solution 2: Tuning kernel swappiness (behavioral shift)#

A large swap file is only half the solution. By default, Linux might still try to use it too early. I wanted the kernel to treat swap as a "last line of defense."

I tuned the swappiness value down to 10:

sudo sysctl vm.swappiness=10
echo 'vm.swappiness=10' >> /etc/sysctl.conf

Traditionally, a value of 60 is the default. Tuning it to 10 tells the kernel: "Only swap if you absolutely have to, prioritize keeping application memory in the physical RAM."

Avoid duplicate directives in sysctl.conf#

Operational tip: Before appending new configurations to /etc/sysctl.conf, check if the variable is already declared to prevent configuration conflicts:

grep -n '^vm.swappiness' /etc/sysctl.conf

If it exists, edit the line instead of appending. Alternatively, manage settings cleanly in dedicated configuration files like /etc/sysctl.d/99-vps-memory.conf:

vm.swappiness=10

Solution 3: SSH protocol hardening#

With the memory pressure addressed, I moved to the communication layer. I needed to ensure that the SSH sessions were more resilient to transient CPU spikes and limited concurrent unauthenticated connections.

Auditing SSH daemon logs#

To diagnose whether connections are dropping due to auth timeouts, TCP drops, or socket resets, monitor the SSH logs:

# View recent logs from the SSH daemon
sudo tail -100 /var/log/auth.log | grep sshd

# Search specifically for session timeout, disconnect, and socket reset entries
sudo grep -i "disconnect\|timeout\|reset" /var/log/auth.log | tail -20

# Search for errors or authentication failures
sudo grep -i "error\|fail" /var/log/auth.log | tail -20

# Check kernel buffer messages for network interface and TCP events
dmesg | grep -i "tcp\|network\|eth0"

Adjusting parameters in sshd_config#

I modified /etc/ssh/sshd_config with the following parameters:

TCPKeepAlive yes
ClientAliveInterval 60
ClientAliveCountMax 3
MaxSessions 50
MaxStartups 10:30:60

Safety Check: Always run sudo sshd -t before restarting the service to catch syntax errors that could lock you out. Apply changes:

sudo systemctl restart sshd

Managing orphaned processes and watchers#

When a remote session drops abruptly, the server often keeps "zombie" processes running. I used the following identifying pattern:

ps aux | grep -Ei 'vscode-server|node|tsserver|kiro|codex'

If necessary, I performed a surgical pkill -f to clean up old, hung instances of the editor backend.

Additionally, I addressed the inotify limit. Large projects can exhaust the number of files the system can watch for changes:

sudo sysctl -w fs.inotify.max_user_watches=524288
echo 'fs.inotify.max_user_watches=524288' >> /etc/sysctl.conf

System resource limits verification#

Check other system-wide resource limits to prevent lockups under heavy concurrent workloads:

# Check resource limits set for the current shell session (ulimits)
ulimit -a

# View the maximum allowed Process IDs (PIDs)
cat /proc/sys/kernel/pid_max

# View system-wide thread limits
cat /proc/sys/kernel/threads-max

# View the maximum number of open file descriptors
cat /proc/sys/fs/file-max

# Audit memory overcommit kernel policies
cat /proc/sys/vm/overcommit_memory
cat /proc/sys/vm/overcommit_ratio

Tuning global limits in systemd#

To prevent system services from hitting PID limits under peak workloads, adjust the global manager configurations:

# Edit global systemd limit file
sudo nano /etc/systemd/system.conf

# Add or uncomment these limit directives
# [Manager]
# DefaultLimitNOFILE=65536
# DefaultLimitNPROC=65536

# Reload the Systemd configuration manager without rebooting
sudo systemctl daemon-reexec

Post-reboot verification & continuous monitoring#

Verify that all memory optimizations and sysctl adjustments persist across server reboots.

Post-reboot audit#

# 1. Verify swap is active and check free RAM
swapon --show
free -h

# 2. Check that swappiness and inotify watches limits persisted
sysctl vm.swappiness
sysctl fs.inotify.max_user_watches

# 3. Verify that the running SSH configuration applies the new keepalive parameters
sshd -T | grep -i "keepalive\|session\|maxstartups"

# 4. Check fstab configuration syntax by performing a safe live remount
sudo mount -a # If this executes without errors, fstab syntax is valid

Continuous resource monitoring#

Set up diagnostic monitoring to trace memory, swap, and process activity over time:

# Track memory metrics in real time
watch -n 1 free -h

# Track active swap paging
watch -n 1 swapon --show

# View the top 10 memory-consuming processes dynamically
watch -n 1 "ps aux --sort=-%mem | head -10"

# Install standard performance tools
sudo apt install htop iotop sysstat -y

# Enable and start the sysstat logging service
sudo systemctl enable sysstat
sudo systemctl start sysstat

The final technical outcome#

After implementing these targeted adjustments, the VPS stabilization was instantaneous. High-load periods were absorbed by the new 8GB swap buffer, the swappiness tuning ensured the RAM stayed snappy, and the SSH hardening prevented the "ghost disconnects" that were previously ruining my productivity.

SRE VPS stabilization checklist#

Diagnosis#

Correction - memory#

Correction - kernel#

Correction - SSH#

Post-correction#

The Core Lesson: In infrastructure management, never treat a disconnection as the root cause. Treat it as a symptom of a deeper resource bottleneck. By combining forensic memory analysis with kernel tuning and protocol hardening, I transformed a struggling VPS into a robust, high-performance development environment. Tuning the kernel and storage to match your actual workload is not just a "nice-to-have" - it's the foundation of a stable professional workflow.

Was this article helpful?

Leave a quick reaction to help prioritize future technical guides:

See also

Linux Why your server needs SWAP even with free disk space: kernel memory mechanics and 4GB setup
DevOps When VS code remote and antigravity stalled my server: anatomy of memory thrashing
Infrastructure Anatomy of a silent deadlock in CloudLinux and cPanel: phantom processes, session locks, and TCP queues
CC BY-NC

This post is licensed under CC BY-NC.

Comments

Join the discussion below.

0 comments