Killing cascading process trees in Linux with pkill without leaving orphaned children

Anyone working in the Linux terminal has encountered this scenario: you run ps aux | grep, locate a frozen service consuming RAM or locking network ports, attempt to kill it by process name, and nothing happens. Worse yet: killing the parent process leaves child workers running as orphaned background tasks.

I recently handled this while managing a background service (antigravity-server) that spawned a multi-tier hierarchy across sh wrappers and node sub-processes.

Here is why standard process killing fails, how the Linux kernel handles process metadata, and the clean workflow to terminate process trees safely.

1. Initial problem: inspecting the process hierarchy and the sl state#

Checking running processes on the system:

ps aux | grep anti

The output showed a three-tier execution chain via fork/exec:

root      619129  0.0  0.0   2584   840 ?        S    15:22   0:00 sh /root/.antigravity-server/bin/1.21.9-hash/bin/antigravity-server --start-server --host=127.0.0.1 --port=0 --connection-token-file /root/.antigravity-server/.hash.token
root      619139  0.3  1.3 11876396 162812 ?     Sl   15:22   0:16 /root/.antigravity-server/bin/1.21.9-hash/node /root/.antigravity-server/bin/1.21.9-hash/out/server-main.js --start-server --host=127.0.0.1 --port=0 --connection-token-file /root/.antigravity-server/.hash.token
root      619356  0.2  0.6 1621680 76072 ?       Sl   15:22   0:14 /root/.antigravity-server/bin/1.21.9-hash/node /root/.antigravity-server/bin/1.21.9-hash/out/bootstrap-fork --type=ptyHost --logsPath /root/.antigravity-server/data/logs/20260405T152231
root      666120  0.0  0.0   6416  1936 pts/0    S+   16:48   0:00 grep anti

Hierarchy breakdown:

1. PID 619129 (sh): The initial shell wrapper script.
2. PID 619139 (node ... server-main.js): The core master application process.
3. PID 619356 (node ... --type=ptyHost): A forked helper managing interactive pseudo-terminals (PTY).

Understanding the sl process state#

In the STAT column, Sl indicates:

• S (Interruptible Sleep): The process is awaiting external events (network I/O, timers, or kernel signals).
• l (Multi-threaded): The runtime manages multiple native worker threads for asynchronous polling and event handling.

2. The default pkill trap and kernel signal mechanics#

Running pkill antigravity fails silently without stopping any workers.

How signal delivery works under the hood#

1. Scanning /proc: pkill checks /proc/[pid]/comm and /proc/[pid]/cmdline.
2. The kill(2) Syscall: Dispatches the default SIGTERM (15) signal to matching PIDs.
3. Queueing in task_struct: The kernel validates user permissions and queues the signal into the process's pending signal set.
4. User-Space Handling: When threads are scheduled or return from syscalls, they process the signal.

However, /proc/[pid]/comm only retains the first 15 characters of the binary name (sh or node). The string antigravity-server exists only within runtime arguments. Therefore, -f (full command line matching) is mandatory.

3. Safe termination strategies#

Strategy 1: Pre-validation with pgrep and pkill#

1. Pre-validation:

   pgrep -a -f "antigravity-server"

1. Graceful shutdown (SIGTERM):

   pkill -15 -f "antigravity-server"

1. Forced termination (SIGKILL fallback):

   pkill -9 -f "antigravity-server"

Strategy 2: Terminating via process group ID (PGID)#

To eliminate parent and child workers in a single atomic syscall, dispatch the signal to the negative Process Group ID:

kill -TERM -619129

Strategy 3: Native lifecycle management via systemd Cgroups#

For persistent background services, manage processes through Systemd units:

[Service]
KillMode=control-group

With KillMode=control-group, systemctl stop signals all processes attached to that specific Cgroup, preventing orphaned subprocesses from escaping cleanup.

4. Kernel mechanics and edge cases#

• Orphaned processes (Init adoption): If you terminate only the wrapper (sh) without propagating signals, the child node processes continue running and get adopted by PID 1 (systemd/init). Using pkill -f or KillMode=control-group prevents this.
• Zombie processes (<defunct>): If a child exits but its parent fails to call waitpid(), it remains in the process table as a zombie. Terminate the parent so PID 1 adopts and reaps the entry.
• Residual lock files: Abruptly killing services can leave dangling lock files or token files on disk. Always verify filesystem state before restarting.

5. Post-execution verification and open descriptors#

After termination, ensure network sockets and open file descriptors are released:

# Check active network sockets
ss -tapen | grep -E '619139|619129'

# Inspect open file descriptors
ls -l /proc/619139/fd

# Inspect signal handling with strace in normal S/R states (if stuck in uninterruptible D-state, wait for disk I/O to complete)
strace -p 619139 -e trace=signal,exit_group,write

Conclusion: practical summary and recommended workflow#

To terminate multi-process command lines cleanly:

1. Verify target PIDs using pgrep -a -f "pattern".
2. Send a graceful shutdown with pkill -15 -f "pattern" or kill -TERM -<PGID>.
3. Apply pkill -9 -f "pattern" only if processes remain unresponsive.
4. For persistent services, encapsulate runtime lifecycles within Systemd units using KillMode=control-group.

See also

Linux Understanding tmpfs in Linux: does reducing temporary partitions really free up RAM?

Linux Why your server needs SWAP even with free disk space: kernel memory mechanics and 4GB setup

Linux Extracting 7z archives in Linux via CLI: from basics to low-level troubleshooting

This post is licensed under CC BY-NC.