In the day-to-day operation of Linux servers, the most disruptive failures often stem not from hacker attacks or code crashes, but from the "OOM Killer" (Out-of-Memory Killer)—a mechanism triggered when system memory becomes critically scarce.
When both physical memory and swap space are exhausted and the kernel’s "memory reclamation" mechanisms fail, the OOM Killer acts as a ruthless arbiter. It calculates an "OOM Score" for every running process and sends a `SIGKILL` signal to the one with the highest score, forcibly terminating it. What makes this particularly frustrating is that the mechanism is entirely indifferent to business logic: it might spare a "zombie" process consuming 8GB of RAM, while ruthlessly killing a MySQL or Nginx instance that uses only 200MB but handles critical business data.
Protecting key applications during this "memory battle royale" doesn't require complex scripts. The following five commands provide a robust defense, ranging from "temporary exemptions" to "hard isolation."
Command 1: Granting an immediate exemption to a high-risk process
echo -500 > /proc/$(pidof -s <process_name>)/oom_score_adj
This is the most direct and effective "get-out-of-jail-free" card. The Linux kernel maintains an adjustment value for each process, located at `/proc/<PID>/oom_score_adj`, which ranges from -1000 to 1000. A lower value makes a process less likely to be killed, while a higher value makes it a prime target.
Setting this value to -500 for a critical process ensures that, even when system memory is exhausted, the kernel assigns it a much lower "risk weight"—prioritizing the termination of other processes with higher scores instead. For processes that absolutely must not be interrupted (such as an SSH daemon), you can set this value to -1000, rendering the process completely immune to the OOM Killer.
Note: This is a temporary measure. If the process restarts, the value will reset to 0. Furthermore, it is not advisable to set a large number of processes to -1000 simultaneously; doing so may prevent the OOM Killer from freeing up sufficient memory, potentially causing the entire system to lock up or hang completely.
Command 2: Configuring Permanent Immunity for systemd Services
Temporary exemptions are only suitable for emergency situations. To ensure that protection persists after a reboot—specifically for applications managed by systemd (such as Nginx, MySQL, or Redis)—you must embed the configuration into the service unit.
sudo systemctl edit --full <service_name>.service
In the editor that opens, add the following line under the `[Service]` section:
ini
OOMScoreAdjust=-900
After saving and exiting, run `sudo systemctl daemon-reload && sudo systemctl restart <service_name>` to apply the changes. This effectively incorporates the effect of the first command into the service's startup configuration, ensuring it is applied automatically every time the service starts. For the vast majority of mission-critical services, -900 is a safer choice than -1000; it provides an extremely high survival priority without completely depriving the system of its "pressure relief valve" in extreme scenarios.
Command 3: Setting an Upper Limit on "Misbehaving" Processes with `MemoryMax`
The first two commands focus on protecting potential "victims." However, the root cause of an OOM (Out-of-Memory) event is often a single process consuming excessive resources. Instead of hoping a process won't be killed when memory runs out, it is better to establish a firm, non-negotiable limit for processes prone to spiraling out of control.
sudo systemctl edit --full <service_name>.service
Set a hard limit within the `[Service]` section:
ini
MemoryMax=2G
The function of `MemoryMax` is as follows: when a service's memory usage reaches 2GB, the cgroup-level OOM handler will specifically terminate that service, rather than allowing the system-wide OOM Killer to select a "victim" at random. This acts as a "targeted elimination" strategy—the process that exceeds its limit is removed, thereby protecting other processes and the system as a whole from being affected.
Command 4: Implementing Flexible Throttling with `MemoryHigh`
Relying solely on `MemoryMax` to terminate a process can sometimes be too blunt an approach. For services that can tolerate brief slowdowns but cannot afford to be interrupted, `MemoryHigh` provides a smoother buffer.
ini
MemoryHigh=1500M
MemoryMax=2G
`MemoryHigh` acts as a "soft limit." When a service’s memory usage exceeds 1.5 GB—but has not yet reached the 2 GB threshold—the kernel will enforce memory reclamation and throttling. This significantly slows down the service, compelling it to free up memory without terminating it. Only if throttling proves ineffective and memory usage continues to climb toward `MemoryMax` will the service be terminated.
This "warn-before-execute" mechanism offers the service a chance to self-adjust while maintaining a final, hard safeguard. It is a recommended practice in production environments for protecting stateful services, such as databases.
Command 5: Enabling "Group-Level Co-termination" to Prevent "Split-Brain" Scenarios
For applications that rely on multi-process collaboration—such as services composed of a master process and multiple worker processes—a major risk is that the OOM (Out-of-Memory) Killer might terminate only one critical child process. This leaves the entire service in a "zombie-like" or "split-brain" state, from which it cannot automatically recover.
echo 1>/sys/fs/cgroup/your-cgroup-path/memory.oom.group
In cgroup v2, `memory.oom.group` governs the behavior of the OOM Killer. When set to 1, the OOM Killer treats the entire cgroup as an indivisible unit: if it decides to trigger a termination, it kills all processes within that group simultaneously, rather than targeting just one.
While this may seem "ruthless," it is actually designed to preserve the consistency of stateful services (such as the master and worker processes of PostgreSQL). If the entire group is terminated, systemd’s `Restart=on-failure` policy can perform a clean, complete restart of the service. Conversely, killing only a single process could leave behind a non-functional "zombie" master process, necessitating manual intervention. Beyond Commands: Securing "Memory Safety" at the Infrastructure Level
While the five commands discussed previously establish a defense-in-depth strategy—ranging from individual processes to cgroups—they only "manage" the consequences of memory pressure at the software layer; they cannot fundamentally "create" more memory. If your VPS suffers from tight memory capacity, or if the host machine is heavily oversubscribed—leading to contention for physical RAM—the OOM (Out-of-Memory) Killer will be triggered far more frequently. In such scenarios, the aforementioned tuning measures serve only as a stopgap.
Jtti’s cloud server solutions provide a more predictable operating environment for memory-sensitive applications at the infrastructure level. Dedicated bandwidth—standard across all plans—ensures that memory swapping and communication with external dependencies are not exacerbated by "noisy neighbors" competing for bandwidth, thereby preventing application-side memory accumulation caused by network congestion. Enterprise-grade NVMe SSDs offer read/write speeds for swap partitions that far exceed those of standard SATA drives; even in extreme cases where swapping occurs, the process completes much faster, minimizing the duration of system "freezes." Furthermore, nodes in Hong Kong and the U.S. are connected via the premium CN2 GIA network, ensuring stable, packet-loss-free SSH connections for monitoring and intervention—guaranteeing that you never miss the critical window for action due to timeouts in your management channel.
The core logic for protecting vital applications from the OOM Killer is to "ensure the kernel knows—before it takes action—which processes are off-limits, which are fair game, and to what extent they can be affected." You can define an exemption list using `oom_score_adj`, establish behavioral boundaries with `MemoryHigh` and `MemoryMax`, and ensure atomic recovery during failures using `memory.oom.group`. When combined with a server that offers guaranteed memory resources and high-performance underlying I/O, this defensive framework can be truly and effectively implemented.