OOMKilled and exit code 137: find the real cause before you raise the limit.
A container terminated with OOMKilled and exit code 137 was killed by the kernel for exceeding a memory limit — usually the container's own, not the node's. Raising the number blindly hides a leak. Leaving it alone leaves the pod in a restart loop. This guide separates the cases.
Confirm it was actually an OOM kill
A crash-looping pod is not automatically an out-of-memory problem. Check the container's last state before assuming anything.
kubectl describe pod <pod> -n <namespace>
# or read it straight out of the status
kubectl get pod <pod> -n <namespace> \
-o jsonpath="{.status.containerStatuses[*].lastState.terminated.reason}"In kubectl describe the decisive block is Last State:
Last State: Terminated
Reason: OOMKilled
Exit Code: 137
Started: Sat, 06 Sep 2026 09:14:22 +0200
Finished: Sat, 06 Sep 2026 09:14:26 +0200
Restart Count: 6992Three things matter. Reason: OOMKilled confirms the kernel killed it. Exit Code: 137 confirms SIGKILL. And the gap between Started and Finished tells you whether the container died on startup, which points at an allocation made immediately, or after hours, which points at a leak.
If the reason is Error or Completed instead, this is not an OOM kill and the memory limit is not your problem.
Why exit code 137 specifically
A process terminated by a signal reports 128 + signal number. SIGKILL is signal 9, so 128 + 9 = 137. The kernel out-of-memory killer uses SIGKILL because it cannot be caught or ignored, which is why your application never gets to log anything on the way down.
That is also why application logs are usually unhelpful for this failure. The last line is whatever the process happened to be doing, not an error. Do not spend time hunting for an exception that does not exist.
The neighbouring code is 143 (128 + 15, SIGTERM), which is a graceful shutdown rather than an OOM kill. If you see 143, look at pod eviction, a rolling update, or a failing liveness probe instead.
Container limit or node memory pressure
These two failures look similar on a dashboard and have opposite fixes.
Check which one you are looking at:
# Node conditions - is MemoryPressure True anywhere?
kubectl describe nodes | grep -A2 MemoryPressure
# Any evicted pods in the namespace?
kubectl get pods -n <namespace> --field-selector status.phase=Failed
# What the container was actually allowed
kubectl get pod <pod> -n <namespace> \
-o jsonpath="{range .spec.containers[*]}{.name}{'\t'}{.resources.limits.memory}{'\n'}{end}"If MemoryPressure is False on every node and you still have exit 137, the container limit is the constraint. Fix the workload, not the cluster.
Five causes, in the order you should rule them out
1. The limit is simply too low. The most common case and the easiest to confirm: steady-state usage sits close to the limit from the moment the container starts. Compare observed usage against the configured limit before changing anything.
kubectl top pod <pod> -n <namespace> --containers
2. A memory leak. Usage climbs steadily over hours or days and the container survives progressively shorter periods between restarts. Raising the limit buys time and does not fix it. The tell is the interval between kills shrinking as traffic accumulates.
3. A workload that allocates more than anyone configured. Check the container's actual command before touching the manifest. A process invoked with an explicit allocation size will breach any limit below it, however carefully the limit is tuned.
kubectl get pod <pod> -n <namespace> \
-o jsonpath="{range .spec.containers[*]}{.name}{': '}{.command}{' '}{.args}{'\n'}{end}"4. A sidecar sharing the pod budget. Memory limits are per container, but the pod is scheduled on the sum. A logging or service-mesh sidecar that grows under load can push the pod past what the node expected even when your application container is well behaved. Check every container in the pod, not only the one you wrote.
5. A burst that only happens under real traffic. Batch import, report generation, a large upload, a cache warm on deploy. Steady state looks comfortable and the peak is three times higher. This is the case that makes teams raise limits repeatedly without ever finding the ceiling. Measure the peak, not the average.
Runtimes that do not see the container limit
A JVM, Node.js or .NET process that is not container-aware sizes its heap from the node's total memory rather than the cgroup limit. The runtime grows to a heap larger than the container is allowed and the kernel kills it. The application never reports an OutOfMemoryError because it never reached its own limit.
Modern JVMs respect cgroup limits, but only with container support active:
# JVM - honour the cgroup limit and cap heap as a share of it -XX:+UseContainerSupport -XX:MaxRAMPercentage=75.0 # Node.js - old-space must sit below the container limit --max-old-space-size=1536 # for a 2Gi container
Leave real headroom between heap and limit. The heap is not the whole process: thread stacks, metaspace, direct byte buffers and the allocator all live outside it. A heap sized to the container limit will be OOMKilled every time.
Choosing a limit from evidence instead of guessing
Doubling the limit until the restarts stop works, and it wastes budget on every replica for as long as the workload runs. The defensible method:
spec:
template:
spec:
containers:
- name: app
resources:
requests:
memory: "256Mi" # steady state - drives scheduling
limits:
memory: "512Mi" # observed peak + headroomThe field to change is spec.template.spec.containers[].resources.limits.memory on the Deployment, StatefulSet or DaemonSet — not on the pod. Editing a pod directly is overwritten by the controller on the next reconcile.
How requests and limits set your eviction risk
The relationship between request and limit decides which pods die first when a node runs short.
If a workload keeps being evicted rather than OOMKilled, raising its request to match its limit moves it to Guaranteed and takes it out of the firing line.
What not to do
Do not remove the memory limit. It is the fastest way to stop the restarts and it converts a contained failure into a node-wide one. An unlimited container can consume node memory until the kubelet evicts unrelated pods, and now you are debugging three services instead of one.
Do not raise the limit past the node's allocatable memory. The pod becomes unschedulable and moves from CrashLoopBackOff to Pending, which looks like a different bug entirely.
Do not raise the request at the same time without thinking. Requests reserve capacity cluster-wide. Doubling requests across a large deployment can exhaust a node pool while actual usage barely moves.
Do not treat one kill as a trend. A single OOMKill during an unusual event is not the same as a pod at 6,992 restarts. Check the restart count and the interval between kills before rewriting a manifest.
Verify the fix
After rolling out a new limit, confirm the restart count stops climbing rather than assuming it did.
kubectl rollout status deployment/<name> -n <namespace> # restart count should stay flat kubectl get pods -n <namespace> -w # and peak usage should sit below the new limit kubectl top pod -n <namespace> --containers
Watch through a full traffic cycle before calling it fixed. A limit that survives a quiet night can still fail at the next batch run.
Frequently asked
What does exit code 137 mean? The process was terminated by SIGKILL (128 + 9). In Kubernetes that is nearly always the kernel OOM killer acting on a container that exceeded its memory limit.
Does OOMKilled mean the node ran out of memory? Usually not. A container is OOMKilled for breaching its own limit while the node still has free memory. Node pressure produces Evicted pods instead.
Why are there no useful application logs? SIGKILL cannot be caught. The process gets no opportunity to flush a log or run a shutdown hook.
Can a pod be OOMKilled with no memory limit set? Yes, through node pressure rather than its own cgroup — and it may take other pods down with it, which is why unlimited containers are worse, not safer.
Why did it work in staging? Staging usually has smaller data, fewer concurrent requests and a colder cache. Peak memory, not average, is what breaches a limit.
Doing this automatically
Everything above is the manual path: read the last state, separate container limit from node pressure, check the container's command, compare observed peak against the configured limit, then choose a new value with headroom.
KrevoPilot runs that pass and returns the cause with its evidence attached — the container's actual command, the limit it breached, the exit code, the restart count, and the exact manifest field to change. It also reports observed peak against configured request across the fleet, so limits come from measurement rather than from doubling until the alerts stop. The in-cluster agent is read-only and cannot change your workloads.
Apply for a trial Troubleshooting guide Agent RBAC reference
