How do I see the real registry error?

Run kubectl describe pod and read the Events section. The Failed event contains the message the registry returned, such as manifest unknown for a missing tag or unauthorized for an authentication failure.

Why does ImagePullBackOff happen only on some nodes?

Nodes that already have the image cached will run the pod while nodes without it fail to pull. This commonly hides a broken imagePullSecret or a node level registry credential that exists on only part of the cluster.

Does imagePullSecret fix every ImagePullBackOff?

No. A pull secret only fixes authentication failures. A missing tag, an unresolvable registry hostname, a rate limit or an architecture mismatch are unaffected by credentials, and adding a secret for those causes wastes time.

← Documentation home

Kubernetes troubleshooting

ImagePullBackOff: five causes that look identical until you read the registry error.

Q: What is the difference between ErrImagePull and ImagePullBackOff?

ErrImagePull is the first failed attempt to pull the image. ImagePullBackOff is the waiting period after repeated failures. They describe the same underlying problem at different stages, and the registry error message in the pod events is what identifies the cause.

Almost every guide answers ImagePullBackOff with “check your imagePullSecret”. That fixes exactly one of five causes. The registry tells you which one you have, in a message most people never open.

ErrImagePull and ImagePullBackOff are the same problem

ErrImagePull is the first failed pull. ImagePullBackOff is the waiting period after repeated failures, with the same exponential backoff the kubelet uses elsewhere. Seeing one turn into the other tells you nothing new — the cause is identical, and it is in the events.

Neither status is itself a cause. Both mean Kubernetes could not obtain the image, and the reason lives in the message the registry returned.

Read the registry error first

kubectl describe pod <pod> -n <namespace>

# or go straight to the events for this pod
kubectl get events -n <namespace> \
  --field-selector involvedObject.name=<pod> \
  --sort-by=.lastTimestamp

The line that matters is the Failed event:

Events:
  Type     Reason   Age                From     Message
  ----     ------   ----               ----     -------
  Warning  Failed   2m (x4 over 3m)    kubelet  Failed to pull image
    "nginx:this-tag-does-not-exist-12345": rpc error: code = NotFound
    desc = failed to pull and unpack image: no match for platform in
    manifest: not found
  Warning  Failed   2m (x4 over 3m)    kubelet  Error: ErrImagePull
  Normal   BackOff  30s (x6 over 3m)   kubelet  Back-off pulling image

Everything below is a translation table for that message. Confirm what the manifest is actually asking for before you change anything:

kubectl get pod <pod> -n <namespace> \
  -o jsonpath="{range .spec.containers[*]}{.name}{': '}{.image}{'\n'}{end}"

Five causes, five different fixes

1Tag or image does not existmanifest unknown, manifest for ... not found, not found: manifest. Credentials are irrelevant.

2Authentication or permissionunauthorized, pull access denied, 403 forbidden. This is the imagePullSecret case.

3Registry unreachableno such host, could not resolve, i/o timeout. DNS, egress policy or firewall.

4Rate limitedtoomanyrequests, You have reached your pull rate limit.

5Architecture mismatchno matching manifest for linux/arm64. The image exists but not for this node's platform.

These are genuinely different failures. Adding a pull secret to a missing-tag problem changes nothing, and it is the single most common wasted step in this whole diagnosis.

Cause 1: the tag does not exist

The most common cause in practice, and the one most often misdiagnosed as an auth problem. A typo in the tag, a tag that was never pushed, or a CI pipeline that failed before publishing while the deployment already moved on.

# does the tag exist at all?
docker manifest inspect <registry>/<image>:<tag>

# or list what is actually published
crane ls <registry>/<image>
skopeo list-tags docker://<registry>/<image>

The fix is the image reference itself, at spec.template.spec.containers[].image on the controller. Pinning by digest rather than a moving tag removes this class of failure entirely:

image: myregistry.io/api@sha256:9f2c...   # immutable
# instead of
image: myregistry.io/api:latest           # moves under you

Cause 2: authentication or pull permission

Messages containing unauthorized, pull access denied or 403 forbidden. Only now is a pull secret the answer.

kubectl create secret docker-registry regcred \
  --docker-server=<registry> \
  --docker-username=<user> \
  --docker-password=<token> \
  -n <namespace>

Then reference it in the pod spec — a secret that exists but is not referenced does nothing:

spec:
  template:
    spec:
      imagePullSecrets:
        - name: regcred
      containers:
        - name: app
          image: myregistry.io/api:1.4.2

Three things that catch people out. Pull secrets are namespaced — a secret in default does nothing for a pod in production. The --docker-server value must match the registry host in the image reference exactly, including the port. And registry tokens expire; a deployment that worked last month and fails today with no manifest change is usually an expired credential.

Cause 3: the registry cannot be reached

Messages containing no such host, could not resolve, server misbehaving or i/o timeout. Nothing is wrong with the image or your credentials — the node cannot get to the registry.

# can a pod on that node resolve the registry?
kubectl run netcheck --rm -it --restart=Never --image=busybox \
  -- nslookup <registry-host>

# is an egress NetworkPolicy blocking it?
kubectl get networkpolicy -n <namespace>

Common in private clusters with restricted egress, behind a proxy that the kubelet is not configured for, or when a NetworkPolicy denies egress by default and nobody allowed the registry.

Cause 4: registry rate limits

Messages containing toomanyrequests or You have reached your pull rate limit. Anonymous pulls from public registries are capped per IP address, which means a cluster behind one NAT gateway shares a single quota across every node.

The symptom is intermittent and correlates with deployment activity: pulls succeed in the morning and fail during a busy afternoon rollout. Fixes, in order of durability: authenticate even for public images, which raises the limit substantially; mirror the images you depend on into your own registry; and set imagePullPolicy: IfNotPresent so cached images are not re-pulled unnecessarily.

Cause 5: architecture mismatch

Messages containing no matching manifest for linux/arm64 or no match for platform in manifest. The image exists and you can pull it, just not onto this node.

Increasingly common with mixed clusters that include ARM nodes such as AWS Graviton, or an image built on an Apple Silicon laptop and pushed as arm64 only.

# which platforms does the image actually publish?
docker manifest inspect <image>:<tag> | grep architecture

# what architecture are the nodes?
kubectl get nodes -o wide \
  -o custom-columns="NAME:.metadata.name,ARCH:.status.nodeInfo.architecture"

The fix is a multi-architecture build (docker buildx build --platform linux/amd64,linux/arm64), or a nodeSelector that keeps the workload on nodes the image supports.

When it fails on some nodes and not others

A deployment where some replicas run and others sit in ImagePullBackOff is almost always a caching artefact. Nodes that already hold the image run happily; nodes that must pull it fail. The image looks fine because it is fine — on the nodes that already have it.

kubectl get pods -n <namespace> -o wide   # which nodes fail?

This pattern hides three things: a pull secret that was only ever applied to part of the cluster, node-level registry credentials configured by hand on older nodes, and architecture mismatch in a mixed node pool. It is also why imagePullPolicy: Always surfaces a problem that IfNotPresent was quietly masking.

Frequently asked

What is the difference between ErrImagePull and ImagePullBackOff? The first is the failed attempt, the second is the backoff after repeated failures. Same cause.

Why does adding an imagePullSecret not help? Because credentials only fix authentication. If the registry said manifest unknown, the tag does not exist and no secret will conjure it.

The image works with docker pull on my laptop. Your laptop has credentials in ~/.docker/config.json, a different network path, and probably a different CPU architecture. None of those apply to the node.

Can a private registry work without a pull secret? Yes, if the nodes carry credentials themselves — common on managed clusters pulling from the same cloud's registry. That is also why it can work on old nodes and fail on new ones.

Is this the same as CrashLoopBackOff? No. ImagePullBackOff happens before the container ever runs. See the CrashLoopBackOff guide for the case where the image ran and exited.

Doing this automatically

The diagnosis above is a matter of reading the registry's message and mapping it to one of five causes. It is mechanical, and it is also the step most often skipped in favour of adding a pull secret and hoping.

KrevoPilot reads that message and names the cause — a missing tag rather than an authentication failure, a rate limit rather than DNS — along with the image reference that failed and the manifest field to change. When the registry does not report a specific reason, it says so rather than picking a plausible one. The in-cluster agent is read-only and cannot change your workloads.

Apply for a trial CrashLoopBackOff guide Troubleshooting guide