Toolbox · Containers
Container Debugging
Get inside containers that have no shell, inspect namespaces, cgroups and capabilities, and trace what a process is doing.
On this page
Get a shell (or not)
| Command | Purpose |
|---|---|
docker exec -it <ctr> sh | If the image has a shell |
docker run --rm -it --pid=container:<ctr> --net=container:<ctr> nicolaka/netshoot | Debug sidecar sharing PID and network namespaces |
kubectl debug -it <pod> --image=busybox:1.36 --target=<container> | Ephemeral container in the pod |
kubectl debug node/<node> -it --image=ubuntu | Caution host-namespace debug pod with the node filesystem mounted at /host |
nsenter -t <pid> -n ss -tulpn | Run a host tool inside a container’s namespace |
Find the process on the host
docker inspect --format '{{.State.Pid}}' <ctr>
crictl ps; crictl inspect <id> | jq .info.pid
ls -l /proc/<pid>/ns/ # namespaces
cat /proc/<pid>/status | grep -E 'Cap(Prm|Eff)' # capabilities (decode with capsh --decode=)
cat /proc/<pid>/cgroupResource limits
| Command | Purpose |
|---|---|
docker stats --no-stream <ctr> | Live usage |
cat /sys/fs/cgroup/memory.max (inside container) | Effective memory limit (cgroup v2) |
cat /sys/fs/cgroup/memory.events | oom_kill counter |
kubectl describe pod <p> | grep -A2 'Last State' | OOMKilled / exit codes |
Filesystem and image
| Command | Purpose |
|---|---|
docker diff <ctr> | Changed files |
docker export <ctr> | tar -tvf - | head | List container FS |
docker save img:tag | tar -xf - -C /tmp/img | Unpack image layers |
dive img:tag | Layer explorer (external tool) |
docker image inspect img:tag --format '{{json .Config}}' | jq | Entrypoint, env, user |
Tracing
| Command | Purpose |
|---|---|
strace -f -p <pid> -e trace=openat,connect | File and network syscalls |
ltrace -p <pid> | Library calls |
tcpdump -ni any -c 20 (in netshoot) | Packets |
Exit codes
| Code | Meaning |
|---|---|
0 | Success |
1 | Application error |
125 | Docker daemon error |
126 | Command not executable |
127 | Command not found |
137 | SIGKILL (often OOM) |
139 | SIGSEGV |
143 | SIGTERM |