A note about how/why this was created.

kube-autopsy

kube-autopsy is a low-overhead Kubernetes diagnostic tool designed to capture the exact system state immediately preceding a pod’s termination (like OOMKilled events). By leveraging native eBPF tracing, it intercepts the Linux Out-Of-Memory (OOM) killer to securely extract high-resolution memory contexts and last-gasp logs before the container runtime destroys the pod’s filesystem and cgroup.

Traditional OOM diagnostic tools rely on polling the memory.events file in cgroups. While this detects an OOM occurred, it cannot reveal which process triggered the OOM, nor can it provide a breakdown of memory.

kube-autopsy compiles portable eBPF bytecode that directly reads the kernel’s mm_struct. When a pod crashes, the agent is instantly notified and streams the precise memory breakdown, the exact triggering PID, the victim PID, and the kernel OOM scores.

The application is deployed as a two-part architecture:

1. Node Agent (DaemonSet): Runs on every node, attaching a kprobe to the kernel’s oom_kill_process.
2. Controller (Deployment): A central operator that manages report lifecycle, garbage collection, and webhook dispatching.

• Automatically generates a PodCrashReport CRD capturing:
  • Trigger Process (name and PID that allocated the breaching memory)
  • Victim Process (process chosen by the OOM killer)
  • Kernel OOM Scores
  • Peak Memory Consumption
  • OOM Context (ContainerLimit vs NodeExhaustion)
  • RSS Memory Dissection (Anon RSS, File RSS, Page Tables)
  • Final 50 Log Lines before termination
• Support for detailed crash payload webhooks directly to Slack.
• k9s plugin for TUI-driven debugging.
• Crash reports bind to their owner Pod and are gracefully garbage collected by the Kubernetes control plane or the controller’s active background GC.

Links

• GitHub Repository
• Documentation