The Complete Guide to eBPF for Security Monitoring

eBPF is the in-kernel virtual machine that lets you attach sandboxed programs to tracepoints, kprobes, uprobes, and network events without loading a kernel module. It sits next to kprobes, tracepoints, perf events, and the BPF verifier, and it matters to this audience because it is the only widely deployed mechanism for observing kernel and userspace behavior at a granularity fine enough to catch the kind of attacks that hardware mitigations are supposed to stop. If you are reverse-engineering a baseband, auditing a UEFI runtime, or trying to understand why a CET shadow stack check failed, eBPF is often the least bad tool you have.

Vendors will tell you eBPF is a safe, production-ready observability layer. The verifier is not a proof assistant. The JIT is not a sandbox. The maps are not a database. Treat every one of those claims as a hypothesis to be falsified, not a design guarantee.

What eBPF Actually Is

eBPF is a register-based virtual machine inside the Linux kernel. Programs are compiled from a restricted C subset to eBPF bytecode, then passed through the verifier, which attempts to prove that the program terminates, does not access out-of-bounds memory, and does not leak kernel pointers to userspace. If the verifier accepts the program, it is either interpreted or JIT-compiled to native x86-64 or ARM64 instructions.

The execution model is event-driven. You attach a program to a hook, and the kernel calls it when that hook fires. Hooks include:

• Tracepoints — stable, low-overhead markers in kernel subsystems.
• kprobes/kretprobes — dynamic instrumentation of almost any kernel function, including ones not exported to modules.
• uprobes/uretprobes — dynamic instrumentation of userspace functions, which is how you watch a baseband daemon or a proprietary userspace helper without modifying its binary.
• LSM hooks — security policy enforcement points, used by BPF LSM programs.
• Network hooks — XDP, TC, cgroup/skb, and socket filters.

For security monitoring, the interesting hooks are the ones that let you observe syscall arguments, memory mappings, page faults, context switches, and network flows without ptrace, without LD_PRELOAD, and without a kernel module.

Why eBPF Beats the Old Tools

The traditional options for security monitoring on Linux are auditd, fanotify, ptrace, and kernel modules. Each has a failure mode that eBPF was designed to avoid.

auditd gives you syscall logging, but the format is verbose, the overhead is real, and the filtering is coarse. You can miss the syscall you care about because the audit rule language does not let you express the condition you actually need.

fanotify is for file access, not for syscall arguments or kernel state. It is useful for watching a firmware update file get written, but it will not tell you which instruction in a proprietary userspace daemon triggered the write.

ptrace is slow, intrusive, and trivially detected by any malware that checks TracerPid or uses PTRACE_TRACEME as an anti-debugging trick. It also changes process semantics in ways that break multi-threaded programs.

Kernel modules give you full access, but they are a stability and security liability. A bug in your module is a bug in the kernel. eBPF programs are verified, sandboxed, and can be updated without a reboot.

None of this means eBPF is free. The verifier rejects valid programs. The JIT has its own bugs. The maps have concurrency semantics that will bite you. But compared to the alternatives, eBPF is the only tool that gives you kernel-level visibility without kernel-level risk.

The Verifier Is Not Your Friend

The eBPF verifier is a static analyzer that tries to prove safety properties about your program. It is not a general-purpose theorem prover, and it is not a security boundary in the way a hypervisor or a hardware enclave is. It is a heuristic filter that rejects some unsafe programs and accepts some safe ones.

In practice, the verifier is the main reason eBPF development is painful. It rejects loops unless they are bounded. It rejects pointer arithmetic unless it can prove the result is in bounds. It rejects programs that are too large, too complex, or too clever. The error messages are often unhelpful, and the fix is usually to restructure your code into a shape the verifier can understand.

For security monitoring, this means you will spend a lot of time fighting the verifier to do things that are trivial in a kernel module. You want to read a string from a syscall argument? You need to copy it into a map first, because the verifier will not let you dereference a userspace pointer directly. You want to iterate over a linked list? You need to bound the loop and hope the verifier can see the bound.

The verifier is also a moving target. New kernel versions add new capabilities and new restrictions. A program that verifies on 5.15 may not verify on 6.1. If you are building a security monitoring tool, you need to test against every kernel you support, and you need to be prepared for the verifier to reject your program for reasons that have nothing to do with safety.

eBPF for Security Monitoring: What You Can Actually Do

Here is what eBPF gives you in practice, with concrete examples that matter for low-level security work.

Syscall Monitoring Without auditd

Attach a program to the raw_syscalls:sys_enter tracepoint and you can log every syscall with its arguments, filtered by process, cgroup, or user. This is the foundation of most eBPF security tools. The overhead is low enough to run in production, and the data is rich enough to catch things like:

• A process calling ptrace on a process it should not be able to touch.
• A process calling memfd_create and then executing from the resulting file descriptor, a classic fileless malware technique.
• A process calling bpf to load its own eBPF programs, which is a red flag if you did not expect it.

The catch is that syscall arguments are raw register values. You need to know the syscall ABI for your architecture, and you need to handle the fact that some arguments are pointers to structures you cannot dereference without copying them into a map first.

File Integrity Monitoring at the VFS Layer

Attach to vfs_write, vfs_read, or the fsnotify hooks and you can watch every file access on the system, including the ones that bypass userspace file watchers. This is how you catch a rootkit that writes to /etc/ld.so.preload or a firmware update tool that writes to a device node it should not touch.

The advantage over fanotify is that you see the kernel-side call, not the userspace wrapper. The disadvantage is that you are now in the business of interpreting VFS data structures, which are not stable across kernel versions.

Network Monitoring at XDP

XDP lets you run eBPF programs at the earliest point in the network stack, before the packet is even allocated an skb. This is the fastest way to drop, redirect, or log packets, and it is the basis for most eBPF-based DDoS mitigation tools.

For security monitoring, XDP is useful for catching port scans, SYN floods, and other network-level attacks before they reach userspace. The limitation is that XDP programs are restricted to a small set of helper functions and cannot access arbitrary kernel state. If you need to correlate a packet with a process, you need to do it in a TC or socket filter program instead.

Watching Kernel Memory Allocation

Attach to kmalloc, kfree, or the slab allocator tracepoints and you can watch kernel memory allocation in real time. This is how you catch a kernel module that is leaking memory, or a driver that is allocating from the wrong zone.

For security work, this is also how you detect heap spraying attacks against the kernel. If you see a process triggering a large number of kmalloc calls with the same size, that is a signal worth investigating.

The Hardware Angle: eBPF and CPU Mitigations

This is where eBPF gets interesting for the counter-x.net audience. eBPF programs run in the kernel, which means they are subject to the same hardware-enforced mitigations as the rest of the kernel. But eBPF also gives you a way to observe those mitigations in action.

For example, you can use eBPF to monitor Control-flow Enforcement Technology (CET) shadow stack violations. When a CET violation occurs, the CPU raises a #CP exception. You can attach a kprobe to the exception handler and log the faulting instruction pointer, the shadow stack pointer, and the process context. This gives you a real-time feed of every CET violation on the system, which is exactly what you want if you are trying to understand whether CET is actually stopping attacks or just generating noise.

Similarly, on ARM64, you can use eBPF to monitor Pointer Authentication Code (PAC) failures. When a PAC check fails, the CPU raises a SP_ALIGN or PAC_FAIL exception depending on the configuration. Attach a kprobe to the exception handler and you can log every PAC failure with the faulting address and the process context. This is how you catch an attacker trying to forge a function pointer on a PAC-enabled kernel.

The caveat is that eBPF itself is a target. If an attacker can load an eBPF program, they can use it to read kernel memory, bypass mitigations, or exfiltrate data. The bpf syscall is a security boundary, and the verifier is the gatekeeper. If the verifier has a bug, eBPF becomes an attack surface, not a defense tool.

eBPF on Embedded Devices: The Baseband Problem

Most of the eBPF tooling assumes a full Linux kernel with BPF support enabled. On embedded devices, that assumption often fails. Baseband processors, automotive ECUs, and UEFI runtime environments typically run proprietary RTOSes or stripped-down Linux kernels without BPF support.

This is a problem for security monitoring, because those are exactly the devices where you need kernel-level visibility. A baseband processor is a black box that talks to the network, parses untrusted input, and has direct access to the application processor. If you cannot instrument it, you cannot monitor it.

There are a few options. Some vendors ship Linux on the application processor with BPF support, and you can use eBPF to monitor the interface between the application processor and the baseband. This is not as good as instrumenting the baseband itself, but it is better than nothing. You can watch the shared memory buffers, the IPC channels, and the network traffic that crosses the boundary.

On devices where the vendor has locked down the kernel and disabled BPF, you are out of luck. You can try to extract the firmware and reverse-engineer it statically, but you will not get runtime visibility without a hardware debugger or a kernel exploit.

Practical eBPF Development: What the Tutorials Do Not Tell You

Most eBPF tutorials start with a hello-world program that prints a message when a syscall fires. That is fine for learning the API, but it does not prepare you for the reality of building a security monitoring tool.

Here are the things that will actually eat your time:

• Verifier errors. You will spend hours restructuring code to satisfy the verifier. The error messages are often misleading, and the fix is usually to simplify your program until the verifier can prove it safe.
• Map concurrency. eBPF maps are shared between kernel and userspace, and between multiple CPUs. If you do not understand the memory ordering guarantees, you will write code that works in testing and fails in production.
• Kernel version differences. eBPF is not stable across kernel versions. Helper functions are added and removed, verifier rules change, and tracepoint formats shift. You need to test against every kernel you support.
• CO-RE and BTF. The modern way to handle kernel version differences is Compile Once, Run Everywhere (CO-RE) with BPF Type Format (BTF). This works, but it adds a layer of complexity that most tutorials skip.
• Performance. eBPF programs run in kernel context, and a slow program can stall the entire system. You need to measure overhead and optimize hot paths, which means understanding the JIT output and the CPU microarchitecture.

If you are serious about eBPF for security monitoring, start with the eBPF.io documentation, then read the kernel source for the verifier and the JIT. The documentation tells you how to use the API. The source tells you what the API actually does.

Tooling: What to Use, What to Avoid

The eBPF tooling ecosystem is fragmented. Here is a quick rundown of what is worth your time.

libbpf is the standard userspace library for loading eBPF programs. It is well-maintained, supports CO-RE, and is the foundation for most modern tools. Use it directly if you want control.

bpftrace is a high-level tracing language that compiles to eBPF. It is great for quick investigations and one-off scripts, but it is not suitable for production monitoring because the scripting language is limited and the overhead is higher than hand-written eBPF.

Falco is a security monitoring tool built on eBPF. It has a large rule set and a lot of community support, but the rule language is its own DSL, and the default rules are noisy. If you use Falco, plan to spend time tuning it.

Cilium and Tetragon are the eBPF-based networking and security tools from Isovalent. Tetragon is interesting because it does syscall and file monitoring with eBPF, but it is still young and the documentation is uneven.

Tracee from Aqua Security is another eBPF-based runtime security tool. It is focused on container security, but the event model is general enough to use outside containers.

For low-level work, I prefer libbpf and hand-written eBPF. The high-level tools hide too many details, and when something breaks, you need to understand the details to fix it.

Limitations and Failure Modes

eBPF is not a panacea. Here are the failure modes you need to plan for.

Verifier bypasses. The verifier is a complex piece of code, and it has had bugs. A verifier bypass means an attacker can load an unsafe eBPF program and use it to read or write kernel memory. This is a kernel-level compromise, and it is not theoretical. The CVE-2021-3490 verifier bug is a good example.

JIT bugs. The eBPF JIT compiles bytecode to native instructions. If the JIT has a bug, a verified program can become an unverified native code execution primitive. This is harder to exploit than a verifier bypass, but it is not impossible.

Map exhaustion. eBPF maps are kernel memory. If an attacker can create maps or fill them with data, they can exhaust kernel memory and cause a denial of service. This is why unprivileged BPF is disabled by default on most distributions.

Observability gaps. eBPF can only observe what the kernel exposes. If an attacker is running in a hypervisor, a firmware environment, or a separate security domain, eBPF will not see them. This is the fundamental limitation of any kernel-level monitoring tool.

What eBPF Cannot Do

eBPF cannot see into hardware. It cannot observe CPU microarchitectural state, cache contents, or branch predictor state. If you are trying to detect a Spectre-class attack, eBPF is the wrong tool. You need performance counters, hardware tracing, or a custom kernel module that reads MSRs.

eBPF cannot see into firmware. UEFI runtime services, SMM, and secure enclaves are outside the kernel’s visibility. If you are trying to monitor a firmware rootkit, eBPF will not help.

eBPF cannot see into other security domains. A hypervisor, a TEE, or a separate VM is outside the kernel’s view. If you are trying to monitor a cross-VM attack, eBPF is the wrong layer.

eBPF is a kernel observability tool. It is very good at what it does, but it is not a universal solution. If you need hardware-level visibility, you need hardware-level tools.

Building a Security Monitoring Stack with eBPF

Here is a concrete architecture for a security monitoring stack built on eBPF, with the tradeoffs spelled out.

Layer 1: Syscall monitoring. Attach to raw_syscalls:sys_enter and raw_syscalls:sys_exit. Log process, syscall number, arguments, and return value. Filter by cgroup, user, or process name. This is your baseline.

Layer 2: File and network monitoring. Attach to VFS hooks and network hooks. Log file opens, writes, and network connections. Correlate with syscall data to get the full picture.

Layer 3: Memory and allocation monitoring. Attach to slab allocator tracepoints and page fault handlers. Log kernel allocations and page faults. This is where you catch heap spraying and memory corruption.

Layer 4: Hardware mitigation monitoring. Attach to exception handlers for CET, PAC, and MTE violations. Log every violation with the faulting address and process context. This is where you catch attacks that bypass software defenses.

Layer 5: Userspace correlation. Use uprobes to instrument critical userspace functions. This is where you catch attacks that target proprietary userspace daemons, like a baseband control daemon or a firmware update tool.

The key is to keep each layer independent. If the verifier rejects a program in one layer, you do not want to lose visibility in the other layers. Use separate eBPF programs for each hook, and use maps to share data between them.

FAQ

Is eBPF safe to run in production?

eBPF is safer than a kernel module, but it is not risk-free. The verifier reduces the risk of memory safety bugs, but it does not eliminate it. The JIT introduces its own risks. The maps can be exhausted. If you run eBPF in production, you need to monitor the eBPF subsystem itself, and you need to have a rollback plan for when a program misbehaves.

Can eBPF detect hardware-level attacks like Spectre or Rowhammer?

No. eBPF operates at the kernel software layer. It cannot observe CPU microarchitectural state, cache contents, or DRAM disturbance. Detecting hardware-level attacks requires performance counters, hardware tracing, or custom kernel modules that read MSRs. eBPF can help you correlate software events with hardware events, but it cannot see the hardware directly.

Does eBPF work on ARM64 embedded devices?

It depends on the kernel configuration. If the vendor enabled CONFIG_BPF and CONFIG_BPF_SYSCALL, eBPF works on ARM64. If the vendor stripped BPF support to save space or lock down the device, you are out of luck. Many embedded devices run kernels without BPF support, and some vendors explicitly disable it to prevent runtime instrumentation.

What is the difference between eBPF and kprobes?

kprobes are a kernel mechanism for dynamic instrumentation. eBPF is a virtual machine that can run programs attached to kprobes. You can use kprobes without eBPF by writing a kernel module, but eBPF gives you a safer, more portable way to use kprobes without loading a module.

Can an attacker use eBPF against me?

Yes. If an attacker can load an eBPF program, they can use it to read kernel memory, bypass mitigations, or exfiltrate data. The bpf syscall is a security boundary, and the verifier is the gatekeeper. If the verifier has a bug, eBPF becomes an attack surface. This is why unprivileged BPF is disabled by default on most distributions.

Next Steps

If you want to go deeper, the next article in this series will cover eBPF verifier internals: how the verifier tracks register state, why it rejects certain loop patterns, and how to write programs that pass verification without fighting the tool. After that, we will look at eBPF on ARM64, including the differences in the JIT, the syscall ABI, and the interaction with PAC and MTE.

If you have a specific eBPF problem you are stuck on, send it in. The best questions are the ones that start with “the verifier rejected this program and I do not understand why.”