Static binary analysis is a quiet, stubborn art—pulling apart compiled code without ever letting it touch a CPU. No sandboxes, no debuggers, no live execution. For reverse engineers, malware analysts, and firmware spelunkers, it’s the first real step into the guts of a program. You don’t need the target OS, and you don’t need to worry about tripping anti-debugging traps. You just need the right tools and a willingness to stare at raw bytes until they start making sense. Here’s a look at the utilities that actually deliver, from the well-known heavyweights to the underground gems.

Why Static Analysis Still Matters

Dynamic analysis gets all the attention—sandboxing, fuzzing, step-through debugging. But static analysis is where you learn the true shape of a binary. There are no anti-debugging tricks to dodge, no environment checks, no encrypted payloads waiting for runtime to unpack. You’re staring at the raw instructions, the import tables, the strings some developer forgot to obfuscate. It’s slow, detail-oriented work, but it shows you the skeleton before the flesh ever twitches.

For malware reverse engineering, static analysis is often the only safe way to handle a sample. For vulnerability research, it’s how you spot the dangerous functions—strcpy, sprintf, gets—without accidentally triggering an exploit. And for firmware dissection, it’s how you map out memory-mapped I/O and undocumented features when you can’t even boot the device.

Disassemblers: The Core of the Toolkit

IDA Pro

Still the king, and for good reason. Hex-Rays’ interactive disassembler has been the standard for decades. Its graph view turns spaghetti x86 into something you can actually follow. The Python scripting engine lets you automate annotation, rename functions in batches, or hunt for known-bad code patterns. The decompiler plugins—Hex-Rays for x86/x64/ARM—are expensive but worth it when you need pseudocode that reads like mangled C. If you’re doing serious vulnerability research or malware triage, you’ll end up here eventually. The freeware version handles x86/x64 but lacks the decompiler; still, it’s enough to learn the craft.

Ghidra

The NSA’s open-source gift to the reversing community. Ghidra’s decompiler is surprisingly good for a free tool, often going toe-to-toe with Hex-Rays on complex functions. It supports a sprawling list of architectures—x86, ARM, MIPS, PowerPC, SPARC, even oddities like Z80 and 6502. The collaborative server mode lets teams work on the same binary at the same time, something IDA still fumbles. The learning curve is steep, and the UI feels like Java from 2005, but once you’re past that, Ghidra is a beast. Scripting in Java or Python (via Jython) is solid, and the community keeps dropping new scripts and processor modules.

radare2 / rizin

This is the terminal-native, script-first, Unix-philosophy disassembler. radare2 (and its modern fork rizin) is what you reach for when you need to pipe analysis into other tools, or when you’re working headless on a server. The learning curve is a cliff face—commands are terse, documentation is scattered, and the defaults are bare-bones. But the power is immense. It handles disassembly, hex editing, binary diffing, and even basic debugging. The r2ghidra plugin pulls Ghidra’s decompiler into the radare2 environment, giving you the best of both worlds. If you live in the terminal, this is your weapon.

Binary Ninja

A commercial tool that’s been quietly eating market share. Binary Ninja’s UI is modern and snappy, its API is clean and well-documented, and its intermediate language (BNIL) makes analysis scripts portable across architectures. The decompiler is good and improving fast. It’s not as feature-complete as IDA, but for many tasks—CTFs, embedded firmware, medium-complexity malware—it’s faster and more pleasant to use. The personal license is affordable compared to IDA Pro, which has made it a favorite among independents.

Hex Editors and Binary Parsers

010 Editor

Not just a hex editor—a binary parsing framework. 010 Editor’s template system lets you define C-like structs that overlay the raw bytes, turning a blob of firmware into labeled fields: headers, checksums, partition tables. The community repository has templates for hundreds of file formats, from ELF and PE to Nintendo ROMs and automotive ECU dumps. The scripting engine (similar to C) can run analysis passes, extract embedded files, or brute-force checksums. For reverse engineering file formats or carving data from unknown binaries, this is the tool.

ImHex

A newer, open-source hex editor with a pattern language inspired by 010 Editor’s templates. ImHex is built by and for reverse engineers: it has a built-in disassembler (via Capstone), data inspector, diffing view, and even a node-based pattern editor. The UI is dark and sleek, and it handles massive files without choking. It’s still maturing, but the pace of development is furious. If you want a free, extensible hex editor that understands binary structures, ImHex is the one to watch.

xxd / hexdump

Sometimes you don’t need a GUI. The classic xxd (or hexdump on BSD) is a command-line hex dumper that can also reverse a hex dump back into binary. It’s everywhere, it’s fast, and it pipes beautifully. Use it to grab a quick visual of a suspicious file, extract a byte range, or convert between formats in a script. Not glamorous, but foundational.

String Analysis and Metadata Extraction

GNU strings

The first thing you run on an unknown binary. strings pulls out printable character sequences, often revealing hardcoded URLs, IP addresses, registry keys, error messages, and function names. The -e flag lets you specify encoding (ASCII, Unicode, etc.), which is critical for modern malware that uses wide strings. Combine with grep to filter for patterns like HTTP, base64, or suspicious file paths. It’s primitive, but it’s the quickest way to get a sense of what a binary is doing.

FLOSS

The FireEye Labs Obfuscated String Solver. Malware authors love to obfuscate strings—XOR them, stack-construct them, or hide them in custom encodings. FLOSS automatically extracts both static and obfuscated strings from a binary by emulating small code sequences. It’s a massive time-saver when dealing with packed or protected samples. Run it before you even open a disassembler; the output often gives you the campaign ID, C2 domains, and mutex names without any manual reversing.

Exeinfo PE / Detect It Easy

Before you dive deep, you need to know what you’re dealing with. These tools identify packers, compilers, and protectors. Exeinfo PE is a Windows classic with a huge signature database. Detect It Easy (DIE) is its open-source, cross-platform cousin with a cleaner UI and scriptable detection engine. Both tell you if the binary is packed with UPX, protected with Themida, or compiled with Visual Studio—information that dictates your entire analysis strategy.

Binary Diffing and Patching

Diaphora

An IDA plugin that performs program binary diffing. Diaphora compares two IDA databases—say, a patched version of a binary against the original—and highlights added, removed, and modified functions. It’s essential for patch analysis: when a vendor silently fixes a vulnerability, diffing the before and after reveals exactly what changed, often pointing straight to the bug. It supports multiple matching heuristics and exports results to SQLite for further querying.

BinDiff

Google’s commercial binary diffing tool, now free after its acquisition from zynamics. BinDiff integrates with IDA and Ghidra, using graph isomorphism algorithms to match functions across binaries. It’s faster and more accurate than Diaphora on heavily optimized code, and its visual call graph diffing is excellent. A must-have for patch diffing and malware variant analysis.

radiff2

Part of the radare2 suite, radiff2 does binary diffing from the command line. It can compare two files byte-by-byte, or use more sophisticated analysis to match functions. The output is raw but scriptable—useful for automated triage pipelines where you need to quickly spot changes across hundreds of samples.

Specialized Static Analysis Engines

Binwalk

The firmware carving tool. Binwalk scans a binary blob for magic bytes that indicate embedded filesystems, compressed archives, or executable code. It can recursively extract a firmware image into its constituent parts—kernel, initramfs, squashfs, bootloader—often revealing hidden filesystems or backdoor binaries. If you’re doing IoT or router analysis, Binwalk is your first step after getting the firmware dump.

Checksec / PwnTools

For exploit development, you need to know the binary’s defenses. Checksec (part of pwntools) reads ELF and PE headers to report stack canaries, NX bit, PIE, RELRO, and other mitigations. It’s a one-liner that tells you whether your target is a hard nut or low-hanging fruit. Pwntools itself is a Python library that wraps a lot of tedious exploit-dev tasks, but its static analysis helpers are worth using even if you never fire up a debugger.

CWE Checker

An open-source tool that uses Ghidra’s headless mode to scan binaries for common weakness patterns—hardcoded passwords, dangerous functions, missing mitigations. It’s essentially a static analysis linter for compiled code. The output maps findings to CWE numbers, which is handy for reporting. It’s not a replacement for manual review, but it catches low-hanging bugs that might otherwise slip by.

Building Your Workflow

Static analysis isn’t about picking one tool—it’s about chaining them. A typical session on an unknown sample might look like this:

1. Triage: Run Exeinfo PE or DIE to identify the packer and compiler. If it’s packed, consider whether to unpack statically or dynamically.
2. Strings: Run FLOSS to pull both static and obfuscated strings. Grep for URLs, IPs, and suspicious patterns.
3. Metadata: Use Binwalk if it’s firmware; otherwise, check the PE/ELF headers manually or with Checksec.
4. Disassembly: Load into Ghidra or IDA. Run initial auto-analysis. Identify the entry point, main function, and any interesting imports.
5. Deep Dive: Annotate functions, trace cross-references, decompile critical sections. Use scripting to automate repetitive tasks.
6. Diffing: If you have a related sample (patched version, earlier variant), use BinDiff or Diaphora to spot changes.
7. Reporting: Export findings, annotate the disassembly database, and generate a report with key IOCs and behavioral summary.

This pipeline works for malware, firmware, and vulnerability research alike. The tools change, but the methodology stays consistent: peel back layers, map the structure, and document everything.

FAQ

What’s the difference between static and dynamic binary analysis?

Static analysis examines a binary without executing it—you’re reading the code, data, and metadata as they exist on disk. Dynamic analysis runs the binary in a controlled environment (sandbox, debugger) to observe its behavior. Static analysis is safer for malicious samples and reveals the full codebase, but it can’t show runtime-decrypted payloads or environment-dependent behavior. Most serious reverse engineering combines both.

Do I need to learn assembly language for static analysis?

Yes, but not all of it at once. Start with the architecture you encounter most—likely x86/x64 or ARM. Focus on calling conventions, control flow instructions, and common patterns like function prologues. Decompilers help by generating pseudocode, but they make mistakes, especially with hand-written assembly or obfuscated code. You need to read the disassembly to verify what the decompiler tells you.

Is Ghidra really a replacement for IDA Pro?

For many users, yes. Ghidra’s decompiler is excellent, its architecture support is broader, and it’s free. IDA still has advantages: a more mature plugin ecosystem, better debugger integration, and some advanced analysis features. High-end vulnerability research shops often keep both. If you’re starting out or working on a budget, Ghidra is the obvious choice.

How do I handle packed or obfuscated binaries statically?

First, identify the packer with tools like DIE or Exeinfo PE. Some packers (UPX, ASPack) can be statically unpacked with their own utilities or with generic unpacking scripts. For custom packers, you may need to manually reconstruct the original entry point by analyzing the unpacking stub in a disassembler. Tools like FLOSS can extract obfuscated strings without full unpacking. In many cases, static unpacking is possible but time-consuming; dynamic unpacking via memory dumps is often faster.

You’ve got a binary. No source. No symbols. No documentation. Just a slab of compiled code sitting on your drive, daring you to figure out what it really does. Maybe it’s a sketchy executable pulled from a phishing campaign, a firmware blob ripped off an IoT gadget, or some legacy app that outlived the developers who wrote it. The first move never changes: static analysis. No execution, no sandbox—just you, the file, and the tools that peel back its layers. This isn’t the sanitized world of automated scanners. This is the raw, hands-on craft of reverse engineering, and the tools you pick decide how deep you can go.

Why Static Analysis Still Matters

Dynamic analysis gets all the hype—sandboxes, debuggers, runtime instrumentation. But static analysis is where the real work begins. Before you ever let a binary touch memory, you need to know what you’re dealing with. File format quirks, embedded strings, suspicious imports, entropy anomalies, packing signatures—these are the breadcrumbs that tell you whether you’re looking at malware, a protection scheme, or just sloppy engineering. Static analysis is also the only safe way to handle truly hostile code. No VM escape worries, no anti-debug tricks, no time bombs waiting for the right system clock. You sit in your terminal, dissecting bytes, and the binary never gets a chance to fight back.

For the underground reverse engineer, static analysis isn’t just a preliminary step—it’s a philosophy. It’s about understanding the artifact as a whole before you ever poke it with a debugger. The tools in this space range from venerable command-line stalwarts to modern graphical suites, but they all share one trait: they give you control. No black-box reports, no sanitized risk scores. Just raw data and the freedom to interpret it.

The Core Toolkit: Disassemblers and Decompilers

At the heart of static analysis sits the disassembler. It translates machine code back into assembly language, giving you a human-readable (if not always human-friendly) view of the program’s logic. The decompiler goes a step further, attempting to reconstruct higher-level C-like pseudocode. These tools are your primary lens into the binary’s soul.

IDA Pro: The Industry Standard

IDA Pro has been the undisputed heavyweight of static analysis for decades. Its interactive disassembler lets you explore binaries with surgical precision—renaming functions, annotating code, defining data structures, and mapping out control flow graphs. The built-in decompiler (via the Hex-Rays plugin) turns assembly back into readable C pseudocode, which is often the fastest way to grasp complex logic. IDA’s plugin ecosystem is vast: Python scripting, custom loaders for obscure file formats, and community tools that automate everything from string deobfuscation to control flow flattening removal. The freeware version is limited to x86/x64 and lacks the decompiler, but it’s still a powerhouse for learning the craft. For serious work, the licensed version is non-negotiable.

What makes IDA truly dangerous in the right hands is its interactivity. You’re not just staring at a dead listing—you’re building a mental model of the program, labeling functions, tracking cross-references, and slowly reconstructing the original developer’s intent. It’s tedious, but it’s the closest thing to reading the programmer’s mind.

Ghidra: The NSA’s Open-Source Beast

When Ghidra dropped in 2019, it sent shockwaves through the reverse engineering community. A full-featured disassembler and decompiler, developed by the NSA and released for free, with support for dozens of processor architectures. Its decompiler is genuinely excellent—often producing cleaner pseudocode than IDA’s Hex-Rays, especially for complex ARM and MIPS binaries. The collaborative features (shared projects, version tracking) make it ideal for team-based analysis, and the scripting framework (Java or Python via Jython) is deep enough to automate almost anything.

Ghidra’s learning curve is steep, and its UI can feel clunky compared to IDA’s polish. But for anyone who can’t drop thousands on a license, or who needs to analyze obscure embedded firmware, Ghidra is a revelation. It’s also become the go-to for malware analysts who want to share annotated databases without worrying about license servers. The underground scene has embraced it hard—custom scripts for deobfuscation, unpacking, and signature generation are everywhere.

Radare2 / Rizin: The Hacker’s Scalpel

If IDA and Ghidra are full surgical suites, radare2 is the scalpel you carry in your pocket. It’s a command-line framework for reverse engineering that can disassemble, analyze, and patch binaries across an absurd range of architectures. The learning curve is vertical—commands are terse, the interface is dense—but once you internalize it, you can tear apart a binary faster than any GUI tool. Rizin is the community fork that’s been cleaning up the codebase and improving usability, but the spirit remains the same: total control, no hand-holding.

Radare2 shines in automation. Need to extract all strings from a hundred firmware samples and cross-reference them with known CVE patterns? A few lines of r2pipe script and you’re done. It’s also the tool of choice for patching binaries on the fly, analyzing esoteric file formats, and doing deep binary diffing. Not for the faint of heart, but indispensable for the underground.

Peeling Back the Layers: File Format Parsers and Unpackers

Before you even fire up a disassembler, you need to know what you’re looking at. Modern binaries are rarely just a clean PE or ELF file. They’re packed, encrypted, obfuscated, or wrapped in custom loaders designed to frustrate analysis. Static unpacking and format parsing tools are your first line of offense.

Detect It Easy (DIE)

DIE is the underground’s replacement for the aging PEiD. It’s a packer identifier, compiler detector, and file format scanner rolled into one. Written in C++ with a Qt GUI, it uses signature-based detection but also heuristics to spot unknown packers, cryptors, and protectors. It handles PE, ELF, Mach-O, and even .NET assemblies. The real power is in its scripting engine—you can write custom detection scripts in JavaScript or Python to catch new obfuscators as they appear in the wild. For anyone analyzing malware or protected binaries, DIE is the first tool you reach for.

UnpacMe and Manual Unpacking

Automated unpacking services like UnpacMe are useful for quick triage, but static unpacking is an art form. Packers like UPX are trivial to reverse, but custom protectors (Themida, VMProtect, Obsidium) require deep knowledge of PE structure, import reconstruction, and sometimes raw hex editing. Tools like PE-bear and CFF Explorer give you surgical control over PE headers, sections, and directories. For ELF, readelf and pyelftools are your friends. The goal is to reconstruct the original binary before it ever executes—a skill that separates script kiddies from serious analysts.

String Analysis and Pattern Matching

Strings are the lowest-hanging fruit in static analysis. They reveal URLs, IP addresses, registry keys, file paths, function names, and sometimes even entire command-and-control protocols. But modern malware rarely leaves strings in plain sight. They’re XOR’d, stacked, or encrypted with custom algorithms. That’s where specialized string tools come in.

FLOSS: Beyond the Basics

FireEye’s FLOSS (now maintained by Mandiant) is the gold standard for string extraction. Unlike the classic strings command, FLOSS uses static analysis to decode obfuscated strings automatically. It emulates small portions of code to resolve stack strings, tight loops, and simple XOR routines—all without executing the binary. It also scores strings by interestingness, helping you cut through the noise. For packed samples, FLOSS can’t always reach the payload, but for unpacked malware, it’s a massive time-saver.

YARA: Signature-Based Hunting

YARA isn’t just a tool—it’s a language for describing binary patterns. You write rules that match on strings, hex sequences, or even regex patterns, and YARA scans files or memory dumps to find hits. It’s the backbone of malware classification and threat intelligence sharing. The underground uses YARA for everything: identifying known packers, flagging suspicious API combinations, hunting for specific crypto constants, and building custom detection sets for private malware families. Combined with a disassembler, YARA rules can be generated automatically from unique code patterns, making it a force multiplier for static analysis.

Binary Diffing and Patching

Sometimes you’re not analyzing a single binary—you’re comparing two versions of the same malware, or a patched and unpatched firmware, to understand what changed. Binary diffing tools highlight the differences at the assembly level, letting you zero in on new functionality, bug fixes, or backdoors.

Diaphora

Diaphora is a binary diffing plugin for IDA Pro that’s become the standard for vulnerability research and malware comparison. It performs multiple rounds of diffing—from basic hash matching to deep structural analysis using ASTs and control flow graphs—to find matches even when code has been heavily modified or recompiled. It’s open source and actively developed by Joxean Koret, a well-known name in the exploit development scene. If you’re trying to figure out what a patch actually fixed, Diaphora is the tool.

PatchDiff2 and BinDiff

PatchDiff2 is another IDA plugin focused on comparing patched binaries, especially for Microsoft Patch Tuesday analysis. It’s simpler than Diaphora but effective for quick comparisons. BinDiff, originally from zynamics (now Google), is a commercial tool that uses graph isomorphism algorithms to match functions across binaries. It’s particularly good at identifying similar code in different malware samples, helping analysts track code reuse and family relationships.

Specialized Analysis: Firmware, Mobile, and .NET

Not all binaries are created equal. Firmware images, Android APKs, and .NET assemblies each require their own tooling. The underground has adapted by building or adopting tools that understand these specific formats.

Firmware Analysis with Binwalk and FACT

Binwalk is the Swiss Army knife of firmware extraction. It scans binary blobs for known file signatures—filesystems, compressed archives, bootloaders, kernels—and carves them out automatically. It can even calculate entropy to identify encrypted or compressed regions. For deeper static analysis of Linux-based firmware, the Firmware Analysis and Comparison Tool (FACT) automates extraction, disassembly, and vulnerability scanning. It’s a framework, not just a tool, and it’s invaluable for IoT security research.

.NET Decompilation: dnSpy and ILSpy

.NET binaries are a different beast. They compile to CIL (Common Intermediate Language) rather than native code, which makes decompilation far more accurate. dnSpy is the weapon of choice—it’s a debugger, decompiler, and assembly editor all in one. You can decompile a .NET malware sample to nearly original source code, patch out anti-analysis checks, and recompile on the fly. ILSpy is a solid open-source alternative. For obfuscated .NET binaries, de4dot is a pre-processor that cleans up control flow, decrypts strings, and removes junk code before you feed it to dnSpy.

Android APK Analysis: APKTool and JADX

Android malware often hides its logic in native libraries, but the entry point is the DEX bytecode. APKTool decodes APK resources and disassembles DEX to Smali, a human-readable assembly language. JADX decompiles DEX directly to Java source, which is often surprisingly readable. For native libraries, you fall back to Ghidra or IDA. The combination of JADX for the Java layer and Ghidra for the native layer covers most Android threats.

Automating Static Analysis at Scale

When you’re dealing with hundreds or thousands of samples, manual analysis isn’t feasible. You need pipelines that extract features, classify, and cluster without human intervention. This is where static analysis meets scripting.

Building Your Own Pipeline

A typical static analysis pipeline starts with file identification (magic bytes, format validation), then moves to unpacking (if needed), string extraction, YARA scanning, and finally disassembly-based feature extraction. Tools like radare2 and Ghidra’s headless mode are perfect for this—they can be scripted to output function metadata, call graphs, and instruction histograms. Combine that with a database of known malware features, and you can cluster new samples by similarity, flag anomalies, and prioritize the most interesting binaries for manual review.

The underground scene has built entire ecosystems around these pipelines. Private YARA rule repositories, custom Ghidra scripts for deobfuscation, and radare2-based triage systems are shared in closed circles. The goal isn’t just detection—it’s understanding at scale.

FAQ

What’s the difference between static and dynamic analysis?

Static analysis examines a binary without executing it—looking at its structure, disassembly, strings, and metadata. Dynamic analysis runs the binary in a controlled environment (sandbox, debugger) to observe its behavior. Static analysis is safer and often faster for initial triage, but it can’t reveal runtime behavior like network connections or decryption of deeply obfuscated code. The two approaches are complementary.

Do I need to pay for IDA Pro, or is Ghidra enough?

Ghidra covers most use cases and is free. IDA Pro’s advantages are its mature plugin ecosystem, superior interactive disassembly, and the quality of the Hex-Rays decompiler for certain architectures. If you’re doing professional vulnerability research or analyzing heavily obfuscated x86/x64 malware, IDA Pro is often worth the investment. For learning, firmware analysis, or collaborative work, Ghidra is more than sufficient.

How do I handle packed or obfuscated binaries statically?

Start with detection tools like DIE to identify the packer. If it’s a known packer, use its unpacker (e.g., UPX -d). For custom packers, you’ll need to manually reconstruct the original entry point by analyzing the unpacking stub in a disassembler. Tools like FLOSS can still extract strings from packed samples by emulating the unpacking code. In many cases, static unpacking is possible but requires deep knowledge of the file format and the packer’s techniques.

What’s the best way to learn static analysis?

Start with simple crackmes from sites like crackmes.one. Use Ghidra or radare2 to analyze them, focusing on understanding control flow and identifying key functions. Read write-ups from other reversers to see how they approach problems. Practice on real malware samples from repositories like theZoo or MalwareBazaar, but always in a safe, isolated environment. The skill comes from repetition and curiosity—there’s no shortcut.

Why Static Analysis Still Matters

It’s 2 a.m. and you’re staring down a stripped ELF binary. You can’t just fire it up in a sandbox—maybe the sample phones home and burns your analysis environment, maybe it’s targeting an architecture you don’t have lying around, or maybe you just need a fast answer before the morning briefing. This is where static analysis earns its stripes. Not the polished, academic kind you read about in sanitized papers, but the messy, hands-on dissection that reverse engineers and malware analysts do when the clock is ticking and the sample is hostile.

Static analysis means tearing a binary apart without ever letting it execute. You’re parsing headers, stepping through disassembly, mapping control flow, and sniffing out suspicious strings or imports. The right tool can mean the difference between spotting a packed dropper in five minutes and spending hours lost in a hex dump. This isn’t a roundup of every tool on GitHub. It’s a curated set of what actually holds up in the field, with a bias toward Linux and cross-platform targets.

Disassemblers and Interactive Analysis

If you’re doing serious static work, you practically live inside a disassembler. The heavy hitters get most of the attention, but a few lesser-known options still have cult followings for specific niches.

Ghidra: The NSA’s Open-Source Powerhouse

Ghidra changed the game when it landed. A full reverse engineering framework with a decompiler that goes toe-to-toe with Hex-Rays, and it costs nothing. You can script tedious chores—unpacking, identifying crypto constants—in Java or Python. The learning curve is real; the UI feels like it was designed by a committee that never met. But once you remap the keybindings and get a feel for the project management, it’s hard to live without. The decompiler output is clean enough to refactor into readable pseudocode, and the function graph view untangles spaghetti logic faster than you’d expect.

For static analysis in particular, Ghidra’s headless mode is a quiet killer feature. You can script entire analysis pipelines: import a binary, run auto-analysis, apply signatures, dump decompiled functions to text—all without ever opening the GUI. That’s a lifesaver when you’re processing dozens of samples from the same campaign.

radare2 / rizin: The Terminal Workhorse

If you’re a terminal native, radare2—or its more actively maintained fork, rizin—is your scalpel. It’s not pretty. But it’s fast and scriptable to a degree that GUI tools can’t touch. Static analysis features include recursive disassembly, entropy analysis for spotting packed sections, and a dense set of commands for hunting ROP gadgets. The learning curve is brutal; the command syntax feels like an arcane dialect you have to learn by osmosis. Once it clicks, though, you can dismantle a binary faster than any point-and-click tool. Pair it with r2pipe for Python scripting and you’ve got a static analysis engine that slots neatly into custom pipelines.

Binary Ninja: The Hacker’s Disassembler

Binary Ninja sits somewhere between Ghidra and radare2 in complexity, with a focus on clean design and a seriously capable API. Its intermediate language (IL) system is a standout for static analysis: you can lift binaries to a platform-agnostic IL and write analysis passes that work across architectures. The decompiler is solid, though not as battle-hardened as Ghidra’s, and the collaborative features make it a decent pick for teams. For static vulnerability research, the automated function signature matching and type propagation save hours of manual annotation.

Specialized Static Analysis Tools

Disassemblers are the foundation, but specialized tools zero in on specific tasks with way more efficiency. These don’t always make the top-10 lists, but they earn their keep in real engagements.

FLOSS: FireEye Labs Obfuscated String Solver

Malware authors love to hide strings—stack-based construction, XOR loops, custom decoders. FLOSS automates pulling these hidden strings out of static binaries. Under the hood, it uses vivisect to statically emulate decryption routines, extracting C2 URLs, registry keys, and other indicators without ever executing the sample. The tool has evolved to handle increasingly twisted obfuscation patterns, and its output is structured for easy ingestion into threat intelligence platforms. If you’re doing malware triage, this one’s non-negotiable.

Binwalk: Firmware and Embedded Analysis

Binwalk is the go-to for extracting filesystems and compressed data from firmware images. It scans for magic bytes, unpacks nested archives, and identifies file types buried inside monolithic blobs. For static analysis of IoT or router firmware, Binwalk is step one—it pulls out the squashfs, jffs2, or cramfs so you can then analyze individual binaries. The entropy graphing feature is also handy for spotting encrypted or compressed sections in any binary, not just firmware.

Checksec: Hardening Verification

Part of the pwntools suite, checksec is a lightweight script that inspects a binary’s security features: RELRO, stack canaries, NX, PIE, and Fortify Source. It’s a quick first pass to understand what mitigations you’re up against. For static analysis, knowing whether a binary has full RELRO or no canaries tells you immediately where to focus your effort. The tool also parses kernel configs for embedded Linux targets, which is handy when assessing attack surface on routers or IoT devices.

Static Analysis for Malware Triage

When you’re staring down a folder of 200 samples from a phishing campaign, speed is everything. These tools prioritize rapid feature extraction over deep disassembly.

YARA: Pattern Matching on Steroids

YARA is the lingua franca of malware classification. Writing effective rules means understanding both the malware family and YARA’s syntax quirks. For static analysis, YARA rules can identify packed samples, detect specific crypto implementations, or flag suspicious API import patterns—all without execution. The real power comes from weaving YARA into your analysis pipeline: scan incoming samples, route matches to appropriate handlers, and enrich results with threat intel. Keep your rule sets lean; bloated rules slow scanning and spike false positives.

CAPA: Capabilities Detection

FireEye’s CAPA tool lifts static analysis to a higher abstraction level. Instead of matching bytes, it identifies capabilities—what the malware does. Using a ruleset mapped to the MITRE ATT&CK framework, CAPA analyzes disassembly and extracts features like keylogging, process injection, or C2 communication. It works with Ghidra or IDA, and the output is a structured list of behaviors with confidence scores. For triage, this tells you immediately whether a sample is a generic stealer or something that deserves a deeper look.

PEFrame and Manalyze: PE Analysis Specialists

For Windows binaries, PEFrame gives you a quick static overview: imports, exports, sections, resources, anomalies. It’s a Python script that wraps pefile and adds heuristics for common malware traits. Manalyze goes deeper, using a YARA-based plugin system to detect packers, cryptors, and suspicious patterns. Both are essential for rapid PE triage when you don’t have time to load every sample in a disassembler.

Deep Static Analysis: Unpacking and Deobfuscation

Packers and obfuscators are the bane of static analysis. These tools help peel back the layers without letting the code execute.

UnpacMe: Automated Unpacking Service

UnpacMe is a web-based platform that automates unpacking for common packers. You upload a sample, and it returns the unpacked binary along with a report. It’s not a silver bullet—custom packers still need manual intervention—but for commodity malware using UPX, ASPack, or similar, it saves hours. The API integration means you can feed it samples directly from your analysis pipeline.

Detect It Easy (DIE): Packer Identification

Before you can unpack, you need to know what you’re dealing with. DIE identifies packers, compilers, and cryptors with a signature-based approach that’s more comprehensive than PEiD. It handles ELF and Mach-O in addition to PE, and its entropy analysis helps spot custom packing. The tool is cross-platform and actively maintained, with a plugin system for extending detection capabilities.

FLARE VM and the Mandiant Stack

While FLARE VM is a full Windows analysis environment, its static analysis components deserve a nod. Tools like FakeNet-NG (for network simulation) and FLOSS (covered above) are part of a curated toolkit that streamlines malware dissection. The VM itself is a time-saver, but the individual tools can be used standalone on any Windows analysis box.

Static Analysis for Vulnerability Research

When you’re auditing code for bugs, static analysis shifts from triage to deep inspection. These tools help find the needle in the haystack.

CodeQL: Semantic Code Analysis

CodeQL treats code as data, letting you write queries that find vulnerability patterns. Originally from Semmle and now part of GitHub, it’s free for open-source projects. For binary analysis, CodeQL can ingest decompiled code from Ghidra and apply the same queries. This means you can hunt for use-after-free, double-fetch, or other complex bugs in closed-source binaries. The query language has a learning curve, but the community provides a growing library of pre-built queries for common vulnerability classes.

BinDiff: Binary Comparison

When you need to understand what changed between two versions of a binary—say, a patched and unpatched firmware—BinDiff is the tool. It matches functions across binaries and highlights differences in control flow and call graphs. This is invaluable for patch diffing, where you reverse the fix to understand the original vulnerability. BinDiff integrates with Ghidra and IDA, and while it’s not free, the time it saves justifies the cost for professional work.

Angr: Concolic Execution Engine

Angr blurs the line between static and dynamic analysis. It’s a Python framework for binary analysis that can perform symbolic execution, control-flow recovery, and data-flow analysis. In static mode, you can use it to generate call graphs, identify dead code, or solve for conditions that reach a specific code path. It’s heavy and complex, but for deep-dive vulnerability research, it’s unmatched in the open-source world.

Building a Static Analysis Pipeline

No single tool does everything. The real power comes from chaining them together. A typical pipeline for unknown samples might look like this:

1. File identification: Use file and DIE to determine the binary format and any known packers.
2. Unpacking: If packed, run through UnpacMe or manual unpacking scripts.
3. String extraction: Run FLOSS to pull obfuscated and plaintext strings.
4. Capability detection: Feed the sample to CAPA for behavioral fingerprinting.
5. YARA scanning: Match against known malware families and packer signatures.
6. Disassembly: Load into Ghidra or Binary Ninja for deep inspection if flagged as interesting.
7. Vulnerability research: Apply CodeQL queries or BinDiff if the sample is a target for exploit development.

Automating this pipeline with scripting is where the real efficiency gains happen. Ghidra’s headless mode, radare2’s r2pipe, and Python wrappers for most of these tools let you process hundreds of samples with minimal human intervention. The goal isn’t to replace the analyst—it’s to filter out the noise so you can focus on the signals.

FAQ

What’s the best free disassembler for static analysis?

Ghidra is the top free option for most use cases. It offers a decompiler, extensive scripting, and a collaborative project model. If you need a lightweight terminal-based tool, radare2/rizin is faster and more scriptable but has a steeper learning curve.

How do I handle obfuscated strings without running the binary?

FLOSS is purpose-built for this. It statically emulates decryption routines to extract hidden strings. For heavily obfuscated samples, you may need to combine it with manual unpacking first, but FLOSS handles most common obfuscation patterns out of the box.

Can static analysis replace dynamic analysis for malware triage?

Not completely, but it can handle a large percentage of initial triage. Tools like CAPA and YARA can classify samples and extract capabilities without execution. Dynamic analysis is still needed for samples with heavy anti-static tricks, but a solid static pipeline reduces the number of samples that require a sandbox.

What’s the best approach for analyzing firmware statically?

Start with Binwalk to extract the filesystem, then use checksec to assess hardening on individual binaries. For deeper analysis, load extracted ELFs into Ghidra and apply CodeQL queries if you’re hunting vulnerabilities. Firmware often contains stripped binaries, so function ID via FLIRT or Ghidra’s BSim can help recover symbol names.

Static analysis isn’t about having the flashiest toolkit—it’s about knowing which tool to reach for when the pressure is on. Master a core set, script the repetitive stuff, and keep your YARA rules fresh. The binaries aren’t going to reverse themselves.

Intel Processor Trace (PT) is one of those hardware features that sits quietly in modern CPUs, rarely touched by most developers but capable of exposing exactly what a processor is doing at the lowest level. If you’ve ever stared at a flame graph wondering why a function call count doesn’t match reality, or tried to catch a transient control-flow hijack that leaves no footprint in logs, PT is the answer you’ve been ignoring. It’s not a magic bullet—it’s a high-bandwidth, low-overhead trace mechanism that demands a certain mindset to wield effectively. This piece breaks down how to use it for both performance dissection and security hardening, without the vendor fluff.

What Intel PT Actually Captures

Intel PT is a hardware tracing facility baked into Intel CPUs since Broadwell (5th gen Core). Unlike software-based instrumentation that injects overhead and skews measurements, PT records control-flow information directly from the processor’s Branch Trace Store (BTS) and other internal buffers. The trace data includes taken branches, indirect branches, far jumps, interrupts, and exceptions—essentially every decision point where the instruction pointer changes in a non-sequential way. You don’t get data values or register contents; that’s not the point. PT gives you the exact path of execution, which is enough to reconstruct what the CPU did and, critically, what it didn’t do.

The output is a highly compressed packet stream. Each packet encodes events like TNT (Taken/Not-Taken) for conditional branches, TIP (Target IP) for indirect branches, and flow update packets for asynchronous events. Because the CPU compresses this on the fly, the overhead stays around 5% even for branch-heavy workloads—far less than binary instrumentation tools like Pin or DynamoRIO. The trade-off is that you need a decoder to make sense of the binary trace, and that’s where tools like perf with libipt come in.

Setting Up Intel PT on Linux

Most modern Linux distributions ship with a kernel that supports PT via the perf subsystem, but you’ll need to verify. Check your CPU with grep intel_pt /proc/cpuinfo—if you see flags, the hardware is there. Then confirm the kernel exposes it: ls /sys/devices/intel_pt/ should show a directory. If not, you might need a kernel rebuild with CONFIG_PERF_EVENTS_INTEL_PT=y, but that’s rare on recent distros.

Install the necessary user-space tools. On Debian/Ubuntu, grab linux-tools-generic and linux-tools-$(uname -r). You’ll also want libipt-dev for the decoder library if you plan to process traces programmatically. The perf tool itself must be built with PT support—check with perf version and look for “intel_pt” in the features list. If it’s missing, compile perf from kernel sources with the required libraries.

Permissions matter. PT uses hardware resources that require root or specific capabilities. Run sudo perf record -e intel_pt// -- sleep 1 as a quick smoke test. If you get a trace file, you’re in business. For production use, set perf_event_paranoid to 0 or add your user to the perf_users group.

Performance Profiling with Intel PT

Traditional sampling profilers like perf record -e cycles capture snapshots at fixed intervals. They’re great for hot spots but blind to execution order and short-lived spikes. PT fills that gap by recording every branch, enabling exact control-flow reconstruction. This is where you catch cache misses that trigger unexpected code paths, or identify mispredicted branches causing pipeline stalls.

Recording a Trace

Start with a simple recording: sudo perf record -e intel_pt// -- your_workload. The double slash lets you pass PT-specific options. For performance analysis, you often want to limit trace scope to avoid drowning in data. Use --filter to trace only specific processes or threads, or --snapshot mode to capture a window around an event of interest. Snapshot mode is underrated—it keeps a circular buffer and dumps it on a trigger, so you can trace a live system for hours without filling your disk.

Decoding the trace is where the real work begins. perf script with the --itrace flag reconstructs instruction flow. For example, perf script --itrace=i0ns synthesizes instruction events with zero skid, meaning you see exactly when each branch occurred, not a sampled approximation. Combine this with perf inject --itrace to convert PT data into a format that perf report can digest, giving you cycle-accurate profiling.

Analyzing Control Flow Anomalies

One of PT’s killer features for performance work is spotting unexpected branches. A function that should be inlined but isn’t, a loop that exits early due to a mispredicted condition, or a hot path that spills into cold code—all visible in the trace. Use perf script --itrace=cr to generate call-return stacks, then feed them into Brendan Gregg’s FlameGraph tools. The resulting flame graph shows exact call stacks, not statistical guesses. You’ll see tail calls, interrupt handlers, and kernel entries that sampling profilers often miss.

For deeper analysis, ptdump from the libipt suite prints raw packet traces. This is low-level but invaluable when you suspect hardware errata or decoder bugs. You can also write custom decoders using libipt to extract specific patterns—like counting indirect branch mispredictions by correlating TIP packets with subsequent execution.

Security Applications: Catching ROP and CFI Violations

Intel PT’s security value comes from its ability to record control flow without trusting software. A rootkit can lie to the kernel, but it can’t hide a branch that the CPU executed. This makes PT a powerful tool for detecting Return-Oriented Programming (ROP) and other control-flow integrity (CFI) violations.

Detecting ROP Gadgets

ROP attacks chain short instruction sequences ending in indirect branches (usually returns). Normal code has a predictable call-return pattern: each call pushes a return address, each ret pops and jumps to it. PT traces expose mismatches. If a ret doesn’t correspond to a prior call, or jumps to an address not preceded by a call, you’ve got a gadget chain. Tools like pt-rop (part of the libipt suite) automate this detection by parsing PT packets and flagging anomalies.

Set up a continuous monitoring session with perf record -e intel_pt// --filter='filter ip 0x400000/0x100000' to trace only specific code regions, reducing noise. Then run pt-rop on the trace to look for mismatched call/return pairs. This is especially useful in production environments where you can’t afford the overhead of full instrumentation but need to verify that critical code paths aren’t being hijacked.

Indirect Branch Tracking

Intel PT can also enforce coarse Control-Flow Integrity (CFI) by comparing indirect branch targets against expected values. Modern processors support CET (Control-flow Enforcement Technology), but PT offers a software-based alternative for older hardware. Record a baseline trace of legitimate execution, extract all indirect branch targets, and then monitor live traces for deviations. A jump to an address outside the known set is a strong indicator of exploitation.

This approach is particularly useful for embedded systems and legacy servers that won’t see CET hardware. The overhead is low enough to run continuously, and you can feed the trace data into an intrusion detection pipeline. Just be aware that JIT-compiled code and some language runtimes (looking at you, V8) generate dynamic indirect branches that complicate baseline creation. You’ll need to whitelist known JIT regions or use Intel’s Processor Trace Decoder Library to filter out legitimate dynamic code.

Advanced Decoding and Tooling

The raw PT trace is a binary blob that requires decoding to be useful. perf script is the easiest entry point, but for custom analysis you’ll want to link against libipt directly. The library provides a packet decoder (pt_pkt_decoder) that yields individual packets, and a higher-level instruction flow decoder (pt_insn_decoder) that reconstructs the execution path. The latter uses sideband information—memory maps and binary images—to resolve addresses to symbols.

Here’s a minimal C snippet to get you started with libipt:

struct pt_config config;
pt_config_init(&config);
config.begin = trace_buffer;
config.end = trace_buffer + trace_size;

struct pt_insn_decoder *decoder = pt_insn_alloc_decoder(&config);
while (1) {
    pt_insn_decode(decoder);
    // process decoder.event, decoder.ip, etc.
}

You’ll need to provide the sideband data yourself—memory mappings and binary files—which is where perf record --intr-regs and perf buildid-list come in. The decoder uses these to translate raw addresses into meaningful symbols. Without sideband, you just get hex dumps, which are nearly useless for complex workloads.

Handling Trace Gaps and Overflows

PT buffers are finite, and when they overflow, you lose data. The trace contains OVF (overflow) packets that mark discontinuities. A resilient decoder must handle these gracefully—resynchronizing at the next PSB (Packet Stream Boundary) packet. PSBs are periodic synchronization points inserted by the hardware; you can control their frequency with the psb_period config option. Shorter periods mean more frequent sync points but higher overhead. For security monitoring, a tight PSB period (e.g., every 4K bytes) ensures you can recover quickly after an overflow. For performance profiling, a longer period reduces trace size.

Another pitfall: trace decode errors due to corrupted packets. Intel PT uses a compressed format that’s sensitive to bit flips. If you’re capturing traces on unreliable media or over a network, implement integrity checks. The libipt decoder returns error codes for malformed packets; your tooling should log these and attempt resynchronization at the next PSB rather than aborting the entire decode.

Integrating PT into Your Workflow

For routine performance work, wrap PT recording into a script that automates decode and flame graph generation. Something like:

perf record -e intel_pt// -- your_workload
perf script --itrace=i0ns --ns -F comm,tid,pid,time,cpu,event,ip,sym,symoff,flags > trace.txt
stackcollapse-perf.pl trace.txt > folded.txt
flamegraph.pl folded.txt > pt_flame.svg

This gives you a precise flame graph with nanosecond timestamps. Compare it against a regular sampling flame graph to see what you’ve been missing—often entire functions that execute too quickly for the sampler to catch.

For security monitoring, consider a daemon that continuously records PT traces in snapshot mode and periodically decodes them looking for ROP signatures. The perf record --snapshot option writes data only when a trigger event occurs, so you can set a USR2 signal handler to dump the buffer on suspicious activity. Combine this with auditd or a custom kernel module that fires the trigger when it detects an anomaly.

Hardware Limitations and Workarounds

Intel PT isn’t available on all SKUs. Some low-end Atom and Celeron processors lack it entirely. On supported CPUs, the feature may be fused off in firmware—check your BIOS for an “Intel PT” toggle. Virtualized environments add another layer: PT can be exposed to guests via Intel VT-x, but the hypervisor must support it. KVM and Xen have PT passthrough, but VMware and Hyper-V lag behind. If you’re in the cloud, you’re likely out of luck unless you’re on bare-metal instances.

Trace bandwidth is another constraint. PT can generate hundreds of megabytes per second per core. The hardware has internal buffers, but if your storage can’t keep up, you’ll get truncated traces. Use perf record --snapshot mode to keep only relevant windows, or filter by process ID and address range to reduce volume. For long-running security monitoring, consider a dedicated trace server with high-speed NVMe storage.

FAQ

What’s the difference between Intel PT and LBR (Last Branch Record)?

LBR records only the last 4–32 branches in a fixed set of MSRs, giving you a tiny window of control flow. PT streams a continuous trace of all branches to memory, limited only by buffer size and storage bandwidth. LBR is simpler to decode and has near-zero overhead, but it’s useless for long-running analysis or detecting rare events. PT is the heavy-duty option for deep dives.

Can Intel PT trace kernel-mode execution?

Yes, but it requires root permissions and careful configuration. By default, PT traces both user and kernel space when run as root. You can filter to kernel-only with perf record -e intel_pt//k or user-only with //u. Tracing kernel code is invaluable for debugging driver bugs or detecting rootkits that hook system calls, but the trace volume can be enormous—use address filters to narrow the scope.

How do I decode PT traces without perf?

Use the standalone ptdump tool from libipt for raw packet inspection, or write a custom decoder using libipt’s C API. The library handles the complex packet decoding and instruction flow reconstruction. You’ll need to provide sideband information (memory maps, binary files) manually, which you can extract from a core dump or /proc/pid/maps. For automated analysis, the pt_insn_decoder API is the way to go.

Is Intel PT useful for debugging multi-threaded race conditions?

Absolutely. PT traces each hardware thread independently, so you can capture exact interleavings of instructions across cores. Use perf record --per-thread to get separate traces per thread, then align them by timestamp. This reveals ordering bugs that are invisible to breakpoint debugging because the act of stopping a thread changes the timing. The trace is a passive observer—it doesn’t perturb the system’s execution.

The Death of the Simple Overflow

Let’s start with the most obvious shift: the classic stack buffer overflow, the kind described in “Smashing the Stack for Fun and Profit,” is practically extinct on modern systems. Not because developers stopped making mistakes. Buffer overflows still happen. What changed is that the exploit path from overwriting a return address to code execution has been systematically dismantled.

Stack Canaries: The First Real Nail

Stack canaries, or stack cookies, were an early and surprisingly effective defense. A random value placed between local variables and the saved return address on the stack, checked before function return. If you overflow a buffer linearly, you clobber the canary. The program crashes instead of redirecting execution.

The immediate counter was to leak the canary, but that requires an information disclosure primitive. Suddenly, your simple overflow wasn’t enough; you needed a second bug. That changed the game from single-shot exploitation to multi-stage attacks. The era of chaining primitives began in earnest.

NX/DEP: Separating Code from Data

Non-executable stack and heap, enforced by hardware NX bits (AMD) or XD bits (Intel) and managed by the OS as DEP on Windows, meant your shellcode on the stack or heap couldn’t just run. The classic response—return-to-libc, then ROP—turned exploit dev into a puzzle of stitching together existing code fragments. That required knowing addresses, which brings us to the next barrier.

ASLR: Making Addresses a Moving Target

Address Space Layout Randomization randomizes the base addresses of key memory regions: the stack, heap, libraries, and on PIE-enabled systems, the main executable itself. Without an info leak, you’re guessing addresses. Early Linux ASLR had weak entropy, especially on 32-bit, and bruteforcing was viable. Modern 64-bit systems offer vast search spaces. On Windows, high-entropy ASLR for 64-bit processes makes blind ROP impractical.

Combined with mandatory ASLR on iOS and Android, and the widespread adoption of PIE (Position Independent Executables), the attacker’s need for an information leak became non-negotiable. Exploit chains now routinely pair a use-after-free or arbitrary read with a write primitive, just to disclose a single code pointer.

This isn’t popping someone else’s proof-of-concept. It’s the quiet, stubborn work of figuring out what really happens when a syscall copies the wrong size from userspace, or a netlink handler forgets to check privileges. Linux kernel exploit development is precision, patience, and a kind of bloody-mindedness you don’t pick up from bug bounty reports. We’ll walk the real workflow—from mapping the attack surface to stabilizing a use-after-free against modern mitigations—with zero marketing fluff.

Why the Kernel Is a Different Beast

Userland exploitation is a playground with boundaries you can see. You’ve got your stack, your heap, your libc. The kernel is a shared, concurrent mess where one bad write panics the box and your target object gets freed under you by a workqueue. You’re not just dodging ASLR and NX. You’re staring down SMAP, SMEP, KPTI, and a growing list of structure-specific hardening tricks. The question stops being “how do I hijack execution” and starts being “how do I massage slab state precisely enough to survive until I win.”

I keep a build environment intentionally behind the latest stable—say, a 5.15 LTS with a known vulnerable driver compiled in. This isn’t about chasing 0-days. It’s about mastering the techniques on a target where you can afford to reboot a thousand times without anyone yelling at you. The workflow always kicks off the same way: static analysis of a driver that handles user-controlled data, usually through ioctl, write, or setsockopt.

Choosing Your First Target

Modern kernels have an absurd attack surface, but not all of it is reachable from an unprivileged namespace. When I’m teaching myself something new, I drift toward out-of-tree drivers or half-forgotten subsystems—think hamradio or android binder backports. The goal is a code path where a length field gets no real check against the destination buffer, or a reference count drops without proper locking. Tools like Syzkaller are fine for fuzzing, but for deliberate exploit development you need to read the code yourself and build a mental model of every allocation and free path.

Heap Grooming and the Slab Allocator

Userland heap exploits often orbit tcache or fastbins. The kernel SLUB allocator is a whole different animal. You’re dealing with dedicated caches for object sizes like kmalloc-192, kmalloc-1024, and structure-specific caches like files_cache. A use-after-free or double-free means you have to reclaim that exact slab slot with a controlled object before the dangling pointer gets dereferenced.

This is where heap spraying shows up, but not the kind you might remember from browser exploits. You can’t just spray ArrayBuffers. Instead, you lean on syscalls that allocate kernel objects of a predictable size: add_key for keyrings, msgget for System V messages, or setsockopt for network-related buffers. The trick is finding an allocation path that lets you control the first few bytes of the object—those bytes often hold function pointers or structural fields like ops vectors or cred pointers.

Crafting a Stable Use-After-Free

Say you’ve found a bug in a driver’s release function: it frees a structure but leaves a file descriptor’s private data pointer dangling. The race window might be tight, so you trigger the free and immediately allocate a new object of the same size. I often use a dedicated thread spinning on userfaultfd or FUSE to pause a copy_from_user midway, stretching the race window artificially. Quiet technique. Doesn’t rely on lucky timing—you choose exactly when the kernel resumes.

Once you’ve reclaimed the slot with a fake object, you need to live through the next few instructions until you can trigger a privilege escalation. That means your fake object’s fields have to satisfy any sanity checks the vulnerable code performs. If you’re overwriting a struct file_operations, you might set the release pointer to a gadget that pivots the stack to a controlled location. But with SMAP, that location can’t be in userspace anymore.

Bypassing Modern Mitigations

SMEP and SMAP stop the kernel from executing or accessing userspace memory directly. KPTI isolates kernel page tables from userspace, so even a leaked kernel address doesn’t hand you a direct map. This forces stack pivoting into the kernel heap or chaining gadgets entirely within kernel space. The ROP chain has to be built from the kernel image itself, which means you need an information leak to beat KASLR.

Information leaks often come from the same bug class you’re exploiting. A heap out-of-bounds read in a syscall might let you leak a nearby object’s slab freelist pointer, which points to another kernel address. From there you can calculate the kernel image base. I’ve also used /proc/kallsyms on older setups, but production systems usually lock that down. Instead, side-channel techniques like prefetch timing or using uninitialized memory in copy_to_user are more practical.

Real-World Example: CVE-2022-1786

A while back I spent time with CVE-2022-1786, a use-after-free in the io_uring subsystem. The bug was in handling IORING_OP_TEE where a pipe buffer could be freed while still referenced. The exploit involved registering a fixed buffer with io_uring_register, triggering the free, and then spraying struct pipe_buffer objects to reclaim the memory. The twist? Modern kernels had randomized slab freelists, so I needed a secondary info leak from an uninitialized io_uring_cqe to locate the reclaimed object. The final payload overwrote the pipe buffer’s ops->release pointer with a gadget that called commit_creds(prepare_kernel_cred(0)).

That gadget, by the way, is a classic. You find it by scanning the kernel’s .text for a call to prepare_kernel_cred followed by a call to commit_creds, usually in run_umount or __sys_setuid code paths. The annoying part is setting up the registers so the result from prepare_kernel_cred (a pointer to a new cred structure) lands as the first argument to commit_creds. Usually that demands a register pivot gadget first.

Tools of the Trade

You can’t do this with just a text editor. My toolkit is minimal but deliberate: a custom QEMU VM with a debug kernel and GDB attached via kgdboc. I use a small Python script that parses System.map and spits out offsets for common structures, and a C program that opens the vulnerable device and triggers the bug with precise timing. For heap visualization, I hacked together a script that parses slabinfo before and after each spray step, so I can see exactly which caches are active.

One undervalued trick is using ftrace to trace the exact function calls leading to the bug. Enable event tracing for kmalloc and kfree on the suspect slab, and you can reconstruct the timeline of allocations and frees from user space. That turns a blind spray into a targeted reclaim, because you know precisely when the vulnerable object gets freed.

Stabilizing the Exploit

A kernel exploit that works once in ten tries is a denial-of-service tool, not a reliable exploit. Stabilization means handling the kernel’s inherent concurrency. You need to account for interrupts, preemption, and other threads that might allocate from the same slab. One approach: set CPU affinity for your exploit process with sched_setaffinity, pinning it to a single core while you do the critical operations. Another: flood the slab with placeholder objects beforehand, so the vulnerable slot is less likely to get snatched by an unrelated allocation.

After the privilege escalation, you need a clean exit. You can’t just call execve("/bin/sh") from kernel context directly, so you usually return to userspace with the new credentials. That means saving the original register state before the ROP chain and restoring it after commit_creds. Mess up the stack frame and you’ll kernel panic right at the finish line—a lesson I’ve learned more times than I’d like to admit.

Defensive Implications

Understanding this craft isn’t just about offense. After spending weeks massaging the slab, you start to see why seemingly innocent code patterns are dangerous. A kfree followed by a goto without clearing the pointer, a copy_from_user without a bounds check on a structure length—these aren’t just bugs. They’re invitations. The hardening features we bypass exist because researchers demonstrated the attack techniques first. Every __randomize_layout annotation in a kernel structure was added because someone, somewhere, managed to overwrite that exact field.

If you’re on the defensive side, pay attention to the slab caches your code uses. Is your structure mixed into a generic cache, or does it have its own dedicated one? A dedicated cache makes heap separation easier for attackers; a generic one forces them to contend with noise. Neither is a silver bullet, but the choice changes how hard a spray-based exploit has to work.

Building Your Own Lab

Start with a kernel that got patched for a known vulnerability, then revert that patch in your local tree. Build it with debug symbols and minimal hardening: nosmep, nosmap, nokaslr on the kernel command line for early stages, then gradually re-enable them as your techniques improve. Write your own vulnerable kernel module—a simple character device that does a bad kfree—and exploit it from first principles. No copy-pasting from exploit-db. The goal is to internalize the state machine of the allocator and the dance of the stack frame.

This path is slow. You’ll read more mm/slub.c than you ever wanted. You’ll stare at register dumps wondering why RAX is zero when it should hold a pointer. But when you finally pop a root shell from a kernel you built and hardened yourself, the quiet satisfaction isn’t about the shell. It’s about knowing exactly why every byte is where it is.

FAQ

• Do I need to know assembly for kernel exploit development? Yes, specifically x86_64 assembly. You’ll need to read disassembly in GDB, understand calling conventions, and manually construct ROP chains. ARM64 is also useful if you’re targeting Android or embedded systems.
• What’s the best way to practice without breaking the law? Build your own vulnerable kernel modules or use deliberately vulnerable virtual machines like those from the pwn.college program. Always work on systems you own or have explicit permission to test.
• How do I handle kernel panics during development? Use a virtual machine with a serial console and configure kexec or panic_on_oops to automatically reboot. Log everything to a host file via virsh console. Expect hundreds of panics; that’s normal.
• Are there any good books on this topic? There are no definitive books, but reading the kernel source itself and studying write-ups from Google Project Zero or the Linux Kernel Exploitation blog posts by various researchers is the most current way to learn. The techniques change faster than any publisher can keep up.