Mitigation bypasses sit at the edge of modern exploitation. When ASLR, DEP, CFG, and their kin lock down a target, a stock ROP chain off GitHub won’t cut it. You need a hand-rolled path through the defenses. This piece walks through the mindset, the primitives, and the concrete techniques for writing your own bypasses, with an eye on real-world constraints: patch levels, compiler quirks, and the grind of debugging weird one-shots.
Understanding the Mitigation Stack You’re Up Against
Before writing a single byte of shellcode or a ROP gadget, map out exactly which mitigations are active on the target binary and the OS. On Windows, /DYNAMICBASE and /HIGHENTROPYVA mean randomized image bases and high-entropy 64-bit ASLR. DEP (Data Execution Prevention) is almost always on, enforced by hardware NX. Control Flow Guard (CFG) validates indirect call targets against a bitmap. CET (Control-flow Enforcement Technology) on newer Intel and AMD chips adds shadow stacks for return addresses—far more punishing for classic stack smashing.
On Linux, PIE (Position Independent Executable) binaries randomize the code section, FORTIFY_SOURCE adds compile-time bounds checks, and stack canaries detect linear buffer overflows. PaX and grsecurity patches layer on stronger ASLR and non-executable pages. The kernel’s KASLR, SMEP, and SMAP raise the bar for local privilege escalation exploits. Don’t assume defaults—check the binary with checksec or dumpbin /headers.
Your bypass strategy depends on which subset of these mitigations your exploit primitive can subvert. A write-what-where primitive is the holy grail, but often you’ll have less: a relative write, an arbitrary decrement, or a constrained heap overflow. The art is chaining the primitive to a specific mitigation’s weak point.
Building Primitives That Respect the Constraints
Custom bypasses start with a solid, reliable primitive. If you’re fuzzing or reversing a network service, the first crash isn’t the exploit—it’s the clue. Turn that crash into an arbitrary read/write, or at least a relative read/write with known offsets. For heap-based bugs, a use-after-free (UAF) on a C++ object with a vtable can give you an arbitrary call. Combine it with a heap spray of fake vtables that land at predictable addresses despite ASLR.
If you’re stuck with a stack buffer overflow and a canary, look for an information leak first. A format string bug or an out-of-bounds read can leak the canary value, a library address, and a stack address, defeating both stack canaries and ASLR in one go. On Windows, the PEB (Process Environment Block) holds the image base and is often reachable via FS:0x30 in 32-bit code. Leaking a single pointer from the PEB gives you the base of the main module—and from there, the IAT to resolve any API.
Primitive Quality and Its Impact on Bypass Design
The quality of your primitive dictates the complexity of the bypass. A write-four-bytes-anywhere primitive is pure gold: you can overwrite a return address, a function pointer, or a structured exception handler. A limited overflow that only lets you corrupt adjacent fields might require more creativity—corrupt a length field to turn a bounded copy into a larger overflow (a technique sometimes called buffer upgrade).
For kernel exploits, a physical memory read/write via DMA or a vulnerable driver gives a completely separate angle on bypassing KASLR and SMEP. Here, you’re not playing by the OS’s rules at all—you’re directly flipping page table entries or patching kernel code in memory. This is one reason why driver bugs remain so valuable.
ASLR Bypass: Information Leaks and Partial Overwrites
ASLR is often the first mitigation you’ll attack. Without knowing where anything lives in memory, even a perfect write primitive is useless. Information leaks are the most direct way: any bug that coughs up a pointer from the heap, stack, or a module’s data section. Leak a return address from the stack and you know the program counter’s location at the time of the bug; subtract the known offset to get the base of that module.
Partial overwrites are a craftier technique when you can only write a few bytes. On 64-bit systems, addresses are typically 48-bit canonical, so the upper 16 bits are just sign extensions. If you overwrite only the lower one or two bytes of a pointer, you can redirect execution within a 256-byte or 64KB range—enough to land on a useful gadget without knowing the full address. This works against function pointers on the heap or GOT entries.
Heap Massaging for ASLR Agnostic Targets
When you can’t get an info leak, heap feng shui can force allocations near known addresses. On older Windows systems, Low Fragmentation Heap (LFH) allocations can be predictable. By spraying objects of the same size, you can force a newly allocated object to land at a specific offset from a previous one. Combine with a UAF to make the dangling pointer point into controlled data.
On Linux, the glibc malloc implementation’s arenas and bins have predictable behavior if you control the allocation and free patterns. For example, fastbin dup attacks can place a chunk at an arbitrary address without needing a leak in some configurations. This kind of heap manipulation is essential when the binary is statically compiled or runs in an environment with no infoleak path.
DEP and NX Bypass: From ROP to JIT Spraying
Data Execution Prevention stops you from jumping to shellcode on the stack or heap. The standard workaround is Return-Oriented Programming (ROP), chaining small snippets of existing code that end in a return. ROP bypasses DEP because it executes code that’s already marked executable. But writing a stable ROP chain by hand is tedious and brittle, especially across different OS builds and patch levels.
Instead of a full ROP chain, consider VirtualProtect or mprotect staging: use a short ROP chain or function call to mark a memory region as executable, then jump to your shellcode. On Windows, you’ll need to locate the VirtualProtect address in kernel32.dll, which requires the module base. Once you have that, a carefully crafted call with the right arguments on the stack makes a page RWX. Linux equivalents use mprotect with the libc address, similarly obtained via leak.
JIT spraying is a niche but powerful technique for browsers. JavaScript engines compile code to executable pages. By crafting specific byte sequences in your JavaScript, you can force the JIT compiler to emit machine code that doubles as your shellcode, even with full DEP enabled. This requires deep knowledge of the target JIT engine’s code generation patterns, but it bypasses DEP without a single ROP gadget.
Write-Execute Regions and W^X Violations
Some systems intentionally keep a few pages both writable and executable, often in older drivers or JIT code caches. If you can find such a region—maybe via a kernel memory leak—you can write your shellcode directly there and jump to it. This sidesteps the need for mprotect/VirtualProtect altogether. Modern OSes enforce strict W^X (write XOR execute) policies in user space, but kernel drivers sometimes violate this, making them prime targets for local privilege escalation.
Control Flow Integrity (CFI) and How to Dance Around It
CFG on Windows and forward-edge CFI on Linux (via clang’s -fsanitize=cfi) validate indirect call and jump targets against a set of allowed functions. A vtable call through a corrupted pointer gets blocked if the target isn’t in the valid set. For coarse-grained CFI like Microsoft’s CFG, the check is just that the target is the start of a function—any function. So you can redirect to any other function entry point, including ones with dangerous side effects (e.g., system() if you control the argument).
Fine-grained CFI is stricter, restricting the target to a specific class of functions. Bypassing it often requires type confusion or counterfeit object-oriented programming (COOP). With COOP, you hijack a legitimate object’s vtable not to arbitrary code, but to a different valid vtable that implements a malicious sequence of virtual calls. This reuses existing code in a valid way, staying within the CFI policy.
Shadow stacks (CET) protect return addresses. Bypassing them typically involves corrupting data pointers instead of return addresses—overwriting a function pointer that’s called before the function returns, so the shadow stack mismatch never fires. Or corrupting a saved frame pointer to redirect a later function’s execution. The key is to never touch a return address that the hardware checks.
Putting It into Practice: A Step-by-Step Chain
Let’s walk through a conceptual chain against a hypothetical Windows 11 x64 binary with all mitigations on: ASLR, DEP, CFG, and CET. Assume we have a relative read/write from a heap overflow.
- Leak the Image Base. The overflow lets us read beyond an object’s bounds into the next heap chunk, which contains a pointer to a vtable inside the main module. Subtract the known offset to get the image base.
- Resolve VirtualProtect. From the image base, parse the PE headers to find the import table, resolve kernel32.dll’s base, and look up VirtualProtect’s address.
- Stage a Fake Vtable. Use the write primitive to craft a fake vtable on the heap. The vtable will have a pointer to VirtualProtect at an offset that gets called when a specific virtual method is invoked.
- Trigger the Call. Corrupt a live object’s vtable pointer to point to our fake vtable. When that object’s method is called, we effectively call VirtualProtect with arguments we’ve placed in the object’s other fields (page address, size, PAGE_EXECUTE_READWRITE).
- Execute Shellcode. After the call, the heap page is now RWX. Write your shellcode to that page and redirect execution to it. Because we never corrupted a return address, CET doesn’t interfere. Because we called VirtualProtect directly (a valid function entry point), CFG passes.
This chain is specific to the bugs and mitigations in play, but the flow—leak, resolve, stage, trigger, execute—is a blueprint for many custom bypasses.
Common Pitfalls and How to Debug Them
Custom bypasses fail in spectacular ways. You’ll see access violations in places you didn’t expect, or the chain will work once and crash on the second run. Here are a few recurring headaches:
- Heap Layout Randomization. Even with heap massaging, the OS may rearrange chunks. Use deterministic free lists and avoid crossing page boundaries that might trigger guard pages. On Windows, the LFH can be disabled for specific sizes; on Linux, stick to fastbins for small allocations.
- Bad Gadget Selection. ROP gadgets that clobber registers you need later are a common source of frustration. Use a gadget search tool like ropper or ROPgadget to find clean gadgets, and simulate the chain in a debugger with register breakpoints.
- Vtable Call Offset Mismatches. When you craft a fake vtable, the compiler may call a method at an offset that depends on the class hierarchy. Single-inheritance objects have predictable offsets; multiple inheritance adds adjustments. Reverse the actual vtable layout before forging.
- Stack Canary Collisions. If you leak a canary and use it, but the program has multiple threads with different canary values, you’ll crash. Leak the canary from the specific thread you’re exploiting, or target the main thread where the canary is static.
Debugging custom bypasses demands a kernel-level debugger (WinDbg with KDNET, or GDB with QEMU for Linux kernel exploits) to see the full memory picture. User-mode debuggers hide too much. Set hardware breakpoints on the instructions that will execute your shellcode or ROP chain, and trace the registers page by page.
FAQ
What’s the difference between a generic and a custom bypass?
A generic bypass is a pre-packaged solution—like a public ROP chain for a specific library version—that works across many targets. A custom bypass is written for a particular binary and mitigation set, often using unique information leaks or primitive combinations that don’t apply elsewhere. Custom bypasses are necessary when the target’s patch level, compiler flags, or runtime environment break the generic ones.
Do I need to learn assembly to write custom bypasses?
Yes, deeply. You’ll need to read disassembly to find information leaks, identify gadget candidates, and understand how the compiler lays out objects. Writing shellcode or ROP chains requires knowing the calling conventions, register clobbers, and alignment constraints of the target architecture (x86, x64, ARM, etc.). Without assembly fluency, you’re guessing, and guessing in exploitation leads to crashes, not shells.
How do I stay updated on new mitigations and bypass techniques?
Follow the research that comes out of security conferences (Black Hat, OffensiveCon, REcon). Read the relevant kernel and compiler source code—Microsoft’s public Windows kernel source, Linux KSPP patches, and LLVM/Clang CFI documentation. Build and test the mitigations yourself in a lab environment; there’s no substitute for watching CET shadow stack enforcement fail in a debugger. The best bypass writers reverse-engineer the mitigations as thoroughly as the targets.
