Build systems are the least trusted part of the software supply chain, and the most trusted by the people who consume the output. A build system is the collection of compilers, linkers, package managers, signing tools, CI runners, and configuration files that turn source code into a distributable artifact. Adjacent concepts include reproducible builds, provenance attestation, hermetic toolchains, and binary transparency. For anyone who has spent time extracting firmware from a locked-down baseband or glitching a secure enclave, the irony is obvious: we spend years reverse-engineering trust boundaries in silicon, while the industry ships binaries produced by a pile of shell scripts running on rented x86-64 instances with no meaningful integrity story. This matters because a compromised build system can inject code that no amount of runtime mitigation will catch. The hardware will faithfully execute whatever the build produced.
I have spent enough time staring at disassembled UEFI images and SEP firmware to know that the most elegant microarchitectural defense is useless if the attacker owns the compiler. Supply chain attacks target build systems because that is where trust is manufactured. The rest of this article is about how that trust gets broken, what the vendors claim, and why the abstractions keep failing.
The Build System as a Trust Anchor
Every signed binary is a statement: the organization that holds the signing key asserts that this artifact corresponds to a particular source revision and was produced by a known process. The signing key is not the interesting part. The interesting part is everything that happens before the signature is applied. Compilers, linkers, post-link optimizers, code-signing wrappers, and the CI orchestrator itself all sit in the trust path. If any of them is subverted, the signature is still valid. The key did exactly what it was told.
This is not a theoretical concern. The 2021 SolarWinds incident showed how a build environment compromise can propagate to thousands of downstream organizations. The attackers did not need to break any cryptographic primitive. They modified source code in the build environment and let the legitimate signing process do the rest. The resulting binaries were signed, distributed, and trusted. The hardware-enforced mitigations on the target machines — SMEP, SMAP, CET, pointer authentication — did not matter because the malicious code was part of the legitimate application.
For firmware and embedded targets, the problem is worse. A compromised build system for a baseband image or a UEFI module can inject code that runs before the OS loads, outside the visibility of most endpoint detection tools. The attacker does not need a zero-day in the target. They need a zero-day in the build pipeline, which is usually a much softer target.
Why Build Systems Are Soft Targets
Build systems are soft targets for the same reason that test infrastructure is a soft target: they are treated as internal plumbing, not as production systems. The people who run them are often under pressure to ship, not to audit. The machines are ephemeral, the configuration is sprawling, and the security model is usually “the CI runner can do anything.”
Consider the typical CI setup for a firmware project. A developer pushes a commit. The CI runner checks out the source, pulls in dependencies from a package registry, runs a build script, and produces a signed artifact. The runner has access to the signing key, or at least to a signing service that does not distinguish between a legitimate build and a malicious one. The build script is often a shell script with no sandboxing, no network egress controls, and no verification of the toolchain it is using. If an attacker can modify the build script, or the package registry, or the toolchain cache, they own the output.
This is not a failure of cryptography. It is a failure of process. The signing key is doing its job. The problem is that the build system is not a trustworthy environment, and the signature does not attest to the environment. It attests to the artifact, which is a different thing entirely.
The Broken Abstraction: Signatures vs. Provenance
The core abstraction that breaks here is the idea that a signature is a statement about the artifact’s origin. It is not. A signature is a statement that the holder of the private key approved the artifact. The holder of the private key is usually a machine, not a human, and the machine does not know whether the artifact is the result of a legitimate build or a compromised one.
Provenance attestation is supposed to fix this. The idea is that the build system records what source revision was used, what toolchain was used, what dependencies were pulled, and what commands were run. The artifact is then signed along with this provenance data. In theory, a consumer can verify that the artifact corresponds to a known source revision and a known build process. In practice, the provenance data is only as trustworthy as the system that generated it. If the build system is compromised, the provenance data is compromised too.
This is the same problem that reproducible builds try to solve from a different angle. A reproducible build is one where the same source revision and the same toolchain produce a byte-for-byte identical artifact, regardless of where the build is run. This makes it possible to detect tampering by comparing artifacts from independent builders. But reproducible builds are hard to achieve in practice, especially for firmware and embedded targets where the toolchain is often proprietary, the build process is not documented, and the output is not deterministic. The vendors claim reproducibility, but the claims rarely survive contact with a real build environment.
What the Vendors Claim
Vendors love to talk about secure boot, signed firmware, and hardware root of trust. These are real mechanisms, and they do provide meaningful protection against runtime tampering. But they do not protect against a compromised build system. If the attacker injects code at build time, the code is signed by the legitimate key, and the secure boot chain will happily load it. The hardware root of trust is not a root of trust in the build process. It is a root of trust in the boot process. Those are different things.
I have seen vendor documentation that describes a “secure development lifecycle” as if it were a technical control. It is not. It is a process document. The actual technical controls are often absent: no hermetic builds, no signed toolchains, no verification of dependencies, no isolation between build stages. The signing key is stored on a CI runner that also runs arbitrary test code. The build script pulls dependencies from a public registry without pinning versions or verifying hashes. The toolchain is installed from a tarball that nobody has audited.
This is not a hypothetical. The 2022 CISA advisory on the XZ Utils backdoor described a supply chain compromise that targeted a widely used compression library. The attacker spent years building trust in the project, then introduced a backdoor into the build process. The backdoor was not in the source code in an obvious way; it was in the build scripts and the test infrastructure. The resulting binaries were signed and distributed by the legitimate maintainers. The attack was caught by accident, not by any systematic control.
The Microarchitectural Angle
For readers of this blog, the interesting question is not whether supply chain attacks happen. It is how they interact with the hardware-enforced mitigations we spend so much time studying. The answer is that they bypass them entirely. A build-time injection is not a runtime exploit. It does not need to defeat ASLR, or bypass CET, or forge a pointer authentication code. It is simply part of the legitimate code. The CPU executes it because the CPU is designed to execute whatever the binary says, and the binary says it is legitimate.
This is why I am skeptical of vendor claims about “hardware-enforced security.” The hardware is not the problem. The problem is the software supply chain that produces the binaries the hardware runs. A secure enclave is only as secure as the code that runs inside it. If that code was compiled by a compromised toolchain, the enclave is compromised. The microarchitectural details do not matter.
There is a deeper point here about the nature of trust. Hardware-enforced mitigations are designed to protect against a particular class of attacks: runtime exploitation of memory corruption vulnerabilities. They do nothing against a build-time injection. The attacker does not need to corrupt memory. They need to corrupt the build process. The result is a binary that is malicious by design, not by accident.
Practical Examples from the Bench
Let me give you a concrete example from my own work. I was extracting firmware from an automotive ECU a while back. The firmware was signed, and the signature verified against a public key stored in the boot ROM. The boot ROM was read-protected, so I had to use fault injection to dump it. That is a hardware problem, and it is the kind of problem this blog usually covers. But once I had the firmware image, I found something more interesting: the build system that produced it had left behind a build path string that included a username and a hostname. The username was “jenkins.” The hostname was an internal CI server. The firmware was built on a Jenkins instance that was probably not isolated from the rest of the corporate network.
That is a supply chain vulnerability hiding in plain sight. The vendor had signed the firmware, but the build environment was a Jenkins server with a predictable username and an internal hostname. If an attacker had compromised that Jenkins server, they could have injected code into the firmware, and the signature would have been valid. The hardware root of trust would have loaded it without complaint. The only reason I noticed the build path string is that I was looking for clues about the build process, not because the vendor had disclosed anything about it.
Another example: I was working on a UEFI module for a laptop. The module was signed, and the signature verified against the platform key. But the build script that produced the module pulled in a precompiled object file from a third-party vendor. The object file was not signed, not hashed, and not documented. It was just a blob that got linked into the final image. If that blob had been malicious, the signature on the final image would have been valid. The platform key would have trusted it. The secure boot chain would have loaded it. The hardware would have executed it.
These are not exotic attacks. They are the result of treating the build system as a black box that produces trustworthy output. The trust is misplaced. The build system is not a black box. It is a collection of scripts, tools, and dependencies that are all subject to compromise.
What Actually Works
If you want to defend against supply chain attacks on build systems, you need to start with the assumption that the build system is compromised. That is the only safe assumption. From there, you can design controls that limit the damage.
The first control is hermetic builds. A hermetic build is one that does not depend on anything outside a declared set of inputs. The toolchain is pinned, the dependencies are pinned, the build script is pinned, and the environment is isolated. This makes it harder for an attacker to inject code through a dependency or a toolchain update. It also makes the build reproducible, which is a prerequisite for independent verification.
The second control is provenance attestation. This is not a silver bullet, but it is better than nothing. The idea is to record what went into the build and sign that record along with the artifact. The record is only as trustworthy as the system that generated it, but if the system is isolated and the record is independently verifiable, it can catch some attacks. The key is to make the provenance data useful, not just a JSON blob that nobody reads.
The third control is independent verification. This is the hardest one, but it is the most effective. If you can build the same source revision on an independent system and compare the output, you can detect tampering. This is what reproducible builds are for. It is not easy, especially for firmware and embedded targets, but it is the only control that actually catches a compromised build system. Everything else is just making the attack harder.
There is a fourth control that is often overlooked: treat the build system as a production system. That means access controls, network segmentation, audit logging, and incident response. The build system is not a development tool. It is the thing that produces the artifacts you ship. If you would not let a developer run arbitrary code on a production server, you should not let them run arbitrary code on a build server.
The Vendor Abstraction Problem
The reason supply chain attacks keep working is that vendors keep selling abstractions that do not hold. “Secure boot” is an abstraction. “Hardware root of trust” is an abstraction. “Signed firmware” is an abstraction. These abstractions are useful, but they are not complete. They protect against runtime tampering, not build-time injection. The vendors know this, but they do not talk about it because it is not a problem they can solve with a marketing slide.
The same is true of “reproducible builds.” The vendors claim reproducibility, but the claims are often aspirational. A build is reproducible only if the toolchain is deterministic, the dependencies are pinned, and the environment is controlled. Most firmware builds are none of those things. The toolchain is a proprietary compiler with undocumented behavior. The dependencies are pulled from a vendor portal that does not provide hashes. The environment is a developer’s laptop with a random set of packages installed. The result is a build that is reproducible only in the sense that it produces the same output on the same machine on the same day.
I am not saying these abstractions are useless. They are useful. But they are not sufficient. The problem is that they are sold as sufficient, and the people who buy them do not ask the hard questions. The hard question is not “is the firmware signed?” The hard question is “what produced the firmware, and can I trust that process?”
What This Means for Firmware Extraction and Analysis
For those of us who spend our time extracting firmware from locked-down devices, supply chain attacks are both a threat and an opportunity. They are a threat because the firmware we extract may already be compromised. They are an opportunity because the build system is often the weakest link in the chain, and the artifacts it leaves behind can be a goldmine of information.
Build path strings, usernames, hostnames, toolchain versions, dependency hashes, and CI configuration files are all clues about the build environment. They can tell you who built the firmware, where it was built, and what tools were used. They can also tell you whether the build environment was isolated or exposed. If you are doing firmware analysis, these clues are worth more than a thousand hours of disassembly. They tell you where to look.
This is a topic I plan to return to. The next logical step is a deep dive into reproducible builds for firmware, with a focus on the specific challenges of embedded toolchains and the ways vendors fail to meet their own claims. If you have a firmware image with a suspicious build path string, or a vendor that claims reproducibility but does not deliver, I would like to hear about it. The bench is always open.
Frequently Asked Questions
Why do attackers target build systems instead of runtime vulnerabilities?
Build systems are softer targets than runtime environments. They are often less monitored, less isolated, and less understood by the people who run them. A successful build system compromise produces a signed, legitimate-looking artifact that bypasses runtime mitigations entirely. The attacker does not need to defeat ASLR, CET, or pointer authentication because the malicious code is part of the legitimate binary. The hardware will execute it without complaint.
Does secure boot protect against supply chain attacks?
No. Secure boot verifies that the firmware or bootloader is signed by a trusted key. It does not verify that the signed artifact is the result of a legitimate build process. If the build system is compromised, the attacker can inject code into the artifact before it is signed. The signature is valid, the secure boot chain is satisfied, and the malicious code runs. Secure boot protects against runtime tampering, not build-time injection.
What is the difference between a signature and provenance attestation?
A signature is a cryptographic statement that the holder of a private key approved an artifact. It says nothing about how the artifact was produced. Provenance attestation is a record of the build process: what source revision was used, what toolchain was used, what dependencies were pulled, and what commands were run. The provenance record is signed along with the artifact. In theory, this gives consumers a way to verify that the artifact corresponds to a known build process. In practice, the provenance record is only as trustworthy as the system that generated it.
Are reproducible builds a practical defense for firmware?
Reproducible builds are the most effective defense against build system compromise, but they are hard to achieve for firmware and embedded targets. The toolchain is often proprietary, the build process is not documented, and the output is not deterministic. Vendors claim reproducibility, but the claims rarely survive contact with a real build environment. Independent verification is the goal, but it requires a level of discipline that most firmware teams do not have.
If you want to dig deeper into the hardware side of this problem, the next article will look at how fault injection can be used to extract boot ROMs from read-protected microcontrollers, and what that means for the trust model of secure boot. The build system is where trust is manufactured. The boot ROM is where it is enforced. Both are worth understanding.
