2026-10-10

What a Manual Audit Adds Over Fuzzing, Formal Verification and Bug Bounties

Automated tools, formal methods, contests and bounties each catch different bugs. Here is where a manual audit adds value, and where it doesn't.

What a Manual Audit Adds Over Fuzzing, Formal Verification and Bug Bounties

If your team already runs Slither in CI, has a Foundry or Echidna invariant suite, and plans a bug bounty, a skeptical CTO will reasonably ask: "What does a paid manual review add that we don't already have?" The answer depends on which bugs each method can find. Fortunately, there is published data on that.

What the evidence says

Two public datasets are especially useful because they look at real findings, not marketing claims.

Trail of Bits, 246 audit findings (2019). Trail of Bits categorized every finding from 23 of its paid smart contract audits and asked, for each one, whether a feasible automated tool could detect it. Its estimates:

  • About 26% of findings could likely be found with feasible static analysis and 37% with dynamic analysis (fuzzing, property-based testing, symbolic execution), usually only with a custom property.
  • For the worst findings (high severity, low difficulty), combining static and dynamic approaches could in the best case catch about 78%.
  • On the other hand, almost 49% of findings are "almost impossible to imagine detecting with a tool," and 35% of high-severity findings are unlikely to be detected automatically.
  • Unit tests, even extensive ones, showed no significant correlation with fewer audit findings.

Web3Bugs / "Demystifying Exploitable Bugs in Smart Contracts" (ICSE 2023). Researchers studied exploitable bugs from Code4rena contest reports. They concluded that "a significant number of exploitable bugs in smart contracts fall under the category of functional bugs, which cannot be detected using simple and general oracles." In other words, these are bugs where the code does exactly what it says, but what it says is wrong for the protocol's intent.

Together they suggest a clear division of labor.

What each method is good at

Method Strong at Weak at
Static analysis (e.g. Slither) Known patterns: reentrancy shapes, unchecked returns, shadowing, access-control smells. Cheap enough to run on every commit. Anything that depends on intent or economics.
Fuzzing / invariant testing (Echidna, Foundry, Medusa) Breaking properties you can write down: conservation of balances, solvency, monotonic counters. Trail of Bits rated dynamic analysis as the best automated bet for severe, easy-to-exploit bugs. Properties nobody thought to write. The fuzzer only checks what you tell it to check.
Formal verification (Certora, Halmos, K) Proving a stated property holds for all inputs, which suits core accounting and token logic. Same blind spot as fuzzing, at higher cost. Trail of Bits notes full formal verification "might take up to 9x the developer effort."
Manual audit Functional and design bugs: wrong assumptions, cross-contract and economic attacks, missing checks, and the question "what property should hold here?" Coverage is bounded by time and reviewer skill. It covers one commit.
Contests Many eyes in a short window, with paid findings. The Web3Bugs data comes from this format. Variable depth on complex systems. Results depend on participation.
Bug bounty Continuous coverage after launch, paid only for real bugs. Wormhole's $10M Immunefi payout shows what a credible bounty can attract. Reactive. Works only if the reward beats the attacker's alternative and you can patch fast.

The specific value of a manual reviewer

From the data above, a good manual audit adds three things that automated tools can't:

  1. It writes the properties. Fuzzers and provers need invariants. A strong reviewer reads your spec, works out the properties that must never break, and often hands them back as tests. Ask whether the firm delivers invariants or fuzz harnesses as part of the engagement.
  2. It finds functional bugs. The Web3Bugs authors found that many exploitable bugs need high-level semantic oracles. In plain terms, someone has to understand what the protocol is supposed to do.
  3. It reviews context. Trail of Bits flags front-running, configuration and documentation issues as almost entirely beyond tools, along with a large share of data-validation and access-control findings.

What a manual audit does not add is continuous coverage. That's the bounty's job. It also doesn't guarantee completeness. That's why the strongest programs layer these methods instead of choosing one.

A layered program for a typical DeFi launch

  1. Before the audit: static analysis in CI with every finding triaged, plus an invariant suite for core accounting. Fixing what tools can find before the audit means you aren't paying expert rates for those bugs.
  2. Audit: scope it around design and economic risk. Give the reviewers your invariants and threat model (see the Rekt Test question "Do you document the best ways to attack your system?").
  3. Optional contest: for broad surface area, after the audit fixes land.
  4. Formal verification: for a small, high-value core where a proof is worth the effort.
  5. After launch: a bug bounty with a payout scaled to extractable value, monitoring, and a diff review for every upgrade.

How to judge the incremental value in a proposal

When a firm sends a proposal, ask:

  • Which of our existing tools' output will you take as given, and where will you spend reviewer time instead?
  • Will you deliver new invariants or fuzz harnesses we can keep running?
  • What classes of bug do you expect tools to miss in our design, and how will you look for them?

A firm that answers with your architecture in mind is selling the part tools can't do. A firm that answers with a generic checklist is mostly selling what your CI already does.

Sources

Sponsored