2026-10-10

How to Tell If Your Audit Was Generic or Junior

Signs that an audit report is mostly checklist output or a thin review, and how to check depth before you pay the final invoice.

How to Tell If Your Audit Was Generic or Junior

Two audit reports can have the same logo, the same page count and the same "no critical issues remaining" summary, and still represent very different amounts of work. One is the output of senior reviewers who learned your system and attacked it. The other is tool output and a checklist, written up by someone who never really understood the design. A CTO who can't tell them apart is paying for a document, not for assurance.

This guide lists the warning signs and the checks you can run yourself.

Why this matters

Published data shows where generic reviews fall short. Trail of Bits' analysis of 246 findings from its own audits estimated that almost half are unlikely to ever be found by automated tools. The ICSE 2023 Web3Bugs study found that many exploitable bugs are functional: they can't be caught by simple, general checks and need someone who understands what the protocol is supposed to do. A review that stays at the level of patterns and checklists will, by construction, miss the bugs that most need a human.

Red flags in the report

1. Findings that could apply to any codebase. "Use a fixed pragma," "missing zero-address check," "consider using SafeERC20," "floating pragma," "events not emitted." These are real but low-value notes, and a static analyzer produces most of them. If they make up most of the report, the reviewer probably spent their time on things your CI already does.

2. No system description in the reviewer's own words. A deep review explains your architecture back to you: actors, trust assumptions, money flows, and what must always be true. If the report just copies your README, or has no system overview at all, ask how the reviewers built their mental model.

3. No stated assumptions or threat model. Who is trusted? Admins, oracles, keepers, bridges, governance? A report that never says what it assumed can't tell you what it didn't check. The Rekt Test asks whether you document "all actors, roles, and privileges" and "the best ways to attack your system." A good auditor will want those documents, and will challenge them.

4. Severity without reasoning. Contest platforms publish explicit severity rules (Code4rena's severity categorization, Sherlock's judging guidelines, Immunefi's classification system) because severity depends on impact and likelihood in context. A finding marked "Medium" with no explanation of the impact, preconditions and realistic exploit path is a weak finding, whatever its label.

5. No proof of concept for serious issues. For high-severity findings, strong reports include a test, a transaction sequence or at least a step-by-step exploit path. Without one, you can't tell whether the issue is real, and you can't test your fix against it.

6. Nothing on economics or composition. If your protocol prices assets, liquidates positions, or depends on external liquidity, a review with no discussion of oracle manipulation, flash-loan-funded paths or cross-contract interaction has probably skipped the attack class behind many large DeFi losses.

7. Coverage that is never stated. Which files and functions were reviewed, at which commit, for how many reviewer-days? If the report won't say, you can't compare it to anything.

Red flags in the engagement

  • The team you met isn't the team that did the work. Ask for the names and public track records (contest results, published findings, talks) of the reviewers who will actually read the code, and get them in the contract.
  • Turnaround that doesn't fit the code size. Ask for the reviewer-days behind the quote and compare them with the line count and complexity of what's in scope. If a large, novel codebase is quoted at very few days, ask what is being skipped.
  • No questions from the reviewers. Deep reviews generate questions about intent ("is this rounding direction deliberate?"). A quiet engagement is a warning sign.
  • No fix review. If the firm doesn't re-check your fixes against the original findings, nobody has verified the version you actually deploy.

Five checks you can run yourself

  1. Run Slither (or your CI's analyzers) on the audited commit. Compare the output with the report's findings. A large overlap with nothing beyond it is a strong signal of a generic review.
  2. Pick your three riskiest functions (anything that moves funds or changes privileges) and look for them in the report. A deep review usually discusses them even when it finds nothing wrong.
  3. Ask for the invariants the reviewers used. Strong teams write down what must never break and test it. If they can't list any, they probably didn't work that way.
  4. Ask one design question on a call. For example: "What happens to solvency if the oracle is stale for an hour?" A reviewer who studied your system will answer specifically.
  5. Compare against a public report from the same firm. Many firms publish their reviews (Trail of Bits keeps a public repository of past reviews, and OpenZeppelin publishes audits for its contracts library). Your report should show the same depth they show in public.

What "good" looks like

A strong report explains your system accurately, states its assumptions and scope by commit, argues each serious finding with impact and likelihood, includes a proof of concept where it matters, discusses economic and cross-contract attacks relevant to your design, and records how each fix was verified. The next article in this series covers the specific deliverables that prove that depth.

Sources

Sponsored