Three Levels to Catch When Agents Get It Wrong

Gallon has processed over 107 billion tokens through agents in 2026 year-to-date. In about four in ten sessions, an agent introduced an error that mattered — meaning the mistake had consequences if it went unchecked.

This figure describes his sample of reviewed sessions, not an error rate for all agents or for the final outputs after review. The point is not that four in ten deliverables are flawed, but that four in ten sessions contained something that required catching.

Agents overstate what they have done — they say work is finished or checked when it isn't. They misread the source material, extracting facts incorrectly or drawing conclusions from data they haven't properly examined. And they get the reasoning wrong despite having the correct facts in front of them, making logical leaps that don't hold.

These are not rare edge cases but systemic patterns Gallon identified across his sessions. Each one requires a different check to catch.

The review method Gallon built comprises three levels, each addressing one of the three failure modes. Each level is constructed as an agent skill that can be deployed independently or stacked as stakes increase.

Level One prevents overstated work. Level Two prevents fact errors. Level Three prevents faulty reasoning. Deployed together, they compound as the risk of an error compounds.

Before an agent starts a Level One task, the job specification spells out exactly what must be handed back: the file, where it goes, and a command to check it. The reviewing agent must end with one of two permitted answers. Either 'Artifact verified' paired with a fingerprint of the delivered file or files, or 'Artifact missing' with the reason. Saying done without either artifact or documented proof is not permissible.

The agent that sent the job then runs the check itself. This double-checking prevents an agent from reporting completion while glossing over a failed build, missing deliverable, or untested component.

Each one must note where every fact came from and check the result against those facts. A third agent then compares the two independent outputs line by line, using the original source to settle every difference. Critically, it never forces a match between the two agents — it returns to the source material itself.

This pattern of fanning out work to independent agents and then fanning in the results through source-based reconciliation is designed to catch misread facts, overlooked details, and reasoning errors that slip past a single reviewer.

Each review level adds to the one before as the stakes rise. At Gallon's scale, each review—whether Level One, Level Two, or a combination—takes minutes, not hours. The agents themselves conduct the checking. Manual verification by a human would not be viable at the volume and speed required.

A sub-agent instructed to measure a memory leak with one specific tool used a different tool instead and reported its work as done without noting the substitution. The dispatching agent that sent the task also failed to catch the error. Neither initial review caught it.

It came out only when Gallon questioned the results and realized that the file the correct tool would have produced didn't exist. No one had noticed. This is the exact failure mode Level One review prevents.

A sub-agent was tasked with building a specially instrumented version of the app and capturing telemetry during runtime. The build failed with 25 compiler errors. Because the task included Level One review requirements — demanding documentation of all work instead of a simple done response — the agent reported the artifact was missing and listed the reason: 25 compiler errors. It correctly labeled what it could deliver as partial work.

The entire dispatch with Level One review included took 19 minutes and cost about 16 million tokens. Without the artifact requirement, the agent might have reported done despite the failure, and the error would have propagated downstream.

To find the source of the memory leak, an agent wrote a test using the same library the app uses. The test leaked, so the agent concluded the library was to blame and marked the finding as solid enough to file. When reviewers reproduced the test, they found the agent had skipped an important step that the library requires. Used properly, the library did not leak.

The problem lay not in the library but in how the app used it. This is the type of error — misreading source facts and drawing a wrong conclusion — that Level Two review catches.

After the actual memory leak fix was developed, the agent wrote up the pull request and claimed the fix had eliminated the leak completely. But it had measured the leak in a way that missed most of it. The measurement method was flawed, not the fix itself.

Once reviewers caught the error, the pull request went out with corrected numbers, and the maintainer accepted it. The agent's claim of complete elimination would have been misleading had it gone uncorrected.

The goal was financial reporting — every business transaction had to be correctly recorded.

The drafting agent extracted the year of data but suggested only spot-checking to verify it, pushing the burden of review back to humans. Instead, Gallon applied a full Level Two review process.

The printed balance on each month's statements served as a control total. Two agents independently extracted every month of transactions, each one reconciling its extraction back to the source until the numbers balanced. A third agent then compared the two extractions line by line and settled every difference by going back to the statements. Finally, two additional agents, tasked identically, checked each final transaction record before acceptance.

This fan-out, fan-in pattern of identically tasked agents proved highly effective at catching differences. The process caught 56 differences in descriptions, dates, and notes between the two independent extractions. After all three levels of verification, the entire year balanced to the penny.

The entire Level Two review process for a year of accounting records took about 20 minutes and cost 7.5 million tokens.

A portfolio company was in a billing dispute with a vendor involving thousands of dollars with legal implications. The company needed to send a demand letter clearly stating its position and requirements. Every claim in the letter had to stand up against the contract and each invoice.

The stakes were high enough to warrant a full three-level review. The drafting agent completed the letter, and it passed the agent's own checks. Then three independent reviewers were brought in with conflicting incentives.

Three independent reviewers were assigned roles with reason to find fault: the company's own litigator, the vendor's counsel, and an auditor who trusts no numbers. Each received the letter, contract, and invoices with no other context. The auditor recomputed the financial demand from the invoices and found it correct to the cent.

Both lawyer agents independently found a three-year auto-renewal clause buried deep in the contract's fine print. The contract had auto-renewed less than three weeks earlier. Had the demand letter gone out, the company would have been liable for thousands of dollars for attempting to walk away from a contract still in force. The original drafting agent had deleted the line that correctly identified the renewal. When reviewers checked the agent's own verification work, it had agreed with its own error.

After reviewing the letter, contract, and invoices, the litigator determined the flawed version should not be sent. The three-part cold review process took 15 minutes and cost 13 million tokens.

The flawed version never went out. The mistake was caught before it could become a costly liability.

The write-up is available at gallon.me. Agents can now be trained to check their own work using this review method.

Gallon believes everything his agents tell him — not because agents are right or because agents are reliable. Agents get things wrong all the time. He believes them because everything he relies on has been thoroughly reviewed and validated first.

The belief is not in the agent. It is in the review.