The Review Wasn't Wrong. It Was a Review of Different Code.

2026-11-08

Agent A writes a change. Agent B reviews it and says it looks great. Agent A then edits one more file “real quick” and ships. Nobody lied. The review was fine. It was a review of different code.

I have been doing this for roughly thirty years, and I can report that “the thing we checked is not the thing we shipped” is the oldest bug in the business. We have simply found a way to do it faster and with more confidence. I wrote Uncle (github.com/unclehq/uncle) because I got tired of it.

The agent is not the authority

At Yahoo I built Yahoo Mail's anti-spam system. Billions of messages a month, six patents, and one durable lesson: a component does not get to declare its own output trustworthy because it produced it. Spammers taught me that. Apparently it also applies to the thing we now let write our code, review our code, and then tell us how it went.

Today's workflow: the agent implements, tests, reviews itself, and reports “done, all green.” Did the tests run? Did they finish? Did the reviewer look at the final code? Who knows, ask the agent. (It will say yes. It is very confident. It is always very confident.)

A second reviewing agent does not fix this. Adding reviewers to an unbound review is just adding more people to a meeting that has already ended.

The change is the unit of trust

A Git tree has a hash. That is a concrete thing to staple evidence to. So record that the review passed against a particular tree, that the tests passed against that same tree, that a human approved it, and then check, before publishing, that you are still holding that tree. Here is a real run from a demo project:

➜  uncle git:(main) uncle verify
UNCLE CHANGE VERIFICATION

verifying the attestation on refs/heads/main
✓ attestation canonical, in-toto Statement, schema v1
○ signature present (ssh); no trust root at .uncle/allowed_signers
✓ approved adversarial review b178…4e2e
✓ approved baseline report 8eb8…00d4
✓ approved change plan e552…0841
✓ approved change spec 7462…8531
✓ implementation artifact 9edd…fedc
✗ verification result is fail, not pass; 3 commands: 1 pass, 1 pre-existing, 1 regressed
✓ blocking findings dispositioned: 0 of 0
✗ required gate ADVERSARIAL_REVIEW has no human approval
  approved_by:  empty
  delegated_by: unattended
✓ head tree matches release artifact 9edd…fedc

The last green check is the one that matters here: the tree you are holding, 9edd…fedc, is the tree that was reviewed, so the evidence applies to this code. That is all it tells you. Two lines are red. The verification run failed (one command regressed against the baseline), and the adversarial-review gate was approved by the agent on nobody's behalf, because the run was unattended. Uncle prints both instead of rounding up. Also note the signature line: a signature is present, but with no trust root to check it against, it proves nothing about who signed. Verified, signed and a human looked at it are different claims, and the tool refuses to blur them.

Revolutionary, I know. Change the tree and the evidence stops applying. The only twist is that the workflow has to enforce it, because “please remember to re-run the review” in a prompt is a wish, not a control.

Same for the evidence itself. If the agent writes the JSON that says "tests": "pass", you have learned what the agent would like you to believe. The driver runs the checks, reads the exit codes, and writes the record. Three outcomes: pass, fail, unavailable. A test that never ran is not a pass. A reviewer that timed out did not approve. Missing evidence is not good news, whatever the dashboard colour.

Isn't this just branch protection?

Mostly, if everything happens inside a pull request. GitHub will dismiss stale approvals and run required checks on the head commit. Good. Now the parts it can't see:

The agent's review and verification happen locally, before a PR exists. GitHub is not in the room.
An LLM reviewer's thumbs-up is not a GitHub approval, so nothing dismisses it when the code changes.
The evidence travels with the change, so you can check it without trusting my CI.
It pins the plan and the requirements, not only the diff.
It records whether a human approved or an agent approved on a human's behalf. GitHub cannot tell these apart.

None of this is a new idea. SLSA and in-toto exist, and several newer tools attach provenance to builds and PRs. Uncle's narrow bit is binding an agent's internal stages (plan, implement, review, verify, approve) to one tree.

Does it work? Here is the actual output

And here is the failure case, running uncle verify on a PR in this site's own repo:

$ uncle verify 4
UNCLE CHANGE VERIFICATION

verifying the attestation on refs/heads/main
✓ attestation canonical, in-toto Statement, schema v1
○ no signature (authentication: none)
✓ approved adversarial review 85cc…c1a4
✓ approved project plan 713a…9bea
✓ approved requirements interpretation 17f9…6bda
✓ approved updated project plan 618a…2c6f
✓ implementation artifact 252a…1816
✓ independent review (unavailable, pass, 21 findings, 21 dispositions)
✓ verification pass
✓ audit pass
✓ blocking findings dispositioned: 6 of 6
✗ required gate ADVERSARIAL_REVIEW has no human approval
  approved_by:  empty
  delegated_by: unattended
✓ head tree matches release artifact 252a…1816

NOT HUMAN VERIFIED

Yes, that is my own PR, and yes, it merged anyway. The verifier didn't block it; it just declined to call it verified, which is exactly the job. A tool that only ever says “verified” is called a green checkmark, and we already have plenty of those. Note the unsigned line too: integrity, not authenticity.

What I'm defending against

In scope: an agent that is mistaken or careless: misreads a tool result, forgets a failure, edits after review. The driver records exit codes and tree identities itself, so this shows up as a mismatch.
Partly in scope: a hostile agent with unsandboxed shell access. If it can write where the driver writes, it can tamper with local state. You get detection when someone later checks evidence against the tree, plus signatures where a trust root exists. Unsigned means no protection here. Use a sandbox.
Out of scope: a compromised machine or verifier, and bugs every reviewer and test misses.

Verification is not correctness

A change can pass every stage and still be wrong. Reviewers miss things, tests are incomplete, specs are wrong, and two models can make the same mistake in perfect harmony. A valid attestation says the evidence applies to this artifact and the recorded process met the verifier's requirements. That is all. It is still more than “the agent said so.”

Uncle is young and I'd bet on it having holes. It's mostly Bash and Python, supports several agent runners, and not every change carries a signed attestation yet. I would much rather you break it than compliment it. A convincing counterexample beats a compliment every time.

Delegate the work, not the responsibility

I don't think the future is humans reading every line an agent writes. I also don't think it's trusting the agent because it sounds sure, or because a second agent nodded along. Let them do the work. Keep “what was checked, by whom, against what” somewhere an agent can't edit.

Agents are getting very good at writing software. Time we got equally good at proving what they did.

Uncle is open source and looking for contributors. The repository is at github.com/unclehq/uncle. If you're interested in coding agents, verification, developer tooling, or software supply-chain integrity, I'd welcome contributions, criticism, and attempts to break its assumptions.

ai-agents verification provenance uncle