If AI writes your code, AI has to verify it — and not the AI that wrote it

Code written by AI needs a check that does three things: it checks against intent, it comes from a different party than the author, and it hands over evidence. Ask the agent that wrote the code to test it in the same session, and you get none of the three.

For decades the person who wrote the code also knew what it was supposed to do. When they wrote a test, the test carried that knowledge. Now a model writes the code. It answers every open question in the ticket by writing code, and whatever it chose becomes the behavior. Nobody wrote the expected result down.

Human reviewers do not close that gap. There are not enough of them. In one company that measured its AI rollout, pull requests grew about twice as fast as the number of reviewers. Fewer changes got a human look at all, and only about one in five got a written comment.¹

So the check has to be automated, and AI is the only thing fast enough to keep up. The question is which AI does it, and what it is given.

The model is already open, and asking it for tests costs cents. Here is what those tests do.

Say the ticket reads: "Round totals to the cent, in the customer's favor." The model writes the function and rounds $10.005 up to $10.01. Then you ask it for tests. It reads its own function and writes: for $10.005, expect $10.01. The test passes. It will pass forever. And when somebody later fixes the rounding, that test turns red and the fix looks like a regression. A green test with a plausible name now protects the bug.

This is not a corner case. Tests written from buggy code find about half as many bugs as tests written from the fixed code.²

About four in five tests that coding agents add to a pull request run the code but barely check what it returns. A pull request full of test files can verify almost nothing.³

The usual loop is: write code, write tests, fix until everything passes. When the agent edits the code and its own tests until they agree, the code drifts toward the tests, not toward the task. More of the "fixed" changes then fail the real checks the agent never saw.⁴

Code and tests from the same source agree with each other. That proves they are consistent. It does not prove they are right.

The last defense in the shortcut is to ask the agent to review what it just wrote. That does not work either. A model reviewing a patch it produced a moment ago, in the same conversation, approves more bad patches than when the same patch is shown to it as someone else's work. Nobody is fooling it. It is lenient with itself. If a hidden instruction in the inputs made it into the code, the self-review is far more likely to approve that too.⁵

So in the write, test, review loop, one model writes the buggy code, writes the test that confirms the bug, then reviews both and passes them. Every step is green. Nothing in the loop knows what the code was supposed to do.

A real verification layer needs three properties: it checks against intent, it comes from a different party than the author, and it hands over evidence. Each one answers a failure above.

Give the checker a description of what the code should do: the ticket, the API contract, the docs, the behavior the system already ships. It writes tests from that, not from the implementation. In the billing example, the checker never reads the rounding function. It reads the ticket, writes "for $10.005 the customer pays $10.00," runs it, and the test fails.

This works. Tests written from a description of the task catch far more bugs than tests written from buggy code, and when the code cannot be trusted, tests written from the description alone do best of all.²

Often nobody wrote that description down. Then the checker has to reconstruct it from the ticket, the docs, the contract, and what the system already does, and then compare the code against it. That reconstruction is not a preliminary to verification. It is most of it.

"Different party" has a precise meaning here. The bias comes from the conversation, not from a label. Telling a model "you wrote this" changes little. Having the patch in its own conversation as its own output changes a lot.⁵

So the minimum is a clean context: a fresh session that receives the code as input and did not write it. A different model is a stronger separation. The same agent that wrote the code, asked in the same session to check it, is not a separation at all.

A separate checker can still share the author's misunderstanding of the ticket. That is why it also needs the intent written down, and why it needs the third property.

A checker that says "this looks wrong" asks a busy reviewer to investigate. A checker that hands over a failing test, runnable against the current code with the input that triggers it, has done the investigation. The reviewer runs it and sees the failure.

This runs in production. At Meta, a system generates tests built to fail on a wrong change and shows engineers only the failures that survive automated filtering. It caught real bugs, and the authors judged half of them serious enough to have broken production.⁶

Almost none of the generated tests reach a person. That is how it should be. Generating a test is cheap and an engineer's attention is not, so the filtering is what makes the system usable. Without it, engineers would have looked at five times as many failures to find the same bugs.⁶

There are two kinds of generated test. A hardening test passes the moment it is written, because it was written from the code as it stands; it guards against future regressions. A catching test is built to fail if the change is wrong. Only the second kind checks the change in front of you. Both raise coverage by the same amount, so coverage cannot tell you which one you have.⁶

Nobody has tested this whole process in a controlled study. Meta's numbers come from Meta. The other studies each test one piece: tests from intent, review in a clean context, filtering before escalation. So this is a recommendation built from parts, not a proven method.

The bug it is built for is the one that is almost right. A linter does not catch it, because the code breaks no rule. A test written from the code does not catch it, because the test copies the wrong behavior. Only a check that knows what the code should do catches it. That check cannot be the model that wrote the code, working in the same session.

Many tools now claim to be this check. They differ a lot. Some only read the diff. Some run the code. Some run the whole system. When you look at one, ask one question: does it give you a failing test you can run, or only an opinion?