AI-Generated Code Can Pass Tests and Still Be Wrong
An AI assistant writes a function. You ask it for tests. It generates a dozen, runs them, fixes a small issue, and reports that everything passes.
That's useful progress. It still leaves a question unanswered: did the tests check what the application was supposed to do?
An implementation and its tests can agree perfectly while sharing the same mistake. This is possible with human-written code too. Fast generation just makes it easier to produce a convincing amount of evidence around an unexamined assumption.
A small mistake with a green test suite
Imagine a reservation expires at a particular instant. The product rule is:
A reservation is valid only while the current time is strictly before its expiration time.
An assistant produces this Python function:
def is_valid(now, expires_at):
return now <= expires_at
Then it writes these tests:
def test_before_expiration():
assert is_valid(9, 10)
def test_after_expiration():
assert not is_valid(11, 10)
Both tests pass. The boundary is still wrong.
The integers here are a deliberately simplified model of time. You don't need a production clock to expose the missing case:
def test_exactly_at_expiration():
assert not is_valid(10, 10)
That test comes from the requirement, not from copying the function's behavior. It fails against the original implementation. Changing <= to < makes the function match the rule.
The important improvement was an independent statement of expected behavior.
Write acceptance examples before implementation
Before asking an assistant to write code, I would give it examples like these:
Expiration: 10
Now: 9 -> valid
Now: 10 -> expired
Now: 11 -> expired
For a real application, the requirements must also define how time is represented, where the current time comes from, and what happens with invalid input. Those decisions should not emerge accidentally from whatever code the assistant generates first.
My AI workflow article emphasizes giving the assistant context. Tests need the same context, especially the exceptions and boundaries that are easy to leave out of a short prompt.
Ask for missing behavior, not more tests
“Write more tests” can produce five variations of the happy path.
A more useful request is:
Here is the requirement and the current test suite.
Identify behavior the tests do not distinguish.
For each gap, give an incorrect implementation that would still pass.
Then propose the smallest test that rejects that implementation.
That framing encourages the assistant to challenge the evidence. In the reservation example, both < and <= satisfy the original tests. Finding a case that distinguishes them is more useful than adding ten more timestamps far from the boundary.
You can make the same check manually: change the comparison, remove the ownership check, or return a constant in a temporary local experiment. If the relevant tests remain green, investigate what they actually assert. Revert the deliberate defect immediately after the experiment.
Mocking can erase the behavior you needed to check
Suppose a repository method is mocked to return an order. Your service test can verify how that result is handled. It cannot tell you whether the real query filters by the correct account.
Keep the unit test, but add a focused integration case: create records for two accounts and verify that one account cannot retrieve the other's record through the real data-access path.
That check is based on an application boundary. It does not need to duplicate every line of the implementation.
Properties give you another kind of evidence
For suitable functions, describe relationships that should hold across many inputs. A reservation valid at one instant should remain valid at an earlier instant, assuming the same expiration and valid time representation.
Property-based testing tools such as Hypothesis generate inputs from strategies and search for counterexamples to the properties you specify. They can reduce a failing example to something easier to inspect. They still rely on you choosing a meaningful property; a weak invariant can pass for a broken implementation. Hypothesis documentation
Pair broad properties with explicit examples like the expiration boundary. They answer different questions.
Review the claim behind the green check
For an AI-assisted change, I would want to see the requirement, the important boundary cases, the code change, and the tests that were actually executed. If integration tests couldn't run, that should be stated plainly.
The goal isn't to distrust every generated line. It's to keep the evidence independent enough to catch a shared misunderstanding.
A passing suite tells you that the code satisfied its assertions under the tested conditions. Good engineering is deciding whether those assertions cover the behavior that matters.
# Comments (0)
No comments yet. Be the first.