Full coverage, zero protection: the AI testing trap

During our account-deletion work, a coding agent produced an implementation and passing tests. Further testing exposed a gap: a failed subscription lookup was being treated as “no active subscription”. The application allowed deletion when it could not establish whether the account was eligible.

We agreed that deletion should be blocked in that situation and added a regression test. Reviewing its assertions raised another concern. Checking that the endpoint returned an error would be insufficient if the account had already been deleted.

That distinction matters when using AI in software testing. A generated test can raise coverage while accepting the wrong behaviour, so we need to examine which incorrect behaviours could still satisfy its assertions.

Coverage tells us where the test went

Statement coverage records which executable statements ran. Branch coverage adds information about which control-flow transitions were exercised. These measurements help identify code that tests have not reached. Coverage.py’s documentation illustrates how a function can have complete statement coverage while still containing an untested branch.

However, executing a statement does not establish that its effect was correct.

A test can execute the deletion, receive an error response, and pass because its only assertion checks the response. The coverage report can accurately show that the deletion line ran. The missing piece is an assertion that rejects the resulting state.

This problem applies to human-written tests too. With AI, we need to pay particular attention to what we ask the agent to optimise. “Increase coverage” gives it a measurable target, but leaves the expected behaviour underspecified.

A passing test can accept a broken implementation

Let us use a small Python example to isolate the problem. This is an illustrative model of the assertion gap, rather than our production implementation.

Suppose the agreed rule is that deletion requires confirmed eligibility. Unknown or ineligible accounts must remain unchanged.

The following implementation violates that rule:

def delete_account(account, eligibility):
    account["deleted"] = True

    if eligibility != "eligible":
        return 409

    return 204

Deletion happens before the eligibility check. Yet this test passes:

def test_unknown_eligibility_returns_conflict():
    account = {"deleted": False}

    status = delete_account(account, "unknown")

    assert status == 409

The function returned the expected status. It also deleted the account.

Adding an eligible-account test could exercise the remaining return statement. We could then execute every statement in this function while still failing to detect its central defect, provided we continued checking only response codes.

The test needs to express the preservation requirement:

def test_unknown_eligibility_preserves_account():
    account = {"deleted": False}

    status = delete_account(account, "unknown")

    assert status == 409
    assert account["deleted"] is False

The additional assertion catches the premature deletion. Moving the eligibility check before the state change addresses this simplified failure.

In an application test, the equivalent check should inspect the relevant persisted state. Checking an unchanged in-memory object would be insufficient if the endpoint updated the database through another instance or query.

Give AI the rule independently of the code

If we provide an implementation and ask an agent to write tests, the implementation becomes one source from which it infers expected behaviour.

That can be useful for understanding interfaces and constructing fixtures. However, the implementation may contain the very misunderstanding we want the test to expose.

For account deletion, “return an error when eligibility is unknown” leaves room for the broken example above. A more complete requirement is:

When eligibility cannot be established, reject deletion and preserve the account.

We can give the agent that requirement alongside the code and ask it to identify the observations needed to verify it.

A useful prompt is:

Generate tests for these reviewed acceptance criteria. For each test, explain which incorrect behaviour would make it fail. Include the relevant resulting state, not only the response. Flag any expected behaviour that the criteria leave unresolved.

This gives the agent a clearer task. It also makes ambiguity visible before it becomes an assertion.

The engineer still needs to review those expectations. An agent may correctly translate an incorrect business rule into executable tests. Passing them would establish agreement with that rule, while the product decision remained wrong.

Check whether the test can reject the defect

Once a regression test exists, I want evidence that it detects the failure it was written for.

In our account-deletion work, we checked that the regression test failed against the broken implementation and passed after the correction. The reason for failure mattered. A missing fixture or an unrelated exception would not demonstrate that the assertion detected the eligibility problem.

For a new test without a historical defect, we can make a small, deliberate change in a local working copy. In this example, move deletion ahead of the eligibility check and rerun the test. If it still passes, inspect what it observes.

We should also test the legitimate success case. An implementation that rejects every deletion request could satisfy all the rejection tests while making the feature unusable.

These checks give us evidence about particular behaviours. They do not prove that every possible defect is detectable. A concurrency issue, for example, may require a test that controls the sequence of reads and writes across competing operations.

Preserve the assertion when fixing the implementation

A failing test gives an agent feedback, but “make the suite green” leaves an important decision open: whether to change the implementation or the expectation.

Sometimes the test is wrong. However, changing a reviewed assertion should require an explanation tied to the intended behaviour.

For a confirmed regression, I would ask the agent to preserve the agreed expectation, propose the implementation fix, and return the test result with the diff. If it believes the assertion needs to change, that disagreement should come back for review.

This keeps the feedback loop connected to the product requirement. Otherwise, the agent can remove the signal that exposed the problem.

Review one behaviour before generating more tests

For a lean engineering team, a practical starting point is one important failure path in an upcoming change.

Write down what must happen and what must remain unchanged. Ask AI to generate the test, then inspect whether a plausible incorrect implementation could still pass. Check the failure against a broken version and retain a legitimate success case.

Coverage remains useful for finding code the suite has not exercised. Alongside it, we need evidence that the tests can distinguish acceptable behaviour from the failures we care about.

The engineering judgement is in defining that distinction. AI can help turn it into checks that run on every subsequent change.