The testing conversation hasn't caught up to the production conversation.
AI made code generation fast. It didn't make verification fast. That's the bottleneck now. And the testing strategies most teams use were designed for a world where code volume was proportional to review capacity. That world is gone.
The old testing pyramid assumed human review
The testing pyramid was designed for a specific constraint: humans write code, humans review code, and tests catch what review misses.
Unit tests at the bottom. Integration tests in the middle. End-to-end tests on top.
That shape assumes most bugs are local, some are interaction-level, and a few are system-level.
That assumption held when humans wrote code. Humans make local mistakes: off-by-ones, missing null checks, wrong variable names. Unit tests catch those. Humans also make integration mistakes: wrong API contracts, mismatched assumptions. Integration tests catch those.
The pyramid worked because the distribution of human mistakes matched the shape of the pyramid.
AI does not make the same mistakes. The code is syntactically clean. The unit tests pass. The functions are well-organized.
The mistakes moved. They are wrong consistency models, incorrect failure handling, and architectural decisions that look right but do not fit the system. These are not unit-testable. They are barely integration-testable in the traditional sense. They are system-level mistakes that only show up under real conditions.
The pyramid is upside down for AI-generated code. The easy layer (unit tests) catches almost nothing. The hard layer (system behavior under real conditions) is where the bugs live. And most teams are still investing in the base of the pyramid.
What testing needs to become
Testing in the AI era has to shift from "does the code work" to "does the code work in the system." That's a different question, and it requires different tools.
Property-based testing over example-based testing. Example-based tests verify specific cases: "given input X, expect output Y." They are easy to generate and easy to pass. They are also the tests AI is best at writing, which means they provide the least signal. The model generates the code and the tests from the same understanding, so the tests encode the same assumptions as the code. They cannot catch what the code gets wrong if they get it wrong the same way.
Property-based tests verify invariants: "for any valid input, the output satisfies this property." They do not encode a single expected answer. They encode a constraint. The test framework generates inputs and checks the constraint holds. That catches edge cases the author did not think of, which is exactly what AI-generated code needs.
Integration testing against real dependencies. The old advice was to mock external dependencies in integration tests. That advice is dangerous with AI-generated code. The model makes assumptions about how the dependency behaves: its consistency model, failure modes, ordering guarantees. Mocks encode your assumptions about the dependency, not the dependency's actual behavior. If the model's assumptions are wrong, the mock confirms them, and the test passes.
Testing against real dependencies catches the assumption mismatches. A real database. A real queue. A real cache. Slower, more expensive, and often the only way to catch the bugs AI-generated code actually produces.
Testing failure paths alongside happy paths. AI is good at generating code that handles the happy path. It is also good at generating code that looks like it handles failure paths: try/catch blocks, error logging, fallback returns.
Looking like error handling and being error handling are different things. The model generates the shape of error handling without necessarily understanding the failure mode.
Testing failure paths means deliberately breaking dependencies and checking that the system degrades correctly. Kill the database. Add latency. Return errors from the queue. These are the tests that matter, because distributed systems are mostly apologies, and the apologies are where AI-generated code is weakest.
The specification problem
You can't prove the bug isn't there. Testing can only prove the presence of bugs, not their absence. That was always true. It matters more now.
The old testing model assumed the specification was clear: the requirements document, the ticket, the acceptance criteria. Tests verified the code against the spec. If the spec was right and the tests passed, you had reasonable confidence.
AI complicates this because the prompt is doing specification work without behaving like a real specification. It describes intent, then the model fills in gaps with assumptions. Those assumptions become part of what the code implements. Testing against the prompt can show whether the code matches the description, but it will not expose every hidden assumption.
Testing strategy now has to include specification verification. Does the code match the spec? Is the spec correct? Does the code make assumptions the spec does not mention?
That work overlaps with testing, but it is closer to review. It is the review that moved up the stack, and it is where much of the leverage now lives.
When code volume rises faster than review capacity, the testing strategy has to absorb more of the uncertainty. The useful tests are the ones that exercise system behavior, dependency behavior, and failure behavior, because that is where generated code tends to be weakest.
The one-line version
AI-era testing needs to exercise the system conditions generated code is most likely to misunderstand.