Practice · 2024–now · AI Present
AI-generated test theater
Coverage theater with a chat box. Tests that mirror the bug and never catch it.
Why AI-generated test theater became a costly fad
AI test-generation theater failed because coverage without behavioral contracts is decoration. Generated tests stick when humans specify invariants and review assertions. The fad was green dashboards from mirrored implementation; the stuck practice is property tests and parity checks owned by people.
Cost of the fad
What AI-generated test theater cost
LLMs wrote tests that boosted coverage and asserted nothing. Green CI, false confidence — eval theater's cousin in the test suite.
Case studies
Patterns
Compare with
Related
Practice · 2024–now
AI review bottleneck
Copilot ships PRs faster than teams can review. Generation outran ownership and security review — the real 2026 invoice.
Practice · 2024–now
Vibe coding
Accept AI output on feel, skip the diff, hope tests exist. Fast demos; CVEs and nobody owns the middleware.
$ Ship-to-prod without reading the diff became a flex until incidents revealed nobody knew which prompt wrote the auth middleware. Review debt arrived in one weekend demo.
Practice · 2024–now
Eval theater
LLM evals as slideware — metrics that look scientific and never gate a deploy. The foil to real eval pipelines in CI.
$ Dashboards of vibe-check scores that never blocked a release. Green charts for leadership; prod still hallucinated.
Practice · 2021–now
AI pair programming
Coding agents that edit the repo — now table stakes. Generation outran review capacity; ownership is the bottleneck.