← All posts
Concepts

Vibe coding won. Now it needs a safety net.

AI-assisted development can increase the volume of code to review. Assess your team’s actual usage and failure patterns rather than treating secondary statistics as universal evidence.

Use your team’s observed failure patterns to choose review and verification effort. Behavioral tests, code review and security checks provide different evidence; keep all three in the workflow.

The trust gap is earned, not irrational

Veracode’s July 2025 report summary reports that 45% of code samples in its evaluation failed security tests. This applies to its evaluated models and tasks, not all AI code or your current project. Check security explicitly alongside functional behavior.

Higher code volume can increase review pressure. Risk-based review, security checks and behavioral tests address different failure classes.

Karpathy's reframe: keep the leverage, add the scrutiny

Karpathy’s Sequoia Ascent 2026 summary emphasizes specifications, verifiable feedback, security and human understanding. That supports adding a test loop while retaining accountable review.

That last one carries the weight. Scaling scrutiny doesn't mean reading more code; it means verifying more behavior. A feedback loop built on a verifiable signal doesn't care whether a human or a model wrote the line that broke. It tells you the same thing every time, mechanically: whether the app still does what it's supposed to do. That signal keeps working when the diff is 40 lines or 4,000, whether you read it or not.

For end-to-end products, the verifiable signal has a name that predates the AI era: a regression suite. You describe what the app must do once, in plain language, and a test holds that line every time the code changes afterward. The behavior is pinned. The implementation underneath is free to churn.

Why behavior verification is the right layer

A diff review can tell you the code looks correct. It cannot tell you the app still works, and that second thing is the only one your users ever experience.

Consider a checkout flow. An agent refactors the payment handler across a dozen files. You can read all twelve diffs and convince yourself they look reasonable — review can still miss an integrated behavior failure. Or you can have one test that says: a user with items in their cart can enter card details, submit, and land on an order-confirmation page. If the refactor broke checkout, the test goes red regardless of how clean the diffs looked. If it didn't, you're free to keep moving. The behavioral assertion is doing work that twelve careful reads can't guarantee.

This is exactly the trade the economics of test maintenance push you toward. A suite anchored to observable behavior, meaning what the user sees rather than how the code happens to be structured, survives the constant internal churn that AI-assisted development throws at it. Anchor those same assertions to implementation details instead and the suite turns into noise. Where you point the assertions decides which of those you get, and it's the same line that separates a test catching real regressions from one that just flakes and trains you to hit re-run.

The description is the artifact you actually maintain

This has to stay practical, especially for the growing number of non-traditional builders shipping real products with agents. If the verification layer made you hand-write Playwright for every flow, you'd have just moved the bottleneck around: vibe coding the app, then painstakingly hand-coding the tests that check it.

The move that keeps the leverage intact is to describe the behavior in plain language and let generation produce the test. You write: "A signed-in user can add a product to the cart and complete checkout with a saved card." A system generates the Playwright, runs it against your real URL in an isolated session, and marks generated status only after completed verification; drafts can exist earlier. That last condition is the non-negotiable part. A test that references a button that isn't there gets caught and rewritten before generated status is reported, not merged on the strength of looking plausible.

That inverts the artifact you maintain. Instead of babysitting brittle selector code, you maintain a plain-language description of intended behavior, the thing you actually understand and the thing that should only change when the product itself is meant to change. Writing that description well becomes the skill, the same way Karpathy puts spec design at the center of agentic engineering. Human review covers intent and the executable checks, where a human belongs, and the machine takes over the tedious but essential job of running the thing until it genuinely passes.

None of this requires slowing down or trusting your agents less than you already do. The point is to aim that distrust somewhere useful. You keep vibing on the implementation, and a behavioral suite you described once holds the line underneath, going red the moment the app stops doing what you said it must. That's the safety net that lets you keep the speed while retaining review and security verification.

The takeaway

Vibe coding won, and reading more diffs won't close the trust gap it opened, because diff review is the exact bottleneck the tools were adopted to remove. Add behavioral scrutiny alongside code and security review: describe what the app must do once, then let a regression suite that has to pass a real run enforce it on every change that follows. Describe the flow, watch it fail the second a button goes missing, watch it stay green while the code underneath churns.

Want to see it in practice? Describe your first flow in plain language and watch it become a verified test in Getting started with your first test, or get early access to try it against your own app.

Coming soon

Turn a described flow into a reviewable test.

Describe a flow in plain English. Generate a Playwright test, run it on your schedule, and review proposed fixes when your UI changes. Join the waitlist for early access.

  • Plain English in, real Playwright out
  • Runs in your CI, on your schedule
  • No credit card, unsubscribe anytime