← All posts
Concepts

End-to-end tests are agent infrastructure now

When several coding agents change the same app, reviewers need repeatable evidence of the behaviors those changes affect. An end-to-end suite can supply part of that evidence: run the relevant journeys, inspect failures, and keep the expectations under review. The suite’s coverage and execution time determine how useful that feedback is.

A reviewer may remember a redirect, a fragile export order, or an important billing edge case. Record those expectations in specifications and checks so a new teammate or coding agent can use them too.

The checks around the model do the real work

The engineers running agent fleets at AI labs keep landing on the same unglamorous observation. The leverage doesn't come from a smarter model. It comes from lint rules, type checks, CI, and end-to-end suites: the automated verification that turns "the agent produced a plausible diff" into "the agent produced a diff that passed the configured checks." Anthropic's own 2026 Agentic Coding Trends Report describes exactly this pattern, where automated feedback loops are what let agentic coding scale past a single supervised session.

Say the logic out loud and it's obvious. An agent's output is only as trustworthy as the checks it has to pass. Lint, type checks and an e2e suite can catch the specific violations they cover. They do not eliminate all style, runtime or login regressions, and passing output still needs review. Add checks for important risks, and assess whether they catch useful failures without overwhelming the review workflow.

Manual browser exploration and reusable automated checks do different work. Exploration can reveal an unexpected path; an automated scenario can repeat a known expectation after later edits. Use both alongside code review, rather than treating either as a complete oracle.

Knowledge in a head blocks agents and new hires alike

There's a second, older problem that agents make suddenly acute. Every codebase carries a load of domain knowledge that lives in exactly one place: someone's head. You can't delete that account or the webhook re-fires. The export has to run before the nightly job or the numbers are stale. Don't touch the currency formatter, it's load-bearing in three countries. That knowledge is a bottleneck for every new engineer who has to absorb it by breaking things and getting corrected in review.

Agents hit the same wall, except they hit it constantly and at speed. An agent doesn't know the export has to run first unless something tells it. If that constraint lives only in a senior engineer's memory, the agent will cheerfully violate it, the diff will look clean, and the tests, assuming they don't cover the case, will stay green. Whatever you never encode is exactly what the agents will eventually trample.

Encoded as a test, that same constraint is checked whenever the maintained test actually runs, without anyone remembering to mention it. This is the real argument for shifting quality left. Catching bugs early is cheaper, sure, but that's the small part. The bigger part is that a behavior expressed as a test becomes a fact the whole system can rely on, instead of a rumor one person happens to know. The suite is where your team's hard-won domain knowledge stops being tribal and starts being executable.

The suite can't be the new bottleneck

Here's the trap. If e2e tests are now the infrastructure that lets you run N agents in parallel, then the cost of producing and maintaining that suite becomes the thing that caps N. And traditional e2e authoring is brutally expensive: brittle selectors, flaky waits, a maintenance tax that compounds with every feature you add and every refactor that quietly moves a button somewhere the old locator can't find it. You'd be swapping one bottleneck (humans reviewing every agent diff) for another (humans hand-writing and repairing the tests those diffs have to pass).

The way out is to make the suite itself cheap to produce and cheap to keep green, which means the tests can't be a pile of hand-tuned selectors only one person understands. When a test is written in plain language and generated against the running app, an agent can extend the suite the same way it extends the code: describe the new behavior, let generation produce and verify the Playwright, and the constraint is now encoded. The suite grows with the codebase instead of lagging behind it.

Two properties matter more than anything else here, because agents will lean on this suite unsupervised:

  • Every test has actually run before it counts. A generated test that was never executed is a guess, and a guess that agents treat as ground truth is worse than no test at all. Verification-by-execution is the whole game, and it's why we're blunt that you should never trust a test just because it looks right, whether a human or a model wrote it.
  • The suite runs in CI on every change, agent-authored or not. The check only multiplies output if it's wired into the loop. A suite that runs when someone remembers to run it is back to being a rumor. Put it in the CI/CD pipeline so every agent's diff has to clear the same bar automatically, and gate the merge on it.

What this changes about how you invest

If you buy the framing, a few priorities reorder themselves. Coverage of your critical user journeys stops being a QA metric and becomes a capacity metric: it can affect confidence in parallel work, but does not establish a safe agent count. Time spent encoding domain constraints as tests stops competing with feature work and starts enabling it, because that encoding is what keeps the agents from re-breaking the same thing next Tuesday. And the maintenance cost of the suite climbs to the top of your risk register, because a suite that's expensive to keep green will quietly cap your throughput no matter how many agents you spin up. Getting the strategy right for 2026 has less to do with test count than with which behaviors are encoded and how cheaply the encoding stays true.

For years the only real reason to write e2e tests was defensive: don't be the one who ships the broken checkout. That reason still holds. It's just no longer the biggest one. The suite is now what lets a fleet of agents move fast without breaking things behind your back, and the teams that pull ahead get there on the strength of what their agents can check against, not the size of the model they picked, and on keeping that suite cheap enough to grow alongside the code. Point three agents at a repo with a thin suite and you'll spend your week reviewing diffs. Point them at a repo where the behavior is encoded and running in CI, and the suite reviews the diffs for you.

Want a suite your agents can extend and rely on — plain-language tests, verified by running against your real app, wired into CI? Get early access, or read how generation works.

Coming soon

Turn a described flow into a reviewable test.

Describe a flow in plain English. Generate a Playwright test, run it on your schedule, and review proposed fixes when your UI changes. Join the waitlist for early access.

  • Plain English in, real Playwright out
  • Runs in your CI, on your schedule
  • No credit card, unsubscribe anytime