Writing a test is the first cost. Keeping its locators, data, and expectations aligned with the app is recurring work. Budget for that upkeep before expanding coverage, and use triage history to see where your suite spends the most effort.
Authoring and upkeep are different costs
Writing a test happens once. You understand the flow, you have the app in front of you, the selectors are fresh, and you close the PR. That cost is real but bounded.
Maintenance is unbounded. Every test you keep is a standing liability that has to survive every change to the application it covers, and a suite of 500 tests is 500 things that can break next Tuesday for reasons that have nothing to do with a bug. The more coverage you add, the more surface you own. That is the uncomfortable part of doing the right thing: adding coverage, the exact move everyone tells you to make, grows the recurring bill rather than shrinking it.
When upkeep has no owner, small failures can accumulate. Reruns and skipped checks may hide them until the suite becomes hard to trust. Track those exceptions and assign maintenance work before the backlog grows.
Three ways a suite rots
A suite rots from three separate forces, and they compound on each other.
App drift. The product changes and the test doesn't know. A button is renamed from "Save" to "Save changes," a two-step checkout becomes three steps, a settings page moves under a new menu. The behavior the test asserts is still correct; only the path to it moved. This is one legitimate kind of breakage. The app genuinely changed, and the test is now describing a version of the UI that no longer exists.
Selector decay. The test targets the DOM by structure or styling that was never meant to be a contract. CSS classes, nth-child positions, and auto-generated IDs shift every time someone refactors markup or a component library bumps a version — with no behavior change at all. This is the purest waste: a red test caused by a change that a user would never notice.
// Brittle: coupled to layout and generated class names
await page.locator('div.card > button.btn-primary.css-1x9f2a').click();
// Resilient: coupled to what the user actually sees
await page.getByRole('button', { name: 'Add to cart' }).click();The second locator survives a restyle. The first breaks the moment a designer touches the component, and then someone spends an afternoon rediscovering that the feature works fine.
Data drift. The test depends on state that doesn't hold still. It signs up casey@example.com, and on the second run that email already exists. It asserts "3 orders" against a shared staging database that now has four. It expects a coupon that expired. Data drift produces the worst failures because they're intermittent (green locally, red in CI, green on rerun), and intermittent failures train everyone to hit "retry" instead of reading the result.
A resilient test generates the state it needs instead of assuming it. Anywhere a test creates something, the value has to be unique per run, or the second run collides with the first:
When the user signs up with email "casey+{{unique}}@example.com"
And the user creates a project named "Project {{unique}}"
Count triage as well as repair
Add the three forces up over a test's life and the ledger inverts. Authoring happens once, in a burst, and then the maintenance bill comes due every sprint for as long as the test lives in the suite. For an illustrative budget, a test written in an hour might consume that hour again several times in triage, re-selection, and reruns, and triage costs more than the original authoring did, because the person doing it has to first prove the failure isn't a real bug before anyone will let them touch the test.
That last point is the hidden cost. Every red build forces a question: is this the app breaking, or the test breaking? Answering it takes a human, some context, and a few minutes of attention that always arrive at the wrong time. Multiply by a flaky suite and you get alert fatigue, and a suite quietly stops being an asset and becomes background noise. Once nobody trusts the results enough to read them, a genuine regression scrolls past in the same red wash as all the false alarms, and the suite has failed at the one job it existed for.
Testing is still worth it. The point is narrower than that: count the whole cost before you commit to coverage you can't maintain.
What regeneration changes
Traditional maintenance means patching: a test broke, so a person opens the file and edits the selector or the step that moved. You're hand-repairing an artifact. The economics only improve if the repair gets cheaper, or stops being necessary in the first place.
The shift is generating the executable test from readable intent instead of hand-writing it. In TestVibe, you describe behavior in Gherkin (the entrypoint, the action, and the visible outcome that proves it worked), and AI generates the Playwright code from that intent:
Scenario: User signs in with valid credentials
Given the user is on the sign-in page
When the user enters a valid email and password
And the user clicks Sign in
Then the dashboard should open
And the user name should be visible in the headerWhen the app drifts — the sign-in label changes, the page moves — you don't hunt for the broken selector. The intent above is still true, so you regenerate against the current app instead: labels, pages, or behavior changing since the last generation is a standard reason to regenerate. That re-derives the executable code from the app as it is now, so keep behavioral intent in the Gherkin and review it before regenerating. You can also save authored Playwright code explicitly in the Code tab; review concurrent-edit conflicts, and remember that regeneration can replace that code.
Two honest caveats keep this from being a silver bullet. First, generated code is not trusted blindly — it's verified by actually running it against your site in an isolated cloud session, and you review the result before relying on it. A failed run can still mean a real bug, missing data, or a test that needs improvement; the tool doesn't decide that for you. Second, regeneration doesn't repeal the three forces — it changes what a fix costs. Generated selectors can still become stale. Regeneration is a possible repair path, and its output still needs review. Data drift still needs unique values, but those are declared in intent, not scattered across code.
The other half is catching drift before it catches you. A scheduled automation, a nightly full-suite run against a stable environment, surfaces breakage on its own schedule, so you learn the login flow moved from a 2 a.m. report instead of a customer ticket.
Maintenance still needs an owner. Regeneration offers another repair path, while state, expectations, and generated output still need review. Track how long that path takes on your own suite before changing the maintenance budget. Get early access to TestVibe to evaluate it on one changing flow.