Most test failures die in a triage tab: someone reads the error, files a ticket, and the fix waits for whoever owns that file to find a spare afternoon. The failures worth chasing are the ones that point to a real defect in the app, not a flaky selector. TestVibe's Debug with AI flow shortens the distance between one of those failures and a change your team can merge: it diagnoses the failure against the Playwright trace and, when your source code is connected, can open a pull request that proposes the fix.
Where the flow starts: a failure that means something
Every fix PR begins at a failed run. In TestVibe, you open Runs, select the failed run, and open the Features tab to find the failed scenario row. Expanding it gives you the failure panel: the layer where you decide what kind of failure this is before touching any code.
Read it top to bottom:
| Area | What it tells you |
|---|---|
| Error | The Playwright error text, assertion output, and failure source. |
| Location | The spec file and configuration where the failure occurred. |
| Final screenshot | The browser state at the end of the failed scenario. |
| Video | The session leading up to the failure. |
The panel is also where you form a hypothesis, not a verdict. A screenshot showing the expected page but a failed assertion usually points to a stale test expectation. A screenshot showing the wrong page, or an error naming a missing selector, can mean the UI changed — but it can just as easily mean navigation, login, or environment setup drifted. Deciding which one you're looking at is exactly the judgment call the next step is built to help with, not replace.
Debug with AI: diagnosis on top of the trace
On a failed result, the Debug with AI button opens the full action-by-action Playwright trace with an assistant docked beside it. The assistant diagnoses the failure automatically: it reads the trace, the error, and the surrounding evidence, rather than waiting for you to paste a stack trace into a chat box.
Because the assistant is project-aware, it already knows the feature the scenario belongs to, the environment it ran against, and, if you've connected one, your application's source. From there it can do one of two things: fix the test, or open a source-code pull request against your repository.
Connect the repository that makes fix PRs possible
The PR path depends on one optional setup step: connecting your application's code under Settings → App Repository. This links the repository that contains your app's own code, on GitHub, GitLab, or Azure DevOps, one active source at a time. TestVibe reads it to ground the AI while generating and debugging tests, and to map coverage of your app's areas.
Two things are worth being precise about:
- The App Repository connection is read-only for grounding. TestVibe does not write to it, and repository permissions are requested only when you connect, never at sign-in.
- A fix PR is a proposal, not a push. The assistant drafts a change and opens a pull request you review and merge yourself. As the assistant's own chat footer puts it: it won't push changes without your confirmation.
This is a different corpus from Settings → Test Files Sync, which two-way mirrors your project's own test files. App Repository is your application's code; Test Files Sync is the tests. Debug with AI reads the former to understand why the app behaved the way the trace shows.
From diagnosis to a drafted change
When the failure is app-side, the assistant uses the connected source to locate the relevant code and drafts the change into a pull request. Say a run fails because a checkout button lost its accessible label after a refactor, and the test can no longer find it:
// The scenario that failed — the locator no longer resolves
await page.getByRole('button', { name: 'Checkout' }).click();
// Error: locator resolved to 0 elementsThe trace shows the button is on the page but rendered without its label. That's an app regression, not a test problem, so the fix belongs in the app: the drafted PR touches the component, not the spec.
- <button className="btn-primary" onClick={checkout}>
+ <button className="btn-primary" aria-label="Checkout" onClick={checkout}>
<CartIcon />
</button>The example is illustrative, but the shape is the honest one: the assistant proposes a small, reviewable diff scoped to what the trace implicates, and hands it to you as a PR. You read the diff the same way you'd read a teammate's: nothing lands on your default branch without your review.
Verify the fix by re-running it
A drafted PR is a hypothesis until a test proves it. Once you've merged the change, or deployed it to your test environment, open the failed result and choose Re-run. The run composer opens pre-scoped to the same test selection; narrow it to just the scenario that failed, then start the run to confirm it now passes.
TestVibe's whole model is that AI output is trusted only after it actually runs green: a generated test only reaches generated status after TestVibe replays the whole assembled spec end-to-end and it passes, and a fix is no exception. If the re-run passes, the loop closes with evidence instead of optimism. If it still fails, the fresh trace tells you what the drafted change missed, and you go around again from the same Debug with AI button.
Two entry points, one idea
TestVibe exposes this from two buttons, and it helps to know which one you're using:
- Debug with AI opens the trace viewer with the docked assistant that diagnoses automatically — the surface described above, and the one that can open a source-code pull request.
- Fix with assistant opens a guided flow that reproduces the failure and proposes a fix, for when you want a more step-by-step path.
Both start from a failed result, and both keep you in the loop for the decision that matters: whether the change is right.
The value here isn't "AI fixes your bugs." It's that the diagnosis, the source context, and the drafted change arrive together, attached to the exact trace that caught the regression, so whoever reviews the PR has everything in one place. You still own the merge. To try the flow end to end, connect a repository and let a failing run walk you from trace to PR. Get early access, or read the Assistant guide first.