The whole thing in five words

TestVibe words you will see

Project

One web app under test, identified by a Base URL. It owns your test features, runs, settings and variables.

Test feature

What you write: a .feature file in Gherkin, with Feature, Background, Scenario and Given/When/Then steps. Each Scenario becomes a Playwright test(), each step a test.step(), and Background becomes setup that runs before every scenario — which is why results read as scenarios and steps.

Generation

Turning a test feature into a runnable Playwright spec. Costs credits, takes minutes.

Run

One execution of generated tests against a chosen environment. It produces artifacts: traces, screenshots, videos, logs, reports.

Workspace

The organisation that owns the projects, the members and the credits.

Signing in

• Email and password. Enter your Email and Password, then press Sign in. Create an account switches the card to sign-up mode; passwords need at least 8 characters including a letter and a number. Forgot password? leads to Send reset link.
• Single sign-on. You may also see Continue with GitHub, Continue with Google, or your company's own button. Providers that are not configured show no button at all.

Creating the project

1. On a brand-new account a dialog titled Add your app opens by itself. Otherwise press New project.
2. Fill in Base URL — the site TestVibe generates and runs tests against, for example https://app.example.com. You should see Project name fill in from the host as you type. The name is only a label; edit it or leave it.
3. Optionally pick a Target framework chip: React, Angular, Next.js, Vue, Wisej.NET or Other. TestVibe auto-detects on the first visit, so blank is fine. But if your app is built with a widget framework that renders anonymous <div>s instead of real buttons, setting this is the highest-value thing you can do before generating.
4. Press Add app (or Create project). You should land on the project dashboard.

You can change the Base URL later in Settings → Overview.

The left nav

Your map for the rest of this page. You will spend this hour in the first two.

Choosing the right URL

• Include the scheme. https://app.example.com, not app.example.com.
• Use test or staging, not production. The tests really click the buttons. If a scenario creates an order, an order gets created.
• It must be reachable from where the tests execute — a cloud browser, not your laptop. A VPN-only name or localhost fails with a connection error. For a private app, TestVibe can reach it without exposing it: a runner you host inside your network, or an outbound-only tunnel. Both live in Settings; ask your administrator which you have.
• Point at a stable starting page — the home page, or the login page. Every test begins at whatever that URL opens, so a deep link quietly limits everything.
• Never put credentials in the URL. Put them in Settings → Variables & Secrets: a variable for the username, a secret for the password. Secrets are write-only. You reference them from test text as {{var:NAME}} and {{secret:NAME}}.

To run the same tests against staging and a preview build later, add those under Settings → Environments and pick one per run. You do not need a second project.

Opening the dialog

1. Go to Test Features and press New test feature. On a brand-new project it opens by itself. You should see five ways to start:
   • Blank Gherkin file — a starter scenario you fill in.
   • Describe it — AI drafts it — plain English in, Gherkin out. Start here.
   • Record in browser — click through your site and let TestVibe draft from the recording.
   • Import from issue — needs Jira or GitHub connected in Settings → Connectors.
   • Import from document — PDF, Word, Excel, CSV, text or Markdown.
2. Choose Describe it — AI drafts it. The dialog retitles to Describe your test and gives you a text box. You can also paste, drop or Attach image a screenshot or mockup.
3. Press Draft with AI. You should land in the Gherkin editor with a draft in it.

Six rules for a good description

• Start where the app starts. Home page, or login page. Do not open on a deep screen — say how you get there.
• Name real things. "the Add to cart button", not "the relevant control".
• State the exact values entered. Never "valid credentials".
• End with something observable. A test with no assertion cannot fail, which makes it worse than no test.
• No URLs, hosts or localhost. The runner is already pointed at your site; a literal address breaks against another environment.
• One behaviour per test feature. Small features generate faster and fail more legibly.

One example, before and after

No data (which account?), no observable outcome ("works properly" cannot fail), and no boundary (does login include forgotten-password? session expiry?). When it fails you will not be able to tell whether the test or the app was wrong.

The repair is mechanical — add the entry point, the data and the assertion:

Names the entry point, names the controls, quotes the literal on-screen text, and asserts two independent things. Negative paths are worth as much as happy ones and generate fastest: error messages, empty states, validation, "no results found".

Check the draft before you generate

Reading the draft is far cheaper than reading a failed generation. Three checks:

• Does the first step start at the landing page? If it begins inside the app, fix it here.
• Are credentials tokens, not literals? You should see {{var:...}} and {{secret:...}} with the names you defined. An undefined token is flagged, and Generate stays disabled until you define it or remove it.
• Does anything create data? That value must be unique per run or the second run collides with the first. Put {{unique}} inside it, for example casey+{{unique}}@example.com. Every {{unique}} in one run resolves to the same stamp, so a later step can log in with what it just created.

Save. The test feature should now show Ready to Generate.

The editor also has AI actions that tighten a draft in place — rough notes to Gherkin, adding edge cases, adding assertions. A few credits each, far cheaper than a wasted generation.

The three phases

The drawer narrates activity rather than dumping a console. It loops back to phase 2 whenever a check fails.

1. Exploring the app. An agent opens your Base URL in a real Chromium in an isolated cloud sandbox and probes it — logging in, opening menus, reading the controls your app renders. It uses what it already knows about this project: your notes, facts earlier sessions learned, and patterns from tests already generated here.
2. Writing the test. One Playwright spec file, one test per Gherkin Scenario, using role-and-name locators against the controls it just saw.
3. Verifying · run 1, 2, 3… It executes the spec through the same harness a published run uses. Red sends it back to phase 2. The last attempt is Final verification.

How long it takes

You do not have to sit there. The drawer offers Run in background. Generation runs server-side, so closing the drawer, navigating away, reloading, or signing in from another machine all pick the session back up; only Cancel cancels. Good use of the wait: write your next two test features, which queue and generate one after another.

When it finishes

• Success. The header reads Generation complete, the feature moves to Ready to Test, and the footer offers Run tests. A What the Assistant learned card lists anything durable it discovered — that knowledge is kept and reused.
• Failure. Generation failed, with the reason at the top and the How this test was built log already expanded. If a spec was written but not proven, the header says Generation failed · test saved and the file is kept. The footer offers Retry generation and, when a trace was captured, Debug with AI.
• Environmental causes are stated plainly — the site was unreachable, the fleet was at capacity, the workspace ran out of credits. None means your app or your Gherkin is wrong. Retry.

What was saved: one Playwright spec file plus a small sidecar mapping its steps. Read it any time on the feature's Code tab.

Starting a run

1. From the test feature, press Run. (From a just-finished generation drawer, Run tests does the same.) The run composer opens.
2. Check the four sections. Defaults are usually right:
   • Environment — which target to run against. Your Base URL is the default.
   • Configurations — which Playwright configuration to run under.
   • Execution — Auto (recommended), Sequential — one at a time, or a custom concurrency. Leave it on Auto.
   • Test Features — which tests are in scope. Pre-ticked.
   The line under the title does the arithmetic: N test features × M configs = K test jobs.
3. Press Start Run. You should land on the run page, which updates live.

If TestVibe cannot reach the target from where tests execute, it tells you before dispatching and offers to fix the connection, run anyway, or cancel. Nothing is spent on a run that was never going to connect.

What you see while it runs

The phase banner walks through Queued — waiting for a runner → Allocating a cloud test session → Running tests. Then scenarios report individually: Pending, Running, then Passed, Failed or Skipped. Tiles above count scenarios, running, queued and passed as they land.

Anatomy of a run page

A header with the outcome badge, the environment and who triggered it; five tiles; a tab strip; then the scenario tree.

The tabs are Test Features · Images · Videos · Network · Console · Artifacts · Server · Report. The first is the scenario tree; start there. Watch the Server tab: it grows a red dot when the app under test threw or restarted during this run, and that dot is one of the strongest signals you will get all day.

Reading a failure

1. Start in the tree. Failures are grouped by test feature and highlighted, each group carrying a failed count. Above 20 scenarios you also get a filter row — All · Failed · Passed · Skipped.
   • The shape of the failure is itself information. One scenario out of forty: probably a real, specific defect. Every scenario in one feature failing at the same early step: something shared broke — the login, a navigation path, a fixture. Everything in the run failing: look at the environment first.
2. Open the failing scenario. A drawer slides in with a Steps timeline split into Background and Scenario.
3. Read the steps. Each row is the Gherkin step in plain language with its own tick or cross. The last green step tells you how far your app really got. The failed row is already expanded and shows the exact Playwright calls, with the failing line marked.
4. Read the error. Under Error is the assertion message — usually "expected X, received Y" or "waited N ms for element". Internal stack frames are hidden; Copy puts the complete trace on your clipboard.
5. Look at the picture. Scroll down to the Final screenshot — the page as it was when the test gave up. Nine times out of ten this alone answers it. Below it sit the Video and the Artifacts list.

The Playwright trace

A trace is captured for every scenario, passing or failing: a complete, replayable recording of what the browser did. It holds a timeline of every action with a snapshot of the page before, during and after — real inspectable pages, not screenshots — plus console output, network requests and the source line each action came from.

• Opening it. On a passing or skipped scenario the drawer footer shows Open Playwright trace. On a failing one that button becomes Debug with AI, which opens the same viewer with an assistant docked beside it. Traces are also on the run's Artifacts tab.
• The move that answers most questions. Click the failing action and look at the "before" snapshot. That is the page exactly as it was when the test tried to act. Was the button there? Covered by a cookie banner or a modal? Still a spinner? Was there an error message the test never mentioned?

Is my app broken, or is my test wrong?

Work down this table. The first row that matches usually settles it.

AI shortcuts on a run page

• Ask AI — in the run header. Summarises the run, or explains why it failed. Good when there are many failures and you want the pattern before the detail.
• Debug with AI — on a failing scenario. Opens the trace with an assistant docked beside it. This is repair 1 in section 07.
• Report tab — a written report of the run, with failure screenshots, to paste into a ticket or a stand-up.
• Comments — a rail on any run for a note, optionally pinned to one scenario. A pinned comment follows that scenario into later runs, so "we know about this one, it's the flaky search filter" stops being tribal knowledge.

Running it again

• Re-run test feature — in the scenario drawer. Opens the composer with that test feature and this run's configuration preselected. Runs are dispatched per test feature, not per scenario, which is why the label says so.
• Re-run — in the run header. Repeats the whole run.
• Both go through Start Run, so you get one more chance to change the environment before anything is spent.

Loop until green. Then the feature's Runs tab shows the history for that one test, and its History tab shows who changed it, when, and which lines.

Drive it from your terminal and CI

Create a key in Settings → CLI & API keys — it is shown once — then:

npm install -g testvibe

testvibe login --server https://your-testvibe --key tvb_...
testvibe use my-project

testvibe features list --status generated
testvibe run Login --wait
testvibe runs list --status failed

testvibe verify --git --wait maps your changed files to the tests that cover them and runs exactly those — the natural pull-request gate. The same package runs as an MCP server (testvibe mcp) so AI coding tools can drive it.

The rest, when you want it

• Load Testing — replay your functional journeys as virtual users, or drive protocol-level traffic, with thresholds and live throughput.
• Settings → Connectors — Jira, GitHub, GitLab, Azure DevOps. A failed run then gets a file-an-issue action, with deduplication.
• Settings → Environments — run the same suite against staging, preview and production-like targets.
• Settings → Test Files Sync — mirror your features and specs into a Git repository.
• Settings → AI Model — the model, and the generation timeout ceiling.
• Plugins — extra help for particular UI frameworks.
• Knowledge Base — turn your test features into end-user help articles.

The habit that makes all of it work: describe one behaviour at a time, insist on an assertion, and read the trace before you blame anything.