Getting started with TestVibe: from sign-up to your first passing test

TestVibe turns a plain-English (or Gherkin) description of what your app should do into a runnable Playwright test, executes it in an isolated cloud sandbox against your site's URL, and hands you back screenshots, video, and a full Playwright trace. Nothing to install, no CI pipeline to wire up, no repository required.

This post walks the shortest successful path end to end: sign up, create a project, describe one feature, generate the test, run it, and read the result. Budget about ten minutes plus a few minutes of generation time.

What you need before you start

Two things:

• A way to sign in. Email and password, GitHub, or Google. Every user gets a personal workspace automatically.
• A URL to test. The address of a deployed, staging, or shared QA environment that a cloud session can reach from the internet. (Private apps behind a VPN work too, via a tunnel device — but skip that for your first test.)

That's the entire prerequisite list. Your first project is one screen: a name, a site URL, and an optional target framework.

The walkthrough

1. Sign up and sign in

1. Open TestVibe in your browser.
2. Pick a sign-in method: Continue with GitHub, Continue with Google, or create a local account with email and password (Create an account).
3. If you created a local account, verify your email — verification is required before your first sign-in. OAuth sign-ins skip this.
4. Land in your personal workspace.

One thing worth knowing up front: sign-in is identity-only. TestVibe never asks for repository permissions at sign-in — connecting a source repo is a separate, optional step you can take later from project settings, and you don't need it at all to test.

2. Create a project

A project is one app under test: a name, the site URL(s) to test, and everything that follows — tests, runs, evidence. If this is the first project on your account, TestVibe shows you the creation screen automatically right after sign-in.

1. Enter a project name.
2. Enter the base URL of the site you want to test — the address your tests will open.
3. Optionally pick the target framework(s) so TestVibe can better read your app's UI (it also auto-detects on first visit).
4. Create the project.

Creation is instant and free — it doesn't spend credits and doesn't probe your site. You land directly in the Test Suite with the Describe a flow dialog open, ready for step 3.

3. Describe a feature

The Test Suite is where test intent lives: groups on the left (think Authentication, Shopping), features and their details on the right.

1. Create or select a group for the feature area you want to test.
2. Create a new feature and give it a clear name.
3. Write the behavior as Gherkin-style steps.
4. Save. It's stored in your project.

Keep the first test small: one page, one action, one result you can verify. A good template:

Feature: Sign in

  Scenario: User signs in with valid credentials
    Given the user is on the sign-in page
    When the user enters valid credentials
    And the user clicks Sign in
    Then the dashboard should open
    And the user name should be visible in the header

What makes this work: a visible starting point, a concrete user action, and an expected result the generated test can actually assert. Use visible text from your application where possible, and avoid vague phrases like "it should work." If your flow needs credentials, don't hardcode them — add them in Settings → Variables & Secrets and reference them from the test as {{var:NAME}} or {{secret:NAME}}.

4. Generate the Playwright code

Now hand the intent to the AI.

1. Select your feature in the Test Suite.
2. Start generation from the test details or generation action.
3. Confirm the site URL and options if prompted; keep the default agent and model.
4. Watch the progress view: queued → setting up → running → finalizing → succeeded.

Under the hood, a cloud agent takes your Gherkin as the source of intent, works against your live site, and produces Playwright code that's saved to your project. This takes a few minutes, and generation usage is metered in credits.

When it succeeds, the feature is ready to run. Before you do, skim the generated output on the Code tab: does it target the right page, and does it check the expected result you wrote? If generation fails instead, open the failure details before touching the test — the cause is usually a missing variable or secret, an unreachable URL, or an instruction that was too vague.

5. Run it

Each run executes in an isolated cloud test session. Run just the one test first — a single-test run is faster to troubleshoot than a suite.

1. Select the generated test and choose the run action.
2. Confirm the site URL.
3. Keep the default browser, viewport, and configuration.
4. Start the run.
5. Watch the live status: Queued → Preparing → Running, then live per-scenario and per-step progress until it finishes Passed or Failed.

6. Read the result — and the trace

When the run finishes, open the result in the Runs section. Start with the summary: overall status, scenarios passed or failed, duration, browser, and site URL.

Then drill into the evidence, all uploaded from the cloud session to your project:

• Screenshots — what the page looked like at important moments
• Videos — a replay of the browser session
• Trace — the Playwright trace: every action, DOM snapshots, and timing (open it with Open Playwright trace from a scenario)
• Console and network — browser errors and failed requests
• Failure details — which step or expectation didn't pass

If it passed: you have a working, regenerable test — add another scenario to the same feature, then a related feature, then run a small group. If it failed, resist the urge to immediately edit the test. A failure can mean a real app bug, missing test data, a wrong URL, or a selector that needs refinement — the evidence tells you which. If the test checked the wrong thing, improve the Gherkin and hit Regenerate; the code is disposable, the intent is the source of truth.

Wrapping up

That's the whole loop: describe behavior in plain language, let AI turn it into Playwright code, run it in an isolated cloud sandbox, and get evidence you can actually act on. The intent stays readable, so when your app changes you edit a sentence and regenerate instead of rewriting selectors.

Ready to try it with your own app? TestVibe is in early access — get early access and you'll be first in line to run this exact path: one project, one feature, one passing test. In the meantime, the quickstart docs cover every step in more depth.