Functional tests answer "does it work?" and visual regression tests answer "does it look right?" — different questions, and a green functional suite can sit happily on top of a page that renders as a garbled mess. Here's what each approach catches, where each one wastes your time, and how to combine them without drowning in false-failure noise.
What functional testing catches
A functional test drives the app the way a user would and asserts on behavior. Click "Add to cart," expect the cart badge to read 1. Submit the login form with bad credentials, expect an error message. Fill checkout, expect an order confirmation.
await page.getByRole('button', { name: 'Add to cart' }).click();
await expect(page.getByTestId('cart-count')).toHaveText('1');This assertion queries the DOM. It passes whether the cart badge is a crisp blue pill or an unstyled black 1 jammed into the corner because a stylesheet failed to load. The button could be invisible, one pixel tall, or stacked behind a modal — as long as Playwright can resolve the locator and click it, the behavior check passes. Functional tests are deliberately blind to appearance, and most of the time that blindness is a feature: it keeps them stable across a redesign that changes every color and margin but none of the logic.
What visual regression catches that functional misses
Now flip it. A CSS refactor drops a z-index, and a cookie banner slides on top of the primary CTA. A design-token rename turns every "danger" button the same grey as the page background. A font swap pushes a headline onto three lines and shoves the hero image below the fold. None of these break a single functional assertion. Every one of them ships a broken page.
Visual regression catches exactly this class of bug: unintended change to what the user sees. It works by capturing a screenshot, storing it as a baseline the first time, and on every later run comparing the current render against that baseline. Playwright ships this natively:
await expect(page).toHaveScreenshot('pricing.png', {
maxDiffPixelRatio: 0.01,
});When the render drifts beyond the threshold, the test fails and hands you an expected/actual/diff triptych. You don't read a stack trace — you look at two pictures and immediately see the banner, the grey button, the reflowed headline.
What visual testing can't tell you
The reverse gap is just as real. A pixel-perfect screenshot proves nothing about behavior. The "Buy" button can look flawless and do nothing on click. A form can render every field correctly and still POST to the wrong endpoint. Numbers can be laid out beautifully and be wrong. Screenshots freeze a moment; they don't exercise flows, follow redirects, check network calls, or verify that state actually changed. Anyone who tries to replace functional coverage with visual snapshots ends up with a suite that's confident about typography and silent about correctness.
Where visual diffs pay off, and where they drown you
Visual regression earns its keep where appearance is the contract:
- Design systems and component libraries. A button, badge, or card is defined by how it looks. Snapshot each component in each state and you catch the token rename that quietly recolors half the app.
- Marketing and landing pages. Layout, spacing, and hero imagery are the product. There's little behavior to assert, so a screenshot check is often the highest-value test on the page.
- Print-style or export views — invoices, receipts, PDFs — where a shifted total or clipped line is a real defect.
It drowns you everywhere content moves on its own:
- Timestamps, live counts, "3 minutes ago," randomized promos, A/B variants. Every run diffs against the baseline and every run fails, for reasons that have nothing to do with a regression.
- Animations and lazy-loaded content, where the screenshot fires mid-transition and captures a different frame each time.
- Cross-browser and cross-OS font rendering, where sub-pixel antialiasing differs between the machine that made the baseline and the one running CI.
The fix isn't to abandon visual testing — it's to scope it. Mask the volatile regions, freeze the clock, seed deterministic data, and pin the rendering environment so the only thing that can move the pixels is an actual change.
Pixel-diff or AI comparison
Strict pixel diffs are unforgiving by design, which is what makes them noisy on anything dynamic. The alternative is a comparison that judges intent rather than exact pixels. In TestVibe you can install a Visual Regression plugin that does a deterministic pixel diff against a baseline in your project's Asset Library, or TestVibe Assistant's AI comparison, which looks at the live screenshot and the baseline and decides whether they still match the described intent — tolerating minor font and date noise while still flagging a meaningfully different page. Use the pixel diff where you want zero drift; use the AI check where a strict diff would fail on rendering jitter you don't care about. Both live as Gherkin steps alongside your functional assertions, so a single run checks behavior and appearance together.
Run both, scoped deliberately
Functional tests are your floor: they prove the app does what it's supposed to, and they survive redesigns. Visual tests are the layer that catches the regressions your assertions are blind to — best aimed at design systems, marketing pages, and export views, and kept away from anything that changes on its own. Neither replaces the other, and the teams who treat it as "either/or" always find out which half they skipped at the worst possible time.
Want AI-generated Playwright tests, functional and visual, verified by actually running against your app? Get early access, or read more in the plugin docs.