Bring your own model: custom LLM endpoints in self-hosted TestVibe

On the cloud, TestVibe's AI generation runs on a first-party metered service and you never touch a model provider. Self-hosted is different: you decide where the model lives. One of those choices is to point generation at your own OpenAI-compatible endpoint — an internal gateway, an on-prem inference server, or a model running as part of the stack — so generation can use that internal endpoint; all enabled integrations must also be internal for a no-egress claim.

This guide covers when that option makes sense, how it's wired, and the caveats that decide whether it actually works.

What the model is (and isn't) used for

Before you stand up any infrastructure, get the scope right. On a self-hosted instance, AI powers generation and the assistant, and can be used by scans, triage or enabled AI plugins. Ordinary manual authoring, existing Playwright tests and viewing results do not require a generation model; any AI-dependent operation still needs a configured provider. If you never set up a model, generation is simply unavailable and AI-dependent scans, plugins and triage will also need a provider.

That scoping matters for capacity planning: you're not sizing an inference cluster for constant traffic, just for bursts of generation work — which is where a GPU earns its keep.

Two ways to supply a model

Self-hosted TestVibe gives you two paths:

• Hosted keys. Provide an Anthropic or OpenAI API key. The app calls that provider directly. A hosted-provider setup — but generation prompts leave your network and go to the provider.
• Bring your own model. Point TestVibe at your own OpenAI-compatible endpoint — Ollama, vLLM, LM Studio, or anything that speaks the same API. Generation stays inside your network when the model endpoint, browser target, and any enabled integrations are also internal.

The decision usually comes down to one axis:

If prompts must never leave your perimeter, the endpoint path is the only one that qualifies.

What you configure

Self-hosted installations support environment-based bootstrap configuration and an installation-administrator AI models panel for providers and model choices. Use the panel to manage the catalog, or supply the bundle's documented environment settings for a custom endpoint:

• The endpoint URL of your inference server.
• The model ids it serves. These become the choices in the app's AI Model picker under Settings → AI Model.
• Optionally an API token, a display label, and a multimodal flag if the model handles vision (used for visual comparison features).

For environment-based secrets that support a {NAME}_FILE form, point that variable at a mounted secret file instead of putting the token in .env. That keeps a gateway token out of your .env and inside a Docker or Kubernetes secret.

The bundle can also run a local model as part of the stack — an optional inference sidecar — so the whole deployment is self-contained without calling out to an external gateway.

The one caveat that breaks generation

Read this before you pick a model. TestVibe does not generate a test by asking for a block of code. It drives a browser step by step, and it does that by having the model call tools. The generation loop is a sequence of function calls — navigate, click, assert, and so on.

That means:

The model must support tool/function calling. A model without it will stall during generation rather than fail cleanly.

Plenty of otherwise-capable local models don't advertise tool support, or implement it unreliably. Choose one that explicitly lists tools in its capabilities — a recent coding-oriented model is your best bet — and verify before you rely on it. A quick check against your endpoint, using the same OpenAI-compatible shape TestVibe will use:

curl http://inference.internal:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-model-id",
    "messages": [{"role": "user", "content": "Click the submit button on the page."}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "click",
        "description": "Click an element on the page",
        "parameters": {
          "type": "object",
          "properties": { "selector": { "type": "string" } },
          "required": ["selector"]
        }
      }
    }],
    "tool_choice": "auto"
  }'

A tool_calls array shows the endpoint can return a tool call for this request. It does not prove all tool schemas, streaming or a complete TestVibe generation work with that serving stack. If it answers in text and ignores the tool, it will stall inside TestVibe the same way.

Two more practical notes: a GPU is strongly recommended — CPU-only inference makes generation painfully slow — and the model's quality directly caps the quality of the generated tests. Hosted frontier models set a high bar; measure output quality, latency and cost on representative flows instead of inferring suitability solely from model size.

Where the safety net still applies

The honest part of TestVibe's generation story survives the model swap. A generated test only reaches generated status after TestVibe replays the whole assembled spec end-to-end and it passes. That validation always runs and can't be turned off. So even a weaker local model can't silently ship a broken test — a test that doesn't actually pass never gets marked generated. A passing verification proves the recorded checks passed on that run. It does not prove the assertions cover the right behavior; review the Gherkin and generated code before relying on them.

One last thing worth knowing: on-prem, credit gating and token metering are off entirely. There are no credits to buy and nothing is metered. Your only cost controls are your own inference hardware and, if you chose hosted keys instead, your provider's bill.

Running TestVibe fully inside your own network — model included — keeps every generation prompt on infrastructure you control. Get early access to the self-hosted bundle, and see the self-hosted AI models guide for the full configuration reference.