What works without it, and what does not

Setting it up

Everything is in the admin console under Mail. There are two ways to send, and you pick one.

1. Choose a transport.
   • Your own SMTP relay — preferred, and the only option that works without outbound internet.
   • Mandrill — one HTTPS call per message to mandrillapp.com, with no relay to run, but this install then needs outbound internet. Set TESTVIBE_MANDRILL_API_KEY (Mandrill API key). Your from address must be on a domain verified in your Mandrill account or every message is rejected.
   If both are set, the relay wins and the Mandrill key is ignored.
2. Fill in the fields for the transport you chose (table below).
3. Set a from address on a domain your relay will accept. Most relays reject a sender on a domain they do not own.
4. Press Send test message. You should receive the test mail. Do not move on until you do — every other mail problem in this product traces back to this step.

One more optional field lives here: TESTVIBE_SIGNUP_NOTIFY_EMAIL (Notify on new sign-ups) tells an address of yours when someone registers. Blank means nobody is notified — this install never reports sign-ups outside your network.

How it is put together

• A connection is the credential, and it belongs to the workspace. Created once, in Settings → Connectors, by an owner or admin. It is encrypted at rest and never returned to a browser.
• A binding says what this project uses that connection for, and which target — which Jira project, which repository. It holds a pointer, never a credential.

What it does once connected

• A File issue button appears on run, load-run and generation detail screens, one per connection that can file issues. Automations can file issues too, and an automated ticket is identical to a hand-filed one.
• The issue body carries the permanent TestVibe deep link and names the artifacts. It deliberately does not embed artifact links — those expire, and a dead link in a two-year-old ticket helps nobody.
• Repeat failures with the same signature comment on the existing ticket rather than opening a new one each night. A person clicking the button always gets a new ticket, because they meant it.

Connector credentials are separate from the jira test plugin's credentials, deliberately. The plugin runs inside the test sandbox; connectors run in the application. Same service, different privilege boundary — do not reuse one token for both.

How a hook works

• Six events: before and after each of run, load test and generation. Suite-level only — there are no per-scenario hooks, because scenarios run in parallel and concurrent "reset the database" calls would stomp each other.
• Five fields: name, event, URL, failure policy, enabled. That is the whole configuration surface, on purpose.
• TestVibe sends a fixed JSON body and reads only the status code. No custom body, no custom headers, no response parsing — the same contract GitHub and Stripe webhooks use.
• A before hook can refuse. With the failure policy set to fail, a non-2xx response stops the run before any capacity is taken. An after hook can never block.
• Signed with a TestVibe-Signature header — t=<unix>,v1=<hex HMAC-SHA256> over "{t}.{body}", Stripe's construction, so most integrators already have verification code. The secret is shown exactly once when the hook is created.
• Fixed 30-second timeout and two retries.

Correlating a hook with the tests it set up

Both sides see the same value. The hook payload carries a per-run unique token, and the tests get the same value as the TESTVIBE_RUN_UNIQUE environment variable. So your endpoint creates tenant-<unique> and the tests find it, with nothing parsed out of the hook's response.

That is also the answer when the private-address restriction bites: if your seeding endpoint cannot be exposed, have the tests do the seeding, keyed on the same value. Tests run in the runner, which can reach your network.

Hook operator notes

• Hooks run in the application, not in the test sandbox. That is why they get secrets and retries — and why section 02 applies to them.
• A run blocked by a failing before-hook ends as error with a banner naming the hook. It is not a test failure, and no capacity was consumed.
• Three consecutive failures of a blocking hook force a notification even for scheduled runs, which are otherwise quiet — so a broken hook cannot kill a nightly schedule in silence.
• The execution log stores failure responses but never a successful one — a setup endpoint's 200 is the response most likely to contain a freshly-minted credential.
• Where: Settings → Project → Hooks. Use the Send test request button, because it is the only place a successful response body is ever shown.

Channels

Per channel and per event type — run, generation, load — a user chooses all, failures only, or off. Cancellations never notify.

There is also a per-automation AI triage on failure toggle: when a run this automation triggered fails, the assistant investigates and the outcome carries a root cause instead of just "failed" — shown on the run, in the failure notification, and in the automation's history.

Automation operator notes

• Scheduling is safe across restarts and replicas. A due automation is claimed atomically, so a restart mid-tick cannot double-fire and a second instance cannot either.
• An automation-produced run cannot re-trigger an automation. There is a loop guard; you will not wake up to a runaway chain.
• Time zone matters. A blank or unrecognised timezone is treated as UTC. On a "why did it run at the wrong time" question, check that field first.
• Automations consume capacity like any other dispatch. Several nightly schedules landing on the same minute will queue against your concurrency caps — see Capacity.
• Assistant automations run inline on the scheduler tick with a 5-minute ceiling, so many of them due at once will serialise. Their output arrives by email.
• Automation-dispatched runs are silent by default — see section 06. Enabling AI triage makes its failures deliver anyway, because you opted in.

What a user needs

1. An API key. Created in the application under Settings → CLI & API keys. Keys begin tvb_ and are shown once. They are sent as Authorization: Bearer tvb_….
2. Your install's address. Everything is under {your base URL}/api/v1/ops. There is no TestVibe-hosted endpoint involved.
3. The CLI, if they want one. You should see your projects listed back.
   shellcopy

   npm install -g testvibe
   export TESTVIBE_SERVER=https://testvibe.example.com
   export TESTVIBE_API_KEY=tvb_…
   testvibe projects

   The same package runs as an MCP stdio server, so an AI assistant can drive the same operations. TESTVIBE_PROJECT pins a default project.

API operator notes

• Plan gating does not apply. On the cloud, programmatic access is a paid capability; on a self-hosted install that check is skipped entirely. Every workspace has full API, CLI and MCP access.
• You can still put a ceiling on it. TESTVIBE_OPS_RATE_LIMIT_PER_HOUR caps requests per workspace per hour when set — unset means no per-workspace limit on self-hosted. A separate per-IP limiter, TESTVIBE_OPS_RATE_LIMIT_IP_PER_HOUR, defaults to 10,000 per hour. Both are environment-only; neither is in the console. Clients get the standard RateLimit-* response headers whenever limiting is on.
• The API is a full control plane — it can create features, dispatch runs and generations, and edit project files. Treat a tvb_ key as equivalent to the user's access, and make sure your reverse proxy is not caching authenticated responses.

Where to go next

• Capacity — what happens when automations and CI dispatch at the same time.
• Licensing · Sign-in · AI providers · Storage.
• The setup checklist — the rest of the first-hour configuration.
• Product documentation — the per-feature reference your users will want.