Variables and secrets: keeping credentials out of your specs

Almost every useful test signs in as someone. That means a username, a password, maybe an API token for setup — and the moment those land in a spec file, they leak into version control, logs, screenshots, and anything an AI reads while generating code. TestVibe keeps them out with two per-project stores and a token syntax that resolves the real value only inside the sandbox at run time.

This guide covers the difference between variables and secrets, how to reference them from Gherkin, what the AI does and doesn't see, and how to rotate a credential safely.

Variables vs secrets

Both live in one place: Settings → Variables & Secrets, scoped to a single project. The difference is how they're handled.

The rule of thumb: use a secret whenever the value should not appear in logs, screenshots, or shared docs. A staging URL is a variable. A staging password is a secret. Never store a password as a variable just because it's convenient — once it's plaintext, it's plaintext everywhere.

Secrets are write-only. Once you save one, the value is never displayed back to anyone, including you. That's the trade-off that makes it safe, and it's why rotation (below) means replacing rather than editing.

Adding a value

The flow is the same for both types:

1. Select Settings from the navigation rail.
2. Open Variables & Secrets under Project.
3. Add a name and value in the Variables or Secrets list and save.

Names matter more than they look. They're matched exactly — spelling and case — against the tokens in your specs, so pick clear, stable names you'll recognize months later:

TEST_USER_EMAIL
TEST_USER_PASSWORD
STAGING_API_TOKEN

Uppercase-with-underscores reads like an environment variable, which is exactly what a variable becomes inside the runner. Consistency here saves you from the most common failure mode: a token that resolves as empty because the entry was saved under a slightly different name.

Referencing values from Gherkin

You reference stored values with runtime tokens. Type @ in the feature editor to autocomplete the names you've saved, so you don't have to remember exact spelling:

Given I am on the "Login" page
When I sign in with username "{{var:TEST_USER_EMAIL}}" and password "{{secret:TEST_USER_PASSWORD}}"
Then I should see the account dashboard

{{var:NAME}} pulls from the variables list; {{secret:NAME}} pulls from the secrets list. The token stays in the feature file as literal text — the actual value is substituted only when the test runs inside the sandbox. So the spec you commit, share, or export contains {{secret:TEST_USER_PASSWORD}}, never the password itself.

This works the same way for setup calls that run before the browser opens. If a scenario needs a fresh record created through an API first, reference the token there too — the entrypoint step still comes first, then the sign-in step, exactly as in any other feature:

Background:
  Given a test account exists via "{{secret:STAGING_API_TOKEN}}"
  And I am on the "Login" page
  When I sign in with username "{{var:TEST_USER_EMAIL}}" and password "{{secret:TEST_USER_PASSWORD}}"

What the AI sees, and what it never sees

TestVibe generates Playwright code from your Gherkin, and generation is given access to your project's configuration so the code fits — including the names of your variables and secrets. That's how the generated script knows to read TEST_USER_PASSWORD from the environment instead of inventing a hard-coded string.

What generation never receives is the secret value. A {{secret:...}} token is resolved at run time inside the sandbox, after code generation is finished. So the value never reaches the generation context, never appears in the generated code, and never shows up in run logs. The AI writes code that reads a credential; it never handles the credential.

The generated code looks roughly like this — reading the injected environment variable, not a literal:

await page.getByLabel('Username').fill(process.env.TEST_USER_EMAIL);
await page.getByLabel('Password').fill(process.env.TEST_USER_PASSWORD);
await page.getByRole('button', { name: 'Sign in' }).click();

Rotating a credential

Rotation is deliberately simple because secrets are write-only: you replace the value, you don't edit it.

1. Rotate the credential in the source system (your app, your identity provider, your API).
2. In Settings → Variables & Secrets, open the entry and save the new value over the old one.
3. Re-run the test or regenerate. Nothing in your specs changes — the token name is stable, only the value behind it moved.

Because the old value was never readable, there's nothing to clean up in the UI. If a key was ever pasted somewhere it shouldn't have been — a step description, a chat message, a commit — rotate it. That's the whole point of keeping the plaintext out of the spec in the first place: rotation stays cheap.

Practical habits

A few conventions keep this friction-free across a growing suite:

• Use dedicated test accounts, not personal credentials. A test account you can reset, lock down, and rotate freely is worth setting up once.
• Never paste a key into a step description. If you're typing a password into Gherkin, stop and make it a {{secret:...}} token instead.
• Keep variables and secrets in the right buckets. A value that resolves as empty is almost always a name mismatch or a value saved in the wrong project — both are avoidable with disciplined naming.
• Scope by project. Values live per project, so staging and production credentials for the same app stay separated by design.

Credentials are the one thing a test genuinely needs and the one thing you never want committed. Storing them as scoped variables and secrets, and referencing them by token, means your specs stay shareable while the real values stay in exactly one encrypted place. See the Add API keys and secrets guide for the full reference, and get early access to try it against your own app.