Variables and secrets: keeping credentials out of your specs

Almost every useful test signs in as someone. That means a username, a password, maybe an API token for setup — and the moment those land in a spec file, they leak into version control, logs, screenshots, and anything an AI reads while generating code. TestVibe keeps them out with two per-project stores and a token syntax that uses runtime references and injects configuration into generation and execution environments as needed.

This guide covers the difference between variables and secrets, how to reference them from Gherkin, what the AI does and doesn't see, and how to rotate a credential safely.

Variables vs secrets

Both live in one place: Settings → Variables & Secrets, scoped to a single project. The difference is how they're handled.

The rule of thumb: use a secret whenever the value should not appear in logs, screenshots, or shared docs. A staging URL is a variable. A staging password is a secret. Never store a password as a variable just because it's convenient — once it's plaintext, it's plaintext everywhere.

Secrets are write-only. Once you save one, the value is never displayed back to anyone, including you. That's the trade-off that makes it safe, and it's why rotation (below) means replacing rather than editing.

Use project variables for non-sensitive configuration and secret references for credentials. Keep values out of instructions and authored files; browser evidence can still reveal data entered into the application.

Adding a value

The flow is the same for both types:

1. Select Settings from the navigation rail.
2. Open Variables & Secrets under Project.
3. Add a name and value in the Variables or Secrets list and save.

Names matter more than they look. They're matched exactly — spelling and case — against the tokens in your specs, so pick clear, stable names you'll recognize months later:

TEST_USER_EMAIL
TEST_USER_PASSWORD
STAGING_API_TOKEN

Uppercase-with-underscores reads like an environment variable, which is exactly what a variable becomes inside the runner. Consistency here saves you from the most common failure mode: a missing-reference or environment-value error when the stored name does not match.

Referencing values from Gherkin

You reference stored values with runtime tokens. Type @ in the feature editor to autocomplete the names you've saved, so you don't have to remember exact spelling:

Given I am on the "Login" page
When I sign in with username "{{var:TEST_USER_EMAIL}}" and password "{{secret:TEST_USER_PASSWORD}}"
Then I should see the account dashboard

{{var:NAME}} pulls from the variables list; {{secret:NAME}} pulls from the secrets list. The token stays in the feature file as literal text — the actual value can be resolved during generation and test execution. So the spec you commit, share, or export contains {{secret:TEST_USER_PASSWORD}}, never the password itself.

This works the same way for setup calls that run before the browser opens. If a scenario needs a fresh record created through an API first, reference the token there too — the entrypoint step still comes first, then the sign-in step, exactly as in any other feature:

Background:
  Given a test account exists via "{{secret:STAGING_API_TOKEN}}"
  And I am on the "Login" page
  When I sign in with username "{{var:TEST_USER_EMAIL}}" and password "{{secret:TEST_USER_PASSWORD}}"

Secret references and browser evidence

TestVibe generates Playwright code from your Gherkin, and generation is given access to your project's configuration so the code fits — including the names of your variables and secrets. That's how the generated script knows to read TEST_USER_PASSWORD from the environment instead of inventing a hard-coded string.

Secret values are injected into the execution environment for generation and test runs, because the browser may need them to sign in. Generated code should read the named environment variable rather than contain a literal credential. Known secret values are redacted from generation transcripts, but page content, screenshots, and traces can still contain sensitive data. Use dedicated test identities and review evidence before sharing it.

The generated code looks roughly like this — reading the injected environment variable, not a literal:

await page.getByLabel('Username').fill(process.env.TEST_USER_EMAIL);
await page.getByLabel('Password').fill(process.env.TEST_USER_PASSWORD);
await page.getByRole('button', { name: 'Sign in' }).click();

Rotating a credential

Rotation is deliberately simple because secrets are write-only: you replace the value, you don't edit it.

1. Rotate the credential in the source system (your app, your identity provider, your API).
2. In Settings → Variables & Secrets, open the entry and save the new value over the old one.
3. Re-run the test or regenerate. Nothing in your specs changes — the token name is stable, only the value behind it moved.

Because the old value was never readable, there's nothing to clean up in the UI. If a key was ever pasted somewhere it shouldn't have been — a step description, a chat message, a commit — rotate it. That's the whole point of keeping the plaintext out of the spec in the first place: rotation stays cheap.

Practical habits

A few conventions keep this friction-free across a growing suite:

• Use dedicated test accounts, not personal credentials. A test account you can reset, lock down, and rotate freely is worth setting up once.
• Never paste a key into a step description. If you're typing a password into Gherkin, stop and make it a {{secret:...}} token instead.
• Keep variables and secrets in the right buckets. A missing or empty value can come from a name mismatch, namespace, wrong project or unset runtime environment — both are avoidable with disciplined naming.
• Scope by project. Values live per project. Within one project, use explicit environment-specific names/setup; selecting a target URL does not automatically isolate credentials.

Credentials are the one thing a test genuinely needs and the one thing you never want committed. Storing them as scoped variables and secrets, and referencing them by token, means your specs stay shareable while stored secret values are encrypted, while generation and execution must access plaintext values when required. See the Add API keys and secrets guide for the full reference, and get early access to try it against your own app.