FAQ

Answers to the questions we hear most. Each answer links to the guide or concept with the full story.

General#

What is BugBrain?#

BugBrain is an AI QA platform. Its agents open a real browser, explore your app like a careful user, drive high-value flows end to end, and report the problems they find — broken interactions, console errors, failed network calls, dead ends, accessibility issues, and more, all captured with screenshots and a step-by-step log. See What is BugBrain?

Does BugBrain replace my QA team?#

No — it augments them. BugBrain takes on broad, repeatable coverage and first-pass triage so your people spend their time on judgment, tricky edge cases, and product decisions. See For QA managers.

Who is BugBrain for?#

Engineers and SDETs who want coverage without authoring scripts, QA managers who need consistent regression coverage and reporting, and PMs who want a clear read on quality. There are tailored intros for developers and PMs.

What's the difference between a run, a test case, and a test plan?#

A run is one execution against a project (most often an open-ended exploratory run). A test case is a saved, repeatable check. A test plan is a group of test cases run together. See Key concepts.

Getting started#

Can I get started without writing test scripts?#

Yes. Point BugBrain at a reachable URL and start an exploratory run — the AI decides what to test, no scripts required. See the Quickstart.

How do I run my first test?#

Create a project with a start URL, then choose Start a run. The run appears under Active Runs with live progress; open it to see screenshots, the issues found, and the AI action log. See Test runs.

Do I need to set anything up first?#

Just a project with a reachable URL. To test signed-in areas, add login credentials to the project. See Set up your workspace and Projects.

How do I turn exploratory findings into repeatable tests?#

Once exploratory runs reveal your real flows, promote the important ones to test cases for repeatable regression coverage. See Test cases.

Runs & verdicts#

Can BugBrain test a localhost or staging app?#

Staging and other reachable URLs work directly. For a local app, the MCP server's local bridge can create a tunnel so the cloud runner can reach it. See MCP server and MCP tools.

How do I test a signed-in area?#

Add login credentials to the project; the agent logs in before exploring. See Projects.

What does INCONCLUSIVE mean?#

It's the third verdict alongside PASS and FAIL, used when the agent couldn't confidently confirm or deny a check — element not found, a CAPTCHA, or confidence below the bar. It's used on purpose instead of forcing a false FAIL, so a FAIL stays trustworthy. See Test-run scoring.

Why did a run find nothing?#

Usually the agent never reached your key pages. Open the action log and timeline, then check the start URL and (if used) that login succeeded — those are the two most common causes. See Test runs.

Why is my run FAILED?#

FAILED means the run itself couldn't complete — the site was unreachable, or login failed. (That's different from an INCONCLUSIVE check, which is about an individual assertion.) The run viewer's error message and action log show exactly where it stopped.

Can I control how far a run explores?#

Yes. Project settings cap things like max steps per flow, max flows per run, and crawl depth. Risk-based runs (such as PR checks) tune these automatically. See Test runs.

Can I run tests on a schedule or on every pull request?#

Yes — schedule runs from the project, and connect the GitHub App for pre-merge runs. See Test plans and PR automation.

Test cases#

What's the difference between an exploratory run and running test cases?#

An exploratory run is open-ended — the AI decides what to test. Running test cases (or a plan) executes specific, saved checks. Use exploratory runs for discovery and case runs for repeatable regression coverage. See Test cases.

How are test cases created?#

You can author them by hand, have the AI generate them from natural language, or import them in bulk from a CSV. See Test cases.

Do I have to write selectors?#

No. Test cases describe intent in plain steps; the agent resolves the right elements at run time and re-resolves them when the UI changes. See Self-healing tests.

What is self-healing?#

When a UI change moves an element, BugBrain re-resolves the locator so the test keeps passing instead of failing on a brittle selector. See Self-healing tests.

Issues#

How does BugBrain avoid reporting the same bug twice?#

Each issue gets a stable signature used to deduplicate it across runs, so the same defect isn't reported many times. See Issue confidence & dedup.

What is a confidence score?#

A 0-to-1 measure of how sure the agent is about an issue or step. Low confidence pushes a result toward INCONCLUSIVE rather than a false FAIL, and very low-confidence findings are filtered out. See Issue confidence & dedup.

How do I act on what a run finds?#

Open the issue detail for severity, category, and evidence, triage it, and push it to your tracker. See Issues & bug triage.

Integrations#

How do I connect Jira or Slack?#

From the Integrations settings, pick the provider and follow its connect step (an API credential, a webhook URL, or a connect flow, depending on the provider). See Integrations and the Integrations matrix.

Which integrations are supported?#

Ten providers: Jira, Linear, Slack, Microsoft Teams, Email, GitHub Issues, GitLab Issues, Azure DevOps, Asana, and ClickUp. Some are issue trackers (BugBrain files bugs in them), some are notifiers (BugBrain sends alerts). See the Integrations matrix.

Why can't I connect a particular integration?#

Availability is gated three ways: the integration's feature flag must be on, and the provider must be allowed by your plan or granted to your workspace. Connecting also requires active billing. See the Integrations matrix.

Can I get notified over the API instead?#

Yes — register an outbound webhook for run and PR-check events, or pull data through the REST API. See Webhooks, Webhook events, and the REST API.

Billing & quotas#

What plans are available?#

Launch, Startup, and Scale, plus a 14-day trial for new workspaces. Exact quota numbers are on the in-app Billing page. See Billing & plans and Plans & quotas.

How does the 14-day trial work?#

New workspaces start on a BugBrain-managed 14-day trial. When it ends without a paid plan, the workspace moves to a past-due state until billing is set up. See Billing & plans.

How do I change plans?#

From the Billing page. Plan changes take effect through your subscription; switching plans resets the standard usage counters. See Billing & plans.

What are the add-ons and how are they enabled?#

The metered add-ons are PR automation, load testing, and AI / LLM app testing. Each is off by default and turns on per workspace when its feature flag is on and its monthly quota is set above zero — the quota, not the plan, is the gate. See Plans & quotas.

Can I get a refund?#

Refunds are issued by the BugBrain team rather than self-served. Raise a refund request against a paid order from billing, and an operator reviews it. A refund doesn't automatically downgrade your workspace. See Billing & plans.

Security#

How is my data isolated between workspaces?#

BugBrain enforces strict per-workspace data isolation at the data layer, so one workspace can never read another's data. See Security & data handling.

How are credentials and secrets protected?#

Sensitive credentials (such as login details and integration tokens) are encrypted at rest, and secrets are redacted from logs. See Security & data handling.

Are you SOC 2 certified?#

No. BugBrain is not SOC 2 certified, and we don't claim to be. The security pages describe only the controls actually in place. See Security & data handling.

How do I keep an API key safe?#

Treat an sk_live_ key like a password: store it in a secrets manager or CI secret, never in source control, and scope it to the least privilege it needs. If it leaks, revoke it and mint a new one. See API keys.

Troubleshooting#

My run failed — what should I check?#

FAILED means the run couldn't complete. Open the run viewer's error message and action log, then confirm the start URL is reachable and (if used) the login credentials are correct. See Test runs.

Login failed during a run — why?#

The agent couldn't sign in. Check the project's credentials are correct and current, and that the login page is reachable. CAPTCHA or multi-factor prompts can also block automated login. See Projects.

A run timed out or seems stuck.#

Check the start URL responds and isn't behind something the agent can't pass (a CAPTCHA, an unreachable network, or a very slow page). The action log shows the last step it reached. Bounding exploration in project settings also keeps runs fast and focused. See Test runs.

I'm getting rate-limited (429) on the API.#

The API is rate limited per principal. Back off and retry rather than polling in a tight loop — a poll storm only counts against your own limit. See the REST API.

My webhook isn't being delivered, or signature verification fails.#

Confirm your endpoint returns a 2xx quickly and that your URL is publicly reachable (private or loopback URLs are rejected). For signature failures, verify the HMAC over the raw request body and check the timestamp tolerance. See Webhooks and Webhook events.

An MCP tool I expected isn't showing up.#

The MCP server only advertises tools your API key's scopes permit. Re-scope or re-mint the key with the scopes you need, then restart the server. See MCP tools and API keys.

Related#