CI/CD Setup Guide

Learn how to integrate BugBrain with your CI/CD pipeline to automatically run tests on every code change.

Prerequisites

• BugBrain Account with a project created
• Test Plan — At least one test plan must exist
• Growth+ Plan or Higher — CI/CD integration requires Growth+ subscription
• CI/CD Platform Access — GitHub, GitLab, or Jenkins administrator access

Getting Your CI/CD Credentials

CI/CD Credentials Explained

When you generate a CI/CD token, you receive:

Types of CI/CD Credentials

CI/CD Webhook Token (Recommended)

Used for webhook-based triggering (GitHub Actions, GitLab CI):

• Scope: Single project + test plan
• Method: POST webhook request
• Security: HMAC-SHA256 signature verification
• Permissions: Execute tests only
• Rotation: Create new token to revoke

Service API Keys

For advanced scenarios (Jenkins, custom integrations):

• Scope: Entire organization (all projects)
• Method: Bearer token in Authorization header
• Security: bcrypt hashed, IP allowlisting available
• Permissions: Full API access
• Format: bugbrain_sk_* (production) or bugbrain_dev_* (development)

Request service keys from your account manager or support.

Credential Management

Viewing Your Tokens

1. Settings → CI/CD Integration
2. Active Tokens table shows:
   • Token name and creation date
   • Last used timestamp
   • Associated test plan
   • Revocation option

Rotating Credentials

To rotate/revoke old credentials:

1. Create new token
2. Update CI/CD platform to use new token
3. Wait 24 hours (keep old token active during transition)
4. Revoke old token

Webhook Secret Verification

BugBrain signs all webhook requests with HMAC-SHA256. Verify in your pipeline:

Python Example:

import hmac
import hashlib
 
def verify_webhook(payload, signature, secret):
    expected_sig = hmac.new(
        secret.encode(),
        payload.encode(),
        hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(signature, expected_sig)

JavaScript Example:

const crypto = require('crypto');
 
function verifyWebhook(payload, signature, secret) {
  const expectedSig = crypto
    .createHmac('sha256', secret)
    .update(payload)
    .digest('hex');
  return crypto.timingSafeEqual(
    signature,
    expectedSig
  );
}

Node.js/Express Example:

app.post('/webhook/bugbrain', express.raw({type: 'application/json'}), (req, res) => {
  const signature = req.headers['x-bugbrain-signature'];
  const payload = req.body.toString();
 
  const expectedSig = crypto
    .createHmac('sha256', process.env.BUGBRAIN_WEBHOOK_SECRET)
    .update(payload)
    .digest('hex');
 
  if (!crypto.timingSafeEqual(signature, expectedSig)) {
    return res.status(401).json({ error: 'Invalid signature' });
  }
 
  // Process webhook...
});

Test Plan Selection

Each CI/CD token is associated with one test plan. To run different test plans:

Option 1: Multiple Tokens

• Create separate CI/CD tokens for each test plan
• Use different tokens in different pipeline stages

Option 2: Multiple Pipelines

• Run different workflows with different tokens
• Example: “smoke-tests” vs “full-tests”

Recommended Setup

Production Tests (High Priority)
  ├─ Smoke Tests → token_smoke_abc123
  ├─ Regression Tests → token_regression_def456
  └─ Security Tests → token_security_ghi789

Staging Tests (Lower Priority)
  └─ Full Test Suite → token_staging_jkl012

Webhook URL Format

The webhook URL has this structure:

https://api.bugbrain.tech/webhooks/ci/{project_id}/{test_plan_id}

Example:

https://api.bugbrain.tech/webhooks/ci/proj_abc123/tp_prod_xyz789

Webhook Payload (what BugBrain sends):

{
  "event": "execution.started",
  "execution_id": "exec_abc123",
  "test_plan_id": "tp_prod_xyz789",
  "project_id": "proj_abc123",
  "status": "running",
  "timestamp": "2025-03-08T10:30:00Z"
}

Webhook Events:

• execution.started — Test run initiated
• execution.progress — Tests running (periodic updates)
• execution.completed — All tests finished
• execution.failed — Error or timeout

Setting Up in Your Platform

Quick Platform Selection

GitHub Actions — GitHub native workflows

• File: .github/workflows/bugbrain-tests.yml
• Trigger: Push, Pull Request
• Integration: Simple webhook call

GitLab CI — GitLab pipeline

• File: .gitlab-ci.yml stage
• Trigger: Merge requests, commits
• Integration: YAML stage configuration

Jenkins — Jenkins pipelines

• Method: Freestyle jobs or declarative pipelines
• Trigger: Post-build hooks
• Integration: Shell/Groovy scripting

Webhook Response Format

When BugBrain receives a webhook trigger, it responds:

{
  "status": "queued",
  "execution_id": "exec_abc123def456",
  "estimated_wait": "30s",
  "polling_url": "https://api.bugbrain.tech/api/executions/exec_abc123def456/status"
}

Status Codes:

• 202 Accepted — Test queued, will start shortly
• 401 Unauthorized — Invalid or missing credentials
• 403 Forbidden — Plan exceeded quota
• 429 Too Many Requests — Rate limit exceeded
• 500 Server Error — Try again in 1 minute

Monitoring Execution from CI/CD

Polling for Results

Your CI/CD job should poll for results:

curl -H "Authorization: Bearer $BUGBRAIN_TOKEN" \
  https://api.bugbrain.tech/api/executions/exec_abc123/status

Response:

{
  "status": "completed",
  "passed_tests": 45,
  "failed_tests": 2,
  "execution_time": 120,
  "report_url": "https://app.bugbrain.tech/executions/exec_abc123"
}

Exit Codes

Your pipeline should set appropriate exit codes:

if [ "$FAILED_TESTS" -gt 0 ]; then
  exit 1  # Pipeline fails
else
  exit 0  # Pipeline succeeds
fi

Troubleshooting

Webhook Not Triggering

Issue: Test run not starting when code is pushed

• Verify webhook URL is correct (no typos)
• Check CI/CD credentials are set as environment variables
• Ensure test plan exists and is not archived
• Check BugBrain webhook logs (Settings → Webhooks → Recent Requests)

Authentication Fails

Issue: 401 Unauthorized response

• Verify CI/CD token is set correctly
• Check token hasn’t expired (create new if needed)
• Ensure token is for correct project/test plan

Rate Limiting

Issue: 429 Too Many Requests response

• Plan quota exceeded
• Upgrade to Growth+ or Pro plan for higher limits
• Reduce test execution frequency

Timeout in Pipeline

Issue: Pipeline job times out waiting for results

• Increase timeout in your CI/CD config
• Use asynchronous polling with longer intervals
• Check BugBrain service status at status.bugbrain.tech