Webhooks Reference

BugBrain webhooks allow you to receive real-time notifications about test execution events.

Webhook Events

BugBrain supports 20+ event types:

execution.created - Test execution started
execution.queued - Test added to queue
execution.started - Tests begin running
execution.progress - Progress update
execution.completed - All tests finished
execution.failed - Execution error/timeout
test.passed - Individual test passed
test.failed - Individual test failed
discovery.started - Crawl session started
discovery.completed - Crawl finished

Webhook Payload

All webhooks include:

{
  "event": "execution.completed",
  "id": "evt_abc123xyz789",
  "timestamp": "2025-03-08T10:32:00Z",
  "data": {
    "execution_id": "exec_abc123def456",
    "test_plan_id": "tp_prod_xyz789",
    "status": "completed",
    "passed_tests": 45,
    "failed_tests": 2,
    "duration_seconds": 120
  }
}

Signature Verification

All webhooks are signed with HMAC-SHA256. Verify using your webhook secret:

Python:

import hmac
import hashlib
import json
 
def verify_webhook(payload_bytes, signature, secret):
    expected_signature = hmac.new(
        secret.encode('utf-8'),
        payload_bytes,
        hashlib.sha256
    ).hexdigest()
    return hmac.compare_digest(signature, expected_signature)

Node.js:

const crypto = require('crypto');
 
function verifyWebhook(payload, signature, secret) {
  const expectedSignature = crypto
    .createHmac('sha256', secret)
    .update(payload, 'utf-8')
    .digest('hex');
 
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(expectedSignature)
  );
}

Webhook Headers

Every request includes:

X-Bugbrain-Signature - HMAC-SHA256 signature
X-Bugbrain-Event - Event type
X-Bugbrain-Delivery - Unique delivery ID
Content-Type: application/json

Retry Logic

BugBrain retries failed deliveries with exponential backoff:

Attempt 1: Immediately
Attempt 2: After 5 seconds
Attempt 3: After 25 seconds
Attempt 4: After 2 minutes
Attempt 5: After 10 minutes
Attempt 6: After 1 hour
Attempt 7: After 5 hours

Total retry window: approximately 6 hours

Idempotency

Use the X-Bugbrain-Delivery header for deduplication:

delivered_webhooks = set()
 
@app.post('/webhooks/bugbrain')
def handle_webhook(request):
    delivery_id = request.headers.get('X-Bugbrain-Delivery')
 
    if delivery_id in delivered_webhooks:
        return {'status': 'already_processed'}, 200
 
    process_event(request.json)
    delivered_webhooks.add(delivery_id)
 
    return {'status': 'ok'}, 200

Response Requirements

Your webhook endpoint must:

Return 2xx status code within 30 seconds
Handle duplicate deliveries (use delivery ID)
Not retry internally (BugBrain handles retries)
Not block on slow operations (process async)

Common Use Cases

Create Jira Issue on Failure:

if payload['event'] == 'execution.completed':
    if payload['data']['failed_tests'] > 0:
        jira.create_issue(
            project='QA',
            summary=f"BugBrain: {payload['data']['failed_tests']} failures"
        )

Post to Slack:

if payload['event'] == 'execution.completed':
    message = f"✅ {payload['data']['passed_tests']} passed, " \
              f"❌ {payload['data']['failed_tests']} failed"
    slack.post_message(channel='#qa', text=message)

Update CI Status:

if payload['event'] == 'execution.completed':
    if 'ci_metadata' in payload['data']:
        ci = payload['data']['ci_metadata']
        status = 'success' if payload['data']['failed_tests'] == 0 else 'failure'
        github.update_check_run(
            repo=ci['repo'],
            head_sha=ci['commit_sha'],
            status=status
        )

Debugging: Use webhook.site during development to inspect payloads and test signature verification.

Discovery API