Executions API

The Executions API allows you to programmatically run tests, monitor their progress, and retrieve results.

Base URL

https://api.bugbrain.tech/api/v1

Authentication

All requests require a Bearer token:

curl https://api.bugbrain.tech/api/v1/executions \
  -H "Authorization: Bearer bugbrain_sk_prod_..."

Endpoints

List Executions

Retrieve all test executions for a project

GET /executions?project_id=proj_abc123&limit=20&offset=0

Parameters:

Parameter	Type	Required	Description
`project_id`	string	Yes	Project ID
`limit`	integer	No	Results per page (default: 20, max: 100)
`offset`	integer	No	Pagination offset (default: 0)
`status`	string	No	Filter by status: pending, running, completed, failed
`created_after`	ISO 8601	No	Filter by creation date
`created_before`	ISO 8601	No	Filter by creation date

Example Request:

curl "https://api.bugbrain.tech/api/v1/executions?project_id=proj_abc123&limit=10" \
  -H "Authorization: Bearer $API_KEY"

Response:

{
  "data": [
    {
      "id": "exec_abc123",
      "test_plan_id": "tp_prod_xyz",
      "project_id": "proj_abc123",
      "status": "completed",
      "passed_tests": 45,
      "failed_tests": 2,
      "total_tests": 47,
      "duration_seconds": 120,
      "created_at": "2025-03-08T10:30:00Z",
      "completed_at": "2025-03-08T10:32:00Z"
    }
  ],
  "pagination": {
    "limit": 10,
    "offset": 0,
    "total": 150
  }
}

Create Execution (Sync)

Run a test plan synchronously (wait for results)

POST /executions

Request Body:

{
  "test_plan_id": "tp_prod_xyz789",
  "project_id": "proj_abc123",
  "timeout_seconds": 300,
  "wait_for_completion": true
}

Parameters:

Parameter	Type	Required	Description
`test_plan_id`	string	Yes	Test plan to execute
`project_id`	string	Yes	Project ID
`timeout_seconds`	integer	No	Max execution time (default: 300)
`wait_for_completion`	boolean	No	Block until complete (default: true)
`environment`	string	No	Override environment (staging/prod)
`variables`	object	No	Runtime variables (key-value pairs)

Example Request:

curl -X POST https://api.bugbrain.tech/api/v1/executions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "test_plan_id": "tp_prod_xyz",
    "project_id": "proj_abc123",
    "timeout_seconds": 300
  }'

Response (Completed):

{
  "id": "exec_abc123def456",
  "test_plan_id": "tp_prod_xyz",
  "project_id": "proj_abc123",
  "status": "completed",
  "passed_tests": 45,
  "failed_tests": 2,
  "total_tests": 47,
  "duration_seconds": 120,
  "created_at": "2025-03-08T10:30:00Z",
  "completed_at": "2025-03-08T10:32:00Z",
  "results": {
    "test_results": [
      {
        "test_id": "test_1",
        "name": "Login Flow",
        "status": "passed",
        "duration_ms": 2500
      },
      {
        "test_id": "test_2",
        "name": "Add to Cart",
        "status": "failed",
        "duration_ms": 3200,
        "error": "Element not found: .add-to-cart-btn"
      }
    ],
    "summary": {
      "total": 47,
      "passed": 45,
      "failed": 2,
      "skipped": 0
    }
  }
}

Create Execution (Async)

Run a test plan asynchronously (return immediately)

POST /executions/async

Request Body:

{
  "test_plan_id": "tp_prod_xyz789",
  "project_id": "proj_abc123"
}

Response (Queued):

{
  "id": "exec_xyz789abc",
  "status": "queued",
  "estimated_wait_seconds": 30,
  "polling_url": "/api/v1/executions/exec_xyz789abc/status"
}

Your application should poll the polling_url to check for completion.

Get Execution Status

Check execution progress and status

GET /executions/{execution_id}/status

Parameters:

Parameter	Type	Required	Description
`execution_id`	string	Yes	Execution ID from create response

Example Request:

curl https://api.bugbrain.tech/api/v1/executions/exec_abc123/status \
  -H "Authorization: Bearer $API_KEY"

Response (Running):

{
  "id": "exec_abc123",
  "status": "running",
  "progress": {
    "current": 32,
    "total": 47,
    "percent": 68
  },
  "started_at": "2025-03-08T10:30:00Z",
  "estimated_completion": "2025-03-08T10:32:30Z"
}

Response (Completed):

{
  "id": "exec_abc123",
  "status": "completed",
  "passed_tests": 45,
  "failed_tests": 2,
  "total_tests": 47,
  "duration_seconds": 120,
  "completed_at": "2025-03-08T10:32:00Z"
}

Get Execution Results

Retrieve detailed test results and logs

GET /executions/{execution_id}/results

Response:

{
  "id": "exec_abc123",
  "test_plan_id": "tp_prod_xyz",
  "test_results": [
    {
      "test_id": "test_1",
      "name": "Login Flow",
      "status": "passed",
      "duration_ms": 2500,
      "steps": [
        {
          "step_id": "step_1",
          "name": "Navigate to login page",
          "status": "passed"
        },
        {
          "step_id": "step_2",
          "name": "Enter credentials",
          "status": "passed"
        },
        {
          "step_id": "step_3",
          "name": "Click login button",
          "status": "passed"
        }
      ]
    },
    {
      "test_id": "test_2",
      "name": "Add to Cart",
      "status": "failed",
      "duration_ms": 3200,
      "error": "Element not found: .add-to-cart-btn",
      "screenshot": "https://cdn.bugbrain.tech/screenshots/exec_abc123/test_2.png"
    }
  ]
}

Get Failure Analysis

Retrieve AI-generated failure analysis (if available)

GET /executions/{execution_id}/failure-analysis

Response:

{
  "id": "exec_abc123",
  "analysis": [
    {
      "test_id": "test_2",
      "test_name": "Add to Cart",
      "failure_category": "selector_changed",
      "confidence": 0.95,
      "description": "The '.add-to-cart-btn' selector no longer exists in the DOM. The button may have been renamed or removed.",
      "evidence": {
        "expected_selector": ".add-to-cart-btn",
        "suggested_alternative": ".product-add-btn",
        "screenshot": "https://cdn.bugbrain.tech/screenshots/exec_abc123/test_2_failure.png"
      },
      "fix_suggestions": [
        "Update test selector from '.add-to-cart-btn' to '.product-add-btn'",
        "Verify the button is visible in the current version of the application"
      ]
    }
  ]
}

Cancel Execution

Stop a running test execution

DELETE /executions/{execution_id}

Example Request:

curl -X DELETE https://api.bugbrain.tech/api/v1/executions/exec_abc123 \
  -H "Authorization: Bearer $API_KEY"

Response:

{
  "id": "exec_abc123",
  "status": "cancelled",
  "cancellation_reason": "requested",
  "cancelled_at": "2025-03-08T10:31:00Z"
}

Status Values

Status	Meaning	Terminal
`queued`	Waiting to start	No
`pending`	Initializing	No
`running`	Tests executing	No
`completed`	All tests finished (may have failures)	Yes
`failed`	Execution error (crashed, timeout)	Yes
`cancelled`	User-requested cancellation	Yes

Polling Example

Python:

import time
import requests
 
def run_test_async(test_plan_id, project_id):
    # Start execution
    response = requests.post(
        'https://api.bugbrain.tech/api/v1/executions/async',
        headers={'Authorization': f'Bearer {api_key}'},
        json={
            'test_plan_id': test_plan_id,
            'project_id': project_id
        }
    )
    execution_id = response.json()['id']
    print(f"Started execution: {execution_id}")
 
    # Poll for results
    max_wait = 600  # 10 minutes
    elapsed = 0
 
    while elapsed < max_wait:
        response = requests.get(
            f'https://api.bugbrain.tech/api/v1/executions/{execution_id}/status',
            headers={'Authorization': f'Bearer {api_key}'}
        )
        status = response.json()
 
        if status['status'] in ['completed', 'failed', 'cancelled']:
            return status
 
        progress = status.get('progress', {})
        print(f"Progress: {progress.get('percent', 0)}%")
 
        time.sleep(5)
        elapsed += 5
 
    raise TimeoutError(f"Test execution did not complete within {max_wait}s")

JavaScript:

async function runTestAsync(testPlanId, projectId) {
  // Start execution
  const startResponse = await fetch(
    'https://api.bugbrain.tech/api/v1/executions/async',
    {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${apiKey}`,
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({
        test_plan_id: testPlanId,
        project_id: projectId
      })
    }
  );
 
  const startData = await startResponse.json();
  const executionId = startData.id;
  console.log(`Started execution: ${executionId}`);
 
  // Poll for results
  const maxWait = 600; // 10 minutes
  let elapsed = 0;
 
  while (elapsed < maxWait) {
    const statusResponse = await fetch(
      `https://api.bugbrain.tech/api/v1/executions/${executionId}/status`,
      {
        headers: { 'Authorization': `Bearer ${apiKey}` }
      }
    );
 
    const status = await statusResponse.json();
 
    if (['completed', 'failed', 'cancelled'].includes(status.status)) {
      return status;
    }
 
    console.log(`Progress: ${status.progress.percent}%`);
 
    await new Promise(resolve => setTimeout(resolve, 5000));
    elapsed += 5;
  }
 
  throw new Error(`Test execution did not complete within ${maxWait}s`);
}

Recommended Pattern: Use async execution with polling for CI/CD integrations. This allows your pipeline to maintain control and set appropriate timeouts.

Authentication Test Cases API