Test Plans

Organize related test cases into test plans and run them together. Perfect for regression testing, smoke tests, and scheduled test runs.

What are Test Plans?

Test Plans are collections of test cases that run together as a batch. Instead of running tests one by one, you can:

• Group related tests (e.g., “Login Flow”, “Checkout Tests”)
• Run all tests in the group with one click
• Schedule automatic runs (hourly, daily, weekly)
• Get aggregated results for the entire suite

Why Use Test Plans?

Without Test Plans

• Run tests manually, one at a time
• Hard to ensure all critical tests run
• Time-consuming to execute test suites
• Difficult to schedule regular testing

With Test Plans

• ✅ Run entire test suites with one click
• ✅ Ensure comprehensive coverage
• ✅ Schedule automatic runs
• ✅ Get suite-level pass/fail reports
• ✅ Integrate with CI/CD pipelines

Creating a Test Plan

1. Go to “Test Plans” in your project sidebar
2. Click “New Test Plan”
3. Enter plan details:
   • Name: Descriptive name (e.g., “Smoke Tests”, “Regression Suite”)
   • Description: What this plan tests
   • Tags: For organization
4. Click “Create”

Adding Tests to Plans

Method 1: From Test Plan

1. Open the Test Plan
2. Click “Add Tests”
3. Select tests from the list
4. Click “Add Selected”

Method 2: From Test Case

1. Open a Test Case
2. Click “Add to Test Plan”
3. Select which plan(s)
4. Click “Add”

Method 3: Bulk Selection

1. Go to Test Cases page
2. Check boxes next to tests
3. Click “Add to Test Plan”
4. Select plan
5. Click “Add”

Organizing Test Plans

Common Test Plan Types

Smoke Tests

• Quick sanity checks
• Run before every deployment
• 5-10 critical tests
• Execute in 2-5 minutes

Regression Tests

• Comprehensive coverage
• Run before releases
• All critical + important tests
• May take 30-60 minutes

Feature-Specific Plans

• Tests for one feature
• “Login & Auth”, “Shopping Cart”, “Search”
• Run when that feature changes

Environment-Specific Plans

• “Production Smoke Tests”
• “Staging Regression”
• Run on specific environments

Priority-Based Plans

• “Critical Tests” - P0 priority
• “High Priority Tests” - P1 priority
• Run based on importance

Running Test Plans

Manual Execution

1. Go to Test Plans
2. Click on a plan
3. Click “Run Test Plan”
4. All tests execute in parallel (based on your plan limits)
5. View aggregated results

Scheduled Execution

Set up automatic runs:

1. Open Test Plan
2. Click “Schedule”
3. Choose frequency:
   • Hourly
   • Daily (specify time)
   • Weekly (specify days and time)
   • Custom cron expression
4. Select time zone
5. Click “Save Schedule”

Example schedules:

• “Every day at 2 AM” - Nightly regression
• “Every Monday at 9 AM” - Weekly full suite
• “Every hour” - Continuous monitoring
• “Weekdays at 6 PM” - End of business day checks

Viewing Results

Plan-Level Results

After a test plan runs, view:

Summary:

• Total tests: 25
• Passed: 23 (92%)
• Failed: 2 (8%)
• Duration: 12 minutes

Status:

• ✅ Passed - All tests passed
• ❌ Failed - One or more tests failed
• ⚠️ Partial - Some tests couldn’t complete

Test Breakdown:

• List of all tests with individual status
• Click any test to see detailed results
• Filter by pass/fail status

History

View past executions:

• All previous runs
• Pass rate trends
• Duration trends
• Compare runs

Test Execution Order

Tests in a plan run:

In Parallel:

• Multiple tests execute simultaneously
• Based on your concurrent execution limit
• Faster completion

Order Options:

• Alphabetical (default) - By test name
• Priority - Critical tests first
• Custom - Manual ordering
• Random - Catch order-dependent bugs

Change order:

1. Open Test Plan
2. Click “Reorder Tests”
3. Drag and drop
4. Click “Save Order”

Managing Test Plans

Editing Plans

1. Open Test Plan
2. Click “Edit”
3. Update name, description, or tags
4. Click “Save Changes”

Duplicating Plans

Create a copy of a plan:

1. Open Test Plan
2. Click “Duplicate”
3. Modify as needed
4. Save as new plan

Archiving Plans

Archive unused plans:

1. Open Test Plan
2. Click “Archive”
3. Archived plans don’t appear in main list
4. Can be restored later

Deleting Plans

Best Practices

CI/CD Integration

Trigger test plans from your pipeline:

Via Webhooks

1. Set up webhook in your CI/CD tool
2. Call BugBrain API to run test plan
3. Wait for results
4. Fail build if tests fail

Via Schedule

1. Schedule test plan after deployment time
2. Get notifications in Slack/Jira
3. Team responds to failures

Learn about webhook integration →

Notifications

Get notified when plans complete:

Configure notifications:

1. Open Test Plan
2. Click “Notifications”
3. Enable/disable notification types:
   • On failure (recommended)
   • On completion (all results)
   • On first failure (stop after first fail)
4. Choose notification channels:
   • Email
   • Slack
   • Jira
   • Teams

Set up integrations →

Analytics

Track test plan metrics:

Pass Rate Trends:

• Historical pass rate
• Identify degrading quality
• See improvement over time

Execution Duration:

• How long plans take
• Identify slow tests
• Optimize for speed

Failure Patterns:

• Which tests fail most
• When failures occur
• Common failure reasons

View analytics →

Advanced Features

Test Plan Templates

Create reusable templates:

1. Create a well-organized plan
2. Save as template
3. Use template for new projects
4. Customize as needed

Conditional Execution

Run only certain tests based on:

• Changed files (via CI/CD)
• Specific tags
• Environment variables
• Previous test results

Parallel Strategies

Configure how tests run:

• Max parallel: Limit concurrent tests
• Timeout per test: Individual timeouts
• Fail fast: Stop on first failure
• Retry failed: Auto-retry failures

Quota Limits

Test plan execution counts against quotas:

Troubleshooting

Plan won’t run?

• Check concurrent execution limits
• Verify all tests are active (not archived)
• Ensure no tests are currently running
• Check quota limits

Tests skipped in plan?

• Verify tests are active
• Check if tests were deleted
• Ensure tests aren’t in draft status

Scheduled plan didn’t run?

• Verify schedule is enabled
• Check quota limits
• Review execution history for errors
• Ensure payment method is valid

Next Steps

• Create your first test plan
• Schedule automated runs
• Set up notifications
• View test analytics