Skip to main content
In betaThe Keystone GitHub App provides the easiest way to integrate testing into your GitHub workflow with automatic test management and no configuration required. This will be ready for general usage soon.

​
Why GitHub App?

Zero Configuration

  • No API keys in your repo
  • Suite selection in Keystone UI
  • Automatic branch detection
  • Smart test promotion

Branch-based Testing

  • Tests track their creation branch
  • Automatic promotion on PR merge
  • Dev overrides for staging URLs
  • Production-ready test management

PR Status Checks

  • Automatic check creation
  • Real-time status updates
  • Detailed failure reports
  • Direct links to test results

Test Isolation

  • Branch-specific test runs
  • Staging URL overrides
  • Clean test promotion
  • No test pollution

​
Quick Start

1

Install GitHub App

Visit the GitHub Marketplace and install the Keystone Tests app to your organization or repository.
2

Authorize & Configure

After installation, you’ll be redirected to Keystone to:
  • Select your default test suite
  • Enable PR testing
  • Configure branch promotion rules
3

Add Workflow

Create .github/workflows/keystone.yml:
name: Deploy and Test

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  deploy-and-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Deploy to Vercel
        id: deploy
        uses: vercel/action@v1
        # ... your existing deployment

      - name: Run Keystone Tests
        uses: keystone-platform/test-action@v1
        with:
          staging_url: ${{ steps.deploy.outputs.deployment_url }}
          # Suite ID comes from your Keystone configuration!
4

Create & Merge

Tests created on feature branches automatically promote to production when PRs merge.

​
How It Works

​
Development Workflow

​
Test Promotion

When a PR is merged, all tests created on that branch are automatically:
  1. Promoted to production status - marked as stable
  2. Cleared of dev overrides - staging URLs removed
  3. Ready for monitoring - available for scheduled runs

​
Configuration

​
In Keystone UI

Configure your GitHub App settings:
  • Default Suite: Which test suite runs on PRs
  • Branch Rules: Which branches trigger tests
  • Required Checks: Make tests required for merging

​
In Your Repository

The only configuration needed in your repo: 
# .github/workflows/keystone.yml
- name: Run Keystone Tests
  uses: keystone-platform/test-action@v1
  with:
    staging_url: ${{ steps.deploy.outputs.url }}
Optional parameters:
  • suite_id: Override the default suite for specific workflows
  • wait_for_results: Set to false for async execution
  • fail_on_error: Set to false to allow PR merge despite test failures

​
Security

The GitHub App requires minimal permissions:
  • Read: Repository metadata, pull requests
  • Write: Commit statuses, checks
  • No code access: We never read your source code
  • OAuth flow for initial setup
  • App installation tokens for API calls
  • No long-lived credentials stored
  • Automatic token rotation

​
Common Patterns

​
Feature Development

# Developer workflow
git checkout -b feature/new-checkout

# Write tests that automatically use preview URLs
# Tests are tagged with branch name

# Open PR - tests run automatically
# No configuration needed!

# Merge PR - tests promoted to production
# Automatic cleanup of dev overrides

​
Monitoring Setup

Once tests are promoted:
  1. They run on production URLs
  2. Available for scheduled execution
  3. Can be included in monitoring suites
  4. Alert on failures

​
Troubleshooting

Common IssuesApp not running tests
  • Check GitHub App is installed on the repository
  • Verify webhook delivery in GitHub settings
  • Ensure workflow file exists
Wrong suite running
  • Update default suite in Keystone settings
  • Check for suite_id override in workflow
Tests not promoting
  • Ensure PR was merged (not just closed)
  • Check branch promotion rules in settings
  • Verify tests were created on feature branch

​
Next Steps

Configure Suites

Organize tests into logical groups

View Results

Monitor test execution in the dashboard