Skip to main content

​
Overview

Manual execution is perfect for test development, debugging, and ad-hoc validation. Run tests interactively with full control over execution.

​
Keystone Studio

​
Quick Start

1

Open Studio

Navigate to your test in Keystone Studio
2

Select Environment

Choose between development, staging, or production
3

Run Test

Click “Run Test” and watch real-time execution
4

Review Results

Analyze results with video playback and logs

​
Execution Options

Run individual tests for quick validation
  • Ideal for debugging specific issues
  • Fast feedback loop
  • Interactive breakpoints available

​
CLI Execution

For local development and automation:

​
Basic Commands

# Start local runner
keystone-runner start

# Run a specific test
keystone run --test "Login Flow"

# Run a test suite
keystone run --suite "Smoke Tests"

# Run with specific environment
keystone run --env staging --test "Checkout"

​
Advanced Options

# Debug mode with headed browser
keystone run --test "Login" --debug --headed

# Custom timeout and retries
keystone run --suite "Critical" --timeout 60000 --retries 3

# With environment variables
BASE_URL=http://localhost:3000 keystone run --test "Local Test"

​
Environment Configuration

​
Studio Configuration

Configure environments in the Keystone UI: 
{
  "name": "development",
  "baseUrl": "http://localhost:3000",
  "variables": {
    "slowMo": 100,
    "headless": false
  }
}

{
  "name": "staging", 
  "baseUrl": "https://staging.example.com",
  "auth": {
    "username": "[email protected]",
    "password": "$STAGING_PASSWORD"
  }
}

​
CLI Configuration

Create .keystone/config.json: 
{
  "runner": {
    "port": 9223,
    "headless": false
  },
  "environments": {
    "local": {
      "baseUrl": "http://localhost:3000"
    }
  }
}

​
Debugging Features

​
Studio Debugging

  • Breakpoints: Click step numbers to add breakpoints
  • Variable Inspector: View all test variables in real-time
  • Step Over/Into: Control execution flow
  • Console Access: Execute JavaScript in test context

​
CLI Debugging

# Enable verbose logging
keystone run --test "Login" --verbose

# Save debug artifacts
keystone run --test "Checkout" --artifacts ./debug-output

# Interactive REPL mode
keystone run --test "Complex Flow" --interactive

​
Best Practices

Use Debug Mode

Enable debug mode when developing new tests to see exactly what’s happening

Start Small

Test individual steps before running entire suites

Save Artifacts

Always save artifacts when debugging failures

Use Variables

Parameterize tests for easy environment switching

​
Common Use Cases

​
Development Workflow

# 1. Start local app
npm run dev

# 2. Start Keystone runner
keystone-runner start

# 3. Run tests against local
keystone run --env local --test "New Feature"

​
Debugging Failed CI Tests

# Download artifacts from CI
keystone artifacts download --run-id abc123

# Replay locally with same config
keystone replay --artifacts ./artifacts/abc123

​
Quick Validation

Perfect for:
  • Verifying bug fixes
  • Testing new features
  • Exploratory testing
  • Demo preparation

​
Troubleshooting

  • Verify runner is started: keystone-runner status
  • Check firewall settings for port 9223
  • Try connecting to 127.0.0.1 instead of localhost
  • Enable verbose logging: --verbose
  • Check browser compatibility
  • Verify environment configuration
  • Review network connectivity

​
Next Steps

Debug Tests

Master debugging techniques

Automate

Set up automated execution