Running Tests Locally

Local test execution

The keystone start command launches a local runner that can execute tests on your machine, providing full visibility into test execution and powerful debugging capabilities.

Starting the runner

Basic usage

# Start the local runner
keystone start

This starts a WebSocket server on port 9223 that can receive and execute test commands. The runner provides:

Visual test execution with browser UI
Real-time step progression
Network request capture
Console log collection
Screenshot generation
Frame/screencast recording

Runner modes

Local mode (default)

keystone start

Direct connection from Keystone Studio running locally
Browser runs on your machine with full UI
Best for development and debugging
Access to localhost and private networks

Proxy mode (cloud tunnel)

keystone start --proxy --api-key your-api-key

Secure tunnel to cloud-based Keystone Studio
Allows cloud recording against local environments
Best for team collaboration and recording
Maintains cloud test management with local execution

Headless mode

keystone start --headless

Browser runs without UI for faster execution
Perfect for CI/CD and automated testing
Reduced resource usage
Still captures screenshots and recordings

Command options

Port configuration

# Use custom port
keystone start --port 9224

# Check available ports
keystone start --port 0  # Auto-assign available port

Debug and logging

# Enable debug logging
keystone start --debug

# Combine with other options
keystone start --debug --headless --port 9224

API and backend settings

# Custom backend URL
keystone start --backend-url https://custom-api.com

# Proxy mode with API key
keystone start --proxy --api-key $KEYSTONE_API_KEY

# Override environment variables
keystone start --backend-url $CUSTOM_BACKEND

Execution capabilities

What the runner captures

Visual elements:

Screenshots before and after each action
Full page screenshots on errors
Element highlighting and selection
Frame recordings of test execution

Network activity:

HTTP requests and responses
WebSocket connections
API calls and responses
Network timing and performance metrics

Browser state:

Console logs and errors
Local storage and session storage
Cookies and authentication state
Page source and DOM snapshots

Performance data:

Page load times
JavaScript execution time
Memory usage patterns
Resource loading metrics

Browser management

The runner automatically:

Launches Chrome/Chromium browser
Manages browser lifecycle
Handles browser crashes and recovery
Cleans up browser data between tests
Manages multiple tabs and windows

Session handling

# Health check endpoint
curl http://localhost:9223/health

# Response includes:
{
  "status": "healthy",
  "capabilities": ["recording", "execution", "screenshots"],
  "version": "0.1.1",
  "browser": "chrome-118.0.5993.88"
}

Local development workflow

Development testing cycle

# Terminal 1: Start your app
npm run dev
# App running on http://localhost:3000

# Terminal 2: Start Keystone runner
keystone start --proxy --debug
# Runner ready on ws://localhost:9223

# Browser: Use Keystone Studio
# Record tests against http://localhost:3000
# Tests execute in real-time with full visibility

Debugging test execution

# Start with maximum debugging
keystone start --debug --headed

# Watch tests execute with:
# - Step-by-step browser actions
# - Console output in terminal
# - Network request logs
# - Screenshot capture at each step

Testing different environments

# Test against local development
export APP_URL="http://localhost:3000"
keystone start --proxy

# Test against staging
export APP_URL="https://staging.example.com"
keystone start --proxy

# Test against review apps
export APP_URL="https://pr-123.preview.example.com"
keystone start --proxy

Runner API endpoints

Health and status

# Check runner health
GET http://localhost:9223/health

# Response:
{
  "status": "healthy",
  "uptime": 3600,
  "capabilities": ["recording", "execution"],
  "browser": {
    "name": "chrome",
    "version": "118.0.5993.88"
  }
}

Session management

# Start test session
POST http://localhost:9223/session/test-session-id
{
  "test": {...},
  "environment": {...}
}

# Session provides WebSocket for real-time communication

Performance and resource management

Resource requirements

Minimum requirements:

2GB RAM available
1 CPU core
100MB disk space for recordings
Port 9223 available

Recommended:

4GB+ RAM for smooth execution
2+ CPU cores for faster test execution
1GB+ disk space for extensive recordings
SSD for better I/O performance

Optimizing performance

# Reduce resource usage
keystone start --headless

# Limit recording features
keystone start --no-screenshots --no-recording

# Use faster execution
keystone start --fast-mode

Managing disk space

# Recordings stored in:
# ~/.keystone/recordings/
# ~/.keystone/screenshots/
# ~/.keystone/logs/

# Clean up old recordings
find ~/.keystone/recordings -mtime +7 -delete

# Or configure automatic cleanup
export KEYSTONE_CLEANUP_DAYS=3
keystone start

Integration with development tools

IDE integration

VS Code tasks.json:

{
  "version": "2.0.0",
  "tasks": [
    {
      "label": "Start Keystone Runner",
      "type": "shell",
      "command": "keystone",
      "args": ["start", "--debug"],
      "group": "test",
      "isBackground": true,
      "problemMatcher": []
    }
  ]
}

IntelliJ run configuration:

# Command: keystone
# Arguments: start --debug --port 9223
# Working directory: $ProjectFileDir$

Git workflow integration

# .git/hooks/post-checkout
#!/bin/bash
# Auto-restart runner when switching branches
if [ "$3" = "1" ]; then
  echo "Branch changed, restarting Keystone runner..."
  pkill -f "keystone start"
  keystone start --proxy &
fi

Docker integration

FROM node:18-slim

# Install Chrome and Keystone CLI
RUN apt-get update && apt-get install -y google-chrome-stable
RUN npm install -g @keystone/cli

# Set up runner
EXPOSE 9223
ENV KEYSTONE_HEADLESS=true
CMD ["keystone", "start", "--headless", "--port", "9223"]

Monitoring and logs

Log locations

# Runner logs
tail -f ~/.keystone/logs/runner.log

# Browser logs
tail -f ~/.keystone/logs/browser.log

# Test execution logs
tail -f ~/.keystone/logs/execution.log

Monitoring runner health

# Continuous health check
watch -n 5 'curl -s http://localhost:9223/health | jq'

# Check resource usage
ps aux | grep keystone
netstat -an | grep 9223

Troubleshooting

Common issues

Port already in use:

# Find what's using the port
lsof -i :9223

# Kill conflicting process
kill $(lsof -t -i:9223)

# Or use different port
keystone start --port 9224

Chrome launch failures:

# Check Chrome installation
which google-chrome
google-chrome --version

# Debug Chrome launch
keystone start --debug

Connection issues:

# Test runner connectivity
curl http://localhost:9223/health

# Check firewall settings
# Ensure port 9223 is accessible

Debug mode output

keystone start --debug

Shows detailed information about:

Runner initialization
Browser launch process
WebSocket connections
Test execution steps
Network requests
Performance metrics

Getting Started

Commands

​Local test execution

​Starting the runner

​Basic usage

​Runner modes

​Local mode (default)

​Proxy mode (cloud tunnel)

​Headless mode

​Command options

​Port configuration

​Debug and logging

​API and backend settings

​Execution capabilities

​What the runner captures

​Browser management

​Session handling

​Local development workflow

​Development testing cycle

​Debugging test execution

​Testing different environments

​Runner API endpoints

​Health and status

​Session management

​Performance and resource management

​Resource requirements

​Optimizing performance

​Managing disk space

​Integration with development tools

​IDE integration

​Git workflow integration

​Docker integration

​Monitoring and logs

​Log locations

​Monitoring runner health

​Troubleshooting

​Common issues

​Debug mode output

​Next steps

Recording Tests

Debugging