Skip to content

PLAYWRIGHT_E2E_TESTING.md

Latest commit

 

History

History
353 lines (255 loc) · 8.07 KB

PLAYWRIGHT_E2E_TESTING.md

File metadata and controls

353 lines (255 loc) · 8.07 KB

Playwright E2E Testing Guide

End-to-end testing infrastructure for the Powernode platform using Playwright.

Overview

Playwright provides cross-browser E2E testing for AI functionality and other platform features. Tests are organized using the Page Object Model pattern and cover the 25 phases from the manual testing guide.

Directory Structure

/e2e/
├── fixtures/
│   ├── auth.ts           # Authentication helpers
│   └── test-data.ts      # Test data and constants
├── pages/                # Page Object Models
│   ├── login.page.ts
│   └── ai/
│       ├── providers.page.ts
│       ├── agents.page.ts
│       ├── workflows.page.ts
│       ├── conversations.page.ts
│       ├── agent-teams.page.ts
│       └── monitoring.page.ts
├── ai/                   # AI test specs
│   ├── providers.spec.ts
│   ├── agents.spec.ts
│   ├── conversations.spec.ts
│   ├── workflows.spec.ts
│   ├── agent-teams.spec.ts
│   └── monitoring.spec.ts
└── global-setup.ts       # Auth state setup

Quick Start

1. Install Playwright Browsers

npx playwright install

2. Configure Test Credentials

Tests use credentials from test-credentials.json (generated by rails db:seed).

Alternatively, set environment variables:

export TEST_USER_EMAIL="admin@powernode.test"
export TEST_USER_PASSWORD="password123"

3. Start Development Servers

sudo systemctl start powernode.target

4. Run Tests

# Run all tests
npm run test:e2e

# Run with UI (interactive mode)
npm run test:e2e:ui

# Run in headed mode (see browser)
npm run test:e2e:headed

# Run with debugging
npm run test:e2e:debug

# View HTML report
npm run test:e2e:report

Running Specific Tests

# Run specific spec file
npm run test:e2e -- e2e/ai/providers.spec.ts

# Run tests matching pattern
npm run test:e2e -- --grep "providers"

# Run only chromium
npm run test:e2e:chromium

# Run only firefox
npm run test:e2e:firefox

# Run only webkit (Safari)
npm run test:e2e:webkit

Test Coverage by Phase

Phase Spec File Coverage
Phase 1: Providers providers.spec.ts Connection testing, model sync, credentials
Phase 2: Agents agents.spec.ts CRUD, execution, history
Phase 3: Conversations conversations.spec.ts Multi-turn chat, context retention
Phase 4: Workflows workflows.spec.ts Builder, execution, validation
Phase 5: Agent Teams agent-teams.spec.ts Team creation, execution, monitoring
Phase 10: Monitoring monitoring.spec.ts Dashboard, tabs, circuit breakers
Phase 17: Messages conversations.spec.ts Rating, copy, regenerate
Phase 18: Validation workflows.spec.ts Health score, issues
Phase 19: Circuit Breakers monitoring.spec.ts States, reset

Writing New Tests

Using Page Objects

import { test, expect } from '@playwright/test';
import { ProvidersPage } from '../pages/ai/providers.page';

test.describe('AI Providers', () => {
  let providersPage: ProvidersPage;

  test.beforeEach(async ({ page }) => {
    providersPage = new ProvidersPage(page);
    await providersPage.goto();
    await providersPage.waitForReady();
  });

  test('should test provider connection', async () => {
    await providersPage.testConnection('Ollama');
    await providersPage.verifyConnectionSuccess();
  });
});

Using Test Data Fixtures

import { TEST_AGENT, uniqueTestData } from '../fixtures/test-data';

test('should create agent', async () => {
  const testAgent = uniqueTestData(TEST_AGENT); // Adds unique suffix
  await agentsPage.createAgent(testAgent);
  await agentsPage.verifyAgentExists(testAgent.name);
});

Test Data Constants

Available in e2e/fixtures/test-data.ts:

  • TEST_PROVIDER - Provider test data
  • TEST_AGENT - Agent test data
  • TEST_CONVERSATION - Conversation test data
  • TEST_WORKFLOW - Workflow test data
  • TEST_AGENT_TEAM - Agent team test data
  • TEST_CONTEXT - Context/memory test data
  • ROUTES - Frontend route constants
  • API_ENDPOINTS - API endpoint constants

Configuration

playwright.config.ts

Key configuration options:

{
  testDir: './e2e',
  fullyParallel: true,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,

  use: {
    baseURL: 'http://localhost:3001',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'on-first-retry',
  },

  projects: [
    { name: 'setup', testMatch: /global-setup\.ts/ },
    { name: 'chromium', dependencies: ['setup'] },
    { name: 'firefox', dependencies: ['setup'] },
    { name: 'webkit', dependencies: ['setup'] },
  ],
}

Environment Variables

Variable Description Default
PLAYWRIGHT_BASE_URL Frontend URL http://localhost:3001
TEST_USER_EMAIL Test user email admin@powernode.test
TEST_USER_PASSWORD Test user password password123
CI CI environment flag -

Authentication

Tests use stored authentication state for efficiency:

  1. global-setup.ts logs in once
  2. Auth state saved to e2e/.auth/user.json
  3. All test projects reuse stored state

To reset authentication:

rm -rf e2e/.auth/
npm run test:e2e

Debugging

VS Code Extension

Install "Playwright Test for VS Code" for:

  • Run tests from editor
  • Debug with breakpoints
  • View traces

Debug Mode

npm run test:e2e:debug

Opens Playwright Inspector with step-through debugging.

Traces

View traces for failed tests:

npx playwright show-trace test-results/path/to/trace.zip

Screenshots

Failed test screenshots are saved to test-results/.

CI/CD Integration

GitHub Actions Example

- name: Install Playwright
  run: npx playwright install --with-deps

- name: Run E2E Tests
  run: npm run test:e2e

- name: Upload Report
  uses: actions/upload-artifact@v3
  if: always()
  with:
    name: playwright-report
    path: playwright-report/

Best Practices

1. Use Page Objects

Encapsulate page interactions for maintainability:

// Good
await providersPage.testConnection('Ollama');

// Avoid
await page.locator('button:has-text("Test Connection")').click();

2. Wait for State, Not Time

// Good
await page.waitForSelector('[class*="success"]');

// Avoid
await page.waitForTimeout(2000);

3. Use Unique Test Data

const testAgent = uniqueTestData(TEST_AGENT);

4. Handle Empty States

const hasAgents = await page.locator('[class*="card"]').count() > 0;
const hasEmptyState = await page.locator(':text("No agents")').count() > 0;
expect(hasAgents || hasEmptyState).toBeTruthy();

5. Respect Timeouts

AI operations may take time:

await page.waitForSelector('[class*="response"]', { timeout: 60000 });

Troubleshooting

Tests Failing on CI

  1. Increase timeouts for CI
  2. Use single worker: workers: 1
  3. Check server startup times

Authentication Issues

  1. Delete e2e/.auth/ directory
  2. Verify credentials in test-credentials.json
  3. Check login page selectors

Flaky Tests

  1. Add explicit waits
  2. Use networkidle load state
  3. Handle async operations properly

Browser Installation

# Install all browsers
npx playwright install

# Install specific browser
npx playwright install chromium

Migration from Cypress

Playwright supersedes Cypress for E2E tests. Key differences:

Aspect Cypress Playwright
Syntax cy.get() page.locator()
Waits Automatic Explicit await
Multi-browser Limited Full support
Parallel Plugin required Built-in

Cypress tests remain in frontend/cypress/ for reference during migration.

Related Documentation