Skip to content

chore(tests): introduce Playwright cross-browser regression testing - #449

Open
harshbhx wants to merge 7 commits into
yegor256:masterfrom
harshbhx:playwright-setup
Open

chore(tests): introduce Playwright cross-browser regression testing#449
harshbhx wants to merge 7 commits into
yegor256:masterfrom
harshbhx:playwright-setup

Conversation

@harshbhx

Copy link
Copy Markdown

Summary

This PR introduces a fully automated, CI driven cross browser testing setup using [Playwright](https://playwright.dev) for Tacit. It covers smoke tests, computed style assertions, and visual regression tests across Chromium, Firefox, and WebKit, all without requiring contributors to install browsers or generate snapshots locally.

Closes #15

Motivation

Tacit is a classless CSS framework, which means every style change can unintentionally affect the rendered output of any HTML element. Right now, visual regressions are only caught through manual inspection. Automated cross browser coverage gives maintainers confidence that design changes render correctly everywhere, and makes future contributions safer for everyone involved.

What Changed

Dependencies

  • Added @playwright/test as a devDependency.

Configuration

  • Created playwright.config.js defining three projects: chromium, firefox, and webkit, along with a built in static server using python -m http.server.

Test Suites

  • tests/tacit.spec.js: a basic smoke test that verifies the index page loads.
  • tests/styles.spec.js: deterministic computed style assertions for headings, tables, inputs, links, and strong text.
  • tests/visual.spec.js: screenshot regression tests covering the full page, headings, table, and form components.

Continuous Integration

  • Added .github/workflows/playwright.yml, which installs dependencies, builds the CSS, installs Playwright browsers, generates Linux baselines when missing, runs the full test suite, and uploads the Playwright report if anything fails.
  • Added .github/workflows/playwright update snapshots.yml, a manual workflow_dispatch workflow for regenerating baselines when a visual change is intentional.

Developer Experience

  • Added convenience scripts to package.json:
    • npm run playwright: runs the full suite.
    • npm run playwright:update: updates baselines.
  • Updated .gitignore to exclude test-results/ and playwright-report/.
  • Added a Playwright CI badge to README.md.

How It Works: CI Driven Baselines

Screenshot baselines are generated on Linux inside the GitHub Action runner (ubuntu-24.04), so contributors never need to run --update-snapshots locally or install any browsers themselves.

  1. On the first run, if tests/visual.spec.js-snapshots does not exist yet, the action generates baseline screenshots, commits them, and pushes back to the same branch.
  2. On every subsequent run, the action compares new screenshots against the existing baselines.
  3. When a visual change is intentional, maintainers can trigger the manual update playwright snapshots workflow to refresh all baselines at once.

Test Plan

To verify the setup locally:

npm install
npx playwright install
npm run playwright

# to update baselines instead, if needed:
npm run playwright:update

The Playwright GitHub Action also runs automatically on every push, so no manual step is required once this PR is merged.

Checklist

  • Tests pass locally across all three browser projects.
  • CI workflow runs successfully on a clean branch with no existing baselines.
  • Manual snapshot update workflow tested and confirmed working.
  • Documentation and README badge updated.
  • No hardcoded paths or machine specific assumptions in config.

harshbhx and others added 6 commits July 22, 2026 00:34
This sets up @playwright/test with a minimal GitHub Actions workflow
that runs a smoke test on Chromium, Firefox, and WebKit. It builds the
demo CSS with grunt sass:dev before testing so index.html renders
properly. This is the first step toward automating cross-browser
testing for issue yegor256#15.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
  Sets up Chromium, Firefox, and WebKit screenshot tests for index.html
  and key components, plus deterministic computed-style assertions. The
  GitHub Actions workflow auto-generates Linux baselines on the first
  push and re-runs tests; a separate workflow_dispatch job lets
  maintainers update baselines after intentional visual changes.

  Closes yegor256#15.
Copilot AI review requested due to automatic review settings July 21, 2026 20:27

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Playwright-based cross-browser regression testing to Tacit, including computed-style assertions and visual snapshot testing, and wires it into GitHub Actions so regressions are detected automatically.

Changes:

  • Introduces Playwright test suites for smoke, computed styles, and visual regression.
  • Adds Playwright configuration + npm scripts to run/update tests.
  • Adds CI workflows to run Playwright and to manually refresh visual baselines; updates docs and gitignore accordingly.

Reviewed changes

Copilot reviewed 8 out of 21 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/visual.spec.js Adds visual snapshot tests for the index page and key components.
tests/tacit.spec.js Adds a cross-browser smoke test for page load/title.
tests/styles.spec.js Adds computed-style assertions to catch CSS regressions deterministically.
playwright.config.js Configures Playwright projects (Chromium/Firefox/WebKit) and a local web server.
package.json Adds @playwright/test and convenience scripts for running/updating tests.
.github/workflows/playwright.yml Adds CI workflow to install deps, build CSS, run Playwright, and upload reports on failure.
.github/workflows/playwright-update-snapshots.yml Adds manual workflow to regenerate/commit snapshot baselines.
.gitignore Ignores Playwright output directories (test-results/, playwright-report/).
README.md Documents the Playwright setup and adds a CI badge.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread package.json Outdated
Comment thread playwright.config.js
Comment thread tests/visual.spec.js
Comment thread README.md
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

automate cross browser testing

2 participants