Visual Regression Setup

You are a QA automation engineer. The user wants to set up visual regression testing that captures screenshots and compares them against baselines to detect unintended visual changes.

What to check first

• Run npm list playwright or npm list puppeteer to confirm a browser automation library is installed
• Verify you have a baseline screenshots directory structure planned (e.g., ./tests/baselines/ and ./tests/screenshots/)
• Check if your project has a test runner configured (Jest, Mocha, or Playwright's built-in runner)

Steps

1. Install Playwright (recommended for visual testing) with npm install --save-dev @playwright/test if not already present
2. Create a baseline directory at ./tests/baselines/ where initial reference screenshots will be stored
3. Create a helper function that uses page.screenshot() to capture full-page or element-specific screenshots
4. Set up a comparator utility using pixelmatch or resemble.js to diff current screenshots against baselines with configurable threshold (typically 1-2% variance allowed)
5. Configure a test fixture that initializes the browser context with consistent viewport size (e.g., 1920x1080) to ensure reproducible captures
6. Create a baseline update workflow — add a CLI flag like --update-baselines that overwrites baseline images instead of comparing
7. Integrate snapshot storage in CI/CD — store baselines as artifacts and download them in pipeline before test runs
8. Add a report generator that outputs HTML with side-by-side diff images showing what changed

Code

// visual-regression-helper.js
const { test, expect } = require('@playwright/test');
const fs = require('fs');
const path = require('path');
const pixelmatch = require('pixelmatch');
const PNG = require('pngjs').PNG;

const BASELINE_DIR = './tests/baselines';
const ACTUAL_DIR = './tests/screenshots/actual';
const DIFF_DIR = './tests/screenshots/diff';

// Ensure directories exist
[BASELINE_DIR, ACTUAL_DIR, DIFF_DIR].forEach(dir => {
  if (!fs.existsSync(dir)) fs.mkdirSync(dir, { recursive: true });
});

async function captureScreenshot(page, name, options = {}) {
  const {
    fullPage = true,
    selector = null,
    updateBaseline = process.env.UPDATE_BASELINES === 'true'
  } = options;

  const screenshotOptions = { fullPage };
  if (selector) screenshotOptions.clip = await page.locator(selector).boundingBox();

  const actualPath = path.join(ACTUAL_DIR, `${name}.png`);
  const baselinePath = path.join(BASELINE_DIR, `${name}.png`);

  // Capture current screenshot
  await page.screenshot({ path: actualPath, ...screenshotOptions });

  // If updating baselines, copy actual

Note: this example was truncated in the source. See the GitHub repo for the latest full version.

Common Pitfalls

• Treating this skill as a one-shot solution — most workflows need iteration and verification
• Skipping the verification steps — you don't know it worked until you measure
• Applying this skill without understanding the underlying problem — read the related docs first

When NOT to Use This Skill

• When a simpler manual approach would take less than 10 minutes
• On critical production systems without testing in staging first
• When you don't have permission or authorization to make these changes

How to Verify It Worked

• Run the verification steps documented above
• Compare the output against your expected baseline
• Check logs for any warnings or errors — silent failures are the worst kind

Production Considerations

• Test in staging before deploying to production
• Have a rollback plan — every change should be reversible
• Monitor the affected systems for at least 24 hours after the change