Playwright Visual Regression Testing

This chapter introduces Playwright's visual regression testing features, detecting unexpected UI changes through screenshot comparison.

What is Visual Regression Testing

Visual Regression Testing compares the current screenshot withthe baseline screenshot (Golden File)to detect UI changes on the page through pixel differences.

When a CSS change unexpectedly affects the layout of other pages, visual regression testing can directly catch this issue.

expect(page).toHaveScreenshot() Page Screenshot Comparison

Example

test('EXAMPLE Homepage Visual Regression', async ({ page }) => {
  await page.goto('https://www.example.com/');

  // First run: save baseline screenshot
  // Subsequent runs: compare with baseline screenshot
  await expect(page).toHaveScreenshot('example-homepage.png');
});

On the first run, Playwright will report an error and generate a baseline screenshot:

Error: A snapshot doesn't exist at example-homepage-chromium-darwin.png,
writing actual.

On the second run, it will automatically compare against the baseline screenshot.

Complete toHaveScreenshot Options

Example

await expect(page).toHaveScreenshot('homepage.png', {
  fullPage: true,                // Full-page screenshot
  maxDiffPixelRatio: 0.01,       // Allow 1% pixel difference
  maxDiffPixels: 100,            // Allow up to 100 pixel differences
  threshold: 0.2,                // Pixel difference threshold (0~1)
  animations: 'disabled',        // Disable animations (recommended)
  mask: [                        // Ignore specific areas
    page.getByText('Current Time'),
  ],
});

expect(locator).toHaveScreenshot() Element Screenshot Comparison

Example

test('Navigation Bar Visual Regression', async ({ page }) => {
  await page.goto('https://www.example.com/');

  // Only compare the navigation bar screenshot
  const navbar = page.getByRole('navigation');
  await expect(navbar).toHaveScreenshot('navbar.png');
});

Golden File Management

Generate Golden File

On the first run of a screenshot assertion, Playwright automatically generates a Golden File and saves it in the-snapshotsdirectory next to the test file:

tests/
├── example.spec.ts
└── example.spec.ts-snapshots/
    └── example-homepage-chromium-darwin.png

The file name contains the browser name and operating system, ensuring baselines for different platforms are managed independently.

Update Golden File

When the UI undergoes expected design changes, you need to update the baseline screenshot:

# 更新所有截图基准
npx playwright test --update-snapshots

# 只更新特定文件
npx playwright test tests/example.spec.ts --update-snapshots

Visual Comparison Options Explained

OptionTypeDescription
maxDiffPixelRationumberAllowed pixel difference ratio (0~1), default 0
maxDiffPixelsnumberAllowed number of pixel differences, default 0
thresholdnumberPer-pixel difference tolerance (0~1), default 0.2
animationsstringWhether to disable animations:'disabled' | 'allow'
maskLocator[]Ignored areas (not considered during difference comparison)

CI Environment Considerations

Browser rendering results are affected by factors such as the operating system, browser version, hardware, power state, and headless mode. To ensure consistency, generate Golden Files and run tests in the same environment. It is recommended to use a Docker container or the same platform as CI to eliminate differences.

Visual Testing in Docker

# 使用 Playwright Docker 镜像确保渲染一致性
docker run --rm -v $(pwd):/work -w /work \
  mcr.microsoft.com/playwright:v1.52.0-jammy \
  npx playwright test --update-snapshots

Best Practices for Visual Testing

  • Generate Golden Files and run comparisons on the same platform
  • Useanimations: 'disabled'to eliminate animation uncertainty
  • Usemaskto ignore dynamic content (such as timestamps, ad slots)
  • Prioritize visual regression testing for critical pages (homepage, login, payment)
  • Commit Golden Files to the version control system
  • Use fixed versions of browsers and operating systems in CI
Other Extensions