2021-05-28 06:30:03 +03:00
---
id: test-retries
title: "Test retry"
---
2022-09-07 13:30:31 +03:00
Test retries are a way to automatically re-run a test when it fails. This is useful when a test is flaky and fails intermittently. Test retries are configured in the [configuration file ](./test-configuration.md ).
2021-09-08 17:44:26 +03:00
## Failures
Playwright Test runs tests in worker processes. These processes are OS processes, running independently, orchestrated by the test runner. All workers have identical environments and each starts its own browser.
Consider the following snippet:
2022-06-11 03:34:31 +03:00
```js tab=js-js
2021-09-08 17:44:26 +03:00
const { test } = require('@playwright/test');
test.describe('suite', () => {
2021-11-20 00:47:30 +03:00
test.beforeAll(async () => { /* ... */ });
2021-09-08 17:44:26 +03:00
test('first good', async ({ page }) => { /* ... */ });
test('second flaky', async ({ page }) => { /* ... */ });
test('third good', async ({ page }) => { /* ... */ });
});
```
2022-06-11 03:34:31 +03:00
```js tab=js-ts
2021-09-08 17:44:26 +03:00
import { test } from '@playwright/test';
test.describe('suite', () => {
2021-11-20 00:47:30 +03:00
test.beforeAll(async () => { /* ... */ });
2021-09-08 17:44:26 +03:00
test('first good', async ({ page }) => { /* ... */ });
test('second flaky', async ({ page }) => { /* ... */ });
test('third good', async ({ page }) => { /* ... */ });
});
```
When **all tests pass** , they will run in order in the same worker process.
* Worker process starts
2021-11-20 00:47:30 +03:00
* `beforeAll` hook runs
2021-09-08 17:44:26 +03:00
* `first good` passes
* `second flaky` passes
* `third good` passes
Should **any test fail** , Playwright Test will discard the entire worker process along with the browser and will start a new one. Testing will continue in the new worker process starting with the next test.
* Worker process #1 starts
2021-11-20 00:47:30 +03:00
* `beforeAll` hook runs
2021-09-08 17:44:26 +03:00
* `first good` passes
* `second flaky` fails
* Worker process #2 starts
2021-11-20 00:47:30 +03:00
* `beforeAll` hook runs again
2021-09-08 17:44:26 +03:00
* `third good` passes
If you **enable [retries](#retries)** , second worker process will start by retrying the failed test and continue from there.
* Worker process #1 starts
2021-11-20 00:47:30 +03:00
* `beforeAll` hook runs
2021-09-08 17:44:26 +03:00
* `first good` passes
* `second flaky` fails
* Worker process #2 starts
2021-11-20 00:47:30 +03:00
* `beforeAll` hook runs again
2021-09-08 17:44:26 +03:00
* `second flaky` is retried and passes
* `third good` passes
This scheme works perfectly for independent tests and guarantees that failing tests can't affect healthy ones.
## Retries
2021-10-15 05:28:47 +03:00
Playwright Test supports **test retries** . When enabled, failing tests will be retried multiple times until they pass, or until the maximum number of retries is reached. By default failing tests are not retried.
2021-05-28 06:30:03 +03:00
2021-06-02 19:23:06 +03:00
```bash
2021-09-08 17:44:26 +03:00
# Give failing tests 3 retry attempts
2021-05-28 06:30:03 +03:00
npx playwright test --retries=3
```
2022-06-11 03:34:31 +03:00
```js tab=js-js
2021-05-31 05:46:16 +03:00
// playwright.config.js
2021-06-21 20:56:30 +03:00
// @ts -check
/** @type {import('@playwright/test').PlaywrightTestConfig} */
const config = {
2021-09-08 17:44:26 +03:00
// Give failing tests 3 retry attempts
2021-05-31 05:46:16 +03:00
retries: 3,
};
2021-06-21 20:56:30 +03:00
module.exports = config;
2021-05-31 05:46:16 +03:00
```
2022-06-11 03:34:31 +03:00
```js tab=js-ts
2021-05-31 05:46:16 +03:00
// playwright.config.ts
2022-05-27 22:36:59 +03:00
import type { PlaywrightTestConfig } from '@playwright/test';
2021-05-31 05:46:16 +03:00
const config: PlaywrightTestConfig = {
2021-09-08 17:44:26 +03:00
// Give failing tests 3 retry attempts
2021-05-31 05:46:16 +03:00
retries: 3,
};
export default config;
```
2021-09-08 17:44:26 +03:00
Playwright Test will categorize tests as follows:
- "passed" - tests that passed on the first run;
- "flaky" - tests that failed on the first run, but passed when retried;
- "failed" - tests that failed on the first run and failed all retries.
2021-05-28 06:30:03 +03:00
2021-06-02 19:23:06 +03:00
```bash
2021-09-08 17:44:26 +03:00
Running 3 tests using 1 worker
✓ example.spec.ts:4:2 › first passes (438ms)
x example.spec.ts:5:2 › second flaky (691ms)
✓ example.spec.ts:5:2 › second flaky (522ms)
✓ example.spec.ts:6:2 › third passes (932ms)
1 flaky
example.spec.ts:5:2 › second flaky
2 passed (4s)
```
2021-11-22 21:06:20 +03:00
You can detect retries at runtime with [`property: TestInfo.retry`], which is accessible to any test, hook or fixture. Here is an example that clears some server-side state before a retry.
2022-06-11 03:34:31 +03:00
```js tab=js-js
2021-11-22 21:06:20 +03:00
const { test, expect } = require('@playwright/test');
test('my test', async ({ page }, testInfo) => {
if (testInfo.retry)
await cleanSomeCachesOnTheServer();
// ...
});
```
2022-06-11 03:34:31 +03:00
```js tab=js-ts
2021-11-22 21:06:20 +03:00
import { test, expect } from '@playwright/test';
test('my test', async ({ page }, testInfo) => {
if (testInfo.retry)
await cleanSomeCachesOnTheServer();
// ...
});
```
2022-10-28 01:53:27 +03:00
You can specify retries for a specific group of tests or a single file with [`method: Test.describe.configure`].
```js tab=js-js
const { test, expect } = require('@playwright/test');
test.describe(() => {
// All tests in this describe group will get 2 retry attempts.
test.describe.configure({ retries: 2 });
test('test 1', async ({ page }) => {
// ...
});
test('test 2', async ({ page }) => {
// ...
});
});
```
```js tab=js-ts
import { test, expect } from '@playwright/test';
test.describe(() => {
// All tests in this describe group will get 2 retry attempts.
test.describe.configure({ retries: 2 });
test('test 1', async ({ page }) => {
// ...
});
test('test 2', async ({ page }) => {
// ...
});
});
```
2021-09-08 17:44:26 +03:00
## Serial mode
Use [`method: Test.describe.serial`] to group dependent tests to ensure they will always run together and in order. If one of the tests fails, all subsequent tests are skipped. All tests in the group are retried together.
Consider the following snippet that uses `test.describe.serial` :
2022-06-11 03:34:31 +03:00
```js tab=js-js
2021-09-08 17:44:26 +03:00
const { test } = require('@playwright/test');
2022-02-03 07:44:11 +03:00
test.describe.configure({ mode: 'serial' });
test.beforeAll(async () => { /* ... */ });
test('first good', async ({ page }) => { /* ... */ });
test('second flaky', async ({ page }) => { /* ... */ });
test('third good', async ({ page }) => { /* ... */ });
2021-09-08 17:44:26 +03:00
```
2022-06-11 03:34:31 +03:00
```js tab=js-ts
2021-09-08 17:44:26 +03:00
import { test } from '@playwright/test';
2022-02-03 07:44:11 +03:00
test.describe.configure({ mode: 'serial' });
test.beforeAll(async () => { /* ... */ });
test('first good', async ({ page }) => { /* ... */ });
test('second flaky', async ({ page }) => { /* ... */ });
test('third good', async ({ page }) => { /* ... */ });
2021-09-08 17:44:26 +03:00
```
When running without [retries ](#retries ), all tests after the failure are skipped:
* Worker process #1:
2021-11-20 00:47:30 +03:00
* `beforeAll` hook runs
2021-09-08 17:44:26 +03:00
* `first good` passes
* `second flaky` fails
* `third good` is skipped entirely
When running with [retries ](#retries ), all tests are retried together:
* Worker process #1:
2021-11-20 00:47:30 +03:00
* `beforeAll` hook runs
2021-09-08 17:44:26 +03:00
* `first good` passes
* `second flaky` fails
* `third good` is skipped
* Worker process #2:
2021-11-20 00:47:30 +03:00
* `beforeAll` hook runs again
2021-09-08 17:44:26 +03:00
* `first good` passes again
* `second flaky` passes
* `third good` passes
:::note
It is usually better to make your tests isolated, so they can be efficiently run and retried independently.
:::
## Reuse single page between tests
Playwright Test creates an isolated [Page] object for each test. However, if you'd like to reuse a single [Page] object between multiple tests, you can create your own in [`method: Test.beforeAll`] and close it in [`method: Test.afterAll`].
2022-06-11 03:34:31 +03:00
```js tab=js-js
2021-09-08 17:44:26 +03:00
// example.spec.js
// @ts -check
const { test } = require('@playwright/test');
2022-02-03 07:44:11 +03:00
test.describe.configure({ mode: 'serial' });
2021-09-08 17:44:26 +03:00
2022-02-03 07:44:11 +03:00
/** @type {import('@playwright/test').Page} */
let page;
2021-09-08 17:44:26 +03:00
2022-02-03 07:44:11 +03:00
test.beforeAll(async ({ browser }) => {
page = await browser.newPage();
});
test.afterAll(async () => {
await page.close();
});
2021-09-08 17:44:26 +03:00
2022-02-03 07:44:11 +03:00
test('runs first', async () => {
await page.goto('https://playwright.dev/');
});
2021-09-08 17:44:26 +03:00
2022-02-03 07:44:11 +03:00
test('runs second', async () => {
2022-10-04 03:02:46 +03:00
await page.getByText('Get Started').click();
2021-09-08 17:44:26 +03:00
});
```
2022-06-11 03:34:31 +03:00
```js tab=js-ts
2021-09-08 17:44:26 +03:00
// example.spec.ts
import { test, Page } from '@playwright/test';
2022-02-03 07:44:11 +03:00
test.describe.configure({ mode: 'serial' });
2021-09-08 17:44:26 +03:00
2022-02-03 07:44:11 +03:00
let page: Page;
2021-09-08 17:44:26 +03:00
2022-02-03 07:44:11 +03:00
test.beforeAll(async ({ browser }) => {
page = await browser.newPage();
});
2021-09-08 17:44:26 +03:00
2022-02-03 07:44:11 +03:00
test.afterAll(async () => {
await page.close();
});
test('runs first', async () => {
await page.goto('https://playwright.dev/');
});
2021-09-08 17:44:26 +03:00
2022-02-03 07:44:11 +03:00
test('runs second', async () => {
2022-10-04 03:02:46 +03:00
await page.getByText('Get Started').click();
2021-09-08 17:44:26 +03:00
});
2021-05-28 06:30:03 +03:00
```