New default is based on the very (unscientific) output of repeated a few
times:
```
$ for i in `seq 5 13`; do time npm run ctest -- --reporter=line --workers=$i 'page/*' 'selector*' 'channels*' 'resource-timing*'; done
```
This patch reverts 2 commits that removed the feature from the method:
- "fix: explicitly ignore maxDiffPixels in toMatchSnapshot (#12570)"
commit b8af8458d6.
- "chore: remove `maxDiffPixels` from toMatchSnapshot (#12539)"
commit a3dff45974.
The `screenshotsDir` option controls the expectation storage
for `toHaveScreenshot()` function.
The new expectation management for screenshots has the following
key properties:
- All screenshots are stored in a single folder called `screenshotsDir`.
- Screenshot names **do not** respect `snapshotDir` and `snapshotSuffix`
configurations.
- `screenshotsDir` is configurable per project. This way a "smoke tests"
project can re-use screenshots from "all tests" project.
- Host platform is a top-level folder.
For example, given the following config:
```js
// playwright.config.ts
module.exports = {
projects: [
{ name: 'Mobile Safari' },
{ name: 'Desktop Chrome' },
],
};
```
And the following test structure:
```
smoke-tests/
└── basic.spec.ts
```
Will result in the following screenshots folder structure by default:
```
__screenshots__/
└── darwin/
├── Mobile Safari/
│ └── smoke-tests/
│ └── basic.spec.ts/
│ └── screenshot-expectation.png
└── Desktop Chrome/
└── smoke-tests/
└── basic.spec.ts/
└── screenshot-expectation.png
```
Previously, we used to skip all the tests from the same file when
any `beforeAll` fails in the file.
Now, we only skip the rest of the tests affected by this particular
`beforeAll` and continue with other tests in the new worker.
Reland: worker.stop() before worker.run() was hanging because
`_runFinished` promise was not initially resolved.
---
This moves `beforeAll`, `afterAll` and some modifiers from running
as a separate entity into running inside a test.
Pros:
- All errors are reported as test errors.
- All artifacts are collected as test artifacts.
- Reporters support this out of the box.
Details:
- Each test computes the necessary hooks to run and runs them.
- Teardown is usually performed during the test (on test failure or worker stop).
- `skipRemaining` is added to `DonePayload` to preserve the behavior
where `beforeAll` hook failure skips subsequent tests.
This behavior can now be improved to only target tests affected by this hook.
