Since yesterday all our windows bots are dying while trying to
install npm@8.
Turns out this is due to recent release of npm@8.4. This patch
moves our CI to use npm@8.3
https://github.com/npm/cli/issues/4341
This patch:
- rolls stable-test-runner to Nov 2, 2021 tip-of-tree
- introduces a new npm script, `npm run vtest`, to run Visual Regression
Tests for our HTML reporter
The common pattern today is to have two commands to configure CI:
```sh
npx playwright install-deps chromium
npx playwright install chromium
```
With this patch, this becomes:
```sh
npx playwright install --with-deps chromium
```
Note: `--with-deps` might call `sudo`.
- Source now lives at `src/test`.
- Former folio tests live at `tests/playwright-test`.
- We use `src/test/internal.ts` that exposes base test without
Playwright fixtures for most tests (to avoid modifications for now).
- Test types live in `types/testFoo.d.ts`.
- Stable test runner is installed to `tests/config/test-runner` during `npm install`.
- All deps including test-only are now listed in `package.json`.
Non-test deps must also be listed in `build_package.js` to get included.
With this patch, we have the following behaviors:
- For PR's, we will run only Linux tests for Chromium, Firefox and
Webkit
- For commits to master and release-*, we'll run all the tests
To run all tests on PR, assign a "CQ1" label to the PR.
