Dmitry Gozman
bb34d7a953
fix(test-runner): sanitize snapshot name before constructing a path ( #7620 )
...
This avoids problems with `toMatchSnapshot('../../dir/file.png')`
where we append this path to `snapshotDir` and end up in some random
place.
Also added a note to documentation.
2021-07-14 16:31:19 -07:00
Max Schmitt
6cc2fe178e
feat(test-runner): migrate to launch config/server ( #7603 )
2021-07-15 01:19:45 +02:00
Max Schmitt
a26fe65db3
fix(test-runner): start webServer before globalSetup/teardown ( #7604 )
2021-07-14 19:01:46 +02:00
Dmitry Gozman
0742cb9076
chore: move working with browser channels to Registry Executables ( #7581 )
2021-07-13 19:03:49 -07:00
Dmitry Gozman
57c5e4d8cf
chore: replace Registry api with Executable list ( #7544 )
2021-07-13 15:57:40 -07:00
Yury Semikhatsky
5d62d01450
fix(runner): mark tests skipped due to sharding in junit report ( #7578 )
2021-07-13 05:06:08 -07:00
Joel Einbinder
eb31b9e4a9
feat(test-runner): support esm modules in more places ( #7542 )
2021-07-12 11:59:58 -05:00
Dmitry Gozman
65606c093a
chore: simplify Registry api ( #7451 )
2021-07-09 16:10:23 -07:00
Max Schmitt
bc2f161a55
chore: roll test-runner to 1.13.0-next-1625774143000 ( #7519 )
2021-07-08 23:36:11 -07:00
Yury Semikhatsky
f5b552fde7
test(runner): new worker after test failure ( #7518 )
2021-07-08 23:30:15 -07:00
Dmitry Gozman
77deca1d6b
feat(test runner): export testInfo.data ( #7525 )
...
This is a key-value storage for any information that goes into the report.
Also export JSONReport types.
2021-07-08 17:16:36 -07:00
Max Schmitt
e604f185ca
fix(test-runner): with automatic fixtures workerInfo was undefined after conditional skip ( #7521 )
2021-07-08 21:55:43 +02:00
Yury Semikhatsky
b2742976a8
fix(runner): do not restart worker after skipping tests ( #7511 )
2021-07-08 09:36:28 -07:00
Max Schmitt
ae489b1c43
fix(test-runner): do not override error with unhandled error ( #7507 )
2021-07-08 18:24:07 +02:00
Dmitry Zakharov
5464ad849e
fix(test-runner): handle negated toMatchSnapshot result ( #7345 )
2021-07-07 18:51:38 -07:00
Dmitry Gozman
2073193c36
fix(test runner): keep track of remaining tests on the runner side ( #7486 )
...
This fixes two issues:
- Sudden worker process exit is properly accounted for.
- We can stop() workers willy-nilly, e.g. after reaching maxFailures.
Details:
- DonePayload does not send `reamining` anymore, and worker does not track it.
- Instead, `Dispatcher._runJob` track remaining tests and acts accordingly.
- Upon worker exit, we emulate a fatal error for all remaining tests.
Drive-by:
- Do not report onTestBegin after reaching maxFailures to avoid confusion.
Before, we did report onTestBegin, but not onTestEnd.
- List reporter aligned between "running" and "finished" state - it was
one character misplaced.
- Added a SIGINT test.
2021-07-07 12:04:43 -07:00
Max Schmitt
98bcf26656
feat(test-runner): add webServer ( #7368 )
2021-07-07 20:19:42 +02:00
Dmitry Gozman
444d1eb51a
feat(test runner): run modifier functions once if they do not depend on test fixtures ( #7436 )
2021-07-02 15:49:05 -07:00
Joel Einbinder
7629587914
fix(test-runner): work with .mjs files ( #7373 )
2021-06-29 15:28:41 -07:00
Dmitry Gozman
6b3614fd4c
feat(test runner): support test.setTimeout for the whole block ( #7387 )
...
This enables `test.setTimeout()` outside of the test body, that
affects all tests in the block (either file or describe).
2021-06-29 13:33:13 -07:00
Joel Einbinder
368880962f
fix(test-runner): better test match glob ( #7382 )
2021-06-29 11:49:50 -07:00
Dmitry Gozman
6aefa02e91
feat(test runner): improve reporters api ( #7370 )
...
- onEnd may return a Promise
- onEnd now takes a result for the full run
- onTimeout is replaced with onEnd(result)
2021-06-29 10:55:46 -07:00
Max Schmitt
0776cf76a2
feat(test-runner): do only allow unique spec titles per suite ( #7300 )
2021-06-28 22:13:35 +02:00
Max Schmitt
bd86e70465
feat(test-runner): allow to focus a test in a location ( #7208 )
2021-06-24 10:02:34 +02:00
Dmitry Gozman
4c6fa42810
fix(test runner): expose real stack traces and speed up locations ( #7265 )
...
Stop wrapping/prepending error messages so that we do not loose the stack trace. For this, update a few manually thrown errors with better messages (usually including a file path).
Speed up locations by doing manual `sourceMapSupport.wrapCallSite()` for a single call site. Performance gain in the runner process with 100 files x 100 tests each:
- 25% on the fresh run without babel cache;
- 80% on the cached run where babel is almost instant.
Also some obvious cleanups around stack traces (removing unused code).
2021-06-23 10:30:54 -07:00
Max Schmitt
103f8ddd06
Revert "chore: update eslint config ( #6840 )" partly ( #7268 )
...
This reverts commit ec7d37d92f
2021-06-23 11:08:35 +02:00
Max Schmitt
99bbc51760
fix(test-runner): support ANSII terminals with list reporter ( #7258 )
2021-06-22 19:04:24 +02:00
Dmitry Gozman
6118d16edd
fix(test runner): properly keep track of requireFile to support helpers/wrappers ( #7243 )
...
This fixes an issue where we incorrectly labeled and assigned ids for tests
that declared tests in require'd files or used test wrappers.
See new tests for examples.
2021-06-21 11:25:15 -07:00
Pavel Feldman
ea4eebeb2d
feat(test-runner): document tagging, implement grep-invert ( #7227 )
2021-06-18 17:56:59 -07:00
Dmitry Gozman
388c6f5196
feature(test runner): capture error and pending calls when timing out ( #7193 )
2021-06-17 15:09:38 -07:00
Pavel Feldman
82a50b0e1d
fix(test-runner): property handle artifacts in context of preserveOutput ( #7181 )
2021-06-16 16:05:30 -07:00
Pavel Feldman
184f2c2e93
feat(test-runner): allow specifying video size ( #7158 )
2021-06-16 07:51:54 -07:00
Joel Einbinder
38e27c9c6c
fix(test-runner): case insensitive test filtering
2021-06-15 17:27:52 -07:00
Joel Einbinder
4c2a3fb443
fix(test-runner): accept relative paths for outputDir ( #7149 )
...
fixes #7124
2021-06-15 13:39:07 -07:00
Dmitry Gozman
742cce3a1d
feat(test runner): configurable reportSlowTests ( #7120 )
...
Also splits tests by projects and reports them with nice relative paths.
2021-06-14 22:45:58 -07:00
Joel Einbinder
aa72b2b9bb
fix(pwt): max-failures should work with retries ( #7127 )
...
fixes #7112
2021-06-14 22:16:16 -07:00
Pavel Feldman
970bb6a70d
feat(test-runner): allow setting pixel match threshold for project ( #7123 )
2021-06-14 21:52:10 -07:00
Max Schmitt
a3a9ccad6f
fix(test-runner): do not require esModuleInterop=true ( #7100 )
2021-06-14 21:58:10 +02:00
Andrey Lushnikov
8a8b3932f8
Revert "fix(test-runner): support passing slowMo option ( #6991 )" ( #7077 )
...
This reverts commit 178489d091
2021-06-11 16:19:50 -07:00
Pavel Feldman
49a8f67c0f
fix(test-runner): resolve global hooks relative to the config dir ( #7061 )
2021-06-10 22:31:27 -07:00
Max Schmitt
05382c997b
fix(test-runner): do only match JS/TS files when collecting ( #7014 )
2021-06-10 16:41:57 +02:00
Max Schmitt
178489d091
fix(test-runner): support passing slowMo option ( #6991 )
...
Fixes #6984
Reverts #6967
2021-06-09 14:02:05 -07:00
Dmitry Gozman
1d6ca5284c
feat(test runner): add tests for playwright-specific fixtures ( #6952 )
2021-06-08 15:52:08 -07:00
Andrey Lushnikov
37af9ce3cf
test(playwright-test): fix fixtures test ( #6967 )
...
🤷♂️
2021-06-08 13:10:15 -07:00
Dmitry Gozman
8c13f679b7
fix(test runner): remove folio/jest namespaces in expect matchers ( #6930 )
2021-06-07 08:02:01 -07:00
Dmitry Gozman
46a0213769
chore: remove internal uses of "folio" ( #6931 )
...
Replaced by "pwt" or "playwright test".
2021-06-06 22:07:07 -07:00
Dmitry Gozman
b556ee6f5b
chore: brush up playwright-test types ( #6928 )
2021-06-06 20:18:47 -07:00
Dmitry Gozman
f745bf1fbc
chore: bring in folio source ( #6923 )
...
- Source now lives at `src/test`.
- Former folio tests live at `tests/playwright-test`.
- We use `src/test/internal.ts` that exposes base test without
Playwright fixtures for most tests (to avoid modifications for now).
- Test types live in `types/testFoo.d.ts`.
- Stable test runner is installed to `tests/config/test-runner` during `npm install`.
- All deps including test-only are now listed in `package.json`.
Non-test deps must also be listed in `build_package.js` to get included.
2021-06-06 17:09:53 -07:00
