Dmitry Gozman
23aa0be15b
docs: update afterEach docs with an example ( #9727 )
...
Also add a test for TestInfo.status in afterEach.
2021-10-22 16:32:22 -07:00
Joel Einbinder
f2888395a6
fix: support even older node 12 ( #9718 )
2021-10-22 15:59:52 -04:00
Dmitry Gozman
193c79a685
feat(test runner): rewrite dispatcher to avoid unneded stalls ( #9629 )
...
In some circumstances, dispatcher was waiting for all exisitng jobs
to finish before scheduling a new one. This leads to unneded stalls.
Instead, we can schedule jobs right away, if we have a worker
available.
2021-10-22 11:10:37 -07:00
Pavel Feldman
6a3e08d1ac
fix(reporter): fix locator stacks to hide internal detail ( #9693 )
2021-10-21 16:10:47 -07:00
Pavel Feldman
299dffbdb3
fix(expect): fix toHaveCount(0) ( #9690 )
2021-10-21 14:46:54 -07:00
Max Schmitt
5426e24248
chore: roll stable-test-runner to ToT ( #9655 )
2021-10-20 10:35:37 -07:00
Joel Einbinder
53c866c0f2
fix(monorepo): explicitly list package.json exports instead of using glob patterns ( #9584 )
2021-10-19 12:28:02 -04:00
Dmitry Gozman
fc6a93d24e
test: expect.not.toBeChecked when no elements match ( #9607 )
2021-10-19 09:13:16 -07:00
Dmitry Gozman
6d554a5e30
feat(reporter): expose more apis ( #9603 )
2021-10-19 08:38:04 -07:00
Pavel Feldman
c06a6e1f63
fix(html): always highlight error code ( #9604 )
2021-10-18 21:37:19 -07:00
Pavel Feldman
e37660b068
feat(api): expose step location ( #9602 )
2021-10-18 21:06:18 -07:00
Dmitry Gozman
4977edcaf3
fix(tracing): fix inconsistent calls of tracing.{start,stop} in fixtures ( #9594 )
...
Drive-by: fix error logging.
2021-10-18 21:05:59 -07:00
Pavel Feldman
432fb453e4
chore(html): only copy trace viewer for reports with traces ( #9579 )
2021-10-18 08:03:04 -07:00
Pavel Feldman
6afa85927e
chore(ui): redesign html report a bit ( #9577 )
2021-10-17 20:58:06 -07:00
Pavel Feldman
99f3389b3b
feat(html): automatically use console reporter with html ( #9559 )
2021-10-15 19:18:56 -07:00
Pavel Feldman
9135847950
feat(ui): merge stdio in html report ( #9560 )
2021-10-15 16:15:06 -07:00
Yury Semikhatsky
235cd10a43
fix(runner): clarify where test.use() can be called ( #9486 )
2021-10-15 12:03:26 -07:00
Pavel Feldman
75cfe5d1f5
chore: start adding html runner e2e tests ( #9518 )
2021-10-14 15:48:05 -07:00
Pavel Feldman
584014f6fa
feat(html): unhide html reporter ( #9512 )
2021-10-14 11:17:35 -07:00
Yury Semikhatsky
e827bde1c2
fix(runner): skip global hooks in --list mode ( #9480 )
2021-10-13 14:18:15 -07:00
Dmitry Gozman
64a3099655
chore(expect): simplify expect implementation ( #9459 )
2021-10-13 08:56:57 -07:00
Pavel Feldman
876e08315b
fix(expect): continue polling in array-based matchers, fix edge cases ( #9430 )
2021-10-11 14:01:09 -07:00
Max Schmitt
b1160ec239
chore(test-runner): make outputDir based on cwd instead of testDir ( #9360 )
2021-10-11 22:24:46 +02:00
Joel Einbinder
d34b53a0d7
feat(test runner): allow comparing binary files to strings ( #9315 )
...
Co-authored-by: Max Schmitt <max@schmitt.mx>
2021-10-11 21:57:59 +02:00
Max Schmitt
76142a33ae
test: fix tests which include special signal handling ( #9433 )
2021-10-11 12:07:40 -07:00
Max Schmitt
9dd6bb8c6a
Revert "feat(test-runner): allow parametrized tests that don't extend base ( #9301 )"
...
This reverts commit f63af830de
2021-10-11 17:32:31 +02:00
Joel Einbinder
c89d5a50dd
chore: migrate to monorepo ( #9421 )
2021-10-11 16:52:17 +02:00
Max Schmitt
b69b8889fe
test: fix broken relative outputDir test
...
#9380
2021-10-09 00:12:21 +02:00
Max Schmitt
804ee6e979
chore: roll stable-test-runner to ToT ( #9397 )
2021-10-08 18:37:06 +02:00
Max Schmitt
b3a8ccd064
test(test-runner): add test for relative outputDir ( #9381 )
2021-10-08 09:20:41 +02:00
Pavel Feldman
083d882b2b
test: add missing request tests ( #9343 )
2021-10-06 10:15:13 -07:00
Pavel Feldman
bc71d20d0f
feat(request): add global request fixture ( #9332 )
2021-10-06 10:09:27 -07:00
Pavel Feldman
f63af830de
feat(test-runner): allow parametrized tests that don't extend base ( #9301 )
2021-10-04 17:16:33 -07:00
Sidharth Vinod
f4cbd9d574
test: remove checking duration in github reporter to reduce flakiness
2021-10-04 15:53:26 +02:00
Sidharth Vinod
be30f9f1c4
feat(test-runner): Add GitHub Actions reporter ( #9191 )
2021-10-04 10:32:56 +02:00
Pavel Feldman
913821f675
chore: roll Electron to 12.2.1 ( #9271 )
2021-10-01 19:40:47 -07:00
Nick Partridge
b126a5685b
feat: add path option to toMatchSnapshot ( #9156 )
2021-10-01 09:15:44 -07:00
Max Schmitt
e674d873a3
feat(test-runner): add reuse context mode to share a single context between tests ( #9115 )
2021-10-01 09:16:03 +02:00
Dmitry Gozman
5e3ad63b42
fix(test runner): do not write missing snapshot until the last retry ( #9246 )
...
This prevents future retries from passing because of the actual
snapshot being written.
In theory, we can avoid running the retry since it should fail anyway.
However, this brings problems, for example in the `describe.serial` mode
where running a test also has some side effects and so it should not be
skipped. Since running a test without a snapshot is rare, it should be
fine to retry it.
2021-09-30 16:44:52 -07:00
Sidharth Vinod
fcb7d2b15a
feat(reporters): Add error position to JSON Report ( #9151 )
2021-09-30 14:18:36 -07:00
Pavel Feldman
f78302e8dd
fix(toBeHidden): return true to missing elements ( #9205 )
2021-09-28 17:11:04 -07:00
Dmitry Gozman
ed9b42a92d
feat(test runner): collect test error from worker teardown ( #9190 )
...
When the test fails (usually with timeout), we wait until all hooks are run
and worker scope is teared down before reporting test end result.
This allows us to collect any error details populated by teardown
in addition to the "timed out" message.
2021-09-28 10:56:50 -07:00
Dmitry Gozman
fa536786f2
fix(test runner): proper serial mode with beforeAll/afterAll failures ( #9183 )
2021-09-27 15:58:26 -07:00
Dmitry Gozman
8dc8777ab4
feat(expect): toContainText(array) ( #9160 )
...
This matches when each expected item from the array
is matched to one of the resolved elements, in order.
Note this performs both "sub-array" and "substring" matching.
Drive-by: documentation fixes.
Drive-by: added "selector resolved to 3 elements" log line
when expecting arrays.
2021-09-27 11:14:35 -07:00
Max Schmitt
cd22072685
chore: enable object-curly-spacing in ESLint ( #9168 )
2021-09-27 18:58:08 +02:00
Pavel Feldman
241411ad42
chore: render expect in trace viewer ( #9141 )
2021-09-27 09:19:59 -07:00
Max Schmitt
4573ce0cf8
chore(test-runner): launch -> webServer ( #9167 )
2021-09-27 11:32:57 +02:00
Dmitry Gozman
d22dd4a4e7
feat(test runner): improve fixture typings for function fixtures ( #9138 )
...
When fixture value `R` is a function, TypeScript sometimes confuses
function `R` and function `async ({}, use) => {}`. This leads to
`any` types in the latter because it could be either of the functions
as TS thinks.
The solution is to only accept the second syntax, assuming that noone
passes fixture value that is a function as is:
```js
// This will stop working.
test.extend<{ foo: (x: number) => number }>({
foo: x => 2 * x,
});
// This will get inferred types and autocomplete.
test.extend<{ foo: (x: number) => number }>({
foo: async ({}, use) => {
await use(x => 2 * x);
},
});
```
2021-09-24 19:59:30 -07:00
Dmitry Gozman
f4aaebfba0
fix(expect): produce "waiting for selector" log, corner cases ( #9140 )
2021-09-24 18:55:45 -07:00
Pavel Feldman
db6c55af51
chore: add a test for the log scale polling ( #9136 )
2021-09-24 12:54:33 -07:00
Pavel Feldman
0908dc98c8
chore: migrate expect(locator).toMatchText to protocol ( #9117 )
2021-09-23 16:46:46 -07:00
Dmitry Gozman
568ec05a97
fix(junit): produce a single system-out entry ( #9091 )
...
Instead of multiple `system-out` entries we produce a single one
with concatenated content. This is compatible with various junit xml
parsers in the wild.
2021-09-22 15:28:57 -07:00
Dmitry Gozman
018467911b
test: introduce some common test fixtures ( #9060 )
2021-09-21 16:24:48 -07:00
Max Schmitt
f9c5279c3a
fix(test-runner): toHaveURL respect baseURL ( #9050 )
2021-09-21 21:41:24 +02:00
Dmitry Gozman
7ec1035b98
test: improve child process utilities in tests ( #9036 )
2021-09-20 17:17:12 -07:00
Pavel Feldman
63ff405e6e
fix(inspector): stop on all snapshottable actions ( #8990 )
2021-09-17 15:24:15 -07:00
Dmitry Gozman
43213614a1
fix(test runner): after hooks step should not be nested ( #8969 )
2021-09-16 15:51:27 -07:00
Dmitry Gozman
5379b2dcba
fix(test runner): account for errors with inconsistent stack/message ( #8950 )
2021-09-15 21:28:36 -07:00
Darrell Breeden
d82cb9a2ff
feat: addition of tags to JSON reporter output ( #8920 )
2021-09-15 12:30:22 -07:00
Pavel Feldman
14bc663742
chrome: don't nest pw:api steps ( #8932 )
2021-09-15 11:34:23 -07:00
Dmitry Gozman
b8a46580dd
fix(expect): toHaveText, toContainText and toHaveTitle normalize whitespace ( #8929 )
2021-09-14 19:24:29 -07:00
Pavel Feldman
5253a7eb54
feat(html): restore trace, video, screenshot ( #8925 )
2021-09-14 16:26:31 -07:00
Pavel Feldman
16baaa317d
chore: remove stale html experiments ( #8905 )
2021-09-13 20:34:46 -07:00
Pavel Feldman
1925c85dfb
feat(report): render attachment as a part of failure ( #8903 )
2021-09-13 18:07:40 -07:00
Pavel Feldman
b76e993951
chore: remove log from htmlreport ( #8898 )
2021-09-13 18:07:15 -07:00
Dmitry Gozman
d9d2d809a2
fix(test): make use in config accept option values only ( #8828 )
...
Also include default options in FullConfig/FullProject.
Also make examples compile and add a test.
2021-09-13 17:50:31 -07:00
Dmitry Gozman
ed34a67d4a
fix(junit reporter): put stdio under testcase ( #8900 )
...
When output happened during test execution, it should be
under `<testcase><system-out>...</system-out></testcase>`.
2021-09-13 17:50:08 -07:00
Dmitry Gozman
440651e05c
test(test runner): calling test.step() from fixture ( #8899 )
2021-09-13 17:49:58 -07:00
Dmitry Gozman
6c41ad3329
test: expect(locator).toHaveCount should wait ( #8901 )
2021-09-13 17:06:19 -07:00
Joel Einbinder
bf35da3656
fix(test-runner): accept unix separators even on windows ( #8881 )
...
.
2021-09-13 12:09:38 -04:00
Dmitry Gozman
bcb0c1745b
fix(waitForEventInfo): reply from the server side ( #8825 )
...
Otherwise, client thinks that `waitForEventInfo` is a pending operation.
2021-09-09 21:20:26 -07:00
Pavel Feldman
665143d629
chore(report): don't generate file per test ( #8822 )
2021-09-09 17:35:31 -07:00
Pavel Feldman
7bbb63d143
feat(test): start authoring raw reporter ( #8790 )
2021-09-09 14:17:18 -07:00
Yury Semikhatsky
afe92a6fcf
fix(runner): avoid line wrapping in list reporter ( #8705 )
2021-09-07 17:42:17 -07:00
Daniel Dyssegaard Kallick
b2c59a15dd
feat(test-runner): include test step in json report ( #8476 )
2021-09-07 13:35:30 -07:00
Dmitry Gozman
3739113e74
fix(test runner): allow dot-files and dot-directories with tests ( #8751 )
2021-09-07 10:32:47 -07:00
Max Schmitt
e40b805782
feat(test-runner): support baseURL in toHaveURL ( #8743 )
2021-09-07 18:34:02 +02:00
Yury Semikhatsky
e5e461c0de
fix: use logger from config if specified ( #8697 )
2021-09-03 14:21:25 -07:00
Dmitry Gozman
e2b092c1a0
feat(html reporter): show log for pw:api and expect steps ( #8692 )
2021-09-03 13:08:17 -07:00
Dmitry Gozman
2b4a93972d
test: use ToT test runner for non-test-runner tests ( #8671 )
2021-09-03 11:22:25 -07:00
Dmitry Gozman
620712a5d9
feat(expect): support array of RegExp objects in toHaveText/toHaveClass ( #8667 )
2021-09-02 15:48:04 -07:00
Dmitry Gozman
e691b649de
feat(test runner): describe.parallel ( #8662 )
2021-09-02 15:42:07 -07:00
Fokke Zandbergen
0ae38b5aec
fix(test-runner): WebServer: use socket for reuseExistingServer to detect servers that aren't using SO_REUSEADDR ( #8537 )
2021-09-02 18:39:41 +02:00
Yury Semikhatsky
bafa426231
feat(runner): support multiple names in project filter ( #8600 )
2021-09-02 09:29:55 -07:00
Dmitry Gozman
b1260602ac
docs: update global setup docs ( #8637 )
...
Changing example to "authenticate once", and also updating the auth doc.
Adding a test with the same setup.
2021-09-01 15:35:46 -07:00
Dmitry Gozman
4f4cf448c2
fix(test runner): generate unique outputDir for beforeAll/afterAll ( #8633 )
2021-09-01 13:41:35 -07:00
Dmitry Gozman
1a9215a6ec
fix(html reporter): do not throw when attachment is actually missing ( #8630 )
2021-09-01 12:20:28 -07:00
Dmitry Gozman
b8f8ca7493
feat(tracing): tracing.{start,stop}Chunk instead of tracing._export ( #8521 )
2021-08-31 17:03:31 -07:00
Dmitry Gozman
bee8ed117b
feat(html reporter): preview source code, steps and step errors ( #8598 )
2021-08-31 16:34:52 -07:00
Dmitry Gozman
900362ec0b
fix(test runner): report unhandled rejections during worker teardown ( #8592 )
2021-08-31 10:50:30 -07:00
Dmitry Gozman
37a897c9b4
fix(test runner): report beforeAll timeout instead of hanging ( #8529 )
...
We used to not report fatal error and hang forever because worker
did not run any tests but also did not report any errors.
Also properly show stack-less errors.
2021-08-28 07:19:45 -07:00
Michael Rienstra
ed0c47a06f
fix(test-runner): minor error message typo ( #8489 )
2021-08-27 10:15:16 +02:00
Dmitry Gozman
a479cb6f52
fix(test runner): add an overload for test.skip(title, fn) ( #8454 )
...
We shipped this feature, but forgot to add the right overload to d.ts.
2021-08-25 14:36:36 -07:00
Dmitry Gozman
de85d8bb83
fix(test runner): do not special case test.fail ( #8447 )
...
This makes `test.fail` tests considered as passing when they actually fail:
- Stop restarting the worker.
- Retry when it passes instead of a fail.
- Behaves similar to regular tests in a `describe.serial` suite.
2021-08-25 12:19:50 -07:00
Dmitry Gozman
75fb77355a
test(test runner): check retries with beforeAll failure and multiple tests ( #8413 )
2021-08-24 12:22:16 -07:00
Joel Einbinder
a8a3799e9d
fix(test-runner): use describe name in output dir ( #8282 )
2021-08-24 10:33:40 -04:00
Dmitry Gozman
f9b87268a7
fix(test runner): afterAll error should not mask beforeAll error ( #8358 )
2021-08-23 09:21:40 -07:00
Pavel Feldman
0997c13151
fix(test-runner): do not attach non-existent diff ( #8297 )
2021-08-20 13:40:27 -07:00
Dmitry Gozman
e5be2c9205
feat(test runner): show failure details for flaky test runs ( #8332 )
...
Currently, we just say "foo.spec.ts > my test" is flaky, but do not
show how exactly the failed run went.
2021-08-19 18:20:53 -07:00
Dmitry Gozman
70eca0401d
fix(test runner): avoid internal error for step end without begin ( #8327 )
...
fix(test runner): avoid internal error for step end without begin
Consider the following scenario:
- Test finishes and starts tearing down fixtures.
- Fixture teardown starts a step S and then times out.
- We declare the test finished (with timeout).
- Dispatcher shuts down the worker and spins a new one for a retry.
Additionally, it clears steps information for the test to be
ready for the new retry. Step S information is lost.
- Meanwhile, during worker teardown, the step S does
actually finish (usually with an error), and we send stepEnd for S.
- Dispatcher does not know what to do with step S end and
prints an internal error.
The fix is to ignore certain messages from the shutting down worker that failed.
2021-08-19 17:04:09 -07:00
