Commit Graph

36 Commits

Author SHA1 Message Date
Dmitry Gozman
49fd9500fe
fix: handle worker process start failure (#27249)
Worker process start failure is reported as a test error and skips other
tests from the group.
If happened during stop (e.g. from a Ctrl+C) before worker has fully
initialized, this error is ignored.

Drive-by: send SIGINT in tests to the whole tree, to better emulate
Ctrl+C behavior.
2023-09-22 10:57:35 -07:00
Dmitry Gozman
186f86905c
chore: make @playwright/test depend on playwright (#26946) 2023-09-08 14:23:35 -07:00
Dmitry Gozman
ed99ac7395
fix(sigint): make sure we do not add handler twice (#24413)
In the following scenario, we were adding SIGINT handler twice, but
removing it just once:
- Task runner starts testing, creates SigIntWatcher, installs SIGINT
handler.
- Press Ctrl+C, task runner interrupts, disarms SigIntWatcher, SIGINT
handler is not removed due to 1000ms cooldown.
- Task runner starts cleanup, creates SigIntWatcher, installs another
SIGINT handler.
- Cleanup finishes, SigIntWatcher disarms, could remove or not remove
SIGINT handler based on timing (same 1000ms cooldown). In any case, we
have one or two SIGINT handlers still on.
- HTML reporter hangs in onExit, while we still have SIGINT handler up,
so Ctrl+C does not exit.

Regressed in #24265.
2023-07-25 18:35:38 -07:00
Dmitry Gozman
ed14bf2103
test: replace sendSIGINTAfter with interactWithTestRunner (#24411)
This way we can send multiple SIGINTs in tests.
2023-07-25 15:46:39 -07:00
Dmitry Gozman
767addec8c
chore: make sure to call task's teardown if it has ever started (#24317)
This way things like WebServerPlugin can cleanup after themselves even
if they failed to start or were interrupted mid-way.
2023-07-20 17:16:22 -07:00
Dmitry Gozman
2c8dd67b0e
fix(test runner): improve error message for non-stable test titles (#23792)
Fixes #23779.
2023-06-19 14:13:11 -07:00
Tomáš Hübelbauer
e550df060e
Enhance the forbidOnly mode message to guide the user towards the configuration option (#23146)
Hi, I am putting this PR out as a feeler to see if there's interested in
improving this error message, but the copy is by no means final and I am
open to improvement suggestions.

My intention here is to:
- Explain what a "focused item" is - that we're talking about a test and
it being focused is most likely down it using `only`
  
Are there other types of "items"? Are there other ways to make them
focused other than `only`?

- Explain why we're even in focused mode and how to control it

The default scaffolded Playwright config file includes a forbidMode
expression driven by whether `CI=1` is set.
I ran into this when trying to reproduce a CI issue locally so I had it
set and unknowingly entered focus only mode.
I wasn't aware this mode was a thing because I was using the default
configuration from `npm init` and did not familiarize myself with all
the options in it.

Is there a way to tell if we're in a TypeScript or JavaScript project in
this function? I would use that to display the configuration file name
with the right extension.

---------

Signed-off-by: Tomáš Hübelbauer <tomas@hubelbauer.net>
2023-05-25 13:32:49 -07:00
Yury Semikhatsky
a4e90f20dc
fix(runner): do not hang on worker exit before tests (#22742) 2023-05-01 14:54:48 -07:00
Pavel Feldman
87acda74ff
chore: respect source map sources when filtering in CLI (#22180)
Fixes #22123
2023-04-03 19:49:01 -07:00
Dmitry Gozman
eb3f8bfba8
fix(test runner): do not optimize filtering when sourcemap is present (#21359)
Fixes #21204.
2023-03-03 07:49:19 -08:00
Dmitry Gozman
91da67fab1
test: remove magic headers in ttest (#20867)
Instead, explicitly import from '@playwright/test'.
2023-02-14 19:20:56 -08:00
Pavel Feldman
1b941bcf2e
chore: simplify ttests (#20733) 2023-02-07 15:11:44 -08:00
Dmitry Gozman
5fb430a743
fix(test runner): do not print timeout message upon interrupt (#20574)
We should only print "Test was interrupted."

Regressed in #18321.
2023-02-01 19:39:43 -08:00
Pavel Feldman
2c27bd3b07
chore: remove plugin registration sink (#20460) 2023-01-27 14:36:41 -08:00
Pavel Feldman
675f0eb4a0
chore: report error location for fatal errors (#19610) 2022-12-21 09:36:59 -08:00
Yury Semikhatsky
31d45ad4dc
chore(runner): add all setup files in separate pass (#18978) 2022-11-22 16:22:48 -08:00
Pavel Feldman
5424c8c385
fix(test): unflake workers test (#17728) 2022-09-30 08:07:20 -07:00
Dmitry Gozman
baa2ef2700
fix(test runner): show interrupted as yellow (#16396)
Drive-by: fix unreadable character in "duplicate titles" error.
2022-08-09 21:17:30 -07:00
Dmitry Gozman
445fe032f5
feat(test runner): separate interrupted status from skipped (#16124) 2022-08-02 12:55:43 -07:00
Pavel Feldman
8c190c3e09
feat(testId): expose persistent test id (#16003) 2022-07-27 20:17:19 -07:00
Dmitry Gozman
c02d83fb11
fix(runner): filter only after duplicate titles check (#15401) 2022-07-06 12:51:26 -07:00
Pavel Feldman
41529bb1a5
chore: respect sigint in global setup (#14805) 2022-06-12 13:06:00 -07:00
Dmitry Gozman
79d356f0cc
fix(test runner): when worker exits unexpectedly, fail a single test (#14608)
All remaining tests will continue in a new worker.
2022-06-02 21:13:47 -07:00
Andrey Lushnikov
8801f79742
Revert "fix(playwright-test): prohibit async functions passed to describe (#14538)" (#14591)
This reverts commit bd0fe50a0e.
2022-06-02 10:31:10 -07:00
Andrey Lushnikov
bd0fe50a0e
fix(playwright-test): prohibit async functions passed to describe (#14538)
References #14533
2022-06-02 09:44:47 -07:00
Dmitry Gozman
305afcdacf
fix(test runner): fix duplicate titles error when multiple issues are present (#14090) 2022-05-11 11:53:16 +01:00
Yury Semikhatsky
2e6ef8f622
fix(runner): fail if worker cannot find some of the tests (#13666) 2022-04-25 09:05:40 -07:00
Dmitry Gozman
f6ccd4847e
fix(test runner): default workerIndex to -1 (#13440) 2022-04-08 15:23:23 -07:00
Dmitry Gozman
9814c592d2
fix(test runner): shutdown redundant workers (#12062) 2022-02-14 10:57:15 -08:00
Andrey Lushnikov
2336692e8a
feat: support clarification message for expect (#11735)
The clarification message is displayed in the HTML report as the name of the step:

![image](https://user-images.githubusercontent.com/746130/151852652-48194140-5ea4-439d-afee-12583a8caf71.png)

It is also shown in terminal output:

![image](https://user-images.githubusercontent.com/746130/151852666-5c956ef1-6e94-4bc2-8e55-b58688dfc7e0.png)

Fixes #7816
2022-01-31 17:14:59 -08:00
Dmitry Gozman
193c79a685
feat(test runner): rewrite dispatcher to avoid unneded stalls (#9629)
In some circumstances, dispatcher was waiting for all exisitng jobs
to finish before scheduling a new one. This leads to unneded stalls.
Instead, we can schedule jobs right away, if we have a worker
available.
2021-10-22 11:10:37 -07:00
Dmitry Gozman
1bbf86d060
feat(test runner): file scope fixtures (#7969)
These are reset after running tests from a single file.
2021-08-04 21:11:02 -07:00
Dmitry Gozman
3f0485486d
feat(test runner): show errors from interrupted tests when available (#7874)
This shows the exact operation that is timing out (like click) when
user hits Ctrl+C.
2021-07-28 15:43:37 -07:00
Max Schmitt
ae489b1c43
fix(test-runner): do not override error with unhandled error (#7507) 2021-07-08 18:24:07 +02:00
Dmitry Gozman
2073193c36
fix(test runner): keep track of remaining tests on the runner side (#7486)
This fixes two issues:
- Sudden worker process exit is properly accounted for.
- We can stop() workers willy-nilly, e.g. after reaching maxFailures.

Details:
- DonePayload does not send `reamining` anymore, and worker does not track it.
- Instead, `Dispatcher._runJob` track remaining tests and acts accordingly.
- Upon worker exit, we emulate a fatal error for all remaining tests.

Drive-by:
- Do not report onTestBegin after reaching maxFailures to avoid confusion.
  Before, we did report onTestBegin, but not onTestEnd.
- List reporter aligned between "running" and "finished" state - it was
  one character misplaced.
- Added a SIGINT test.
2021-07-07 12:04:43 -07:00
Max Schmitt
0776cf76a2
feat(test-runner): do only allow unique spec titles per suite (#7300) 2021-06-28 22:13:35 +02:00