Commit Graph

529 Commits

Author SHA1 Message Date
Dmitry Gozman
800b813d4b
chore(test runner): prepare to per-fixture timeout (#11605)
This reworks DeadlineRunner to use exception to signal timeout. This way,
we'll be able to run fixtures against a shared deadline vs their own
deadline and still get an easy control-flow timeout handling.
2022-01-25 11:22:28 -08:00
Andrey Lushnikov
42876a0528
fix: support PWDEBUG=0 to disable debug (#11611)
Fixes #11606
2022-01-25 08:40:24 -08:00
Ross Wollman
64e7557fb9
fix: falsey behavior in route.continue, page.post, testInfo.attach (#11421)
In several of the Playwright APIs, falsey values were not handled correctly. This changeset adds tests (and some fixes):

- route.continue: If options.postData was the empty string, the continue failed to override the post data.
- page.post (application/json with options.data: false|''|0|null): Raw falsey values were getting dropped (i.e. you can't do the equivalent of curl --header application/json … -d 'false'). This has been fixed with most values across all browsers, but an additional fix is needed for 'null' which the channel serializer treats extra specially.
- testInfo.attach: This didn't get reported as an error when options.path was the empty string, but should have been.
#11413 (and its fix #11414) inspired this search as they are the same
class of bug.
2022-01-24 15:06:36 -08:00
Pavel Feldman
9542b007cf
fix(esm): don't emit module shorthand warning (#11596) 2022-01-24 11:22:56 -08:00
Pavel Feldman
0b99d83911
chore: include config dir in quick list report (#11563) 2022-01-22 17:02:46 -08:00
Pavel Feldman
baf0e5b47a
chore: implement lightweight test listing (#11551) 2022-01-21 19:11:22 -08:00
Pavel Feldman
6b3e596fd8
fix(baseurl): support path-less baseurl (#11527) 2022-01-20 18:11:56 -08:00
Pavel Feldman
50fa839e64
chore: include transpilation mode in the compute cache hash (#11521) 2022-01-20 13:33:40 -08:00
Ross Wollman
33f32368e5
fix(test runner): dispatcher transformation of empty attachments (#11414) 2022-01-17 18:47:24 -08:00
Pavel Feldman
8ecf581b45
chore: focus by line should override only (#11427) 2022-01-16 08:47:09 -08:00
Pavel Feldman
8b0ef4ad6b
chore: introduce oop testing mode for debugging (#11428) 2022-01-15 19:25:32 -08:00
Max Schmitt
badb5b4d13
chore(pw-test): forward web-server stdout to debug target (#11384) 2022-01-13 23:55:46 +01:00
Dmitry Gozman
9d5bf0e90d
fix(test runner): fixture teardown double error, testInfo.attach() (#11365)
- Use file path, not content to calculate the attachment hash.
- Always cleanup fixture from the list on teardown, to avoid reporting
  teardown error multiple times: from the test, and from the cleanup.
2022-01-13 10:38:47 -08:00
Pavel Feldman
feb7148b3f
chore: don't write into stdout from the test runner, use reporters instead (#11367) 2022-01-12 19:52:40 -08:00
Andrey Lushnikov
9285596806
chore: cut v1.18.0 (#11358)
Drive-by: fix the `//utils/bump_package_versions.js` script.
2022-01-12 11:50:44 -08:00
Andrey Lushnikov
6f932fcb4a
fix: default list reporter should dump stderr to stderr (#11351)
This is consistent with all other reporters we have in place.
2022-01-12 07:43:25 -08:00
Dmitry Gozman
4efb30999f
feat(request): show request context methods as steps (#11337) 2022-01-11 17:33:41 -08:00
Andrey Lushnikov
2a0930c7a4
docs: clarification about trailing slash in docs (#11326)
Fixes #10557
2022-01-11 07:50:58 -08:00
Dmitry Gozman
da1f39fb29
chore(test runner): refactor worker runner parts (#11316)
This will make it easier to change in the future.
2022-01-10 20:25:56 -08:00
Dmitry Gozman
d36ff8a96c
fix(test runner): remove test output for failures (#11308) 2022-01-10 13:22:09 -08:00
Dmitry Gozman
14fd837e94
fix(test runner): hide beforeAll/afterAll hooks from the reporter api (#11306)
This api is not ready yet.
2022-01-10 12:09:51 -08:00
Andrey Lushnikov
7b1fecc009
fix: default to en-US locale in all browsers in Playwright Test (#11133)
It makes sense to default to en-US by default across all our browsers in Playwright Test.

Fixes #11127
2022-01-10 08:37:55 -08:00
Michaël De Boey
ff5f75dc10
fix: pin colors to 1.4.0 (#11274) 2022-01-09 16:18:21 -08:00
Dmitry Gozman
058f98d3dd
fix(test runner): revert error location from top frame (#11250)
This does not play nicely with some internal Playwright errors, so
it needs more tweaks.
2022-01-07 11:06:47 -08:00
Dmitry Gozman
19460d50e4
chore(test runner): process-wide cache of required test files (#11230)
This is a preparation to TestRunner api, where we should be
able to run tests multiple times in the same runner process.
2022-01-06 15:38:10 -08:00
Pavel Feldman
373042ed89
chore: don't allow importing @playwright/test twice (#11218) 2022-01-06 09:29:05 -08:00
Dmitry Gozman
3ecac56cc0
fix(test runner): testInfo.attach api review changes (#11211)
Remove overload, require name, merge options.
2022-01-05 16:39:33 -08:00
Dmitry Gozman
f77c874e8a
feat(test runner): make _extendTest experimental (#11210)
Hidden from types and docs.
2022-01-05 15:54:00 -08:00
Dmitry Gozman
ccc61e31ea
chore(test runner): preparation for TestRunner api (#11209) 2022-01-05 15:49:01 -08:00
Dmitry Gozman
3839917eb6
chore(test runner): move config loading to runner (#11186)
In preparation for the TestRunner api.
2022-01-05 13:44:29 -08:00
Dmitry Gozman
16a779a5ff
fix(test runner): show codeframe and location from the error top stack frame (#11179)
Previously, reporter would look for a stack frame directly in the test file.
Often times, that is not a top stack frame, especially when the test uses
some helper functions.

This changes error snippets and locations to use the top frame. When top
frame does not match the test file, we additionally show the location
to avoid confusion:

```
  1) a.spec.ts:7:7 › foobar ========================================================================

    Error: oh my

       at helper.ts:5

      3 |
      4 |       export function ohMy() {
    > 5 |         throw new Error('oh my');
        |               ^
      6 |       }
      7 |

        at ohMy (.../reporter-base-should-print-codeframe-from-a-helper/helper.ts:5:15)
        at .../reporter-base-should-print-codeframe-from-a-helper/a.spec.ts:8:9
        at FixtureRunner.resolveParametersAndRunHookOrTest (.../src/fixtures.ts:281:12)
```
2022-01-04 16:00:55 -08:00
Pavel Feldman
7bfa6f9b5f
feat(html): group similar items in the report (#11160) 2022-01-03 21:17:17 -08:00
Pavel Feldman
a0aeaeb929
test: expose repeatEachIndex (#11158) 2022-01-03 17:29:54 -08:00
Ross Wollman
dadb5cbc30
fix(html reporter): fix too much strikethrough in diffs (#11115)
Textual snapshot diffs were previously broken in the HTML Report. The strikethrough'd text extended beyond the intended region.

HTML Report Before: 
<img width="693" alt="Screen Shot 2021-12-27 at 4 43 35 PM" src="https://user-images.githubusercontent.com/11915034/147518750-a60f9002-6eed-48a1-a412-20fabd076fa6.png">

HTML Report After:
<img width="206" alt="Screen Shot 2021-12-27 at 4 48 37 PM" src="https://user-images.githubusercontent.com/11915034/147518762-19a4c8f9-ccc3-4a3c-a962-5a42edc6fc5d.png">

This now matches what's expected and shown in the terminal (which has always been correct):

<img width="1384" alt="Screen Shot 2021-12-27 at 4 36 29 PM" src="https://user-images.githubusercontent.com/11915034/147518799-f538259e-5a45-4d6f-916c-a12ccb620c5b.png">

NB: This MR is a workaround, but not a root cause fix. It works, but I never fully got to the root cause so a bug upstream may be required. It's unclear whether it's (1) in [`colors`](https://www.npmjs.com/package/colors), (2) in [`ansi-to-html`](https://www.npmjs.com/package/ansi-to-html), or (3) Playwright's use of the two. Since the terminal output is correct, I suspect it is in `ansi-to-html`. For example:

```js
const colors = require("colors");
const Convert = require('ansi-to-html');
const convert = new Convert();

// original (strike incorrectly wraps everything in the HTML)
console.log(convert.toHtml(colors.strikethrough("crossed out") + ' ' + colors.red("red")))
// prints: <strike>crossed out <span style="color:#A00">red<span style="color:#FFF"></span></span></strike>

// workaround
console.log(convert.toHtml(colors.reset(colors.strikethrough("crossed out")) + ' ' + colors.red("red")))
// prints: <strike>crossed out</strike> <span style="color:#A00">red<span style="color:#FFF"></span></span>
```

Fixes #11116
2021-12-28 09:56:34 -08:00
Dmitry Gozman
f933759ad1
chore(test runner): minor improvements (#11067)
- Types for fixture options and more.
- Refined type for deadline runner.
2021-12-22 09:59:58 -08:00
Dmitry Gozman
f5304e3bda
fix(fixtures): await fixture teardown when shutting down the worker (#11033) 2021-12-20 16:19:21 -08:00
Dmitry Gozman
2d00836f0e
fix(test runner): show the location of afterAll timeout (#11007) 2021-12-18 09:32:41 -08:00
Dmitry Gozman
c9ba49936f
feat(reporters): show retry #x when running a test (#11003) 2021-12-17 21:07:04 -08:00
Pavel Feldman
8f98074fc8
chore: add blink-diff third party library (#10984) 2021-12-17 15:53:37 -08:00
Dmitry Gozman
b6aad54b9f
fix(test runner): test.setTimeout whould not break debugging (#11004)
We ignore test.setTimeout() when timeout is already zero for debugging.
2021-12-17 15:17:48 -08:00
Dmitry Gozman
f5780be41b
fix(list reporter): make sure that duration suffix survives truncation (#11002) 2021-12-17 13:08:02 -08:00
Dmitry Gozman
93ad12978c
fix(test runner): disable expect, action and navigation timeouts on debug (#10958)
We disable these timeouts when test timeout is set to zero.
This covers PWDEBUG=1, --debug and manual `test.setTimeout(0)` scenarios.
2021-12-16 18:32:46 -08:00
Dmitry Gozman
192071d5bc
feat(test runner): save traces for beforeAll/afterAll hooks (#10950) 2021-12-15 16:06:10 -08:00
Dmitry Gozman
19b08332ce
feat(test runner): disable test timeout on page.pause() call (#10944) 2021-12-15 11:12:52 -08:00
Dmitry Gozman
0d54afab9c
feat(test runner): show beforeAll/afterAll hooks similar to tests (#10923)
Reporters now get notified about hooks start/end/steps.
2021-12-15 10:39:49 -08:00
Pavel Feldman
f579f9c806
chore: parse tsx tests (#10917) 2021-12-14 19:25:07 -08:00
Dmitry Gozman
34b84841b0
chore(test runner): create TestResult instances lazily in dispatcher (#10921)
This prepares for beforeAll/afterAll hooks to be handled in the same way.
Since we do not know in advance whether a hook will run, we must create
TestResults lazily.
2021-12-14 14:10:56 -08:00
Dmitry Gozman
5c4ebdce54
fix(line reporter): clarify about retries when going over total counter (#10901) 2021-12-13 19:06:13 -08:00
Joel Einbinder
c27491cd4d
feat(test-runner): shorten long output paths (#10523) 2021-12-13 10:56:03 -08:00
Joel Einbinder
7a02c52144
feat(test-runner): specific playwright types for expect (#10670) 2021-12-13 13:42:36 -05:00
Dmitry Gozman
308c7b4e32
docs: update test advanced guides (#10861)
Linking to the API reference, using better examples and newer docs.
2021-12-10 11:15:01 -08:00
Dmitry Gozman
897e41c6c1
docs: document TestOptions.video.size (#10827) 2021-12-09 07:38:58 -08:00
Pavel Feldman
b5933db279
feat(tsconfig): respect baseUrl and paths from tsconfig (#10525) 2021-12-08 22:43:00 -08:00
Pavel Feldman
4d683cef7f
fix(html): render text attachments as text (#10778) 2021-12-08 08:51:44 -08:00
Pavel Feldman
a08a41f6c9
chore: render annotations in html report (#10774) 2021-12-07 18:35:06 -08:00
Pavel Feldman
feb4c62da1
fix(html): html reporter fixes (#10770) 2021-12-07 16:47:47 -08:00
Pavel Feldman
7765131a14
feat(acceptDownload): revert acceptDownload (#10709) 2021-12-06 09:25:24 -08:00
Pavel Feldman
518d67add5
feat(test.info): expose information on the currently running test (#10708) 2021-12-06 09:25:11 -08:00
Dmitry Gozman
98e2f40bb0
docs: replace TestCase.suite with TestCase.parent (#10687)
It is there by mistake.
2021-12-02 14:24:43 -08:00
Pavel Feldman
31e0a63fcd
feat(toBeChecked): allow passing checked: false (#10665) 2021-12-02 10:31:26 -08:00
Pavel Feldman
b9731a904e
chore: add validations into check_deps (#10661) 2021-12-01 18:14:13 -08:00
Saransh Miglani
f05252874a
chore: minor code rearrangement (#10650) 2021-12-01 09:18:16 -08:00
Yury Semikhatsky
d66b7aab3b
feat(expext): toBeOK for APIResponse (#10596) 2021-11-30 18:12:19 -08:00
Dmitry Gozman
729da65eba
fix(test runner): allow multiple missing snapshots per test (#10621)
Instead of failing right away, continue test execution but mark
the test as failed.
2021-11-30 17:50:19 -08:00
Max Schmitt
3337920c76
docs: clarify webServer ipv4/ipv6 and baseURL protocol (#10610) 2021-11-30 20:01:28 +01:00
Max Schmitt
682d1f41c7
fix(test-runner): wrong test-runner options got determined (#10607)
Follow-up for #10507
2021-11-30 19:59:35 +01:00
Max Schmitt
9337aa3cdf
fix(test-runner): webServer port detection on Node.js 17 (#10538) 2021-11-29 19:36:35 +01:00
Joel Einbinder
15053d9b3b
fix(test-runner): apply default options to all browserTypes (#10507) 2021-11-29 12:21:15 -05:00
Max Schmitt
293c233a49
docs(test-runner): add webServer (#10531) 2021-11-29 18:01:14 +01:00
Joel Einbinder
20c0facfb9
fix(test runner): dont mangle test names with multiple dashes (#10447) 2021-11-24 19:36:38 -05:00
Pavel Feldman
5d19f16601
feat(esm): introduce experimental PW_EXPERIMENTAL_TS_ESM option (#10519) 2021-11-24 14:17:01 -08:00
Pavel Feldman
7eb3f76f49
feat(esm): allow running tests in type module projects (#10503) 2021-11-24 12:42:48 -08:00
Max Schmitt
85197e68c9
chore: support range requests in trace viewer http server (#10434) 2021-11-23 21:37:55 +01:00
Ross Wollman
854f321532
feat(api): add explicit async testInfo.attach (#10121)
feat(api): add explicit async testInfo.attach

We add an explicit async API for attaching file paths (and Buffers) to
tests that can be awaited to help users ensure they are attaching files
that actually exist at both the time of the invocation and later when
reporters (like the HTML Reporter) run and package up test artifacts.

This is intended to help surface attachment issues as soon as possible
so you aren't silently left with a missing attachment
minutes/days/months later when you go to debug a suddenly breaking test
expecting an attachment to be there.

NB: The current implemntation incurs an extra file copy compared to
manipulating the raw attachments array. If users encounter performance
issues because of this, we can consider an option parameter that uses
rename under the hood instead of copy. However, that would need to be
used with care if the file were to be accessed later in the test.
2021-11-23 09:30:53 -08:00
Dmitry Gozman
e647f0420c
docs: add more references to TestInfo.retry (#10472) 2021-11-22 10:06:20 -08:00
Dmitry Gozman
fde2f6a77f
docs: separate doc for test timeouts (#10448) 2021-11-19 17:06:46 -08:00
Dmitry Gozman
4eaeb3b59c
docs: explain that beforeAll/afterAll run again in the new worker process (#10446) 2021-11-19 13:47:30 -08:00
Dmitry Gozman
0302e759df
feat(test runner): allow top-level test.fixme similar to test.skip (#10250)
```js
test.fixme('my test name', () => {});
```
2021-11-19 11:40:40 -08:00
Dmitry Gozman
d9f849fb14
feat(test runner): replace declare/define with "options" (#10293)
1. Fixtures defined in test.extend() can now have `{ option: true }` configuration that makes them overridable in the config. Options support all other properties of fixtures - value/function, scope, auto.
```
const test = base.extend<MyOptions>({
  foo: ['default', { option: true }],
});
```

2. test.declare() and project.define are removed.

3. project.use applies overrides to default option values and nothing else. Any test.extend() and test.use() calls take priority over config options.

Required user changes: if someone used to define fixture options with test.extend(), overriding them in config will stop working. The solution is to add `{ option: true }`.

```
// Old code
export const test = base.extend<{ myOption: number, myFixture: number }>({
  myOption: 123,
  myFixture: ({ myOption }, use) => use(2 * myOption),
});

// New code
export const test = base.extend<{ myOption: number, myFixture: number }>({
  myOption: [123, { option: true }],
  myFixture: ({ myOption }, use) => use(2 * myOption),
});
```
2021-11-18 15:45:52 -08:00
Dmitry Gozman
8f43f4c98f
feat(serial): better errors from beforeAll (#10419)
When beforeAll hook times out or fails with an exception, we now
close the context and show a nice error.
2021-11-18 14:36:55 -08:00
Dmitry Gozman
bd93fc499f
fix(html reporter): show missing attachments as warnings (#10400) 2021-11-17 18:03:13 -08:00
Dmitry Gozman
ce2c0c59a7
feat(expect): show expect timeout in the error message (#10388)
Makes it easier to understand that expect does indeed have a separate timeout.

```
    Error: expect(received).toHaveCount(expected) // deep equality

    Expected: 0
    Received: 1

    Call log:
      - expect.toHaveCount with timeout 500ms
      - waiting for selector "span"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
```
2021-11-17 17:28:30 -08:00
Max Schmitt
0781d0303b
docs(python): enable web-first assertions (#10390) 2021-11-18 00:46:30 +01:00
Dmitry Gozman
638ebd6dd6
fix(test runner): do not validate fixtures in tests/hooks that are never run (#10328) 2021-11-15 13:17:26 -08:00
Dmitry Gozman
4bb123d4b7
fix(test runner): route more errors to reporter (#10263)
For example, top-level errors in files or global setup issues.
2021-11-12 06:47:41 -08:00
Andrey Lushnikov
bd1ce399e3
fix(html-report): fix command to open HTML report (#10231) 2021-11-12 00:12:23 -08:00
Pavel Feldman
8dcd77270f
fix(sigint): remove sigint handler early (#10266) 2021-11-11 17:14:36 -08:00
Dmitry Gozman
7eec66d0f9
chore(test runner): route runner errors through Reporter.onError (#10257) 2021-11-11 16:48:08 -08:00
Dmitry Gozman
f38f611478
chore: add npm run lint-tests (#10252) 2021-11-11 13:27:50 -08:00
Dmitry Gozman
4c93417e8a
fix(github reporter): handle global onError (#10256) 2021-11-11 13:25:38 -08:00
Dmitry Gozman
9622704a8a
fix(test runner): update TestInfo.duration before running afterEach hooks (#10228) 2021-11-10 16:02:27 -08:00
Andrey Lushnikov
b2af576796
chore: cut v1.17 (#10203) 2021-11-09 17:56:26 -08:00
Pavel Feldman
7b64161a37
feat(test-runner): allow specifying fine-grained trace options (#10147) 2021-11-08 15:39:58 -08:00
Joel Einbinder
0cad0de3e3
fix(test runner): better error message when importing typescript from esmodule (#10061) 2021-11-08 16:25:40 -05:00
Dmitry Gozman
0a104bc500
docs: clarify that slow tests are about files (#10134) 2021-11-08 12:54:18 -08:00
Dmitry Gozman
3c1aaa5338
fix(trace): do not call tracing.stopChunk() twice (#10054) 2021-11-04 21:08:42 -07:00
Dmitry Gozman
3120f81629
fix(reporters): move Running X tests using Y workers to reporters (#10014) 2021-11-03 11:17:23 -07:00
Dmitry Gozman
9cebe60831
feat(reporters): augment non-stdio reporters with dot/line (#10003) 2021-11-03 08:25:16 -07:00
Andrey Lushnikov
62a01a41c6
feat: various docker integration improvements (#9988)
- shorter docker terminal message
- terminate testrunner if docker cannot be found
- use `-docker` suffix for snapshots / screenshots.
2021-11-02 13:58:26 -07:00
Pavel Feldman
6a30c90590
feat(trace-viewer): render wall time for each action (#9982) 2021-11-02 12:16:12 -07:00
Max Schmitt
bedb817a00
docs(test-runner): worker fixtures only in before/after-All (#9967) 2021-11-02 16:51:41 +01:00
Nick Partridge
a51ac39275
feat: add snapshotDir to set base snapshot directory (#9260) 2021-11-02 08:02:49 -07:00
Pavel Feldman
56ca3a18f5
feat(trace-viewer): show test name (#9957) 2021-11-01 21:23:35 -07:00
Pavel Feldman
3673776330
Revert "chore(test-runner): use test name as trace file name prefix (#9619)" (#9956)
This reverts commit 3c420a7cf1.
2021-11-01 20:27:41 -07:00
Pavel Feldman
85497c5509
feat(tracing): allow including source files in traces (#9946) 2021-11-01 19:57:29 -07:00
Pavel Feldman
2bfbf65b8d
fix(html): strip ansi escaping from stdio (#9944) 2021-11-01 16:39:54 -07:00
Pavel Feldman
9ac8829583
feat(html): bake report zip into the html report, allow opening from fs (#9939) 2021-11-01 16:14:52 -07:00
Pavel Feldman
d79aae633c
feat(html): render the suite name before test title (#9909) 2021-11-01 10:53:42 -07:00
Dmitry Gozman
4f1027bdd0
feat(test runner): introduce TestInfo.parallelIndex (#9762)
This is a worker number between `0` and `workers - 1` that
does not change after worker process restart.
2021-11-01 10:37:34 -07:00
Max Schmitt
3c420a7cf1
chore(test-runner): use test name as trace file name prefix (#9619) 2021-11-01 18:15:19 +01:00
Dmitry Gozman
13844a5b0a
feat(html reporter): open test details page when running a single test (#9910) 2021-11-01 09:54:53 -07:00
Pavel Feldman
d6adff7792
fix(html): include html into the reporter types (#9907) 2021-10-31 11:06:28 -07:00
Pavel Feldman
8991bbde33
feat(html): live filtering, opt-out from auto-open (#9889) 2021-10-29 16:24:08 -07:00
Dmitry Gozman
49337890d2
docs: improve documentation for assertions and test config (#9890) 2021-10-29 15:25:52 -07:00
Dmitry Gozman
227e3da62f
fix(test runner): do not reuse worker that did not teardown scopes (#9872)
Two bug fixes:
- Do not use the worker that is being shutdown for a new job.
- Report unhandled errors during "expected to fail" tests as
  fatal errors.
2021-10-29 13:36:12 -07:00
Pavel Feldman
2e4722d460
test: migrate to upstream fixtures (#9835) 2021-10-28 08:31:30 -07:00
Joel Einbinder
c8addef03a
fix(monorepo): export package.json from every package (#9803) 2021-10-26 18:41:18 -04:00
Dmitry Gozman
22e4a0d580
fix(snapshots): --update-snapshots + retries should work (#9799) 2021-10-26 13:50:16 -07:00
Pavel Feldman
273122b761
chore: align internal test harness w/ @playwright/test (#9796) 2021-10-26 13:45:53 -07:00
Pavel Feldman
87c64b2c1c
chore: align client side instrumentations (#9771) 2021-10-26 11:13:35 -07:00
Pavel Feldman
5451808df1
chore: append sources to traces (#9736) 2021-10-25 18:56:57 -07:00
Dmitry Gozman
13ed1dee50
fix(test runner): do not mask uncaught error in beforeEach (#9764) 2021-10-25 14:17:27 -07:00
Pavel Feldman
f08c22b467
fix(trace-viewer): show source files in local version (#9732) 2021-10-23 11:23:39 -07:00
Dmitry Gozman
23aa0be15b
docs: update afterEach docs with an example (#9727)
Also add a test for TestInfo.status in afterEach.
2021-10-22 16:32:22 -07:00
Joel Einbinder
940466f830
fix(npm): don't ship src in @playwright/test and ensure we ship cli.js in playwright-core 2021-10-22 16:15:57 -04:00
Joel Einbinder
f2888395a6
fix: support even older node 12 (#9718) 2021-10-22 15:59:52 -04:00
Dmitry Gozman
193c79a685
feat(test runner): rewrite dispatcher to avoid unneded stalls (#9629)
In some circumstances, dispatcher was waiting for all exisitng jobs
to finish before scheduling a new one. This leads to unneded stalls.
Instead, we can schedule jobs right away, if we have a worker
available.
2021-10-22 11:10:37 -07:00
Max Schmitt
e17dbe8d01
chore: pin expect package (#9713) 2021-10-22 08:07:33 -07:00
Pavel Feldman
225145fc3e
fix(expect): do not fail on navigated frames while polling (#9659) 2021-10-20 13:01:05 -07:00
Pavel Feldman
b6a9a8a34a
Revert "chore: print global errors when running html report (#9657)" (#9661)
This reverts commit 68c9fce507.
2021-10-20 12:51:12 -07:00
Andrey Lushnikov
6b1767efbb
chore: cut v1.16.0 branch (#9656) 2021-10-20 12:44:32 -07:00
Pavel Feldman
68c9fce507
chore: print global errors when running html report (#9657) 2021-10-20 11:57:21 -07:00
Dmitry Gozman
fb421e0a65
feat(github reporter): update docs, strip ansi escapes (#9640) 2021-10-20 07:56:03 -07:00
Pavel Feldman
4423de9996
chore: speed up snippet generation 50 times (#9638) 2021-10-19 21:10:14 -07:00
Andrey Lushnikov
983cfde4d4
feat: introduce docker integration for Playwright Test (#9599) 2021-10-19 16:10:24 -07:00
Pavel Feldman
9d03a85c30
chore: make web-first matchers work outside of pw tests (#9624) 2021-10-19 14:46:56 -07:00
Joel Einbinder
adb5aa4544
fix(monorepo): support yarn2/pnp by specifying all deps and bins (#9620) 2021-10-19 14:49:47 -04:00
Joel Einbinder
53c866c0f2
fix(monorepo): explicitly list package.json exports instead of using glob patterns (#9584) 2021-10-19 12:28:02 -04:00
Dmitry Gozman
6d554a5e30
feat(reporter): expose more apis (#9603) 2021-10-19 08:38:04 -07:00
Yury Semikhatsky
6d727401bf
chore(fetch): rename ApiRequest* to APIRequest* (#9606) 2021-10-19 07:38:27 -07:00
Pavel Feldman
bccd4c8906
feat(api): expose step location UI (#9605) 2021-10-18 22:14:01 -07:00
Pavel Feldman
c06a6e1f63
fix(html): always highlight error code (#9604) 2021-10-18 21:37:19 -07:00
Pavel Feldman
e37660b068
feat(api): expose step location (#9602) 2021-10-18 21:06:18 -07:00
Dmitry Gozman
4977edcaf3
fix(tracing): fix inconsistent calls of tracing.{start,stop} in fixtures (#9594)
Drive-by: fix error logging.
2021-10-18 21:05:59 -07:00
Pavel Feldman
99a2bd604d
chore: minor UI updates (#9590) 2021-10-18 13:34:02 -07:00
Pavel Feldman
432fb453e4
chore(html): only copy trace viewer for reports with traces (#9579) 2021-10-18 08:03:04 -07:00
Pavel Feldman
6afa85927e
chore(ui): redesign html report a bit (#9577) 2021-10-17 20:58:06 -07:00
Paul Biggar
e473fc6a10
docs(test-runner): afterEach JSDoc description typo (#9542) 2021-10-16 12:36:07 +02:00
Feng Yu
0160c18f7a
docs: fix typo retried (#9562) 2021-10-15 19:24:03 -07:00
Pavel Feldman
99f3389b3b
feat(html): automatically use console reporter with html (#9559) 2021-10-15 19:18:56 -07:00
Pavel Feldman
9135847950
feat(ui): merge stdio in html report (#9560) 2021-10-15 16:15:06 -07:00
Pavel Feldman
e31b96cc26
feat(tracing): make context.request appear in the trace (#9555) 2021-10-15 15:22:49 -07:00
Yury Semikhatsky
235cd10a43
fix(runner): clarify where test.use() can be called (#9486) 2021-10-15 12:03:26 -07:00
Pavel Feldman
458945821f
doc(html): document html reporter (#9528) 2021-10-15 08:15:30 -07:00
Pavel Feldman
5ea4ec2f7a
feat(html): open show html report when there are failures (#9526) 2021-10-14 21:09:41 -07:00
Yury Semikhatsky
4680ef46de
docs: clarify default retry behavior (#9525) 2021-10-14 19:28:47 -07:00
Dmitry Gozman
c711fb35ad
fix(expect): do not rely on displayName (#9523)
Support for displayName was removed in Node 16.
Switching to Function.name instead.

Relevant V8 change: https://chromium-review.googlesource.com/c/v8/v8/+/2692189
2021-10-14 19:23:45 -07:00
Pavel Feldman
75cfe5d1f5
chore: start adding html runner e2e tests (#9518) 2021-10-14 15:48:05 -07:00
Pavel Feldman
584014f6fa
feat(html): unhide html reporter (#9512) 2021-10-14 11:17:35 -07:00
Joel Einbinder
c8f875a193
chore: move test runner types into the playwright test package (#9472)
Co-authored-by: Max Schmitt <max@schmitt.mx>
2021-10-14 11:55:08 +02:00
Pavel Feldman
bb77912aee
chore: don't include babel compiled files in webpack bundles (#9491) 2021-10-13 19:27:50 -07:00
Yury Semikhatsky
e827bde1c2
fix(runner): skip global hooks in --list mode (#9480) 2021-10-13 14:18:15 -07:00
Joel Einbinder
2a0a44bc09
chore: move diff_match_patch into the test package (#9475) 2021-10-13 13:38:26 -07:00
Pavel Feldman
cd99ad0da2
feat(trace): allow multiple traces in a single app, gc traces (#9478) 2021-10-13 13:31:54 -07:00
Pavel Feldman
8b1a887756
feat(html): link traces from html report (#9473) 2021-10-13 11:07:29 -07:00
Dmitry Gozman
64a3099655
chore(expect): simplify expect implementation (#9459) 2021-10-13 08:56:57 -07:00
Yury Semikhatsky
82ff85b106
fix(matchers): make isNot required in protocol (#9436) 2021-10-12 09:01:46 -07:00
Max Schmitt
b1160ec239
chore(test-runner): make outputDir based on cwd instead of testDir (#9360) 2021-10-11 22:24:46 +02:00
Joel Einbinder
d34b53a0d7
feat(test runner): allow comparing binary files to strings (#9315)
Co-authored-by: Max Schmitt <max@schmitt.mx>
2021-10-11 21:57:59 +02:00
Max Schmitt
f0ff2a23d2
chore: add README's for packages (#9432) 2021-10-11 19:28:10 +02:00
Joel Einbinder
c89d5a50dd
chore: migrate to monorepo (#9421) 2021-10-11 16:52:17 +02:00
Dmitry Gozman
167db03f05
feat(test-runner): export reporter api as @playwright/test/reporter (#7692) 2021-07-16 21:15:03 -07:00
Joel Einbinder
7629587914
fix(test-runner): work with .mjs files (#7373) 2021-06-29 15:28:41 -07:00
Joel Einbinder
0af1f4399d
fix(test-runner): fix default export breaking on installation tests (#7331) 2021-06-25 22:52:04 -07:00
Joel Einbinder
7caf05b24a
fix(playwright-test): have the proper default export (#7328)
There are 3 ways to import `@playwright/test` library in the modern Node.js ecosystem:
- Using `require`: works great, this patch doesn't change it
- Using `import` statement from `.mjs` file - we have wrong `default` for @playwright/test that should be a `test`. This is what test checks for
- Using `import test from '@playwright/test'` from `.ts` file - was broken because TypeScript thought it's a CJS module, whereas it's a ESM module in reality.

Also, typescript types import from `.d.ts` file was broken because we had no default export (`export *` syntax does not export default).
2021-06-25 15:29:22 -07:00
Pavel Feldman
deec8a4adb
docs: make docs test-first (#7214) 2021-06-18 10:09:38 -07:00
Dmitry Gozman
b625a04adc
types: move exported objects from index.d.ts to overrides.d.ts (#7152)
This way we generate these types right away, without duplicating them everywhere.
2021-06-15 15:06:18 -07:00
Dmitry Gozman
f745bf1fbc
chore: bring in folio source (#6923)
- Source now lives at `src/test`.
- Former folio tests live at `tests/playwright-test`.
- We use `src/test/internal.ts` that exposes base test without
  Playwright fixtures for most tests (to avoid modifications for now).
- Test types live in `types/testFoo.d.ts`.
- Stable test runner is installed to `tests/config/test-runner` during `npm install`.
- All deps including test-only are now listed in `package.json`.
  Non-test deps must also be listed in `build_package.js` to get included.
2021-06-06 17:09:53 -07:00
Dmitry Gozman
3de3a88930
feat(test): introduce npx playwright test (#6816) 2021-06-03 08:07:55 -07:00