Commit Graph

115 Commits

Author SHA1 Message Date
Max Schmitt
3337920c76
docs: clarify webServer ipv4/ipv6 and baseURL protocol (#10610) 2021-11-30 20:01:28 +01:00
Max Schmitt
682d1f41c7
fix(test-runner): wrong test-runner options got determined (#10607)
Follow-up for #10507
2021-11-30 19:59:35 +01:00
Max Schmitt
9337aa3cdf
fix(test-runner): webServer port detection on Node.js 17 (#10538) 2021-11-29 19:36:35 +01:00
Joel Einbinder
15053d9b3b
fix(test-runner): apply default options to all browserTypes (#10507) 2021-11-29 12:21:15 -05:00
Max Schmitt
293c233a49
docs(test-runner): add webServer (#10531) 2021-11-29 18:01:14 +01:00
Joel Einbinder
20c0facfb9
fix(test runner): dont mangle test names with multiple dashes (#10447) 2021-11-24 19:36:38 -05:00
Pavel Feldman
5d19f16601
feat(esm): introduce experimental PW_EXPERIMENTAL_TS_ESM option (#10519) 2021-11-24 14:17:01 -08:00
Pavel Feldman
7eb3f76f49
feat(esm): allow running tests in type module projects (#10503) 2021-11-24 12:42:48 -08:00
Max Schmitt
85197e68c9
chore: support range requests in trace viewer http server (#10434) 2021-11-23 21:37:55 +01:00
Ross Wollman
854f321532
feat(api): add explicit async testInfo.attach (#10121)
feat(api): add explicit async testInfo.attach

We add an explicit async API for attaching file paths (and Buffers) to
tests that can be awaited to help users ensure they are attaching files
that actually exist at both the time of the invocation and later when
reporters (like the HTML Reporter) run and package up test artifacts.

This is intended to help surface attachment issues as soon as possible
so you aren't silently left with a missing attachment
minutes/days/months later when you go to debug a suddenly breaking test
expecting an attachment to be there.

NB: The current implemntation incurs an extra file copy compared to
manipulating the raw attachments array. If users encounter performance
issues because of this, we can consider an option parameter that uses
rename under the hood instead of copy. However, that would need to be
used with care if the file were to be accessed later in the test.
2021-11-23 09:30:53 -08:00
Dmitry Gozman
e647f0420c
docs: add more references to TestInfo.retry (#10472) 2021-11-22 10:06:20 -08:00
Dmitry Gozman
fde2f6a77f
docs: separate doc for test timeouts (#10448) 2021-11-19 17:06:46 -08:00
Dmitry Gozman
4eaeb3b59c
docs: explain that beforeAll/afterAll run again in the new worker process (#10446) 2021-11-19 13:47:30 -08:00
Dmitry Gozman
0302e759df
feat(test runner): allow top-level test.fixme similar to test.skip (#10250)
```js
test.fixme('my test name', () => {});
```
2021-11-19 11:40:40 -08:00
Dmitry Gozman
d9f849fb14
feat(test runner): replace declare/define with "options" (#10293)
1. Fixtures defined in test.extend() can now have `{ option: true }` configuration that makes them overridable in the config. Options support all other properties of fixtures - value/function, scope, auto.
```
const test = base.extend<MyOptions>({
  foo: ['default', { option: true }],
});
```

2. test.declare() and project.define are removed.

3. project.use applies overrides to default option values and nothing else. Any test.extend() and test.use() calls take priority over config options.

Required user changes: if someone used to define fixture options with test.extend(), overriding them in config will stop working. The solution is to add `{ option: true }`.

```
// Old code
export const test = base.extend<{ myOption: number, myFixture: number }>({
  myOption: 123,
  myFixture: ({ myOption }, use) => use(2 * myOption),
});

// New code
export const test = base.extend<{ myOption: number, myFixture: number }>({
  myOption: [123, { option: true }],
  myFixture: ({ myOption }, use) => use(2 * myOption),
});
```
2021-11-18 15:45:52 -08:00
Dmitry Gozman
8f43f4c98f
feat(serial): better errors from beforeAll (#10419)
When beforeAll hook times out or fails with an exception, we now
close the context and show a nice error.
2021-11-18 14:36:55 -08:00
Dmitry Gozman
bd93fc499f
fix(html reporter): show missing attachments as warnings (#10400) 2021-11-17 18:03:13 -08:00
Dmitry Gozman
ce2c0c59a7
feat(expect): show expect timeout in the error message (#10388)
Makes it easier to understand that expect does indeed have a separate timeout.

```
    Error: expect(received).toHaveCount(expected) // deep equality

    Expected: 0
    Received: 1

    Call log:
      - expect.toHaveCount with timeout 500ms
      - waiting for selector "span"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
      -   selector resolved to 1 element
      -   unexpected value "1"
```
2021-11-17 17:28:30 -08:00
Max Schmitt
0781d0303b
docs(python): enable web-first assertions (#10390) 2021-11-18 00:46:30 +01:00
Dmitry Gozman
638ebd6dd6
fix(test runner): do not validate fixtures in tests/hooks that are never run (#10328) 2021-11-15 13:17:26 -08:00
Dmitry Gozman
4bb123d4b7
fix(test runner): route more errors to reporter (#10263)
For example, top-level errors in files or global setup issues.
2021-11-12 06:47:41 -08:00
Andrey Lushnikov
bd1ce399e3
fix(html-report): fix command to open HTML report (#10231) 2021-11-12 00:12:23 -08:00
Pavel Feldman
8dcd77270f
fix(sigint): remove sigint handler early (#10266) 2021-11-11 17:14:36 -08:00
Dmitry Gozman
7eec66d0f9
chore(test runner): route runner errors through Reporter.onError (#10257) 2021-11-11 16:48:08 -08:00
Dmitry Gozman
f38f611478
chore: add npm run lint-tests (#10252) 2021-11-11 13:27:50 -08:00
Dmitry Gozman
4c93417e8a
fix(github reporter): handle global onError (#10256) 2021-11-11 13:25:38 -08:00
Dmitry Gozman
9622704a8a
fix(test runner): update TestInfo.duration before running afterEach hooks (#10228) 2021-11-10 16:02:27 -08:00
Andrey Lushnikov
b2af576796
chore: cut v1.17 (#10203) 2021-11-09 17:56:26 -08:00
Pavel Feldman
7b64161a37
feat(test-runner): allow specifying fine-grained trace options (#10147) 2021-11-08 15:39:58 -08:00
Joel Einbinder
0cad0de3e3
fix(test runner): better error message when importing typescript from esmodule (#10061) 2021-11-08 16:25:40 -05:00
Dmitry Gozman
0a104bc500
docs: clarify that slow tests are about files (#10134) 2021-11-08 12:54:18 -08:00
Dmitry Gozman
3c1aaa5338
fix(trace): do not call tracing.stopChunk() twice (#10054) 2021-11-04 21:08:42 -07:00
Dmitry Gozman
3120f81629
fix(reporters): move Running X tests using Y workers to reporters (#10014) 2021-11-03 11:17:23 -07:00
Dmitry Gozman
9cebe60831
feat(reporters): augment non-stdio reporters with dot/line (#10003) 2021-11-03 08:25:16 -07:00
Andrey Lushnikov
62a01a41c6
feat: various docker integration improvements (#9988)
- shorter docker terminal message
- terminate testrunner if docker cannot be found
- use `-docker` suffix for snapshots / screenshots.
2021-11-02 13:58:26 -07:00
Pavel Feldman
6a30c90590
feat(trace-viewer): render wall time for each action (#9982) 2021-11-02 12:16:12 -07:00
Max Schmitt
bedb817a00
docs(test-runner): worker fixtures only in before/after-All (#9967) 2021-11-02 16:51:41 +01:00
Nick Partridge
a51ac39275
feat: add snapshotDir to set base snapshot directory (#9260) 2021-11-02 08:02:49 -07:00
Pavel Feldman
56ca3a18f5
feat(trace-viewer): show test name (#9957) 2021-11-01 21:23:35 -07:00
Pavel Feldman
3673776330
Revert "chore(test-runner): use test name as trace file name prefix (#9619)" (#9956)
This reverts commit 3c420a7cf1.
2021-11-01 20:27:41 -07:00
Pavel Feldman
85497c5509
feat(tracing): allow including source files in traces (#9946) 2021-11-01 19:57:29 -07:00
Pavel Feldman
2bfbf65b8d
fix(html): strip ansi escaping from stdio (#9944) 2021-11-01 16:39:54 -07:00
Pavel Feldman
9ac8829583
feat(html): bake report zip into the html report, allow opening from fs (#9939) 2021-11-01 16:14:52 -07:00
Pavel Feldman
d79aae633c
feat(html): render the suite name before test title (#9909) 2021-11-01 10:53:42 -07:00
Dmitry Gozman
4f1027bdd0
feat(test runner): introduce TestInfo.parallelIndex (#9762)
This is a worker number between `0` and `workers - 1` that
does not change after worker process restart.
2021-11-01 10:37:34 -07:00
Max Schmitt
3c420a7cf1
chore(test-runner): use test name as trace file name prefix (#9619) 2021-11-01 18:15:19 +01:00
Dmitry Gozman
13844a5b0a
feat(html reporter): open test details page when running a single test (#9910) 2021-11-01 09:54:53 -07:00
Pavel Feldman
d6adff7792
fix(html): include html into the reporter types (#9907) 2021-10-31 11:06:28 -07:00
Pavel Feldman
8991bbde33
feat(html): live filtering, opt-out from auto-open (#9889) 2021-10-29 16:24:08 -07:00
Dmitry Gozman
49337890d2
docs: improve documentation for assertions and test config (#9890) 2021-10-29 15:25:52 -07:00