Commit Graph

165 Commits

Author SHA1 Message Date
Andrey Lushnikov
9285596806
chore: cut v1.18.0 (#11358)
Drive-by: fix the `//utils/bump_package_versions.js` script.
2022-01-12 11:50:44 -08:00
Andrey Lushnikov
6f932fcb4a
fix: default list reporter should dump stderr to stderr (#11351)
This is consistent with all other reporters we have in place.
2022-01-12 07:43:25 -08:00
Dmitry Gozman
4efb30999f
feat(request): show request context methods as steps (#11337) 2022-01-11 17:33:41 -08:00
Andrey Lushnikov
2a0930c7a4
docs: clarification about trailing slash in docs (#11326)
Fixes #10557
2022-01-11 07:50:58 -08:00
Dmitry Gozman
da1f39fb29
chore(test runner): refactor worker runner parts (#11316)
This will make it easier to change in the future.
2022-01-10 20:25:56 -08:00
Dmitry Gozman
d36ff8a96c
fix(test runner): remove test output for failures (#11308) 2022-01-10 13:22:09 -08:00
Dmitry Gozman
14fd837e94
fix(test runner): hide beforeAll/afterAll hooks from the reporter api (#11306)
This api is not ready yet.
2022-01-10 12:09:51 -08:00
Andrey Lushnikov
7b1fecc009
fix: default to en-US locale in all browsers in Playwright Test (#11133)
It makes sense to default to en-US by default across all our browsers in Playwright Test.

Fixes #11127
2022-01-10 08:37:55 -08:00
Michaël De Boey
ff5f75dc10
fix: pin colors to 1.4.0 (#11274) 2022-01-09 16:18:21 -08:00
Dmitry Gozman
058f98d3dd
fix(test runner): revert error location from top frame (#11250)
This does not play nicely with some internal Playwright errors, so
it needs more tweaks.
2022-01-07 11:06:47 -08:00
Dmitry Gozman
19460d50e4
chore(test runner): process-wide cache of required test files (#11230)
This is a preparation to TestRunner api, where we should be
able to run tests multiple times in the same runner process.
2022-01-06 15:38:10 -08:00
Pavel Feldman
373042ed89
chore: don't allow importing @playwright/test twice (#11218) 2022-01-06 09:29:05 -08:00
Dmitry Gozman
3ecac56cc0
fix(test runner): testInfo.attach api review changes (#11211)
Remove overload, require name, merge options.
2022-01-05 16:39:33 -08:00
Dmitry Gozman
f77c874e8a
feat(test runner): make _extendTest experimental (#11210)
Hidden from types and docs.
2022-01-05 15:54:00 -08:00
Dmitry Gozman
ccc61e31ea
chore(test runner): preparation for TestRunner api (#11209) 2022-01-05 15:49:01 -08:00
Dmitry Gozman
3839917eb6
chore(test runner): move config loading to runner (#11186)
In preparation for the TestRunner api.
2022-01-05 13:44:29 -08:00
Dmitry Gozman
16a779a5ff
fix(test runner): show codeframe and location from the error top stack frame (#11179)
Previously, reporter would look for a stack frame directly in the test file.
Often times, that is not a top stack frame, especially when the test uses
some helper functions.

This changes error snippets and locations to use the top frame. When top
frame does not match the test file, we additionally show the location
to avoid confusion:

```
  1) a.spec.ts:7:7 › foobar ========================================================================

    Error: oh my

       at helper.ts:5

      3 |
      4 |       export function ohMy() {
    > 5 |         throw new Error('oh my');
        |               ^
      6 |       }
      7 |

        at ohMy (.../reporter-base-should-print-codeframe-from-a-helper/helper.ts:5:15)
        at .../reporter-base-should-print-codeframe-from-a-helper/a.spec.ts:8:9
        at FixtureRunner.resolveParametersAndRunHookOrTest (.../src/fixtures.ts:281:12)
```
2022-01-04 16:00:55 -08:00
Pavel Feldman
7bfa6f9b5f
feat(html): group similar items in the report (#11160) 2022-01-03 21:17:17 -08:00
Pavel Feldman
a0aeaeb929
test: expose repeatEachIndex (#11158) 2022-01-03 17:29:54 -08:00
Ross Wollman
dadb5cbc30
fix(html reporter): fix too much strikethrough in diffs (#11115)
Textual snapshot diffs were previously broken in the HTML Report. The strikethrough'd text extended beyond the intended region.

HTML Report Before: 
<img width="693" alt="Screen Shot 2021-12-27 at 4 43 35 PM" src="https://user-images.githubusercontent.com/11915034/147518750-a60f9002-6eed-48a1-a412-20fabd076fa6.png">

HTML Report After:
<img width="206" alt="Screen Shot 2021-12-27 at 4 48 37 PM" src="https://user-images.githubusercontent.com/11915034/147518762-19a4c8f9-ccc3-4a3c-a962-5a42edc6fc5d.png">

This now matches what's expected and shown in the terminal (which has always been correct):

<img width="1384" alt="Screen Shot 2021-12-27 at 4 36 29 PM" src="https://user-images.githubusercontent.com/11915034/147518799-f538259e-5a45-4d6f-916c-a12ccb620c5b.png">

NB: This MR is a workaround, but not a root cause fix. It works, but I never fully got to the root cause so a bug upstream may be required. It's unclear whether it's (1) in [`colors`](https://www.npmjs.com/package/colors), (2) in [`ansi-to-html`](https://www.npmjs.com/package/ansi-to-html), or (3) Playwright's use of the two. Since the terminal output is correct, I suspect it is in `ansi-to-html`. For example:

```js
const colors = require("colors");
const Convert = require('ansi-to-html');
const convert = new Convert();

// original (strike incorrectly wraps everything in the HTML)
console.log(convert.toHtml(colors.strikethrough("crossed out") + ' ' + colors.red("red")))
// prints: <strike>crossed out <span style="color:#A00">red<span style="color:#FFF"></span></span></strike>

// workaround
console.log(convert.toHtml(colors.reset(colors.strikethrough("crossed out")) + ' ' + colors.red("red")))
// prints: <strike>crossed out</strike> <span style="color:#A00">red<span style="color:#FFF"></span></span>
```

Fixes #11116
2021-12-28 09:56:34 -08:00
Dmitry Gozman
f933759ad1
chore(test runner): minor improvements (#11067)
- Types for fixture options and more.
- Refined type for deadline runner.
2021-12-22 09:59:58 -08:00
Dmitry Gozman
f5304e3bda
fix(fixtures): await fixture teardown when shutting down the worker (#11033) 2021-12-20 16:19:21 -08:00
Dmitry Gozman
2d00836f0e
fix(test runner): show the location of afterAll timeout (#11007) 2021-12-18 09:32:41 -08:00
Dmitry Gozman
c9ba49936f
feat(reporters): show retry #x when running a test (#11003) 2021-12-17 21:07:04 -08:00
Pavel Feldman
8f98074fc8
chore: add blink-diff third party library (#10984) 2021-12-17 15:53:37 -08:00
Dmitry Gozman
b6aad54b9f
fix(test runner): test.setTimeout whould not break debugging (#11004)
We ignore test.setTimeout() when timeout is already zero for debugging.
2021-12-17 15:17:48 -08:00
Dmitry Gozman
f5780be41b
fix(list reporter): make sure that duration suffix survives truncation (#11002) 2021-12-17 13:08:02 -08:00
Dmitry Gozman
93ad12978c
fix(test runner): disable expect, action and navigation timeouts on debug (#10958)
We disable these timeouts when test timeout is set to zero.
This covers PWDEBUG=1, --debug and manual `test.setTimeout(0)` scenarios.
2021-12-16 18:32:46 -08:00
Dmitry Gozman
192071d5bc
feat(test runner): save traces for beforeAll/afterAll hooks (#10950) 2021-12-15 16:06:10 -08:00
Dmitry Gozman
19b08332ce
feat(test runner): disable test timeout on page.pause() call (#10944) 2021-12-15 11:12:52 -08:00
Dmitry Gozman
0d54afab9c
feat(test runner): show beforeAll/afterAll hooks similar to tests (#10923)
Reporters now get notified about hooks start/end/steps.
2021-12-15 10:39:49 -08:00
Pavel Feldman
f579f9c806
chore: parse tsx tests (#10917) 2021-12-14 19:25:07 -08:00
Dmitry Gozman
34b84841b0
chore(test runner): create TestResult instances lazily in dispatcher (#10921)
This prepares for beforeAll/afterAll hooks to be handled in the same way.
Since we do not know in advance whether a hook will run, we must create
TestResults lazily.
2021-12-14 14:10:56 -08:00
Dmitry Gozman
5c4ebdce54
fix(line reporter): clarify about retries when going over total counter (#10901) 2021-12-13 19:06:13 -08:00
Joel Einbinder
c27491cd4d
feat(test-runner): shorten long output paths (#10523) 2021-12-13 10:56:03 -08:00
Joel Einbinder
7a02c52144
feat(test-runner): specific playwright types for expect (#10670) 2021-12-13 13:42:36 -05:00
Dmitry Gozman
308c7b4e32
docs: update test advanced guides (#10861)
Linking to the API reference, using better examples and newer docs.
2021-12-10 11:15:01 -08:00
Dmitry Gozman
897e41c6c1
docs: document TestOptions.video.size (#10827) 2021-12-09 07:38:58 -08:00
Pavel Feldman
b5933db279
feat(tsconfig): respect baseUrl and paths from tsconfig (#10525) 2021-12-08 22:43:00 -08:00
Pavel Feldman
4d683cef7f
fix(html): render text attachments as text (#10778) 2021-12-08 08:51:44 -08:00
Pavel Feldman
a08a41f6c9
chore: render annotations in html report (#10774) 2021-12-07 18:35:06 -08:00
Pavel Feldman
feb4c62da1
fix(html): html reporter fixes (#10770) 2021-12-07 16:47:47 -08:00
Pavel Feldman
7765131a14
feat(acceptDownload): revert acceptDownload (#10709) 2021-12-06 09:25:24 -08:00
Pavel Feldman
518d67add5
feat(test.info): expose information on the currently running test (#10708) 2021-12-06 09:25:11 -08:00
Dmitry Gozman
98e2f40bb0
docs: replace TestCase.suite with TestCase.parent (#10687)
It is there by mistake.
2021-12-02 14:24:43 -08:00
Pavel Feldman
31e0a63fcd
feat(toBeChecked): allow passing checked: false (#10665) 2021-12-02 10:31:26 -08:00
Pavel Feldman
b9731a904e
chore: add validations into check_deps (#10661) 2021-12-01 18:14:13 -08:00
Saransh Miglani
f05252874a
chore: minor code rearrangement (#10650) 2021-12-01 09:18:16 -08:00
Yury Semikhatsky
d66b7aab3b
feat(expext): toBeOK for APIResponse (#10596) 2021-11-30 18:12:19 -08:00
Dmitry Gozman
729da65eba
fix(test runner): allow multiple missing snapshots per test (#10621)
Instead of failing right away, continue test execution but mark
the test as failed.
2021-11-30 17:50:19 -08:00