Commit Graph

1105 Commits

Author SHA1 Message Date
Yury Semikhatsky
cdef8171c1
chore: extract remote connection to a separate file (#13331) 2022-04-05 14:47:11 -07:00
github-actions[bot]
67c251f483
feat(chromium): roll to r1000 (#13327)
References #13198

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-04-05 11:05:01 -07:00
Max Schmitt
844ecbe2eb
chore: better Electron error message when no executable path is given (#13318) 2022-04-05 18:45:21 +02:00
Dmitry Gozman
e31a5b690a
chore: unrelease toHaveScreenshot and screenshotsDir (#13304) 2022-04-05 08:34:51 -07:00
Pavel Feldman
a2f94f6336
fix: trace viewer pwa should have no base (#13308) 2022-04-04 21:53:19 -07:00
Pavel Feldman
23d9dbc011
chore: rewrite non-web protocol for Electron tracing (#13305) 2022-04-04 20:56:04 -07:00
Dmitry Gozman
a71c151438
chore: introduce FullProjectInternal (#13301)
This is similar to FullConfigInternal to put private stuff on.
2022-04-04 16:07:04 -07:00
Dmitry Gozman
a09b8dfa36
chore: unrelease 'fonts' screenshot option (#13300) 2022-04-04 14:51:11 -07:00
Thomas Scholtes
3e65ef35cf
fix: wait for worker to be ready before initializing (#13270)
fix: wait for worker to be ready before initializing

The test dispatcher waits until a worker is ready before sending the
`init` message. This guarantees that the worker process is listening for the
message. Otherwise, the worker process might miss the `init` message and
the subsequent `run` message would crash the worker.
2022-04-04 13:56:57 -07:00
Dmitry Gozman
4e1fb1728f
docs: small clarifications to hasText and has-text (#13298) 2022-04-04 13:18:03 -07:00
Pavel Feldman
1bebc28aed
chore: reset page/context for reuse in component tests (#13264) 2022-04-04 12:39:43 -07:00
Pavel Feldman
8232497c88
feat(electron): expose app process(), detach on exit (#13280) 2022-04-04 11:50:46 -07:00
Ross Wollman
42798b5857
chore: warn against using globally scoped install (#13196) 2022-04-03 20:26:23 -07:00
Pavel Feldman
b0103566c9
fix(addInitScript): tolerate trailing comments (#13275) 2022-04-03 18:47:12 -07:00
Pavel Feldman
bcb12fcf7f
chore: mark public methods on server as such (#13271) 2022-04-02 19:02:27 -07:00
kaivean
5d2e8918d8
feat(android): add androidDevice.options.omitDriverInstall (#13249) 2022-04-02 16:00:38 -07:00
Dmitry Gozman
f6936c0461
chore: move private fields to FullConfigInternal (#13261) 2022-04-01 18:32:34 -07:00
Pavel Feldman
66cf82766e
test(html-reporter): add image diff tests (#13262) 2022-04-01 18:11:15 -07:00
Pavel Feldman
55ee41c848
feat(html): render image diff slider (#13257) 2022-04-01 15:27:51 -07:00
Ross Wollman
12abae7f31
feat(test-runner): friendly expect errors for typos (#13229)
If you typo'd an `expect` property, you got a cryptic error message:

```
Uncaught TypeError: Cannot create proxy with a non-object as target or handler
```

Now we get this nice friendly message:

```
  1) a.spec.ts:6:9 › explodes ======================================================================

    Error: expect: Property 'toBeLessThen' not found.

    Did you mean 'toBeLessThan'?

    See https://playwright.dev/docs/test-assertions for available options and documentation.

      5 |         const { test } = pwt;
      6 |         test('explodes', () => {
    > 7 |           expect.soft(1).toBeLessThen();
        |           ^
      8 |         });
      9 |

```

Fixes #13218
2022-04-01 13:38:22 -07:00
Pavel Feldman
c262a80762
chore: extract image diff component (#13221) 2022-04-01 12:53:56 -07:00
Ross Wollman
16efbdef98
chore: remove/rename FullConfig._attachments (#13233)
This was originally introduced in #12734.

It will be replaced with GlobalInfo (#13083), but not before the 1.21
release.
2022-04-01 12:36:05 -07:00
Dmitry Gozman
a200fe3528
feat(screenshot): rename "size" option to "scale" (#13254)
Drive-by: fix `caret` handling in `toHaveScreenshot`.
2022-04-01 12:28:40 -07:00
liuxingbaoyu
2eb1c8bc42
fix(show-trace): traceModel was removed by mistake (#12968) 2022-04-01 12:20:05 -07:00
Dmitry Gozman
356fc35b85
feat(role selector): allow unquoted name attribute (#13224)
- This supports `role=button[name=Hello]` similarly to CSS selectors.
- Does not change `_react` or `_vue` behavior that insist on quoting the string.
- Uses CSS notion of "identifier" characters.
2022-04-01 10:08:02 -07:00
Andrey Lushnikov
a87794dae6
fix: demand --force to re-install stable browser channels (#13200)
This patch will check if browser channel is already installed.
If it is, it'll abort installation with the following error:

```
aslushnikov:~/prog/playwright$ npx playwright install msedge
Failed to install browsers
Error:
╔═════════════════════════════════════════════════════════════════╗
║ ATTENTION: "msedge" is already installed on the system!         ║
║                                                                 ║
║ "msedge" installation is not hermetic; installing newer version ║
║ requires *removal* of a current installation first.             ║
║                                                                 ║
║ To *uninstall* current version and re-install latest "msedge":  ║
║                                                                 ║
║ - Close all running instances of "msedge", if any               ║
║ - Use "--force" to install browser:                             ║
║                                                                 ║
║     npx playwright install --force msedge                       ║
║                                                                 ║
║ <3 Playwright Team                                              ║
╚═════════════════════════════════════════════════════════════════╝
```

To re-install browser channel, use `--force`.

Fixes https://github.com/microsoft/playwright/issues/13061
2022-04-01 10:05:53 -07:00
Yury Semikhatsky
7d7fe3c618
fix(route): remove cors option, compare origin (#13231) 2022-03-31 19:21:21 -07:00
Dmitry Gozman
9fc95dda84
fix(screenshot): do not stall on hideHiglight attempt 2 (#13222)
It turns out that "non stalling evaluate" can stall in Chromium
in some weird conditions, like `document.open` after some weird
`iframe.src` value.

We now only hide highlight in those frames where we did install
highlight in the first place.
2022-03-31 15:33:29 -07:00
Pavel Feldman
6a463195c4
chore: allow multiple image diffs (#13202) 2022-03-31 14:11:34 -07:00
Dmitry Gozman
e5182259b1
feat(role selector): docs and minor fixes (#13203)
- Added docs to `selectors.md`.
- `[pressed]` and `[checked]` do not match `"mixed"` states.
- Disallow `[name]` shorthand without a value.
- Renamed `includeHidden` to `include-hidden`.
2022-03-31 13:06:39 -07:00
Pavel Feldman
b2c863f6a3
fix(tracing): account for screencast timestamps when computing timeline boundary of empty trace (#13214) 2022-03-31 09:45:45 -07:00
github-actions[bot]
28234b6540
feat(webkit): roll to r1623 (#13195) 2022-03-31 07:40:08 -07:00
Pavel Feldman
923f74c5a6
chore: allow matchers decorate step title (#13199) 2022-03-30 21:52:00 -07:00
Dmitry Gozman
ecc804d808
chore: prepare types to stricter checks in TS 4.7.0 (#13193) 2022-03-30 20:33:36 -07:00
Pavel Feldman
42f260c688
chore: prepare image diff for refactornig (#13197) 2022-03-30 17:42:08 -07:00
github-actions[bot]
6157c3e743
feat(webkit): roll to r1622 (#13187) 2022-03-30 15:39:14 -07:00
Yury Semikhatsky
c200321402
fix: export request from index.jsm (#13158) 2022-03-30 09:35:47 -07:00
Dmitry Gozman
a8d4a8aa52
fix(text selector): ignore non-leading quote when parsing (#13170)
Previously, any unpaired quote in the text selector "escaped"
everything till the end of the selector string, and so any
subsequent chained selectors, including ">>" separator were ignored.

An example of misbehaving selector: `text=19" >> nth=1`.

Now, when text selector contains a non-leading quote, selector parser
does not assume it should escape ">>" separator and correctly
tokenizes all selectors from the chain.

Note that this behavior is a workaround for the fact that our
text selectors is somewhat poorly defined in this area. That said,
this workaround seems to be safe enough. It still does not work for
unpaired leading quotes like this: `text="19 >> nth=1`.
2022-03-30 09:33:32 -07:00
Ross Wollman
2932da33f2
chore(html-reporter): remove no-longer-needed code (#13167) 2022-03-29 21:08:10 -07:00
Pavel Feldman
5e30375c8c
test(components): add more tests (#13168) 2022-03-29 21:06:11 -07:00
Pavel Feldman
81e7c0a77c
chore(html-report): render metainfo as a chip (#13166) 2022-03-29 18:13:08 -07:00
Andrey Lushnikov
a9989852d5
feat(vrt): new option "caret" for taking screenshots (#13164)
This has two values:
- `"hide"` to hide input caret for taking screenshot
- `"initial"` to keep caret behavior unchanged

Defaults to `"hide"`.

Fixes #12643
2022-03-29 17:48:13 -07:00
Pavel Feldman
5e17ed137b
fix(html-report): only invoke git once (#13165) 2022-03-29 15:49:58 -07:00
Dmitry Gozman
eb09306db2
feat(connect): add connectOptions.timeout (#13163) 2022-03-29 15:03:43 -07:00
Andrey Lushnikov
aa1daeba85
fix(html): put HTML report next to package.json by default (#13141)
Fixes #12970
2022-03-29 14:19:31 -07:00
Pavel Feldman
bd6e88b8be
test: start adding component tests (#13159) 2022-03-29 13:27:37 -07:00
Pavel Feldman
5575476697
chore: make webserver try index.html for / (#13155) 2022-03-29 12:19:56 -07:00
Dmitry Gozman
6b48631eed
feat(role): add more tests for accessible name (#13154) 2022-03-29 11:59:44 -07:00
github-actions[bot]
54f6e5db19
feat(firefox): roll to r1322 (#13150)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-29 11:28:20 -07:00
github-actions[bot]
29e0525926
feat(firefox-beta): roll to r1321 (#13142)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-29 01:16:25 -07:00
Pavel Feldman
1961959dcb
chore: migrate injected scripts to esbuild (#13143) 2022-03-28 23:10:17 -07:00
Dmitry Gozman
de0af27837
feat(test runner): show last tests in the worker on teardown error (#13139) 2022-03-28 19:58:24 -07:00
Pavel Feldman
8758cf8cbf
chore: migrate html reporter to vite (#13116) 2022-03-28 18:21:19 -07:00
Dmitry Gozman
d59c2b996f
fix(parallel): minimize the number of beforeAll/afterAll hooks in parallel mode (#13138)
Previously, we always formed groups consisting of a single test.
Now, we group tests that share `beforeAll`/`afterAll` hooks into
`config.workers` equally-sized groups.
2022-03-28 16:10:32 -07:00
Ross Wollman
ae365dda3b
chore: remove internal dep on FullConfig (#13074)
Resolves #13058.

This blocks adding #8206 since we don't want the new launch configs
publicly available on FullConfig.
2022-03-28 15:53:42 -07:00
Dmitry Gozman
01a39e5b4c
test(role): add tests for implicit role calculation (#13132) 2022-03-28 15:22:50 -07:00
Philip Sanetra
be41c4a35d
fix(registry): Fix support for generic-linux (#13129)
Fixes https://github.com/microsoft/playwright/issues/13128
2022-03-28 14:50:56 -07:00
Dmitry Gozman
9420a53939
fix(esm): make sure import from './foo.js' is supported (#13137)
Drive-by: migrate all @esm tests to esm.spec.ts.
2022-03-28 14:37:46 -07:00
Dmitry Gozman
8c19f71c36
feat(selectors): role selector engine (#12999)
This introduces `role=button[name="Click me"][pressed]` attribute-style
role selector. It is only available under `env.PLAYWRIGHT_EXPERIMENTAL_FEATURES`.

Supported attributes:
- `role` is required, for example `role=button`;
- `name` is accessible name, supports matching operators and regular expressions:
  `role=button[name=/Click(me)?/]`;
- `checked` boolean/mixed, for example `role=checkbox[checked=false]`;
- `selected` boolean, for example `role=option[selected]`;
- `expanded` boolean, for example `role=button[expanded=true]`;
- `disabled` boolean, for example `role=button[disabled]`;
- `level` number, for example `role=heading[level=3]`;
- `pressed` boolean/mixed, for example `role=button[pressed="mixed"]`;
- `includeHidden` - by default, only non-hidden elements are considered.
   Passing `role=button[includeHidden]` matches hidden elements as well.
2022-03-28 09:24:58 -07:00
Yury Semikhatsky
5734c18ef8
feat(route): add cors header in route.fulfill (#12943) 2022-03-25 14:56:57 -07:00
Andrey Lushnikov
4ab4c0bda1
feat: detect docker version and Playwright version mismatch (#12806)
This patch prints a friendly instructions in case Docker image version
mismatches Playwright version and there are missing browser
dependencies.

With this patch, Playwright will yield the following error:

```
root@f0774d2b2097:~# node a.mjs
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

browserType.launch:
╔════════════════════════════════════════════════════════════════════════════════════════════╗
║ Host system is missing dependencies to run browsers.                                       ║
║ This is most likely due to docker image version not matching Playwright version:           ║
║ - Playwright: 1.22.0                                                                       ║
║ -     Docker: 1.21.0                                                                       ║
║                                                                                            ║
║ Either:                                                                                    ║
║ - (recommended) use docker image "mcr.microsoft.com/playwright:v1.22.0-focal"              ║
║ - (alternative 1) run the following command inside docker to install missing dependencies: ║
║                                                                                            ║
║     npx playwright install-deps                                                            ║
║                                                                                            ║
║ - (alternative 2) use Aptitude inside docker:                                              ║
║                                                                                            ║
║     apt-get install libgbm1                                                                ║
║                                                                                            ║
║ <3 Playwright Team                                                                         ║
╚════════════════════════════════════════════════════════════════════════════════════════════╝
    at file:///root/a.mjs:3:10 {
  name: 'Error'
}```

Fixes #12796

Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2022-03-25 14:45:53 -07:00
Pavel Feldman
6b81e76c2b
chore: migrate trace-viewer and recorder to vite (#13076) 2022-03-25 14:12:00 -07:00
Yury Semikhatsky
97e8ead57c
feat: large files upload in ff and wk (#12937) 2022-03-25 13:26:12 -07:00
Max Schmitt
c8cc62a2e6
chore: add link validation for docs (#13047) 2022-03-25 19:30:45 +01:00
Pavel Feldman
3f1cb7b8e6
chore: only restart esm on Node 16+ (#12955) 2022-03-25 08:44:42 -07:00
Pavel Feldman
81d412216a
fix: list files when config writes to output (#13023) 2022-03-25 08:44:27 -07:00
Pavel Feldman
07c1a27732
chore(deps): prepare deps for monorepo (#13057) 2022-03-25 08:43:29 -07:00
github-actions[bot]
7a67c6d264
feat(webkit): roll to r1621 (#13056) 2022-03-25 08:40:19 -07:00
Max Schmitt
1797c5c249
chore(test-runner): support self signed certificate in webServer (#13032) 2022-03-24 17:30:52 +01:00
github-actions[bot]
8f6df440d5
feat(firefox-beta): roll to r1320 (#13010) 2022-03-24 07:49:50 -07:00
github-actions[bot]
920df240d1
feat(firefox): roll to r1321 (#13017) 2022-03-24 07:49:27 -07:00
Yury Semikhatsky
136c4e10e6
Revert "chore: align setInputFiles in webkit with other browsers (#13011)" (#13021)
This reverts commit 7fe763c1fc.
2022-03-24 07:46:37 -07:00
Dmitry Gozman
91408f2c5e
fix(storageState): provide nice error message for storage state issues (#13019) 2022-03-24 07:33:51 -07:00
Dmitry Gozman
3688e74e3e
fix(list mode): print errors to stderr (#13016)
`--list` mode now prints any errors encountered during test collection,
for example syntax errors, to `stderr`.
2022-03-24 07:33:33 -07:00
Yury Semikhatsky
7fe763c1fc
chore: align setInputFiles in webkit with other browsers (#13011) 2022-03-23 16:07:47 -07:00
Andrey Lushnikov
e1700bd167
feat: use package.json directory as a default for outputDir (#12942)
This patch:
- starts using directory of `package.json` to resolve default
  output directory path
- starts using either `package.json` directory or configuration
  directory to resolve all relative paths

References #12970
2022-03-23 16:05:49 -07:00
Ross Wollman
39376cccd8
fix: allow unnamed cookies (#12991)
Fixes #12808.
2022-03-23 15:50:43 -07:00
Andrey Lushnikov
bbc1a4fea0
fix: print all missing libraries when used on unsupported linux distro (#12966)
This patch:
- Adds 3 new host platform types:
  * `generic-linux` and `generic-linux-arm64` for the unsupported
    linux distributions
  * `<unknown>` for non-supported OS versions
- Prints a warning when downloading Ubuntu browser builds on
  unsupported Linux distribution
- Makes sure launch doctor prints all missing shared libraries
  on unknown Linux distributions
- Also prints an `apt` command as an alternative to Playwright CLI
  dependency installation.
2022-03-23 14:06:14 -07:00
github-actions[bot]
53b84541ab
feat(chromium): roll to r983172 (#12995)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-23 20:34:23 +01:00
github-actions[bot]
2d5762c842
feat(webkit): roll to r1620 (#12971) 2022-03-23 11:30:42 -07:00
Max Schmitt
b69d0566bb
chore: disable Chromium CertificateTransparencyComponentUpdater feature (#12992) 2022-03-23 19:14:22 +01:00
Andrew Hayward
ab39cfcb18
docs: fixing example code in accessibility API docs (#12664)
Co-authored-by: Andrey Lushnikov <aslushnikov@gmail.com>
2022-03-23 11:50:35 +01:00
Dmitry Gozman
722302799e
feat(selectors): support regular expressions in attribute selectors (#12960)
Supports inline regex in addition to string: `_react=BookItem[author = /Ann?a/i]`.
This is similar to `text=` selector, but applies to `_react` and `_vue`
selectors. In the future, will also apply to `role=` selector.
2022-03-22 17:00:56 -07:00
Ross Wollman
541fb39a51
feat(html-reporter): add report context header (#12734)
Resolves #11318.

* Adds `TestConfig.attachments` public API. (We opted to not implement an analog to the async `TestInfo.attach(…)` API.)
* Adds `TestConfig.attachments` to common reporters.
* Dogfoods some git and CI-info inference to generate useful atttachments
* Updates HTML Reporter to include a side bar to present a pre-defined set of attachments (a.k.a git/commit context sidebar)

Here's what it looks like:

<img width="1738" alt="Screen Shot 2022-03-21 at 3 23 28 PM" src="https://user-images.githubusercontent.com/11915034/159373291-8b937d30-fba3-472a-853a-766018f6b3e2.png">

See `tests/playwright-test/reporter-html.spec.ts` for an example of usage (for dogfood-ing only). In the future, if this becomes user-facing, there the Global Setup bit would likely become unnecessary (as would interaction with attachments array); there would likely just be a nice top-level config and/or CLI flag to enable collecting of info.
2022-03-22 16:28:04 -07:00
Brian Rhoten
fb83d4b42c
fix(codegen): do not consider empty attributes for selector generation (#12880)
Co-authored-by: Brian Rhoten <brhoten@appriss.com>
2022-03-22 13:56:33 -07:00
Brandon Walderman
7db40071a8
fix(selenium): Use ms:edgeOptions with Edge. (#12933) 2022-03-22 12:46:00 -07:00
Max Schmitt
2a7370bad2
fix: upgrade json5 dependency (#12946)
Fixes #12940
2022-03-22 11:33:23 -07:00
Max Schmitt
d8ab76bf64
chore: throw when a Promise was passed to toMatchSnapshot (#12906) 2022-03-22 16:36:09 +01:00
Dmitry Gozman
4b877213a1
fix(tracing): preserve control values without modifying DOM (#12939)
Previously, we preserved input/textarea values by providing
`value` attribute or text child. This produces DOM that does not
actually match the original page.

This change starts using special attributes to modify values
directly when rendering.

Same treatment is also applied to options in `select` and
`checked` property of checkboxes and radio buttons.
2022-03-21 18:51:48 -07:00
Dmitry Gozman
dea6528c0c
feat(aria): introduce role utils (#12916)
This includes aria role and accessible name computation.
Accessible name is covered by wpt tests.
2022-03-21 17:26:45 -07:00
Andrey Lushnikov
7f51336068
fix: allow toMatchSnapshot to use text comparator for text data (#12934)
This was regressed awhile ago.

In v1.17 we shipped the following code: 30e15ad36f/packages/playwright-test/src/matchers/golden.ts (L122-L131)

`toMatchSnapshot` should fallback to text comparator in case of
unknown extension and string data.

Fixes #12862
2022-03-21 16:42:21 -07:00
github-actions[bot]
478a75401a
feat(webkit): roll to r1619 (#12905) 2022-03-21 15:15:56 -07:00
Andrey Lushnikov
c18077c0de
feat(toHaveScreenshot): align screenshot generation & comparison (#12812)
This patch aligns the strategies that are used to generate new
screnshot expectations and to compare screenshot expectations against
baseline.

With this patch, `toHaveScreenshot` will:
- when generating a new expectation: will wait for 2 consecutive
  screenshots to match and accept the last one as expectation.
- when given an expectation:
  * will compare first screenshot against expectation. If matches,
    resolve successfully
  * if first screenshot doesn't match, then wait for 2 consecutive
    screenshots to match and then compare last screenshot with the
    expectation.

An example of a new detailed call log:

```
  1) a.spec.ts:3:1 › should work ===================================================================

    Error: Screenshot comparison failed:

      20000 pixels (ratio 0.03 of all image pixels) are different

    Call log:
      - expect.toHaveScreenshot with timeout 5000ms
      -   verifying given screenshot expectation
      - fast-path: checking first screenshot to match expectation
      - taking page screenshot
      -   disabled all CSS animations
      -   waiting for fonts to load...
      -   fonts in all frames are loaded
      - fast-path failed: first screenshot did not match expectation - 20000 pixels (ratio 0.03 of all image pixels) are different
      - waiting for 2 consecutive screenshots to match
      - waiting 100ms before taking screenshot
      - taking page screenshot
      -   disabled all CSS animations
      -   waiting for fonts to load...
      -   fonts in all frames are loaded
      - 2 consecutive screenshots matched
      - final screenshot did not match expectation - 20000 pixels (ratio 0.03 of all image pixels) are different
      - 20000 pixels (ratio 0.03 of all image pixels) are different

    Expected: /Users/andreylushnikov/tmp/test-results/a-should-work/should-work-1-expected.png
    Received: /Users/andreylushnikov/tmp/test-results/a-should-work/should-work-1-actual.png
        Diff: /Users/andreylushnikov/tmp/test-results/a-should-work/should-work-1-diff.png

      3 | test('should work', async ({ page }) => {
      4 |   await page.goto('file:///Users/andreylushnikov/prog/playwright/tests/assets/rotate-z.html');
    > 5 |   await expect(page).toHaveScreenshot();
        |                      ^
      6 | });
      7 |
```
2022-03-21 15:10:33 -07:00
github-actions[bot]
67e754f6b5
feat(firefox): roll to r1320 (#12928) 2022-03-21 13:40:41 -07:00
Yury Semikhatsky
0adf5536af
docs(fetch): clarify cookie handling (#12892) 2022-03-21 13:20:17 -07:00
github-actions[bot]
9a60067fc2
feat(chromium): roll to r982481 (#12903)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-19 13:06:55 +01:00
Pavel Feldman
e044bae888
chore: make count() work with _vue selectors (#12899) 2022-03-18 21:22:33 -07:00
Dmitry Gozman
85b01056b6
fix(click): workaround elementsFromPoint issue in Chromium (#12888)
Block elements inside elements are not returned correctly
from `document.elementsFromPoint(x, y)` in some cases.
See https://bugs.chromium.org/p/chromium/issues/detail?id=1307458 for details.
2022-03-18 18:20:48 -07:00
Pavel Feldman
98ed81dc00
chore: do not initialize full sessions for pages used in session restore (#12886) 2022-03-18 18:17:37 -07:00
Andrey Lushnikov
cfe92e1608
feat(test-runner): implement expect.poll (#12815)
This patch implements `expect.poll()` method that polls given
predicate until a given synchronous predicate completes.

Usage:

```js
// wait until page gets 3 frames.
await expect.poll(() => page.frames().length, {
  timeout: 1000,
  message: 'custom error message',
}).toBe(3);
```

Fixes #10235
2022-03-18 16:31:26 -07:00
Dmitry Gozman
009185bb89
fix(test runner): do not print fixture location without a separate timeout (#12894)
Otherwise, user randomly sees some "extend" calls that have no real meaning.
2022-03-18 16:09:41 -07:00
Dmitry Gozman
209bde5000
feat(test runner): descrbe.skip (#12865)
`describe.skip` declares a test group that is skipped.
2022-03-18 16:07:11 -07:00
Max Schmitt
dd32956ccc
chore: improve handling with missing xserver (#12724) 2022-03-18 22:39:11 +01:00
Max Schmitt
d795f5dd64
chore: print error if install-deps is used != ubuntu (#12877) 2022-03-18 19:13:11 +01:00
Yury Semikhatsky
a8d80621b2
feat(chromium): large file uploads (#12860) 2022-03-18 09:00:52 -07:00
Dmitry Gozman
3a009531b1
feat(hooks): add a step per hook (#12867)
It is now possible to see which hooks were run in the html report.
2022-03-17 19:33:01 -07:00
Pavel Feldman
c7d6f96328
chore: disambiguate internal flags (#12866) 2022-03-17 18:27:33 -07:00
Dmitry Gozman
25483452c0
feat(fixtures): per-fixture timeout (#12751)
By default, fixtures share timeout with the test they are instantiated for.
However, for more heavy fixtures, especially worker-scoped ones, it makes
sense to have a separate timeout.

This introduces `{ timeout: number }` option to the list of fixture options
that opts the fixture into a dedicated timeout rather than sharing it
with the test.
2022-03-17 09:36:03 -07:00
Benny Neugebauer
2e434f725b
chore: fix markdown links in NPM README's (#12833) 2022-03-17 15:28:57 +01:00
Henrik Storck
05387b7835
fix: fix folder traversal for isModule check (#12836) (#12837) 2022-03-16 15:52:39 -08:00
Max Schmitt
e231db1810
chore: restore expect.any()/expect.anything() (#12820) 2022-03-16 21:34:41 +01:00
Yury Semikhatsky
1284ab8101
feat(webkit): bump to 1617, freeze macOS 10.15 (#12790) 2022-03-15 16:09:56 -07:00
Dmitry Gozman
f8c4cb3d24
fix(screenshot): do not stall on hideHighlight (#12764) 2022-03-15 14:13:45 -07:00
Pavel Feldman
6b324d3780
chore(vue): allow adding events in jsx templates (#12746) 2022-03-15 13:02:10 -07:00
github-actions[bot]
08c29b305b
feat(chromium): roll to r980398 (#12757)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-14 18:51:46 -07:00
Andrey Lushnikov
88610c8b4c
fix: properly define apiName for web-first assertions (#12706)
Turns out relying on PWTRAP in stack is not reliable: depending on the
call structure, the stack might be cut unpredictably by Node.js.

This patch removes PWTRAP and instead plumbs explicit stack and
pre-set `apiName` all the way down to `wrapApiCall`.
2022-03-14 18:01:13 -07:00
Andrey Lushnikov
e3bd7ce119
fix(test-runner): resolve outputDir wrt rootDir (#12755)
Currently, running `npx playwright test` in a subfolder of your project
will result in a `test-results` folder created in your `cwd`.

This is unexpected; instead, we should always resolve all paths
against `rootDir` - directory that contains config.
2022-03-14 17:20:17 -07:00
Yury Semikhatsky
990224bde4
Revert "feat(webkit): bump to 1617 (#12738)" (#12760)
This reverts commit 77824ee19e.
2022-03-14 17:08:32 -07:00
Yury Semikhatsky
77824ee19e
feat(webkit): bump to 1617 (#12738) 2022-03-14 15:11:21 -07:00
Max Schmitt
01beb6310a
docs: mention locator.nth is zero based (#12750)
Co-authored-by: Pavel Feldman <pavel.feldman@gmail.com>
2022-03-14 21:06:44 +01:00
Dmitry Gozman
237954212c
Revert "feat(line reporter): show stats, handle tty" (#12735)
This reverts commit be817d1a53, PR #12695.
Reason: found issues with stdout messing with stats.
2022-03-14 10:37:43 -07:00
Pavel Feldman
3b0a5b4753
chore: mount should return locator pointing to the component element / fragment (#12718) 2022-03-13 21:26:13 -07:00
Pavel Feldman
25c0369eaf
chore: prepare to publish component packages (#12697) 2022-03-12 13:51:27 -08:00
Yury Semikhatsky
19de9cacc3
docs: delete verification guide (#12698) 2022-03-12 09:32:54 -08:00
Max Schmitt
adcd32fc6e
fix(expect): toHaveCSS with custom CSS properties (#12709) 2022-03-12 09:32:40 -08:00
github-actions[bot]
ffc8031bad
feat(chromium): roll to r980066 (#12705)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-12 09:07:57 +01:00
Andrey Lushnikov
ef4b34fb7a
feat(toHaveScreenshot): enhance messaging in case of failures (#12677)
This patch:
- adds call logs to track screenshot timeouts, e.g. due to
  waiting for web fonts
- makes sure all snapshot expectations have `.png` extension
- throws a polite error when given a buffer or a string instead of a
  page or a locator
- removes stray NL between error description and call log
- makes sure `apiName` is always correct (and adds a test for it)
2022-03-11 22:40:28 -08:00
Dmitry Gozman
be817d1a53
feat(line reporter): show stats, handle tty (#12695)
- Line reporter now shows stats in addition to the test name:
  ```log
  [chromium] › page/page-click-react.spec.ts:108:1 › should not retarget when element changes on hover
  23% [21/93] Passed: 17 Flaky: 0 Failed: 0 Skipped: 4 (7s)
  ```

- When connected to a TTY or with `env.PLAYWRIGHT_LIVE_TERMINAL`
set to anything but `'0'` or `'false'`, line reporter updates in place.

- When not connected to a TTY, line reporter prints an update
after each ~1% of tests done, so it never prints more than 100 lines.

- Updated tests to the golden style.
2022-03-11 18:19:33 -08:00
Pavel Feldman
e9fa098bf3
chore: add ct examples (#12693) 2022-03-11 15:46:11 -08:00
Andrey Lushnikov
95c774cf84
feat: support multiple screenshots in HTML report (#12577)
This patch adds support to multiple diffs. These are possible
due to soft assertions.

Drive-by: rename second screenshot in `toHaveScreenshot` failure when
re-generating expectations from "expected" to "previous".
2022-03-11 08:46:13 -08:00
Andrey Lushnikov
e3ab6388a9
feat(toHaveScreenshot): enable sensible defaults (#12675)
This patch:
- Enables configuration of certain defaults for some options of `expect.toHaveScreenshot` method via `TestProject.expect.toHaveScreenshot` property
- Sets sensible defaults for these options:
  * `fonts: "ready"`
  * `animations: "disabled"`
  * `size: "css"`
2022-03-11 08:45:36 -08:00
Pavel Feldman
af55f48754
chore: add experimental mount (#12657) 2022-03-11 08:00:46 -08:00
Ross Wollman
ebae5e76df
feat(test-runner): more default workers for M1 (#12660)
New default is based on the very (unscientific) output of repeated a few
times:

```
$ for i in `seq 5 13`; do time npm run ctest -- --reporter=line --workers=$i 'page/*' 'selector*' 'channels*' 'resource-timing*'; done
```
2022-03-11 02:12:07 -08:00
Andrey Lushnikov
b51157bdea
feat: bring back maxDiffPixels in toMatchSnapshot (#12627)
This patch reverts 2 commits that removed the feature from the method:
- "fix: explicitly ignore maxDiffPixels in toMatchSnapshot (#12570)"
  commit b8af8458d6.
- "chore: remove `maxDiffPixels` from toMatchSnapshot (#12539)"
  commit a3dff45974.
2022-03-10 18:41:16 -08:00
Andrey Lushnikov
e83549e8a0
fix: support all commong screenshot options in toHaveScreenshot (#12670)
This patch adds support for all common options in
`expect.toHaveScreenshot`.
2022-03-10 18:36:20 -08:00
Andrey Lushnikov
65b9cba143
chore: follow-up to animations default option (#12669)
This is a follow-up to 42765804bc
that was landed without bots.

I ran the bots manually on Linux & Mac.
2022-03-10 17:26:50 -08:00
Andrey Lushnikov
689d8196e0
chore: follow-up to fix docs (#12668)
This is a follow-up to 49e66c7f08 that was landed without bots.

I ran bots manually on Mac & Linux.
2022-03-10 17:19:44 -08:00
Andrey Lushnikov
42765804bc
chore: add "allow" as a default value for animations. (#12663) 2022-03-10 17:15:36 -08:00
Andrey Lushnikov
49e66c7f08
feat(screenshot): introduce new "fonts" option for screenshots (#12661)
This option will wait for webfonts to load before taking screenshots.
2022-03-10 16:54:36 -08:00
Andrey Lushnikov
12d8a262be
feat: support screenshotsDir option (#12642)
The `screenshotsDir` option controls the expectation storage
for `toHaveScreenshot()` function.

The new expectation management for screenshots has the following
key properties:
- All screenshots are stored in a single folder called `screenshotsDir`.
- Screenshot names **do not** respect `snapshotDir` and `snapshotSuffix`
  configurations.
- `screenshotsDir` is configurable per project. This way a "smoke tests"
  project can re-use screenshots from "all tests" project.
- Host platform is a top-level folder.

For example, given the following config:

```js
// playwright.config.ts
module.exports = {
  projects: [
    { name: 'Mobile Safari' },
    { name: 'Desktop Chrome' },
  ],
};
```

And the following test structure:

```
smoke-tests/
└── basic.spec.ts
```

Will result in the following screenshots folder structure by default:

```
__screenshots__/
└── darwin/
    ├── Mobile Safari/
    │   └── smoke-tests/
    │       └── basic.spec.ts/
    │           └── screenshot-expectation.png
    └── Desktop Chrome/
        └── smoke-tests/
            └── basic.spec.ts/
                └── screenshot-expectation.png
```
2022-03-10 16:50:26 -08:00
Pavel Feldman
4d3d0e2bda
chore: enable ts-esm support by default (#12658) 2022-03-10 14:48:33 -08:00
Yury Semikhatsky
cf643ff04c
docs: remove unsupported permission "push" from docs (#12654) 2022-03-10 14:00:25 -08:00
Dmitry Gozman
a388bb2302
feat(screenshot): size:'css'|'device' option (#12634)
With this experimental option, screenshot dimensions are in CSS pixels,
not physical device pixels, effectively ignoring the device scale factor.
2022-03-10 13:07:10 -08:00
Max Schmitt
3f468abedd
fix(test-runner): ESM compatibly on Windows (#12615) 2022-03-10 19:27:25 +01:00
Andrey Lushnikov
75eef09c0d
feat: show expectation name as part of toHaveScreenshot title (#12612)
This patch adds snapshot file name as part of `toHaveScreenshot`
and `toMatchSnapshot` step title.
2022-03-09 20:09:45 -08:00
Yury Semikhatsky
d49843edaa
fix(firefox): add missing dependency on Ubuntu 18.04 (#12625) 2022-03-09 11:21:15 -08:00
Dmitry Gozman
92430cc56c
chore(fixtures): do not force-close contexts in beforeAll (#12416)
This now happens the same way as for the test, in the worker cleanup
when browser closes, which is reported as a part of the test.
2022-03-09 07:26:46 -08:00
Dmitry Gozman
e8ce5d0258
feat(test runner): on beforeAll failure, precisely skip the tests (#12417)
Previously, we used to skip all the tests from the same file when
any `beforeAll` fails in the file.

Now, we only skip the rest of the tests affected by this particular
`beforeAll` and continue with other tests in the new worker.
2022-03-08 20:29:31 -08:00
Dmitry Gozman
4a768294b4
chore: cleanup hooks notion from raw/html reporters (#12610) 2022-03-08 19:08:31 -08:00
Dmitry Gozman
e895bc2751
reland(test-runner): rely on test title paths instead of ordinal (#12611)
Fixes #11904

Co-authored-by: Andrey Lushnikov <lushnikov@chromium.org>
2022-03-08 19:06:25 -08:00