Commit Graph

1141 Commits

Author SHA1 Message Date
Dmitry Gozman
6b48631eed
feat(role): add more tests for accessible name (#13154) 2022-03-29 11:59:44 -07:00
Pavel Feldman
1961959dcb
chore: migrate injected scripts to esbuild (#13143) 2022-03-28 23:10:17 -07:00
Dmitry Gozman
de0af27837
feat(test runner): show last tests in the worker on teardown error (#13139) 2022-03-28 19:58:24 -07:00
Dmitry Gozman
d59c2b996f
fix(parallel): minimize the number of beforeAll/afterAll hooks in parallel mode (#13138)
Previously, we always formed groups consisting of a single test.
Now, we group tests that share `beforeAll`/`afterAll` hooks into
`config.workers` equally-sized groups.
2022-03-28 16:10:32 -07:00
Ross Wollman
13dedd27ca
test: out-of-viewport should be considered visible (#13134)
Closes #13131.

Per the visibility spec on https://playwright.dev/docs/next/actionability#visible:

> Element is considered visible when it has non-empty bounding box and does not have visibility:hidden computed style. Note that elements of zero size or with display:none are not considered visible.

 non-empty bounding box
 does not have visibility:hidden

Given the above conditions are satisfied, the locator is considered visible.

https://github.com/microsoft/playwright/issues/8740 proposes something like `isInViewport()` that would be better suited for checking if an element is offscreen.
2022-03-28 16:10:16 -07:00
Dmitry Gozman
01a39e5b4c
test(role): add tests for implicit role calculation (#13132) 2022-03-28 15:22:50 -07:00
Philip Sanetra
be41c4a35d
fix(registry): Fix support for generic-linux (#13129)
Fixes https://github.com/microsoft/playwright/issues/13128
2022-03-28 14:50:56 -07:00
Dmitry Gozman
9420a53939
fix(esm): make sure import from './foo.js' is supported (#13137)
Drive-by: migrate all @esm tests to esm.spec.ts.
2022-03-28 14:37:46 -07:00
Dmitry Gozman
8c19f71c36
feat(selectors): role selector engine (#12999)
This introduces `role=button[name="Click me"][pressed]` attribute-style
role selector. It is only available under `env.PLAYWRIGHT_EXPERIMENTAL_FEATURES`.

Supported attributes:
- `role` is required, for example `role=button`;
- `name` is accessible name, supports matching operators and regular expressions:
  `role=button[name=/Click(me)?/]`;
- `checked` boolean/mixed, for example `role=checkbox[checked=false]`;
- `selected` boolean, for example `role=option[selected]`;
- `expanded` boolean, for example `role=button[expanded=true]`;
- `disabled` boolean, for example `role=button[disabled]`;
- `level` number, for example `role=heading[level=3]`;
- `pressed` boolean/mixed, for example `role=button[pressed="mixed"]`;
- `includeHidden` - by default, only non-hidden elements are considered.
   Passing `role=button[includeHidden]` matches hidden elements as well.
2022-03-28 09:24:58 -07:00
Pavel Feldman
1471bb7177
fix(tests): remove lock from component tests, test real experience (#13095) 2022-03-26 13:54:33 -07:00
Pavel Feldman
a4a2d41705 fix(tests): fix selenium installation tests 2022-03-26 08:36:25 -07:00
Pavel Feldman
a58707b8a6
chore: group tests under tests/ (2) (#13082) 2022-03-25 23:09:02 -07:00
Pavel Feldman
a454328c25
fix(test): follow up to tests refactoring (#13085) 2022-03-25 22:11:29 -07:00
Pavel Feldman
02cac8a066
chore: group tests under tests/ (1) (#13081) 2022-03-25 16:05:50 -07:00
Yury Semikhatsky
5734c18ef8
feat(route): add cors header in route.fulfill (#12943) 2022-03-25 14:56:57 -07:00
Pavel Feldman
6b81e76c2b
chore: migrate trace-viewer and recorder to vite (#13076) 2022-03-25 14:12:00 -07:00
Yury Semikhatsky
97e8ead57c
feat: large files upload in ff and wk (#12937) 2022-03-25 13:26:12 -07:00
Ross Wollman
ddc2cd4a3e
test: mark flaky proxy tests (#13050) 2022-03-24 14:50:59 -07:00
Max Schmitt
1797c5c249
chore(test-runner): support self signed certificate in webServer (#13032) 2022-03-24 17:30:52 +01:00
Dmitry Gozman
91408f2c5e
fix(storageState): provide nice error message for storage state issues (#13019) 2022-03-24 07:33:51 -07:00
Dmitry Gozman
3688e74e3e
fix(list mode): print errors to stderr (#13016)
`--list` mode now prints any errors encountered during test collection,
for example syntax errors, to `stderr`.
2022-03-24 07:33:33 -07:00
Andrey Lushnikov
e1700bd167
feat: use package.json directory as a default for outputDir (#12942)
This patch:
- starts using directory of `package.json` to resolve default
  output directory path
- starts using either `package.json` directory or configuration
  directory to resolve all relative paths

References #12970
2022-03-23 16:05:49 -07:00
Ross Wollman
39376cccd8
fix: allow unnamed cookies (#12991)
Fixes #12808.
2022-03-23 15:50:43 -07:00
Xiaoxing Ye
04e9d2ec08
test(test-runner): fix reporter-list test cannot be run on win32 vscode (#12951) 2022-03-23 11:48:05 +01:00
Dmitry Gozman
722302799e
feat(selectors): support regular expressions in attribute selectors (#12960)
Supports inline regex in addition to string: `_react=BookItem[author = /Ann?a/i]`.
This is similar to `text=` selector, but applies to `_react` and `_vue`
selectors. In the future, will also apply to `role=` selector.
2022-03-22 17:00:56 -07:00
Ross Wollman
541fb39a51
feat(html-reporter): add report context header (#12734)
Resolves #11318.

* Adds `TestConfig.attachments` public API. (We opted to not implement an analog to the async `TestInfo.attach(…)` API.)
* Adds `TestConfig.attachments` to common reporters.
* Dogfoods some git and CI-info inference to generate useful atttachments
* Updates HTML Reporter to include a side bar to present a pre-defined set of attachments (a.k.a git/commit context sidebar)

Here's what it looks like:

<img width="1738" alt="Screen Shot 2022-03-21 at 3 23 28 PM" src="https://user-images.githubusercontent.com/11915034/159373291-8b937d30-fba3-472a-853a-766018f6b3e2.png">

See `tests/playwright-test/reporter-html.spec.ts` for an example of usage (for dogfood-ing only). In the future, if this becomes user-facing, there the Global Setup bit would likely become unnecessary (as would interaction with attachments array); there would likely just be a nice top-level config and/or CLI flag to enable collecting of info.
2022-03-22 16:28:04 -07:00
Brian Rhoten
fb83d4b42c
fix(codegen): do not consider empty attributes for selector generation (#12880)
Co-authored-by: Brian Rhoten <brhoten@appriss.com>
2022-03-22 13:56:33 -07:00
Max Schmitt
d8ab76bf64
chore: throw when a Promise was passed to toMatchSnapshot (#12906) 2022-03-22 16:36:09 +01:00
Dmitry Gozman
4b877213a1
fix(tracing): preserve control values without modifying DOM (#12939)
Previously, we preserved input/textarea values by providing
`value` attribute or text child. This produces DOM that does not
actually match the original page.

This change starts using special attributes to modify values
directly when rendering.

Same treatment is also applied to options in `select` and
`checked` property of checkboxes and radio buttons.
2022-03-21 18:51:48 -07:00
Dmitry Gozman
dea6528c0c
feat(aria): introduce role utils (#12916)
This includes aria role and accessible name computation.
Accessible name is covered by wpt tests.
2022-03-21 17:26:45 -07:00
Andrey Lushnikov
7f51336068
fix: allow toMatchSnapshot to use text comparator for text data (#12934)
This was regressed awhile ago.

In v1.17 we shipped the following code: 30e15ad36f/packages/playwright-test/src/matchers/golden.ts (L122-L131)

`toMatchSnapshot` should fallback to text comparator in case of
unknown extension and string data.

Fixes #12862
2022-03-21 16:42:21 -07:00
Andrey Lushnikov
c18077c0de
feat(toHaveScreenshot): align screenshot generation & comparison (#12812)
This patch aligns the strategies that are used to generate new
screnshot expectations and to compare screenshot expectations against
baseline.

With this patch, `toHaveScreenshot` will:
- when generating a new expectation: will wait for 2 consecutive
  screenshots to match and accept the last one as expectation.
- when given an expectation:
  * will compare first screenshot against expectation. If matches,
    resolve successfully
  * if first screenshot doesn't match, then wait for 2 consecutive
    screenshots to match and then compare last screenshot with the
    expectation.

An example of a new detailed call log:

```
  1) a.spec.ts:3:1 › should work ===================================================================

    Error: Screenshot comparison failed:

      20000 pixels (ratio 0.03 of all image pixels) are different

    Call log:
      - expect.toHaveScreenshot with timeout 5000ms
      -   verifying given screenshot expectation
      - fast-path: checking first screenshot to match expectation
      - taking page screenshot
      -   disabled all CSS animations
      -   waiting for fonts to load...
      -   fonts in all frames are loaded
      - fast-path failed: first screenshot did not match expectation - 20000 pixels (ratio 0.03 of all image pixels) are different
      - waiting for 2 consecutive screenshots to match
      - waiting 100ms before taking screenshot
      - taking page screenshot
      -   disabled all CSS animations
      -   waiting for fonts to load...
      -   fonts in all frames are loaded
      - 2 consecutive screenshots matched
      - final screenshot did not match expectation - 20000 pixels (ratio 0.03 of all image pixels) are different
      - 20000 pixels (ratio 0.03 of all image pixels) are different

    Expected: /Users/andreylushnikov/tmp/test-results/a-should-work/should-work-1-expected.png
    Received: /Users/andreylushnikov/tmp/test-results/a-should-work/should-work-1-actual.png
        Diff: /Users/andreylushnikov/tmp/test-results/a-should-work/should-work-1-diff.png

      3 | test('should work', async ({ page }) => {
      4 |   await page.goto('file:///Users/andreylushnikov/prog/playwright/tests/assets/rotate-z.html');
    > 5 |   await expect(page).toHaveScreenshot();
        |                      ^
      6 | });
      7 |
```
2022-03-21 15:10:33 -07:00
Pavel Feldman
e044bae888
chore: make count() work with _vue selectors (#12899) 2022-03-18 21:22:33 -07:00
Dmitry Gozman
85b01056b6
fix(click): workaround elementsFromPoint issue in Chromium (#12888)
Block elements inside elements are not returned correctly
from `document.elementsFromPoint(x, y)` in some cases.
See https://bugs.chromium.org/p/chromium/issues/detail?id=1307458 for details.
2022-03-18 18:20:48 -07:00
Pavel Feldman
98ed81dc00
chore: do not initialize full sessions for pages used in session restore (#12886) 2022-03-18 18:17:37 -07:00
Pavel Feldman
4aaa63beaa
test: add a test for #12837 (#12838) 2022-03-18 18:09:59 -07:00
Andrey Lushnikov
cfe92e1608
feat(test-runner): implement expect.poll (#12815)
This patch implements `expect.poll()` method that polls given
predicate until a given synchronous predicate completes.

Usage:

```js
// wait until page gets 3 frames.
await expect.poll(() => page.frames().length, {
  timeout: 1000,
  message: 'custom error message',
}).toBe(3);
```

Fixes #10235
2022-03-18 16:31:26 -07:00
Dmitry Gozman
009185bb89
fix(test runner): do not print fixture location without a separate timeout (#12894)
Otherwise, user randomly sees some "extend" calls that have no real meaning.
2022-03-18 16:09:41 -07:00
Dmitry Gozman
209bde5000
feat(test runner): descrbe.skip (#12865)
`describe.skip` declares a test group that is skipped.
2022-03-18 16:07:11 -07:00
Max Schmitt
dd32956ccc
chore: improve handling with missing xserver (#12724) 2022-03-18 22:39:11 +01:00
Andrey Lushnikov
bb756c5b23
test: mark webkit ubuntu fonts test as fixme (#12885)
References https://github.com/microsoft/playwright/issues/12839
2022-03-18 09:35:41 -07:00
Yury Semikhatsky
a8d80621b2
feat(chromium): large file uploads (#12860) 2022-03-18 09:00:52 -07:00
Dmitry Gozman
3a009531b1
feat(hooks): add a step per hook (#12867)
It is now possible to see which hooks were run in the html report.
2022-03-17 19:33:01 -07:00
Pavel Feldman
c7d6f96328
chore: disambiguate internal flags (#12866) 2022-03-17 18:27:33 -07:00
Dmitry Gozman
25483452c0
feat(fixtures): per-fixture timeout (#12751)
By default, fixtures share timeout with the test they are instantiated for.
However, for more heavy fixtures, especially worker-scoped ones, it makes
sense to have a separate timeout.

This introduces `{ timeout: number }` option to the list of fixture options
that opts the fixture into a dedicated timeout rather than sharing it
with the test.
2022-03-17 09:36:03 -07:00
Max Schmitt
e231db1810
chore: restore expect.any()/expect.anything() (#12820) 2022-03-16 21:34:41 +01:00
Dmitry Gozman
f8c4cb3d24
fix(screenshot): do not stall on hideHighlight (#12764) 2022-03-15 14:13:45 -07:00
Andrey Lushnikov
88610c8b4c
fix: properly define apiName for web-first assertions (#12706)
Turns out relying on PWTRAP in stack is not reliable: depending on the
call structure, the stack might be cut unpredictably by Node.js.

This patch removes PWTRAP and instead plumbs explicit stack and
pre-set `apiName` all the way down to `wrapApiCall`.
2022-03-14 18:01:13 -07:00
Dmitry Gozman
237954212c
Revert "feat(line reporter): show stats, handle tty" (#12735)
This reverts commit be817d1a53, PR #12695.
Reason: found issues with stdout messing with stats.
2022-03-14 10:37:43 -07:00
Max Schmitt
adcd32fc6e
fix(expect): toHaveCSS with custom CSS properties (#12709) 2022-03-12 09:32:40 -08:00