Commit Graph

7214 Commits

Author SHA1 Message Date
Andrey Lushnikov
49e66c7f08
feat(screenshot): introduce new "fonts" option for screenshots (#12661)
This option will wait for webfonts to load before taking screenshots.
2022-03-10 16:54:36 -08:00
Andrey Lushnikov
12d8a262be
feat: support screenshotsDir option (#12642)
The `screenshotsDir` option controls the expectation storage
for `toHaveScreenshot()` function.

The new expectation management for screenshots has the following
key properties:
- All screenshots are stored in a single folder called `screenshotsDir`.
- Screenshot names **do not** respect `snapshotDir` and `snapshotSuffix`
  configurations.
- `screenshotsDir` is configurable per project. This way a "smoke tests"
  project can re-use screenshots from "all tests" project.
- Host platform is a top-level folder.

For example, given the following config:

```js
// playwright.config.ts
module.exports = {
  projects: [
    { name: 'Mobile Safari' },
    { name: 'Desktop Chrome' },
  ],
};
```

And the following test structure:

```
smoke-tests/
└── basic.spec.ts
```

Will result in the following screenshots folder structure by default:

```
__screenshots__/
└── darwin/
    ├── Mobile Safari/
    │   └── smoke-tests/
    │       └── basic.spec.ts/
    │           └── screenshot-expectation.png
    └── Desktop Chrome/
        └── smoke-tests/
            └── basic.spec.ts/
                └── screenshot-expectation.png
```
2022-03-10 16:50:26 -08:00
Yury Semikhatsky
10bf5f3e49
docs: update links to assertions docs (#12662) 2022-03-10 15:45:18 -08:00
Pavel Feldman
4d3d0e2bda
chore: enable ts-esm support by default (#12658) 2022-03-10 14:48:33 -08:00
Yury Semikhatsky
cf643ff04c
docs: remove unsupported permission "push" from docs (#12654) 2022-03-10 14:00:25 -08:00
Max Schmitt
324e3ff3ae
docs: add .NET as a new Docker image (#12659) 2022-03-10 22:20:04 +01:00
Dmitry Gozman
a388bb2302
feat(screenshot): size:'css'|'device' option (#12634)
With this experimental option, screenshot dimensions are in CSS pixels,
not physical device pixels, effectively ignoring the device scale factor.
2022-03-10 13:07:10 -08:00
Max Schmitt
372a3219f3
chore: remove Python from main Docker image (#12431) 2022-03-10 19:51:35 +01:00
Max Schmitt
90e76f9adb
test: rename #smoke to @smoke as test tags (#12652) 2022-03-10 19:42:52 +01:00
Max Schmitt
3f468abedd
fix(test-runner): ESM compatibly on Windows (#12615) 2022-03-10 19:27:25 +01:00
Max Schmitt
d805cd9097
chore: skip failing android tests 2 (#12647) 2022-03-10 18:46:50 +01:00
Max Schmitt
49fd77d9af
docs: reference to external allure reporter (#12646) 2022-03-10 09:01:42 -08:00
Yury Semikhatsky
e270adf437
docs: fix broken link (#12638) 2022-03-09 22:00:16 -08:00
Andrey Lushnikov
75eef09c0d
feat: show expectation name as part of toHaveScreenshot title (#12612)
This patch adds snapshot file name as part of `toHaveScreenshot`
and `toMatchSnapshot` step title.
2022-03-09 20:09:45 -08:00
Yury Semikhatsky
5f1188d195
docs: fix js example formatting (#12635) 2022-03-09 17:10:37 -08:00
Yury Semikhatsky
d49843edaa
fix(firefox): add missing dependency on Ubuntu 18.04 (#12625) 2022-03-09 11:21:15 -08:00
Dmitry Gozman
92430cc56c
chore(fixtures): do not force-close contexts in beforeAll (#12416)
This now happens the same way as for the test, in the worker cleanup
when browser closes, which is reported as a part of the test.
2022-03-09 07:26:46 -08:00
Dmitry Gozman
e8ce5d0258
feat(test runner): on beforeAll failure, precisely skip the tests (#12417)
Previously, we used to skip all the tests from the same file when
any `beforeAll` fails in the file.

Now, we only skip the rest of the tests affected by this particular
`beforeAll` and continue with other tests in the new worker.
2022-03-08 20:29:31 -08:00
Dmitry Gozman
4a768294b4
chore: cleanup hooks notion from raw/html reporters (#12610) 2022-03-08 19:08:31 -08:00
Dmitry Gozman
e895bc2751
reland(test-runner): rely on test title paths instead of ordinal (#12611)
Fixes #11904

Co-authored-by: Andrey Lushnikov <lushnikov@chromium.org>
2022-03-08 19:06:25 -08:00
Dmitry Gozman
682eeb4ae0
reland(hooks): separate test timeout from beforeAll/afterAll timeouts (#12609)
This makes it possible to have longer `beforeAll`/`afterAll` and not
affect first/last test timeout.
2022-03-08 19:05:23 -08:00
Yury Semikhatsky
176846672f
docs: clarify multiple before/after hooks execution order (#12606) 2022-03-08 17:42:50 -08:00
Dmitry Gozman
62a032baa0
chore: remove env.PLAYWRIGHT_NO_LAYOUT_SHIFT_CHECK (#12605) 2022-03-08 16:42:39 -08:00
Dmitry Gozman
9cd704220c
reland(test runner): run hooks/modifiers as a part of the test (#12518)
Reland: worker.stop() before worker.run() was hanging because
`_runFinished` promise was not initially resolved.
---
This moves `beforeAll`, `afterAll` and some modifiers from running
as a separate entity into running inside a test.

Pros:
- All errors are reported as test errors.
- All artifacts are collected as test artifacts.
- Reporters support this out of the box.

Details:
- Each test computes the necessary hooks to run and runs them.
- Teardown is usually performed during the test (on test failure or worker stop).
- `skipRemaining` is added to `DonePayload` to preserve the behavior
  where `beforeAll` hook failure skips subsequent tests.
  This behavior can now be improved to only target tests affected by this hook.
2022-03-08 16:35:14 -08:00
Yury Semikhatsky
79c8746fd2
docs: add name for screenshot animations option (#12601) 2022-03-08 16:02:31 -08:00
Yury Semikhatsky
03150265db
docs: update POM guide (#12598) 2022-03-08 14:01:50 -08:00
Andrey Lushnikov
4bccf2f6aa
chore: cut v1.20 branch (#12593) 2022-03-08 14:01:20 -08:00
Meir Blachman
7e53facf1e
docs: Update pom.md to use locators (#12592) 2022-03-08 21:39:40 +01:00
Max Schmitt
30c954a3ba
docs: add Docker image for Python (#12595) 2022-03-08 21:21:46 +01:00
Andrey Lushnikov
422df4cbbe
docs: add js release notes for 1.20 (#12575) 2022-03-08 12:15:55 -08:00
github-actions[bot]
04e03728b8
feat(chromium): roll to r978106 (#12585)
Co-authored-by: Max Schmitt <max@schmitt.mx>
2022-03-08 21:05:56 +01:00
Andrey Lushnikov
497b51ad8c
fix(toHaveScreenshot): fix initial screenshot time (#12576)
Instead of being taken immediately, first screenshot was actually
taken after 1 second.
2022-03-08 09:30:14 -08:00
github-actions[bot]
18e6aa3f15
feat(firefox): roll to r1319 (#12587)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-08 09:19:00 -08:00
Andrey Lushnikov
b8af8458d6
fix: explicitly ignore maxDiffPixels in toMatchSnapshot (#12570)
Fixes #12564
2022-03-07 16:55:35 -08:00
Andrey Lushnikov
e8ad7eb3f2
browser(firefox): roll Firefox to M97 (#12574)
Firefox 98 doesn't work on Ubuntu aarch64 yet.

https://github.com/microsoft/playwright/issues/12557
2022-03-07 16:16:42 -08:00
Andrey Lushnikov
b0a1b412c9
devops: use xcode 13.2 when building WebKit @ MacOS 11 (#12573) 2022-03-07 15:45:35 -08:00
Yury Semikhatsky
e3e29e25be
docs: update links to assertions class (#12562) 2022-03-07 10:37:36 -08:00
Dmitry Gozman
d836ed41d3
feat(remote): send 'x-playwright-browser' header (#12556) 2022-03-07 10:30:53 -08:00
Max Schmitt
5ca7858ace
docs: use npm init @latest (#12561) 2022-03-07 19:30:03 +01:00
Siddharth Singha Roy
a2b3d4f570
feat(cli) : allow passing PlaywrightServer.startDefault parameters via cli (#12530) 2022-03-07 09:15:46 -08:00
github-actions[bot]
d81330f28f
browser(chromium): roll to r978106 (#12550)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-07 11:42:14 +01:00
Diego Pino
cd0074fa41
browser(webkit): rebase to 03/04/22 (r290853) (#12551) 2022-03-07 11:39:09 +01:00
github-actions[bot]
702536d962
feat(webkit): roll to r1616 (#12541)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Yury Semikhatsky <yurys@chromium.org>
2022-03-06 12:22:47 +01:00
Andrey Lushnikov
d7648d390c
fix(toHaveScreenshot): round ratio up (#12540)
This way the ratio value could be used right away in configuation.
2022-03-04 18:55:48 -08:00
Andrey Lushnikov
a3dff45974
chore: remove maxDiffPixels from toMatchSnapshot (#12539)
References #12441
2022-03-04 18:27:04 -08:00
Andrey Lushnikov
6dac01aec7
fix: implement log scale back-off for screenshots (#12504)
References https://github.com/microsoft/playwright/issues/12441
2022-03-04 18:17:57 -08:00
Andrey Lushnikov
4953fc4845
chore: introduce toHaveScreenshot configuration options (#12507)
References https://github.com/microsoft/playwright/issues/12441
2022-03-04 17:30:43 -08:00
github-actions[bot]
adb2847d84
feat(firefox): roll to r1318 (#12537)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-03-04 17:29:35 -08:00
Ross Wollman
281945c8b6
test: typo fix (#12531) 2022-03-04 17:09:18 -08:00
Max Schmitt
5fd43edcbb
chore: fix android tests after #12220 (#12535) 2022-03-05 00:18:03 +01:00