Commit Graph

842 Commits

Author SHA1 Message Date
Andrey Lushnikov
a9989852d5
feat(vrt): new option "caret" for taking screenshots (#13164)
This has two values:
- `"hide"` to hide input caret for taking screenshot
- `"initial"` to keep caret behavior unchanged

Defaults to `"hide"`.

Fixes #12643
2022-03-29 17:48:13 -07:00
Dmitry Gozman
eb09306db2
feat(connect): add connectOptions.timeout (#13163) 2022-03-29 15:03:43 -07:00
Pavel Feldman
1961959dcb
chore: migrate injected scripts to esbuild (#13143) 2022-03-28 23:10:17 -07:00
Pavel Feldman
8758cf8cbf
chore: migrate html reporter to vite (#13116) 2022-03-28 18:21:19 -07:00
Ross Wollman
ae365dda3b
chore: remove internal dep on FullConfig (#13074)
Resolves #13058.

This blocks adding #8206 since we don't want the new launch configs
publicly available on FullConfig.
2022-03-28 15:53:42 -07:00
Pavel Feldman
a454328c25
fix(test): follow up to tests refactoring (#13085) 2022-03-25 22:11:29 -07:00
Andrey Lushnikov
4ab4c0bda1
feat: detect docker version and Playwright version mismatch (#12806)
This patch prints a friendly instructions in case Docker image version
mismatches Playwright version and there are missing browser
dependencies.

With this patch, Playwright will yield the following error:

```
root@f0774d2b2097:~# node a.mjs
node:internal/process/promises:279
            triggerUncaughtException(err, true /* fromPromise */);
            ^

browserType.launch:
╔════════════════════════════════════════════════════════════════════════════════════════════╗
║ Host system is missing dependencies to run browsers.                                       ║
║ This is most likely due to docker image version not matching Playwright version:           ║
║ - Playwright: 1.22.0                                                                       ║
║ -     Docker: 1.21.0                                                                       ║
║                                                                                            ║
║ Either:                                                                                    ║
║ - (recommended) use docker image "mcr.microsoft.com/playwright:v1.22.0-focal"              ║
║ - (alternative 1) run the following command inside docker to install missing dependencies: ║
║                                                                                            ║
║     npx playwright install-deps                                                            ║
║                                                                                            ║
║ - (alternative 2) use Aptitude inside docker:                                              ║
║                                                                                            ║
║     apt-get install libgbm1                                                                ║
║                                                                                            ║
║ <3 Playwright Team                                                                         ║
╚════════════════════════════════════════════════════════════════════════════════════════════╝
    at file:///root/a.mjs:3:10 {
  name: 'Error'
}```

Fixes #12796

Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
2022-03-25 14:45:53 -07:00
Pavel Feldman
6b81e76c2b
chore: migrate trace-viewer and recorder to vite (#13076) 2022-03-25 14:12:00 -07:00
Max Schmitt
c8cc62a2e6
chore: add link validation for docs (#13047) 2022-03-25 19:30:45 +01:00
Pavel Feldman
07c1a27732
chore(deps): prepare deps for monorepo (#13057) 2022-03-25 08:43:29 -07:00
Pavel Feldman
71dd960fec
fix(watch): throttle / batch watch processing (#13054) 2022-03-24 17:38:03 -07:00
Max Schmitt
1797c5c249
chore(test-runner): support self signed certificate in webServer (#13032) 2022-03-24 17:30:52 +01:00
Ross Wollman
541fb39a51
feat(html-reporter): add report context header (#12734)
Resolves #11318.

* Adds `TestConfig.attachments` public API. (We opted to not implement an analog to the async `TestInfo.attach(…)` API.)
* Adds `TestConfig.attachments` to common reporters.
* Dogfoods some git and CI-info inference to generate useful atttachments
* Updates HTML Reporter to include a side bar to present a pre-defined set of attachments (a.k.a git/commit context sidebar)

Here's what it looks like:

<img width="1738" alt="Screen Shot 2022-03-21 at 3 23 28 PM" src="https://user-images.githubusercontent.com/11915034/159373291-8b937d30-fba3-472a-853a-766018f6b3e2.png">

See `tests/playwright-test/reporter-html.spec.ts` for an example of usage (for dogfood-ing only). In the future, if this becomes user-facing, there the Global Setup bit would likely become unnecessary (as would interaction with attachments array); there would likely just be a nice top-level config and/or CLI flag to enable collecting of info.
2022-03-22 16:28:04 -07:00
Dmitry Gozman
209bde5000
feat(test runner): descrbe.skip (#12865)
`describe.skip` declares a test group that is skipped.
2022-03-18 16:07:11 -07:00
Yury Semikhatsky
a8d80621b2
feat(chromium): large file uploads (#12860) 2022-03-18 09:00:52 -07:00
Max Schmitt
f3ffd328dd
chore: fix modification dates in driver (#12864) 2022-03-18 13:03:09 +01:00
Dmitry Gozman
25483452c0
feat(fixtures): per-fixture timeout (#12751)
By default, fixtures share timeout with the test they are instantiated for.
However, for more heavy fixtures, especially worker-scoped ones, it makes
sense to have a separate timeout.

This introduces `{ timeout: number }` option to the list of fixture options
that opts the fixture into a dedicated timeout rather than sharing it
with the test.
2022-03-17 09:36:03 -07:00
Andrey Lushnikov
e16f3c46b4
docs: always keep docker references up-to-date in documentation (#12795)
References #12796
2022-03-16 13:51:39 -07:00
Pavel Feldman
25c0369eaf
chore: prepare to publish component packages (#12697) 2022-03-12 13:51:27 -08:00
Pavel Feldman
e9fa098bf3
chore: add ct examples (#12693) 2022-03-11 15:46:11 -08:00
Andrey Lushnikov
e3ab6388a9
feat(toHaveScreenshot): enable sensible defaults (#12675)
This patch:
- Enables configuration of certain defaults for some options of `expect.toHaveScreenshot` method via `TestProject.expect.toHaveScreenshot` property
- Sets sensible defaults for these options:
  * `fonts: "ready"`
  * `animations: "disabled"`
  * `size: "css"`
2022-03-11 08:45:36 -08:00
Pavel Feldman
af55f48754
chore: add experimental mount (#12657) 2022-03-11 08:00:46 -08:00
Andrey Lushnikov
b51157bdea
feat: bring back maxDiffPixels in toMatchSnapshot (#12627)
This patch reverts 2 commits that removed the feature from the method:
- "fix: explicitly ignore maxDiffPixels in toMatchSnapshot (#12570)"
  commit b8af8458d6.
- "chore: remove `maxDiffPixels` from toMatchSnapshot (#12539)"
  commit a3dff45974.
2022-03-10 18:41:16 -08:00
Andrey Lushnikov
12d8a262be
feat: support screenshotsDir option (#12642)
The `screenshotsDir` option controls the expectation storage
for `toHaveScreenshot()` function.

The new expectation management for screenshots has the following
key properties:
- All screenshots are stored in a single folder called `screenshotsDir`.
- Screenshot names **do not** respect `snapshotDir` and `snapshotSuffix`
  configurations.
- `screenshotsDir` is configurable per project. This way a "smoke tests"
  project can re-use screenshots from "all tests" project.
- Host platform is a top-level folder.

For example, given the following config:

```js
// playwright.config.ts
module.exports = {
  projects: [
    { name: 'Mobile Safari' },
    { name: 'Desktop Chrome' },
  ],
};
```

And the following test structure:

```
smoke-tests/
└── basic.spec.ts
```

Will result in the following screenshots folder structure by default:

```
__screenshots__/
└── darwin/
    ├── Mobile Safari/
    │   └── smoke-tests/
    │       └── basic.spec.ts/
    │           └── screenshot-expectation.png
    └── Desktop Chrome/
        └── smoke-tests/
            └── basic.spec.ts/
                └── screenshot-expectation.png
```
2022-03-10 16:50:26 -08:00
Max Schmitt
372a3219f3
chore: remove Python from main Docker image (#12431) 2022-03-10 19:51:35 +01:00
Yury Semikhatsky
79c8746fd2
docs: add name for screenshot animations option (#12601) 2022-03-08 16:02:31 -08:00
Andrey Lushnikov
a3dff45974
chore: remove maxDiffPixels from toMatchSnapshot (#12539)
References #12441
2022-03-04 18:27:04 -08:00
Andrey Lushnikov
4953fc4845
chore: introduce toHaveScreenshot configuration options (#12507)
References https://github.com/microsoft/playwright/issues/12441
2022-03-04 17:30:43 -08:00
Yury Semikhatsky
f96a1a3ccd
fix(devops): list dependencies error (#12521) 2022-03-04 09:23:05 -08:00
Andrey Lushnikov
279e579097
chore: address pixelCount and pixelRatio review comments (#12501)
- documentation fixes
- rename pixelCount to maxDiffPixels
- rename pixelRatio to maxDiffPixelRatio
2022-03-03 23:17:31 -08:00
Yury Semikhatsky
b96bbc6ad7
docs: simplify test-assertions.md, avoid duplication (#12475) 2022-03-03 10:06:14 -08:00
Pavel Feldman
34bf0d4b87
feat(test-runner): allow grepping in project (#12469) 2022-03-02 15:02:35 -08:00
Yury Semikhatsky
a413c0f94c
docs: restore assertions timeout option in js (#12467) 2022-03-02 12:43:16 -08:00
Pavel Feldman
4b19d59ec5
feat(test): introduce fully parallel mode (#12446) 2022-03-01 18:12:21 -08:00
Andrey Lushnikov
33655ff168
devops(flakiness-dashboard): another fix to support empty metadata (#12430) 2022-03-01 01:50:40 -08:00
Andrey Lushnikov
4a1878e00e
devops: fix flakiness dashboard for projects with no metadata (#12426) 2022-03-01 01:21:00 -08:00
Andrey Lushnikov
396d920145
feat(test-runner): implement expect(pageOrLocator).toHaveScreenshot (#12242)
Fixes #9938
2022-02-28 12:25:59 -08:00
Max Schmitt
428a014cad
devops: Docker publishing cleanups (#12341) 2022-02-24 18:10:34 +01:00
Max Schmitt
0b55950d6a
chore: fix Android tests (#12325) 2022-02-24 06:57:32 +01:00
Andrey Lushnikov
a98babec69
feat(test-runner): introduce pixelCount and pixelRatio options (#12169)
This patch adds additional options to `toMatchSnapshot` method:
- `pixelCount` - acceptable number of pixels that differ to still
  consider images equal. Unset by default.
- `pixelRatio` - acceptable ratio of all image pixels (from 0 to 1) that differ to still
  consider images equal. Unset by default.

Fixes #12167, #10219
2022-02-17 15:44:03 -08:00
Yury Semikhatsky
abd7084bcc
fix: match default font families in headless chromium (#11340) 2022-02-11 09:06:17 -08:00
Dmitry Gozman
66b5cf5ae1
feat(remote): make PlaywrightServer work with browserType.connect (#11849)
This changes PlaywrigtServer to serve connections like `ws://localhost:3333/?browser=chromium`:
- launches the browser;
- talks `browserType.connect`-style protocol over websocket;
- compatible with `connectOptions` fixture.

```js
await playwright.chromium.connect({ wsEndpoint: 'ws://localhost:3333/?browser=chrome' });
```
2022-02-10 16:36:23 -08:00
Max Schmitt
b8d588aa47
chore: fix .NET generation script for .NET 6 (#11965) 2022-02-09 18:03:37 +01:00
Dmitry Gozman
19368e93af
feat(test runner): support connectOptions (#11919)
This allows to specify `connectOptions` in the config that
switch built-in `browser` to be remotely connected.
2022-02-08 20:45:42 -08:00
Andrey Lushnikov
39ed705904
chore: unify workspace helper scripts (#11925)
This patch unifies a variety of different workspace
scripts into a single `//utils/workspace.js`.

Fixes #11362
2022-02-08 10:35:00 -08:00
Pavel Feldman
1215057ca1
chore: use ipc transport for out-of-process driver (#11826) 2022-02-02 21:26:45 -08:00
Pavel Feldman
fdda759a9d
feat(parallel): allow setting enclosing scope parallel mode (#11822) 2022-02-02 20:44:11 -08:00
Andrey Lushnikov
ba0c7e679b
feat(test-runner): support expect.soft (#11800)
Soft expects will still fail the test, but will not abort it's execution. As a consequence of this:
-  `TestResult` now might have multiple errors, which is reflected with a new `testResult.erros: TestError[]` field.
- `TestInfo` now might have multiple errors as well, which is reflected with a new `testInfo.errors: TestError[]` field.

Fixes #7819
2022-02-02 18:33:51 -08:00
Max Schmitt
c752b28516
chore: pin production dependencies (#11793) 2022-02-02 11:14:41 +01:00
Pavel Feldman
6e2fcc4700
chore: do not expose suite load error (#11797) 2022-02-01 15:34:16 -08:00