feat: support experimental doc entries
- Params/options/members are marked as experimental in the docs.
- `experimental.d.ts` is generated that contains all types and
includes experimental features.
- `experimental.d.ts` is references in our tests so that we
can test experimental features.
- `fonts` option is restored as experimental.
Previously, we were guessing whether something is optional from the
comment (looking for stuff like "defaults to" or "optional").
To remove this error-prone technique, every optional property or
argument (but not option) is explicitly marked with "?".
As a result, fix documentation issues:
- Removed `TestStep.data` that is not a thing.
- Added `TestConfig.name` documentation.
- Refined a lot of optional types in the documentation.
- Aligned `test.fail()` and `test.slow()` docs with `test.skip()`.
- Strict set of docs vs types inconsistensies in the generator
This has two values:
- `"hide"` to hide input caret for taking screenshot
- `"initial"` to keep caret behavior unchanged
Defaults to `"hide"`.
Fixes#12643
This patch prints a friendly instructions in case Docker image version
mismatches Playwright version and there are missing browser
dependencies.
With this patch, Playwright will yield the following error:
```
root@f0774d2b2097:~# node a.mjs
node:internal/process/promises:279
triggerUncaughtException(err, true /* fromPromise */);
^
browserType.launch:
╔════════════════════════════════════════════════════════════════════════════════════════════╗
║ Host system is missing dependencies to run browsers. ║
║ This is most likely due to docker image version not matching Playwright version: ║
║ - Playwright: 1.22.0 ║
║ - Docker: 1.21.0 ║
║ ║
║ Either: ║
║ - (recommended) use docker image "mcr.microsoft.com/playwright:v1.22.0-focal" ║
║ - (alternative 1) run the following command inside docker to install missing dependencies: ║
║ ║
║ npx playwright install-deps ║
║ ║
║ - (alternative 2) use Aptitude inside docker: ║
║ ║
║ apt-get install libgbm1 ║
║ ║
║ <3 Playwright Team ║
╚════════════════════════════════════════════════════════════════════════════════════════════╝
at file:///root/a.mjs:3:10 {
name: 'Error'
}```
Fixes#12796
Co-authored-by: Dmitry Gozman <dgozman@gmail.com>
Resolves#11318.
* Adds `TestConfig.attachments` public API. (We opted to not implement an analog to the async `TestInfo.attach(…)` API.)
* Adds `TestConfig.attachments` to common reporters.
* Dogfoods some git and CI-info inference to generate useful atttachments
* Updates HTML Reporter to include a side bar to present a pre-defined set of attachments (a.k.a git/commit context sidebar)
Here's what it looks like:
<img width="1738" alt="Screen Shot 2022-03-21 at 3 23 28 PM" src="https://user-images.githubusercontent.com/11915034/159373291-8b937d30-fba3-472a-853a-766018f6b3e2.png">
See `tests/playwright-test/reporter-html.spec.ts` for an example of usage (for dogfood-ing only). In the future, if this becomes user-facing, there the Global Setup bit would likely become unnecessary (as would interaction with attachments array); there would likely just be a nice top-level config and/or CLI flag to enable collecting of info.
By default, fixtures share timeout with the test they are instantiated for.
However, for more heavy fixtures, especially worker-scoped ones, it makes
sense to have a separate timeout.
This introduces `{ timeout: number }` option to the list of fixture options
that opts the fixture into a dedicated timeout rather than sharing it
with the test.
This patch:
- Enables configuration of certain defaults for some options of `expect.toHaveScreenshot` method via `TestProject.expect.toHaveScreenshot` property
- Sets sensible defaults for these options:
* `fonts: "ready"`
* `animations: "disabled"`
* `size: "css"`
This patch reverts 2 commits that removed the feature from the method:
- "fix: explicitly ignore maxDiffPixels in toMatchSnapshot (#12570)"
commit b8af8458d6.
- "chore: remove `maxDiffPixels` from toMatchSnapshot (#12539)"
commit a3dff45974.
The `screenshotsDir` option controls the expectation storage
for `toHaveScreenshot()` function.
The new expectation management for screenshots has the following
key properties:
- All screenshots are stored in a single folder called `screenshotsDir`.
- Screenshot names **do not** respect `snapshotDir` and `snapshotSuffix`
configurations.
- `screenshotsDir` is configurable per project. This way a "smoke tests"
project can re-use screenshots from "all tests" project.
- Host platform is a top-level folder.
For example, given the following config:
```js
// playwright.config.ts
module.exports = {
projects: [
{ name: 'Mobile Safari' },
{ name: 'Desktop Chrome' },
],
};
```
And the following test structure:
```
smoke-tests/
└── basic.spec.ts
```
Will result in the following screenshots folder structure by default:
```
__screenshots__/
└── darwin/
├── Mobile Safari/
│ └── smoke-tests/
│ └── basic.spec.ts/
│ └── screenshot-expectation.png
└── Desktop Chrome/
└── smoke-tests/
└── basic.spec.ts/
└── screenshot-expectation.png
```
This patch adds additional options to `toMatchSnapshot` method:
- `pixelCount` - acceptable number of pixels that differ to still
consider images equal. Unset by default.
- `pixelRatio` - acceptable ratio of all image pixels (from 0 to 1) that differ to still
consider images equal. Unset by default.
Fixes#12167, #10219
This changes PlaywrigtServer to serve connections like `ws://localhost:3333/?browser=chromium`:
- launches the browser;
- talks `browserType.connect`-style protocol over websocket;
- compatible with `connectOptions` fixture.
```js
await playwright.chromium.connect({ wsEndpoint: 'ws://localhost:3333/?browser=chrome' });
```
Soft expects will still fail the test, but will not abort it's execution. As a consequence of this:
- `TestResult` now might have multiple errors, which is reflected with a new `testResult.erros: TestError[]` field.
- `TestInfo` now might have multiple errors as well, which is reflected with a new `testInfo.errors: TestError[]` field.
Fixes#7819
This patch:
- moves docker publishing to main repo
- extracts all the logic from the GHA YML file to Bash
- starts using the script in both `publish_canary.yml` and
`publish_release.yml`
Fixes#10351
feat(api): add explicit async testInfo.attach
We add an explicit async API for attaching file paths (and Buffers) to
tests that can be awaited to help users ensure they are attaching files
that actually exist at both the time of the invocation and later when
reporters (like the HTML Reporter) run and package up test artifacts.
This is intended to help surface attachment issues as soon as possible
so you aren't silently left with a missing attachment
minutes/days/months later when you go to debug a suddenly breaking test
expecting an attachment to be there.
NB: The current implemntation incurs an extra file copy compared to
manipulating the raw attachments array. If users encounter performance
issues because of this, we can consider an option parameter that uses
rename under the hood instead of copy. However, that would need to be
used with care if the file were to be accessed later in the test.
1. Fixtures defined in test.extend() can now have `{ option: true }` configuration that makes them overridable in the config. Options support all other properties of fixtures - value/function, scope, auto.
```
const test = base.extend<MyOptions>({
foo: ['default', { option: true }],
});
```
2. test.declare() and project.define are removed.
3. project.use applies overrides to default option values and nothing else. Any test.extend() and test.use() calls take priority over config options.
Required user changes: if someone used to define fixture options with test.extend(), overriding them in config will stop working. The solution is to add `{ option: true }`.
```
// Old code
export const test = base.extend<{ myOption: number, myFixture: number }>({
myOption: 123,
myFixture: ({ myOption }, use) => use(2 * myOption),
});
// New code
export const test = base.extend<{ myOption: number, myFixture: number }>({
myOption: [123, { option: true }],
myFixture: ({ myOption }, use) => use(2 * myOption),
});
```
This patch:
- removes the Visual Regression bits from the images. These will
probably be added back when it's ready for the prime time.
* removes VNC && noVNC
* removes Playwright Agent
- cleans up apt caches
This drops image size from 2.03GB to 1.91GB.
This patch:
- removes the "is this upstream check?" when publishing daily version
- fixes the bug when manually triggered builds were pushed under @beta
tag
Drive-by: rename "tip-of-tree" flag to "next" for consistency.
When fixture value `R` is a function, TypeScript sometimes confuses
function `R` and function `async ({}, use) => {}`. This leads to
`any` types in the latter because it could be either of the functions
as TS thinks.
The solution is to only accept the second syntax, assuming that noone
passes fixture value that is a function as is:
```js
// This will stop working.
test.extend<{ foo: (x: number) => number }>({
foo: x => 2 * x,
});
// This will get inferred types and autocomplete.
test.extend<{ foo: (x: number) => number }>({
foo: async ({}, use) => {
await use(x => 2 * x);
},
});
```
This patch adds a general-purpose grid framework to parallelize
Playwright across multiple agents.
This patch adds two CLI commands to manage grid:
- `npx playwright experimental-grid-server` - to launch grid
- `npx playwrigth experimental-grid-agent` - to launch agent in a host
environment.
Grid server accepts an `--agent-factory` argument. A simple
`factory.js` might look like this:
```js
const child_process = require('child_process');
module.exports = {
name: 'My Simple Factory',
capacity: Infinity, // How many workers launch per agent
timeout: 10_000, // 10 seconds timeout to create agent
launch: ({agentId, gridURL, playwrightVersion}) => child_process.spawn(`npx`, [
'playwright'
'experimental-grid-agent',
'--grid-url', gridURL,
'--agent-id', agentId,
], {
cwd: __dirname,
shell: true,
stdio: 'inherit',
}),
};
```
With this `factory.js`, grid server could be launched like this:
```bash
npx playwright experimental-grid-server --factory=./factory.js
```
Once launched, it could be used with Playwright Test using env variable:
```bash
PW_GRID=http://localhost:3000 npx playwright test
```
