Dmitry Gozman
503f8f51dc
fix(reuse): stop pending operations upon reuse/disconnect ( #18997 )
2022-11-22 15:21:20 -08:00
Yury Semikhatsky
2dc51f6c46
fix(runner): run all setup files when there is test.only ( #18967 )
2022-11-21 16:33:23 -08:00
Yury Semikhatsky
e1189a96b6
feat(runner): run all setup files if none matched the filter ( #18922 )
...
The behavior regarding filters (both in config, command line and .only)
is the following:
- if some of tests match and none of setup match then we'll run all
setup files and all matching tests
- otherwise the filters apply to setup files the same way as to regular
tests
2022-11-21 09:23:28 -08:00
Yury Semikhatsky
c0d0f54a12
feat(runner): allow filtering setup and tests ( #18900 )
...
Running `npx playwright test file:123` will have the following behavior
- if only test files match then only matching subset of tests will run
but all setup files will run as well
- if only setup files match the filter then only those setup tests will
run
- if both setup and test files match an error will be thrown
2022-11-18 11:35:29 -08:00
Dmitry Gozman
1ec0bb277d
feat(expect): ensure at least one expectation check, no matter the timeout ( #18895 )
...
References #18859 .
2022-11-17 19:43:10 -08:00
Yury Semikhatsky
0f4b67bc6d
test(cli): setup files are in list-files and test --list output ( #18890 )
2022-11-17 16:31:04 -08:00
Pavel Feldman
4e58b0c2ea
chore: render timed out error message when expect timeouts ( #18863 )
...
Fixes https://github.com/microsoft/playwright/issues/18859
2022-11-16 17:00:42 -08:00
Yury Semikhatsky
05fb3e6de8
Revert "chore: hide TestProject.setup and TestInfo.storage ( #18800 )" ( #18836 )
...
Now that the patch has been cherry-picked to the release branch
reverting it on main.
This reverts commit 6deba5dc21
2022-11-16 11:23:10 -08:00
Yury Semikhatsky
6deba5dc21
chore: hide TestProject.setup and TestInfo.storage ( #18800 )
2022-11-14 17:03:01 -08:00
Andrey Lushnikov
f3a99fdd69
chore: address API review comments for the snapshotPathTemplate ( #18716 )
...
This patch:
- updates documentation to lead users from `TestConfig.snapshotDir` and
`testInfo.snapshotSuffix` to `TestConfig.snapshotPathTemplate` as a
better and more flexible alternative.
- drops `{snapshotSuffix}` from documentation
- stops using `snapshotSuffix = ''` in our own tests and switches us
to the `snapshotPathTemplate`.
- adds `{testName}` token.
2022-11-10 17:23:57 -08:00
Andrey Lushnikov
db826c9c8c
chore: drive-by fixes for the snapshotPathTemplate ( #18715 )
...
- Use `snapshotPathTemplate` for docker screenshots in html-reporter
- Mark the snapshot path template test as slow since it re-spawns
worker for each project.
- Fix docker smoke tests
2022-11-10 15:50:13 -08:00
Dmitry Gozman
194abc35cd
Revert #18561 "feat(test runner): test.reset() to reset options to default/config value" ( #18704 )
...
This reverts commit 6fef227f43
2022-11-10 14:45:05 -08:00
Andrey Lushnikov
6d491f928d
feat(playwright-test): introduce snapshotPathTemplate configuration ( #18568 )
...
This configuration option allows to set a string with template
values for precise control over snapshot path location.
An example of `snapshotPathTemplate` usage:
```ts
// playwright.config.ts
// Notice the `testDir` configuration!
export default {
testDir: './tests',
snapshotPathTemplate: './__screenshots__/{platform}/{projectName}/{testFilePath}/{arg}{ext}',
}
```
Currently supported "magic tokens" inside the `snapshotPathTemplate`
are:
- `{testDir}` - project's `testDir`
- `{snapshotDir}` - project's `snapshotDir`
- `{platform}` - `process.platform`
- `{projectName}` - Project's sanitized name
- `{testFileDir}` - Directories in relative path from `testDir` to test
file path (e.g. `page/` in the example below)
- `{testFileName}` - Test file name (with extension) (e.g.
`page-click.spec.ts` in the example below)
- `{testFilePath}` - Relative path from `testDir` to test file path
(e.g. `page/page-click.spec.ts` in the example below)
- `{ext}` - snapshot extension (with dots)
- `{arg}` - joined snapshot name parts, without extension (e.g.
`foo/bar/baz` in the example below)
- `{snapshotSuffix}` - `testInfo.snapshotSuffix` value.
Consider the following file structure:
```
playwright.config.ts
tests/
└── page/
└── page-click.spec.ts
```
The following `page-click.spec.ts`:
```ts
// page-click.spec.ts
import { test, expect } from '@playwright/test';
test('should work', async ({ page }) => {
await expect(page).toHaveScreenshot(['foo', 'bar', 'baz.png']);
});
```
Fixes #7792
2022-11-09 15:29:07 -08:00
Andrey Lushnikov
9bcb28f25a
Revert "feat: send SIGTERM to webserver before SIGKILL'ing it. ( #18220 )" ( #18661 )
...
This reverts commit c63a0b536dhttps://github.com/microsoft/playwright/pull/18564
2022-11-09 09:18:33 -08:00
Yury Semikhatsky
d5494edf71
feat(runner): TestOptions.storageStateName ( #18587 )
2022-11-07 16:27:38 -08:00
Dmitry Gozman
9172a2ca5a
test: use strict png comparator in tests ( #18622 )
2022-11-07 13:32:46 -08:00
Dmitry Gozman
6a65a43e9a
chore: use consistent asLocator() in all logs ( #18586 )
...
References #18524 .
2022-11-04 15:19:16 -07:00
Yury Semikhatsky
25dc0bfacb
feat(runner): change storage fixture to TestInfo.storage() ( #18584 )
2022-11-04 14:28:25 -07:00
Yury Semikhatsky
a9c15a25f8
test: storageState from config key ( #18581 )
2022-11-04 13:10:33 -07:00
Dmitry Gozman
6fef227f43
feat(test runner): test.reset() to reset options to default/config value ( #18561 )
2022-11-04 09:03:29 -07:00
Yury Semikhatsky
c8cd07594c
feat(runner): initialize context.storageState from storage entry ( #18549 )
2022-11-03 14:20:39 -07:00
Xiaoxing Ye
9338355e47
feat(testinfo): add name to attachment output name ( #18440 )
...
Per discussion in #12950 , adding sanitized name to the output filename
prefix. This can make debugging easier, and the filename structure more
human friendly.
2022-11-03 13:54:51 -07:00
Yury Semikhatsky
45aa82242d
feat(runner): storage fixture ( #18522 )
2022-11-02 20:22:35 -07:00
Yury Semikhatsky
1d2fc1e963
feat(runner): project.setup ( #18486 )
2022-11-01 23:44:30 -07:00
Pavel Feldman
2183d9e9a2
chore: use codemirror for editor ( #18482 )
2022-11-01 15:04:30 -07:00
Yury Semikhatsky
7337dd4e28
chore: revert project.{stage,run} ( #18462 )
2022-10-31 14:04:24 -07:00
Dmitry Gozman
2d3b2a0768
feat(test runner): test.describe.configure({ retries, timeout }) ( #18388 )
...
References #10825 .
2022-10-27 15:53:27 -07:00
Pavel Feldman
db456a020c
chore: do not point to node_modules in code frames ( #18358 )
...
Fixes https://github.com/microsoft/playwright/issues/18330
2022-10-26 15:18:31 -07:00
Pavel Feldman
1505a952fe
chore: make handle beforeunload when reusing the context ( #18357 )
...
Fixes: https://github.com/microsoft/playwright/issues/17903
2022-10-26 15:17:40 -07:00
Max Schmitt
ffc47271f2
fix(junit): escape null control character ( #18325 )
...
Fixes https://github.com/microsoft/playwright/issues/18322
2022-10-25 14:39:16 -07:00
Dmitry Gozman
caa9c6a597
fix(test runner): make sure soft expect error does not mask a timeout flag ( #18321 )
...
We have to reliably know whether test timed out or not, and soft expect
error could have marked it with `status=failed` but it would still time
out. Now we have a separate `_didTimeout` flag for this.
Fixes #18023 .
2022-10-25 12:34:15 -07:00
Pavel Feldman
ab78865a8d
chore: render steps in list reporter ( #18269 )
2022-10-24 15:54:53 -07:00
Andrey Lushnikov
c63a0b536d
feat: send SIGTERM to webserver before SIGKILL'ing it. ( #18220 )
...
We now will send `SIGTERM` to the webserver and wait for the `timeout`
before sending `SIGKILL` to it.
Fixes #18209
2022-10-21 08:55:06 -07:00
Pavel Feldman
fb9555fb5d
fix(ts): resolve .js to .ts in non-ESM mode too ( #18219 )
...
Fixes https://github.com/microsoft/playwright/issues/18077
2022-10-20 12:21:22 -07:00
sand4rt
4ed2a01d9c
core(ct): import paths ( #18131 )
2022-10-19 19:41:21 -07:00
Pavel Feldman
ad9729f246
chore: respect tsconfig paths in js files ( #18191 )
...
Fixes: https://github.com/microsoft/playwright/issues/17804
2022-10-19 19:38:14 -07:00
Max Schmitt
af38449f42
fix(pwtest): fix focused line with dirname specified ( #18189 )
2022-10-19 15:05:59 -07:00
Dmitry Gozman
9fe72a1da8
fix(test runner): remove unused type imports during transform ( #18157 )
...
Fixes #18117 .
2022-10-19 13:06:11 -07:00
Yury Semikhatsky
11eb719d13
feat(runner): project run: "always" ( #18160 )
...
Projects marked with `run: 'always'` are non shard-able and run after
failures.
2022-10-18 17:18:45 -07:00
Pavel Feldman
739b64a09a
chore: render typed locators in the recorder ( #18162 )
2022-10-18 16:39:58 -07:00
Pavel Feldman
7ab4c17519
chore: support jsx/tsx syntax in test files ( #18123 )
...
Fixes https://github.com/microsoft/playwright/issues/17964
2022-10-18 12:45:33 -07:00
Yury Semikhatsky
08a3a269cd
feat(runner): project.canShard ( #18037 )
2022-10-12 14:34:22 -07:00
Yury Semikhatsky
3b8f63d703
feat(runner): project.stopOnFailure ( #18009 )
2022-10-11 17:04:01 -07:00
Yury Semikhatsky
3592269caf
feat(runner): project.stage ( #17971 )
2022-10-10 17:56:18 -07:00
Yury Semikhatsky
2d72d0ba03
chore: remove config.groups ( #17974 )
2022-10-10 16:42:48 -07:00
Dmitry Gozman
b64457d8ce
fix(locators): make regex escape work when multiple spaces are present ( #17893 )
2022-10-06 16:23:40 -07:00
Pavel Feldman
c0e4caa604
fix(esm+tsconfig): allow mapped ts files in esm mode ( #17862 )
...
Fixes https://github.com/microsoft/playwright/issues/17840
2022-10-05 18:00:49 -07:00
Pavel Feldman
5424c8c385
fix(test): unflake workers test ( #17728 )
2022-09-30 08:07:20 -07:00
Yury Semikhatsky
51966bc045
fix: --grep and --grep-invert should intersect with config ( #17716 )
...
Fixes https://github.com/microsoft/playwright/issues/17405
2022-09-29 16:39:21 -07:00
Yury Semikhatsky
9f17ee6871
feat: group filter ( #17646 )
2022-09-28 18:45:01 -07:00
Pavel Feldman
cadd4d1dd0
chore: migrate http server to ts ( #17677 )
2022-09-28 17:01:13 -07:00
Pavel Feldman
d8f67eb75d
feat(api): introduce getByTestId ( #17645 )
2022-09-27 21:06:07 -07:00
Pavel Feldman
8ad201b802
chore: add snippet to the json report ( #17567 )
2022-09-27 13:23:08 -07:00
Dmitry Gozman
50d4a5844f
test: mark "should cache build" as slow ( #17637 )
...
Frequently timing out on the bots.
2022-09-27 13:09:05 -07:00
Ross Wollman
e73676d094
fix(reporters): output relative to config ( #17518 )
...
Fixes #17412 .
Supercedes #17413 .
- if configured via playwright.config.ts, relative paths should be
relative to the config.
- if configured via env var, should be relative to `cwd`
2022-09-26 11:01:43 -07:00
Yury Semikhatsky
10d7c60abf
feat(runner): project execution schedule ( #17548 )
2022-09-23 20:01:27 -07:00
Jean-François Greffier
a15fe50e7b
feat(test runner): workers as percentage ( #17400 )
...
Allows to set workers as a percentage of logical CPUs, for example
"50%".
Examples :
```bash
npx playwright test --workers 3
npx playwright test --workers 50%
```
```js
const config: PlaywrightTestConfig = {
// ...
workers: '33%',
}
```
2022-09-21 11:17:36 -07:00
Dmitry Gozman
da7feb9cc6
fix(esm): silence warning when running with experimental loader ( #17493 )
...
Unfortunately, this silences all warnings, not just the experimental
loader warning.
2022-09-21 10:56:17 -07:00
Ross Wollman
8d639ae50e
chore: revert toHaveAttribute type sig and overloads ( #17406 )
...
Relates #16517 .
Revert "docs(python): add missing NotToHaveAttribute overloads (#17371 )"
This reverts commit 2e1ea29614#17345 )"
This reverts commit 4b8a85e69d#17024 )"
This reverts commit 1dc05bd4c6#16941 )"
This reverts commit f30ac1d678#16767 )"
This reverts commit 622c73cc1e
2022-09-16 11:17:35 -07:00
Pavel Feldman
30ff27843a
chore: rebuild components on new vite ( #17367 )
2022-09-15 15:24:01 -07:00
Pavel Feldman
016883602d
chore: roll test runner to tot ( #17351 )
2022-09-14 19:51:22 -07:00
Andrey Lushnikov
8466c436c9
fix(list reporter): properly count new lines for long stdout ( #17339 )
2022-09-14 15:25:24 -07:00
Andrey Lushnikov
4f11a4d5e7
feat(docker): use the __screenshots__ dir for snapshots by default ( #17311 )
...
This patch opts into the `__screnshots__` folder snapshot management
for docker.
With this patch, docker-originating snapshots will be stored in the
following folder:
```
{testDir}/__screenshots__/{projectName}/{testFilePath}/{snapshotName}
```
Where `{testFilePath}` is a test file path relative to `testDir`
Drive-by: introduce and document the `PLAYWRIGHT_DOCKER` environment
variable that enables docker integration.
2022-09-13 15:49:04 -07:00
Dmitry Gozman
48ac918cec
test: unflake "should load web server w/o esm loader in ems module" ( #17269 )
2022-09-12 11:42:04 -07:00
sand4rt
72a18754ef
core(ct): rename tests ( #17216 )
2022-09-12 09:27:53 -07:00
Dmitry Gozman
bca13bc35a
feat(assertions): support toBeVisible({ visible }) ( #17207 )
2022-09-09 08:33:23 -07:00
Pavel Feldman
80e9c5dc55
fix(reuse): restore storage state ( #17175 )
2022-09-07 19:15:04 -07:00
Pavel Feldman
c91df61ca3
chore: optionally connect instead of launching ( #17174 )
2022-09-07 15:57:20 -07:00
Max Schmitt
14ec71b905
chore: roll stable test-runner to 1.26.0-alpha-sep-7-2022 ( #17166 )
2022-09-07 20:11:09 +02:00
Dmitry Gozman
f0c5810609
feat(assertions): support toBeEditable({ editable }) ( #17065 )
2022-09-06 12:50:45 -07:00
Dmitry Gozman
306ab34aa3
feat(assertions): support toBeEnabled({ enabled }) ( #17058 )
2022-09-06 11:40:34 -07:00
Andrey Lushnikov
fafd9837ba
feat: introduce the --ignore-snapshots option ( #17004 )
...
This patch introduces `--ignore-snapshots` Playwright Test CLI option,
and `ignoreSnapshots` configuration argument.
2022-09-01 05:34:36 -07:00
Yury Semikhatsky
5d6253f743
fix: stop har recording when APIRequestContext is disposed ( #17007 )
2022-08-31 21:51:38 -07:00
nate-ro
409bab59ea
chore: added starting time of test to the base json reporter ( #16885 )
2022-08-30 20:03:21 -07:00
Yury Semikhatsky
f30ac1d678
fix: support toHaveAttribute(name, options) ( #16941 )
2022-08-30 17:53:00 -07:00
Yury Semikhatsky
11dfd31dd9
feat: --pass-with-no-tests option ( #16902 )
2022-08-29 15:46:34 -07:00
Pavel Feldman
4d892475da
chore: do not use experimental loader for web server ( #16733 )
2022-08-23 10:22:05 -07:00
Pavel Feldman
d7be1fcca8
fix(esm): allow importing ts from esm ( #16735 )
2022-08-23 10:20:56 -07:00
Sergio Freire
abe7cf23a7
fix(junit reporter): remove source location from classname attribute ( #16499 )
2022-08-19 16:42:21 -07:00
Max Schmitt
7e2aec7454
chore: align more ESLint rules with VSCode formatting ( #16647 )
2022-08-18 20:12:33 +02:00
Pavel Feldman
bd06d1604f
chore: render original exception location in error frame ( #16515 )
2022-08-15 09:28:55 -07:00
Pavel Feldman
e1d3246d1c
fix(ct): pass local config to preview ( #16481 )
2022-08-12 10:37:35 -07:00
Dmitry Gozman
7acbd052e4
test: migrate some expect() tests to be more readable ( #16394 )
...
This moves some expect() matchers tests from test runner tests to page tests,
because these are implemented through a library call anyway.
Makes tests more readbable, faster and easier to test specific details.
2022-08-10 15:10:25 -07:00
Johannes Loher
3e67a7c836
feat(plugins): send accept header in webServer url checking ( #16251 )
2022-08-10 12:48:37 -07:00
Dmitry Gozman
baa2ef2700
fix(test runner): show interrupted as yellow ( #16396 )
...
Drive-by: fix unreadable character in "duplicate titles" error.
2022-08-09 21:17:30 -07:00
Dmitry Gozman
f58c376443
fix(test runner): do not resolve relative imports through baseUrl ( #16395 )
2022-08-09 21:16:27 -07:00
Denis Sokolov
92aacb9345
test: another test for baseUrl/relative imports ( #16338 )
...
#15891
2022-08-10 01:05:48 +02:00
Ross Wollman
a3d99f1b4a
chore: trace viewer fallback error ( #16365 )
...
Fixes #16349 .
2022-08-09 15:53:11 -07:00
Ross Wollman
36b92d8847
fix: toBeFocused should match shadow elements ( #16362 )
...
Fixes #16268 .
2022-08-08 15:34:58 -07:00
Dmitry Gozman
f6d94f0ac9
chore: update "X fatal errors" message ( #16325 )
...
Also, exlude certain errors from triggering this message:
- `no tests found`
- `duplicate test titles are not allowed`
- `--forbid-only found a focused test`
- `Timed out waiting 3600s for the entire test run`
2022-08-05 21:21:43 -07:00
Dmitry Gozman
7a86e140f5
chore: remove TestProject.projectSetup ( #16321 )
...
There are better ideas to address this issue in more general manner.
2022-08-05 15:24:30 -07:00
Dmitry Gozman
cb30cb4eb7
feat(test runner): add --trace cli option ( #16277 )
2022-08-05 09:20:39 -07:00
Dmitry Gozman
919b1cbb22
fix(test runner): do not swallow afterEach errors after skipped tests ( #16276 )
...
Drive-by: add more test runner logging.
2022-08-04 18:49:23 -07:00
Pavel Feldman
97fa2518e9
fix(reuse): clear storage after stopping all scripts ( #16275 )
2022-08-04 16:39:18 -07:00
Pavel Feldman
b3d30a808f
fix(reuse): reset sw, db, storages ( #16265 )
2022-08-04 15:01:34 -07:00
Dmitry Gozman
2fa48483af
test: unflake "should retain traces for interrupted tests" ( #16235 )
...
Ensure both tests start before one of them finishes.
2022-08-04 09:18:18 -07:00
Pavel Feldman
74f7005c02
chore: preserve window while reusing window ( #16225 )
2022-08-03 16:14:28 -07:00
Dmitry Gozman
03fe75251b
fix(test runner): show tests as interrupted when maxFailures stops them ( #16178 )
...
Previously, we marked these tests as skipped, and it was sometimes
confusing, because they did actually run and produce some output/artifacts.
2022-08-03 15:25:25 -07:00
Dmitry Gozman
445fe032f5
feat(test runner): separate interrupted status from skipped ( #16124 )
2022-08-02 12:55:43 -07:00
Dmitry Gozman
3112edb4ca
feat(test runner): TestProject.projectSetup ( #16063 )
...
`projectSetup` is a project-scoped alternative to `globalSetup`.
It is only executed if at least one test from the project is scheduled to run.
2022-08-01 09:01:23 -07:00
Yury Semikhatsky
7d306bbc66
feat: return value from step ( #16060 )
2022-07-29 15:16:07 -07:00
Dmitry Gozman
e830fe821d
feat(test runner): introduce test.describe.fixme ( #16059 )
2022-07-29 12:44:22 -07:00
Yury Semikhatsky
af8e3e7afa
feat: print response text when toBeOK fails ( #16040 )
2022-07-29 11:46:48 -07:00
Dmitry Gozman
62e4e80599
feat(test runner): show the number of fatal errors at the end ( #15975 )
2022-07-28 14:46:21 -07:00
Max Schmitt
a089bf3df5
chore(test-runner): better error message if page gets used inside beforeAll ( #16012 )
2022-07-28 23:07:28 +02:00
Dmitry Gozman
3d89506704
fix(fixtures): make undefined option mean "default value" ( #16026 )
...
In the following example, `locale` inside the `describe`
would be reverted to the default value:
```js
test.use({ locale: 'en-GB' });
test.decsribe(() => {
test.use({ locale: undefined });
});
```
2022-07-28 12:57:05 -07:00
Pavel Feldman
8c190c3e09
feat(testId): expose persistent test id ( #16003 )
2022-07-27 20:17:19 -07:00
Dmitry Gozman
a94fe361a2
test: unflake a few tests ( #15959 )
2022-07-27 11:27:53 -07:00
Dmitry Gozman
6009804e0e
Revert "fix(test runner): ignore undefined values in fixtures definit…ions ( #15119 )" ( #15979 )
...
Revert "fix(test runner): ignore undefined values in fixtures definitions (#15119 )"
Revert commit d7b63fa0b4
2022-07-27 08:51:45 -07:00
Dmitry Gozman
cd9dccbe27
fix(test runner): serial suites inside parallel suite should run in parallel ( #15769 )
2022-07-26 13:38:25 -07:00
Dmitry Gozman
c0d78c5642
fix(runner): friendly error message instead of "digests do not match" ( #15939 )
2022-07-26 08:53:32 -07:00
Pavel Feldman
6fcc916f00
test: passing test for issue 15891 ( #15930 )
2022-07-25 15:28:14 -07:00
Dmitry Gozman
9ce727c936
feat(list reporter): number test runs to see testing progress ( #15693 )
...
Drive-by: replace a few colors.gray with colors.dim for better contrast.
2022-07-25 13:22:25 -07:00
Max Schmitt
004cd9273c
fix(test-runner): multiple webServer types ( #15901 )
2022-07-25 10:39:06 +02:00
Dmitry Gozman
f954891491
fix(runner): globalTeardown without globalSetup should work ( #15814 )
2022-07-20 12:41:35 -07:00
Dmitry Gozman
6a60de1b2f
test: add failing test for traces after interruption ( #15796 )
2022-07-19 13:50:52 -07:00
Dmitry Gozman
a56d801352
fix(test runner): show fixture name when worker times out ( #15724 )
2022-07-15 13:05:48 -07:00
Max Schmitt
4bba41ab8f
Revert "feat(matchers): add toContainClass ( #15491 )" ( #15670 )
...
This reverts commit e4debd0bf6
2022-07-14 22:03:37 +02:00
Pavel Feldman
d02914fa3a
chore: relax transpilation criteria to allow ems imports of .vue files ( #15592 )
2022-07-13 16:11:38 -07:00
Dmitry Gozman
22ead7b5ab
Revert "fix(test runner): handle istty in line reporter ( #15455 )" ( #15633 )
...
This reverts commit 767babc3a2
2022-07-13 15:11:56 -07:00
Dmitry Gozman
51c3ea55ed
Revert "feat(line reporter): show testing progress ( #15547 )" ( #15631 )
...
This reverts commit e76d9b3b28
2022-07-13 13:54:26 -07:00
Pavel Feldman
00848b1bcf
fix(ct): restore public folder deployment ( #15594 )
2022-07-12 16:14:48 -07:00
Yury Semikhatsky
a6daf600a9
fix: support hasColors and getColorDepth in worker ( #15590 )
2022-07-12 14:47:14 -07:00
Pavel Feldman
3939b9f36e
chore: migrate component testing to server-side page reuse ( #15477 )
2022-07-12 14:30:24 -07:00
Max Schmitt
e4debd0bf6
feat(matchers): add toContainClass ( #15491 )
2022-07-12 23:17:10 +02:00
Max Schmitt
7e1801bd30
fix(matchers): repeating values lead to no error ( #15559 )
2022-07-12 20:03:27 +02:00
Dmitry Gozman
64353c4b96
fix(test runner): make sure worker cleans up if last test was skipped ( #15552 )
2022-07-11 21:33:56 -07:00
Dmitry Gozman
e76d9b3b28
feat(line reporter): show testing progress ( #15547 )
...
Line reporter now shows stats in addition to the test name:
```
[chromium] › page/page-click-react.spec.ts:108:1 › should not retarget when element changes on hover
[21/93] Passed: 17 Flaky: 0 Failed: 0 Skipped: 4 (7s)
```
2022-07-11 20:44:50 -07:00
Max Schmitt
98ea418124
fix(expect): expect.not types ( #15487 )
2022-07-11 18:31:53 +02:00
Dmitry Gozman
6dc4a51c92
fix(test runner): more friendly test duration ( #15476 )
...
fix(test runner): more firendly test duration
Reported test duration now does not include time spent in
`beforeAll`, `afterAll` and fixtures that have a separate timeout.
This is to avoid different reported test execution time,
depending on the test execution order.
2022-07-08 11:45:44 -07:00
Dmitry Gozman
894ead5198
fix(test runner): mention fixture in worker teardown timeout error message ( #15499 )
2022-07-08 11:44:37 -07:00
Yury Semikhatsky
51aa5fea78
fix: colors in console output from worker ( #15421 )
...
Color highlights regressed in #14455
Fixes #15366
2022-07-07 17:56:41 -07:00
Ross Wollman
799d4703bc
feature(test-runner): multiple web servers ( #15388 )
...
Fixes #8206 .
Since #8206 is a long-awaited (~ 1 year old), popular (~ 45 reactions, frequently requested in community channels, etc.), this PR aims to unblock folks.
Notably, we do not innovate on the `webServer` API, despite knowing we're not in love with it. We'll save the innovation for either Plugins or a new `LaunchConfigs` option. (We haven't yet arrived at a Plugin API we like, and instead of launching a new option guessing what the "better" launchConfig API would be, let's wait and see how folks use this new Array-variant of `webServer` which—despite its name—can be used for non-Web Server launches!
2022-07-07 15:27:21 -07:00
Dmitry Gozman
767babc3a2
fix(test runner): handle istty in line reporter ( #15455 )
...
When running without tty, line reporter outputs a line for each
percent of the tests, thus limiting the output to ~100 lines.
In addition, reporters now support PLAYWRIGHT_LIVE_TERMINAL
env variable to force tty mode.
2022-07-07 09:55:12 -07:00
Dmitry Gozman
89c16cbc8b
feat(runner): anonymous describes ( #15424 )
...
Useful to set some options without coming up with an artificial name.
```js
test.describe(() => {
test.use({ colorScheme: 'dark' });
test('my test', () => {});
})
```
2022-07-06 13:54:11 -07:00
Dmitry Gozman
c02d83fb11
fix(runner): filter only after duplicate titles check ( #15401 )
2022-07-06 12:51:26 -07:00
Dmitry Gozman
ee82837fb5
fix(fixtures): do not run user function when required fixture has failed ( #15385 )
...
Currently, it is possible to run a function, e.g. a second `beforeEach` hook,
that will receive `null` for the fixture that has already failed before.
This PR skips running any user function that uses a fixture that has already
failed, just like if the fixture would be initialized again and failing for
the second time.
2022-07-05 17:15:28 -07:00
Ross Wollman
981f5ab8c1
fix(test-runner): apply fixme v. skip annotations ( #15277 )
...
Currently, if `text.fixme()` or `test.skip()` is used within a test, we
add a `fixme` or `skip` annotation. However, if the wrapper style is
used:
```
test.fixme('should work', () => {…})
```
the annotations were missing. This change adds annotations for the
above.
These annotations are important for reporting purposes and knowing
exactly what flavor of "skipped" was used.
Fixes #15239 .
2022-07-05 10:46:30 -07:00
Dmitry Gozman
2a805c1f1c
fix(line reporter): print currently running test ( #15339 )
2022-07-05 09:07:55 -07:00
Pavel Feldman
295ea7a3cb
fix(config): fall back to context options ( #15309 )
2022-07-01 10:20:13 -07:00
Pavel Feldman
c4d2342339
fix(config): fall back to launch options ( #15293 )
2022-07-01 08:12:57 -07:00
Dmitry Gozman
5083da9d98
test: unflake "should truncate long test names" ( #15286 )
2022-06-30 20:24:00 -07:00
Dmitry Gozman
b86926e691
fix(test runner): update timeout error messages ( #15117 )
...
Some of the current timeout error messages are confusing, because they do not suggest that the issue is most likely a slow test. This PR updates timeout messages as follows:
- Test timeout of 30000ms exceeded.
- Test timeout of 30000ms exceeded while setting up "browser".
- Test timeout of 30000ms exceeded while tearing down "context".
- Test timeout of 30000ms exceeded while setting up "playwright configuration".
- Test timeout of 30000ms exceeded while running "beforeEach" hook.
- Test timeout of 30000ms exceeded while running "afterEach" hook.
- "beforeAll" hook timeout of 30000ms exceeded.
- "afterAll" hook timeout of 30000ms exceeded.
- Worker teardown timeout of 30000ms exceeded.
- "skip" modifier timeout of 30000ms exceeded.
- Fixture "myCustomFixture" timeout of 5000ms exceeded.
2022-06-30 17:05:08 -07:00
Dmitry Gozman
dc1d8dc3da
test: improve a few test runner tests that time out ( #15287 )
2022-06-30 17:03:01 -07:00
Max Schmitt
3d1d723c56
fix(esm): show codeframe when errors get reported ( #15262 )
2022-06-30 21:17:08 +02:00
Dmitry Gozman
f0b3b280a5
test: increase small timeouts to accomodate two workers ( #15257 )
2022-06-30 09:05:39 -07:00
Max Schmitt
71fc53bbf8
fix(matchers): toHaveClass on SVG elements ( #15267 )
...
Fixes #15260
2022-06-30 07:01:26 -07:00
Ross Wollman
9fb80c905b
test: match more of output to ensure source map support is good ( #15206 )
...
Co-authored-by: Max Schmitt <max@schmitt.mx>
2022-06-29 22:26:48 +02:00
Dmitry Gozman
79163e802a
fix(test runner): screenshot immediately after failure ( #15159 )
...
Previously, screenshot was taken after hooks and fixtures teardown.
However, hooks can easily modify the state of the page, and
screenshot would not reflect the moment of failure.
Instead, we take screenshots immediately after the test function
finishes with an error.
2022-06-27 17:46:39 -07:00
Dmitry Gozman
d7b63fa0b4
fix(test runner): ignore undefined values in fixtures definitions ( #15119 )
...
These mean "I don't want to specify this fixture/option"
instead of "I want the value of undefined", aligned with how TypeScript works.
We already do similar things in the config.
2022-06-27 11:31:41 -07:00
Dmitry Gozman
a93db3cf11
fix(reporter): line reporter should not swallow half-line stdout ( #15114 )
2022-06-27 11:25:42 -07:00
Pavel Feldman
1ce1246a0e
test: stress runners ( #15078 )
2022-06-23 15:55:12 -07:00
Dmitry Gozman
a46aaee6e8
fix(reporters): truncate long test titles from the start ( #15052 )
...
Most useful information is at the end - test name, current step, retry.
We truncate the repetitive project + suites at the start.
2022-06-22 17:03:54 -07:00
Pavel Feldman
9525bedc1f
feat(har): re-add routeFromHAR ( #15024 )
2022-06-21 22:12:37 -07:00
Dmitry Gozman
c02e165eb6
fix(test runner): toHaveScreenshot should not overwrite matching expectations ( #15028 )
...
Even in the `--update-snapshots` mode we should keep existing files if
they are matching under the threshold, to avoid needless churn.
2022-06-21 18:01:25 -07:00
Max Schmitt
5b5d24a7a5
chore: roll stable test-runner to 1.23.0-alpha-1655778679000 ( #15015 )
2022-06-21 18:53:38 +02:00
Dmitry Gozman
5397394653
feature(har): add testOptions.har ( #14991 )
...
Can now be used with `test.use({ har })`.
Also added more tests for latest har features.
2022-06-20 13:37:31 -07:00
Pavel Feldman
e8070ee1b2
chore: warn users on clashing test output and html reporter folders ( #14964 )
2022-06-18 15:47:26 -07:00
Pavel Feldman
030e7d211c
chore(har): allow replaying from zip har ( #14962 )
2022-06-17 16:11:22 -07:00
Ross Wollman
b9114f9cbc
fix(html-reporter): file-browser friendly extensions ( #14943 )
...
Fixes #14904 .
This is done to make looking at the raw contents of the report
friendlier when using a file browser. However, it should be noted, the
public API of the HTML Reporter makes no guarantees of its contents
structure/layout/naming-conventions.
2022-06-17 08:09:49 -07:00
Ross Wollman
3974d1b359
revert( #14797 ): fix(test runner): collect artifacts when calling browser.close() ( #14846 )
...
Revert "fix(test runner): collect artifacts when calling `browser.close()` (#14797 )"
This reverts commit c7a28ac7e9https://github.com/microsoft/playwright/runs/6838098316?check_suite_focus=true
* First Bad: https://github.com/microsoft/playwright/runs/6838104691?check_suite_focus=true
* Still bad on HEAD (88664c39c9https://github.com/microsoft/playwright/runs/6868333846?check_suite_focus=true
2022-06-13 17:20:59 -07:00
Pavel Feldman
88664c39c9
chore: remove esm error interception, it only confuses users ( #14829 )
2022-06-13 12:32:42 -07:00
Pavel Feldman
41529bb1a5
chore: respect sigint in global setup ( #14805 )
2022-06-12 13:06:00 -07:00
Pavel Feldman
380e787065
fix(ct): respect boolean shorthands ( #14798 )
2022-06-10 17:34:21 -07:00
Pavel Feldman
30f3d526eb
feat(server): consider 3XX, 400-403 responses as started server
2022-06-10 14:47:29 -07:00
Dmitry Gozman
c7a28ac7e9
fix(test runner): collect artifacts when calling browser.close() ( #14797 )
...
Previously, we only collected artifacts on context closure.
However, in serial mode it is possible to close the browser instead.
2022-06-10 14:11:34 -07:00
Sergio Freire
060cd9d97c
feat(junit reporter): link testcases to Xray test issues and provide additional metadata for Xray Test Management ( #11374 )
...
Co-authored-by: Sergio Freire <sergio.freire@xpand-it.com>
2022-06-10 10:31:48 +02:00
Sergio Freire
d193bd64c4
feat(junit reporter): add option to force usage of CDATA sections for content in XML elements ( #12744 )
...
Co-authored-by: Sergio Freire <sergio.freire@xpand-it.com>
2022-06-10 10:25:52 +02:00
Yury Semikhatsky
21d570f62e
test: update expectation after webkit roll ( #14718 )
2022-06-08 12:46:39 -07:00
Pavel Feldman
e3a590ed80
chore: update to react18, but use react17 mode ( #14645 )
2022-06-06 17:14:01 -07:00
Pavel Feldman
2d5572abd8
fix(ct): return locator to root for fragments ( #14639 )
...
fix(fragments): return locator to root for fragments
2022-06-04 14:07:06 -07:00
Pavel Feldman
c4b003a29a
chore: don't mask ESM error ( #14628 )
2022-06-03 15:28:53 -07:00
Dmitry Gozman
79d356f0cc
fix(test runner): when worker exits unexpectedly, fail a single test ( #14608 )
...
All remaining tests will continue in a new worker.
2022-06-02 21:13:47 -07:00
Ross Wollman
978854b859
chore: move multi-select assertion to toHaveValues ( #14595 )
...
Follow-up to e0a87e52d7
2022-06-02 16:01:34 -04:00
Dmitry Gozman
3a3aa023ad
chore: simplify context/page reuse in ct ( #14565 )
2022-06-02 12:16:07 -07:00
Andrey Lushnikov
8801f79742
Revert "fix(playwright-test): prohibit async functions passed to describe ( #14538 )" ( #14591 )
...
This reverts commit bd0fe50a0e
2022-06-02 10:31:10 -07:00
Andrey Lushnikov
bd0fe50a0e
fix(playwright-test): prohibit async functions passed to describe ( #14538 )
...
References #14533
2022-06-02 09:44:47 -07:00
Ross Wollman
e0a87e52d7
feat: support multi-select/combo box with toHaveValue ( #14555 )
2022-06-02 12:10:28 -04:00
Dmitry Gozman
d00efa0dfe
feat(expect): add ignoreCase option to toHaveText and toContainText ( #14534 )
2022-06-02 05:52:53 -07:00
Pavel Feldman
e912b6897d
fix(esm): respect source maps in esm-transpiled code ( #14561 )
2022-06-01 16:50:23 -07:00
Dmitry Gozman
dbcf039717
chore: remove experimental types ( #14560 )
2022-06-01 15:22:43 -07:00
Pavel Feldman
95672765bc
fix(ct): isolate component tests when recording video / trace ( #14531 )
2022-05-31 15:59:36 -07:00
Dmitry Gozman
a7500c18d6
feat(test runner): allow serial suites inside parallel ( #14530 )
...
Also works for `fullyParallel` mode.
2022-05-31 15:24:20 -07:00
Yury Semikhatsky
2bcdf68ef5
fix(runner): do not override trace files in afterAll ( #14529 )
2022-05-31 15:21:51 -07:00
Yury Semikhatsky
b164d82ba3
fix(runner): ignore .gitignore if testDir is explicitly configured ( #14430 )
...
If the tests are in an explicitly configured testDir (either at the global config level or per project) .gitignore filters are not applied.
Fixes #14381
2022-05-26 14:39:51 -07:00
Pavel Feldman
ea07ff3ae9
fix(jsx): allow passing variables into mount ( #14412 )
2022-05-25 13:59:45 -07:00
Pavel Feldman
9440f52b88
feat(ct): support cra w/ .js ( #14326 )
2022-05-24 19:43:28 -07:00
Pavel Feldman
d97c9bd5b6
test(ct): start covering components with tests ( #14369 )
2022-05-24 13:54:12 -07:00
Ross Wollman
9b225f2ad4
fix: expect.toHaveScreenshot.animations types ( #14387 )
...
This makes the docs/types match the code which has already been
released.
Relevant code to traverse up from:
- 3e084829c0/packages/playwright-core/src/server/screenshotter.ts (L89)3e084829c0/packages/playwright-core/src/server/screenshotter.ts (L118)Fixes #14385
2022-05-24 11:34:29 -07:00
Pavel Feldman
738d5e5b3e
chore: pass fixture defaults different from falsy ( #14237 )
2022-05-18 12:03:47 -07:00
Pavel Feldman
fe0afd6b5c
fix(toHaveProperty): serialize falsy arguments as well ( #14232 )
2022-05-17 14:44:12 -07:00
Pavel Feldman
56a5f2c506
fix(types): fix the toHaveScreenshot types ( #14174 )
...
Co-authored-by: Andrey Lushnikov <aslushnikov@gmail.com>
2022-05-16 07:53:46 -07:00
Dmitry Gozman
c3beb71b07
fix(test runner): do not run automatic fixtures for beforeAll hooks ( #14104 )
...
There are a few issues this covers:
- Some fixtures like `page` and `context` are not allowed in `beforeAll`
hooks, so using them in automatic fixture makes it throw.
- Running automatic fixture solely for `afterAll` is unexpected.
This currently happens when `afterAll` is run for cleanup after
fixture timeout/throw.
For built-in playwright fixture, we keep `'all-hooks-included'` auto mode.
Added a doc explaining the execution order.
2022-05-13 11:17:20 +01:00
Max Schmitt
cdd589da91
chore: roll stable test-runner to 1.22.0-alpha-may-11-2022 ( #14102 )
2022-05-11 16:06:52 +01:00
Dmitry Gozman
305afcdacf
fix(test runner): fix duplicate titles error when multiple issues are present ( #14090 )
2022-05-11 11:53:16 +01:00
Pavel Feldman
d83de4f567
chore: remove stale plugins test
2022-05-09 12:54:06 -07:00
Dmitry Gozman
e8fb5a6337
fix(test runner): ensure that hooks run before fixtures teardown after timeout ( #14035 )
...
We had common cleanup exiting early after timeout, because we did not
reset the time slot.
2022-05-09 20:38:20 +01:00
Andrey Lushnikov
5aa82dc5e4
chore: un-experiment expect(pageOrLocator).toHaveScreenshot() ( #14033 )
...
This patch un-experiments `expect(pageOrLocator).toHaveScreenshot()`
method.
Fixes #13526
2022-05-09 08:34:03 -07:00
Andrey Lushnikov
98945a81a8
feat: move toHaveScreenshot to use old snapshot paths by default ( #14006 )
...
Note: all toHaveScreenshot tests still use `__screenshots__` directory
for their expectations. One more test was added to make sure that
by default, `toHaveScreenshot` uses old snapshots.
2022-05-09 07:34:53 -07:00
Andrey Lushnikov
5a5bb36d28
chore: nuke "fonts" screenshot option ( #14004 )
...
It was never released since it wasn't working as expected on WebKit WPE.
Fixes #12839
2022-05-06 17:54:17 -07:00
