Commit Graph

290 Commits

Author SHA1 Message Date
Pavel Feldman
c9cc8478b3
chore: minor trace viewer UI tweaks (#20937) 2023-02-16 07:59:21 -08:00
Dmitry Gozman
91da67fab1
test: remove magic headers in ttest (#20867)
Instead, explicitly import from '@playwright/test'.
2023-02-14 19:20:56 -08:00
Andrey Lushnikov
fdcd7b549d
chore: mark comparator option back as experimental (#20816)
This reverts commit 303c5998f8.

Reason for revert: I tried enabling `ssim-cie94` by default on
ionic-framework test suite, and it proves to be overly strict for their
usecase.
2023-02-13 11:11:44 -08:00
Dmitry Gozman
de69b766d9
test: code health in ttest (#20837)
- Remove duplicate tests.
- Remove unused test helpers.
- Print full watch output on failures.
- Unflake some tests.
2023-02-10 20:26:19 -08:00
Andrey Lushnikov
bcb2d67c5d
chore: remove experimental dockerfile.remote (#20790)
We didn't find a compelling-enough use case to release this.
2023-02-09 12:18:02 -08:00
Pavel Feldman
b247bfe153
test(watch): start adding tests (#20764) 2023-02-09 08:31:02 -08:00
Dmitry Gozman
bc74383480
test: remove service2 mode and experimental-grid (#20730) 2023-02-07 14:19:37 -08:00
Andrey Lushnikov
b67cef2c4d
feat: introduce Dockerfile.remote image (#20691)
When this image is launched, it exposes a single endpoint
that can be used to connect to and to launch browsers.
2023-02-07 10:50:44 -08:00
Andrey Lushnikov
303c5998f8
feat: release "comparator" option from experiment (#20720)
The option defines a comparator to be used to compare images.
Possible values are `"pixelmatch"` and `"ssim-cie94"`.

Note: This reverts commit 8167f8bf54.
2023-02-07 08:51:48 -08:00
Andrey Lushnikov
0a752f3fd6
chore(docker): remove experimental docker integration (#20676)
This removes everything related to docker integration experiments that
we conducted over the last 6 months.

I'll send a follow-up with an alternative suggestion that was demo'ed on
a team meeting in the end of December.
2023-02-06 10:50:45 -08:00
Playwright Service
f56d29709e
feat(firefox-beta): roll to r1377 (#20516)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Andrey Lushnikov <aslushnikov@gmail.com>
2023-02-02 07:14:18 -08:00
Etienne
252b489a7f
feat(trace-viewer): allow host and port to be specified (#20258) 2023-01-27 14:20:25 -08:00
Pavel Feldman
d1c161ce99
chore: merge the util exports (#20110) 2023-01-13 13:50:38 -08:00
Dmitry Gozman
711a1aadbf
chore: update tracing tests helper (#19848)
To be reused in future tests.

Signed-off-by: Dmitry Gozman <dgozman@gmail.com>
Co-authored-by: Yury Semikhatsky <yurys@chromium.org>
2023-01-03 17:30:36 -08:00
Andrey Lushnikov
caec93ef42
test: enable ssim-cie94 image comparison algorithm for our tests (#19313) 2022-12-29 23:48:28 -08:00
Dmitry Gozman
83418aa8ab
fix(socks): cleanup event listeners upon disconnect (#19671)
References #19661.
2022-12-22 17:15:51 -08:00
Dmitry Gozman
256e9fd443
feat(connect): allow exposing local network to the remote browser (experimental) (#19372)
`connectOptions: { _exposeNetwork: '*' | 'localhost' }`

References #19287.
2022-12-09 11:16:29 -08:00
Dmitry Gozman
622c1a8da6
tests: run browserType.connect tests against launchServer and run-server (#19340) 2022-12-07 14:36:32 -08:00
Dmitry Gozman
81c8620bfe
chore: upgrade internal types to support TypeScript 4.9.3 (#19280)
Fixes https://github.com/microsoft/TypeScript/issues/51619

Co-authored-by: Max Schmitt <max@schmitt.mx>
2022-12-05 17:22:25 -08:00
Max Schmitt
b84a7a9e57
test: fix WebKit socks proxy tests (#19035)
Fixes https://github.com/microsoft/playwright/issues/18301
2022-11-23 13:03:38 -10:00
Pavel Feldman
228f78c89d
chore: render browser window in trace (#18870) 2022-11-22 08:41:52 -08:00
Andrey Lushnikov
f3a99fdd69
chore: address API review comments for the snapshotPathTemplate (#18716)
This patch:
- updates documentation to lead users from `TestConfig.snapshotDir` and
  `testInfo.snapshotSuffix` to `TestConfig.snapshotPathTemplate` as a
  better and more flexible alternative.
- drops `{snapshotSuffix}` from documentation
- stops using `snapshotSuffix = ''` in our own tests and switches us
  to the `snapshotPathTemplate`.
- adds `{testName}` token.
2022-11-10 17:23:57 -08:00
Pavel Feldman
ca2e7ef199
chore: report paused signal to the debug controller clients (#18701) 2022-11-10 12:15:29 -08:00
Pavel Feldman
0355d8618f
chore: use provided value for the generated test id (#18631) 2022-11-08 12:04:43 -08:00
Dmitry Gozman
9172a2ca5a
test: use strict png comparator in tests (#18622) 2022-11-07 13:32:46 -08:00
Andrey Lushnikov
8538f61a72
feat(containers): implement global network tethering for playwright server (#17719)
This patch implements a new mode of network tethering for Playwright
server & its clients.
With this patch:
- playwright server could be launched with the
`--browser-proxy-mode=tether` flag to engage in the new mode
- a new type of client, "Network Tethering Client" can connect to the
server to provide network traffic to the browsers
- all clients that connect to the server with the `x-playwright-proxy:
*` header will get traffic from the "Network Tethering Client"

This patch also adds an environment variable
`PW_OWNED_BY_TETHER_CLIENT`. With this env, playwright server will
auto-close when the network tethering client disconnects. It will also
auto-close if the network client does not connect to the server in the
first 10 seconds of the server existence. This way we can ensure that
`npx playwright docker start` blocks terminal & controls the lifetime of
the started container.
2022-11-03 13:47:51 -07:00
Pavel Feldman
d819f97f40
test: clean up connect options set for debug controller (#18317) 2022-10-25 09:03:04 -07:00
Pavel Feldman
d3948d1308
chore: enable debug controller testing (#18270) 2022-10-24 16:19:58 -07:00
Dmitry Gozman
3409a37f77
feat: allow opening multiple html reporters and trace viewers (#17636)
This makes `HttpServer` accept `preferredPort` option that will first
try to listen on that port, and if that port is already in use, listen
on some available port instead.

Fixes #17201.
2022-09-27 12:45:42 -07:00
Pavel Feldman
df143031e7
chore: move protocol and trace types into the top-level packages (#17486) 2022-09-20 18:41:51 -07:00
Andrey Lushnikov
a12112c24d
devops(docker): add docker integration smoke tests (#17267) 2022-09-14 15:05:18 -07:00
Max Schmitt
904801a5eb
test: add initial webview2 tests (#16827) 2022-09-07 20:09:22 +02:00
Pavel Feldman
8d25f2ef59
fix(selectors): allow custom engines in out-of-process (#17139) 2022-09-06 14:15:53 -07:00
Max Schmitt
7e2aec7454
chore: align more ESLint rules with VSCode formatting (#16647) 2022-08-18 20:12:33 +02:00
Dmitry Gozman
7acbd052e4
test: migrate some expect() tests to be more readable (#16394)
This moves some expect() matchers tests from test runner tests to page tests,
because these are implemented through a library call anyway.

Makes tests more readbable, faster and easier to test specific details.
2022-08-10 15:10:25 -07:00
Andrey Lushnikov
4cbb95e06d
test: fix firefox-beta tests (#16278) 2022-08-10 11:00:59 +02:00
Andrey Lushnikov
fb76d62a2b
feat(firefox): roll Firefox stable to 103 (#16224) 2022-08-03 16:55:19 -07:00
Pavel Feldman
5a79054544
feat(innerloop): allow reusing browsers over the remote connection (#16065) 2022-07-31 14:31:17 -07:00
Andrey Lushnikov
3b269d0ed7
browser(ff-beta): roll Firefox-Beta to 103.0b5 (#15394) 2022-07-05 15:47:35 -07:00
Dmitry Gozman
71dcad3b2e
test: fix some flaky failures (#15314)
- Never use open shadow root for highlight. This messes up
  our selectors that accidentally match internal preview elements.
- Remove failing electron test that we do not care about.
- Skip `channels.spec.ts` in non-default mode.
2022-07-01 13:57:33 -07:00
Ross Wollman
6cb3236acd
feat(chromium): Service Worker Network Instrumentation and Inspection (#14716)
Adds Chromium support for Service Worker Networking (interception/routing, Request/Response events, and HAR).

Resolves #1090.
Depends on #14714 and #14714.
Supercedes #14321.
Follow up #14711.

Landed upstream patches:
- https://chromium-review.googlesource.com/c/chromium/src/+/3510917
- https://chromium-review.googlesource.com/c/chromium/src/+/3526571
- https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3566669
- https://chromium-review.googlesource.com/c/chromium/src/+/3544685
- https://chromium-review.googlesource.com/c/devtools/devtools-frontend/+/3610924
- https://chromium-review.googlesource.com/c/chromium/src/+/3689949
2022-07-01 12:49:43 -07:00
Dmitry Gozman
82032be368
chore(server): validate/convert protocol both ways (#14811)
Previously, we only validated/converted on the way to the server,
but not from the server.

Validating both ways catches issues earlier, and allows us to
perform automatic conversions, for example only converting
buffers to base64 when sending over wire.
2022-07-01 09:58:07 -07:00
Pavel Feldman
e8070ee1b2
chore: warn users on clashing test output and html reporter folders (#14964) 2022-06-18 15:47:26 -07:00
Pavel Feldman
030e7d211c
chore(har): allow replaying from zip har (#14962) 2022-06-17 16:11:22 -07:00
Pavel Feldman
245c33a5d4
feat(har): allow storing content as separate files (#14934) 2022-06-16 16:33:32 -07:00
Dmitry Gozman
dbcf039717
chore: remove experimental types (#14560) 2022-06-01 15:22:43 -07:00
Yury Semikhatsky
c1aecc7d50
docs: maxDiffPixels description typo (#14456) 2022-05-29 09:46:06 -07:00
Pavel Feldman
f724fa570f
docs: make ts snippets esm-friendly (#14451) 2022-05-27 12:36:59 -07:00
Ross Wollman
9b225f2ad4
fix: expect.toHaveScreenshot.animations types (#14387)
This makes the docs/types match the code which has already been
released.

Relevant code to traverse up from:
- 3e084829c0/packages/playwright-core/src/server/screenshotter.ts (L89)
- 3e084829c0/packages/playwright-core/src/server/screenshotter.ts (L118)

Fixes #14385
2022-05-24 11:34:29 -07:00
Andrey Lushnikov
3fd435d1d3
docs: fix docs for toHaveScreenshot (#14280) 2022-05-24 10:54:32 -07:00
Pavel Feldman
71a55c74da
fix(leak): do not retain test fixtures in worker fixtures (#14363) 2022-05-23 16:54:56 -07:00
Max Schmitt
9256de5f48
chore: fix wrong toHaveScreenshot defaults in docs (#14200) 2022-05-16 22:26:23 +03:00
Pavel Feldman
56a5f2c506
fix(types): fix the toHaveScreenshot types (#14174)
Co-authored-by: Andrey Lushnikov <aslushnikov@gmail.com>
2022-05-16 07:53:46 -07:00
Dmitry Gozman
0e2855348c
feat(locators): remove layout locators (#14129) 2022-05-12 18:50:19 +01:00
Max Schmitt
2844623109
docs: add route class -> networking guide reference (#14092) 2022-05-11 16:07:00 +01:00
Pavel Feldman
e9378ba5fc
chore: hide plugins again (#14038) 2022-05-09 10:06:13 -07:00
Dmitry Gozman
6931d89eaf
feat(locators): rename locator.that to locator.filter (#14025) 2022-05-08 21:59:40 +01:00
Dmitry Gozman
c3cf7ee4a7
feat(layout locators): remove maxDistance option (#14013) 2022-05-08 07:18:16 +01:00
Andrey Lushnikov
5a5bb36d28
chore: nuke "fonts" screenshot option (#14004)
It was never released since it wasn't working as expected on WebKit WPE.

Fixes #12839
2022-05-06 17:54:17 -07:00
Pavel Feldman
1bbefce9af
Revert "chore: use plugins for component testing again (#13977)" (#13998)
This reverts commit a2f9f15e3e.
2022-05-06 12:02:07 -07:00
Pavel Feldman
a2f9f15e3e
chore: use plugins for component testing again (#13977) 2022-05-05 14:26:56 -07:00
Pavel Feldman
058f32caff
chore: fixtures-via-plugin implementation (#13950) 2022-05-05 10:14:00 -07:00
Ross Wollman
95f7acf1e4
chore: widen Metadata type (#13942) 2022-05-04 13:16:11 -07:00
Pavel Feldman
29fd1d86df
chore: hide plugins for now (#13908) 2022-05-03 14:25:56 -07:00
Dmitry Gozman
54dd6d01e5
feat(locator): layout options (leftOf, rightOf, above, below, near) (#13821)
This also includes corresponding selector engines `left-of` and others,
modeled after existing `has` selector engine.
2022-05-03 10:33:33 +01:00
Ross Wollman
3b3cad7d69
feat: rewrite gitCommitInfo plugin, drop GlobalInfo & attachments (#13837) 2022-05-02 16:28:14 -07:00
Dmitry Gozman
dc3f2d26c4
docs: add details to the element screenshot methods (#13847) 2022-04-29 17:36:09 +01:00
Ross Wollman
45972a0b6a
feat: add name for plugins (#13826)
Useful for debugging and eventually for reporters to give status updates on which phase of which plugin they are currently running.
2022-04-28 11:43:39 -07:00
Pavel Feldman
e756528ad2
feat(plugins): expose suite to plugin setup (#13807) 2022-04-28 08:16:17 -07:00
Dmitry Gozman
3aba94dbef
chore: remove unnecessary types from overrides-test (#13794) 2022-04-27 16:14:37 +01:00
Max Schmitt
a0c24c01c8
chore: fix docs links for .NET roll (#13787) 2022-04-27 16:06:30 +02:00
Pavel Feldman
bc6f8e1f20
feat(poll): expose custom poll interval (#13776) 2022-04-26 21:32:38 -07:00
Dmitry Gozman
39cfa2b6e3
test: switch temp dir tests to use userDataDir (#13745) 2022-04-26 09:32:58 +01:00
Dmitry Gozman
97750ccf9a
feat: locator.that (#13731)
Filters existing locator by options, currently `has` and `hasText`.
2022-04-25 20:06:18 +01:00
Pavel Feldman
5e51c17d41
chore: use plugin story for components (#13717) 2022-04-25 10:40:58 -07:00
Dmitry Gozman
8e5854033a
fix: types generation for attach and GlobalInfo (#13727) 2022-04-25 13:56:37 +01:00
Dmitry Gozman
d2bdb1d7cf
docs: explain label retargeting in selectors doc and some input actions (#13725) 2022-04-25 13:55:41 +01:00
Ross Wollman
3eac9c694c
fix(test-runner): mark attach as async (#13687)
Also brings back generation of experimental types as part of lint.
2022-04-22 09:27:02 -07:00
Max Schmitt
ca4cfca8ad
chore: make tests pass on Node.js 17+ (#13678) 2022-04-22 13:42:52 +02:00
Yury Semikhatsky
0a401b2d86
docs: clarify semantics of redirected response (#13636) 2022-04-19 09:36:38 -07:00
Yury Semikhatsky
ed0dcdabc9
docs: clarify toBeDisabled behavior (#13616) 2022-04-18 17:06:01 -07:00
Pavel Feldman
eafb92f6a3
chore: embed expect types (#13609) 2022-04-18 10:54:40 -07:00
Dmitry Gozman
02aa4fe617
chore: generate types for TestProject/TestConfig (#13574) 2022-04-15 10:27:38 -07:00
Dmitry Gozman
189a5fa0f8
chore: freez FullProject public type (#13570)
This is similar to FullConfig.
Properties `expect` and `fullyParallel` are not exposed.
2022-04-14 16:58:01 -07:00
Yury Semikhatsky
aee6ba299a
chore: remove GridClient, run only page tests in service mode (#13566) 2022-04-14 15:30:04 -07:00
Wojciech Jureczka
7ba527c65f
chore: expose JSON Reporter types (#13241)
Resolves #13208
2022-04-14 13:55:29 -07:00
Dmitry Gozman
c86c2e8762
feat: restore toHaveScreenshot as experimental feature (#13549)
- Restore docs.
- Make `TestConfig.expect` generated.
- Allow experimental properties with "e" marker: `- foo e<float>`.
2022-04-14 13:22:42 -07:00
Dmitry Gozman
20dcc45afa
feat: support experimental doc entries (#13446)
feat: support experimental doc entries

- Params/options/members are marked as experimental in the docs.
- `experimental.d.ts` is generated that contains all types and
  includes experimental features.
- `experimental.d.ts` is references in our tests so that we
  can test experimental features.
- `fonts` option is restored as experimental.
2022-04-13 16:13:30 -07:00
Dmitry Gozman
2a97ad2487
test: check that we remove temp directories on process exit (#13494) 2022-04-12 12:28:33 -07:00
Ross Wollman
1af32e400f
feat(test-runner): introduce GlobalInfo (#13083) 2022-04-08 13:22:14 -07:00
Pavel Feldman
e79b90f454
chore: use utils via index export (5) (#13413) 2022-04-07 20:18:22 -07:00
Dmitry Gozman
155bb7fcae
fix: wait for cleanup on double SIGINT (#13411)
This is a speculative fix to leftover tmp directories.
When users issues SIGINT twice, we enter `gracefullyClose()`
twice, and shortcut the second time. It turns out, we do
not wait for directories removal.

Note: it is unknown how often we reach this codepath in practice.
2022-04-07 19:20:54 -07:00
Pavel Feldman
40d5e3a3c9
chore: use utils via index export (3) (#13403) 2022-04-07 13:55:44 -07:00
Pavel Feldman
bde7bf4ea9
chore: move registry & dispatchers to under server (#13370) 2022-04-06 22:21:27 -07:00
Pavel Feldman
5ae2017a5b
chore: always import type (#13365) 2022-04-06 14:57:14 -07:00
Pavel Feldman
6ca58e18cb
fix(electron): better support for custom schemas (#13329) 2022-04-05 16:10:12 -07:00
Ross Wollman
16efbdef98
chore: remove/rename FullConfig._attachments (#13233)
This was originally introduced in #12734.

It will be replaced with GlobalInfo (#13083), but not before the 1.21
release.
2022-04-01 12:36:05 -07:00
Pavel Feldman
5e17ed137b
fix(html-report): only invoke git once (#13165) 2022-03-29 15:49:58 -07:00
Pavel Feldman
a58707b8a6
chore: group tests under tests/ (2) (#13082) 2022-03-25 23:09:02 -07:00
Ross Wollman
541fb39a51
feat(html-reporter): add report context header (#12734)
Resolves #11318.

* Adds `TestConfig.attachments` public API. (We opted to not implement an analog to the async `TestInfo.attach(…)` API.)
* Adds `TestConfig.attachments` to common reporters.
* Dogfoods some git and CI-info inference to generate useful atttachments
* Updates HTML Reporter to include a side bar to present a pre-defined set of attachments (a.k.a git/commit context sidebar)

Here's what it looks like:

<img width="1738" alt="Screen Shot 2022-03-21 at 3 23 28 PM" src="https://user-images.githubusercontent.com/11915034/159373291-8b937d30-fba3-472a-853a-766018f6b3e2.png">

See `tests/playwright-test/reporter-html.spec.ts` for an example of usage (for dogfood-ing only). In the future, if this becomes user-facing, there the Global Setup bit would likely become unnecessary (as would interaction with attachments array); there would likely just be a nice top-level config and/or CLI flag to enable collecting of info.
2022-03-22 16:28:04 -07:00