Commit Graph

2897 Commits

Author SHA1 Message Date
Pavel Feldman
555a8d0d10
fix(testrunner): include fixture teardown into timeout, add global timeout (#3685) 2020-08-28 19:27:49 -07:00
Pavel Feldman
c47af2d4f3
fix(testrunner): report unhandled rejection ones, allow retry (#3686) 2020-08-28 18:25:09 -07:00
Joel Einbinder
abb50a79bd
browser(firefox): fix request frame attribution (#3657)
Firefox will sometimes send multiple requests with the same http channel id. When a frame is loaded, the favicon is requested in the parent frame, but with the same channel id. This can cause the document request to report the wrong frame, causing the test 'should capture iframe navigation request' to fail. It fails consistently on my computer.

This patch adds the content policy type into the http channelId to better distinguish requests. Maybe there is something better we can do? It looks like we use channelId has request ids, so there might be more bugs with these favicon requests in playwright?
2020-08-28 17:55:05 -07:00
Joel Einbinder
6a93cb9749
fix(types): don't show types that we don't export (#3185)
Today we have a bunch of types used by the d.ts file that are not exported. We don't want to export them because it would greatly increase our semver API surface area, so this patch inlines them. Now users will not see names of types they can't import.
2020-08-28 17:53:03 -07:00
Joel Einbinder
4e5007ae1a
fix(rpc): nice error stacks when running tests (#3507) 2020-08-28 16:47:06 -07:00
Max Schmitt
c2cd963407
chore: added envinfo to the bug issue template (#2237)
* chore: added envinfo to the bug issue template

* Update bug.md

Co-authored-by: Arjun Attam <arjunattam@gmail.com>
2020-08-28 16:24:42 -07:00
Pavel Feldman
657cc9b630
feat(test): use metafunc in describes (#3682) 2020-08-28 15:45:09 -07:00
Arjun Attam
fb6d1ad591
docs(docker): add link to mcr status ui (#3679) 2020-08-28 14:30:51 -07:00
Andrey Lushnikov
45e178f8b5
fix: support IP:PORT short notation to specify proxy server (#3568)
Short notation implies `http://` scheme.

Fixes #3233
2020-08-28 14:17:16 -07:00
Pavel Feldman
97e4561ee1
feat(test): introduce metafunc for skip (#3676) 2020-08-28 13:53:47 -07:00
Dmitry Gozman
e5ff283a40
fix(trace): only enable on separate tracing bots (#3677) 2020-08-28 13:50:06 -07:00
Max Schmitt
63a0e0c1d0
chore: bump dev dependencies (#3659) 2020-08-28 13:43:57 -07:00
Dmitry Gozman
7b1fac90bb
test: mark all crash tests as flaky on firefox win (#3675)
All crash tests do not actually crash sometimes on firefox win.
2020-08-28 12:05:11 -07:00
Dmitry Gozman
3d6051adb7
test: mark "should work for webgl" as fixme on webkit linux (#3674)
It sometimes fails all 3 retries.
2020-08-28 12:05:01 -07:00
Darío Kondratiuk
90408aa4f4
test: Remove "request interception" from oopif tests (#3671)
We don't have request interceptions :)
2020-08-28 11:37:16 -07:00
Dmitry Gozman
b34d9aba25
feat(trace): experimental traces for our tests (#3567)
This introduces basic tracing enabled in our tests.

What is captured:
- network resources;
- snapshots at the start of most actions;
- snapshot after the test failure.

How this integrates with test runner:
- context fixture calls private method context._initSnapshotter() and uses Tracer to trace all events;
- all tests share a single test-results/trace-storage directory to store blobs;
- each test has its own trace file.
- npm run show-trace opens a bare-minimum trace viewer that renders snapshots.
2020-08-28 10:51:55 -07:00
Carlos Alberto Lopez Perez
19f21b1bde
browser(webkit): use webkit generate-bundle tool to generate the bundles (#3563)
* This changes the archive.sh script to use the webkit generate-bundle
 tool to generate the MiniBrowser bundles instead of using our custom
 logic.

 * pw_run.sh is simplified because the new bundles include a wrapper
 script that sets the needed LD_LIBRARY_PATH environment variables.
 pw_run.sh now calls this wrapper script (for GTK or WPE) instead of
 calling the MiniBrowser binary directly when running from a bundle.

Co-authored-by: Andrey Lushnikov <aslushnikov@gmail.com>
2020-08-28 10:48:57 -07:00
Dmitry Gozman
4386cd4e7f
test: mark "headless should be able to read cookies written by headful" as flaky on firefox (#3673) 2020-08-28 10:38:13 -07:00
Dmitry Gozman
744af78dc2
feat(rpc): simplify browser name detection on the client side (#3670) 2020-08-28 10:23:02 -07:00
Pavel Feldman
eec92630df
test: make some tests as flaky (#3672) 2020-08-28 10:03:52 -07:00
Arjun Attam
2edd6f2824
docs: introduce why-playwright.md (#3666) 2020-08-28 09:38:12 -07:00
Joel Einbinder
e2057fb8bb
chore(test): run eslint on tests (#3638) 2020-08-28 04:20:29 -07:00
Pavel Feldman
6ffdd4dfa1
feat(testrunner): allow unexpected passes (#3665) 2020-08-28 00:32:00 -07:00
Dmitry Gozman
5c0f93301d
test: always setUnderTest in index.js, rename to setDevMode (#3662)
Root index.js is only used for local development, so
assuming dev mode there is fine. This way we do not have
to worry about calling setUnderTest early enough.
2020-08-27 21:08:33 -07:00
Arjun Attam
7444de4b73
docs: update navigation and loading page (#3655)
* docs: update navigation and loading doc

* dgozman comments

* doclint
2020-08-27 17:47:23 -07:00
Pavel Feldman
cfbec44285
feat(testrunner): allow annotating tests as flaky (#3663) 2020-08-27 16:34:34 -07:00
Pavel Feldman
6a0f587fae
fix(testrunner): report suite-level errors (#3661) 2020-08-27 11:41:53 -07:00
Pavel Feldman
2b7d79d7fa
fix(testrunner): fix windows bots (#3660) 2020-08-27 10:14:27 -07:00
Pavel Feldman
efd45f8b51
fix(testrunner): report tests as passed in the trial-run mode (#3654) 2020-08-26 22:29:23 -07:00
Pavel Feldman
06154060a2
feat(testrunner): support repeat-each (#3652) 2020-08-26 20:18:44 -07:00
Andrey Lushnikov
3ea3cf0373
devops: add yarn and git in the docker container (#3651)
This will help consuming docker image as an env for test execution.

Fixes #3529
2020-08-26 20:16:24 -07:00
Pavel Feldman
15ec87dbcc
feat(testrunner): support --retries, flaky tests (#3649) 2020-08-26 16:32:47 -07:00
Max Schmitt
254238cdce
enh: bake browser revisions and api into driver (#3514) 2020-08-26 16:01:42 -07:00
Andrey Lushnikov
c96ea4b6de
chore: remove docker image size computation scripts (#3650)
The scripts are broken atm. They're re-implemented in the
devops website: https://devops.aslushnikov.com for now.
2020-08-26 16:00:34 -07:00
Yury Semikhatsky
5f9407a0d1
feat(webkit): bump to 1334 (#3643) 2020-08-26 15:18:39 -07:00
Joel Einbinder
1a5f22d327
fix(test): import playwright types with import type (#3647)
* fix(test): import playwright types with import type

* add a check
2020-08-26 14:16:35 -07:00
Pavel Feldman
a20bb949ea
chore(testrunner): introduce test result, reuse it in ipc (#3644) 2020-08-26 14:14:23 -07:00
Dmitry Gozman
9e2e87060a
test: switch browserType.connect tests to use remoteServer (#3646)
This way we test the real scenario where server is not launched
in the same node process.

Additionally, move all connect-related tests to a single file and
clean them up.
2020-08-26 13:33:37 -07:00
Dmitry Gozman
8d7ec3aca3
fix(downloads): make path/saveAs work when connected remotely (#3634)
- saveAs uses a stream internally and pipes it to the local file;
- path throws an error when called on a remote browser.
2020-08-26 12:46:30 -07:00
Pavel Feldman
a87614a266
feat(test): shrink api to run only, rename pending to skipped (#3636) 2020-08-26 09:38:19 -07:00
Darío Kondratiuk
9b50a6d259
test: Fix Chromium JSCoverage reportAnonumousScripts test (#3641) 2020-08-26 08:59:07 -07:00
Joel Einbinder
4249a11d7e
chore(types): upgrade to TypeScript 4.0.2 (#3637) 2020-08-26 08:57:03 -07:00
Yury Semikhatsky
80cf7e9fac
browser(webkit): do not crash when opening web inspector (#3631) 2020-08-26 08:55:18 -07:00
Joel Einbinder
f9eeb2984a
fix playwright being imported before toImpl could be registered (#3632) 2020-08-26 02:52:23 -07:00
Joel Einbinder
a6b9922eb8
fix(testrunner): console.log in color (#3633) 2020-08-25 21:06:36 -07:00
Blake Williams
5f86253ad4
docs: add more detail to waitForNavigation API method (#3635) 2020-08-25 21:05:27 -07:00
Dmitry Gozman
25381cfa43
test: add some tests for remote connect (#3614)
Our current tests excercise the same-process launchServer/connect
combination. This change adds tests for remote server and immediately
exposes a bug with selector engines.
2020-08-25 19:09:30 -07:00
Yury Semikhatsky
db0fa07330
fix(screencast): replace ScreencastStopped with async path() (#3626) 2020-08-25 17:18:28 -07:00
Yury Semikhatsky
1a37f8ba7a
browser(webkit): remove browserContextId from some events (#3628) 2020-08-25 17:15:21 -07:00
Yury Semikhatsky
22b9246692
chore(context): unify browser context id handling (#3629) 2020-08-25 16:33:16 -07:00