Commit Graph

1220 Commits

Author SHA1 Message Date
Dmitry Gozman
b34d9aba25
feat(trace): experimental traces for our tests (#3567)
This introduces basic tracing enabled in our tests.

What is captured:
- network resources;
- snapshots at the start of most actions;
- snapshot after the test failure.

How this integrates with test runner:
- context fixture calls private method context._initSnapshotter() and uses Tracer to trace all events;
- all tests share a single test-results/trace-storage directory to store blobs;
- each test has its own trace file.
- npm run show-trace opens a bare-minimum trace viewer that renders snapshots.
2020-08-28 10:51:55 -07:00
Dmitry Gozman
4386cd4e7f
test: mark "headless should be able to read cookies written by headful" as flaky on firefox (#3673) 2020-08-28 10:38:13 -07:00
Pavel Feldman
eec92630df
test: make some tests as flaky (#3672) 2020-08-28 10:03:52 -07:00
Joel Einbinder
e2057fb8bb
chore(test): run eslint on tests (#3638) 2020-08-28 04:20:29 -07:00
Pavel Feldman
6ffdd4dfa1
feat(testrunner): allow unexpected passes (#3665) 2020-08-28 00:32:00 -07:00
Dmitry Gozman
5c0f93301d
test: always setUnderTest in index.js, rename to setDevMode (#3662)
Root index.js is only used for local development, so
assuming dev mode there is fine. This way we do not have
to worry about calling setUnderTest early enough.
2020-08-27 21:08:33 -07:00
Joel Einbinder
1a5f22d327
fix(test): import playwright types with import type (#3647)
* fix(test): import playwright types with import type

* add a check
2020-08-26 14:16:35 -07:00
Pavel Feldman
a20bb949ea
chore(testrunner): introduce test result, reuse it in ipc (#3644) 2020-08-26 14:14:23 -07:00
Dmitry Gozman
9e2e87060a
test: switch browserType.connect tests to use remoteServer (#3646)
This way we test the real scenario where server is not launched
in the same node process.

Additionally, move all connect-related tests to a single file and
clean them up.
2020-08-26 13:33:37 -07:00
Dmitry Gozman
8d7ec3aca3
fix(downloads): make path/saveAs work when connected remotely (#3634)
- saveAs uses a stream internally and pipes it to the local file;
- path throws an error when called on a remote browser.
2020-08-26 12:46:30 -07:00
Darío Kondratiuk
9b50a6d259
test: Fix Chromium JSCoverage reportAnonumousScripts test (#3641) 2020-08-26 08:59:07 -07:00
Joel Einbinder
f9eeb2984a
fix playwright being imported before toImpl could be registered (#3632) 2020-08-26 02:52:23 -07:00
Dmitry Gozman
25381cfa43
test: add some tests for remote connect (#3614)
Our current tests excercise the same-process launchServer/connect
combination. This change adds tests for remote server and immediately
exposes a bug with selector engines.
2020-08-25 19:09:30 -07:00
Yury Semikhatsky
a38564b7ca
fix(screencast): replace ScreencastStopped event with async path() (#3612) 2020-08-25 13:07:32 -07:00
Pavel Feldman
e215461898
chore: split tests for faster execution (#3613) 2020-08-24 20:26:06 -07:00
Dmitry Gozman
3b2f14fcee
test: fix wire tests (#3609) 2020-08-24 17:32:55 -07:00
Yury Semikhatsky
b9d6324d14
feat(screencast): fire start evet for popups (#3600) 2020-08-24 17:23:54 -07:00
Dmitry Gozman
a099e941d6
chore: move last rpc files to their place (#3604)
Also update check-deps to be more user-friendly.
2020-08-24 14:48:03 -07:00
Dmitry Gozman
cd220daa83
chore: move src files to server (#3593) 2020-08-24 06:51:51 -07:00
Pavel Feldman
4025f9f1ef
feat(testrunner): expose test and runner config to fixtures (#3580) 2020-08-22 16:44:56 -07:00
Dmitry Gozman
9fca63f8ec
chore: move src/rpc/client to src/client (#3581) 2020-08-22 15:13:51 -07:00
Darío Kondratiuk
72f11fdb0b
test: Remove duplicated expect (#3579) 2020-08-22 10:23:54 -07:00
Dmitry Gozman
655013d025
chore: move shared utilities to src/utils (#3575) 2020-08-22 07:07:13 -07:00
Pavel Feldman
b909924a61
test: remove mocha dependency (#3576) 2020-08-22 00:05:24 -07:00
Dmitry Gozman
6fe1cd9812
chore: move protocol files to src/protocol (#3571) 2020-08-21 18:46:11 -07:00
Pavel Feldman
398bd477ac
test: translate tests into ts, extract mocha (#3565) 2020-08-21 17:14:11 -07:00
Dmitry Gozman
57e8617474
chore: refactor impl-side events to be per-class (#3569) 2020-08-21 16:26:33 -07:00
Joel Einbinder
8ae3c4be06
feat(testrunner): delete types.d.ts (#3551) 2020-08-21 09:53:02 -07:00
Yury Semikhatsky
1f0e9db0a6
feat(firefox): support context-level screencast api (#3555) 2020-08-20 19:49:30 -07:00
Yury Semikhatsky
7a492831a1
fix(test): fix the popup test on Windows (#3558) 2020-08-20 19:48:56 -07:00
Pavel Feldman
83f399534c
test: take a screenshot upon failure example (#3556) 2020-08-20 17:45:10 -07:00
Yury Semikhatsky
071931ebb1
feat(firefox): bump to 1166 (#3557) 2020-08-20 16:51:17 -07:00
Dmitry Gozman
e2bb6a07cd
fix(click): allow clicking 1x1 sized elements (#3538)
One by one seems like a resonable minimum size for clicking.

It is not surprising to see a 1x1 native accessible control
that is covered by a custom control that handles input instead.
2020-08-20 16:49:19 -07:00
Joel Einbinder
012f9425bf
chore(test-runner): move into its own folder and typescript project (#3548) 2020-08-20 16:04:27 -07:00
Dmitry Gozman
4c5635434a
fix(permissions): browserContext.grantPermissions to respect the origin (#3542)
Due to wrong type usage, we ignored the origin while granting permissions.
Switching to generated types revealed this issue. We should follow up
with switching all dispatchers to the generated types.
2020-08-20 14:19:27 -07:00
Pavel Feldman
db2e66aa76
test: introduce global setup (#3544) 2020-08-20 12:51:05 -07:00
Max Schmitt
e32a496e2e
devops(browser-roll): fix fixture tests (#3547) 2020-08-20 11:10:57 -07:00
Pavel Feldman
86815d70e0
test: convert rename options to parameters, remove options magic (#3543) 2020-08-19 21:32:12 -07:00
Pavel Feldman
30f4c0c9fe
test runner: remove dependencies on playwright (#3539) 2020-08-19 17:47:24 -07:00
Dmitry Gozman
18292325b6
api: add waitForElementState('disabled') (#3537)
Allows waiting for the element to be disabled.
2020-08-19 17:20:10 -07:00
Dmitry Gozman
0a22e2758a
fix(chromium): disable lazy loading iframes (#3535)
These do not play nicely with our "page is loaded when all frames
are loaded" logic.
2020-08-19 15:58:13 -07:00
Max Schmitt
e679b82339
fix(devops): auto roll tests (#3536) 2020-08-19 15:04:52 -07:00
Pavel Feldman
a65b0bba5d
test: merge test options into options (#3531) 2020-08-19 13:30:54 -07:00
Dmitry Gozman
9ac1bbc2a5
chore: remove more paths and url matches from the server side (#3528) 2020-08-19 13:27:58 -07:00
Yury Semikhatsky
83de0071c9
feat(screencast): add start/stop events on context (#3483) 2020-08-19 12:45:31 -07:00
Pavel Feldman
8989d66bda
test: introduce options (#3525) 2020-08-18 23:00:45 -07:00
Joel Einbinder
97157520a6
feat(slowmo): only slowmo once per user action (#3012)
This changes the behavior of slowmo to slow down user actions instead of every protocol command. This makes slowmo a lot more predictable. Without this, there is no way to set slowmo to a good value without incurring a huge delay at the start of your test when it sets things up.
2020-08-18 19:13:40 -07:00
Pavel Feldman
b0667e8bc5
test: fix fit, do not rely upon mocha suite (#3520) 2020-08-18 18:55:26 -07:00
Dmitry Gozman
e7e8524e14
chore: remove screenshot path from the server side (#3519)
Also fixes auto-detection of mime type based on path and adds tests.
2020-08-18 18:46:56 -07:00
Dmitry Gozman
aeadf50165
chore: use HeadersArray instead of Headers object on the server side (#3512)
This simplifies implementation and avoids multiple conversions.
Also adding some tests around lowercase and wrong types.
2020-08-18 15:38:29 -07:00
Pavel Feldman
77cab8bed3
test: introduce test collector (#3515) 2020-08-18 14:12:31 -07:00
Pavel Feldman
510182f0b9
test: use isChromium, etc fixtures for browser name sniffing (#3508) 2020-08-18 12:48:32 -07:00
Joel Einbinder
b2228a660a
fix(test): disable more screenshot tests on headful firefox (#3513) 2020-08-18 11:39:10 -07:00
Darío Kondratiuk
5a964f7f99
tests: fix should get the same headers as the server (#3510)
`server.setRoute` won't return a promise
2020-08-18 09:58:00 -07:00
Dmitry Gozman
7a77faf194
fix(testrunner): do not override debug.log (#3505)
Otherwise, our DEBUG_FILE is ignored and we dump all DEBUG on the bots.
2020-08-17 21:02:26 -07:00
Joel Einbinder
dfa1f103af
feat(screenshot): create directories for screenshot file 2020-08-17 16:53:19 -07:00
Dmitry Gozman
0e9793c452
api: ElementHandle.waitForElementState (#3501)
This method waits for visible, hidden, stable or enabled state,
similar to the actionability checks performed before actions.

This gives a bit more control to the user. Some examples:
- Allows to wait for something to be stable before taking a screenshot.
- Allows to wait for the element to be hidden/detached after a specific action.
2020-08-17 16:22:34 -07:00
Max Schmitt
59e3326fc0
tests: add test for page.focus() in Firefox (#3478) 2020-08-17 16:08:47 -07:00
Dmitry Gozman
1e9c0eb705
chore: remove logger infrastructure from server side (#3487)
We do not implement LoggerSink on the server, so we can
use a simple debugLogger.
2020-08-17 14:12:31 -07:00
Pavel Feldman
3aae8c6be1
test: run tests by ordinals, not ranges (#3497) 2020-08-17 10:33:42 -07:00
Pavel Feldman
262e886940
test: organize golden files under __snapshots__ folder (#3494) 2020-08-16 19:19:52 -07:00
Andrey Lushnikov
f983432536
feat(firefox): roll firefox to r1160 (#3468)
Since this is a beta build, cookie tests are restored to the
proper non-lax-by-default behavior.

Fixes #3215
2020-08-15 17:15:21 -07:00
Pavel Feldman
c90039586d
test: restart worker upon any test failure (#3492) 2020-08-15 13:40:19 -07:00
Pavel Feldman
c44f841f33
test: reverse dumpio into quiet, serialize output (#3491) 2020-08-15 00:19:20 -07:00
Pavel Feldman
73d2dc11d7
test: encapsulate mocha into test runner (#3490) 2020-08-14 22:26:16 -07:00
Max Schmitt
5410c30908
chore(test): fix tests when using browser path overwrites (#3453) 2020-08-14 19:34:42 -07:00
Pavel Feldman
35fbd588c8
test: implement in-process debug mode (#3486) 2020-08-14 19:31:04 -07:00
Dmitry Gozman
bc23324878
chore: remove apiName plumbing and some unused methods from server side (#3481)
We append apiName where needed on the client instead.
2020-08-14 18:25:32 -07:00
Dmitry Gozman
244c2f37b6
feat(rpc): make sure filechooser is only intercepted when needed (#3482)
So that user can choose a file manually in headful mode.
2020-08-14 18:24:36 -07:00
Joel Einbinder
0c798f0572
fix(testrunner): properly run tests when the first arg is a file (#3472) 2020-08-14 16:49:49 -07:00
Dmitry Gozman
69e1e713ef
feat(click): provide preview of the element intercepting pointer events (#3449) 2020-08-14 14:48:36 -07:00
Dmitry Gozman
85c93e91a7
api: introduce ElementHandle.waitForSelector (#3452)
This is similar to Frame.waitForSelector, but relative to the handle.
2020-08-14 14:47:24 -07:00
Pavel Feldman
a03c761201
test: unconditionally plumb debug to parent process (#3479) 2020-08-14 13:46:43 -07:00
Pavel Feldman
d537088af8
test: deliver colorful debug messages, do not pipe stdio (#3477) 2020-08-14 13:42:03 -07:00
Dmitry Gozman
dec8fb7890
fix(hover): do not require the element to be enabled before hovering (#3445) 2020-08-14 13:18:32 -07:00
Joel Einbinder
c1de95f91f
feat(testrunner): pretty error messages (#3469) 2020-08-14 07:28:35 -07:00
Pavel Feldman
2f5a0a6c07
test: slowly removing testOptions (#3464) 2020-08-14 07:22:54 -07:00
Joel Einbinder
036cd5ca41
feat(testrunner): use ring character for skips (#3454) 2020-08-13 23:58:03 -07:00
Joel Einbinder
f45791dd8b
feat(testrunner): support sourcemaps (#3459) 2020-08-13 23:17:46 -07:00
Pavel Feldman
ee02702203
test: allow overriding test fixtures, add some test runner tests (#3463) 2020-08-13 22:35:54 -07:00
Pavel Feldman
31fac37780
test: allow overriding the worker fixtures (#3460) 2020-08-13 20:35:26 -07:00
Yury Semikhatsky
737bfa2676
test(screencast): skip test that depends on accelerated compositing (#3458) 2020-08-13 18:01:06 -07:00
Pavel Feldman
6abc352498
test: remove output and golden directory notions (#3456) 2020-08-13 17:32:27 -07:00
Dmitry Gozman
9b52ca8676
chore: remove unused non-rpc code, test options, infra, bots (#3444) 2020-08-13 16:00:23 -07:00
Joel Einbinder
176227549d
feat(testrunner): cache transformed files (#3451)
This makes parsing faster to the point where it is not worth it to report when files are parsed.
2020-08-13 14:35:05 -07:00
Joel Einbinder
84441f8f77
chore(test): run doclint tests with mocha, delete testrunner again (#3447) 2020-08-13 13:57:27 -07:00
Pavel Feldman
e2cfb05786
test: print stderr upon test failure (#3448) 2020-08-13 13:32:15 -07:00
Dmitry Gozman
18b2cf5ec7
feat(rpc): use rpc protocol for browserType.connect (#3380) 2020-08-13 13:24:49 -07:00
Pavel Feldman
5498ed10a8
test: introduce --trial-run mode to capture test model (#3436) 2020-08-13 11:48:58 -07:00
Yury Semikhatsky
4bad89fa37
test(screencast): do print actual pixels on failure (#3442) 2020-08-13 10:45:06 -07:00
Yury Semikhatsky
68e6ab888c
test(screencast): test that css animations are recorded (#3427) 2020-08-13 09:12:13 -07:00
Pavel Feldman
4bb2658e74
test: fix create output folder (#3431) 2020-08-12 22:41:40 -07:00
Pavel Feldman
51bd3709ff
Revert "chore(test): run doclint tests with mocha, delete utils/testrunner (#3428)" (#3432) 2020-08-12 22:01:37 -07:00
Joel Einbinder
061ff25741
chore(test): run doclint tests with mocha, delete utils/testrunner (#3428) 2020-08-12 20:02:48 -07:00
Joel Einbinder
d3677357b8
feat(testrunner): take the first argument as the test root dir (#3423) 2020-08-12 20:01:13 -07:00
Joel Einbinder
ec24516e66
chore(test): remove try/finally pattern from fixtures (#3409) 2020-08-12 17:51:07 -07:00
Dmitry Gozman
f4e65f6dca
test: implement spec builder (#3422) 2020-08-12 16:49:55 -07:00
Yury Semikhatsky
f0fcdc8f51
test(firefox): make headful screencast tests work under xvfb (#3421) 2020-08-12 16:07:01 -07:00
Dmitry Gozman
eb67c86213
test: fix it.fail().slow is not a function (#3420) 2020-08-12 15:22:26 -07:00
Pavel Feldman
a4a07c46c3
test: support --grep, --forbid-only (#3417) 2020-08-12 14:57:37 -07:00
Dmitry Gozman
a574fa6edb
api: add Frame.page() getter (#3392) 2020-08-12 14:45:36 -07:00
Dmitry Gozman
adfeaa49ed
feat(firefox): bump to 1157 (#3395) 2020-08-12 14:45:00 -07:00
Pavel Feldman
962ddc09c4
test: consolidate runner files (#3415) 2020-08-12 13:47:44 -07:00
Pavel Feldman
7e07634cc6
test: use mocha in ci/cd (#3406) 2020-08-12 11:48:30 -07:00
Yury Semikhatsky
16b471fac6
test(screencast): video recording during cros-process navigation (#3396) 2020-08-12 10:18:41 -07:00
Pavel Feldman
4061bc696a
test: make fit run single test (#3394) 2020-08-11 19:44:13 -07:00
Joel Einbinder
6054f14794
chore(tests): convert all tests to typescript (#3384) 2020-08-11 15:50:53 -07:00
Yury Semikhatsky
812d7eefe9
feat(webkit): bump to 1325 (#3388) 2020-08-11 13:59:23 -07:00
Andrey Lushnikov
2db97e3b2d
feat(firefox): migrate to protocol-based proxy implementation (#3362)
This migrates Firefox to the protocol-based proxy implementation.

Benefits:
- supports secure web proxies (already supported by Chromium)
- unlocks support for SOCKS proxies with authentication
2020-08-11 11:36:27 -07:00
Pavel Feldman
bfdb59eada
test: make mocha runner work in parallel (#3383) 2020-08-11 10:57:30 -07:00
Dmitry Gozman
9697ad635f
fix(chromium): handle the case when new pending comes before old commit (#3370) 2020-08-11 08:59:00 -07:00
Darío Kondratiuk
ff2c2b299f
test: Fix insertText test title (#3386) 2020-08-11 08:43:38 -07:00
Joel Einbinder
77e75b447b
chore(test): move electron tests to typescript (#3379) 2020-08-10 23:04:38 -07:00
Yury Semikhatsky
d8d845afcc
feat(screencast): add private recording APIs and basic test (#3296) 2020-08-10 21:22:57 -07:00
Dmitry Gozman
8bb6d73b44
feat(rpc): keep non-rpc linux bots for now (#3381) 2020-08-10 20:49:53 -07:00
Pavel Feldman
823ef86470
test: add support for mocha (#3376) 2020-08-10 20:10:39 -07:00
Joel Einbinder
d76166beca
chore(test): require playwright fixtures from userland (#3355)
This patch moves fixtures.js to base.fixtures.ts that sits next to tests. All tests get an extra import to get the base fixtures (both types and implementations).
2020-08-10 16:48:34 -07:00
Dmitry Gozman
3179e71912
feat(rpc): in-process rpc on by default (#3104) 2020-08-10 14:18:58 -07:00
Pavel Feldman
6f09590c9a
test: restore nojest runner (#3359) 2020-08-08 19:43:00 -07:00
Dmitry Gozman
7e2cc77524
test: add a test for newCDPSession rejecting on non-pages (#3353) 2020-08-07 15:40:46 -07:00
Dmitry Gozman
83f5628549
feat(rpc): misc fixes (#3351) 2020-08-07 15:40:21 -07:00
Pavel Feldman
f6d321fb6a
test: do not inherit from the Node environment (#3348) 2020-08-07 14:27:04 -07:00
Pavel Feldman
b3091deb78
fix(cors): allow routing the cors request with credentials (#3336) 2020-08-07 14:02:41 -07:00
Joel Einbinder
eac8aeedaf
chore(types): convert tests to typescript part 3 (#3340) 2020-08-07 11:19:15 -07:00
Yury Semikhatsky
1bcbf19e86
test(video): mark test as failing in WebKit Linux (#3344) 2020-08-07 09:53:34 -07:00
Andrey Lushnikov
411c738026
feat(firefox): roll to r1154 (#3333)
This roll includes firefox upstream roll. A few tests were hard on me,
so they are currently disabled:

- `should work with subframes return 204`
- `should work when subframe issues window.stop()`

Additionally, new firefox changes cookie treatment that affected a
portion of our cookie tests.

References #3215
2020-08-06 17:16:52 -07:00
Joel Einbinder
83ac3f43f3
chore(test): convert some more tests to typescript (#3329) 2020-08-06 14:12:14 -07:00
Pavel Feldman
4b3fb6dcca
chore(test): convert tests to typescript (1) (#3307) 2020-08-06 08:27:00 -07:00
Pavel Feldman
5c0b88fb10
feat(test): add dot report for aslushnikov (#3317) 2020-08-05 21:18:20 -07:00
Pavel Feldman
0a5d340e8e
test: print failed tests upon interrupt (#3316) 2020-08-05 17:31:13 -07:00
Yury Semikhatsky
5f7b5469b9
test: overridden timezone does not change default in another context (#3313) 2020-08-05 16:31:43 -07:00
Yury Semikhatsky
aa2ec09e97
test: default and overriden locale isolation between contexts (#3308) 2020-08-05 13:43:38 -07:00
Pavel Feldman
9280037d0f
chore(test): add blank lines (#3303) 2020-08-05 11:43:40 -07:00
Yury Semikhatsky
9ec026730f
chore(test): use pathToFileURL (#3292) 2020-08-04 17:26:42 -07:00
Pavel Feldman
57490b74c6
test: remove describes (6) (#3295) 2020-08-04 16:32:10 -07:00
Pavel Feldman
4cbfa09c2c
test: remove describes (5) (#3294) 2020-08-04 15:57:25 -07:00
Pavel Feldman
2e65b0afff
test: remove describes (4) (#3286) 2020-08-04 15:09:24 -07:00
Pavel Feldman
573f580fa8
test: remove describes (3) (#3278) 2020-08-03 16:30:37 -07:00
Yury Semikhatsky
de55fa6482
fix(webkit): ensure WebKit can play h264 video (#3272) 2020-08-03 16:06:57 -07:00
Pavel Feldman
bb267356fd
test: remove describes (2) (#3276) 2020-08-03 15:23:53 -07:00
Pavel Feldman
5c4f06703d
test: remove describes (#3274) 2020-08-03 13:41:48 -07:00
Dmitry Gozman
3edfb2a9d8
test: add REPORT_ONLY mode for test collection (#3225) 2020-07-31 11:02:11 -07:00
Ross Wollman
ce0ddd270b
feat(download): create directories for saveAs (#3249)
This is to match the behavior for screenshots path added in #3247.
2020-07-30 23:21:03 -07:00
Dmitry Gozman
21eafbcdae
test: unflake screenshot test (#3245) 2020-07-30 16:49:58 -07:00
Dmitry Gozman
2f95b6e34e
feat(selectors): auto-detect xpath starting with ".." (#3239) 2020-07-30 16:21:48 -07:00
Joel Einbinder
08916781a8
fix(test): display correct error when golden files mismatch (#3234) 2020-07-30 11:20:48 -07:00
Dmitry Gozman
cefb1b9727
feat(rpc): run fixtures.jest.js with channel (#3227)
Also, introduce setupInProcess wrapper to be used for in-process rpc.
2020-07-30 10:22:28 -07:00
Ross Wollman
7dd9f2c290
test(iframes): add x-frame-options display test (#3217)
Firefox is marked as failing since Playwright's build of Firefox happily
displays the iframe while the consumer build of Firefox refuses display
of the iframe.
2020-07-29 16:49:02 -07:00
Dmitry Gozman
9132d23b2b
fix(screenshot): wait for stable position before taking element screenshot (#3216)
Same goes for scrollIntoViewIfNeeded.
2020-07-29 16:36:02 -07:00
Dmitry Gozman
10225d1983
test: fix a race in the oopif test (#3211) 2020-07-29 14:42:28 -07:00