Commit Graph

199 Commits

Author SHA1 Message Date
Dmitry Gozman
d38baae01c
feat(testrunner): nested environments (#1635) 2020-04-02 13:12:22 -07:00
Pavel Feldman
e241c1bef8
chore: remove web mode (#1625) 2020-04-01 14:42:47 -07:00
Arjun Attam
1f0b7bff04
docs(api): update ordering for $eval and $$eval (#1623) 2020-04-01 13:09:24 -07:00
Dmitry Gozman
f87e64544c
feat(testrunner): introduce environments (#1593) 2020-04-01 10:49:47 -07:00
Joel Einbinder
a853690c16
fix(types): don't export derived types (#1598)
I was playing around today with different ways of changing the way we export types for #1439. I looked at only exporting 'Parameter' types, only exporting 'Return' types, only exporting a manual list of 'important' types. They all had different pros and cons, and it was very difficult to settle on a good answer.

For now, let's not export any parameter/return types. We can whitelist some types upon user request. I'm thinking `LaunchOptions` and `AccessibilitySnapshot` could be quite useful. We can always add new types after 1.0, but we can't remove them.

The patch looks funny because this was my original intent for the types, but I didn't know I had to `export {}` to tell typescript that my .d.ts shouldn't export everything.
2020-03-31 16:19:20 -07:00
Joel Einbinder
92c5ab3b7e
fix(types): correctly infer type in $eval and $$eval (#1603) 2020-03-31 13:22:38 -07:00
Joel Einbinder
08aebc72fa
fix(types): add types for waitForEvent (#1601) 2020-03-31 09:17:54 -07:00
Dmitry Gozman
b6166c9669
chore(testrunner): introduce Location class (#1585)
Drive-by: fix an edge when testing continued after termination.
2020-03-29 21:38:30 -07:00
Dmitry Gozman
c49b856d6d
chore(testrunner): remove setup() helper (#1584) 2020-03-29 11:37:45 -07:00
Dmitry Gozman
a41836b1f1
chore(testrunner): introduce TestRun (#1582) 2020-03-28 14:25:57 -07:00
Dmitry Gozman
b85ab891a1
chore(testrunner): make most modifiers external (#1581) 2020-03-28 08:49:00 -07:00
Dmitry Gozman
4bd46bafa9
feat(testurnner): allow multiple hooks isntances and per-test hooks (#1571) 2020-03-27 22:57:22 -07:00
Dmitry Gozman
6503c83302
fix(install): speculative fix for generateChromiumProtocol (#1583) 2020-03-27 22:42:39 -07:00
Pavel Feldman
6903496f2d
fix(build): generate protocol in chromium (#1579) 2020-03-27 17:11:54 -07:00
Dmitry Gozman
a042466125
chore(testrunner): remove effectiveMode and effectiveExpectation (#1569) 2020-03-27 15:35:14 -07:00
Dmitry Gozman
af7a16c360
chore(testrunner): merge test spec with test, suite spec with suite (#1566) 2020-03-26 22:47:13 -07:00
Dmitry Gozman
aad82e00bd
chore(testrunner): decouple UserCallback from location and timeout (#1557)
This will make it easier to change lifetimes of Test and Suite.
2020-03-26 14:43:28 -07:00
Dmitry Gozman
81bd8de00b
feat(testrunner): composable and bindable attributes and modifiers (#1547)
This allows to make things like `fit`, `skip()` and soon even `dit` be implemented externally.
2020-03-25 22:42:09 -07:00
Andrey Lushnikov
7e75cefd20
devops: restore publishing @next version (#1540)
This merges the `//utils/apply_next_version.js` in to
`//utils/update_version.js`.
2020-03-25 17:31:19 -07:00
Dmitry Gozman
b61198458c
feat(testrunner): modifiers and attributes (#1538)
This generalizes test modifiers to support custom ones.
2020-03-25 14:40:57 -07:00
Dmitry Gozman
60942d0af5
chore(selectors): move selectors logic to selectors.ts (#1536)
This encapsulates selectors logic in one place, in a preparation for more complex scenarios like main-world selectors or piercing frames.

Note: we had `Page.fill should wait for visible visibilty` test, but we do not actually wait for visible in page.fill(). It happened to pass due to lucky evaluation order.

References #1316.
2020-03-25 14:08:46 -07:00
Dmitry Gozman
6ee7852f0f
chore: update release guide and helper script (#1521) 2020-03-25 13:15:29 -07:00
Dmitry Gozman
a2ee7a1f2a
fix(testrunner): do not spam output after termination (#1511) 2020-03-24 14:40:59 -07:00
Pavel Feldman
222560866a
chore: add logging to the testing server (#1505) 2020-03-24 09:11:56 -07:00
Joel Einbinder
5bf9f223db
fix(docs): consider argument to be optional in evaluate (#1500) 2020-03-23 17:22:10 -07:00
Pavel Feldman
c0c9b7f137
test: make debugp collect IO (take 2) (#1493) 2020-03-23 15:08:02 -07:00
Joel Einbinder
1a25a4efcf
fix(doclint): support lists in comments (#1492)
Adds logging comments to the doclint tests, and adds a new one with a bulleted list in a comment. Lists can only be used in comments where extra properties would be unexpected.
2020-03-23 14:50:32 -07:00
Dmitry Gozman
63906454a7
fix(testrunner): attribute unhandle promise reject to a single worker (#1491)
When there is a single worker, we are almost sure the error originated in that worker. Attributing it helps with context by showing last run tests.
2020-03-23 14:45:21 -07:00
Pavel
1ddf05113b Revert "test: make debugp collect IO (#1485)"
This reverts commit b1bebdad2e.
2020-03-23 13:48:22 -07:00
Pavel Feldman
b1bebdad2e
test: make debugp collect IO (#1485) 2020-03-23 13:42:51 -07:00
Dmitry Gozman
16c7a5bd5c
api(eval): accept zero or one arguments in all evaluation functions (#1431) 2020-03-20 15:08:17 -07:00
Joel Einbinder
825555cddf
types: better types (#1166)
This generates typescript definitions based on the api.md, instead of autogenerating them from the typescript source code.

Now types
 - only include the public api
 - work with older versions of typescript
 - include descriptions
 - are more consistent
 - are more complete

#6
2020-03-20 01:30:35 -07:00
Andrey Lushnikov
f5ecbff16e
devops: remake downloading logic (#1419)
This patch:
- removes `browserType.downloadBrowserIfNeeded()` method. The method
  turned out to be ill-behaving and cannot not be used as we'd like to (see #1085)
- adds a `browserType.setExecutablePath` method to set a browser
  exectuable.

With this patch, we take the following approach towards managing browser downloads:
- `playwright-core` doesn't download any browsers. In `playwright-core`, `playwright.chromium.executablePath()` returns `null` (same for firefox and webkit).
- clients of `playwright-core` (e.g. `playwright` and others) download browsers one way or another.
They can then configure `playwright` with executable paths and re-export the `playwright` object to their clients.
- `playwright`, `playwright-firefox`, `playwright-chromium` and `playwright-webkit` download 
browsers. Once browsers are downloaded, their executable paths are saved to a `.downloaded-browsers.json` file. This file is read in `playwright/index.js` to configure browser executable paths and re-export the API.
- special case is `install-from-github.js` that also cleans up old browsers.
2020-03-19 11:43:35 -07:00
Joel Einbinder
b0749e3a6d
fix(docs): fixup and lint optionals and return values in api.md (#1376)
Currently in our API `?` means null, but sometimes it means optional. Linting optional/nulls with this patch is required for #1166 to land nicely.

Previously, return types were not being linted in `api.md`. This is fixed, along with many broken return types.

This patch considers `?` to mean nullable, and has some heuristics to determine optionality. I believe this to be the minimal patch needed to unblock #1166. After it lands, we can consider changing the api docs to hopefully remove some heuristics and strangeness.
2020-03-18 16:23:25 -07:00
Joel Einbinder
741e2d19d6
fix(docs): lint and fix all internal links in api.md
We have had a lot of churn in the api, which has caused a lot of our links to break.
2020-03-18 15:34:53 -07:00
Andrey Lushnikov
a9ab9b0dc0
fix(testrunner): sourcemapify stack traces for test errors (#1409)
Fixes #1403
2020-03-17 09:04:44 -07:00
Dmitry Gozman
3960b179a1
fix(testrunner): fit.fail should run the test (#1407) 2020-03-16 19:12:52 -07:00
Andrey Lushnikov
5816ec53f7
fix(testrunner): dedup focused tests and suites by id (#1393) 2020-03-16 09:04:55 -07:00
Dmitry Gozman
e7eeefe4c7
chore(testrunner): separate expectations from run mode (#1395)
Run/Focus/Skip is orthogonal to expect to Pass/Fail/Flake.
This change separates the two, in a preparation to run Fail/Flaky tests.
2020-03-15 23:10:49 -07:00
Pavel Feldman
e692dd6e62
api(cdp): rename ChromiumSession to CDPSession (#1380) 2020-03-13 16:16:54 -07:00
Dmitry Gozman
0d7cb29329
test: continue running tests after crash, report crashes separately (#1362) 2020-03-12 17:32:53 -07:00
Joel Einbinder
be83cba409
fix(doclint): correctly get versions on windows (#1350)
--version doesn't work on windows.
2020-03-12 14:03:01 -07:00
Andrey Lushnikov
704fe6dc73 fix(testrunner): fix reporting focused tests 2020-03-11 18:53:24 -07:00
Andrey Lushnikov
1cd00bd068
feat(testrunner): allow filtering by name and show all focused tests (#1354) 2020-03-11 18:30:43 -07:00
Max Schmitt
a24cce8b74
devops: fix protocol generation with root on Linux (#1327)
Currently it was leading to an error if I tried to build it with a root user on a Linux environment. So in the end the type generation was just skipped:

```
22:51:55.757 $ node install-from-github.js
22:51:55.813 Building playwright...
22:52:14.094 chromium downloaded to /11994741/playwright/.local-chromium/linux-747023 22:52:14.123 Failed to launch browser!
22:52:14.123 [0310/215214.121201:ERROR:zygote_host_impl_linux.cc(89)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
22:52:14.123 TROUBLESHOOTING: https://github.com/Microsoft/playwright/blob/master/docs/troubleshooting.md
```

Maybe we should also exit the script with an error code if the protocol generation was not successful.
2020-03-10 15:21:08 -07:00
Yury Semikhatsky
16d5a9cb24
tests(runner): support DEBUGP for timing out tests (#1324) 2020-03-10 14:48:13 -07:00
Dmitry Gozman
0ce8efab7b
test: rework testrunner workers (#1296)
This change introduces a TestWorker that can be in a certain state,
meaning it has run some beforeAll hooks of a certain test suite stack.

TestWorker can be created at any time, which allows for a number of features:
- don't run hooks for suites with no runnable tests;
- smarter test distribution (and possibility for variuos strategies);
- recovering from hook failures and test failure by creating a new worker;
- possible isolation between workers by running them in separate environments.
2020-03-10 11:30:02 -07:00
Andrey Lushnikov
d542ef849f
fix(testrunner): handle uncaught errors (#1317) 2020-03-10 11:16:54 -07:00
Andrey Lushnikov
3dc48f96c0
chore: output both received value and diff for string expected results (#1287) 2020-03-07 17:29:41 -08:00
Andrey Lushnikov
c881248062
docs(contributing.md): update CONTRIBUTING.md (#1286)
Fixes #1257
2020-03-07 17:29:22 -08:00
Yohix
bfd32fe6d2
doc: fix typos (#1284) 2020-03-07 08:37:47 -08:00
Pavel Feldman
3c35d7b058
api(waitFor): click(waitFor) -> click(force) (#1275) 2020-03-06 16:24:21 -08:00
Andrey Lushnikov
a0e12e0a14
feat(testrunner): support .slow() for slow tests (#1274)
Slow tests have 3x timeout.
2020-03-06 15:52:24 -08:00
Joel Einbinder
8aa88d5021
fix(doc): check and update optional types in the api (#1206)
This adds a new check to doclint for whether a member is correctly marked as optional. 
part of #6
2020-03-03 17:29:12 -08:00
Andrey Lushnikov
019eaa4470 chore(ci): different attempt to publish on Travis 2020-03-02 19:00:47 -08:00
Andrey Lushnikov
01888897b7 chore(ci): fix publish_all_packages.sh on travis 2020-03-02 18:53:57 -08:00
Andrey Lushnikov
64e5e213d8 chore(ci): forcefully login NPM on CI if NPM_AUTH_TOKEN is set 2020-03-02 18:38:39 -08:00
Andrey Lushnikov
a40f562662 chore(ci): add debug info for publish_all_packages 2020-03-02 18:35:04 -08:00
Andrey Lushnikov
497a74d8e2 chore(ci): fix publishing @next on travis 2020-03-02 18:26:45 -08:00
Andrey Lushnikov
62e2570e67 chore: fix utils/apply_next_version.js 2020-03-02 18:17:16 -08:00
Andrey Lushnikov
57c45f0d1f
fix: properly publish all packages on travis (#1187) 2020-03-02 17:40:48 -08:00
Pavel
ba06fb2f0d test: mark some tests as skipped 2020-03-02 14:57:54 -08:00
Pavel Feldman
5bd6e4970b
test: it.skip skips and it.fail expects to fail now (#1178) 2020-03-02 13:47:08 -08:00
Pavel Feldman
cbf65a9c56
test: chain test modifiers (#1175) 2020-03-02 11:18:42 -08:00
Andrey Lushnikov
b20b323106 chore(ci): another attempt to program in .travis.yml 2020-02-28 16:17:59 -08:00
Dmitry Gozman
ac2f04f10f
api(selectors): pass selector name when registering, allow file path (#1162) 2020-02-28 15:34:07 -08:00
Andrey Lushnikov
d511d7dd99 chore(ci): use TRAVIS_NEXT_NUMBER instead of Date.now() 2020-02-28 15:20:07 -08:00
Dmitry Gozman
c4f55bf22c
chore: guide for producing release notes (#1165) 2020-02-28 14:40:50 -08:00
Andrey Lushnikov
9b51febf50
feat: setup continuous deployment (#1159)
This patch sets continuous deployment of playwright packages
from tip-of-tree. These packages are released under the "@next",
so it should be possible to install tip-of-tree version of Playwright
from NPM using

```
npm i playwright@next
```

A few technical details:
- This uses travis to drive deployments
- This deploys from master and only if all browsers pass all tests on linux
- This is an attempt to deploy just two packages: `playwright-core` and `playwright-firefox`. If this works out, I'll follow-up with other packages as well
2020-02-28 11:02:42 -08:00
Joel Einbinder
b50e8b377f
chore: fix doclint tests (#1098)
This fixes the doclint tests so that `npm test` works. It also adds all the browsers to npm test.

Fixes #8
2020-02-24 18:24:02 -08:00
Andrey Lushnikov
0ded511d0b
feat(testrunner): better matchers (#1077)
This patch re-implements matching and reporting for test runner.
Among other improvements:
- test failures now show a short snippet from test
- test failures now explicitly say what received and what was expected
- `expect.toBe()` now does text diff when gets strings as input
- `expect.toEqual` now does object diff
2020-02-20 22:55:39 -08:00
Andrey Lushnikov
53a7e342e9 fix(testrunner): support throwing non-errors 2020-02-20 17:06:05 -08:00
Andrey Lushnikov
8487ef2821
feat(testrunner): add DEBUG to testrunner (#1014)
The plan is to collect logs for the whole test run and upload it later on
using https://github.com/actions/upload-artifact

Produced log size:
- 163MB (8.5MB zipped) for Chromium: `DEBUG=* npm run unit 2>log`
- 135MB (4.8MB zipped) for WebKit: `DEBUG=*,-pw:wrapped* npm run wunit 2>log`
- 29MB (4.0MB zipped) for Firefox: `DEBUG=* npm run funit 2>log`
2020-02-14 15:21:08 -08:00
Andrey Lushnikov
d29625c281
chore: generate browser versions when doing release (#999)
This introduces a handful of new markdown preprocessor commands to
insert browser versions:
- `GEN:chromium-version-if-release` - inserts current Chromium version
  if we're doing release; noop otherwise.
- `GEN:firefox-version-if-release` - inserts current Firefox version
  if we're doing release; noop otherwise.

And to generate badge links:
- `GEN:chromium-version-badge-if-release` - inserts current Chromium version
  badge if we're doing release; noop otherwise.
- `GEN:firefox-version-badge-if-release` - inserts current Firefox version
  badge if we're doing release; noop otherwise.

This doesn't touch webkit at all - we're yet to figure what to do with
webkit version.

NOTE: versions will be updated only once we release. This way our
README.md always represents last released version.
2020-02-13 18:26:38 -08:00
Joel Einbinder
251ad38824
fix(navigations): remove LifecycleWatcher, fix flakes (#882) 2020-02-10 18:35:47 -08:00
Joel Einbinder
c03e8b7946
chore(tests): add types for tests (#915)
I enabled vscode autocomplete in our test files. Typechecking had too many errors to enable, but it caught some real bugs that I will fix in a follow up.

This patch contains:
* `test/types.d.ts` - d.ts file for our test runner.
* `test/tsconfig.json` - typescript project for our tests.
* JSDoc header in all specs to mark the describe as a TestSuite
* Drive-by fix of a launcher test that was using `if` instead of `it`
* Some drive-by fixes of unimpactful typos in tests.
2020-02-10 13:20:13 -08:00
Andrey Lushnikov
4fcc63f2e1 chore: remove usages of mime module from infrastructure
Follow-up to 4d84e35096
2020-02-10 11:26:08 -08:00
Andrey Lushnikov
f4734ef570 feat(testrunner): show workerId in verbose mode 2020-02-07 17:16:31 -08:00
Joel Einbinder
ffc1022717
fix(doclint): fix doclint for new typescript (#879)
Some types now have `innerTypeNames.length === 0`. Fun!
2020-02-06 16:26:36 -08:00
Pavel Feldman
99d06892c5
tests: explicitly close contexts for browser.newPage (#875) 2020-02-06 15:18:07 -08:00
Andrey Lushnikov
cd68619786 chore: fix protocol-types-generator to work with new API
Follow-up to a2ab645e63
2020-02-05 16:51:16 -08:00
Pavel Feldman
55b6fe241e
feat(launch): introduce client, server & persistent launch modes (3) (#854) 2020-02-05 16:36:36 -08:00
Pavel Feldman
a2ab645e63
feat(launch): introduce client, server & persistent launch modes (2) (#840) 2020-02-05 12:41:55 -08:00
Andrey Lushnikov
0f1a42a5d3 docs(readme): fix API link to always point to last released API 2020-02-05 12:05:36 -08:00
Pavel Feldman
0518625dcc
feat(launch): introduce client, server & persistent launch modes (1) (#838) 2020-02-04 19:41:38 -08:00
Andrey Lushnikov
7128628d70 feat(testrunner): ability to repeat test suites (#681)
Now you can `fdescribe.repeat(10)` to repeat test suites.
2020-01-27 12:48:11 -08:00
Pavel Feldman
5a5016fa7d docs: inline superclass toc into classes for convenience (#663) 2020-01-27 10:05:04 -08:00
Yury Semikhatsky
b64604c1ea
chore: replace pptr with pw (#643) 2020-01-24 16:15:41 -08:00
Joel Einbinder
ff877014cd
doc(troubleshooting): add note about lack of node 8 support (#623) 2020-01-24 09:04:54 -08:00
Dmitry Gozman
056fbbd53e
fix(api): make pipe connection the default, expose webSocket launch option (#562) 2020-01-23 17:45:31 -08:00
Joel Einbinder
3b2993ffe9 fix(docs): add back troubleshooting.md (#605)
This is mostly copied from Puppeteer's troubleshooting.md, with everything under a Chromium header and a few tweaks to make things make sense for Playwright.

We can add other browsers to this document once we figure out how to troubleshoot them.

Fixes #583
2020-01-23 15:54:09 -08:00
Dmitry Gozman
ac2ba3cbd9
fix(api): BrowserServer -> BrowserApp, resuse it between browsers (#599) 2020-01-23 14:40:37 -08:00
Dmitry Gozman
fa2f321fdd
fix(api): remove BrowserServer.connect (#574) 2020-01-23 08:51:43 -08:00
Andrey Lushnikov
710554bff5
feat(testrunner): add it.repeat to repeat test multiple times (#568) 2020-01-22 15:31:52 -08:00
Andrey Lushnikov
eab8f92ac8 docs: create a single top-level TOC for api.md (#561)
This patch:
- creates a single top-level TOC for api.md
- adds a TOC for each class in the api.md
2020-01-22 12:21:45 -08:00
Joel Einbinder
4573cd3fec chore(packages): copy readme from root folder into packages/playwright (#552) 2020-01-21 17:24:47 -08:00
Joel Einbinder
fa1d286d56
chore: npm packages (#525)
## playwright-chromium
- installs chromium
- exposes chromium api from playwright-core

## playwright-firefox
- installs firefox
- exposes firefox api from playwright-core

## playwright-webkit
- installs webkit
- exposes webkit api from playwright-core

## playwright-core
- downloads no browsers
- contains all of the js code
- designed for internal use

## playwright
- downloads all browsers
- exposes the entire api from playwright-core

## github
- downloads all browsers, generates protocol definitions, builds typescript
- exposes "playwright-core" api
2020-01-21 12:22:17 -08:00
Pavel Feldman
80c3b46a54
feat(websockets): implement support for ws on cr/wk (#542) 2020-01-21 11:48:48 -08:00
Yury Semikhatsky
fc9ddb7c3c chore(webkit): simplify session init logic (#523)
* chore(webkit): simplify session init logic

* update remaining license headers
2020-01-17 14:02:57 -08:00
Andrey Lushnikov
e03f1e4c2c feat(firefox): roll firefox to r1012 (#493)
This roll puts Firefox protocol types under related protocol domains,
so protocol type generator had to be updated as well.
2020-01-14 15:32:31 -08:00