Commit Graph

185 Commits

Author SHA1 Message Date
Dmitry Gozman
a042466125
chore(testrunner): remove effectiveMode and effectiveExpectation (#1569) 2020-03-27 15:35:14 -07:00
Dmitry Gozman
af7a16c360
chore(testrunner): merge test spec with test, suite spec with suite (#1566) 2020-03-26 22:47:13 -07:00
Dmitry Gozman
aad82e00bd
chore(testrunner): decouple UserCallback from location and timeout (#1557)
This will make it easier to change lifetimes of Test and Suite.
2020-03-26 14:43:28 -07:00
Dmitry Gozman
81bd8de00b
feat(testrunner): composable and bindable attributes and modifiers (#1547)
This allows to make things like `fit`, `skip()` and soon even `dit` be implemented externally.
2020-03-25 22:42:09 -07:00
Andrey Lushnikov
7e75cefd20
devops: restore publishing @next version (#1540)
This merges the `//utils/apply_next_version.js` in to
`//utils/update_version.js`.
2020-03-25 17:31:19 -07:00
Dmitry Gozman
b61198458c
feat(testrunner): modifiers and attributes (#1538)
This generalizes test modifiers to support custom ones.
2020-03-25 14:40:57 -07:00
Dmitry Gozman
60942d0af5
chore(selectors): move selectors logic to selectors.ts (#1536)
This encapsulates selectors logic in one place, in a preparation for more complex scenarios like main-world selectors or piercing frames.

Note: we had `Page.fill should wait for visible visibilty` test, but we do not actually wait for visible in page.fill(). It happened to pass due to lucky evaluation order.

References #1316.
2020-03-25 14:08:46 -07:00
Dmitry Gozman
6ee7852f0f
chore: update release guide and helper script (#1521) 2020-03-25 13:15:29 -07:00
Dmitry Gozman
a2ee7a1f2a
fix(testrunner): do not spam output after termination (#1511) 2020-03-24 14:40:59 -07:00
Pavel Feldman
222560866a
chore: add logging to the testing server (#1505) 2020-03-24 09:11:56 -07:00
Joel Einbinder
5bf9f223db
fix(docs): consider argument to be optional in evaluate (#1500) 2020-03-23 17:22:10 -07:00
Pavel Feldman
c0c9b7f137
test: make debugp collect IO (take 2) (#1493) 2020-03-23 15:08:02 -07:00
Joel Einbinder
1a25a4efcf
fix(doclint): support lists in comments (#1492)
Adds logging comments to the doclint tests, and adds a new one with a bulleted list in a comment. Lists can only be used in comments where extra properties would be unexpected.
2020-03-23 14:50:32 -07:00
Dmitry Gozman
63906454a7
fix(testrunner): attribute unhandle promise reject to a single worker (#1491)
When there is a single worker, we are almost sure the error originated in that worker. Attributing it helps with context by showing last run tests.
2020-03-23 14:45:21 -07:00
Pavel
1ddf05113b Revert "test: make debugp collect IO (#1485)"
This reverts commit b1bebdad2e.
2020-03-23 13:48:22 -07:00
Pavel Feldman
b1bebdad2e
test: make debugp collect IO (#1485) 2020-03-23 13:42:51 -07:00
Dmitry Gozman
16c7a5bd5c
api(eval): accept zero or one arguments in all evaluation functions (#1431) 2020-03-20 15:08:17 -07:00
Joel Einbinder
825555cddf
types: better types (#1166)
This generates typescript definitions based on the api.md, instead of autogenerating them from the typescript source code.

Now types
 - only include the public api
 - work with older versions of typescript
 - include descriptions
 - are more consistent
 - are more complete

#6
2020-03-20 01:30:35 -07:00
Andrey Lushnikov
f5ecbff16e
devops: remake downloading logic (#1419)
This patch:
- removes `browserType.downloadBrowserIfNeeded()` method. The method
  turned out to be ill-behaving and cannot not be used as we'd like to (see #1085)
- adds a `browserType.setExecutablePath` method to set a browser
  exectuable.

With this patch, we take the following approach towards managing browser downloads:
- `playwright-core` doesn't download any browsers. In `playwright-core`, `playwright.chromium.executablePath()` returns `null` (same for firefox and webkit).
- clients of `playwright-core` (e.g. `playwright` and others) download browsers one way or another.
They can then configure `playwright` with executable paths and re-export the `playwright` object to their clients.
- `playwright`, `playwright-firefox`, `playwright-chromium` and `playwright-webkit` download 
browsers. Once browsers are downloaded, their executable paths are saved to a `.downloaded-browsers.json` file. This file is read in `playwright/index.js` to configure browser executable paths and re-export the API.
- special case is `install-from-github.js` that also cleans up old browsers.
2020-03-19 11:43:35 -07:00
Joel Einbinder
b0749e3a6d
fix(docs): fixup and lint optionals and return values in api.md (#1376)
Currently in our API `?` means null, but sometimes it means optional. Linting optional/nulls with this patch is required for #1166 to land nicely.

Previously, return types were not being linted in `api.md`. This is fixed, along with many broken return types.

This patch considers `?` to mean nullable, and has some heuristics to determine optionality. I believe this to be the minimal patch needed to unblock #1166. After it lands, we can consider changing the api docs to hopefully remove some heuristics and strangeness.
2020-03-18 16:23:25 -07:00
Joel Einbinder
741e2d19d6
fix(docs): lint and fix all internal links in api.md
We have had a lot of churn in the api, which has caused a lot of our links to break.
2020-03-18 15:34:53 -07:00
Andrey Lushnikov
a9ab9b0dc0
fix(testrunner): sourcemapify stack traces for test errors (#1409)
Fixes #1403
2020-03-17 09:04:44 -07:00
Dmitry Gozman
3960b179a1
fix(testrunner): fit.fail should run the test (#1407) 2020-03-16 19:12:52 -07:00
Andrey Lushnikov
5816ec53f7
fix(testrunner): dedup focused tests and suites by id (#1393) 2020-03-16 09:04:55 -07:00
Dmitry Gozman
e7eeefe4c7
chore(testrunner): separate expectations from run mode (#1395)
Run/Focus/Skip is orthogonal to expect to Pass/Fail/Flake.
This change separates the two, in a preparation to run Fail/Flaky tests.
2020-03-15 23:10:49 -07:00
Pavel Feldman
e692dd6e62
api(cdp): rename ChromiumSession to CDPSession (#1380) 2020-03-13 16:16:54 -07:00
Dmitry Gozman
0d7cb29329
test: continue running tests after crash, report crashes separately (#1362) 2020-03-12 17:32:53 -07:00
Joel Einbinder
be83cba409
fix(doclint): correctly get versions on windows (#1350)
--version doesn't work on windows.
2020-03-12 14:03:01 -07:00
Andrey Lushnikov
704fe6dc73 fix(testrunner): fix reporting focused tests 2020-03-11 18:53:24 -07:00
Andrey Lushnikov
1cd00bd068
feat(testrunner): allow filtering by name and show all focused tests (#1354) 2020-03-11 18:30:43 -07:00
Max Schmitt
a24cce8b74
devops: fix protocol generation with root on Linux (#1327)
Currently it was leading to an error if I tried to build it with a root user on a Linux environment. So in the end the type generation was just skipped:

```
22:51:55.757 $ node install-from-github.js
22:51:55.813 Building playwright...
22:52:14.094 chromium downloaded to /11994741/playwright/.local-chromium/linux-747023 22:52:14.123 Failed to launch browser!
22:52:14.123 [0310/215214.121201:ERROR:zygote_host_impl_linux.cc(89)] Running as root without --no-sandbox is not supported. See https://crbug.com/638180.
22:52:14.123 TROUBLESHOOTING: https://github.com/Microsoft/playwright/blob/master/docs/troubleshooting.md
```

Maybe we should also exit the script with an error code if the protocol generation was not successful.
2020-03-10 15:21:08 -07:00
Yury Semikhatsky
16d5a9cb24
tests(runner): support DEBUGP for timing out tests (#1324) 2020-03-10 14:48:13 -07:00
Dmitry Gozman
0ce8efab7b
test: rework testrunner workers (#1296)
This change introduces a TestWorker that can be in a certain state,
meaning it has run some beforeAll hooks of a certain test suite stack.

TestWorker can be created at any time, which allows for a number of features:
- don't run hooks for suites with no runnable tests;
- smarter test distribution (and possibility for variuos strategies);
- recovering from hook failures and test failure by creating a new worker;
- possible isolation between workers by running them in separate environments.
2020-03-10 11:30:02 -07:00
Andrey Lushnikov
d542ef849f
fix(testrunner): handle uncaught errors (#1317) 2020-03-10 11:16:54 -07:00
Andrey Lushnikov
3dc48f96c0
chore: output both received value and diff for string expected results (#1287) 2020-03-07 17:29:41 -08:00
Andrey Lushnikov
c881248062
docs(contributing.md): update CONTRIBUTING.md (#1286)
Fixes #1257
2020-03-07 17:29:22 -08:00
Yohix
bfd32fe6d2
doc: fix typos (#1284) 2020-03-07 08:37:47 -08:00
Pavel Feldman
3c35d7b058
api(waitFor): click(waitFor) -> click(force) (#1275) 2020-03-06 16:24:21 -08:00
Andrey Lushnikov
a0e12e0a14
feat(testrunner): support .slow() for slow tests (#1274)
Slow tests have 3x timeout.
2020-03-06 15:52:24 -08:00
Joel Einbinder
8aa88d5021
fix(doc): check and update optional types in the api (#1206)
This adds a new check to doclint for whether a member is correctly marked as optional. 
part of #6
2020-03-03 17:29:12 -08:00
Andrey Lushnikov
019eaa4470 chore(ci): different attempt to publish on Travis 2020-03-02 19:00:47 -08:00
Andrey Lushnikov
01888897b7 chore(ci): fix publish_all_packages.sh on travis 2020-03-02 18:53:57 -08:00
Andrey Lushnikov
64e5e213d8 chore(ci): forcefully login NPM on CI if NPM_AUTH_TOKEN is set 2020-03-02 18:38:39 -08:00
Andrey Lushnikov
a40f562662 chore(ci): add debug info for publish_all_packages 2020-03-02 18:35:04 -08:00
Andrey Lushnikov
497a74d8e2 chore(ci): fix publishing @next on travis 2020-03-02 18:26:45 -08:00
Andrey Lushnikov
62e2570e67 chore: fix utils/apply_next_version.js 2020-03-02 18:17:16 -08:00
Andrey Lushnikov
57c45f0d1f
fix: properly publish all packages on travis (#1187) 2020-03-02 17:40:48 -08:00
Pavel
ba06fb2f0d test: mark some tests as skipped 2020-03-02 14:57:54 -08:00
Pavel Feldman
5bd6e4970b
test: it.skip skips and it.fail expects to fail now (#1178) 2020-03-02 13:47:08 -08:00
Pavel Feldman
cbf65a9c56
test: chain test modifiers (#1175) 2020-03-02 11:18:42 -08:00
Andrey Lushnikov
b20b323106 chore(ci): another attempt to program in .travis.yml 2020-02-28 16:17:59 -08:00
Dmitry Gozman
ac2f04f10f
api(selectors): pass selector name when registering, allow file path (#1162) 2020-02-28 15:34:07 -08:00
Andrey Lushnikov
d511d7dd99 chore(ci): use TRAVIS_NEXT_NUMBER instead of Date.now() 2020-02-28 15:20:07 -08:00
Dmitry Gozman
c4f55bf22c
chore: guide for producing release notes (#1165) 2020-02-28 14:40:50 -08:00
Andrey Lushnikov
9b51febf50
feat: setup continuous deployment (#1159)
This patch sets continuous deployment of playwright packages
from tip-of-tree. These packages are released under the "@next",
so it should be possible to install tip-of-tree version of Playwright
from NPM using

```
npm i playwright@next
```

A few technical details:
- This uses travis to drive deployments
- This deploys from master and only if all browsers pass all tests on linux
- This is an attempt to deploy just two packages: `playwright-core` and `playwright-firefox`. If this works out, I'll follow-up with other packages as well
2020-02-28 11:02:42 -08:00
Joel Einbinder
b50e8b377f
chore: fix doclint tests (#1098)
This fixes the doclint tests so that `npm test` works. It also adds all the browsers to npm test.

Fixes #8
2020-02-24 18:24:02 -08:00
Andrey Lushnikov
0ded511d0b
feat(testrunner): better matchers (#1077)
This patch re-implements matching and reporting for test runner.
Among other improvements:
- test failures now show a short snippet from test
- test failures now explicitly say what received and what was expected
- `expect.toBe()` now does text diff when gets strings as input
- `expect.toEqual` now does object diff
2020-02-20 22:55:39 -08:00
Andrey Lushnikov
53a7e342e9 fix(testrunner): support throwing non-errors 2020-02-20 17:06:05 -08:00
Andrey Lushnikov
8487ef2821
feat(testrunner): add DEBUG to testrunner (#1014)
The plan is to collect logs for the whole test run and upload it later on
using https://github.com/actions/upload-artifact

Produced log size:
- 163MB (8.5MB zipped) for Chromium: `DEBUG=* npm run unit 2>log`
- 135MB (4.8MB zipped) for WebKit: `DEBUG=*,-pw:wrapped* npm run wunit 2>log`
- 29MB (4.0MB zipped) for Firefox: `DEBUG=* npm run funit 2>log`
2020-02-14 15:21:08 -08:00
Andrey Lushnikov
d29625c281
chore: generate browser versions when doing release (#999)
This introduces a handful of new markdown preprocessor commands to
insert browser versions:
- `GEN:chromium-version-if-release` - inserts current Chromium version
  if we're doing release; noop otherwise.
- `GEN:firefox-version-if-release` - inserts current Firefox version
  if we're doing release; noop otherwise.

And to generate badge links:
- `GEN:chromium-version-badge-if-release` - inserts current Chromium version
  badge if we're doing release; noop otherwise.
- `GEN:firefox-version-badge-if-release` - inserts current Firefox version
  badge if we're doing release; noop otherwise.

This doesn't touch webkit at all - we're yet to figure what to do with
webkit version.

NOTE: versions will be updated only once we release. This way our
README.md always represents last released version.
2020-02-13 18:26:38 -08:00
Joel Einbinder
251ad38824
fix(navigations): remove LifecycleWatcher, fix flakes (#882) 2020-02-10 18:35:47 -08:00
Joel Einbinder
c03e8b7946
chore(tests): add types for tests (#915)
I enabled vscode autocomplete in our test files. Typechecking had too many errors to enable, but it caught some real bugs that I will fix in a follow up.

This patch contains:
* `test/types.d.ts` - d.ts file for our test runner.
* `test/tsconfig.json` - typescript project for our tests.
* JSDoc header in all specs to mark the describe as a TestSuite
* Drive-by fix of a launcher test that was using `if` instead of `it`
* Some drive-by fixes of unimpactful typos in tests.
2020-02-10 13:20:13 -08:00
Andrey Lushnikov
4fcc63f2e1 chore: remove usages of mime module from infrastructure
Follow-up to 4d84e35096
2020-02-10 11:26:08 -08:00
Andrey Lushnikov
f4734ef570 feat(testrunner): show workerId in verbose mode 2020-02-07 17:16:31 -08:00
Joel Einbinder
ffc1022717
fix(doclint): fix doclint for new typescript (#879)
Some types now have `innerTypeNames.length === 0`. Fun!
2020-02-06 16:26:36 -08:00
Pavel Feldman
99d06892c5
tests: explicitly close contexts for browser.newPage (#875) 2020-02-06 15:18:07 -08:00
Andrey Lushnikov
cd68619786 chore: fix protocol-types-generator to work with new API
Follow-up to a2ab645e63
2020-02-05 16:51:16 -08:00
Pavel Feldman
55b6fe241e
feat(launch): introduce client, server & persistent launch modes (3) (#854) 2020-02-05 16:36:36 -08:00
Pavel Feldman
a2ab645e63
feat(launch): introduce client, server & persistent launch modes (2) (#840) 2020-02-05 12:41:55 -08:00
Andrey Lushnikov
0f1a42a5d3 docs(readme): fix API link to always point to last released API 2020-02-05 12:05:36 -08:00
Pavel Feldman
0518625dcc
feat(launch): introduce client, server & persistent launch modes (1) (#838) 2020-02-04 19:41:38 -08:00
Andrey Lushnikov
7128628d70 feat(testrunner): ability to repeat test suites (#681)
Now you can `fdescribe.repeat(10)` to repeat test suites.
2020-01-27 12:48:11 -08:00
Pavel Feldman
5a5016fa7d docs: inline superclass toc into classes for convenience (#663) 2020-01-27 10:05:04 -08:00
Yury Semikhatsky
b64604c1ea
chore: replace pptr with pw (#643) 2020-01-24 16:15:41 -08:00
Joel Einbinder
ff877014cd
doc(troubleshooting): add note about lack of node 8 support (#623) 2020-01-24 09:04:54 -08:00
Dmitry Gozman
056fbbd53e
fix(api): make pipe connection the default, expose webSocket launch option (#562) 2020-01-23 17:45:31 -08:00
Joel Einbinder
3b2993ffe9 fix(docs): add back troubleshooting.md (#605)
This is mostly copied from Puppeteer's troubleshooting.md, with everything under a Chromium header and a few tweaks to make things make sense for Playwright.

We can add other browsers to this document once we figure out how to troubleshoot them.

Fixes #583
2020-01-23 15:54:09 -08:00
Dmitry Gozman
ac2ba3cbd9
fix(api): BrowserServer -> BrowserApp, resuse it between browsers (#599) 2020-01-23 14:40:37 -08:00
Dmitry Gozman
fa2f321fdd
fix(api): remove BrowserServer.connect (#574) 2020-01-23 08:51:43 -08:00
Andrey Lushnikov
710554bff5
feat(testrunner): add it.repeat to repeat test multiple times (#568) 2020-01-22 15:31:52 -08:00
Andrey Lushnikov
eab8f92ac8 docs: create a single top-level TOC for api.md (#561)
This patch:
- creates a single top-level TOC for api.md
- adds a TOC for each class in the api.md
2020-01-22 12:21:45 -08:00
Joel Einbinder
4573cd3fec chore(packages): copy readme from root folder into packages/playwright (#552) 2020-01-21 17:24:47 -08:00
Joel Einbinder
fa1d286d56
chore: npm packages (#525)
## playwright-chromium
- installs chromium
- exposes chromium api from playwright-core

## playwright-firefox
- installs firefox
- exposes firefox api from playwright-core

## playwright-webkit
- installs webkit
- exposes webkit api from playwright-core

## playwright-core
- downloads no browsers
- contains all of the js code
- designed for internal use

## playwright
- downloads all browsers
- exposes the entire api from playwright-core

## github
- downloads all browsers, generates protocol definitions, builds typescript
- exposes "playwright-core" api
2020-01-21 12:22:17 -08:00
Pavel Feldman
80c3b46a54
feat(websockets): implement support for ws on cr/wk (#542) 2020-01-21 11:48:48 -08:00
Yury Semikhatsky
fc9ddb7c3c chore(webkit): simplify session init logic (#523)
* chore(webkit): simplify session init logic

* update remaining license headers
2020-01-17 14:02:57 -08:00
Andrey Lushnikov
e03f1e4c2c feat(firefox): roll firefox to r1012 (#493)
This roll puts Firefox protocol types under related protocol domains,
so protocol type generator had to be updated as well.
2020-01-14 15:32:31 -08:00
Dmitry Gozman
5b1aef4c20 api: remove BrowserFetcher from api, update docs (#487) 2020-01-14 10:07:26 -08:00
Dmitry Gozman
fb1b3d9a89 chore: finish strict type checks across src (#482) 2020-01-13 22:08:35 -08:00
Dmitry Gozman
b388722777
chore: update top-level api (#479)
```
require('playwright').chromium.launch(...)
window.playwrightweb.chromium.connect(...)
```
2020-01-13 17:36:46 -08:00
Andrey Lushnikov
98bf9ac1d5
fix(testrunner): return correct exit code (#474) 2020-01-13 15:30:16 -08:00
Yury Semikhatsky
778ab3deff chore: replace CHROME with CHROMIUM (#465) 2020-01-13 10:13:28 -08:00
Dmitry Gozman
c77fd5e6cb
feat(pw-web): generate playwright/web.js which can be used in the browser (#455) 2020-01-10 16:03:10 -08:00
Andrey Lushnikov
02c504ca59
feat(testrunner): "sourcemapify" crash error stacktraces (#441)
Whenever test runner crashes due to UnhandledPromiseRejection, the
related error should be run through "sourcemap decoding".
2020-01-09 16:37:19 +00:00
Dmitry Gozman
57c3916b0c
docs: support interfaces in doclint (#420) 2020-01-08 14:04:33 -08:00
Andrey Lushnikov
a18777673e
devops(testrunner): support source maps (#340)
This patch adds a basic source map support to test runner.

SourceMap support is powered by Chromium DevTools source map
implementation (thus copyright). Unlike popular `source-map` npm
module, it's sync and pretty straight-forward.

The `SourceMap.js` file has a few modifications wrt upstream
Chromium version:
- reverse mappings API is removed. There's no need to ever compute them
- the `upperBoundary` function from DevTools' platform is inlined
2020-01-08 16:16:54 +00:00
Dmitry Gozman
6318b1de75 chore: move server-side code to src/server (#411) 2020-01-07 15:27:45 -08:00
Pavel Feldman
e0693654b7 docs: do not yell at templated types in doc lint (#409) 2020-01-07 14:19:58 -08:00
Dmitry Gozman
9c966c8b19
feat(pw-web): introduce platform.ts to absract between node and browser platforms (#392) 2020-01-07 11:55:24 -08:00
Arjun Attam
562e6f5fe1 chore(license): use Apache 2.0 (#389) 2020-01-06 18:22:35 -08:00
Dmitry Gozman
04bf728364 test: make network cache test pass on all browsers (#374) 2020-01-03 15:34:51 -08:00