Commit Graph

290 Commits

Author SHA1 Message Date
Ross Wollman
72b95b9ffa
chore: dogfood dotenv (#12673)
Use a top-level .env file to control the internal testing setup.

This allows for easy manipulation of environment variables regardless
of your setup (VSCode Extension, CLI, etc.).
2022-03-11 15:12:25 -08:00
Andrey Lushnikov
e8ad7eb3f2
browser(firefox): roll Firefox to M97 (#12574)
Firefox 98 doesn't work on Ubuntu aarch64 yet.

https://github.com/microsoft/playwright/issues/12557
2022-03-07 16:16:42 -08:00
Pavel Feldman
4b19d59ec5
feat(test): introduce fully parallel mode (#12446) 2022-03-01 18:12:21 -08:00
Andrey Lushnikov
1a7a3bf99d
chore: fix types for tests (#12429) 2022-03-01 01:32:40 -08:00
Andrey Lushnikov
66eda836c9
browser(ff-beta): roll Firefox-Beta to 98b10 (#12421)
Firefox reverted the third-party cookies recently:
https://bugzilla.mozilla.org/show_bug.cgi?id=1751435

Thus changes in tests.

References #12225
2022-03-01 00:17:33 -08:00
Dmitry Gozman
edac84d072
feat(remote): send browser/channel/headless in a header (#12205) 2022-02-18 07:54:33 -08:00
Dmitry Gozman
47cc7c4ae8
chore: refactor internal cli commands to use commander (#12146) 2022-02-16 09:54:12 -08:00
Dmitry Gozman
66b5cf5ae1
feat(remote): make PlaywrightServer work with browserType.connect (#11849)
This changes PlaywrigtServer to serve connections like `ws://localhost:3333/?browser=chromium`:
- launches the browser;
- talks `browserType.connect`-style protocol over websocket;
- compatible with `connectOptions` fixture.

```js
await playwright.chromium.connect({ wsEndpoint: 'ws://localhost:3333/?browser=chrome' });
```
2022-02-10 16:36:23 -08:00
Dmitry Gozman
284f76357f
test: unflake browsertype-connect.spec (#11994)
Consider the scenario:
- First test starts a remote server, connects to it and does not close.
- Remote server fixture stops the server in teardown. Meanwhile,
  the connected browser did not get notified about disconnect just yet.
- Second test starts and sets up tracing in the old connected browser.
- Tracing fails because the browser now realises it has disconnected.
2022-02-09 17:00:33 -08:00
Max Schmitt
c752b28516
chore: pin production dependencies (#11793) 2022-02-02 11:14:41 +01:00
Caio Agiani
b6b60decdf
fix: typos (#11789) 2022-02-01 11:09:41 -08:00
Dmitry Gozman
30f3cbd6da
fix(fixtures): make sure defaultLaunchOptions are always there (#11713) 2022-01-28 10:51:55 -08:00
Pavel Feldman
29a84b2df8
chore: allow pre-processing scripts (#11702) 2022-01-27 14:32:23 -08:00
Dmitry Gozman
b1fbc4fdbe
fix(test runner): resolve tsconfig for each file (#11662)
This allows us to properly handle path mappings
that are not too ambiguous.
2022-01-26 18:28:42 -08:00
Pavel Feldman
db0cda9df5
chore: rename default config to match extension pattern (#11446) 2022-01-18 13:31:12 -08:00
Pavel Feldman
8ecf581b45
chore: focus by line should override only (#11427) 2022-01-16 08:47:09 -08:00
Pavel Feldman
373042ed89
chore: don't allow importing @playwright/test twice (#11218) 2022-01-06 09:29:05 -08:00
Dmitry Gozman
f77c874e8a
feat(test runner): make _extendTest experimental (#11210)
Hidden from types and docs.
2022-01-05 15:54:00 -08:00
Ross Wollman
fde427d890
feat(proxy): unify local network proxy behavior (#10719)
When configuring a proxy, Chromium requires a magic tokens to get some
local network requests to go through the proxy. This has tripped up a
few users, so we make the behavior default to the expected: proxy
everything including the local requests. This matches the other vendors
as well.

NB: This can be disabled via
`PLAYWRIGHT_DISABLE_FORCED_CHROMIUM_PROXIED_LOOPBACK=1`

Supercedes: #8345
Fixes: #10631
2021-12-10 14:01:56 -08:00
Yury Semikhatsky
976af162b0
feat(tracing): pack sources to trace on the driver side (#10815) 2021-12-09 17:21:17 -08:00
Dmitry Gozman
d9f849fb14
feat(test runner): replace declare/define with "options" (#10293)
1. Fixtures defined in test.extend() can now have `{ option: true }` configuration that makes them overridable in the config. Options support all other properties of fixtures - value/function, scope, auto.
```
const test = base.extend<MyOptions>({
  foo: ['default', { option: true }],
});
```

2. test.declare() and project.define are removed.

3. project.use applies overrides to default option values and nothing else. Any test.extend() and test.use() calls take priority over config options.

Required user changes: if someone used to define fixture options with test.extend(), overriding them in config will stop working. The solution is to add `{ option: true }`.

```
// Old code
export const test = base.extend<{ myOption: number, myFixture: number }>({
  myOption: 123,
  myFixture: ({ myOption }, use) => use(2 * myOption),
});

// New code
export const test = base.extend<{ myOption: number, myFixture: number }>({
  myOption: [123, { option: true }],
  myFixture: ({ myOption }, use) => use(2 * myOption),
});
```
2021-11-18 15:45:52 -08:00
Dmitry Gozman
04dc935265
test: do not print output from child processes in skipped tests (#10422) 2021-11-18 14:35:51 -08:00
Andrey Lushnikov
dc89738233
test: bump expect timeout (#10384)
This should make trace viewer tests less flaky.

References #10383
2021-11-17 11:56:24 -08:00
Max Schmitt
3a64da7a54
fix: do not show taskkill.exe output (#10244) 2021-11-11 18:32:22 +01:00
Pavel Feldman
8991bbde33
feat(html): live filtering, opt-out from auto-open (#9889) 2021-10-29 16:24:08 -07:00
Max Schmitt
21c4435060
test: do kill testProcesses accordingly (#9818)
* test: do kill testProcesses accordingly

* fix it
2021-10-28 23:08:04 +02:00
Pavel Feldman
2e4722d460
test: migrate to upstream fixtures (#9835) 2021-10-28 08:31:30 -07:00
Pavel Feldman
16afb5064f
test: remove browserOptions (#9829) 2021-10-27 19:00:06 -07:00
Pavel Feldman
31faa7d9a7
chore: sort out base fixtures (#9809) 2021-10-27 08:28:53 -07:00
Joel Einbinder
c8addef03a
fix(monorepo): export package.json from every package (#9803) 2021-10-26 18:41:18 -04:00
Pavel Feldman
273122b761
chore: align internal test harness w/ @playwright/test (#9796) 2021-10-26 13:45:53 -07:00
Pavel Feldman
87c64b2c1c
chore: align client side instrumentations (#9771) 2021-10-26 11:13:35 -07:00
Pavel Feldman
5451808df1
chore: append sources to traces (#9736) 2021-10-25 18:56:57 -07:00
Joel Einbinder
f2888395a6
fix: support even older node 12 (#9718) 2021-10-22 15:59:52 -04:00
Pavel Feldman
2b9d7815b2
chore: enable html report by default (#9639) 2021-10-19 21:31:59 -07:00
Joel Einbinder
53c866c0f2
fix(monorepo): explicitly list package.json exports instead of using glob patterns (#9584) 2021-10-19 12:28:02 -04:00
Pavel Feldman
e37660b068
feat(api): expose step location (#9602) 2021-10-18 21:06:18 -07:00
Andrey Lushnikov
01c702adbb
chore: move "service" mode tests to experimental grid (#9531)
This patch moves service tests to use experimental grid.
Test changes:
- `proxy.spec.ts` - proxy setup is not supported at all in service mode.
  This is because we already use proxy for browsers to properly resolve
  localhost.
- `har.spec.ts` - tests disabled since connection port is different.
2021-10-15 16:11:53 -07:00
Pavel Feldman
75cfe5d1f5
chore: start adding html runner e2e tests (#9518) 2021-10-14 15:48:05 -07:00
Max Schmitt
96be17463e
feat(codgen): support positioned clicks in a canvas (#9503) 2021-10-14 17:37:29 +02:00
Joel Einbinder
c8f875a193
chore: move test runner types into the playwright test package (#9472)
Co-authored-by: Max Schmitt <max@schmitt.mx>
2021-10-14 11:55:08 +02:00
Joel Einbinder
c89d5a50dd
chore: migrate to monorepo (#9421) 2021-10-11 16:52:17 +02:00
Max Schmitt
b10140d9c2
chore: add reuse context to internal test suite (#9359) 2021-10-07 23:01:08 +02:00
Pavel Feldman
bc71d20d0f
feat(request): add global request fixture (#9332) 2021-10-06 10:09:27 -07:00
Pavel Feldman
913821f675
chore: roll Electron to 12.2.1 (#9271) 2021-10-01 19:40:47 -07:00
Dmitry Gozman
5633520f45
fix(selenium connect): register in gracefullyCloseAll for driver cleanup (#9218)
Otherwise, killing the driver does not cleanup sessions in the grid.
2021-09-29 14:54:24 -07:00
Max Schmitt
cd22072685
chore: enable object-curly-spacing in ESLint (#9168) 2021-09-27 18:58:08 +02:00
Pavel Feldman
241411ad42
chore: render expect in trace viewer (#9141) 2021-09-27 09:19:59 -07:00
Dmitry Gozman
2afe76d86d
feat(launch): connect to process.env.SELENIUM_REMOTE_URL instead of local launch (#9056) 2021-09-22 21:13:32 -07:00
Dmitry Gozman
018467911b
test: introduce some common test fixtures (#9060) 2021-09-21 16:24:48 -07:00
Dmitry Gozman
43213614a1
fix(test runner): after hooks step should not be nested (#8969) 2021-09-16 15:51:27 -07:00
Pavel Feldman
14bc663742
chrome: don't nest pw:api steps (#8932) 2021-09-15 11:34:23 -07:00
Pavel Feldman
aa76cb5610
fix(tracing): fix tests via tracking context close state (#8906) 2021-09-13 21:12:36 -07:00
Pavel Feldman
1925c85dfb
feat(report): render attachment as a part of failure (#8903) 2021-09-13 18:07:40 -07:00
Pavel Feldman
b76e993951
chore: remove log from htmlreport (#8898) 2021-09-13 18:07:15 -07:00
Pavel Feldman
665143d629
chore(report): don't generate file per test (#8822) 2021-09-09 17:35:31 -07:00
Yury Semikhatsky
81ae790288
feat(fetch): support ignoreHTTPSErrors (#8795) 2021-09-08 20:32:52 -07:00
Dmitry Gozman
e2b092c1a0
feat(html reporter): show log for pw:api and expect steps (#8692) 2021-09-03 13:08:17 -07:00
Dmitry Gozman
2b4a93972d
test: use ToT test runner for non-test-runner tests (#8671) 2021-09-03 11:22:25 -07:00
Dmitry Gozman
0f34cfe339
test: roll test runner to 1.15.0-next-1630432230000 (#8599) 2021-08-31 17:06:32 -07:00
Dmitry Gozman
bf13259fa9
test: roll test runner to aug-27 (#8523) 2021-08-27 16:26:51 -07:00
Max Schmitt
8189de42cf
test: fix package installation tests (#8430) 2021-08-25 13:34:54 +02:00
Dmitry Gozman
7818f5fa0b
test: add a tracing bot that collects a trace for most contexts (#8316) 2021-08-19 19:09:19 -07:00
Yury Semikhatsky
356d69ae3f
test: make borwsercontext-proxy tests use test proxy server (#8318) 2021-08-19 13:34:32 -07:00
Yury Semikhatsky
d42169aba1
test: proxy server fixture, new test for https via http proxy (#8299) 2021-08-19 07:36:03 -07:00
Pavel Feldman
4d7956c70b
chore: roll test runner to ToT (#8291) 2021-08-18 14:32:36 -07:00
Andrey Lushnikov
45f51239a6
chore: roll TestRunner to tip-of-tree (#8168) 2021-08-12 10:03:23 -07:00
Yury Semikhatsky
75981bc0aa
fix(tests): do not throw if page did not produce video (#8112)
Fixes #8103
2021-08-10 01:24:14 -07:00
Pavel Feldman
40fb9d85e0
feat(test-result): render image diff (#8061) 2021-08-07 15:47:03 -07:00
Pavel Feldman
559d6242f0
chore: roll test runner to ToT (#8063) 2021-08-07 02:11:01 -07:00
Pavel Feldman
efb4af8489
chore: roll test runner to ToT (#8025) 2021-08-06 16:58:08 -07:00
Pavel Feldman
a8d404cd29
feat(test-runner): basic html reporter (#7994) 2021-08-05 13:36:47 -07:00
Max Schmitt
869f8d541b
chore: PlaywrightClient/Server enhancements (#7980)
- ensure timeout results in a meaningful message
- add onDisconnect handler
2021-08-04 19:45:33 +02:00
Pavel Feldman
542019f3a1
chore: roll test runner to ToT (#7977) 2021-08-03 22:19:27 -07:00
Pavel Feldman
5803035c1b
feat(test-runner): introduce steps (#7952) 2021-08-02 17:17:20 -07:00
Pavel Feldman
07c1c4bfb1
chore: roll test runner to tot (#7912) 2021-07-29 09:42:42 -07:00
Pavel Feldman
b8cfce3b9d
chore: roll test runner to ToT (#7896) 2021-07-28 17:33:14 -07:00
Dmitry Gozman
642d6d4459
test: roll test-runner to 1.13.0-next-1626706866000 (#7719) 2021-07-19 09:10:34 -07:00
Max Schmitt
bc2f161a55
chore: roll test-runner to 1.13.0-next-1625774143000 (#7519) 2021-07-08 23:36:11 -07:00
Max Schmitt
b846ddda04
feat(download): introduce Download.cancel (#7462) 2021-07-06 09:38:50 +02:00
Max Schmitt
701624c484
chore: roll playwright-test to 1.13.0-next-1624575246000 (#7291) 2021-06-25 22:53:14 +02:00
Max Schmitt
4c6deaa449
test: added tests for WebSocket over Socks proxy (#7235) 2021-06-24 18:50:16 +02:00
Yury Semikhatsky
c0cd2d4579
feat: response interception (#7122) 2021-06-18 11:04:48 -07:00
Pavel Feldman
82a50b0e1d
fix(test-runner): property handle artifacts in context of preserveOutput (#7181) 2021-06-16 16:05:30 -07:00
Andrey Lushnikov
34d151a1f9
devops: introduce Docker tests (#7157)
Fixes #7139
2021-06-15 23:59:20 -07:00
Max Schmitt
604cf2b89f
roll(test-runner): to 1.13.0-next-alpha-jun-14-2021 (#7111) 2021-06-14 20:30:42 +02:00
Peng-Yu Chen
5f6d4a7b73
feat(download): adding a new Download._cancel method (#6236) 2021-06-12 22:23:22 +02:00
Dmitry Gozman
eb01844710
chore(tests): roll stable test runner to jun-8-2021 (#6951) 2021-06-08 08:35:04 -07:00
Dmitry Gozman
f745bf1fbc
chore: bring in folio source (#6923)
- Source now lives at `src/test`.
- Former folio tests live at `tests/playwright-test`.
- We use `src/test/internal.ts` that exposes base test without
  Playwright fixtures for most tests (to avoid modifications for now).
- Test types live in `types/testFoo.d.ts`.
- Stable test runner is installed to `tests/config/test-runner` during `npm install`.
- All deps including test-only are now listed in `package.json`.
  Non-test deps must also be listed in `build_package.js` to get included.
2021-06-06 17:09:53 -07:00
Dmitry Gozman
82041b2f74
test: roll to folio@0.4.0-alpha28 (#6918) 2021-06-04 20:54:58 -07:00
Dmitry Gozman
69b734629c
fix: various test-related fixes (#6916)
- Closing inspector window resumes the script.
- Replace FOLIO_WORKER_INDEX with TEST_WORKER_INDEX.
- Account for `@playwright/test` stack traces.
2021-06-04 18:43:54 -07:00
Dmitry Gozman
21b00d0bcc
test: roll to folio@0.4.0-alpha27 (#6897) 2021-06-03 22:06:59 -07:00
Max Schmitt
064150f8dd
chore: use fs.promises API instead of promisify (#6871) 2021-06-03 18:55:33 +02:00
Pavel Feldman
b2143a951b
chore: make tracing zero config (#6859) 2021-06-02 22:00:34 -07:00
Max Schmitt
c09726b023
test: add tests for port-forwarding via playwrightclient (#6860)q 2021-06-03 02:19:01 +02:00
Dmitry Gozman
8fefac9b8c
test: roll to folio@0.4.0-alpha21 (#6789) 2021-05-28 17:03:18 -07:00
Andrey Lushnikov
30e5681b82
chore: report correct browser channel for Android tests (#6733) 2021-05-24 19:06:46 -07:00
Dmitry Gozman
57f3a53a7e
test: roll to folio@0.4.0-alpha16 (#6656) 2021-05-20 09:51:09 -07:00
Dmitry Gozman
4c3bd11820
test: roll to folio@0.4.0-alpha14 (#6602) 2021-05-16 19:58:26 -07:00
Dmitry Gozman
5e84eade85
test: roll to folio@0.4.0-alpha13 (#6570) 2021-05-15 07:42:35 -07:00
Dmitry Gozman
8b6b894dd8
test: prepare test to use options as passed (#6557)
This changes `headful` to `headless` to align with launch options.
Also replaces `isChromium` and friends with `browserName`.
2021-05-13 10:22:23 -07:00
Pavel Feldman
21cb726b7d
chore(tracing): expose tracing api (#6523) 2021-05-12 12:21:54 -07:00
Dmitry Gozman
b01ccc282f
test: roll to folio@0.4.0-alpha11 (#6496) 2021-05-11 06:40:06 -07:00
Dmitry Gozman
76e409637a
test: simplify more tests (#6471) 2021-05-09 17:47:20 -07:00
Dmitry Gozman
2d4538c23d
test: cleanup tests and configs after last folio update (#6463) 2021-05-08 17:45:04 -07:00
Dmitry Gozman
573327b7b5
test: roll to folio@0.4.0-alpha8 (#6451) 2021-05-07 15:25:55 -07:00
Dmitry Gozman
b29b7df47e
fix(connect): handle disconnect in various situations (#6276)
There are a few ways for `connect()` to finish:
- `Browser.close()` from the client side.
- Browser on the server side did exit (e.g. crashed).
- Connection was dropped by either of the sides.

We reduce all the cases to the last one by dropping the
connection when client wants calls `Browser.close()` or
server-side browser exits.

In all these cases we should properly cleanup on the server side,
and ensure that all promises reject on the client side.
2021-05-06 09:34:06 -07:00
Dmitry Gozman
217cbe3e21
test: cleanup bad usages of pageTest (#6430) 2021-05-06 07:08:22 -07:00
Yury Semikhatsky
86df1df8e7
test: update download test expectations (#6394) 2021-05-04 11:59:39 -07:00
Pavel Feldman
1a859ebe68
chore(electron): fix node/browser race conditions, expose browser window asynchronously (#6381) 2021-05-02 22:45:06 -07:00
Dmitry Gozman
de21a94b75
test: roll to folio@0.4.0-alpha6 (#6366) 2021-04-30 13:26:13 -07:00
Joel Einbinder
6ce56dde49
test(accessibility): remove and update tests for new chromium (#6372) 2021-04-30 10:19:37 -07:00
Pavel Feldman
934bc672f7
test(tracing): start adding tracing tests (#6369) 2021-04-30 07:40:22 -07:00
Dmitry Gozman
11882cdda7
test: roll to folio@0.4.0-alpha3 (#6262) 2021-04-29 11:11:32 -07:00
Pavel Feldman
a9219aa8b6
chore: start / stop context tracing (#6309) 2021-04-23 20:39:09 -07:00
Yury Semikhatsky
f63f92be5e
chore: repair run_static_server.js (#6298) 2021-04-23 11:42:54 -07:00
Dmitry Gozman
fff1f3d45c
chore: simplify remote connection protocol (#6164)
This changes the root object from RemoteBrowser to Playwright,
similar to local driver connection. This way, any remote connection
gets a Playwright object.

This also starts reusing PlaywrightServer class, and introduces
`cli run-server` hidden command that runs ws server on the
specified port.

Previous structure:
```
RemoteBrowser
  - browser (using ConnectedBrowser for remote-specific behavior)
  - selectors (special instance for this remote connection)
```

New structure:
```
Playwright
  - ...
  - selectors (special instance for this remote connection)
  - preLaunchedBrowser (using ConnectedBrowser for remote-specific behavior)
```
2021-04-12 11:14:54 -07:00
Dmitry Gozman
bd8433ba49
test: cleanup various testing env variables (#6155) 2021-04-09 07:59:09 -07:00
Andrey Lushnikov
856ced6e65
tests: attribute electron tests to electron on the dashboard (#6156) 2021-04-08 23:19:36 -07:00
Joel Einbinder
779355ad51
feat(types): make the template on BrowserType optional (#6142)
This makes it much nicer to use `BrowserType` because it no longer has a template.

Technically a breaking change because of the rare edge case where someone used their own non-browser type inside the template, but I don't consider that intended behavior and think this is fine.
2021-04-08 10:27:24 -07:00
Dmitry Gozman
310692b101
test: run page tests on electron bot (#6122) 2021-04-08 10:26:26 -07:00
Pavel Feldman
d9546fd098
chore: read all traces from the folder (#6134) 2021-04-08 07:59:05 -07:00
Andrey Lushnikov
632ff111d4
test: properly annotate Android tests for flakiness dashbboard (#6133) 2021-04-07 19:58:49 -07:00
Dmitry Gozman
ee44fbe2bc
devops: upload test results for android bots (#6121) 2021-04-07 08:07:31 -07:00
Andrey Lushnikov
2897df693f
devops: restore flakiness dashboard upload (#6116) 2021-04-06 22:46:52 -07:00
Dmitry Gozman
36a546999e
test: roll to folio 0.3.21-alpha (#6108) 2021-04-06 16:09:54 -07:00
Andrey Lushnikov
ba5ba52e41
test: expose browserVersion in the tests (#6090)
Drive-by: expose browser version in test report as well.
2021-04-05 22:22:07 -07:00
Dmitry Gozman
63e471ca22
test: cleanup proxy and context tests (#6085) 2021-04-05 18:48:46 -07:00
Andrey Lushnikov
1a44f68155
devops: migrate flakiness dashboard to the new folio reporter format (#6089)
New folio changed the JSON report, so we have to keep with the changes.

The most notable changes:
- there are no parameters any more. We recreate these as `testInfo.data`
  arguments that are saved under test result's data.
- `test.runs` are renamed into `test.results`

I didn't find other changes so far - let's see if this works in the
cloud!
2021-04-05 17:35:34 -07:00
Dmitry Gozman
5afe282fbb
test: move remaining files from old test/ directory (#6081) 2021-04-05 15:51:45 -07:00
Dmitry Gozman
e3cf675624
test: remove a copy of folio, use upstream (#6080) 2021-04-05 13:23:49 -07:00
Dmitry Gozman
bd61f863c4
test: print "Using executable at ..." for custom executable (#6077) 2021-04-05 09:19:39 -07:00
Dmitry Gozman
4f7e7450e2
test: migrate last tests to new folio (#6071) 2021-04-05 09:18:56 -07:00
Dmitry Gozman
f21f47889e
test: migrate more page tests to folio (#6062) 2021-04-04 19:32:14 -07:00
Dmitry Gozman
f5781f90ba
test: migrate most non-page tests to new folio (#6059) 2021-04-02 21:07:45 -07:00
Dmitry Gozman
0629922774
test: migrate page tests to new folio (#6054) 2021-04-02 14:23:42 -07:00
Dmitry Gozman
d0afa9d8de
test: migrate cli tests to new folio (#6048) 2021-04-02 11:19:26 -07:00
Dmitry Gozman
1444cc873a
test: migrate electron tests to new folio (#6043) 2021-04-02 09:00:49 -07:00
Dmitry Gozman
a4eb4c0baf
test: migrate remoteServer tests to new folio (#6042) 2021-04-01 19:13:08 -07:00
Dmitry Gozman
be79b3883b
test: bring new folio and migrate small amount of tests to it (#5994) 2021-04-01 16:35:26 -07:00