Joel Einbinder
040c6a6a41
chore(jest): run tests with jest ( #2754 )
...
Experimentally run `npx jest` to run our tests with jest.
2020-07-08 11:13:18 -07:00
Andrey Lushnikov
760283ea4e
testrunner: fix default environment name ( #2870 )
2020-07-07 22:14:04 -07:00
Andrey Lushnikov
05b019f1ba
reland: testrunner: make environment a simple class ( #2812 )
...
This re-lands PR https://github.com/microsoft/playwright/pull/2769
It was reverted before in https://github.com/microsoft/playwright/pull/2790
because it was breaking the new CHANNEL bot.
2020-07-02 11:05:38 -07:00
Pavel Feldman
e467ea57e7
revert: testrunner: make environment a simple class ( #2769 ) ( #2790 )
2020-07-01 10:56:19 -07:00
Andrey Lushnikov
1605cb453c
testrunner: make environment a simple class ( #2769 )
...
This patch:
- makes environment a simple class with optional methods `beforeEach`, `afterEach`, `beforeAll`, `afterAll`, `globalSetup` and `globalTeardown`
- removes capability to have multiple hooks of the same name inside suite
- removes default environment for test. (`dit` now adds a `TraceTestEnvironment` to the test)
- extracts all environments that we use in our tests in `//test/environments.js`
Downsides:
- we no longer know hook locations for the environments. This, however, should not be a big deal since stack traces (if any) will still point into it.
- this also regresses hook locations for suites for simplicity. We can get them back, but it shouldn't be pressing since we now have only one hook of each kind in every suite.
2020-06-30 16:51:03 -07:00
Andrey Lushnikov
1865c5685a
testrunner: support globalSetup and globalTeardown hooks. ( #2686 )
2020-06-24 22:08:46 -07:00
Andrey Lushnikov
07dbd0baae
testrunner: teach runHook to accept hook arguments ( #2682 )
2020-06-23 18:17:14 -07:00
Andrey Lushnikov
c61e2d6cc9
testrunner: drop nested test environments ( #2681 )
2020-06-23 16:45:00 -07:00
Dmitry Gozman
1d37a10558
chore: migrate navigations to Progress ( #2463 )
2020-06-04 16:43:48 -07:00
Dmitry Gozman
963dc72dd2
devops: add headful linux bot ( #2060 )
2020-05-04 15:15:51 -07:00
Andrey Lushnikov
a9f0c40af3
feat(testrunner): improvements to testrunner ( #2070 )
...
This patch:
- teaches test runner to understand custom argument spelling, e.g. `--file=evalu` and `-j10`
- fixes `--file` filter to actually focus file paths instead of focusing
all tests with given path
2020-05-01 10:09:45 -07:00
Andrey Lushnikov
2d68830411
feat(testrunner): support --file to filter tests by test file name. ( #1903 )
2020-04-21 16:47:38 -07:00
Dmitry Gozman
cf82e2c945
fix(testrunner): await terminations before reporting test results ( #1855 )
...
This way we ensure that all errors are picked up.
2020-04-17 18:42:12 -07:00
Andrey Lushnikov
51b8685a8c
feat(testrunner): support --repeat CLI flag to repeat tests ( #1828 )
...
This allows you to run `npm run cunit -- --repeat 10` to run
tests multiple times.
2020-04-16 18:09:25 -07:00
Andrey Lushnikov
167d265e56
fix(testrunner): make .repeat() retain test order ( #1814 )
2020-04-15 18:41:55 -07:00
Dmitry Gozman
a3571c24c4
fix(testrunner): respect timeout=0 in hooks ( #1764 )
2020-04-13 14:30:51 -07:00
Dmitry Gozman
29a6cdff6b
fix(tests): fix a race with golden setup ( #1757 )
...
When running multiple browsers, we should take the golden information from the state.
2020-04-13 14:12:44 -07:00
Dmitry Gozman
2d57fff0be
fix(tests): fix multiple browsers tests ( #1718 )
...
- Use a server only once, not once per browser.
- Avoid require cache for test files.
- Improve testrunner hooks debugging.
2020-04-08 15:54:17 -07:00
Dmitry Gozman
2ef8e26602
test: structure tests to use environments, closer to end user ( #1713 )
2020-04-08 14:17:34 -07:00
Dmitry Gozman
20ff327827
feat(testrunner): catch delegate errors ( #1704 )
...
This ensures we get a proper error when something goes wrong. Should
also help with producing the right error code in the case of internal error.
Drive-by: fix location issue which manifests on the bots.
Drive-by: remove the use of Array.prototype.flat to make it work on bots.
2020-04-07 22:56:21 -07:00
Dmitry Gozman
39e06f0286
feat(testrunner): improve reporting of unhandled errors/rejections ( #1697 )
2020-04-07 16:02:33 -07:00
Dmitry Gozman
cd39053ccc
feat(testrunner): make it easier to setup golden matcher ( #1682 )
2020-04-06 18:01:56 -07:00
Dmitry Gozman
e519a3d762
fix(testrunner): better capture Location for hooks ( #1680 )
2020-04-06 17:47:17 -07:00
Dmitry Gozman
f2b13c0e93
chore(testrunner): split TestRunner into parts ( #1679 )
2020-04-06 17:21:42 -07:00
Dmitry Gozman
823f961d8d
feat(testrunner): migrate from events to a delegate ( #1647 )
...
This allows an async handler for each event that can be awaited.
Drive-by: merge TestPass into TestRunner.
2020-04-03 15:47:25 -07:00
Dmitry Gozman
1f2803bbc6
feat(testrunner): removeEnvironment ( #1650 )
2020-04-03 09:48:01 -07:00
Dmitry Gozman
d38baae01c
feat(testrunner): nested environments ( #1635 )
2020-04-02 13:12:22 -07:00
Dmitry Gozman
f87e64544c
feat(testrunner): introduce environments ( #1593 )
2020-04-01 10:49:47 -07:00
Dmitry Gozman
b6166c9669
chore(testrunner): introduce Location class ( #1585 )
...
Drive-by: fix an edge when testing continued after termination.
2020-03-29 21:38:30 -07:00
Dmitry Gozman
c49b856d6d
chore(testrunner): remove setup() helper ( #1584 )
2020-03-29 11:37:45 -07:00
Dmitry Gozman
a41836b1f1
chore(testrunner): introduce TestRun ( #1582 )
2020-03-28 14:25:57 -07:00
Dmitry Gozman
b85ab891a1
chore(testrunner): make most modifiers external ( #1581 )
2020-03-28 08:49:00 -07:00
Dmitry Gozman
4bd46bafa9
feat(testurnner): allow multiple hooks isntances and per-test hooks ( #1571 )
2020-03-27 22:57:22 -07:00
Dmitry Gozman
a042466125
chore(testrunner): remove effectiveMode and effectiveExpectation ( #1569 )
2020-03-27 15:35:14 -07:00
Dmitry Gozman
af7a16c360
chore(testrunner): merge test spec with test, suite spec with suite ( #1566 )
2020-03-26 22:47:13 -07:00
Dmitry Gozman
aad82e00bd
chore(testrunner): decouple UserCallback from location and timeout ( #1557 )
...
This will make it easier to change lifetimes of Test and Suite.
2020-03-26 14:43:28 -07:00
Dmitry Gozman
81bd8de00b
feat(testrunner): composable and bindable attributes and modifiers ( #1547 )
...
This allows to make things like `fit`, `skip()` and soon even `dit` be implemented externally.
2020-03-25 22:42:09 -07:00
Dmitry Gozman
b61198458c
feat(testrunner): modifiers and attributes ( #1538 )
...
This generalizes test modifiers to support custom ones.
2020-03-25 14:40:57 -07:00
Dmitry Gozman
a2ee7a1f2a
fix(testrunner): do not spam output after termination ( #1511 )
2020-03-24 14:40:59 -07:00
Pavel Feldman
c0c9b7f137
test: make debugp collect IO (take 2) ( #1493 )
2020-03-23 15:08:02 -07:00
Dmitry Gozman
63906454a7
fix(testrunner): attribute unhandle promise reject to a single worker ( #1491 )
...
When there is a single worker, we are almost sure the error originated in that worker. Attributing it helps with context by showing last run tests.
2020-03-23 14:45:21 -07:00
Pavel
1ddf05113b
Revert "test: make debugp collect IO ( #1485 )"
...
This reverts commit b1bebdad2e
2020-03-23 13:48:22 -07:00
Pavel Feldman
b1bebdad2e
test: make debugp collect IO ( #1485 )
2020-03-23 13:42:51 -07:00
Andrey Lushnikov
a9ab9b0dc0
fix(testrunner): sourcemapify stack traces for test errors ( #1409 )
...
Fixes #1403
2020-03-17 09:04:44 -07:00
Dmitry Gozman
3960b179a1
fix(testrunner): fit.fail should run the test ( #1407 )
2020-03-16 19:12:52 -07:00
Andrey Lushnikov
5816ec53f7
fix(testrunner): dedup focused tests and suites by id ( #1393 )
2020-03-16 09:04:55 -07:00
Dmitry Gozman
e7eeefe4c7
chore(testrunner): separate expectations from run mode ( #1395 )
...
Run/Focus/Skip is orthogonal to expect to Pass/Fail/Flake.
This change separates the two, in a preparation to run Fail/Flaky tests.
2020-03-15 23:10:49 -07:00
Dmitry Gozman
0d7cb29329
test: continue running tests after crash, report crashes separately ( #1362 )
2020-03-12 17:32:53 -07:00
Andrey Lushnikov
704fe6dc73
fix(testrunner): fix reporting focused tests
2020-03-11 18:53:24 -07:00
Andrey Lushnikov
1cd00bd068
feat(testrunner): allow filtering by name and show all focused tests ( #1354 )
2020-03-11 18:30:43 -07:00
