Commit Graph

7214 Commits

Author SHA1 Message Date
Andrey Lushnikov
d744a87aee
fix(test-runner): rely on test title paths instead of ordinal (#12414)
Fixes #11904
2022-02-28 15:40:23 -08:00
Max Schmitt
cb41e668fe
devops: fix test runner flakiness uploads (#12415) 2022-03-01 00:37:37 +01:00
Dmitry Gozman
73dee69558
fix(hooks): separate test timeout from beforeAll/afterAll timeouts (#12413)
This makes it possible to have longer `beforeAll`/`afterAll` and not
affect first/last test timeout.
2022-02-28 15:09:04 -08:00
Mateusz Burzyński
eaa98ce53a
feat(keyboard): support simple copy-pasting using meta+c/v (#10828)
It's a straightforward change to support new, common, keyboard commands

Note that I've tested this locally with Chrome on my Mac but it seems that CI doesn't want to pass Chrome tests - it's running on ubuntu though. Does this mean that I should introduce per-platform editing commands? At the moment there is only a single [`macEditingCommands`](0ed33522c5/packages/playwright-core/src/server/macEditingCommands.ts) file.

References https://github.com/microsoft/playwright/issues/12000

Co-authored-by: Andrey Lushnikov <aslushnikov@gmail.com>
2022-02-28 13:43:43 -08:00
Andrey Lushnikov
f47423d315
devops: upload test-runner test results to flakiness dashboard (#12412)
Fixes #12152
2022-02-28 13:41:30 -08:00
Andrey Lushnikov
396d920145
feat(test-runner): implement expect(pageOrLocator).toHaveScreenshot (#12242)
Fixes #9938
2022-02-28 12:25:59 -08:00
Andrey Lushnikov
8d94ed134c
fix: add snapshots for page and elementHandle screenshots (#12411)
Fixes #12280
2022-02-28 11:52:54 -08:00
Dmitry Gozman
47045ba48d
chore(test runner): run hooks/modifiers as a part of the test (#12329)
chore(test runner): run hooks/modifiers as a part of the test

This moves `beforeAll`, `afterAll` and some modifiers from running
as a separate entity into running inside a test.

Pros:
- All errors are reported as test errors.
- All artifacts are collected as test artifacts.
- Reporters support this out of the box.

Details:
- Each test computes the necessary hooks to run and runs them.
- Teardown is usually performed during the test (on test failure or worker stop).
- `skipRemaining` is added to `DonePayload` to preserve the behavior
  where `beforeAll` hook failure skips subsequent tests.
  This behavior can now be improved to only target tests affected by this hook.
2022-02-28 11:42:47 -08:00
Diego Pino
76dedeab15
browser(webkit): Fix build for Ubuntu 18.04 (#12368) 2022-02-28 09:15:39 -08:00
github-actions[bot]
90f1619812
browser(chromium): roll to r975608 (#12395)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-28 11:16:20 +01:00
github-actions[bot]
80a38a39c2
feat(chromium): roll to r974924 (#12377)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-26 07:52:27 +01:00
Max Schmitt
41e9b36f63
fix: install-deps did not throw on non 0 exit code (#12375) 2022-02-26 00:45:27 +01:00
Dmitry Gozman
3c2bca2768
fix(test loader): implement tsconfig paths through resolveFilename (#12357)
This uses `Module._resolveFilename` to intercept module resolution and
check `tsconfig.paths` similarly to pirates usage ot `Module._compile`.

Previously, we resolved during compilation that required reproducible
resolution due to caching. Now we can resolve as we go and support
all `tsconfig.paths`.
2022-02-25 15:43:58 -08:00
Max Schmitt
6438aed36c test: skip test in electron
Follow-up for #12371
2022-02-26 00:17:37 +01:00
Max Schmitt
e035ed488c
test: add test for cookies with expiration (#12304) 2022-02-25 23:00:51 +01:00
Max Schmitt
b8728248fa
docs: merge assertions with verification doc (#12361) 2022-02-25 22:58:41 +01:00
Max Schmitt
a95d2320ad
test: mark failing android tests as fixme / fix them (#12371) 2022-02-25 22:56:51 +01:00
Max Schmitt
2f0183dd1d
docs(cli): add --width-deps to the docs (#12362) 2022-02-25 19:26:32 +01:00
Max Schmitt
c1a1eaec34
docs(trace-viewer): viewing remote traces on trace.playwright.dev (#12363) 2022-02-25 19:23:39 +01:00
Max Schmitt
8292fe8940
devops: run test-runner tests on Node 16 (#12359) 2022-02-25 19:13:45 +01:00
nswbmw
b47c95c20c
feat: add iPhone 12 Mini (#11429)
Co-authored-by: Max Schmitt <max@schmitt.mx>
2022-02-25 12:15:58 +01:00
github-actions[bot]
3c8222056a
browser(chromium): roll to r974924 (#12358)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-25 11:18:52 +01:00
Max Schmitt
821a8e93c7
chore: do not double trim in trace viewer (#12338) 2022-02-24 21:55:50 +01:00
Dmitry Gozman
91672595f2
fix(reporters): normalize usage of isTTY, env.CI and debug env variables (#12295)
- `stdout.isTTY` controls whether list reporter updates lines or just adds them;
- `env.CI` is used in a few places to affect the defaults:
  - whether to open interactive html;
  - default reporter dot/line;
  - default terminal reporter added to non-terminal reporters;
- `env.PWTEST_SKIP_TEST_OUTPUT` is removed;
- `env.PW_TEST_DEBUG_REPORTERS` is introduced specifically for tests.
2022-02-24 12:39:28 -08:00
Max Schmitt
428a014cad
devops: Docker publishing cleanups (#12341) 2022-02-24 18:10:34 +01:00
Max Schmitt
0b55950d6a
chore: fix Android tests (#12325) 2022-02-24 06:57:32 +01:00
Andrey Lushnikov
71edad327b
fix: tolerate EBUSY error when removing output directories (#12323)
- Previous attempt: https://github.com/microsoft/playwright/pull/12300
- Revert with reasoning: ebc52d10e4

Fixes #12106
2022-02-23 14:23:57 -08:00
Andrey Lushnikov
df7fbe7857
Revert "chore: best-effort cleanup for output folders that are mounted (#12300)" (#12321)
This reverts commit e5c9d1e39f.

Reason for revert: turns out this fix results in a 5-second delay
when starting tests in docker, with `test-results` folder being
a non-removable mount.

The reason for the delay is the `maxBusyTries` option that we
supply by default to rimraf when trying to remove the folder.

While this option might come handy when removing temporary
browser profile folder, it doesn't serve us well in this particular
usecase.

References #12106
2022-02-23 14:10:11 -08:00
Andrey Lushnikov
5879c7f362
chore: refactor toMatchSnapshot once again (#12313)
Keep massaging code in preparation for `toHaveScreenshot`.

References #9938
2022-02-23 13:17:37 -08:00
Andrey Lushnikov
e5c9d1e39f
chore: best-effort cleanup for output folders that are mounted (#12300)
Fixes #12106
2022-02-23 12:55:41 -08:00
Dmitry Gozman
4399623f9f
chore(test runner): make timeout a separate error in TestInfo.errors (#12315)
This way we control the timeout error message from the runner,
so that later on we can differentiate between test timeout, fixture
timeout and hook timeout.
2022-02-23 12:32:12 -08:00
Max Schmitt
5be9f68446
docs: fix broken docs by wrong snippet lang (#12306) 2022-02-23 14:35:40 +01:00
Diego Pino
290410e44f
browser(webkit): rebase to 02/18/22 (#12277)
Rebase `webkit/patches/boostrap.diff` to [r290138](https://trac.webkit.org/changeset/290138/webkit).

This likely solves the reported missing libOpenGL library in the CI bot.
2022-02-22 22:01:09 -08:00
Dmitry Gozman
26d6537119
docs: document environment variables usage (#12294) 2022-02-22 15:24:35 -08:00
Dmitry Gozman
114edecd3a
fix(list mode): keep outputDir intact (#12291) 2022-02-22 12:50:26 -08:00
Ben Lu
f933f65d03
feat(html): put diff first in image comparison (#12254) 2022-02-22 11:29:58 -08:00
Max Schmitt
b08000e7c2
docs: advertise how to enable sources with tracing (#12273) 2022-02-22 20:16:05 +01:00
Max Schmitt
52932172e7
docs(python): fix set_viewport_size in emulation.md (#12271) 2022-02-22 20:15:46 +01:00
Max Schmitt
be2e4866b0
test: add test for sendBeacon and asserting request body (#12274) 2022-02-22 20:15:24 +01:00
Max Schmitt
17a922cf5e
docs: fix expect custom matcher global.d.ts (#12268) 2022-02-22 11:09:52 -08:00
github-actions[bot]
cbe075135c
feat(chromium): roll to r973391 (#12279)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-22 11:33:25 +01:00
Max Schmitt
768242db12
docs(dotnet): enable web-first assertions (#12113) 2022-02-21 14:01:53 +01:00
github-actions[bot]
29a2354df8
browser(chromium): roll to r973391 (#12265)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-21 11:30:39 +01:00
github-actions[bot]
222033d6b2
feat(chromium): roll to r972766 (#12248)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2022-02-21 11:30:30 +01:00
Pavel Feldman
b8cf5ac661
fix(electron): do not attach external debugger when running Electron tests (#12250) 2022-02-19 17:46:26 -08:00
Andrey Lushnikov
fe57771869
chore: update package.json (#12246)
These changed after 0682672242
2022-02-18 22:15:48 -08:00
Andrey Lushnikov
18fdf927b7
chore: simplify screenshot option handling (#12245)
#9938
2022-02-18 21:34:56 -08:00
Yury Semikhatsky
a16eaf584a
docs: mock guide (#12241) 2022-02-18 18:28:03 -08:00
Dmitry Gozman
d3c4323021
fix(test runner): improve error message for unexpected calls (#12240) 2022-02-18 18:25:18 -08:00
Dmitry Gozman
ee0dd6ec71
fix(test runner): trim full output path (#12239)
Previously, we only trimmed the "full title" component, but
we should also trim the "relative path" component.
2022-02-18 15:40:36 -08:00