microsoft/playwright
1
1
Fork 0
mirror of https://github.com/microsoft/playwright.git synced 2024-11-28 09:23:42 +03:00
Code Issues Projects Releases Wiki Activity

docs(contributing): add info about skip and fail (#1944)

Browse Source 
When contributing some test cases, I was a bit confused about if I should use `skip` and `fail`, the difference between the two, and how they interact with CI/CD tests passing or failing: https://github.com/microsoft/playwright/pull/1939#discussion_r413946012.

This is my attempt at explaining the difference, but feel free to edit to make it more clear and concise (and correct any inaccuracies).
This commit is contained in:
Ross Wollman 2020-04-23 14:28:40 -07:00 committed by GitHub
parent 05f0797211
commit fd17cfb5c1
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 20 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
CONTRIBUTING.md
View File

@ -198,6 +198,26 @@ HEADLESS=false SLOW_MO=500 npm run wtest
BROWSER=chromium node --inspect-brk test/test.js
```
- When should a test be marked with `skip` or `fail`?
- **`skip(condition)`**: This test *should ***never*** work* for `condition`
where `condition` is usually a certain browser like `FFOX` (for Firefox),
`WEBKIT` (for WebKit), and `CHROMIUM` (for Chromium).
For example, the [alt-click downloads test](https://github.com/microsoft/playwright/blob/471ccc72d3f0847caa36f629b394a028c7750d93/test/download.spec.js#L86) is marked
with `skip(FFOX)` since an alt-click in Firefox will not produce a download
even if a person was driving the browser.
- **`fail(condition)`**: This test *should ***eventually*** work* for `condition`
where `condition` is usually a certain browser like `FFOX` (for Firefox),
`WEBKIT` (for WebKit), and `CHROMIUM` (for Chromium).
For example, the [alt-click downloads test](https://github.com/microsoft/playwright/blob/471ccc72d3f0847caa36f629b394a028c7750d93/test/download.spec.js#L86) is marked
with `fail(CHROMIUM || WEBKIT)` since Playwright performing these actions
currently diverges from what a user would experience driving a Chromium or
WebKit.
## Public API Coverage
Every public API method or event should be called at least once in tests. To ensure this, there's a `coverage` command which tracks calls to public API and reports back if some methods/events were not called.