Commit Graph

106 Commits

Author SHA1 Message Date
Dmitry Gozman
39f92e82f6
test: limit workers to one for Firefox on Windows (#27514) 2023-10-09 12:34:59 -07:00
Max Schmitt
0efa4a90cc
devops: Node.js 16 -> Node.js 18 (#27467)
As per
https://github.com/microsoft/playwright-internal/blob/main/INFRASTRUCTURE.md.


Fixes https://github.com/microsoft/playwright/issues/27148.
2023-10-05 23:08:43 +00:00
Dmitry Gozman
dfc75a1e1a
test: speed up installation tests by not isolating browser unless required (#27370) 2023-10-02 11:26:08 -07:00
Max Schmitt
5811d65b9d
Revert "chore: pin node to 20.5 to work around https://github.com/nod… (#26979)
…ejs/node/issues/49497 (#26921)"

This reverts commit 874e285adf.
2023-09-11 22:33:09 +02:00
Dmitry Gozman
7c0766a273
devops: install pnpm and yarn for secondary installation tests (#26968) 2023-09-08 13:54:05 -07:00
Pavel Feldman
874e285adf
chore: pin node to 20.5 to work around https://github.com/nodejs/node/issues/49497 (#26921)
Works around https://github.com/nodejs/node/issues/49497
2023-09-06 16:38:19 -07:00
Yury Semikhatsky
c344fce7b6
devops: simplify upload-blob-report action (#26548) 2023-08-18 12:16:26 -07:00
Yury Semikhatsky
31652bca59
devops: merge all "tests 2" reports (#26547) 2023-08-18 12:06:17 -07:00
Dmitry Gozman
27c15b705d
fix(blob): replace projectSuffix with reportName (#25017)
Always ensure unique project/test ids across blobs.
Show `reportName` as a label in the html report.

References #24451.
2023-08-07 13:38:09 -07:00
Yury Semikhatsky
a0211924d3
devops: upload blob reports for headed and transport tests (#26313)
This will add another (3*4 + 2)*3k = 42k tests to the combined report
and should give us 100k+ tests in total per report.
2023-08-06 11:10:12 -07:00
Pavel Feldman
ed83d04d3e chore: install deps for grid tests (3) 2023-08-04 18:05:32 -07:00
Pavel Feldman
1fdb8bdbe4 chore: install deps for grid tests (2) 2023-08-04 16:41:41 -07:00
Yury Semikhatsky
4494137814
devops: unique report names (#24596) 2023-08-03 15:34:40 -07:00
Dmitry Gozman
d92fe16b76
fix(blob report): default location relative to package.json (#24481)
Also:
- remove `blob-report` directory at the start;
- markdown's `report.md` next to package.json;
- use default location in playwright's workflows.

References #24451.
2023-07-28 15:49:31 -07:00
Yury Semikhatsky
e036603aa3
devops: upload blob report only for some bots in tests 2 (#24353)
For now merge-reports chokes on that many events and we end up with no
report at all.
2023-07-26 12:33:14 -07:00
Yury Semikhatsky
8b454cb3c3
devops: merge reports fall pw tests in tests 2 (#24309) 2023-07-19 13:01:52 -07:00
Yury Semikhatsky
d92db9a513
devops: always create blob report on CI, write PR number within action (#24241) 2023-07-14 15:10:29 -07:00
Yury Semikhatsky
1b1cf87e0a
devops: add upload report action (#24222)
* Fix report downloading from Azure (reports are now zipped)
* Extracted upload logic into an action
* Extracted PR number file generation into its own job
2023-07-14 08:21:15 -07:00
Yury Semikhatsky
51b8f609fb
Revert "devops: zip blob report artifact before uploading (#23667)" (#23687)
This reverts commit a1cdae6bff.

The problem with this approach is that each job overwrites the zip
artifact whereas previously it was merging all reports in the same
directory. We are going to zip .jsonl files instead.
2023-06-13 12:36:04 -07:00
Yury Semikhatsky
a1cdae6bff
devops: zip blob report artifact before uploading (#23667)
Compressed size for `tests 1` blob report is 19Mb whil uncompressed one
is 211Mb. Also according to [GitHub
policy](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts
) it is uncompressed size that is used for billing:

"Artifacts are uploaded during a workflow run, and you can view an
artifact's name and size in the UI. When an artifact is downloaded using
the GitHub UI, all files that were individually uploaded as part of the
artifact get zipped together into a single file. This means that billing
is calculated based on the size of the uploaded artifact and not the
size of the zip file."
2023-06-12 14:03:22 -07:00
Max Schmitt
b21c81d344
devops: add bot for Chromium --headless=new (#23440)
https://github.com/microsoft/playwright/issues/23389
2023-06-02 20:40:12 +02:00
Yury Semikhatsky
2719f408b2
fix(devops): pass pr number in a file (#23456)
For some reason pull_requests field on workflow_run is empty for pull
requests created from branches in forked repositories, see
https://github.com/orgs/community/discussions/25220. As a workaround we
store triggering pull request number in a file.
2023-06-01 17:12:58 -07:00
Yury Semikhatsky
6cbcf18c48
fix(devops): path to blob report (#23444) 2023-06-01 10:17:34 -07:00
Yury Semikhatsky
972ad89af2
feat(devops): publish merged reports to pull requests (#23419)
* Moved report merging and publishing logic into create_test_report.yml
shared between all workflows
* Merged reports are now published for try jobs on pull requests too. In
order to achieve that the logic had to be extracted into a separate
workflow triggered by
[workflow_run](https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#workflow_run),
this way it can access secrets even if the original workflow was not
able to.
* The blob report data flow is different depending on whether the
workflow is triggered by a pull request or a push:
- For `pull_request` the workflow doesn't have access to the secrets it
uploads the blob report to the GitHub
artifact storage. Later on the merge workflow uploads that blob report
to Azure blob storage.
- Workflows triggered by `push` event can read secrets. They upload blob
report directly to Azure blob storage
and the merge workflow downloads the report from there rather than from
GitHub artifacts.
2023-06-01 08:40:43 -07:00
Yury Semikhatsky
ce9892c694
fix(devops): azure conection string var name (#23416) 2023-05-31 16:54:44 -07:00
Max Schmitt
92d650c317
devops: add missing workflow permission (#23413) 2023-06-01 01:19:23 +02:00
Yury Semikhatsky
90aa378e8f
chore: extract batch download action (#23410) 2023-05-31 16:07:29 -07:00
Max Schmitt
40581ed906
devops: post merged test results comment as GitHub Status Check (#23409) 2023-06-01 00:29:57 +02:00
Yury Semikhatsky
38c89df330
chore: replace "blob download-batch" with a bash script (#23381)
`az storage blob download-batch` has been timing out over the last few
days, see upstream issue
https://github.com/Azure/azure-cli/issues/26567. Replacing it with a
simple bash script that discovers blobs with a given prefix and then
downloads one-by-one.
2023-05-30 16:46:37 -07:00
Max Schmitt
1f7223eb21
chore: test on Node.js 20 (#22651)
Fixes https://github.com/microsoft/playwright/issues/22582
2023-05-30 18:16:34 +02:00
Yury Semikhatsky
1e64fa4f7c
chore: no empty dash suffix in blob report (#23285) 2023-05-25 13:18:57 -07:00
Yury Semikhatsky
0a65a0b8e8
devops: post merged report URL to the PR (#23266)
Posting a comment to the PR using GITHUB_TOKEN requires elevated
permissions, see
https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs
2023-05-24 14:43:36 -07:00
Yury Semikhatsky
10b7cb3979
chore: blob report project suffix (#23212) 2023-05-22 17:54:37 -07:00
Yury Semikhatsky
c8a42af11f
devops: merged report for tracing tests (#23167) 2023-05-19 13:31:37 -07:00
Yury Semikhatsky
5004f908d9
devops: do not upload test-results artifacts (#23168) 2023-05-19 13:06:42 -07:00
Yury Semikhatsky
27403dcbb4
devops: remove explicit npm@8 installation (#23096)
Node 16+ comes with npm 8+ by default.
2023-05-17 14:32:38 -07:00
Max Schmitt
89aa02228f
devops: add macOS 13 coverage (#23065)
Coverage https://github.com/microsoft/playwright/issues/22226
2023-05-17 01:13:09 +02:00
Max Schmitt
44b63b042e
devops: do not download electron on non-electron workflows (#22232) 2023-04-06 19:49:29 +02:00
Max Schmitt
6682fb6075
devops: bump Node.js version 14 -> 16 (#20800)
Fixes https://github.com/microsoft/playwright/issues/20788
2023-02-10 00:53:32 +01:00
Dmitry Gozman
edfff8cd8c
test: run more tests in 'service' mode (#20734) 2023-02-07 18:25:33 -08:00
Andrey Lushnikov
0a752f3fd6
chore(docker): remove experimental docker integration (#20676)
This removes everything related to docker integration experiments that
we conducted over the last 6 months.

I'll send a follow-up with an alternative suggestion that was demo'ed on
a team meeting in the end of December.
2023-02-06 10:50:45 -08:00
Max Schmitt
3acf3f5cae
devops: fix service tests 2
Signed-off-by: Max Schmitt <max@schmitt.mx>
2022-12-21 21:10:44 +01:00
Max Schmitt
2c801f9309
test: fix service tests (#19592) 2022-12-20 15:37:30 +01:00
Max Schmitt
6a5552ba24
chore: drop Ubuntu 18.04 support (#18924) 2022-11-21 11:23:45 -10:00
Andrey Lushnikov
db826c9c8c
chore: drive-by fixes for the snapshotPathTemplate (#18715)
- Use `snapshotPathTemplate` for docker screenshots in html-reporter
- Mark the snapshot path template test as slow since it re-spawns
  worker for each project.
- Fix docker smoke tests
2022-11-10 15:50:13 -08:00
Max Schmitt
d1d24bb490
Revert "devops: use debug Node.js build on CI for browser downloads (#17504)" (#17711)
This reverts commit 56df171d17.
2022-09-29 21:08:28 +02:00
Max Schmitt
56df171d17
devops: use debug Node.js build on CI for browser downloads (#17504)
https://github.com/microsoft/playwright/issues/17394
2022-09-28 10:52:30 +02:00
Andrey Lushnikov
a12112c24d
devops(docker): add docker integration smoke tests (#17267) 2022-09-14 15:05:18 -07:00
Max Schmitt
e8b9d4e758
devops: bump github actions (#16952) 2022-08-31 17:23:42 +02:00
Andrey Lushnikov
f3269c6891
tests: do not test on MacOS 10.15 any more (#16465)
Playwright v1.25 was the last release to support macOS 10.15, so we
no longer need to test on macOS 10.15.
2022-08-11 09:50:44 -07:00