Commit Graph

496 Commits

Author SHA1 Message Date
Andrey Lushnikov
c92ad10f60 chore: fix issue template 2023-08-31 09:24:18 -07:00
Yury Semikhatsky
77f4f4d5de
fix: linkify report link in check output (#26696)
Fixes #26597
2023-08-24 17:45:18 -07:00
Yury Semikhatsky
39a6b23309
devops: create blob reports for all "tests 1" (#26694) 2023-08-24 16:06:41 -07:00
Yury Semikhatsky
c344fce7b6
devops: simplify upload-blob-report action (#26548) 2023-08-18 12:16:26 -07:00
Yury Semikhatsky
31652bca59
devops: merge all "tests 2" reports (#26547) 2023-08-18 12:06:17 -07:00
Max Schmitt
3916e36976
devops: filter merged comments to hide better (#26546) 2023-08-18 21:05:03 +02:00
Max Schmitt
2c4897094b
devops: hide previous Test results comments (#26516)
There is no REST call for it, so we need to use GraphQL.

References:
- https://github.com/actions/github-script#run-custom-graphql-queries
- https://docs.github.com/en/graphql/reference/mutations#minimizecomment
- https://github.com/isaacs/github/issues/1480
2023-08-17 17:56:32 +02:00
Pavel Feldman
576608b69d
chore: revert #24598 (#26484) 2023-08-15 11:29:33 -07:00
Pavel Feldman
ba81a23256
chore: enable traces on service2 bots (#26414) 2023-08-10 12:20:52 -07:00
Yury Semikhatsky
6f8f3dddf1
devops(merge): authorize service principal by using a client secret (#26380)
Pass all credentials via environment variable as described on [this
page](https://learn.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-authorize-azure-active-directory#authorize-a-service-principal-by-using-a-client-secret-1)
instead of using `azcopy login`.
2023-08-09 11:26:52 -07:00
Yury Semikhatsky
ff0aef37f8
devops: use azcopy for better upload performance (#26360)
The credentials are created with 
`az ad sp create-for-rbac --name "playwright-github-actions" --role
"Storage Blob Data Contributor" --scopes /subscriptions/<subscription
id>/resourceGroups/<resource
group>/providers/Microsoft.Storage/storageAccounts/<storage account>`

We cannot use `azure/login@v1` for login as it does not see to properly
propagate credentials to `azcopy` in the next step (there are some
reports about keyring problems on linux based actions).
2023-08-08 22:50:36 -07:00
Dmitry Gozman
27c15b705d
fix(blob): replace projectSuffix with reportName (#25017)
Always ensure unique project/test ids across blobs.
Show `reportName` as a label in the html report.

References #24451.
2023-08-07 13:38:09 -07:00
Yury Semikhatsky
a0211924d3
devops: upload blob reports for headed and transport tests (#26313)
This will add another (3*4 + 2)*3k = 42k tests to the combined report
and should give us 100k+ tests in total per report.
2023-08-06 11:10:12 -07:00
Pavel Feldman
ed83d04d3e chore: install deps for grid tests (3) 2023-08-04 18:05:32 -07:00
Pavel Feldman
1fdb8bdbe4 chore: install deps for grid tests (2) 2023-08-04 16:41:41 -07:00
Pavel Feldman
fda578c14d chore: install deps for grid tests 2023-08-04 16:22:43 -07:00
Pavel Feldman
aba6964bd1
chore: add grid tests (#24617) 2023-08-04 14:59:48 -07:00
Yury Semikhatsky
4494137814
devops: unique report names (#24596) 2023-08-03 15:34:40 -07:00
Yury Semikhatsky
8b18722181
chore: drop run_attempt from github artifact name (#24594) 2023-08-03 15:34:23 -07:00
Yury Semikhatsky
d57a3bad39
devops: blob report is already zipped (#24595) 2023-08-03 15:31:39 -07:00
Dmitry Gozman
d92fe16b76
fix(blob report): default location relative to package.json (#24481)
Also:
- remove `blob-report` directory at the start;
- markdown's `report.md` next to package.json;
- use default location in playwright's workflows.

References #24451.
2023-07-28 15:49:31 -07:00
Pavel
0c253dafc7 chore: quote the zip pattern 2023-07-27 16:44:30 -07:00
Pavel Feldman
335d5275cc chore: fix the zip paths in artifact upload 2023-07-27 13:14:33 -07:00
Pavel
fba344d5ed chore: zip artifacts always 2023-07-27 11:31:13 -07:00
Pavel
58548640b3 chore: zip artifacts 2023-07-27 11:22:57 -07:00
Pavel Feldman
dd7e53fcfe
chore: try running s2 with tracing (#24449) 2023-07-26 20:33:10 -07:00
Pavel Feldman
216d08ba94
chore: use shared test run id (#24448) 2023-07-26 18:36:33 -07:00
Pavel Feldman
ececb6d19e
chore: use proper run_id (#24447) 2023-07-26 17:23:10 -07:00
Pavel Feldman
619d326f4b
chore: run os/browser matrix (#24445) 2023-07-26 17:06:09 -07:00
Pavel Feldman
517cc18c9b
chore: remove retries, skip tracing tests (#24443) 2023-07-26 14:56:38 -07:00
Pavel Feldman
0f5b4fc6f2
chore: lower workers from 20 to 10 (#24442) 2023-07-26 14:43:38 -07:00
Pavel Feldman
1ddfcf503d
chore: add yml for service2 test mode (#24441) 2023-07-26 14:32:48 -07:00
Yury Semikhatsky
e036603aa3
devops: upload blob report only for some bots in tests 2 (#24353)
For now merge-reports chokes on that many events and we end up with no
report at all.
2023-07-26 12:33:14 -07:00
Max Schmitt
1634ec8766
chore: introduce code snippet linting infra (#23960) 2023-07-24 22:27:44 +02:00
Yury Semikhatsky
c40a1dd418
devops: drop --attachments missing option (#24349)
Since all resources are uploaded in the separate workflow anyway there
is not much point in passing the options and uploading blob reports as a
separate step.
2023-07-21 11:34:59 -07:00
Yury Semikhatsky
8b454cb3c3
devops: merge reports fall pw tests in tests 2 (#24309) 2023-07-19 13:01:52 -07:00
Yury Semikhatsky
e3ef358906
devops: merge reports on PRs only (#24307) 2023-07-19 12:03:31 -07:00
Andrey Lushnikov
1fdd7541e0
chore: add firefox-asan for stress tests (#24049) 2023-07-19 05:53:52 -07:00
Yury Semikhatsky
91346c7012
chore: focus failed and flaky tests, no links in title (#24269)
Links don't work in the check titles:

<img width="503" alt="image"
src="https://github.com/microsoft/playwright/assets/9798949/6ad8f3d3-b96f-4f62-9d4e-cc3d473fd97a">
2023-07-17 17:48:40 -07:00
Yury Semikhatsky
d92db9a513
devops: always create blob report on CI, write PR number within action (#24241) 2023-07-14 15:10:29 -07:00
Yury Semikhatsky
b11fa7435b
fix: mergeWorkflowUrl initialization (#24238) 2023-07-14 12:24:58 -07:00
Yury Semikhatsky
1b1cf87e0a
devops: add upload report action (#24222)
* Fix report downloading from Azure (reports are now zipped)
* Extracted upload logic into an action
* Extracted PR number file generation into its own job
2023-07-14 08:21:15 -07:00
Yury Semikhatsky
5ccd4b0632
chore: linkify Test results title (#24220) 2023-07-13 16:46:17 -07:00
Yury Semikhatsky
ea1ec112d8
fix: add missing fs import (#24185)
Fixes 

```
Notice: Report url: https://mspwblobreport.z1.web.core.windows.net/run-5533005176-1-a0b0752662f8af5f841ff7a65b04d02066474ff2/index.html
ReferenceError: fs is not defined
    at eval (eval at callAsyncFunction (/home/runner/work/_actions/actions/github-script/v6/dist/index.js:15143:16), <anonymous>:30:18)
    at callAsyncFunction (/home/runner/work/_actions/actions/github-script/v6/dist/index.js:15144:12)
    at main (/home/runner/work/_actions/actions/github-script/v6/dist/index.js:15236:26)
    at /home/runner/work/_actions/actions/github-script/v6/dist/index.js:15217:1
    at /home/runner/work/_actions/actions/github-script/v6/dist/index.js:15268:3
    at Object.<anonymous> (/home/runner/work/_actions/actions/github-script/v6/dist/index.js:152[71](https://github.com/microsoft/playwright/actions/runs/5533535965/jobs/10097205178#step:12:72):12)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
```
2023-07-12 10:59:44 -07:00
Yury Semikhatsky
8fbcffa43e
chore: publish markdown report as a comment (#23825) 2023-07-11 14:27:58 -07:00
dependabot[bot]
36588ba98e
chore(deps): bump semver and @babel/core in /packages/playwright-test/bundles/babel (#24082)
Fixes https://github.com/microsoft/playwright/issues/23903

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Max Schmitt <max@schmitt.mx>
2023-07-10 23:40:28 +02:00
Max Schmitt
9a580af1e6
devops: disable npm audit on CI (#23902) 2023-06-26 20:00:43 +02:00
Yury Semikhatsky
51b8f609fb
Revert "devops: zip blob report artifact before uploading (#23667)" (#23687)
This reverts commit a1cdae6bff.

The problem with this approach is that each job overwrites the zip
artifact whereas previously it was merging all reports in the same
directory. We are going to zip .jsonl files instead.
2023-06-13 12:36:04 -07:00
Yury Semikhatsky
a1cdae6bff
devops: zip blob report artifact before uploading (#23667)
Compressed size for `tests 1` blob report is 19Mb whil uncompressed one
is 211Mb. Also according to [GitHub
policy](https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts
) it is uncompressed size that is used for billing:

"Artifacts are uploaded during a workflow run, and you can view an
artifact's name and size in the UI. When an artifact is downloaded using
the GitHub UI, all files that were individually uploaded as part of the
artifact get zipped together into a single file. This means that billing
is calculated based on the size of the uploaded artifact and not the
size of the zip file."
2023-06-12 14:03:22 -07:00
Yury Semikhatsky
7d2201561f
chore(devops): add check status for report merger to PRs (#23564)
The check summary has a link to the report and a link to the merge
workflow run. Otherwise it's very hard to tell which merge workflow
corresponds to given PR.
2023-06-06 22:45:57 -07:00