Looks like due to https://github.com/mvdan/sh/issues/636
scorecard can't parse comments quoted with backticks like
```
cmd -a \
-b `# withouth backticks -c below would be a separate command` \
-c
```
and fails with something like
```
error parsing shell code: 82:26: reached EOF without closing quote `
```
This PR turns that message into
```
error parsing shell code: vagrant/bootstrap_scripts/arch-sanitizers-clang.sh: 82:26: reached EOF without closing quote `
```
which is a bit more useful.
Co-authored-by: laurentsimon <64505099+laurentsimon@users.noreply.github.com>
to make it easier to figure out whether those PRs are really merged
without code review or whether there is a bug in scorecard like
https://github.com/ossf/scorecard/issues/1260 that prevents it
from finding reviewed PRs. Other than that, the "CI-Tests" check
already show "untested" PRs so it seems the "Code-Review" check
should follow suit.
In PRs like https://github.com/iovisor/bcc/pull/3626 no checks suites
are triggered:
```
$ curl --silent -H "Accept: application/vnd.github.v3+json" 3fcf0f1b58/check-runs
{
"total_count": 0,
"check_runs": [
]
}
```
```
curl --silent -H "Accept: application/vnd.github.v3+json" 3fcf0f1b58/check-suites
{
"total_count": 0,
"check_suites": [
]
}
```
The check should just keep going because "statuses" still can be
triggered so it should use them instead:
```
Closes https://github.com/ossf/scorecard/issues/1285
Co-authored-by: laurentsimon <64505099+laurentsimon@users.noreply.github.com>
Co-authored-by: Azeem Shaikh <azeemshaikh38@gmail.com>
Parsing errors are meant to be discarded but aren't. This patch
changes the code so that the error is indeed discarded and checking
continues as intended and adds a unit test for it.
Signed-off-by: Arnaud J Le Hors <lehors@us.ibm.com>
Co-authored-by: Azeem Shaikh <azeemshaikh38@gmail.com>
Those files most likely contain binary data used by tests for
example. It should be safe to remove this because executables
disguised as ".bin" files will still be caught and flagged by scorecard
before it even have a chance to look at extensions.
It should address https://github.com/ossf/scorecard/issues/1256
Projects with a lot of different CI services use both and the check
should take that into account so as not to report that PRs
like https://github.com/systemd/systemd/pull/21329
with 28 successful, 4 failing, and 2 neutral checks were merged
without any tests.
Without this patch `scorecard` says that 5 out 30 PRs were merged
without running tests:
```
"Debug: CI test found: pr: 21299, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 472a52d22b",
"Debug: CI test found: pr: 21300, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): cf35602cbc",
"Debug: CI test found: pr: 21301, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 13b7b8bd73",
"Debug: CI test found: pr: 21302, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): dfa4876c41",
"Debug: CI test found: pr: 21304, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 01f6c450b6",
"Debug: CI test found: pr: 21305, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 547f97d571",
"Debug: CI test found: pr: 21310, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 0078bbb232",
"Debug: CI test found: pr: 21312, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): df8a8240d3",
"Debug: merged PR without CI test: 21313",
"Debug: CI test found: pr: 21314, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): a942a27840",
"Debug: CI test found: pr: 21316, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 3fec0e6cbf",
"Debug: CI test found: pr: 21318, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): db4f0788c5",
"Debug: CI test found: pr: 21320, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 875afa02fa",
"Debug: CI test found: pr: 21321, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): a55277b889",
"Debug: CI test found: pr: 21324, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): b9df4a2b20",
"Debug: merged PR without CI test: 21325",
"Debug: CI test found: pr: 21327, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 44ddfb922f",
"Debug: CI test found: pr: 21328, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 5e034d4d32",
"Debug: merged PR without CI test: 21329",
"Debug: CI test found: pr: 21330, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 4df52c20f4",
"Debug: CI test found: pr: 21331, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 5dd57a00d5",
"Debug: merged PR without CI test: 21332",
"Debug: CI test found: pr: 21333, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): e0c311b1aa",
"Debug: CI test found: pr: 21334, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 55caae6a78",
"Debug: CI test found: pr: 21335, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): f1d467af25",
"Debug: merged PR without CI test: 21337",
"Debug: CI test found: pr: 21341, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 17f8d8f9b4",
"Debug: CI test found: pr: 21342, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 38ac3ab10a",
"Debug: CI test found: pr: 21347, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 4f8c9645df",
"Debug: CI test found: pr: 21349, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 502e2b4b9e"
```
With this patch:
```
"Debug: CI test found: pr: 21299, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 472a52d22b",
"Debug: CI test found: pr: 21300, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): cf35602cbc",
"Debug: CI test found: pr: 21301, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 13b7b8bd73",
"Debug: CI test found: pr: 21302, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): dfa4876c41",
"Debug: CI test found: pr: 21304, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 01f6c450b6",
"Debug: CI test found: pr: 21305, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 547f97d571",
"Debug: CI test found: pr: 21310, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 0078bbb232",
"Debug: CI test found: pr: 21312, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): df8a8240d3",
"Debug: CI test found: pr: 21313, context: github-actions: https://api.github.com/repos/systemd/systemd/check-runs/4191612395",
"Debug: CI test found: pr: 21314, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): a942a27840",
"Debug: CI test found: pr: 21316, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 3fec0e6cbf",
"Debug: CI test found: pr: 21318, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): db4f0788c5",
"Debug: CI test found: pr: 21320, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 875afa02fa",
"Debug: CI test found: pr: 21321, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): a55277b889",
"Debug: CI test found: pr: 21324, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): b9df4a2b20",
"Debug: CI test found: pr: 21325, context: github-actions: https://api.github.com/repos/systemd/systemd/check-runs/4191237494",
"Debug: CI test found: pr: 21327, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 44ddfb922f",
"Debug: CI test found: pr: 21328, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 5e034d4d32",
"Debug: CI test found: pr: 21329, context: github-actions: https://api.github.com/repos/systemd/systemd/check-runs/4192198481",
"Debug: CI test found: pr: 21330, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 4df52c20f4",
"Debug: CI test found: pr: 21331, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 5dd57a00d5",
"Debug: CI test found: pr: 21332, context: github-actions: https://api.github.com/repos/systemd/systemd/check-runs/4192365458",
"Debug: CI test found: pr: 21333, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): e0c311b1aa",
"Debug: CI test found: pr: 21334, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 55caae6a78",
"Debug: CI test found: pr: 21335, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): f1d467af25",
"Debug: CI test found: pr: 21337, context: github-actions: https://api.github.com/repos/systemd/systemd/check-runs/4197451714",
"Debug: CI test found: pr: 21341, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 17f8d8f9b4",
"Debug: CI test found: pr: 21342, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 38ac3ab10a",
"Debug: CI test found: pr: 21347, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 4f8c9645df",
"Debug: CI test found: pr: 21349, context: ci/semaphoreci/pr: Debian autopkgtest (LXC): 502e2b4b9e"
```
Co-authored-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <zerjun@eta-hd.com>
Co-authored-by: laurentsimon <64505099+laurentsimon@users.noreply.github.com>
It was tested with the systemd project where the security policy
is kept in docs/SECURITY.md. Without this patch `scorecard`
says that the security policy can't be found.
According to https://github.com/apps/lgtm-com
"LGTM is a code analysis platform for identifying vulnerabilities early and preventing
them from reaching production". It's used by `systemd`, `lxc` and a lot of other large
open source projects. The check is
still kind of broken in the sense that it fails to detect
projects where every PR is analyzed by LGTM before getting merged
but it's better than nothing I guess.
Co-authored-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
* Add ClusterFuzzLite to Fuzzing check.
Check for the existence of ".clusterfuzzlite/Dockerfile".
Fixes#1148.
* comment
Co-authored-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
* Different calculation between github and non-github actions
* Add test case for different kind of github and non-github action
* Modify existing test as score calculation has changed
* Github workflow steps run on Windows should default to pwsh as its shell
* Style change from PR feedback
* Fixing linter error
* MR feedback: simplifying code
* Moving consts to top of file
Co-authored-by: laurentsimon <64505099+laurentsimon@users.noreply.github.com>
Co-authored-by: Azeem Shaikh <azeemshaikh38@gmail.com>
Co-authored-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
* Display line number for github workflow "uses" field
* Adding test for line numbers
* Updating comment
* Updating this log message to use SARIF format
Co-authored-by: laurentsimon <64505099+laurentsimon@users.noreply.github.com>
