scorecard/checks/checks.yaml
Abhishek Arya 5f82d2b9c0
Add checks for workflow action pinning (#466)
Patch by Laurent Simon <laurentsimon@google.com>

Co-authored-by: Laurent Simon <laurentsimon@google.com>
2021-05-17 13:03:39 -07:00

216 lines
11 KiB
YAML

# Copyright 2021 Security Scorecard Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This is the source of truth for all check descriptions and remediation steps.
# Run `cd checks/main && go run /main` to generate `checks.json` and `checks.md`.
checks:
Security-Policy:
description: >-
This check tries to determine if a project has published a security
policy. It works by looking for a file named `SECURITY.md`
(case-insensitive) in a few well-known directories.
remediation:
- >-
Place a security policy file `SECURITY.md` in the root directory of your
repository. This makes it easily discoverable by a vulnerability
reporter.
- >-
The file should contain information on what constitutes a vulnerability
and a way to report it securely (e.g. issue tracker with private issue
support, encrypted email with a published public key).
Contributors:
description: >-
This check tries to determine if a project has a set of contributors from
multiple companies. It works by looking at the authors of recent commits
and checking the `Company` field on the GitHub user profile. A contributor
must have at least 5 commint in the last 30 commits. The check succeeds if
all contributor span at least 2 different companies.
remediation:
- >-
There is *NO* remediation work needed here. This is just to provide some
insights on which organization(s) have contributed to the project and
making trust decision based on that. But you can ask your contributors
to join their respective organization.
Frozen-Deps:
description: >-
This check tries to determine if a project has declared and pinned its
dependencies. It works by (1) looking for the following files in the root
directory: go.mod, go.sum (Golang), package-lock.json, npm-shrinkwrap.json
(Javascript), requirements.txt, pipfile.lock (Python), gemfile.lock
(Ruby), cargo.lock (Rust), yarn.lock (package manager), composer.lock
(PHP), vendor/, third_party/, third-party/; (2) look for github actions
under .github/workflows/ and verifies they are pinned by hash. If one of
the files in (1) AND all the dependencies in (2) are pinned, the check
succeds. This check does not currently look for docker image pinning and
shell script pinning.
remediation:
- >-
Declare all your dependencies with specific versions in your package
format file (e.g. `package.json` for npm, `requirements.txt` for
python). For C/C++, check in the code from a trusted source and add a
`README` on the specific version used (and the archive SHA hashes).
- >-
If the package manager supports lock files (e.g. `package-lock.json` for
npm), make sure to check these in the source code as well. These files
maintain signatures for the entire dependency tree and saves from future
exploitation in case the package is compromised.
Signed-Tags:
description: >-
This check looks for cryptographically signed tags in the last 5 tags. The
check does not verify the signature, but relies on github's verification.
remediation:
- Generate a new signing key.
- Add your key to your source hosting provider.
- Configure your key and email in git.
- Publish the tag and then sign it with this key.
- >-
For GitHub, check out the steps
[here](https://docs.github.com/en/github/authenticating-to-github/signing-tags#further-reading).
Signed-Releases:
description: >-
This check tries to determine if a project cryptographically signs release
artifacts. It works by looking for filenames: *.minisign
(https://github.com/jedisct1/minisign), *.asc (pgp), *.sign. for the last
5 GitHub releases. The check does not verify the signatures.
remediation:
- Publish the release.
- Generate a signing key.
- Download the release as an archive locally.
- Sign the release archive with this key (should output a signature file).
- Attach the signature file next to the release archive.
- >-
For GitHub, check out the steps
[here](https://wiki.debian.org/Creating%20signed%20GitHub%20releases).
CI-Tests:
description: >-
This check tries to determine if the project runs tests before pull
requests are merged. It works by looking for a set of well-known CI-system
names in GitHub `CheckRuns` and `Statuses` among the recent commits (~30).
A CI-system is considered well-known if its names contains any of the
following: appveyor, buildkite, circleci, e2e, github-actions, jenkins,
mergeable, test, travis-ci. The check succeeds if at least 75% of
successful pull requests have at least one successful check associated
with them.
remediation:
- Check-in scripts that run all the tests in your repository.
- >-
Integrate those scripts with a CI/CD platform that runs it on every pull
request (e.g. [GitHub
Actions](https://docs.github.com/en/actions/learn-github-actions/introduction-to-github-actions),
[Prow](https://github.com/kubernetes/test-infra/tree/master/prow), etc).
Code-Review:
description: >-
This check tries to determine if a project requires code review before
pull requests are merged. First it checks if branch-Protection is enabled
on the default branch and the number of reviewers is at least 1. If this
fails, it checks if the recent (~30) commits have a Github-approved
review or if the merger is different from the committer (implicit review).
The check succeeds if at least 75% of commits have a review as described
above. If it fails, it does the same check but looking for reviews by
[Prow](https://github.com/kubernetes/test-infra/tree/master/prow#readme)
(labels "lgtm" or "approved"). If this fails, it does the same but looking
for gerrit-specific commit messages ("Reviewed-on" and "Reviewed-by").
remediation:
- >-
Follow security best practices by performing strict code reviews for
every new pull request.
- >-
Make "code reviews" mandatory in your repository configuration. E.g.
[GitHub](https://docs.github.com/en/github/administering-a-repository/about-protected-branches#require-pull-request-reviews-before-merging).
- >-
Enforce the rule for administrators / code owners as well. E.g.
[GitHub](https://docs.github.com/en/github/administering-a-repository/about-protected-branches#include-administrators)
CII-Best-Practices:
description: >-
This check tries to determine if the project has a [CII Best Practices
Badge](https://bestpractices.coreinfrastructure.org/en). It uses the URL
for the Git repo and the CII API. The check does not consider if the repo
has a solver or gold levels for passing the test.
remediation:
- >-
Sign up for the [CII Best Practices
program](https://bestpractices.coreinfrastructure.org/en).
Pull-Requests:
description: >-
This check tries to determine if the project requires pull requests for
all changes to the default branch. It works by looking at recent commits
(first page, ~30) and uses the GitHub API to search for associated pull
requests. The check discards commits by usernames containing 'bot' or
'gardener'. The check considers a commit containing the string
`Reviewed-on` as being reviewed through gerrit; and does not check for a
corresponding PR.
remediation:
- >-
Always open a pull request for any change you intend to make, big or
small.
- >-
Make "pull requests" mandatory in your repository configuration. E.g.
[GitHub](https://docs.github.com/en/github/administering-a-repository/about-protected-branches#require-pull-request-reviews-before-merging)
- >-
Enforce the rule for administrators / code owners as well. E.g.
[GitHub](https://docs.github.com/en/github/administering-a-repository/about-protected-branches#include-administrators)
Fuzzing:
description: >-
This check tries to determine if the project uses a fuzzing system. It
currently works by checking if the repo name is in the
[OSS-Fuzz](https://github.com/google/oss-fuzz) project list.
remediation:
- >-
Integrate the project with OSS-Fuzz by following the instructions
[here](https://google.github.io/oss-fuzz/).
SAST:
description: >-
This check tries to determine if the project uses static code analysis
systems. It currently works by looking for well-known results in GitHub
pull requests. More specifically, the check first looks for Github apps
named [github-code-scanning](https://securitylab.github.com/tools/codeql)
(codeql) and sonarcloud in the recent (~30) merged PRs. If >75% of
commits contain at least a successful check (by any of the apps above),
the check succeeds. If the above fails, the check instead looks for the
use of "github/codeql-action" in a github workflow.
remediation:
- >-
Run CodeQL checks in your CI/CD by following the instructions
[here](https://github.com/github/codeql-action#usage).
Active:
description: >-
A project which is not active may not be patched, may not have its
dependencies patched, or may not be actively tested and used. So this
check tries to determine if the project is still "actively maintained". It
currently works by looking for commits within the last 90 days, and
succeeds if there are at least 2 commits in the last 90 days.
remediation:
- >-
There is *NO* remediation work needed here. This is just to indicate
your project activity and maintenance commitment.
Branch-Protection:
description: >-
Branch protection allows defining rules to enforce certain workflows for
branches, such as requiring a review or passing certain status checks.
This check would work only when the token has [Admin
access](https://github.community/t/enable-branch-protection-get-api-without-admin/14197)
to the repository. This check determines if the default branch is
protected. More specifically, the checks for AllowForcePushes (disabled),
AllowDeletions (disabled), EnforceAdmins (enabled), RequireLinearHistory
(enabled), RequiredStatusChecks (enabled and must have non-empty context
enabled), RequiredPullRequestReviews (>=1), DismissStaleReviews (enabled),
RequireCodeOwnerReviews (enabled).
remediation:
- >-
Enable branch protection settings in your source hosting provider to
avoid force pushes or deletion of your important branches.
- >-
For GitHub, check out the steps
[here](https://docs.github.com/en/github/administering-a-repository/managing-a-branch-protection-rule).