scorecard/docs/checks/checks.yaml

# Copyright 2021 Security Scorecard Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# This is the source of truth for all check descriptions and remediation steps.
# Run `cd checks/main && go run /main` to generate `checks.json` and `checks.md`.
checks:
  Active:
    risk: High
    description: >-
      This check tries to determine if the project is "actively maintained".

      A project which is not active may not be patched, may not have its
      dependencies patched, or may not be actively tested and used.
      A low score is therefore considered `High` risk.

      The check currently works by looking for commits within the last 90 days, and
      outputs the highest score if there are at least 1 commit/week during this period.      
    remediation:
      - >-
        There is *NO* remediation work needed here. This is just to indicate
        your project activity and maintenance commitment.        
  Automatic-Dependency-Update:
    risk: High
    description: >-
      This check tries to determine if a project has dependencies automatically updated.

      Not updating dependencies makes a project vulnerable to known flaws and prone to attacks.
      A low score is therefore considered `High` risk.

      The checks looks for [dependabot](https://dependabot.com/docs/config-file/) or
      [renovatebot](https://docs.renovatebot.com/configuration-options/). This check only looks if
      it is enabled and does not ensure that it is run and pull requests are merged.      

    remediation:
      - >-
        Signup for automatic dependency updates with dependabot or renovatebot and place the config
        file in the locations that are recommended by these tools.        
  Binary-Artifacts:
    risk: High
    description: >-
      This check tries to determine if a project has binary artifacts in the source repository.

      Binaries are a threat to auditability and vulnerability management.
      In addition, a binary could be compromised or malicious.
      A low score is therefore considered `High` risk.      
    remediation:
      - >-
        Remove the binary artifacts from the repository.        
      - >-
        Build from source.        
  Branch-Protection:
    risk: High
    description: >-
      [Branch protection](https://docs.github.com/en/github/administering-a-repository/defining-the-mergeability-of-pull-requests/about-protected-branches)
      allows defining rules to enforce certain workflows for
      branches, such as requiring a review or passing certain status checks.

      Branch protection ensures compromised contributors cannot
      intentionally inject malicious code. A low score is therefore considered `High` risk.

      This check determines if the default and release branches are
      protected with GitHub's branch protection settings.
      The check only works when the token has [Admin
      access](https://github.community/t/enable-branch-protection-get-api-without-admin/14197)
      to the repository. This check determines if the default and release branches are
      protected.      
    remediation:
      - >-
        Enable branch protection settings in your source hosting provider to
        avoid force pushes or deletion of your important branches.        
      - >-
        For GitHub, check out the steps
        [here](https://docs.github.com/en/github/administering-a-repository/managing-a-branch-protection-rule).        
  CI-Tests:
    risk: Low
    description: >-
      This check tries to determine if the project runs tests before pull
      requests are merged.

      Running tests helps helps developers catch mistakes early on.
      A low score is considered 'Low' risk.

      The check works by looking for a set of well-known CI-system
      names in GitHub `CheckRuns` and `Statuses` among the recent commits (~30).
      A CI-system is considered well-known if its name contains any of the
      following: appveyor, buildkite, circleci, e2e, github-actions, jenkins,
      mergeable, test, travis-ci. The check succeeds if at least 75% of
      successful pull requests have at least one successful check associated
      with them.      
    remediation:
      - Check-in scripts that run all the tests in your repository.
      - >-
        Integrate those scripts with a CI/CD platform that runs it on every pull
        request (e.g. [GitHub
        Actions](https://docs.github.com/en/actions/learn-github-actions/introduction-to-github-actions),
        [Prow](https://github.com/kubernetes/test-infra/tree/master/prow), etc).        
  CII-Best-Practices:
    risk: Low
    description: >-
      This check tries to determine if the project has a [CII Best Practices
      Badge](https://bestpractices.coreinfrastructure.org/en).

      This badges tells us the repo maintainers are aware of best
      development practices. A low score is considered 'Low' risk.

      The check uses the URL for the Git repo and the CII API.      
    remediation:
      - >-
        Sign up for the [CII Best Practices
        program](https://bestpractices.coreinfrastructure.org/en).        
  Code-Review:
    risk: High
    description: >-
      This check tries to determine if a project requires code review before
      pull requests are merged.

      Reviewing code improves quality of code in general. In addition, it ensures
      compromised contributors cannot intentionally inject malicious code. A low
      score is therefore considered `High` risk.

      The check first tries to detect if Branch-Protection is enabled
      on the default branch and the number of reviewers is at least 1. If this
      fails, it checks if the recent (~30) commits have a Github-approved
      review or if the merger is different from the committer (implicit review).
      It also performs similar check for reviews using
      [Prow](https://github.com/kubernetes/test-infra/tree/master/prow#readme)
      (labels "lgtm" or "approved") and Gerrit ("Reviewed-on" and "Reviewed-by").      
    remediation:
      - >-
        Follow security best practices by performing strict code reviews for
        every new pull request.        
      - >-
        Make "code reviews" mandatory in your repository configuration. E.g.
        [GitHub](https://docs.github.com/en/github/administering-a-repository/about-protected-branches#require-pull-request-reviews-before-merging).        
      - >-
        Enforce the rule for administrators / code owners as well. E.g.
        [GitHub](https://docs.github.com/en/github/administering-a-repository/about-protected-branches#include-administrators)        
  Contributors:
    risk: Low
    description: >-
      This check tries to determine if a project has a set of contributors from
      multiple companies.

      Low score has 'Low' risk.

      The check works by looking at the authors of recent commits
      and checking the `Company` field on the GitHub user profile. A contributor
      must have at least 5 commint in the last 30 commits. The check succeeds if
      all contributor span at least 2 different companies.      
    remediation:
      - >-
        There is *NO* remediation work needed here. This is just to provide some
        insights on which organization(s) have contributed to the project and
        making trust decision based on that. But you can ask your contributors
        to join their respective organization.        
  Fuzzing:
    risk: Medium
    description: >-
      This check tries to determine if the project uses a fuzzing system.

      Fuzzing is important to reduce the number of vulnerabilities in code.
      A low score is considered 'Medium' risk.

      The check currently works by checking if the repo name is in the
      [OSS-Fuzz](https://github.com/google/oss-fuzz) project list.      
    remediation:
      - >-
        Integrate the project with OSS-Fuzz by following the instructions
        [here](https://google.github.io/oss-fuzz/).        
  Packaging:
    risk: Medium
    description:
      This check tries to determine if the project is published as a package that other developers can install/download.

      Packaging your project is important for users to receive updates and security patches automatically.
      A low score is considered `Medium` risk.

      The check currently looks for [GitHub packaging workflows]( https://docs.github.com/en/packages/learn-github-packages/publishing-a-package)
      and language-specific GitHub Actions that upload the package to a correpdonsing hub, e.g., [Npm](https://www.npmjs.com/).
      There is a plan to add better support to query package manager hubs directly in the future, e.g., for [Npm](https://www.npmjs.com/), [PyPi](https://pypi.org/).
    remediation:
      - Publish your project as a [downloadable package](https://docs.github.com/en/packages/learn-github-packages/publishing-a-package).
      - Use a GitHub action to release your package to language-specific hubs.
  Pinned-Dependencies:
    risk: Medium
    description: >-
      This check tries to determine if a project has declared and pinned its
      dependencies.

      Pinning dependencies is important to mitigate compromised dependencies
      from undermining the security of the project. Low score is therefore considered
      `Medium` risk.

      The checks works by (1) looking for the following files in the root
      directory: go.mod, go.sum (Golang), package-lock.json, npm-shrinkwrap.json
      (Javascript), requirements.txt, pipfile.lock (Python), gemfile.lock
      (Ruby), cargo.lock (Rust), yarn.lock (package manager), composer.lock
      (PHP), vendor/, third_party/, third-party/; (2) looks for
      unpinned dependencies in Dockerfiles, shell scripts and GitHub workflows.      
    remediation:
      - >-
        Declare all your dependencies with specific versions in your package
        format file (e.g. `package.json` for npm, `requirements.txt` for
        python). For C/C++, check in the code from a trusted source and add a
        `README` on the specific version used (and the archive SHA hashes).        
      - >-
        If the package manager supports lock files (e.g. `package-lock.json` for
        npm), make sure to check these in the source code as well. These files
        maintain signatures for the entire dependency tree and saves from future
        exploitation in case the package is compromised.        
      - >-
        For Dockerfiles and GitHub workflows, pin dependencies by hash. See example
        [gitcache-docker.yaml](https://github.com/ossf/scorecard/blob/main/.github/workflows/gitcache-docker.yaml#L36)
        and [Dockerfile](https://github.com/ossf/scorecard/blob/main/cron/worker/Dockerfile) examples.        
      - >-
        To help update your dependencies after pinning them, use tools such as
        Github's [dependabot](https://github.blog/2020-06-01-keep-all-your-packages-up-to-date-with-dependabot/)
        or [renovate bot](https://github.com/renovatebot/renovate).        
  SAST:
    risk: Medium
    description: >-
      This check tries to determine if the project uses static code analysis
      systems.

      SAST tool may prevent known classes of bugs to be inadvertently
      introduced in the codebase. A low score is considered `Medium` risk.

      The checks currently looks for known Github apps
      such as [github-code-scanning](https://securitylab.github.com/tools/codeql)
      (codeql) and sonarcloud in the recent (~30) merged PRs. The
      check also looks for the use of "github/codeql-action" in a GitHub workflow.      
    remediation:
      - >-
        Run CodeQL checks in your CI/CD by following the instructions
        [here](https://github.com/github/codeql-action#usage).        
  Security-Policy:
    description: >-
      This check tries to determine if a project has published a security
      policy. It works by looking for a file named `SECURITY.md`
      (case-insensitive) in a few well-known directories.      
    remediation:
      - >-
        Place a security policy file `SECURITY.md` in the root directory of your
        repository. This makes it easily discoverable by a vulnerability reporter.        
      - >-
        The file should contain information on what constitutes a vulnerability
        and a way to report it securely (e.g. issue tracker with private issue
        support, encrypted email with a published public key).        
  Signed-Releases:
    risk: High
    description: >-
      This check tries to determine if a project cryptographically signs release
      artifacts.

      Signed releases attest to the provenance of the artifact.
      A low score is considered 'High' risk.

      It works by looking for filenames: *.minisig
      (https://github.com/jedisct1/minisign), *.asc (pgp), *.sign. for the last
      5 GitHub releases. The check does not verify the signatures.      
    remediation:
      - Publish the release.
      - Generate a signing key.
      - Download the release as an archive locally.
      - Sign the release archive with this key (should output a signature file).
      - Attach the signature file next to the release archive.
      - >-
        For GitHub, check out the steps
        [here](https://wiki.debian.org/Creating%20signed%20GitHub%20releases).        
  Signed-Tags:
    risk: Medium
    description: >-
      This check looks for cryptographically signed tags in the last 5 tags.

      Signed tags attest the author of a commit. A low score is considered 'Medium' risk.

      The check does not verify the signature itself and currently relies on GitHub's verification.      
    remediation:
      - Generate a new signing key.
      - Add your key to your source hosting provider.
      - Configure your key and email in git.
      - Publish the tag and then sign it with this key.
      - >-
        For GitHub, check out the steps
        [here](https://docs.github.com/en/github/authenticating-to-github/signing-tags#further-reading).        
  Token-Permissions:
    risk: High
    description: >-
      This check tries to determine if a project's GitHub workflows
      follow the principle of least privilege, i.e. if the GitHub tokens
      are set read-only by default.

      A compromised token with write access may be used by attackers to
      push malicious code into the project. A low score is therefore considered `High` risk.

      For each workflow yaml file, the check looks
      for the permission definitions. To obtain the highest score, the permissions should be set as
      read-only at the [top level](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#permissions)
      and the required write permissions should be declared at the [run-level](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#jobsjob_idpermissions).
      The check cannot detect if the "read-only" GitHub permission settings is enabled, as there is no API available.      
    remediation:
      - >-
        Set permissions as `read-all` or `contents: read` as described in
        GitHub's [documentation](https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#permissions).        
  Vulnerabilities:
    risk: High
    description: >-
      This check determines if there are open, unfixed vulnerabilities
      in the project using the [OSV](https://osv.dev) service.

      An existing vulnerability is can readily be used by attackers,
      so a low score is considered `High` risk.      
    remediation:
      - >-
        Fix the vulnerabilities. The details of each vulnerability can be found
        on <https://osv.dev>.