scorecard/options/options.go
Gabriela Gutierrez 8789bbbbfc
⚠️ Add initial Maintainers Annotation parsing (#3905)
* feat: Get maintainers annotation from repo

This commits adds functionality to read a scorecard.yml file from a repository and parse it to get the maintainers annotation. It introduces the concepts of exemptions, annotations, annotated checks, and annotation reasons.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Hand off maintainers annotation for SARIF

Hnad off maintainers annotation to SARIF formatting so it can decide to skip or not skip checks when creating the output.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: If check is annotated, skip in SARIF output

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Add other annotation reasons

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Add options to show maintainers annotations in output

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Output maintainers annotations in JSON

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Remove unnecessary maintainers annotation param in SARIF

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Output maintainers annotations in string default result

This commit changes how data is appended to the table rows. Previously, we defined the table columns size and added information to each index. To avoid complicating the calculation of the index now that we are adding another optional column, the data is appended to the row as needed.

Also, the maintainers annotation was chosen to be displayed as last column to give space for Scorecard official reasoning and documentation to appear first.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Ignore annotation if check has max score

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* doc: Add documentation for maintainers annotation

Introduce what flag should be used to show maintainers annotation and how to configure maintainers annotation for your repository.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: A maintainers annotation obj can verify if a check is exempted

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Get annotations function can be private

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Find scorecard.yml file in the repository's root

Change to "GetFileContent" method since we're looking for a specific file instead of using "OnMatchingFileContentDo" method that looks files with a specific content.
This also removes the dependency from "checks/fileparser". This is necessary to move "IsCheckExempted" to checker.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: A check should know if it's exempted or not

Moving the verification "IsCheckExempted" from maintainers_annotation package to checker package. This way a check result will define, consulting maintainers annotation, if it is exempted or not.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Maintainers annotation can only be used in experimental mode

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Ignore if scorecard.yml does not exist

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Remove unnecessary maintainers annotation param

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* docs: Move complete mantainers annotation doc to feature folder

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Error logs

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename AnnotationReason to Reason

Avoid repetition in variable references.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Reason documentation

Redo reason documentation as a switch case to be called when necessary instead of defining a global map. Another reason to redo this logic as switch is that switch should be more performatic then instantiating a local map.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename ScorecardYml to ScorecardConfig

This is a better generic name to reference Scorecard configuration file and leave the file format for the implementation.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Check name comparison

The EqualFold comparison is already case insensitive.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename maintainers annotation folder/file to config

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename and simplify parsing the config

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Check parses its reasons

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Is check exempted

Fix config struture renaming and collect all annotation reasons for a check. Don't stop in the first annotation that the check is exempted.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename maintainers annotation to annotations

Renaming flags, function params, docs and fixing config renamings.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Separate annotations content from config parsing

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Omit empty annotations in JSON results

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Read config file content

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: JSON2 result options

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: String result options

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Mock GetFileReader

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotation on Binary-Artifacts check

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Validate annotated checks

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotating all checks

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Validate annotated reasons

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotating all reasons

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Multiple annotations

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Binary-Artifacts exempted for testing

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Binary-Artifacts not exempted

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: No checks exempted

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Exemption is outdated

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Improve reasons error comparison

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Multiple exemption reasons in a single annotation

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Multiple exemption reasons across annotations

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: cmd show annotations flag doc

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Add show annotations flag

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Remove unnecessary function

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotations string format

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotations json format

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter fallthrough

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter imports

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter unnecessart struct type declaration

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter append combine

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter struct memory

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter improve error msg in run scorecard

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter dynamic errors

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* docs: Disable security alerts on SARIF output

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* docs: Redirect to configuration doc on main README

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Invalid check in annotations

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Invalid reason in annotations

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Exempt check on SARIF output clears runs

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Add check1 annotations json

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: On parse error return empty config file not a "dirty" one

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: On parse config error continue execution

We log the error to the user but continue execution with empty config.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Merge conflics importing rules

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Readd is experimental enabled method

This method is necessary to validate if experimental feature is enabled so it can activate show annotations feature.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Wrap config parse under experimental flag

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix unit test by removing unused mock call

Signed-off-by: Spencer Schrock <sschrock@google.com>

---------

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
2024-04-23 20:15:12 +00:00

262 lines
6.8 KiB
Go

// Copyright 2020 OpenSSF Scorecard Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Package options implements Scorecard options.
package options
import (
"errors"
"fmt"
"log"
"os"
"strings"
"github.com/caarlos0/env/v6"
"github.com/ossf/scorecard/v5/clients"
sclog "github.com/ossf/scorecard/v5/log"
)
// Options define common options for configuring scorecard.
type Options struct {
Repo string
Local string
Commit string
LogLevel string
Format string
NPM string
PyPI string
RubyGems string
Nuget string
PolicyFile string
ResultsFile string
ChecksToRun []string
ProbesToRun []string
Metadata []string
CommitDepth int
ShowDetails bool
ShowAnnotations bool
// Feature flags.
EnableSarif bool `env:"ENABLE_SARIF"`
EnableScorecardV6 bool `env:"SCORECARD_V6"`
EnableScorecardExperimental bool `env:"SCORECARD_EXPERIMENTAL"`
}
// New creates a new instance of `Options`.
func New() *Options {
opts := &Options{}
if err := env.Parse(opts); err != nil {
log.Printf("could not parse env vars, using default options: %v", err)
}
// Defaulting.
// TODO(options): Consider moving this to a separate function/method.
if opts.Commit == "" {
opts.Commit = DefaultCommit
}
if opts.Format == "" {
opts.Format = FormatDefault
}
if opts.LogLevel == "" {
opts.LogLevel = DefaultLogLevel
}
return opts
}
const (
// DefaultCommit specifies the default commit reference to use.
DefaultCommit = clients.HeadSHA
// Formats.
// FormatJSON specifies that results should be output in JSON format.
FormatJSON = "json"
// FormatProbe specifies that results should be output in probe JSON format.
FormatProbe = "probe"
// FormatSarif specifies that results should be output in SARIF format.
FormatSarif = "sarif"
// FormatDefault specifies that results should be output in default format.
FormatDefault = "default"
// FormatRaw specifies that results should be output in raw format.
FormatRaw = "raw"
// Environment variables.
// EnvVarEnableSarif is the environment variable which controls enabling
// SARIF logging.
EnvVarEnableSarif = "ENABLE_SARIF"
// EnvVarScorecardV6 is the environment variable which enables scorecard v6
// options.
EnvVarScorecardV6 = "SCORECARD_V6"
// EnvVarScorecardExperimental is the environment variable which enables experimental
// features.
EnvVarScorecardExperimental = "SCORECARD_EXPERIMENTAL"
)
var (
// DefaultLogLevel retrieves the default log level.
DefaultLogLevel = sclog.DefaultLevel.String()
errCommitIsEmpty = errors.New("commit should be non-empty")
errFormatNotSupported = errors.New("unsupported format")
errPolicyFileNotSupported = errors.New("policy file is not supported yet")
errRawOptionNotSupported = errors.New("raw option is not supported yet")
errRepoOptionMustBeSet = errors.New(
"exactly one of `repo`, `npm`, `pypi`, `rubygems`, `nuget` or `local` must be set",
)
errSARIFNotSupported = errors.New("SARIF format is not supported yet")
errValidate = errors.New("some options could not be validated")
)
// Validate validates scorecard configuration options.
// TODO(options): Cleanup error messages.
func (o *Options) Validate() error {
var errs []error
// Validate exactly one of `--repo`, `--npm`, `--pypi`, `--rubygems`, `--nuget`, `--local` is enabled.
if boolSum(o.Repo != "",
o.NPM != "",
o.PyPI != "",
o.RubyGems != "",
o.Nuget != "",
o.Local != "") != 1 {
errs = append(
errs,
errRepoOptionMustBeSet,
)
}
// Validate SARIF features are flag-guarded.
if !o.isSarifEnabled() {
if o.Format == FormatSarif {
errs = append(
errs,
errSARIFNotSupported,
)
}
if o.PolicyFile != "" {
errs = append(
errs,
errPolicyFileNotSupported,
)
}
}
// Validate V6 features are flag-guarded.
if !o.isV6Enabled() {
if o.Format == FormatRaw {
errs = append(
errs,
errRawOptionNotSupported,
)
}
}
// Validate format.
if !validateFormat(o.Format) {
errs = append(
errs,
errFormatNotSupported,
)
}
// Validate `commit` is non-empty.
if o.Commit == "" {
errs = append(
errs,
errCommitIsEmpty,
)
}
if len(errs) != 0 {
return fmt.Errorf(
"%w: %+v",
errValidate,
errs,
)
}
return nil
}
func boolSum(bools ...bool) int {
sum := 0
for _, b := range bools {
if b {
sum++
}
}
return sum
}
// Feature flags.
// GitHub integration support.
// See https://github.com/ossf/scorecard-action/issues/1107.
// NOTE: We don't add a field to to the Option structure to simplify
// integration. If we did, the Action would also need to be aware
// of the integration and pass the relevant values. This
// would add redundancy and complicate maintenance.
func (o *Options) IsInternalGitHubIntegrationEnabled() bool {
return (os.Getenv("CI") == "true") &&
(os.Getenv("SCORECARD_INTERNAL_GITHUB_INTEGRATION") == "1") &&
(os.Getenv("GITHUB_EVENT_NAME") == "dynamic")
}
// Checks returns the list of checks and honours the
// GitHub integration.
func (o *Options) Checks() []string {
if o.IsInternalGitHubIntegrationEnabled() {
// Overwrite the list of checks.
s := os.Getenv("SCORECARD_INTERNAL_GITHUB_CHECKS")
l := strings.Split(s, ",")
for i := range l {
l[i] = strings.TrimSpace(l[i])
}
return l
}
return o.ChecksToRun
}
func (o *Options) Probes() []string {
return o.ProbesToRun
}
// isExperimentalEnabled returns true if experimental features were enabled via
// environment variable.
func (o *Options) isExperimentalEnabled() bool {
value, _ := os.LookupEnv(EnvVarScorecardExperimental)
return value == "1"
}
// isSarifEnabled returns true if SARIF format was specified in options or via
// environment variable.
func (o *Options) isSarifEnabled() bool {
// UPGRADEv4: remove.
_, enabled := os.LookupEnv(EnvVarEnableSarif)
return o.EnableSarif || enabled
}
// isV6Enabled returns true if v6 functionality was specified in options or via
// environment variable.
func (o *Options) isV6Enabled() bool {
_, enabled := os.LookupEnv(EnvVarScorecardV6)
return o.EnableScorecardV6 || enabled
}
func validateFormat(format string) bool {
switch format {
case FormatJSON, FormatProbe, FormatSarif, FormatDefault, FormatRaw:
return true
default:
return false
}
}