scorecard/config/testdata
Gabriela Gutierrez 8789bbbbfc
⚠️ Add initial Maintainers Annotation parsing (#3905)
* feat: Get maintainers annotation from repo

This commits adds functionality to read a scorecard.yml file from a repository and parse it to get the maintainers annotation. It introduces the concepts of exemptions, annotations, annotated checks, and annotation reasons.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Hand off maintainers annotation for SARIF

Hnad off maintainers annotation to SARIF formatting so it can decide to skip or not skip checks when creating the output.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: If check is annotated, skip in SARIF output

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Add other annotation reasons

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Add options to show maintainers annotations in output

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Output maintainers annotations in JSON

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Remove unnecessary maintainers annotation param in SARIF

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Output maintainers annotations in string default result

This commit changes how data is appended to the table rows. Previously, we defined the table columns size and added information to each index. To avoid complicating the calculation of the index now that we are adding another optional column, the data is appended to the row as needed.

Also, the maintainers annotation was chosen to be displayed as last column to give space for Scorecard official reasoning and documentation to appear first.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Ignore annotation if check has max score

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* doc: Add documentation for maintainers annotation

Introduce what flag should be used to show maintainers annotation and how to configure maintainers annotation for your repository.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: A maintainers annotation obj can verify if a check is exempted

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Get annotations function can be private

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Find scorecard.yml file in the repository's root

Change to "GetFileContent" method since we're looking for a specific file instead of using "OnMatchingFileContentDo" method that looks files with a specific content.
This also removes the dependency from "checks/fileparser". This is necessary to move "IsCheckExempted" to checker.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: A check should know if it's exempted or not

Moving the verification "IsCheckExempted" from maintainers_annotation package to checker package. This way a check result will define, consulting maintainers annotation, if it is exempted or not.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Maintainers annotation can only be used in experimental mode

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Ignore if scorecard.yml does not exist

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Remove unnecessary maintainers annotation param

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* docs: Move complete mantainers annotation doc to feature folder

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Error logs

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename AnnotationReason to Reason

Avoid repetition in variable references.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Reason documentation

Redo reason documentation as a switch case to be called when necessary instead of defining a global map. Another reason to redo this logic as switch is that switch should be more performatic then instantiating a local map.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename ScorecardYml to ScorecardConfig

This is a better generic name to reference Scorecard configuration file and leave the file format for the implementation.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Check name comparison

The EqualFold comparison is already case insensitive.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename maintainers annotation folder/file to config

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename and simplify parsing the config

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Check parses its reasons

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Is check exempted

Fix config struture renaming and collect all annotation reasons for a check. Don't stop in the first annotation that the check is exempted.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Rename maintainers annotation to annotations

Renaming flags, function params, docs and fixing config renamings.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Separate annotations content from config parsing

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Omit empty annotations in JSON results

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: Read config file content

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: JSON2 result options

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* refactor: String result options

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Mock GetFileReader

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotation on Binary-Artifacts check

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Validate annotated checks

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotating all checks

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Validate annotated reasons

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotating all reasons

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Multiple annotations

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Binary-Artifacts exempted for testing

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Binary-Artifacts not exempted

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: No checks exempted

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Exemption is outdated

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Improve reasons error comparison

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Multiple exemption reasons in a single annotation

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Multiple exemption reasons across annotations

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: cmd show annotations flag doc

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Add show annotations flag

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Remove unnecessary function

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotations string format

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Annotations json format

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter fallthrough

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter imports

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter unnecessart struct type declaration

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter append combine

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter struct memory

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter improve error msg in run scorecard

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Linter dynamic errors

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* docs: Disable security alerts on SARIF output

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* docs: Redirect to configuration doc on main README

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Invalid check in annotations

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Invalid reason in annotations

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Exempt check on SARIF output clears runs

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* test: Add check1 annotations json

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: On parse error return empty config file not a "dirty" one

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: On parse config error continue execution

We log the error to the user but continue execution with empty config.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Merge conflics importing rules

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix: Readd is experimental enabled method

This method is necessary to validate if experimental feature is enabled so it can activate show annotations feature.

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* feat: Wrap config parse under experimental flag

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>

* fix unit test by removing unused mock call

Signed-off-by: Spencer Schrock <sschrock@google.com>

---------

Signed-off-by: Gabriela Gutierrez <gabigutierrez@google.com>
2024-04-23 20:15:12 +00:00
..
all_checks.yml ⚠️ Add initial Maintainers Annotation parsing (#3905) 2024-04-23 20:15:12 +00:00
all_reasons.yml ⚠️ Add initial Maintainers Annotation parsing (#3905) 2024-04-23 20:15:12 +00:00
invalid_check.yml ⚠️ Add initial Maintainers Annotation parsing (#3905) 2024-04-23 20:15:12 +00:00
invalid_reason.yml ⚠️ Add initial Maintainers Annotation parsing (#3905) 2024-04-23 20:15:12 +00:00
multiple_annotations.yml ⚠️ Add initial Maintainers Annotation parsing (#3905) 2024-04-23 20:15:12 +00:00
single_check.yml ⚠️ Add initial Maintainers Annotation parsing (#3905) 2024-04-23 20:15:12 +00:00