🌱 Add config e2e test and fix README (#4232)

* add config e2e test

Signed-off-by: Spencer Schrock <sschrock@google.com>

* update readme syntax

The old syntax was changed so the README was out of date.
This was exposed when setting up the e2e repo.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* fix rename

Signed-off-by: Spencer Schrock <sschrock@google.com>

---------

Signed-off-by: Spencer Schrock <sschrock@google.com>

config/README.md

@@ -10,40 +10,37 @@ To annotate your repository, create a `scorecard.yml` file in the root of your r
 The file structure is as follows:
 
 ```yml
-exemptions:
+annotations:
   - checks:
       - binary-artifacts
-    annotations:
-      - annotation: test-data # the binary files are only used for testing
+    reasons:
+      - reason: test-data # the binary files are only used for testing
   - checks:
       - dangerous-workflow
-    annotations:
-      - annotation: remediated # the workflow is dangerous but only run under maintainers verification and approval
-      -
+    reasons:
+      - reason: remediated # the workflow is dangerous but only run under maintainers verification and approval
 ```
 
 You can annotate multiple checks at a time:
 
 ```yml
-exemptions:
+annotations:
   - checks:
       - binary-artifacts
       - pinned-dependencies
-    annotations:
-      - annotation: test-data # the binary files and files with unpinned dependencies are only used for testing
- 
+    reasons:
+      - reason: test-data # the binary files and files with unpinned dependencies are only used for testing
 ```
 
 And also provide multiple annotations for checks:
 
 ```yml
-exemptions:
+annotations:
   - checks:
       - binary-artifacts
-    annotations:
-      - annotation: test-data # test.exe is only used for testing 
-      - annotation: remediated # dependency.exe is needed and it's used but the binary signature is verified
-  
+    reasons:
+      - reason: test-data # test.exe is only used for testing 
+      - reason: remediated # dependency.exe is needed and it's used but the binary signature is verified
 ```
 
 The available checks are the Scorecard checks in lower case e.g. Binary-Artifacts is `binary-artifacts`.

e2e/config_test.go

@@ -0,0 +1,40 @@
+// Copyright 2024 OpenSSF Scorecard Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package e2e
+
+import (
+	"context"
+
+	. "github.com/onsi/ginkgo/v2"
+	. "github.com/onsi/gomega"
+
+	"github.com/ossf/scorecard/v5/checks"
+	"github.com/ossf/scorecard/v5/clients/githubrepo"
+	"github.com/ossf/scorecard/v5/pkg/scorecard"
+)
+
+var _ = Describe("E2E TEST: config parsing", func() {
+	Context("E2E TEST:Valid config parsing", func() {
+		It("Should return an annotation from the config", func() {
+			repo, err := githubrepo.MakeGithubRepo("ossf-tests/scorecard-config-e2e")
+			Expect(err).Should(BeNil())
+			results, err := scorecard.Run(context.Background(), repo,
+				scorecard.WithChecks([]string{checks.CheckCodeReview}),
+			)
+			Expect(err).Should(BeNil())
+			Expect(len(results.Config.Annotations)).Should(BeNumerically(">=", 1))
+		})
+	})
+})