6629b09746
Some checks are pending
CodeQL / Analyze (go) (push) Waiting to run
CodeQL / Analyze (javascript) (push) Waiting to run
gitlab-tests / gitlab-integration-trusted (push) Waiting to run
golangci-lint / check-linter (push) Waiting to run
build / unit-test (push) Waiting to run
build / generate-mocks (push) Waiting to run
build / generate-docs (push) Waiting to run
build / build-proto (push) Waiting to run
build / ${{ matrix.target }} (build-add-script) (push) Blocked by required conditions
build / ${{ matrix.target }} (build-bq-transfer) (push) Blocked by required conditions
build / ${{ matrix.target }} (build-cii-worker) (push) Blocked by required conditions
build / ${{ matrix.target }} (build-controller) (push) Blocked by required conditions
build / ${{ matrix.target }} (build-github-server) (push) Blocked by required conditions
build / ${{ matrix.target }} (build-scorecard) (push) Blocked by required conditions
build / ${{ matrix.target }} (build-shuffler) (push) Blocked by required conditions
build / ${{ matrix.target }} (build-validate-script) (push) Blocked by required conditions
build / ${{ matrix.target }} (build-webhook) (push) Blocked by required conditions
build / ${{ matrix.target }} (build-worker) (push) Blocked by required conditions
build / validate-docs (push) Waiting to run
build / add-projects (push) Waiting to run
build / validate-projects (push) Waiting to run
build / license boilerplate check (push) Waiting to run
Scorecard analysis workflow / Scorecard analysis (push) Waiting to run
* add lifecycle field to probe yaml definitions Signed-off-by: Spencer Schrock <sschrock@google.com> * classify existing probes Some are listed as stable if they're not expected to change, others are listed as experimental if there are still expected changes. Signed-off-by: Spencer Schrock <sschrock@google.com> * add lifecycle to probe readme Signed-off-by: Spencer Schrock <sschrock@google.com> * fix linter Signed-off-by: Spencer Schrock <sschrock@google.com> * add lifecycle for new probe Signed-off-by: Spencer Schrock <sschrock@google.com> * add probe lifecycle to documentation Signed-off-by: Spencer Schrock <sschrock@google.com> --------- Signed-off-by: Spencer Schrock <sschrock@google.com> |
||
---|---|---|
.. | ||
archived | ||
blocksDeleteOnBranches | ||
blocksForcePushOnBranches | ||
branchesAreProtected | ||
branchProtectionAppliesToAdmins | ||
codeApproved | ||
codeReviewOneReviewers | ||
contributorsFromOrgOrCompany | ||
createdRecently | ||
dependencyUpdateToolConfigured | ||
dismissesStaleReviews | ||
fuzzed | ||
hasBinaryArtifacts | ||
hasDangerousWorkflowScriptInjection | ||
hasDangerousWorkflowUntrustedCheckout | ||
hasFSFOrOSIApprovedLicense | ||
hasLicenseFile | ||
hasNoGitHubWorkflowPermissionUnknown | ||
hasOpenSSFBadge | ||
hasOSVVulnerabilities | ||
hasPermissiveLicense | ||
hasRecentCommits | ||
hasReleaseSBOM | ||
hasSBOM | ||
hasUnverifiedBinaryArtifacts | ||
internal | ||
issueActivityByProjectMember | ||
jobLevelPermissions | ||
packagedWithAutomatedWorkflow | ||
pinsDependencies | ||
releasesAreSigned | ||
releasesHaveProvenance | ||
releasesHaveVerifiedProvenance | ||
requiresApproversForPullRequests | ||
requiresCodeOwnersReview | ||
requiresLastPushApproval | ||
requiresPRsToChangeCode | ||
requiresUpToDateBranches | ||
runsStatusChecksBeforeMerging | ||
sastToolConfigured | ||
sastToolRunsOnAllCommits | ||
securityPolicyContainsLinks | ||
securityPolicyContainsText | ||
securityPolicyContainsVulnerabilityDisclosure | ||
securityPolicyPresent | ||
testsRunInCI | ||
topLevelPermissions | ||
utils | ||
webhooksUseSecrets | ||
zrunner | ||
entries.go | ||
README.md |
OpenSSF Scorecard Probes
This directory contains all the Scorecard probes. Each probe has its own subdirectory.
A probe is an individual heuristic, which provides information about a distinct behavior a project under analysis may or may not be doing. The probes follow a camelcase naming convention that describe the exact heuristic a particular probe assesses. The name should be phrased in a way that can be answered by "true" or "false".
Probes can return one or more findings, where a finding is a piece of data with an outcome, message, and optionally a location where the behavior was observed.
The primary outcomes are finding.OutcomeTrue
and finding.OutcomeFalse
, but other outcomes are available as well.
For example, the finding.OutcomeNotAvailable
is often used for scenarios, where Scorecard cannot assess a behavior because there is no data to analyze.
A probe consists of three files:
def.yml
: The documentation of the probe.impl.go
: The actual implementation of the probe.impl_test.go
: The probe's test.
Lifecycle
Probes can exist in several different lifecycle states:
Experimental
: The semantics of the probe may change, and there are no stability guarantees.Stable
: The probe behavior and semantics will not change. There may be bug fixes as needed.Deprecated
: The probe is no longer supported and callers should not expect it to be maintained.
Reusing code in probes
When multiple probes use the same code, the reused code can be placed in a package under probes/internal/
How do I know which probes to add?
In general, browsing through the Scorecard GitHub issues is the best way to find new probes to add. Requests for support for new tools, fuzzing engines or other heuristics can often be converted into specific probes.
Probe definition formatting
Probe definitions can display links following standard markdown format.
Probe definitions can display dynamic content. This requires modifications in def.yml
and impl.go
and in the evaluation steps.
The following snippet in def.yml
will display dynamic data provided by impl.go
:
${{ metadata.dataToDisplay }}
And then in impl.go
add the following metadata:
f, err := finding.NewWith(fs, Probe,
"Message", nil,
finding.OutcomeTrue)
f = f.WithRemediationMetadata(map[string]string{
"dataToDisplay": "this is the text we will display",
})
Example
Consider a probe with following line in its def.yml
:
The project ${{ metadata.oss-fuzz-integration-status }} integrated into OSS-Fuzz.
and the probe sets the following metadata:
f, err := finding.NewWith(fs, Probe,
"Message", nil,
finding.OutcomeTrue)
f = f.WithRemediationMetadata(map[string]string{
"oss-fuzz-integration-status": "is",
})
The probe will then output the following text:
The project is integrated into OSS-Fuzz.
Should the changes be in the probe or the evaluation?
The remediation data must be set in the probe.