Commit Graph

263 Commits

Author SHA1 Message Date
Spencer Schrock
347c2a81fe
Add tests for getBucketSummary. (#2310)
Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
2022-09-28 19:30:44 -04:00
Spencer Schrock
ac55bf4cf0
🐛 Prevent partial cron transfers caused by controller failures (#2308)
* Prevent transfer of bq data when .shard_metadata file is missing.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Nack requests whose jobs dont have a shard metadata file.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Add isCompleted tests.

Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
2022-09-28 19:40:21 +00:00
Spencer Schrock
a694cc90d8
Fix k8s yaml errors and document how to prevent them. (#2298)
Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
2022-09-26 19:10:10 +00:00
Azeem Shaikh
7cd6406aef
Reduce build target radius (#2293)
Signed-off-by: Azeem Shaikh <azeemshaikh38@gmail.com>

Signed-off-by: Azeem Shaikh <azeemshaikh38@gmail.com>
2022-09-24 19:58:50 +00:00
Spencer Schrock
a7a503ae54
🌱 cron: pass config as an argument to binaries (4/n) (#2279)
* Explicitly read config file instead of embedding it.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Add CLI config arg and ReadConfig() to existing cron binaries.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Volume mount config

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Ignore CLI flag args when reading local filenames in controller.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Hide --config in the config package.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Add config param to k8s files.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Fix test

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Allow fallback to embedded config if no config is passed as arg. Intended to be temporary to help with GKE rollout.

Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
2022-09-23 13:42:56 -07:00
Spencer Schrock
f017e2e77b
Fix typo which was causing index out of range panics (#2284)
Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
2022-09-22 17:15:19 +00:00
Azeem Shaikh
a6983edf6e
Fix failing linters (#2281)
Signed-off-by: Azeem Shaikh <azeemshaikh38@gmail.com>

Signed-off-by: Azeem Shaikh <azeemshaikh38@gmail.com>
2022-09-21 18:14:58 +00:00
raghavkaul
d75dea8a58
🌱 Feature: Group commits into changesets (#2260)
* Group raw commits into changesets

Signed-off-by: Raghav Kaul <raghavkaul@google.com>

* Add tests, fix golint

Signed-off-by: Raghav Kaul <raghavkaul@google.com>

* Fix lint

Signed-off-by: Raghav Kaul <raghavkaul@google.com>

* Address PR comments

Signed-off-by: Raghav Kaul <raghavkaul@google.com>

* Fix test failures, remove unneeded fields from raw results

Signed-off-by: Raghav Kaul <raghavkaul@google.com>

* Fix lint

Signed-off-by: Raghav Kaul <raghavkaul@google.com>

* Fix tests

* Handle randomized order
* e2e

Signed-off-by: Raghav Kaul <raghavkaul@google.com>

* Accept code reviews on any commit, not just HEAD

Signed-off-by: Raghav Kaul <raghavkaul@google.com>

* Address PR comments

Signed-off-by: Raghav Kaul <raghavkaul@google.com>

Signed-off-by: Raghav Kaul <raghavkaul@google.com>
Co-authored-by: Azeem Shaikh <azeemshaikh38@gmail.com>
2022-09-20 17:53:11 +00:00
Spencer Schrock
2231d1f722
🌱 cron: make CSV header optional (3/n) (#2261)
* Make CSV header optional.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Appease linter.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Address PR feedback.

Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
2022-09-13 21:57:31 -04:00
Spencer Schrock
bde0ae166a
🌱 cron: generalize config and create optional values for scorecard and criticality (2/n) (#2254)
* Add map logic to yaml config.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Add scorecard yaml test

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Separate general config values from scorecard specific values.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Add criticality values to config.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Add test to confirm empty string behavior.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Combine scorecard and criticality values under AdditionalParams.

Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
Co-authored-by: Azeem Shaikh <azeemshaikh38@gmail.com>
2022-09-12 23:25:29 +00:00
Spencer Schrock
c665f271ce
🌱 cron: allow controller to read CSVs from cloud storage (1/n) (#2235)
* Add input bucket config values

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Allow controller to read input files from buckets.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Add nested iterator tests.

Signed-off-by: Spencer Schrock <sschrock@google.com>

* Add blob tests.

Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
2022-09-08 07:32:52 -04:00
Spencer Schrock
bc5a1d6c3d
Enable SAST check in cron by default (#2223)
Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
2022-09-01 17:25:29 +00:00
Spencer Schrock
758cc39b7e
Add k8s README (#2219)
Signed-off-by: Spencer Schrock <sschrock@google.com>

Signed-off-by: Spencer Schrock <sschrock@google.com>
2022-08-30 14:11:22 -04:00
Spencer Schrock
a8e9050ae0
Optimize SAST check (#2191)
* Optimize SAST

* Address PR feedback

* split checkruns into separate graphql query

* Enable SAST check in the releasetest cron worker

Co-authored-by: Azeem Shaikh <azeemshaikh38@gmail.com>
2022-08-26 19:53:48 +00:00
Spencer Schrock
11ff78e35c
Deduplicate projects by excluding URL fragments (#2201) 2022-08-26 15:35:08 -04:00
Naveen
10b6052acf
🌱 Upgrade to go 1.18 (#2143)
* 🌱 Upgrade to go 1.18

- Upgrade to go 1.18
- Updated the deps to avoid critical CVE's

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>

* Updated dockerfile.

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>

* Fixed the linter issues.

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>

* Fixed the CVE dependencies

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>

* Rmoved the cache which is changing between 1.17 and 1.18

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>

* Rmoved the cache which is changing between 1.17 and 1.18

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>

* Updated ko to latest

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>

* Fixed linter issue.

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>

* Fixed linter issue.

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-08-16 20:55:48 -05:00
Azeem Shaikh
d2b3496beb
Remove duplicate projects with different casings (#2155) 2022-08-16 16:53:55 -05:00
Azeem Shaikh
69eb1ccf1d
Fix a bug in cron API data exporting (#2112)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2022-07-31 06:59:56 -05:00
Naveen
e23ee84db0
Export Scorecards results for API (#2081)
* 🌱 Export Scorecards results for API

- Exporting the Scorecard results for the scorecard API.
- The code exports as result.json without the commit SHA and also with
  the commit SHA.

* Some cleanup and tweaks.

* Some cleanup and tweaks.
2022-07-23 02:37:17 +00:00
Bill Nottingham
63e40aea4d
Add a number of new projects to scan. (#2043)
Co-authored-by: Azeem Shaikh <azeemshaikh38@gmail.com>
2022-07-12 21:58:03 +00:00
Bill Nottingham
48291a3dd4
Use the proper repo for lombok. (#2029) 2022-07-08 23:15:13 +00:00
Aiden Wang
64cd05310b
Support user-defined fuzz functions (GoLang) in fuzzing check (#1979)
* temp save 05262022

* finished golang fuzz func check, getLang interface to be done next week

* temp save 05/31/2022

* temp save 06/01/2022

* temp save-2 06/01/2022

* temp save-1 06032022

* temp save-2 06022022

* temp save

* temp save 06032022

* temp save 06032022 (2)

* update err def

* temp save 3

* update docs for fuzzing

* update docs for fuzzing

* update checks.yaml to gen docs

* temp save 0606

* temp save-2 0606

* temp save-3 0606

* temp save-4 0606

* fix linter errors

* fix linter errs-2

* fix e2e errors

* 0608

* 0608-2

Co-authored-by: Aiden Wang <aidenwang@google.com>
2022-06-08 19:17:51 -07:00
Azeem Shaikh
a30bd749cb
Fix bug in move to internal (#1964)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2022-05-26 17:13:04 +00:00
Azeem Shaikh
d1714a289a
Move the cron job to internal package (#1960)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2022-05-25 15:37:22 -07:00
Azeem Shaikh
6a21afb410
Fix bug in cron setup (#1959)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2022-05-25 20:46:50 +00:00
Azeem Shaikh
25c7e1c7f2
Replace checker.Commit with clients.Commit (#1950)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2022-05-24 23:11:37 +00:00
Azeem Shaikh
edd371cf7d
Replace checker.BP with clients.BP (#1953)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2022-05-24 12:34:07 -07:00
dependabot[bot]
7e4cd514fc
🌱 Bump distroless/base in /cron/controller (#1929)
Bumps distroless/base from `764b74b` to `d65ac1a`.

---
updated-dependencies:
- dependency-name: distroless/base
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-22 12:55:12 +00:00
laurentsimon
2fc48e3b38
Use Tool for raw fuzzing results (#1935)
* updates

* updates
2022-05-21 01:43:09 +00:00
dependabot[bot]
fb45cd7e9d 🌱 Bump distroless/base in /cron/webhook
Bumps distroless/base from `764b74b` to `d65ac1a`.

---
updated-dependencies:
- dependency-name: distroless/base
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-19 16:45:34 +00:00
dependabot[bot]
5843c148db 🌱 Bump distroless/base in /cron/worker
Bumps distroless/base from `764b74b` to `d65ac1a`.

---
updated-dependencies:
- dependency-name: distroless/base
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-05-19 12:54:38 +00:00
laurentsimon
b4700ab5df
Raw results for Contributors check (#1919)
* update

* update

* linter

* linter
2022-05-18 18:13:10 +00:00
Azeem Shaikh
8fdb0e767e
Cron cleanup (#1925)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2022-05-18 09:48:40 -07:00
Naveen
bbaf072dd5
⚠️ Remove the oldjson format from cron (#1920)
- removed the old json format from cron
fix https://github.com/ossf/scorecard/pull/1487

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-05-17 17:31:25 -07:00
Azeem Shaikh
236b296403
Do not fail on empty repositories (#1914)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2022-05-16 00:41:17 +00:00
Naveen
0275a94a3f
:warn: Remove the old Details field from CheckResult (#1906)
https://github.com/ossf/scorecard/issues/1393

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-05-12 12:58:12 -07:00
06kellyjac
c5d787a598 pkg: refactor out scorecard_version 2022-05-10 09:51:55 -05:00
Naveen
7ff4b7e050
⚠️ Removing the confidence field from CheckResult struct (#1896)
- Removing the confidence field from `CheckResult` struct
- https://github.com/ossf/scorecard/issues/1393

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-05-09 17:46:24 +00:00
Parth Kanakiya
9a7d030902
Added additional github repositories in projects.csv (#1886)
* Added additional repositories

* Added more repos

* Cleaned the repos
2022-05-06 16:13:50 +00:00
naveensrinivasan
2cb654102d ⚠️ Removing the pass field from result (#1853)
- Removing the pass field from result
    - https://github.com/ossf/scorecard/issues/1393

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-05-03 11:17:47 -05:00
Naveen
44ad5f53ad
⚠️ Removing the error field from result (#1853)
- Removing the error field from result
- https://github.com/ossf/scorecard/issues/1393

Signed-off-by: naveensrinivasan <172697+naveensrinivasan@users.noreply.github.com>
2022-04-22 23:22:43 +00:00
laurentsimon
1f3861b4cc
Update env variables in cron (#1858) 2022-04-22 20:21:08 +00:00
laurentsimon
f99e1a1552
Schema for BQ table for raw results (#1762)
* Fix schemas

* updates

* updates

* Schema for BQ table of raw result

* update

* updates

* create utility function only

* update

* updates

* updates

* manifest
2022-04-15 16:35:01 +00:00
dependabot[bot]
c428e3181e 🌱 Bump distroless/base in /cron/worker
Bumps distroless/base from `792dfe7` to `764b74b`.

---
updated-dependencies:
- dependency-name: distroless/base
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-29 21:28:57 +00:00
dependabot[bot]
ce06ac1a7e
🌱 Bump distroless/base in /cron/webhook (#1794)
Bumps distroless/base from `792dfe7` to `764b74b`.

---
updated-dependencies:
- dependency-name: distroless/base
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Azeem Shaikh <azeemshaikh38@gmail.com>
2022-03-29 19:51:22 +00:00
dependabot[bot]
007156b1d3 🌱 Bump distroless/base in /cron/controller
Bumps distroless/base from `792dfe7` to `764b74b`.

---
updated-dependencies:
- dependency-name: distroless/base
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-29 09:59:08 -05:00
laurentsimon
8150ab0f88
Make Vuln ID field lower case in raw results (#1761)
* case sensitive ID

* updates
2022-03-25 00:24:23 +00:00
dependabot[bot]
66b3d8ce5c
🌱 Bump github.com/golangci/golangci-lint from 1.44.2 to 1.45.0 in /tools (#1757)
* 🌱 Bump github.com/golangci/golangci-lint in /tools

Bumps [github.com/golangci/golangci-lint](https://github.com/golangci/golangci-lint) from 1.44.2 to 1.45.0.
- [Release notes](https://github.com/golangci/golangci-lint/releases)
- [Changelog](https://github.com/golangci/golangci-lint/blob/master/CHANGELOG.md)
- [Commits](https://github.com/golangci/golangci-lint/compare/v1.44.2...v1.45.0)

---
updated-dependencies:
- dependency-name: github.com/golangci/golangci-lint
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* golangci-lint: Surface and fix as many lint warnings automatically

Signed-off-by: Stephen Augustus <foo@auggie.dev>

* generated: Run golangci-lint with `fix: true`

Signed-off-by: Stephen Augustus <foo@auggie.dev>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Stephen Augustus <foo@auggie.dev>
2022-03-23 02:23:39 +00:00
laurentsimon
06efb4a71c
Update BQ table name for raw results (#1759)
* Update name

* comments
2022-03-21 23:50:45 +00:00
laurentsimon
1094680a0f
🐛 Fix schemas from https://github.com/ossf/scorecard/pull/1758 (#1760)
* Fix schemas

* updates

* updates
2022-03-21 21:03:26 +00:00