Commit Graph

42 Commits

Author SHA1 Message Date
Azeem Shaikh
8c2432bd62
Add worker to the PubSub framework. (#463)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2021-05-18 15:18:08 -07:00
laurentsimon
3b1c9b8496
❇️ Pin our docker dependencies by hash (#468)
* check pinning in docker files

* Pin our docker dependencies

* Revert "check pinning in docker files"

This reverts commit c05a5007b1.

* comments

* typo

* fix hashes
2021-05-18 18:05:13 +00:00
Azeem Shaikh
37519d9672
Update RunScorecards API. (#461)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2021-05-16 19:38:46 -07:00
Azeem Shaikh
ba3b5c5979
Refactor Makefile and add proto compile support. (#458)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2021-05-15 13:58:01 -07:00
Azeem Shaikh
c82770397a
Add and use config.yaml for pubsub cron. (#457)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2021-05-15 00:22:45 -07:00
Azeem Shaikh
6437c9324f
Setup PubSub framework code. (#428)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2021-05-14 14:32:23 -07:00
Azeem Shaikh
80203a8351
Move k8s into cron folder. (#410)
Co-authored-by: Azeem Shaikh <azeems@google.com>
2021-05-07 05:08:33 -07:00
Chris McGehee
6a7142fe21 Fix lint issues: golint linter 2021-05-02 14:49:40 -05:00
naveen
c6b4e24218 🌱 Clean and Validation of cron
* Included validation for the cron
* Removed branch protection check from the cron
2021-04-29 11:03:41 -05:00
Azeem Shaikh
d3a59eacff Move Dockerfile.gsutil to inside cron/ 2021-04-27 17:21:53 -05:00
naveen
f7b658d31d 🌱 Fixed lint issues in cron
* Fixed golang ci-lint issues in cron
2021-04-26 08:51:28 -05:00
Azeem Shaikh
bd3eff1fcf
Cron job uses line-delimited JSON (#344)
*  Refactor to reduce code duplication

* 

* Move lib/ back to checker/

* Move lib/ back to checker/

* Move lib/ back to checker/

* Address PR comments.

* Addressing PR comments.

* Separate out ReposURL nito repos/

* Add TODO in gitcache module.

* Add RepoRequest/Response types.

* Avoid printing `ShouldRetry` and `Error` in output JSON.

* Fix JSON output.

* Simplify cmd package.

* Make cron/ a package instead of module.

* Fix TODO.

* Remove binary file.

* go.mod file.

* go.mod updates.

* Refactor cron to use in-memory JSON.

* Fix JSON output.

* Fix go.mod

* Address PR comments.

* Change %w -> %v.

* Address PR comments.

* Fix err.

Co-authored-by: Azeem Shaikh <azeems@google.com>
2021-04-19 12:49:51 -07:00
naveen
9397708318 Handle vendored repos dependency
*Handle vendored repos for go dependency

* Add additional repositories for projects.txt
2021-04-12 15:50:10 -05:00
Naveen
f02df30b61
Included dependency parsing for go (#337)
* Included dependency parsing of go.mod files.
* Parse vanity URL in go.mod to add dependencies
* Updated dependencies for scorecard and cosign based on the vanity URL's.
2021-04-10 12:21:51 -05:00
Azeem Shaikh
a58818d258
🌱 : Reduce code duplication for follow-up cron refactoring (#338)
*  Refactor to reduce code duplication

* 

* Move lib/ back to checker/

* Move lib/ back to checker/

* Move lib/ back to checker/

* Address PR comments.

* Addressing PR comments.

* Avoid printing `ShouldRetry` and `Error` in output JSON.

* Fix JSON output.

Co-authored-by: Azeem Shaikh <azeems@google.com>
2021-04-10 07:26:56 -05:00
naveen
6aad826067 🌱 Included dependencies for k8s
* Included the k8s dependencies.
2021-04-08 14:17:56 -05:00
asraa
8a5f9a8ea7
zero pad dates (#328)
Signed-off-by: Asra Ali <asraa@google.com>
2021-04-05 07:57:37 -07:00
Asra Ali
ed8d5801bc Add updater to collect deps in project files and add to projects.txt
Signed-off-by: Asra Ali <asraa@google.com>
2021-04-02 12:57:57 -05:00
nathannaveen
f5185e4bd6 🌱 included copyright headers. 2021-04-01 21:36:10 -05:00
Naveen
0e5b8e63f2 Eating your own dog food
Eating your own dog food
2021-03-22 18:00:20 -04:00
naveen
688dc5e6c7 Refactor cron job
* Refactored cron job from shell script to go.
* Included metadata to the projects.txt for envoy
* Included checks for duplicate item in projects.txt
* Sorted the projects.txt so that it is easier for someone to look for a
project
2021-03-21 22:31:07 -04:00
Asra Ali
7a2675532a add envoy deps statically
Signed-off-by: Asra Ali <asraa@google.com>
2021-03-19 10:07:33 -04:00
Naveen
1a81741624
🌱Remove branch protection check from cron (#290)
The branch protection check needs an admin access to the repository. All
of the checks from cron would fail and uses another call to the API.

This will reduce usage of the API.
2021-03-19 07:27:09 -04:00
NirmalaY12
6a224d1693 Update projects.txt
Scan on github.com/mwiede/jsch
2021-03-14 21:37:18 -04:00
naveen
0eaa4ff3d0 Fix - Made the results.json wellformed from cron
Fixed the results.json to be wellformed from the cron job.

Changed the docker image from gsutil to cloudsdk:slim to `sed` binary
which is being used with the cron.sh
2021-03-11 21:58:54 -05:00
Naveen
b79b001e4e
Fix - failed cron job docker (#259)
The cron job was failing with the .dockerignore.

Also update the cronjob to pull the latest projects.txt from the GitHub.
2021-03-10 16:01:47 +00:00
naveen
248fda288e Fix - docker builds for scorecard cron
Fixed the docker build for scorecard cron and as well as updated the
integration to test for the docker builds.
2021-03-05 13:14:33 -05:00
Naveen
b4c2e4fd13
feat - migrate to go 1.16 (#233)
Upgrade to go version 1.16
2021-03-03 18:56:29 +00:00
Abhishek Arya
5fcaa98d6f
Add top 200 for all langs from criticality score repo. (#181)
From https://github.com/ossf/criticality_score#public-data
and combine with existing projects list.
2021-02-14 16:39:16 -05:00
James Pether Sörling
127fda75ff
Update projects.txt (#151)
Add 3 projects by https://github.com/Hack23

Co-authored-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
Co-authored-by: dlorenc <lorenc.d@gmail.com>
2021-02-10 21:08:11 +00:00
Naveen
15a1ba0536
feat - nonroot docker container (#114)
* feat - nonroot docker container

Changed the docker container to nonroot

* Feat - New Dockerfile for non-cron job

Created a new Dockerfile for non-cron job.
Moved the existing Dockerfile into cron folder for cron specific.

Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>

* Fix - The Docker version information in the README

Updated the README to include docker version information required for
Dockerfile.

Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
2021-01-05 07:45:15 -06:00
Michael Scovetta
ddc82c6306
Add --show-details to the cron job. (#113)
This change will add a `details` section to the cron job output.
2020-12-28 15:16:44 -06:00
naveen
3de6a1b1c0 fix - shellcheck violations for cron.sh
Fixed the shellcheck violations in cron.sh
2020-12-22 12:04:23 -05:00
Dan Lorenc
cd16def4b4 Add 50 Google projects. 2020-12-07 09:02:58 -06:00
Dan Lorenc
338f9a0874 Add Envoy dependencies to cron. 2020-11-28 09:53:11 -06:00
dlorenc
8dfd873669
Sort/uniquify the projects list before running. (#81)
This way we can add things in multiple sections and not worry about deduping.
2020-11-28 09:36:56 -06:00
Atul Mohan
528a01f619
Add apache druid to projects list (#75) 2020-11-17 09:28:40 -06:00
dlorenc
1d004100ed
Add 70 more projects to the nightly run. (#72) 2020-11-16 10:01:58 -06:00
dlorenc
24fa4cca5e
Add support for and hookup app based authentication for higher rate limiting. (#69)
This also configures it in our nightly cron cluster.
2020-11-13 11:06:46 -06:00
Dan Lorenc
3350a2d0bf Switch the nightly to use json. 2020-11-12 20:13:34 -06:00
dlorenc
ef19bdf032
Add a Dockerfile and k8s cron job to upload files to GCS each night. (#59) 2020-11-12 12:26:38 -06:00
dlorenc
45286f140c
Add a script to output in csv that can be run daily. (#56) 2020-11-10 13:25:57 -06:00