Commit Graph

478 Commits

Author SHA1 Message Date
naveen
c62e667f7c Docs - Included instructions for deploying cron
Included instructions for deployment of the k8s cron job for the daily
score.
2021-03-16 10:15:14 -04:00
naveen
27ec7fff8d Docs - Updated the docs for cron
Included a section within the CONTRIBUTING.md about the dailyscore and
cron job.
2021-03-15 12:38:58 -04:00
Naveen
4b4d0f0a01
Fix - out of memory error for large repository (#276)
The httpcache client caches everything in memory and if the repository
is large then the process gets evicted with oom.

Changed the implementation to use the standard http client to fetch the
tarball.
2021-03-14 21:50:17 -04:00
NirmalaY12
6a224d1693 Update projects.txt
Scan on github.com/mwiede/jsch
2021-03-14 21:37:18 -04:00
naveen
88de2df279 Feat-Use synk to check cron-job security settings
Use synk to check for cron-job yaml for secuity misconfiguration.
2021-03-12 21:03:29 -05:00
naveen
3489c83404 Feat - Include synk check for k8s yaml
Synk has set of rules to validate the k8s yaml for insecure
configuration.

This action will validate the k8s yaml for insecure configuration.
2021-03-12 20:56:00 -05:00
naveen
3d6b080241 Doc - Included gitcache documentation
Included documentation for gitcache.
2021-03-12 19:24:29 -05:00
naveen
0eaa4ff3d0 Fix - Made the results.json wellformed from cron
Fixed the results.json to be wellformed from the cron job.

Changed the docker image from gsutil to cloudsdk:slim to `sed` binary
which is being used with the cron.sh
2021-03-11 21:58:54 -05:00
naveen
b8768a0eb3 Fix - Set resource limits for the cron pod 2021-03-11 12:03:14 -05:00
Naveen
cccf74cb60
Fix - yaml string quotes. (#266)
The `yaml` string was missing quotes.
2021-03-11 16:06:46 +00:00
naveen
2978ae550a Fix - signed-tags e2e tests.
The signed tags e2e tests were failing because apache/airflow pushed
tags without signing.

Changed from apache/airflow to bitcoin/bitcoin.
2021-03-11 10:59:03 -05:00
naveen
cb94f06642 Fix - cron included caching settings
Included caching settings for the cron job.
2021-03-10 12:33:14 -05:00
Naveen
b79b001e4e
Fix - failed cron job docker (#259)
The cron job was failing with the .dockerignore.

Also update the cronjob to pull the latest projects.txt from the GitHub.
2021-03-10 16:01:47 +00:00
naveen
6e8018cf8f chore - Upgrade ginkgo and goomega dependencies
Upgrade version for ginkgo and goomega dependencies.
2021-03-10 09:08:31 -05:00
Naveen
210cf4a31e
Feat - More validation tests for gitcache (#257)
Implemented validation tests for gitcache.
2021-03-08 19:12:13 +00:00
dependabot[bot]
b766fff506 Bump github.com/onsi/ginkgo from 1.15.0 to 1.15.1 in /gitcache
Bumps [github.com/onsi/ginkgo](https://github.com/onsi/ginkgo) from 1.15.0 to 1.15.1.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v1.15.0...v1.15.1)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-08 08:56:19 -05:00
dependabot[bot]
303118fe54 Bump github.com/onsi/gomega from 1.10.5 to 1.11.0 in /gitcache
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.10.5 to 1.11.0.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.10.5...v1.11.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-08 07:58:55 -05:00
naveen
2e2820cfba Secured pod and included readiness and liveness
* Included security context to runas Nonroot
* Turned off privilege escalation
* Implemented readOnly RootFilesystem
* Implemented readiness and livenessprobe
2021-03-07 18:47:17 -05:00
naveen
64ac973acf feat - k8s deployment 2021-03-06 19:24:13 -05:00
naveen
93761ebaa1 Feat- Parmeterize cache folder in gitcache
The cache temp folder was hardcoded to using the current working
directory.

With this it will be using the directory that is configured in
"TEMP_DIR".

The TEMP_DIR would be an emptydir in k8s.

Included tests for basic e2e tests.
2021-03-05 23:05:13 -05:00
naveen
248fda288e Fix - docker builds for scorecard cron
Fixed the docker build for scorecard cron and as well as updated the
integration to test for the docker builds.
2021-03-05 13:14:33 -05:00
dependabot[bot]
cef322b809 Bump github.com/onsi/gomega from 1.10.1 to 1.10.5 in /gitcache
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.10.1 to 1.10.5.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.10.1...v1.10.5)

Signed-off-by: dependabot[bot] <support@github.com>
2021-03-05 08:47:05 -05:00
naveen
018043f4cf Feat - e2e tests for gitcache buckets
Implemented e2e tests for buckets.
2021-03-05 08:08:47 -05:00
naveen
90d3fa7e70 Fix - Change log.fatal to error
Fix the repo parsing from log.fatal to error. This was causing the
process to terminate.
2021-03-04 20:06:46 -05:00
naveen
abb06c9dbc feat- Reorganize the code structure
Reorganize the code structure for testing and maintenance.

Feat - Included http endpoint
2021-03-04 19:08:47 -05:00
Naveen
c5528dba94
Update issue templates (#235) 2021-03-04 03:30:32 +00:00
Naveen
3e979657bf
Implemented docker for gitcache (#231)
* Implemented caching the git folder instead of just a branch.
Implemented logging.
Refactored code.

* Feat - Implemented docker for gitcache
2021-03-04 03:22:17 +00:00
Naveen
b4c2e4fd13
feat - migrate to go 1.16 (#233)
Upgrade to go version 1.16
2021-03-03 18:56:29 +00:00
Naveen
f0ff62d9eb
Feat - Included dependabot for gitcache (#232) 2021-03-02 16:51:04 -08:00
Naveen
c55c380e9b
Updated README (#230)
* Updated README

Updated README to reflect the changes that are implemented.

* Update README.md
2021-03-02 21:16:37 +00:00
Naveen
b1f037172a
gitcache - Scaling the scorecard scans (#227)
* Feature - implemented gitcache to scale scorecard

* Create README.md

* Update README.md

* Feature - implemented gitcache to scale scorecard
2021-03-02 02:00:01 +00:00
Edoardo Tenani
7f7c9fcb89
contributors: use go-github org API (#228)
Replace direct call to HTTP URL with appropriate go-github API call.

Closes #175
2021-03-01 16:24:18 -08:00
Abhishek Arya
a44dd6a758
Add pypi and ruby gems package support. (#226)
Adds some more package managers to
https://github.com/ossf/scorecard/issues/33

Co-authored-by: Naveen <172697+naveensrinivasan@users.noreply.github.com>
2021-03-01 11:21:20 -05:00
Naveen
47eda487f5
Fix - Updated golangci-lint based on warnings (#225)
Fixed these deprecated linters

golangci_lint: unexpected output on stderr: level=warning msg="[runner]
The linter 'interfacer' is deprecated due to: The repository of the linter has been archived by the owner."
level=warning msg="[runner] The linter 'maligned' is deprecated due to:
The repository of the linter has been archived by the owner. Use govet 'fieldalignment' instead."
2021-02-27 18:55:36 -08:00
naveen
7b192a0243 feat - Included tests for disk cache
Included tests for disk cache.
Cleaned up tests.
2021-02-26 15:46:21 -05:00
naveen
c2ff48dc59 feat-Reduced GitHub API calls for security check
Reduced the number of calls to GitHub API from 16 to max of 2 calls.
Utilized tar ball to download and check for the contents of those files.
2021-02-25 21:55:54 -05:00
naveen
6f2a0f43f4 Fix - Output path for the test runs 2021-02-25 15:59:39 -05:00
naveen
a7174d8ad7 Feature - Include e2e tests for docker
Included e2e tests for docker.
Included .Dockerignore to ignore files.
Included Docker build in the Makefile.
2021-02-25 11:02:45 -05:00
naveen
cab29a2747 Feat- Use cloud buckets for caching
Use cloud buckets for httpcache.

The implementation uses https://github.com/google/go-cloud for it to be
cloud vendor agnostic.
2021-02-24 11:17:50 -05:00
naveen
586e3d60be Doc - Update README with the TOC
Updated the README with TOC and included instructions for docker usage.
2021-02-23 10:47:44 -05:00
Naveen
79170187a2
Feat- Included dependabot for docker (#213) 2021-02-23 07:34:12 -08:00
naveen
7726ca7987 Feature - Include metadata in the results
Included metadata that can be passed an argument to the command line.
The same metadata will returned the `json` results.
2021-02-22 19:23:46 -05:00
naveen
9510d3e0d7 Fix - default disk cache size
The default disk cache size is 100mb. Changed the default disk cache to
10gb.
2021-02-22 18:19:56 -05:00
Naveen
db81680172
Feat-Implement httpcache middleware for GitHub API (#203)
The GitHub API supports conditional requests
https://docs.github.com/en/rest/overview/resources-in-the-rest-api#conditional-requests

https://github.com/google/go-github supports Conditional requests
https://github.com/google/go-github#conditional-requests

As we are scaling more and more projects this would add a lot of value.

Initial run fetches information using `httpcache` as a middleware,
which caches the HTTP response initially in a large disk (PVC),
probably move to Redis later as a cache instead of disk.

Subsequent `cron runs` will utilize the `httpcache` for checking content modification and
load it from the cache if it isn't modified, which reduces the hitting the
Rate Limit of the GitHub API.
2021-02-22 17:18:28 +00:00
Naveen
66af8d8235
Doc - Update contributing to reflect changes (#208) 2021-02-21 16:18:03 -08:00
Naveen
e0a02567fb
Fix - Cleanup the makefile targets (#207) 2021-02-21 23:35:39 +00:00
naveen
5018c5012c Fix - GitHub bot message URL for ok-to-test
Fixed the incorrect URL to the ok-to-test bot message
2021-02-19 14:04:24 -05:00
naveen
9c4a4596ed Testing - Slash command 2021-02-19 14:04:24 -05:00
dependabot[bot]
030ab1dbc5 Bump peter-evans/slash-command-dispatch from v1 to v2.1.3
Bumps [peter-evans/slash-command-dispatch](https://github.com/peter-evans/slash-command-dispatch) from v1 to v2.1.3.
- [Release notes](https://github.com/peter-evans/slash-command-dispatch/releases)
- [Commits](https://github.com/peter-evans/slash-command-dispatch/compare/v1...72ab5a2e417e454aa8e89c43b28e36fe331e00a5)

Signed-off-by: dependabot[bot] <support@github.com>
2021-02-19 08:28:45 -05:00
Edoardo Tenani
efd18c84a1 roundtripper: ignore gosec G101 error
As per issue #172 this is not an issue, as there is no real GitHub token
in the constant.
2021-02-19 08:23:57 -05:00