ossf/scorecard
1
1
Fork 0
mirror of https://github.com/ossf/scorecard.git synced 2024-09-19 13:07:17 +03:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Fix - out of memory error for large repository (#276)

Browse Source 
The httpcache client caches everything in memory and if the repository
is large then the process gets evicted with oom.

Changed the implementation to use the standard http client to fetch the
tarball.
This commit is contained in:
Naveen 2021-03-14 21:50:17 -04:00 committed by GitHub
parent 6a224d1693
commit 4b4d0f0a01
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 7 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

9
checks/checkforfile.go
View File

@ -18,6 +18,7 @@ import (
"archive/tar"
"compress/gzip"
"io"
"net/http"
"strings"
"github.com/ossf/scorecard/checker"
@ -35,8 +36,12 @@ func CheckIfFileExists(c checker.Checker, predicate func(name string,
url = strings.Replace(url, "{archive_format}", "tarball/", 1)
url = strings.Replace(url, "{/ref}", r.GetDefaultBranch(), 1)
// Download
resp, err := c.HttpClient.Get(url)
// Using the http.get instead of the checker httpClient because
// the default checker.HTTPClient caches everything in the memory and it causes oom.
//https://securego.io/docs/rules/g107.html
//nolint
resp, err := http.Get(url)
if err != nil {
return checker.RetryResult(err)
}