Commit Graph

16 Commits

Author SHA1 Message Date
Chris McGehee
a4e8751d73 Fix intermittent test failure 2021-05-02 14:13:48 -05:00
nathannaveen
f5185e4bd6 🌱 included copyright headers. 2021-04-01 21:36:10 -05:00
naveen
5b9991e3c4 chore - remove debug log for roundtripper
Remove the debug log for the roundtripper which is flooding.
2021-03-18 10:49:13 -04:00
Naveen
7ff09db2ed
Fix-Using Roundrobin tokens across multiple calls (#284)
The GitHub tokens are picked from a list for each call using Roundrobin approach.
2021-03-17 21:41:29 +00:00
naveen
cab29a2747 Feat- Use cloud buckets for caching
Use cloud buckets for httpcache.

The implementation uses https://github.com/google/go-cloud for it to be
cloud vendor agnostic.
2021-02-24 11:17:50 -05:00
naveen
9510d3e0d7 Fix - default disk cache size
The default disk cache size is 100mb. Changed the default disk cache to
10gb.
2021-02-22 18:19:56 -05:00
Naveen
db81680172
Feat-Implement httpcache middleware for GitHub API (#203)
The GitHub API supports conditional requests
https://docs.github.com/en/rest/overview/resources-in-the-rest-api#conditional-requests

https://github.com/google/go-github supports Conditional requests
https://github.com/google/go-github#conditional-requests

As we are scaling more and more projects this would add a lot of value.

Initial run fetches information using `httpcache` as a middleware,
which caches the HTTP response initially in a large disk (PVC),
probably move to Redis later as a cache instead of disk.

Subsequent `cron runs` will utilize the `httpcache` for checking content modification and
load it from the cache if it isn't modified, which reduces the hitting the
Rate Limit of the GitHub API.
2021-02-22 17:18:28 +00:00
Edoardo Tenani
efd18c84a1 roundtripper: ignore gosec G101 error
As per issue #172 this is not an issue, as there is no real GitHub token
in the constant.
2021-02-19 08:23:57 -05:00
dlorenc
738f152a6c
Add support for multiple auth tokens to round robin requests through. (#87) 2020-12-02 07:59:43 -06:00
Abhishek Arya
f35a1a36f5
Fix typo. (#73) 2020-11-16 11:30:30 -06:00
dlorenc
24fa4cca5e
Add support for and hookup app based authentication for higher rate limiting. (#69)
This also configures it in our nightly cron cluster.
2020-11-13 11:06:46 -06:00
Abhishek Arya
81eab9d2d8
Add license header and code of conduct files. (#34)
* Add license header and code of conduct files.

* Fill missing field.
2020-10-26 15:22:13 -05:00
Dan Lorenc
bdb1aba90e Fix rate limit. 2020-10-13 14:43:12 -05:00
Dan Lorenc
c9596cd09d Add better logging. 2020-10-13 11:29:29 -05:00
Kim Lewandowski
812b3f46f6
fixing lock logic (#4)
Co-authored-by: Kim Lewandowski <klewandowski@google.com>
2020-10-12 11:09:58 -05:00
Dan Lorenc
3ee3c748e9 Initial commit. 2020-10-09 10:08:43 -05:00