Commit Graph

11 Commits

Author SHA1 Message Date
Michael Muré
7348fb9edb
github: fix data race when closing event channel
I believe the issue was twofold:

When done importing, the calling context is likely still valid, so if the output channel is not read enough and reach capacity, some event producer down the line can be blocked trying to send in that channel. When closing it, this send is still trying to proceed, which is illegal in go.

In rateLimitHandlerClient, there was a need to 2 different type of output channel: core.ExportResult and ImportEvent. To do so, the previous code was using a single channel type RateLimitingEvent and a series of goroutines to read/cast/send to the final channel. This could result in more async goroutine being stuck trying to send in an at-capacity channel. Instead, the code now use a simple synchronous callback to directly push to the final output channel. No concurrency needed anymore and the code is simpler.

Any of those fixes could have resolved the data race, but both fixes is more correct.
2022-06-05 15:13:49 +02:00
rng-dynamics
247e1a865d
feature: Github bridge mutation rate limit (#694)
Unified handling of rate limiting of github graphql api
2021-09-14 22:22:28 +02:00
Michael Muré
10a80f1888
github: minor cleanups 2021-04-09 13:07:45 +02:00
Alexander Scharinger
0fd570171d Improve feedback for user when Github rate limiting
The Github bridge itself should not write anything. This commit removes
code writing to stdout and itroduces an event `ImportEventRateLimiting`
to `core.ImportResult` in order to inform about a rate limiting situation
of the Github GraphQL API. Now the communication with the user is
delegated to the various user interfaces.
2021-04-08 23:57:25 +02:00
Alexander Scharinger
db57227ae5 Github bridge: stop sleep-timer on SIGINT 2021-03-27 22:55:47 +01:00
Alexander Scharinger
2646c63213 Github bridge: Refactor 2021-03-22 19:26:59 +01:00
Alexander Scharinger
52fba350d6 Github bridge: send message to user when waiting
When the Github GraphQL API rate limit is exhausted print a message at
the bottom of the terminal so the user knows why the import has been
paused.
2021-03-18 20:59:53 +01:00
Alexander Scharinger
d7f555b437 Github bridge: try again in case of web API error 2021-03-15 08:39:08 +01:00
Alexander Scharinger
93b14c509b Remove maps containing channels.
The old implementation of the github bridge used maps to store several
channels holding data obtained from the Github API. Removing the maps and
simply packing data and channels together in a struct and passing it
through one single channel makes the program simpler in terms of
concurrency and, additionally, enables the garbage collector to free the
memory gradually without any additional provisions.
2021-03-15 07:15:00 +01:00
Alexander Scharinger
9a8e487613 Fix errors: deadlock and empty titles 2021-03-15 07:14:40 +01:00
Alexander Scharinger
689b640bbb Deal with github bridge import rate limit 2021-02-28 21:03:10 +01:00