Commit Graph

340 Commits

Author SHA1 Message Date
Michael Muré
3d454d9dc8
entity/dag: proper base operation for simplified implementation
- reduce boilerplace necessary to implement an operation
- consolidate what an operation is in the core, which in turn pave the way for a generic cache layer mechanism
- avoid the previously complex unmarshalling process
- support operation metadata from the core
- simplified testing
2022-07-25 13:27:17 +02:00
Steve Moyer
1d4667c825
refactor(809): eliminate need to defer CleanupTestRepos() 2022-06-16 09:02:52 -04:00
Michael Muré
7348fb9edb
github: fix data race when closing event channel
I believe the issue was twofold:

When done importing, the calling context is likely still valid, so if the output channel is not read enough and reach capacity, some event producer down the line can be blocked trying to send in that channel. When closing it, this send is still trying to proceed, which is illegal in go.

In rateLimitHandlerClient, there was a need to 2 different type of output channel: core.ExportResult and ImportEvent. To do so, the previous code was using a single channel type RateLimitingEvent and a series of goroutines to read/cast/send to the final channel. This could result in more async goroutine being stuck trying to send in an at-capacity channel. Instead, the code now use a simple synchronous callback to directly push to the final output channel. No concurrency needed anymore and the code is simpler.

Any of those fixes could have resolved the data race, but both fixes is more correct.
2022-06-05 15:13:49 +02:00
Gaël James
09037baa7c Allow new gitlab PAT format 2022-03-31 09:57:50 +02:00
Michael Muré
6f11282451 jira: fix incorrect client creation reusing the same credential 2022-03-10 18:41:54 +01:00
Michael Muré
05d73e1b53
Merge pull request #571 from 5nord/add-new-gitlab-iterator
[gitlab] Add new iterator with state change events
2022-02-01 09:24:58 +01:00
dependabot[bot]
245e61e4f1
Bump github.com/xanzy/go-gitlab from 0.50.4 to 0.54.1 (#732)
* Bump github.com/xanzy/go-gitlab from 0.50.4 to 0.54.1

Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.50.4 to 0.54.1.
- [Release notes](https://github.com/xanzy/go-gitlab/releases)
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.50.4...v0.54.1)

---
updated-dependencies:
- dependency-name: github.com/xanzy/go-gitlab
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-01-05 13:37:15 +01:00
Michael Muré
e888391b36
gitlab: re-enable previously broken test 2021-12-05 11:19:40 +01:00
Michael Muré
e762290e23
gitlab: order events on the fly 2021-12-05 11:07:29 +01:00
Michael Muré
4ab6d91df6
Merge pull request #715 from remram44/gitlab-access-tokens
Fix GitLab access tokens settings link
2021-12-04 12:34:03 +01:00
Michael Muré
cadbccb533
Merge pull request #727 from MichaelMure/rng-dynamics/fix-import-empty-title
Fix: github import, some issue titles cause error
2021-12-04 12:29:24 +01:00
Alexander Scharinger
160ba2420b Fix: github import, some issue titles cause error 2021-12-04 09:50:24 +01:00
Alexander Scharinger
476526ac2b Fix: github bridge: push then pull without duplication 2021-12-03 23:21:11 +01:00
Remi Rampin
882579b129 Update URL of GitLab access token settings 2021-10-13 15:07:03 -04:00
Remi Rampin
f50aea486b Fix GitLab showing https:/../personal_access_token
Using path.Join() to concatenate URLs doesn't really make sense, and in
particular will change "//" to "/".
2021-10-13 15:06:47 -04:00
rng-dynamics
d546cdeee1
Tests for the github bridge (#706)
Add integration test for github bridge
2021-09-14 22:24:34 +02:00
rng-dynamics
247e1a865d
feature: Github bridge mutation rate limit (#694)
Unified handling of rate limiting of github graphql api
2021-09-14 22:22:28 +02:00
dependabot[bot]
322f0639c2
Bump github.com/xanzy/go-gitlab from 0.48.0 to 0.50.0
Bumps [github.com/xanzy/go-gitlab](https://github.com/xanzy/go-gitlab) from 0.48.0 to 0.50.0.
- [Release notes](https://github.com/xanzy/go-gitlab/releases)
- [Changelog](https://github.com/xanzy/go-gitlab/blob/master/releases_test.go)
- [Commits](https://github.com/xanzy/go-gitlab/compare/v0.48.0...v0.50.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-24 10:07:56 +02:00
Sascha
1939949fcd
CLI: Add non-interactive option to interactive commands (#651)
* Add option to skip the AvatarURL input request

Using an empty string for the avatar cli flag e.g. `git-bug user create
-a ""` will still result in a prompt. As the avatar URL is an optional
option, it should be possible to skip asking for it entirely.
Otherwise automated user creation via a script must make use of pipe hacks.

* Add global --non-interactive cmdline option

* Replace --skipAvatar for --non-interactive option

* Cmd BugAdd: respect non-interactive option

* Cmd bridge configure: respect non-interactive opt

* Cmd CommentAdd: respect non-interactive option

* Cmd CommentEdit: respect non-interactive option

* Cmd TermUI: respect non-interactive option

* Cmd TitleEdit: respect non-interactive option

* Remove global non-interactive option

* Cmd UserCreate: Use local non-interactive option

* Cmd BugAdd: Use local non-interactive option

* Cmd BridgeConfigure: Use local non-interactive option

* Cmd CommentAdd: Use local non-interactive option

* Cmd CommentEdit: Use local non-interactive option

* Cmd TermUI: Drop non-interactive option

It should be obviouse that the termui is an interactive command.

* Cmd TitleEdit: Use local non-interactive option

* Update docs

* Bridge GitHub: respect non-interactive option

* Bridge GitLab: respect non-interactive option

* Bridge Jira: respect non-interactive and token opt

* Fix failing compilation

* Bridge launchpad: respect non-interactive option

* bridge: isNonInteractive --> interactive

Co-authored-by: Michael Muré <batolettre@gmail.com>
2021-05-09 11:14:45 +02:00
Matthias Simon
028d5c03f9 Add some documentation comments 2021-04-23 09:17:37 +02:00
Matthias Simon
aa4e225a80 gitlab: Add new iterator with state change events
Retrieving events is spread across various various Gitlab APIs. This
makes importing and sorting Gitlab events by time quite complicated.

This commit replaces the old iterators with a goroutine/channel-based
iterator, which merges the individual Gitlab API streams into a single
(sorted) event stream.
2021-04-23 09:02:48 +02:00
Michael Muré
7a7a402649
github: attempt to fix a CI issue 2021-04-23 00:28:46 +02:00
Ellis Clayton
a80152890c
Support new GitHub token formats
GitHub have introduced a new format for their access tokens, which does
not fit within the rules of the previous regex. For the time being, the
previous token format is still being supported by GitHub, so it makes
sense to continue allowing legacy tokens.

https://github.blog/changelog/2021-03-04-authentication-token-format-updates/
2021-04-19 10:21:11 +10:00
Michael Muré
51a2c85954
make sure every text input is safe and validated
fix #630
2021-04-17 17:40:11 +02:00
Alexander Scharinger
8fb6ea0d95 Github brdige: move credential loading and client creation back
Reason: failing integration tests
2021-04-09 15:56:32 +02:00
Alexander Scharinger
be24cdfec4 Revert "Bridges: move credential loading and client creation"
This reverts commit 3d14e2e67c.
2021-04-09 15:41:27 +02:00
Michael Muré
10a80f1888
github: minor cleanups 2021-04-09 13:07:45 +02:00
Michael Muré
1520f678f7
Merge remote-tracking branch 'origin/master' into dev-gh-bridge 2021-04-09 13:01:14 +02:00
Alexander Scharinger
0fd570171d Improve feedback for user when Github rate limiting
The Github bridge itself should not write anything. This commit removes
code writing to stdout and itroduces an event `ImportEventRateLimiting`
to `core.ImportResult` in order to inform about a rate limiting situation
of the Github GraphQL API. Now the communication with the user is
delegated to the various user interfaces.
2021-04-08 23:57:25 +02:00
Alexander Scharinger
ede5e218ac Add comment to clarify look ahead in import channel 2021-04-08 23:20:05 +02:00
Alexander Scharinger
3d14e2e67c Bridges: move credential loading and client creation
Gitlab and Jira bridge: move credential loading and client creation from
`Init` to `ImportAll` in order to harmonize the behaviour of the
different bridges.
2021-04-08 22:48:31 +02:00
Michael Muré
aa0449a3ea
Merge remote-tracking branch 'origin/master' into dag-entity 2021-03-29 11:02:08 +02:00
Michael Muré
890c014d91
repo: fix various config issues around case insentivity 2021-03-28 22:18:01 +02:00
Alexander Scharinger
b2e98ef07f Github bridge: refactor message handling 2021-03-28 12:16:13 +02:00
Alexander Scharinger
db57227ae5 Github bridge: stop sleep-timer on SIGINT 2021-03-27 22:55:47 +01:00
Alexander Scharinger
21b330dad1 Github bridge: fix message about timeout 2021-03-27 22:45:49 +01:00
Alexander Scharinger
2646c63213 Github bridge: Refactor 2021-03-22 19:26:59 +01:00
Alexander Scharinger
52fba350d6 Github bridge: send message to user when waiting
When the Github GraphQL API rate limit is exhausted print a message at
the bottom of the terminal so the user knows why the import has been
paused.
2021-03-18 20:59:53 +01:00
Alexander Scharinger
d7f555b437 Github bridge: try again in case of web API error 2021-03-15 08:39:08 +01:00
Alexander Scharinger
93b14c509b Remove maps containing channels.
The old implementation of the github bridge used maps to store several
channels holding data obtained from the Github API. Removing the maps and
simply packing data and channels together in a struct and passing it
through one single channel makes the program simpler in terms of
concurrency and, additionally, enables the garbage collector to free the
memory gradually without any additional provisions.
2021-03-15 07:15:00 +01:00
Alexander Scharinger
9a8e487613 Fix errors: deadlock and empty titles 2021-03-15 07:14:40 +01:00
Alexander Scharinger
689b640bbb Deal with github bridge import rate limit 2021-02-28 21:03:10 +01:00
Michael Muré
3f6ef50883
bug: migrate to the DAG entity structure! 2021-02-14 12:19:50 +01:00
Michael Muré
b01aa18d39
identity: PR fixes 2021-02-14 12:17:47 +01:00
Michael Muré
ab57d74a31
deal with the previous changes 2021-02-14 12:17:47 +01:00
Michael Muré
fb0c5fd061
repo: expose all lamport clocks, move clocks in their own folder 2021-02-14 12:15:41 +01:00
Matthias Simon
34e6f3c224 Fix comparison mix-up in gitlab importer 2021-02-07 17:01:59 +01:00
Michael Muré
1871f09345
Merge pull request #534 from MichaelMure/gh-bridge-pull-fix
Fix github bridge import
2021-02-01 11:46:31 +01:00
Alexander Scharinger
1ccbf2cb63 Add comments 2021-01-28 14:42:20 +01:00
Alexander Scharinger
62bcfd985d Fix bug 2021-01-26 12:07:26 +01:00