Summary:
Instead of doing `commit_syncer.get_bookmark_renamer()(&bookmark)` everywhere,
let's allow to just do `commit_syncer.rename_bookmark(&bookmark)`.
Also, let's add helper methods for fetching repo ids.
Also, let's `impl Debug` to use it in logs.
Reviewed By: farnz
Differential Revision: D18353939
fbshipit-source-id: 1cebcad4a8c1523064580594e392b901566d3c28
Summary:
A tool that checks that files in a commit from a large repo match files from a
corresponding commit in a small repo.
The tool does verification in a few stages:
1) Fetch paths and filenodes for small and large repo
2) Move all paths in the large repo
3) For each path check if we have the same path in small repo. If we don't
print a warning
4) For each filenode that are different also compare their content ids, and
repo an error only if content ids are different
Note that we can't rely just on filenode id comparison, because it turns out
that filenodes can legitimately be different between large and small repo.
We could also just compare the content ids, but that's much slower.
Reviewed By: krallin
Differential Revision: D18268939
fbshipit-source-id: 14f091db380e127ce2d926253376fbc0cf1e30b3
Summary:
Use idea suggested by krallin in https://fburl.com/diff/j1pl3kvl, it looks
much nicer.
Reviewed By: krallin
Differential Revision: D18303027
fbshipit-source-id: efcc1f7b618dedda1801320f71914018668acc64
Summary:
1. `CommitSyncer` seems to be the struct that hides all the lowly implementation
details of how commits are synced across repos, so let's make `RepoSyncTarget` own it, rather than
explicit `Mover` instances.
1. while we're at that, let's make `CommitSyncRepos` own `BookmarkRenamer` the
same way it owns `Mover`.
Reviewed By: StanislavGlebik
Differential Revision: D18274007
fbshipit-source-id: 4451980a0714220933a0583a8f964e4278bf1594
Summary:
This table is needed for backsyncer to track commits that are "equivalent"
between large ans small repo i.e. the closest ancestor ancestor of large commit
that was rewritten in the small repo.
It's very similar to normal `synced_commit_mapping`, but with a few important differences:
1) Small bcs id can be NULL, meaning that commit from large repo doesn't remap to any commit from the small repo [1]
2) Multiple large bcs ids can point to the same small bcs id
3) Inserting new entry in synced_commit_mapping automatically inserts entry in synced_working_copy_equivalence - if a commit was remapped, then it's by definition an equivalent working copy
Reviewed By: ikostia
Differential Revision: D18172894
fbshipit-source-id: 14d5f692521282223778e8c9ad46191d0e24d618
Summary:
We are not using them now, and we decide to use them later we can always add
the back
Reviewed By: aslpavel
Differential Revision: D18169226
fbshipit-source-id: 9b964abde2eda4361ad9610075e37d3b11bd0efe
Summary: We're centralising all the commit syncing work in one place, which we called the CommitSyncConfig. This conflicts with the metaconfig CommitSyncConfig, so rename the one that does work to CommitSyncer
Reviewed By: farnz
Differential Revision: D18139821
fbshipit-source-id: 804e293e30167aa36a0818d297dbfd0594f41118
Summary: By looking at whether an ancestor should be synced and is synced, we can get to the same conclusion as before, but faster. In particular, when you have lots of commits to sync, this rapidly notices that we have an ancestor that needs syncing, and says that we can't decide on this commit until that ancestor is synced
Reviewed By: farnz
Differential Revision: D18139085
fbshipit-source-id: 8b7459bafe287f1d7664f8606489dd1c26713637
Summary:
Use functionality added in D18121305 to implement backsyncer that knows how to
rewrite commits (previously it was just copying commits from one repo to
another).
Note that tests were rewritten from scratch, and one of the tests was removed
without replacement - the one that tests backsyncer on diamond merges. It won't
work right now because there are known innefficiencies in
`get_commit_sync_outcome()`, which will be addressed later.
Reviewed By: farnz
Differential Revision: D18134820
fbshipit-source-id: f1ee39842f7e89d057d5be58667dab739df97b95
Summary:
When syncing large to small, we need to distinguish several states: Do I need to sync this commit? Can I sync this commit now? Will there be parent changes involved?
Add `CommitSyncOutcome` to answer this question, and start heading towards a state where commit syncing is contained in a single crate
Reviewed By: StanislavGlebik
Differential Revision: D18121305
fbshipit-source-id: 302b30d98ba1ff2b3e59ab9a40d49b3252686e7f
Summary:
We've changed data model to make sure we have entry for all pre-big merge
commits. This diff removes the checks for non-remapped commits in the target repo - those
shouldn't be necessary anymore.
Reviewed By: farnz
Differential Revision: D18084127
fbshipit-source-id: 6fa00ac59f5d0806035b1a4f590344e800a4c174
Summary:
Now find new commits (i.e. commits from source_repo that are not in target
repo) and backsync them
Reviewed By: krallin
Differential Revision: D17930706
fbshipit-source-id: ca08f2b8c644a4dbab6edec5019e4445f5c5b011
Summary:
In the multi-directional-sync world it is not enough to have a single direction
specified for all small repos, let's move it to the small repo config level.
This prompts some changes in how we do verification:
- no overlapping prefixes are allowed among all small repos with small-to-large
sync direction (when small repos tail into the large repo, we cannot deny
a pushrebase because of the path conflicts (since the small repo has already
accepted the commit in the past), so let's prohibit path conflicts in the first place)
- no overlapping prefixes are allowed between *all* small-to-large repos and
*each* large-to-small repo (same reason, we cannot reject tailed commits,
so we need to protect ourselves from path conflicts)
- overlapping *is* allowed between large-to-small repos, but only when used
prefixes are identical, not when one prefix is a prefix of another prefix.
To be clear, I am not certain that we cannot accommodate the latter case,
but I just want to avoid additional complexity so far. Overlapped prefixes
represent the "locked across repos" directories: ones, which can be changed
from each of the small repos as part of a push redirector pushrebase
Reviewed By: StanislavGlebik
Differential Revision: D17930877
fbshipit-source-id: 97f0ab8f5975f9635e84716851fe42c8cc8800f5
Summary:
While we have the `RepositoryId` dedicated type, the `RepoConfig` struct used
`i32` for its `repoid` field, so in a lot of places, which referenced or used
this config we continued to use `i32`. This is sad, so let's fix this.
This diff changes `RepoConfig` and everything that depends on it.
Ideally we would also just remove the `.id()` method of the `RepositoryId`
struct, so that our clients have to use the struct, but in a few places it
is deserialized into thift/SQL types, so unless we move those transformations
into the same file (which also seems bad and not future-proof), we cannot get
rid of the method.
Reviewed By: farnz
Differential Revision: D17928574
fbshipit-source-id: 3d9355272cfcd20af787edd6417cc529be640356
Summary:
It handles only very specific case of merges - repo imports e.g. no ancestor of
p2 has a parent that's ancestor of p1. Also at the moment it doesn't support
Mover that removes a p2's ancestor commit.
There's no reason to not add support for both of these cases later though.
Reviewed By: ikostia
Differential Revision: D17899741
fbshipit-source-id: 7d96c9dd13790e1fd8ee96a9d2f486d56e25abf9
Summary:
This diff updates all license headers to use the new text and style.
Also, a few internal files were missing the header, but now they have it.
`fbcode/common/rust/netstring/` had the internal header, but now it has
GPLV2PLUS - since that goes to Mononoke's Github too.
Differential Revision: D17881539
fbshipit-source-id: b70d2ee41d2019fc7c2fe458627f0f7c01978186
Summary:
Similarly to `Mover`, which acts on paths according to the sync config,
we need a `BookmarkRenamer`, which acts on bookmark names.
The logic now supports two types of bookmark configs:
- common pushrebase bookmarks: they are common for all small repos and they
don't get renamed. `master` is one example of those, and I don't know if
there can be any other :)
- prefixes: when the bookmark is updated in the small repo, an equivalent,
but prefixed bookmark is updated in the large repo. Correspondingly, when
a prefixed bookmark is updated in a large repo, an equivalent, but unprefixed
bookmark is updated in the small repo.
There are also "error" cases:
- bookmark is neither common pushrebase, nor prefixed with the correct small
repo's prefix. In that case we plan not to update that bookmark (although,
create the corresponding commit(we may have to be careful and make it draft
instead of public or something like this))
Reviewed By: krallin
Differential Revision: D17877749
fbshipit-source-id: 6e2f67456de354c2358376e5762fabffe0de3a42
Summary:
Previously function remapped each copy_from changeset separately, however this
is not necessary. copy_from changeset is guaranteed to be one of the parents
(see https://fburl.com/0i80ojja).
This diff simplifies the function a bit by not doing a separate remap for
copy_from changesets
Reviewed By: farnz
Differential Revision: D17779792
fbshipit-source-id: c4c64bdc295858289a26e1d5b229ced6eca96414
Summary:
Given a list of changesets, a source and a target repo id, this query tells
you the first changeset from the list, which comes from the source repo and
has a synced equivalent in the target repo. Here, the word "first" refers to
the order of changesets in the provided list.
The naming is a little unfortunate, as the use case for this query is to
find the *last* (meaning newest) changeset that had been synced. But I wanted
the name to reflect this particular fn's behavior, rather than the intended
use.
Reviewed By: farnz
Differential Revision: D17600505
fbshipit-source-id: ea7336dcf839302e7b9ca6e1bc2ee88a3a1d99a9
Summary: We've improved the MySQL version in review - make the SQLite version match, to avoid nasty test/prod issues
Reviewed By: StanislavGlebik
Differential Revision: D17528682
fbshipit-source-id: 498467210608cce0ec27d014d64a22115355cda2
Summary: Let's have all concerned parties use the same type under the same name.
Reviewed By: farnz
Differential Revision: D17481135
fbshipit-source-id: d0a99cce19fa0941d351f5754ed4517aea6a6105
Summary: This got amended into the wrong commit - land it so that we've got it
Reviewed By: krallin
Differential Revision: D17426107
fbshipit-source-id: 4bb2fcae1d7ea2db5378f39ec0f22c2abf225d74
Summary: To make merges into our final megarepo world Just Work, we want to sustain an interim period where the legacy repo is still present and up to date. Provide a way to do this by having Mononoke internally remap and pushrebase commits so that we can have two repos kept in sync.
Reviewed By: ikostia, StanislavGlebik
Differential Revision: D16918280
fbshipit-source-id: c7a8d1bb1b1972cb5609490b8fd5b759cb5f9cdd
Summary:
`movers` are functions, that we use to shift file paths when syncing commits.
These functions should be automatically buildable from repo-sync configs
for both small-to-large and large-to-small sync directions.
Reviewed By: farnz
Differential Revision: D17395844
fbshipit-source-id: 25ec9b06ba5908d8c125702a712b3cf782ccffca
Summary: `async-unit` no longer has the old problems around deadlock - instead, it runs the tests in a Tokio runtime and threads the failure state (if any) back to the test runner. Use it.
Reviewed By: StanislavGlebik
Differential Revision: D16937389
fbshipit-source-id: 81b87a32b9c8ffdfd4a2033d10809cdcdb39898b
Summary:
This diff moves initFacebook calls that used to happen just before FFI calls to instead happen at the beginning of main.
The basic assumption of initFacebook is that it happens at the beginning of main before there are additional threads. It must be allowed to modify process-global state like env vars or gflags without the possibility of a data race from other code concurrently reading those things. As such, the previous approach of calling initFacebook through `*fbinit::FACEBOOK` near FFI calls was prone to race conditions.
The new approach is based on attribute macros added in D17245802.
---
The primary remaining situations that still require `*fbinit::FACEBOOK` are when we don't directly control the function arguments surrounding the call to C++, such as in lazy_static:
lazy_static! {
static ref S: Ty = {
let _ = *fbinit::FACEBOOK;
/* call C++ */
};
}
and quickcheck:
quickcheck! {
fn f(/* args that impl Arbitrary */) {
let _ = *fbinit::FACEBOOK;
/* call C++ */
}
}
I will revisit these in a separate diff. They are a small fraction of total uses of fbinit.
Reviewed By: Imxset21
Differential Revision: D17328504
fbshipit-source-id: f80edb763e7f42b3216552dd32f1ea0e6cc8fd12
Summary: I think these are left over from pre-2018 code where they may have been necessary. In 2018 edition, import paths in `use` always begin with a crate name or `crate`/`super`/`self`, so `use $ident;` always refers to a crate. Since extern crates are always in scope in every module, `use $ident` does nothing.
Reviewed By: Imxset21
Differential Revision: D17290473
fbshipit-source-id: 23d86e5d0dcd5c2d4e53c7a36b4267101dd4b45c
Summary: When we sync commits, we will need to be able to cross-reference the commits between the source repo and the target repo. Make it easy to track this mapping.
Reviewed By: StanislavGlebik
Differential Revision: D16896918
fbshipit-source-id: 3738cf2067291f16ce9b62509763ac526d1d35c2
