Summary:
This is prerequisite step before moving hg blob `fetch|upload` functionality to mercurial crate.
- Use `Arc<dyn Blobstore>` instead of `RepoBlobstore`
- do not use `BlobRepo` in `UploadHg*` and `fetch_*` structures and functions
Reviewed By: ikostia
Differential Revision: D16963058
fbshipit-source-id: be36599cb430a342348cc077885b2f32eaa4ee47
Summary:
We now have two Manifest and two Entry traits and it makes it very confusing.
The old Manifest trait (and related EmptyManifest struct and Entry trait) are
mercurial-specific, while new Manifest trait is generic over all kinds of
manifests (unodes, mercurial etc).
In ideal state we should have only new Manifest trait, but it'd require quite a
big refactoring. In short term let's at least rename Manifest to HgManifest and
Entry to HgEntry to make distinction clearer.
Reviewed By: krallin
Differential Revision: D16886032
fbshipit-source-id: 80e3999fae3e89347448579caf7aafec56705500
Summary:
It's used only in a very few places, and most likely that's by accident. We
pass logger via CoreContext now
Reviewed By: krallin
Differential Revision: D16336953
fbshipit-source-id: 36ea4678b3c3df448591c606628b93ff834fae45
Summary: `MemoryManifest` is not longer used anywhere
Reviewed By: farnz
Differential Revision: D16148769
fbshipit-source-id: f4e63fed3d56ade122560d271bd5701110c5dab8
Summary:
Removed references to HgNodeHash in repo_client in the specified functions. In
addition, updated other files due to type related dependencies.
Reviewed By: StanislavGlebik
Differential Revision: D14543934
fbshipit-source-id: b0d860fe7085ed4b91a62ab1e27fb2907a642a1d
Summary: The Copy trait means that something is so cheap to copy that you don't even need to explicitly do `.clone()` on it. As it doesn't make much sense to pass &i64 it also doesn't make much sense to pass &<Something that is Copy>, so I have removed all the occurences of passing one of ouf hashes that are Copy.
Reviewed By: fanzeyi
Differential Revision: D13974622
fbshipit-source-id: 89efc1c1e29269cc2e77dcb124964265c344f519
Summary:
Previously pushrebasing an empty commit failed because we assumed that root
manifest of a commit is always sent in a bundle. This diff removes this
assumption
Reviewed By: lukaspiatkowski
Differential Revision: D13818556
fbshipit-source-id: 44e96374ae343074f48e42a90c691b21e3c41386
Summary:
Currently we read all bookmarks from primary replica a few times during `hg
pull`. First time when we do listkeys, second time when we get heads.
That might create a high load on primary replica.
However the delay between primary and secondary replicas is fairly small, and so it
*should* be fine to read bookmarks from secondary local replica as long as there is only
one replica per region (because if we have a few replicas per region, then
heads and listkeys response might be inconsistent).
Reviewed By: lukaspiatkowski
Differential Revision: D13039779
fbshipit-source-id: e1b8050f63a3a05dc6cf837e17a448c3b346b723
Summary:
Recently blobrepo_utils_tests started to deadlock. And it looks like it happens
because of `Future::wait()` in blobrepo_utils_tests macro - https://fburl.com/2jho0i2e.
I've added some logging in Mononoke and tokio code and got the following
sequence of events:
1) A test calls `Future::wait()`. Note that this blocks one of the worker threads
from the threadpool.
2) In `VisitOne::visit()` function (https://fburl.com/wmxvt7mq) the work is
split between the threads with `tokio::spawn()` function.
3) `tokio::spawn()` adds a future to some worker thread's queue.
4) If this worker thread is the thread that called `Future::wait()` then we get
a deadlock - recently spawned future can't complete because a thread is
blocked, and thread is blocked because `Future::wait()` waits for the spawned
future to complete.
This diff fixes by avoiding using `Future::wait()` and using tokio::spawn
instead.
Reviewed By: farnz
Differential Revision: D13021358
fbshipit-source-id: 6f1adda43f9460dc952e5794c2fa147b234731ee
Summary: Make get_manifest_by_nodeid accept HgManifestId and correct all calls to get_manifest_by_nodeid.
Reviewed By: StanislavGlebik
Differential Revision: D10298425
fbshipit-source-id: 932e2a896657575c8998e5151ae34a96c164e2b2
Summary:
Alas, the diff is huge. One thing is changing Changesets to use ChangesetId.
This is actually quite straightforward. But in order to do this we need to
adapt our test fixtures to also use bonsai changesets. Modifying existing test
fixtures to work with bonsai changesets is very tricky. Besides, existing test
fixtures is a big pile of tech debt anyway, so I used this chance to get rid of
them.
Now test fixtures use `generate_new_fixtures` binary to generate an actual Rust
code that creates a BlobRepo. This Rust code creates a bonsai changeset, that
is converted to hg changeset later.
In many cases it results in the same hg hashes as in old test fixtures.
However, there are a couple of cases where the hashes are different:
1) In the case of merge we are generating different hashes because of different
changed file list (lukaspiatkowski, aslpavel, is it expected?). this is the case for test
fixtures like merge_even, merge_uneven and so on.
2) Old test fixtures used flat manifest hashes while new test fixtures are tree
manifest only.
Reviewed By: jsgf
Differential Revision: D9132296
fbshipit-source-id: 5c4effd8d56dfc0bca13c924683c19665e7bed31
Summary: This verifier works on manifests -- we'll need other ones for filenodes etc.
Reviewed By: StanislavGlebik
Differential Revision: D9105976
fbshipit-source-id: 939b72b6a3f69b716385315f69a57e91d71a45f3
Summary: See the doc comment for what this state represents and why it is necessary.
Reviewed By: StanislavGlebik
Differential Revision: D9025772
fbshipit-source-id: b0588d037365194bbf2d9889ead60237ef097359
Summary: This is a workaround for buggy commits produced by older versions of treemanifest.
Reviewed By: farnz
Differential Revision: D9018502
fbshipit-source-id: 71f2aeeaee01500a1a73814dc9c4979ef2c79746
Summary:
These types are Hg specific. Since we are going to add bonsai changeset
creation soon, let's make it clear in the types
Reviewed By: farnz
Differential Revision: D8911359
fbshipit-source-id: 8b6cc45122402d7b7e074e66d904d979030de705
Summary:
The goal of this tool is to verify that bonsai changesets roundtrip with the same hashes. This uses the `ChangesetVisitor` stuff introduced in the previous diff to walk over a specific number of changesets.
I've been using this tool for a few days, and it's shaken out a number of bugs in the Bonsai support in Mononoke, and even found old bugs in Mercurial. So it's been a pretty useful exercise overall.
Reviewed By: farnz
Differential Revision: D8817725
fbshipit-source-id: b6976e8ff9024098c695e82383d98d95b2701e1b
Summary:
This allows parallel walks across thousands of changesets.
In my testing with the Bonsai verification tool this seemed to work well,
keeping my cores saturated almost all the time.
Something to pay attention to is the way cancellation works -- it seems to make
sense, but I could have gotten something subtly wrong :)
Reviewed By: farnz
Differential Revision: D8909719
fbshipit-source-id: f7f47df479431bca05f2cd1508c9c9a4e8a2af50
