Summary:
As with other methods like gettreepack and getfiles, let's use parallelism.
Later we may make the buffer size configurable if we need to.
Reviewed By: lukaspiatkowski
Differential Revision: D7587674
fbshipit-source-id: 51be6d0e338fb062a116fe6c723f272c66f9da2a
Summary:
Previously tree diff was sequential. That means that this is very slow for any
more or less big trees. By using select_all we can make it parallel.
Reviewed By: lukaspiatkowski
Differential Revision: D7550920
fbshipit-source-id: d0fec1b7387b64e94049c58bbd741885a3d65f21
Summary:
Add CachingBlobstore that uses Asyncmemo as a cache.
Normal Blobstore returns Option<Bytes>. CachingBlobstore doesn't cache `None` response, because this blob may appear later in the blobstore, and we want clients to be able to access it.
Reviewed By: farnz
Differential Revision: D6579385
fbshipit-source-id: 746a5c222dd00a35c3353e76c2792fc8e2323381
Summary:
That prevents us from implementing, for example, Weight for Bytes. There are no
impl specialization in Rust yet, and since Bytes do not implement HeapSizeOf,
we can't implement Weight for Bytes.
However we want to use Asyncmemo with Bytes. So in order to fix the issue let's
remove overgeneric Weight implementation and add implementation for the most
basic things.
Reviewed By: jsgf
Differential Revision: D7639162
fbshipit-source-id: b81d7e50599fdd8bdb8f9903d7aedff71734cdf2
Summary:
This helps out with the stricter-types-in-blobstore work I'm doing,
and is nice cleanup anyway.
Reviewed By: StanislavGlebik
Differential Revision: D7625562
fbshipit-source-id: c27901ef94a03a716368403b63a6afd74a1c2a90
Summary:
Now that `BlobNode` no longer returns `None`:
* don't expose the `BlobNode` API outside the crate because it turns out to not be very useful (it should probably go away eventually?)
* make the `File` API not return `Option` types
* Add a new `file_contents` that returns a brand-new `FileContents` (this is the first time we're tying together Mercurial and Mononoke data structures!)
Also remove a `Symlink` API that isn't really correct honestly.
Reviewed By: StanislavGlebik
Differential Revision: D7624729
fbshipit-source-id: 38443093b8bfea91384c959f3425cf355fac9f65
Summary:
This is not only the newer, more specific type -- it also makes a couple
of upcoming diffs more straightforward.
Reviewed By: StanislavGlebik
Differential Revision: D7622906
fbshipit-source-id: 4e453b827512c538f4f9777ae4d24627f3b124cf
Summary:
This state never actually ended up being used.
There's API cleanup that can happen here, but that's a TODO.
Reviewed By: StanislavGlebik
Differential Revision: D7620266
fbshipit-source-id: 069a890d4f6b7b4e198461a7159015916d0d4109
Summary: Having an implicit `From` parsing makes it hard to track the exceptional places where RevlogChangest can be directly translated to BlobChangeset. Make it explicit for better tracking of this behavior
Reviewed By: StanislavGlebik
Differential Revision: D7637247
fbshipit-source-id: 781341315102ea6b2265c33bb09a89aae3d0c329
Summary:
I'm going to put in some stronger typing around what can be stored in
the blob store. Centralizing the management here makes that much easier.
Reviewed By: StanislavGlebik
Differential Revision: D7619519
fbshipit-source-id: a428679018f0a1571e54bc01bb5483ba9fdf1cb5
Summary: The Hg prefix is unique now so let's not use verbose mercurial::
Reviewed By: sid0
Differential Revision: D7620112
fbshipit-source-id: 0aece310ed817445fef4c94b32f78fda3a3b1c49
Summary: mercurial_types::DBlobNode should be replaced by types from mononoke_types or mercurial in most cases. This rename should help with tracking this
Reviewed By: sid0
Differential Revision: D7619793
fbshipit-source-id: 261fd92acae825dc4bc8011c3716c5585eb0413c
Summary: mercurial_types::DParent should be replaced by types from mononoke_types or mercurial in most cases. This rename should help with tracking this
Reviewed By: sid0
Differential Revision: D7619686
fbshipit-source-id: 5ad105113779387f1408c806860483e06ed5fb3d
Summary: mercurial_types::DEntryId should be replaced by types from mononoke_types or mercurial in most cases. This rename should help with tracking this
Reviewed By: sid0
Differential Revision: D7619571
fbshipit-source-id: bf8d81ec9ffe6a5525d923d7ee67d8e92498aa4d
Summary: mercurial_types::DFileNodeId should be replaced by types from mononoke_types in most cases. This rename should help with tracking this
Reviewed By: sid0
Differential Revision: D7619290
fbshipit-source-id: aa6a8e55ae3810c4531028c3b3db2e5730fe7846
Summary: mercurial_types::DManifestId should be replaced by types from mononoke_types in most cases and by mercurial::HgManifestId in others. This rename should help with tracking this
Reviewed By: sid0
Differential Revision: D7619062
fbshipit-source-id: 447224194c6555334b64dc29ebabe3ef0d0cb87e
Summary: mercurial_types::DChangesetId should be replaced by types from mononoke_types in most cases and by mercurial::HgChangesetId in others. This rename should help with tracking this
Reviewed By: sid0
Differential Revision: D7618897
fbshipit-source-id: 78904f57376606be99b56662164e0c110e632c64
Summary: mercurial_types::NodeHash should be replaced by types from mononoke_types in most cases and by mercurial::NodeHash in others. This rename should help with tracking this fact.
Reviewed By: sid0
Differential Revision: D7618389
fbshipit-source-id: a876e723d911df626c7851fba56a056843b4e049
Summary:
Turns out this trait is actually really useful. In this case it'll be
used to generate blobstore types.
Reviewed By: StanislavGlebik
Differential Revision: D7608152
fbshipit-source-id: 792de037c8b028172f868b9b035ede3db9a39788
Summary:
This is going to make life more convenient in upcoming diffs, when these strings are going to be used to generate blobstore keys.
The switch to lowercase is to make those blobstore keys look nicer.
The switch to string literals is because Rust really supports strings a lot better than it does bytestrings. In particular, `concat!` for literals works with strings, but there's no equivalent for bytestring literals. For current purposes the two are equivalent -- there's no runtime cost to using a string literal here.
Reviewed By: StanislavGlebik
Differential Revision: D7598616
fbshipit-source-id: f2fa543429921cca5fd8dc354ea11000fdab3f76
Summary:
If there are multiple parents, it is important to know which parent to
follow for copy information.
Reviewed By: StanislavGlebik
Differential Revision: D7597811
fbshipit-source-id: 74bb0080671b18beb5483b978016bf66a9496967
Summary:
I haven't actually fixed the bug here because that's quite a bit of work that
will hopefully soon be somewhat moot.
Reviewed By: StanislavGlebik
Differential Revision: D7594278
fbshipit-source-id: b29861c71628a7dc903d6405c3348a9e41002fba
Summary:
I'm adding a non-`HgBlob` path separately, and using `HgBlob` here is
inconvenient.
Reviewed By: StanislavGlebik
Differential Revision: D7589854
fbshipit-source-id: 21e425e7d70691b982dbd66b1ea4996d1e0eef66
Summary: Migrates some uses of `.map_err()` and `.context(format!())` usage in Mononoke to `.with_context()`
Reviewed By: lukaspiatkowski
Differential Revision: D7607935
fbshipit-source-id: 551538c78a1755f7aa0716532ab437a1baf6dd89
Summary:
This should be mostly a no-op change, only the types change so that it prepares this code to be able to remap Root Manifest ids.
There is one small logical change that I am also shipping here, which is that fetching of linknode is happening in parallel with fetching of parents and content of the entry.
Reviewed By: StanislavGlebik
Differential Revision: D7600960
fbshipit-source-id: 2233f1c7d6f14e38cb075b5d7f40339d61cbf574
Summary: getbundle returns changelogs back to Mercurial, so it's IDs should be retrieved from the remapping tables. This diff prepares it for that change
Reviewed By: StanislavGlebik
Differential Revision: D7600959
fbshipit-source-id: e586a92b272e604a1de2619feb9838e982dd802c
Summary:
mercurial::NodeHash is the representation of Mercurial's Sha1 hash, so it should be used to parse and generate hgproto.
This diff doesn't not change two crucial places that are required for full coverage of NodeHash remapping in future:
- Bundle2's tree manifest generation code
- Bundle2's resolver code
Those will be added in dependent diffs
Reviewed By: sid0
Differential Revision: D7600471
fbshipit-source-id: 40e05d5cce6c454200169f6f0049e57d427e9403
Summary:
This is a cleanup of NodeHash API. There were few unused methods and few ways to convert between mercurial and mercurial_types hashes. With this diff it is very easy to identify the places where this converstion happens.
A followup of this diff will be to use this new API to easily replace the NodeHash convertions in places where it requires remapping.
Reviewed By: sid0
Differential Revision: D7592876
fbshipit-source-id: 6875aa6df1a3708ce54ca5724f6eb960d179192b
Summary:
Let's fill cs table even if we import only part of the repo. This let's us
import new changesets incrementally.
That can be dangerous since we don't check if parent commits are present.
However this blobimport is a temporary measure until we get a full-fidelity
blobimport that uses a commit API.
Reviewed By: jsgf
Differential Revision: D7485495
fbshipit-source-id: 63ba91bad4eb1c1662db73293c76a506f48a4753
Summary: They are no longer used in Mononoke and they were replaced with filenodes
Reviewed By: farnz
Differential Revision: D7443319
fbshipit-source-id: 91769de0d4db9f3adf9290070d459c13aa4ed8cc
Summary: They are replaced by filenodes
Reviewed By: farnz
Differential Revision: D7443320
fbshipit-source-id: 13c7d07bc00dcbaa991663c8da8a07fcb0de1332
Summary:
This is used in a bunch of places throughout bundle2-resolver, and is
the default key that should be used throughout Mercurial.
Reviewed By: farnz
Differential Revision: D7578081
fbshipit-source-id: 8f44b54189b22b0aba8f08853bbd2bbbaa3efc5c
Summary:
This will probably go away soon, but for now I want to be able to
disambiguate the new Thrift-encoded blobs in Mononoke from these.
Reviewed By: StanislavGlebik
Differential Revision: D7565808
fbshipit-source-id: d61f3096fa368b934a923dee54a0ea1e3469ae0d
Summary:
`left` and `right` are too vague in this context, since there's
already `Added` and `Removed` which imply an explicit direction.
Reviewed By: StanislavGlebik
Differential Revision: D7560975
fbshipit-source-id: 847f600508434d62857ff60a6e6678a4426ffa88
Summary: Need this for bonsai changeset tests
Reviewed By: StanislavGlebik
Differential Revision: D7560093
fbshipit-source-id: caa0f92fa7ef1c1adb7775835615cb1cb3c7b6aa
Summary:
This is going to be useful to test out some of the work I'm doing with
generating bonsai changesets.
Reviewed By: StanislavGlebik
Differential Revision: D7555735
fbshipit-source-id: 26175244ed59ba080e1740d004d299d4d15ff593
Summary: Found while writing a test against this.
Reviewed By: StanislavGlebik
Differential Revision: D7555159
fbshipit-source-id: c6f279f41974e08ab690befcc84718c2bb22ddc0
Summary:
It increases parallelism, hence speeds up gettreepack request.
Note that currently buffer size is hardcoded. We may change it later, but
right now let's not add too many config knobs.
Reviewed By: farnz
Differential Revision: D7568349
fbshipit-source-id: d1833f1f64438da96c619746d205733af406c921
Summary:
Buffering many file requests increases parallelism.
Note that currently buffer size is hardcoded. We may change it later, but right
now let's not add too many config knobs.
Reviewed By: farnz
Differential Revision: D7568348
fbshipit-source-id: 80456d189cc21d050fcaf7d17cabf8cd59d61d63
Summary:
Trace parent fetching and linknode fetching for gettreepack
And trace blobstore calls. Note that we are interested in tracing both single
blobstore call and in the retrying manifold calls. Because there can be short
single blobstore calls that fail, but overall retrying manifold call can take a few
seconds.
Reviewed By: farnz
Differential Revision: D7550958
fbshipit-source-id: d607812f558900ee55ddd61b529b7b43e9827f49
