Summary:
Use .chain_err() where appropriate to give context to errors coming up from
below. This requires the outer errors to be proper Fail-implementing errors (or
failure::Error), so leave the string wrappers as Context.
Reviewed By: lukaspiatkowski
Differential Revision: D9439058
fbshipit-source-id: 58e08e6b046268332079905cb456ab3e43f5bfcd
Summary:
These are the types that we currently need to be able to serialize if we're to
replace `Asyncmemo`'s caching uses with cachelib. Derive the `Abomonation`
trait for all of them.
Reviewed By: jsgf
Differential Revision: D9082597
fbshipit-source-id: 910e90476a3cc4d18ba758226b8572c3e8d264c6
Summary: See the doc comment for what this state represents and why it is necessary.
Reviewed By: StanislavGlebik
Differential Revision: D9025772
fbshipit-source-id: b0588d037365194bbf2d9889ead60237ef097359
Summary:
Now bonsai changesets are created at the same time as hg changesets, and
the mapping between bonsai and hg changesets is recorded
One important piece is missing. At the moment copy info information is ignored.
I'll add it in the next diffs.
Before diff is landed, I re-run the blobimport to prefill missing bonsai changesets.
Reviewed By: farnz
Differential Revision: D8893504
fbshipit-source-id: 1cc4bbcca2f489a0ef6990d6c04d5b3fd8bef92c
Summary: Previously there was no way to get at the underlying `u64` in a Generation object. I want to use this in order to determine how deep into the commit graph I need to lazily index nodes while answering queries. This will help keep the indexing limited to only what is relevant to incoming queries.
Reviewed By: StanislavGlebik
Differential Revision: D9009058
fbshipit-source-id: 1b4ec44b8245ead75f3097048e85b0a1aafe84f6
Summary:
It's useful to get the hash of the changeset. Looks like we have to clone, but
I'm happy to avoid doing it if that's possible.
Reviewed By: farnz
Differential Revision: D9013977
fbshipit-source-id: bdf06b457fb11c222afddc623c71f3b6b6be30fc
Summary:
Extras value may be non-utf8, for example, hg-git extension generate binary
diff which may be non-utf8.
At the same time it seems that it's better to leave extras key as a utf8
string. However, I'm open to changing it if there is a big pushback
Reviewed By: sunshowers, farnz
Differential Revision: D8991340
fbshipit-source-id: 8f9f33ab3ea77281ae33e0bc735b15201720b761
Summary:
RFC3339 is a standard way to represent dates. See https://tools.ietf.org/html/rfc3339
We're going to use this in code to check for commits older than a certain time.
Reviewed By: farnz
Differential Revision: D9017006
fbshipit-source-id: a4e4325f32d84d4be9b160adf428411d014fb62b
Summary: Those structures will be used in next diffs to store the FilenodeInfo inside memcache for caching purposes
Reviewed By: farnz
Differential Revision: D9014213
fbshipit-source-id: 4952a90415d4b8ab903387fd5cdfaf08d9870c07
Summary: would like there to be an easier way to print these out.
Reviewed By: StanislavGlebik
Differential Revision: D8888556
fbshipit-source-id: 67634ba81ca7ed5789dbc744ef5ab2a4f26be07e
Summary:
I don't like this because particularly the empty string for regular
files looks weird.
Reviewed By: StanislavGlebik
Differential Revision: D8888553
fbshipit-source-id: 20a9048a19b3fdfe681160a637bc2dfc8932c113
Summary:
We need to store relation between Hg changesets and Bonsai changesets.
- `BonsaiHgMapping` is exactly this mapping which establishes injective relation between `{Hg|Bonsai}Changeset`
Reviewed By: StanislavGlebik
Differential Revision: D8801254
fbshipit-source-id: c7df14172e6c2d67c039a24e1bb821e6d92860af
Summary:
This is a series of patches which adds Cargo.toml files to all the crates and tries to build them. There is individual patch for each crate which tells whether that crate build successfully right now using cargo or not, and if not, reason behind that.
Following are the reasons why the crates don't build:
* failure_ext and netstring crates which are internal
* error related to tokio_io, there might be an patched version of tokio_io internally
* actix-web depends on httparse which uses nightly features
All the build is done using rustc version `rustc 1.27.0-dev`.
Pull Request resolved: https://github.com/facebookexperimental/mononoke/pull/7
Differential Revision: D8778746
Pulled By: jsgf
fbshipit-source-id: 927a7a20b1d5c9643869b26c0eab09e90048443e
Summary: Moved the Generation struct, which is a u64 wrapper, from repogen to mononoke-types, and updated the according usages. This should make it easier to phase out RepoGenCache.
Reviewed By: farnz
Differential Revision: D8725538
fbshipit-source-id: cfd39be03bae56d2288053b7b5e212e6d547c833
Summary:
I really liked how expressing the problem in terms of data structures
made it fairly straightforward to understand.
Reviewed By: farnz
Differential Revision: D8586383
fbshipit-source-id: d1e3c92a3a5b760a0f13f4912420e2a73b937e8d
Summary:
Fetching the blob is still required to compute the node hash, but we don't have
to reupload it.
Reviewed By: farnz
Differential Revision: D8508462
fbshipit-source-id: 341a1a2f82d8f8b939ebf3990b3467ed7ad9244c
Summary:
Store manifests as Thrift blobs instead. Required fixing up a lot of
different places, but they should all be pretty clear now.
Reviewed By: farnz
Differential Revision: D8416238
fbshipit-source-id: 523e3054e467e54d180df5ba78445c9b1ccc3b5c
Summary: Added logic to save `FileChange` as a Mercurial format `HgBlobEntry`
Reviewed By: sunshowers
Differential Revision: D8187792
fbshipit-source-id: 4714c81ab23ebac528cfec15c4a9e66083d4fb6c
Summary:
Implementation of generic `store|fetch` for bonsai types.
- bonsai types have unique typed hashes associated with each bonsai type, I'm leveraging this fact to implement generic `store|fetch` methods on `BlobRepo`
Reviewed By: farnz
Differential Revision: D8254810
fbshipit-source-id: 5f798fade4cb8d1ac851f94c7ad7e64636bbca65
Summary:
- `store|fetch` semantic requires `MononkeId::Value` type but `UnodeId` does not have associated value type.
- `UnodeId` is not used anywhere in codebase
Nuking it for now.
Reviewed By: lukaspiatkowski
Differential Revision: D8351415
fbshipit-source-id: 970ab8d3c0cdca6d77c96ea08643f90a4ce0e624
Summary:
We're going to need to be able to edit memory manifests. Provide
remove and set operations, to match the Bonsai Changeset data structures
Reviewed By: StanislavGlebik
Differential Revision: D7620527
fbshipit-source-id: e85459c5dbfa8855267fd2cf6578c9fc39f223f8
Summary:
Rust 1.26 adds many new language features. In particular `impl Trait` is now
stable, so we no longer need `conservative_impl_trait`.
There also seems to have been changed in the (unstable) TryFrom with respect to
usize, and the behaviour of the never type `!`.
There are still a few deprecation warnings, but they don't cause the build to
fail.
Path remapping is now stable, so the buck config needs to change to use it
rather than the unstable command line option.
TODO:
- get aarch64 rust-crates-io build (can defer to a later update)
Reviewed By: Imxset21
Differential Revision: D7966091
fbshipit-source-id: 2e61e262c21eb01c852a36f49c6a6369cdaddcdb
Summary: As with changesets and blobs, let's cache filenodes data
Reviewed By: jsgf
Differential Revision: D7831105
fbshipit-source-id: 334cb474f5cc3ef8dba0945d11273b2b3875e8ad
Summary:
We'd like to move away from `RawNodeBlob` and `RawCSBlob` to data structures
serialized by Thrift. This is the first step to doing that.
The most important thing here is that it reuses file content IDs from native
Mononoke storage.
Reviewed By: jsgf
Differential Revision: D7771990
fbshipit-source-id: de4ee0b56aa6610caeff84b2235e19855df086cb
Summary: Want to get this in now before I forget.
Reviewed By: jsgf
Differential Revision: D7781551
fbshipit-source-id: a5e6fa062514d90113c96307f78e20fa05f770ea
Summary: Will be used for Thrift envelopes.
Reviewed By: jsgf
Differential Revision: D7771214
fbshipit-source-id: 3f0dbf77793064f2606ebe34672629c4e49cc7fe
Summary: This is an important consistency check for bonsai changesets.
Reviewed By: jsgf
Differential Revision: D7755664
fbshipit-source-id: 4e64ed532d0730147efec7a2b10b61f625e50dd0
Summary:
This does mean that we need to adjust QuickCheck generation a bit to ensure
validity.
Reviewed By: jsgf
Differential Revision: D7755635
fbshipit-source-id: ebca6f5a52038b64f83ff599079460daf061cb44
Summary: I got annoyed at the lack of this.
Reviewed By: StanislavGlebik
Differential Revision: D7755636
fbshipit-source-id: 0d519dca94ed93e3efe8d12f9004661b92fb8b1a
Summary:
This redefines file changes and deletes slightly -- see the comments
for a full description.
Reviewed By: jsgf, farnz
Differential Revision: D7750040
fbshipit-source-id: f1c52295cd21150fbdc909198104c0571132431d
Summary:
We know that the hashes for non-root-tree-manifests and filenodes
should always be consistent. Verify that.
Reviewed By: farnz
Differential Revision: D7704087
fbshipit-source-id: 7f6207878c5cd372b272aa6970506dd63b5a3c7c
Summary:
This is the first step to getting bonsai changesets into the blob store.
So far the code just waits for content blob uploads to be done. In future diffs Mononoke will synthesize this information into a bonsai changeset as well.
Reviewed By: lukaspiatkowski
Differential Revision: D7627784
fbshipit-source-id: da0c96772ce4d18aed579cecca6135137a8dbe18
Summary:
I was trying to debug something with the new blobimport, and this was
getting in the way.
Reviewed By: StanislavGlebik
Differential Revision: D7664660
fbshipit-source-id: 2ec4ee79fbe13584f35e7dcd9e8df2b8bdf181c0
Summary:
Currently, any sort of `Bytes` can be stored in the blobstore. That
caused me to make several mistakes while writing the code to store bonsai
changesets, because I'd just end up storing the wrong set of `Bytes`.
Introduce stronger typing so that only types that explicitly implement
`BlobStorable` can be stored in the blobstore.
Currently, these sorts of blobs can be stored in the blob store:
* `ChangesetBlob` and `ContentBlob` in `mononoke-types` (these are Thrift-serialized structures)
* The envelope `RawNodeBlob` and `RawCSBlob` types in `blobrepo`, once converted to `EnvelopeBlob` instances
* `HgBlob`, which contains revlog data (manifests or files) exactly as serialized by Mercurial
Reviewed By: StanislavGlebik
Differential Revision: D7627290
fbshipit-source-id: d1bcbde8881e365dec99618556e7b054985bccf7
Summary:
This is not only the newer, more specific type -- it also makes a couple
of upcoming diffs more straightforward.
Reviewed By: StanislavGlebik
Differential Revision: D7622906
fbshipit-source-id: 4e453b827512c538f4f9777ae4d24627f3b124cf
Summary:
Turns out this trait is actually really useful. In this case it'll be
used to generate blobstore types.
Reviewed By: StanislavGlebik
Differential Revision: D7608152
fbshipit-source-id: 792de037c8b028172f868b9b035ede3db9a39788
Summary:
This is going to make life more convenient in upcoming diffs, when these strings are going to be used to generate blobstore keys.
The switch to lowercase is to make those blobstore keys look nicer.
The switch to string literals is because Rust really supports strings a lot better than it does bytestrings. In particular, `concat!` for literals works with strings, but there's no equivalent for bytestring literals. For current purposes the two are equivalent -- there's no runtime cost to using a string literal here.
Reviewed By: StanislavGlebik
Differential Revision: D7598616
fbshipit-source-id: f2fa543429921cca5fd8dc354ea11000fdab3f76
Summary:
If there are multiple parents, it is important to know which parent to
follow for copy information.
Reviewed By: StanislavGlebik
Differential Revision: D7597811
fbshipit-source-id: 74bb0080671b18beb5483b978016bf66a9496967
Summary:
This will help in case we ever need to abandon this format in the
future.
Reviewed By: jsgf
Differential Revision: D7483425
fbshipit-source-id: 0cca098ca861cca0994c06709849ececffb38fc5
Summary:
The current structure makes defining new manifests really verbose and
doesn't easily support nested manifests. Make it really simple: either accept a
path map or a description (a csv).
Also switch to a `BTreeMap` to allow lookups in the next diff.
This is unfortunately hard to separate out because all the bits are
interconnected.
Reviewed By: StanislavGlebik
Differential Revision: D7531115
fbshipit-source-id: ca0fa35d4e0ac6a4e23adb2a7d2b5679ce52b643
Summary:
Since `FileType` now exists, the `Type` enum can use it instead of
defining its own stuff.
Reviewed By: farnz
Differential Revision: D7526046
fbshipit-source-id: 3b8eb5502bee9bc410ced811dc019c1ce757633f
Summary: Very similar to serialize and deserialize for bonsai changesets.
Reviewed By: farnz
Differential Revision: D7380859
fbshipit-source-id: 42567ae3c2e52597b3708871358165ec54a8c04a
Summary: This name is less misleading. Thanks markbt for suggesting it.
Reviewed By: farnz
Differential Revision: D7380821
fbshipit-source-id: 96abb846fd8979cc322e1f176d15caf24bb00874
Summary: The compact_protocol code is from jsgf.
Reviewed By: farnz
Differential Revision: D7365901
fbshipit-source-id: 4cef925ad60c579188ab016fed3f027b3944a3a1
Summary:
This also means that a direct conversion from a byte slice to a Blake2
is no longer desirable. Forcing people to go through the typed hashes seems
quite desirable.
Reviewed By: farnz
Differential Revision: D7365900
fbshipit-source-id: 36f4afd5a53f857c8952c636b0db0b4b509a6f44
