Summary: Compaction can slow down blobimporting a lot. Let's add an option to postpone it till the end
Reviewed By: farnz
Differential Revision: D5882003
fbshipit-source-id: 0611a8e94b3d7331bdacf909d820526f547414a0
Summary: Removed use of deprecated Tokio APIs. Firstly, removed usage of BoxFuture and BoxStream from futures crate and instead used it from futures_ext crate. Also, replaced stream::iter() with stream::iter_ok().
Reviewed By: sid0
Differential Revision: D5882902
fbshipit-source-id: 329231cca798371701e8966251a43a78db6e4ee1
Summary: Generation has nothing to do with machine words or pointer sizes, so it shouldn't be using usize. Use u64 instead.
Reviewed By: wez
Differential Revision: D5885943
fbshipit-source-id: 49b748cf592e270f78d85fe3c39b8d1a2a3e1671
Summary:
lukaspiatkowski pointed out that I don't test that an input that's NotReady doesn't hit
the panic cases. Add tests to check that this works.
Reviewed By: jsgf
Differential Revision: D5881789
fbshipit-source-id: 6357262c070c8c6a524d7bedeb1966843bd7612b
Summary: Teach the revsets code to do set intersection.
Reviewed By: jsgf
Differential Revision: D5881788
fbshipit-source-id: 972fd85a47ee29a1037f5e4072c03e25189b8300
Summary:
Switch from hand-rolled errors that eat the underlying cause to
`error_chain!` errors that allow you to extract what happened.
Reviewed By: StanislavGlebik, lukaspiatkowski
Differential Revision: D5862059
fbshipit-source-id: 4f7e849249b124cc6f48498045de1d01a717619b
Summary:
Manually adding all the dependencies in the TARGETS file is quite annoying. This simple tool does it for you!
It queries buck to get list of files for the target, and also queries all the rust_library mononoke dependencies.
Then it parses source files and generates TARGETS list.
Tool is super-simple and that's why it's not always correct (for example, it fails to find a deps if hasn't been used in mononoke before), but it works good enough for many use-cases (see test plan).
Reviewed By: lukaspiatkowski
Differential Revision: D5861828
fbshipit-source-id: ab8cc96d8bc394af172ee09da3aceb0ffb7493e8
Summary: This diffs reverts D5842977 and adds fetching mononoke-config inside tupperware, so that it can be served by mononoke server
Reviewed By: StanislavGlebik
Differential Revision: D5866239
fbshipit-source-id: 6f3fd794e5274c8bab0c50efc9c95e31f88be688
Summary: memheads can't fail, so use `!` for its `Error` type
Reviewed By: sid0
Differential Revision: D5849874
fbshipit-source-id: fbefb3cdaa11d9c10cbfc988e43978751c807472
Summary: membookmarks can't fail, so use `!` for its `Error` type
Reviewed By: sid0
Differential Revision: D5849875
fbshipit-source-id: 5dbb4edf9bb57dde55877d2c90936b51dcc5d7ff
Summary: memblob can't fail, so use `!` for its error type
Reviewed By: sid0
Differential Revision: D5849876
fbshipit-source-id: 48ca1c345d788a5f6e632d73f55bc485c036a0e3
Summary:
We can use the "never_type" feature to enable more general use
of the `!` type, which indicates that the type state is impossible (ie,
`!` has no values). Not only is this more expressive than using - say -
`()`, it also allows irrefutable matching against a result (`let Ok(x)
= ...`), since the compiler knows that `Err(!)` can never happen.
Reviewed By: sid0
Differential Revision: D5849877
fbshipit-source-id: 48c14f9c2042fb8168eb1dd308fd3a79c9ed0fc2
Summary:
We want a mechanism to quickly and easily find sets of revisions in
Mononoke. As a first step, introduce a mechanism to find single nodes, and a
mechanism to union two streams of nodes.
Later diffs will add the other 2 set operations, plus range (`a::b` in
Mercurial terminology).
Reviewed By: jsgf
Differential Revision: D5778925
fbshipit-source-id: 9e320ccab21c9b7b9c0fab8fbbc5c7389d2b7da0
Summary: file blobstore was read data from incorrect folders. This diff fixes it.
Reviewed By: farnz
Differential Revision: D5852736
fbshipit-source-id: 1bc08788023ff2cc9d631985d70329475afc589f
Summary:
`indicatif` is a command-line progress bar crate for Rust. It seems
pretty generally useful, and I plan to use it in the Dewey client replacement.
As a side note, it was written by Armin Ronacher (of Flask fame), so it should
hopefully be solid. See https://github.com/mitsuhiko/indicatif.
As part of this update, the `bitflags` crate went from `0.9.1` to `1.0.0`, which adds
a breaking change wherein bitflag constants are now namespaced by their
containing struct. This diff includes the appropriate fixes for revlog and zeus,
which seem to be the only things affected.
Reviewed By: jsgf
Differential Revision: D5855753
fbshipit-source-id: 38d88335126e787f3acd92189a1ed17f55138799
Summary:
Without this, any lines that are too long would cause `rustfmt` to
error out. That seems not very useful.
Reviewed By: kulshrax
Differential Revision: D5853676
fbshipit-source-id: b7335a14079ebf6245da0c028039d88745b32785
Summary:
We're going to add more stores like obsmarkers and linknodes very
soon. Stuffing all of them into type parameters is going to get ugly very soon.
Instead, add a trait which represents all the state, and make `BlobRepo`
generic on that trait. Add a few implementations for common use cases like
files and RocksDB-based stores.
This allows the dependency lists for the Mononoke and Eden servers to be more
straightforward.
Reviewed By: jsgf
Differential Revision: D5850372
fbshipit-source-id: 37a0687611687b9616ebbddce70f53e1e5d3267b
Summary: I'm going to move this to a common location very soon.
Reviewed By: jsgf
Differential Revision: D5850319
fbshipit-source-id: 09937659330de93cf74bf827f225418dd413865d
Summary:
Just like the previous diff with bookmarks, every user needs this and it makes
sense anyway.
Reviewed By: jsgf
Differential Revision: D5847877
fbshipit-source-id: b91a49e94da2d7e207061b6c52b78c55a2229dec
Summary: All its users need this, and it makes sense.
Reviewed By: jsgf
Differential Revision: D5847753
fbshipit-source-id: 042c7b5637e5f83918fa9bc5d131c8745d7bbc3e
Summary: Also ensure that `blobimport` doesn't use its own copy.
Reviewed By: jsgf
Differential Revision: D5847604
fbshipit-source-id: 5390848cd5fab8abd967ef9701720491d703c0f1
Summary:
Finally use hashencode to hash long path.
This is basically a translation of core mercurial python code to rust.
Tests are also copied from core mercurial
Reviewed By: jsgf
Differential Revision: D5719573
fbshipit-source-id: a53bdeafe22cc9f492b8c08d4ae302a393f9b8fb
Summary: Encoding only index path is not enough. It works fine for now because we don't use hash encoding. Next diff adds hash encoding, so we need to encode datapath too.
Reviewed By: jsgf
Differential Revision: D5719574
fbshipit-source-id: 4f2a4a75baad73313e80ffb81031166d4bab3e29
Summary:
Let's use separate function fsencode instead of two methods fsencode_dir and fsencode_file.
There are a few reasons for that:
1) This is similar to upsteram mercurial code - it also uses separate function, not a method of the class.
2) Path is supposed to represent a file in the mercurial vfs. Previously we joined this file with "00manifest.i" - it creates a file that doesn't exist in mercurial vfs. This point is debatable though, so I'm fine with making it a method of the class. But probably it doesn't matter that much.
2) We never actually need to encode directory - even in tree manifest case we use fsencode to find location of`00manifest.*` files. That means, that we don't really need to have separate fsencode_dir function, so I was wrong when I added them in the first place.
3) Special hash encoding is used to encode paths that are longer than 120 chars (will be added in the next diff). `00manifest.i` and `00manifest.d` are used in the hash digest, and that means that one fsencode_dir() method is not enough - we'd need to add separate methods to fsencode idx file and separate method to fsencode data file.
Reviewed By: jsgf
Differential Revision: D5719576
fbshipit-source-id: ca6b38dd7d0c6c0c5a345d8fcbe1b0d6fa10a062
Summary:
We want to be able to test against repo fixtures. Turn repo fixtures
that have been created by blobimport into in-memory BlobRepos for testing.
Reviewed By: jsgf
Differential Revision: D5834109
fbshipit-source-id: d6fc01f19986b5a6951f25c7c8bda8aa7c9dee56
Summary:
Removes usage of BoxFuture and BoxStream from futures crate and instead use
it from internal futures_ext lib.
Reviewed By: sid0
Differential Revision: D5847771
fbshipit-source-id: 01823513176add37caa73c0ea2810e80d7fae955
Summary:
Allow Arc<R> where R: Repo to be used more generally. Also Box for
completeness.
Reviewed By: farnz
Differential Revision: D5846659
fbshipit-source-id: e101647badfb18a4a59d595ee9b21814b332fe34
Summary: Use `impl Future` rather than a boxed future.
Reviewed By: sid0
Differential Revision: D5829773
fbshipit-source-id: 40c4339e96f7194544f416534952b78a23d93fa6
Summary: The idea is to deploy a no-op mononoke server to tupperware and slowly add more parts like fb303 page, proper args for startup etc.
Reviewed By: farnz
Differential Revision: D5842977
fbshipit-source-id: 227d144b71132e73b76ce7067ac998b1945cb5e1
Summary: Makes a formerly manual process automatic.
Reviewed By: jsgf
Differential Revision: D5832456
fbshipit-source-id: 0bfa01aaf08f0794a3b98ab7290d163a65650a7e
Summary:
This is going to be useful for a small regenerate script I want to
write and open source.
Reviewed By: yfeldblum
Differential Revision: D5831942
fbshipit-source-id: c1610ba1aa93061662e8cca3052f842b74fa4838
Summary: Switch to the implementations in `futures-ext` added in the previous diff.
Reviewed By: jsgf
Differential Revision: D5799732
fbshipit-source-id: f5d54afe5847bb09f0231d5b3c58c0e2e54c391f
Summary:
This was deprecated upstream, but is still useful to us.
Also add `nonsend` versions since those are still required in a couple of
places.
Reviewed By: jsgf
Differential Revision: D5817016
fbshipit-source-id: 3753fadcae666099a96b3b849e438c5c16c7232f
Summary: Remove the unused `empty()` constructor, and make the `parse` APIs private.
Reviewed By: lukaspiatkowski
Differential Revision: D5796734
fbshipit-source-id: 84e7b146ca89334fbdfda917d5d8eebf04a3e474
Summary: Going to be making significant changes to these files soon.
Reviewed By: kulshrax
Differential Revision: D5796735
fbshipit-source-id: 879fbca3fc936a538c95e50a3333fc2c312de15b
Summary: This parsing is only to check that the modules work together. The metaconfig module still needs to have some proper parsing implemented, Error handling and tests
Reviewed By: jsgf
Differential Revision: D5698544
fbshipit-source-id: 8cd3254c10f977a17ddc31ac2199ee4c9fa98355
Summary: Simple linear history with 10 changesets as a file blob repo.
Reviewed By: sid0
Differential Revision: D5731223
fbshipit-source-id: bc479304d13f7a2299d3ecc382e052ad5600c46c
Summary:
Serde is overkill for this, and the types we care about for heads can
already convert to/from strings without making bad pathnames.
Reviewed By: lukaspiatkowski
Differential Revision: D5784229
fbshipit-source-id: 28ee55f8c5c34ecd28cd1a7e465436892eb6fee2
