Summary: It will be used by many things, in particular by gettreepack method
Reviewed By: jsgf
Differential Revision: D6845036
fbshipit-source-id: 3174dfafeeab830ededb70ddff3dd62f6a608d54
Summary: Change BlobChangeset and callers to use ChangesetId instead of NodeId
Reviewed By: lukaspiatkowski
Differential Revision: D6835450
fbshipit-source-id: 7b20359837632aef4803e40965380c38f54c9d0a
Summary: Update the bookmarks module to use ChangesetId to represent bookmarks, rather than NodeHash.
Reviewed By: lukaspiatkowski
Differential Revision: D6774650
fbshipit-source-id: 1742e4e78798ad68a7f17ebd345eef14a7de2cec
Summary:
This will (in practice) always be `Arc<Blobstore>`, so push down and
make it the concrete type.
Reviewed By: jsgf
Differential Revision: D6761690
fbshipit-source-id: c279751d98a8882c209b79faa7cbfd3e635beda5
Summary:
We're never going to serve RevlogRepo in production, and we're down to
a single BlobRepo type that will have different backing stores. Remove the
unused trait, and use BlobRepo everywhere bar blobimport and repo_config
(because we previously hardcoded revlog here - we want to change to a BlobRepo
once blobimport is full-fidelity).
Reviewed By: jsgf
Differential Revision: D6596164
fbshipit-source-id: ba6e76e78c495720792cbe77ae6037f7802ec126
Summary:
Remove the last associated types from BlobStore - this means that
BlobStore now has an associated trait object type.
Reviewed By: jsgf
Differential Revision: D6425414
fbshipit-source-id: 7186dab9b56593dd1d70be732d4ad56d1e7b3c63
Summary:
Don't use failure's bail!() and ensure!() macros.
Instead, failure_ext provides:
- bail_err!(err) - Converts its single parameter to the expected error and returns; ie `return Err(From::from(err));`
- bail_msg!(fmt, ...) - takes format string parameters and returns a `failure::err_msg()` error
- ensure_err!(), ensure_msg!() - corresponding changes
Also:
- remove all stray references to error-chain
- remove direct references to failure_derive (it's reexported via failure and failure_ext)
- replace uses of `Err(foo)?;` with `bail_err!()` (since `bail_err` unconditionally returns, but `Err(x)?` does not in principle, which can affect type inference)
Reviewed By: kulshrax
Differential Revision: D6507717
fbshipit-source-id: 635fb6f8c96d185b195dff171ea9c8db9e83af10
Summary:
Convert scm/mononoke to use failure, and update common/rust crates it depends on as well.
What it looks like is a lot of deleted code...
General strategy:
- common/rust/failure_ext adds some things that are in git failure that aren't yet in crates.io (`bail!` and `ensure!`, `Result<T, Error>`)
- everything returns `Result<T, failure::Error>`
- crates with real error get an error type, with a derived Fail implementation
- replicate error-chain by defining an `enum ErrorKind` where the fields match the declared errors in the error! macro
- crates with dummy error-chain (no local errors) lose it
- `.chain_err()` -> `.context()` or `.with_context()`
So far the only place I've needed to extract an error is in a unit test.
Having a single unified error type has simplified a lot of things, and removed a lot of error type parameters, error conversion, etc, etc.
Reviewed By: sid0
Differential Revision: D6446584
fbshipit-source-id: 744640ca2997d4a85513c4519017f2e2e78a73f5
Summary:
BlobStore is entirely generic, and puts no limits on its
implementations. Remove ValueIn and ValueOut type parameters, and insist that
all blobs are Bytes (as per production setups)
Reviewed By: StanislavGlebik
Differential Revision: D6425413
fbshipit-source-id: 455e526d8baebd0d0f1906941648acca89be4881
Summary:
BlobStore is entirely generic, and puts no limits on its
implementations. Remove the "Key" type parameter, and insist that all keys are
String (as per production setups)
Reviewed By: StanislavGlebik
Differential Revision: D6425412
fbshipit-source-id: 1f1229bf8e001bf780964e883c6beb071e9ef1d8
Summary:
As part of removing excess genericism, make Bookmarks a trait with no
associated types or type parameters. I will revisit error types here once I
have removed the Repo trait - these are not ideal, but enough to get things
going
Reviewed By: StanislavGlebik
Differential Revision: D6405315
fbshipit-source-id: 814c106612a061e1766f1ea9a9428a13a73bd007
Summary:
Returning the bookmarks object gets in the way of degenericising
bookmarks. It's also not in line with other methods on the Repo trait - the
repo handles querying the underlying storage, not the user.
Switch to providing pass-through interfaces for bookmarks.
Reviewed By: StanislavGlebik
Differential Revision: D6408644
fbshipit-source-id: 2808850a070b7bcc478cd40d824bdc8d3acb8b0f
Summary:
As part of removing excess genericism, make Heads a trait with no
associated types or type parameters.
Reviewed By: StanislavGlebik
Differential Revision: D6352727
fbshipit-source-id: df9ef87e0e0abe43c30e7318da38d7f930c37c6e
Summary: We are going to use them to fetch not only file blobs, but also tree blobs.
Reviewed By: kulshrax
Differential Revision: D6336010
fbshipit-source-id: 77fcc45698a447c10d180baf929465f1d7e4ee72
Summary: Also support them in `generate_memblob_repo.py`.
Reviewed By: jsgf
Differential Revision: D6215721
fbshipit-source-id: c8b855bbc74b619bc093b3aac449a283a1ad33ec
Summary:
Bookmarks and heads are not used in eden server, but user still have to specify
--blobrepo-folder. Let's use MemHeads and MemBookmarks instead to avoid
specifying this parameter
Reviewed By: sid0
Differential Revision: D6099149
fbshipit-source-id: 26927bdd5c5dba2164688e72d238587f5535ccfc
Summary:
`RepoPath` represents any absolute path -- root, directory or file. There's a
lot of code that manually switches between directory and file entries --
abstract all of that away.
Reviewed By: farnz
Differential Revision: D6201383
fbshipit-source-id: 0047023a67a5484ddbdd00bb57bca3bfb7d4dd3f
Summary:
I spent too much time debugging something that was made a lot easier
once I used `impl Future`.
Reviewed By: StanislavGlebik
Differential Revision: D6094810
fbshipit-source-id: 6bdcf6159a87df56eb61d85a51ade31397ff6994
Summary:
Manifold requires a separate thread to do the IO, so we create a separate
detached thread.
Reviewed By: sid0
Differential Revision: D6063729
fbshipit-source-id: a67d1da955aa2cd34e923a0762bbb76c858728b1
Summary:
Blobrepo state with file-based bookmarks and heads and manifold blobstore.
It will be used for eden server testing.
Reviewed By: sid0
Differential Revision: D6063728
fbshipit-source-id: f0542265af015d5c20db225ed6bf85cae954a3ab
Summary:
Mercurial filelog entries may have metadata fields in the beginning, usually used to track copies/renames. Previously mononoke eden server returned this metadata as part of the file blob.
This diff changes it. Now `get_content()` method returns file content without metadata, and to make it consistent, both `get_content()` of the blobrepo and revlog repo do the same.
This decision certainly has it's tradeoffs, because now it's more difficult to get metadata (`get_raw_content` needs to be used).
But we'll probably change how metadata is stored anyway, that's why I think changing `get_content` method is fine.
This diff also cleans up server/src/main.rs file, because previously it had to strip metadata itself.
Also diff fixes problem in metadata parsing - it previously failed if file is less than 2 bytes
Reviewed By: farnz
Differential Revision: D5901476
fbshipit-source-id: f3ade0179710352590068c238e6a733aab68a512
Summary:
`Path` has the potential to be confused with `std::path::Path`.
`MPath` is nice, concise, and clearly different from `Path`.
Reviewed By: jsgf
Differential Revision: D5895665
fbshipit-source-id: dc5ed5c3866b227d753c6d904d3c6d213c882cd7
Summary: file blobstore was read data from incorrect folders. This diff fixes it.
Reviewed By: farnz
Differential Revision: D5852736
fbshipit-source-id: 1bc08788023ff2cc9d631985d70329475afc589f
Summary:
We're going to add more stores like obsmarkers and linknodes very
soon. Stuffing all of them into type parameters is going to get ugly very soon.
Instead, add a trait which represents all the state, and make `BlobRepo`
generic on that trait. Add a few implementations for common use cases like
files and RocksDB-based stores.
This allows the dependency lists for the Mononoke and Eden servers to be more
straightforward.
Reviewed By: jsgf
Differential Revision: D5850372
fbshipit-source-id: 37a0687611687b9616ebbddce70f53e1e5d3267b
Summary:
Just like the previous diff with bookmarks, every user needs this and it makes
sense anyway.
Reviewed By: jsgf
Differential Revision: D5847877
fbshipit-source-id: b91a49e94da2d7e207061b6c52b78c55a2229dec
Summary: All its users need this, and it makes sense.
Reviewed By: jsgf
Differential Revision: D5847753
fbshipit-source-id: 042c7b5637e5f83918fa9bc5d131c8745d7bbc3e
Summary: Also ensure that `blobimport` doesn't use its own copy.
Reviewed By: jsgf
Differential Revision: D5847604
fbshipit-source-id: 5390848cd5fab8abd967ef9701720491d703c0f1
Summary:
Removes usage of BoxFuture and BoxStream from futures crate and instead use
it from internal futures_ext lib.
Reviewed By: sid0
Differential Revision: D5847771
fbshipit-source-id: 01823513176add37caa73c0ea2810e80d7fae955
Summary:
Finally got an update working by removing the `mysql_async` crate.
Some notes:
* The `mysql_async` crate was responsible in this case: see https://github.com/rust-lang/cargo/issues/4066#issuecomment-328210204 for why.
* tokio/futures deprecated a bunch of stuff. I've filed a TODO for now.
* We finally pulled in error-chain 0.11, which has a bunch of nice improvements.
Reviewed By: kulshrax
Differential Revision: D5798282
fbshipit-source-id: a38a7b17ee0205428e2ea63334722aa408582493
Summary:
`:` is a reserved character for Windows paths, so Mercurial rejects
them from being committed. Use `-` instead, so that we can commit file blob
repo test fixtures.
Reviewed By: kulshrax
Differential Revision: D5731525
fbshipit-source-id: 8d14fc03f1b135cbc4d42aeaf2f3a0ae6d13f956
Summary: Would be nice to be able to return future that references a boxed Entry
Reviewed By: jsgf
Differential Revision: D5688859
fbshipit-source-id: acb7bcc461f886bca08b84f31dbebe4f692187f1
Summary: Main part is `get_stream_of_manifest_entries` that creates a stream of all tree manifest entries by recursively going through all of them.
Reviewed By: jsgf
Differential Revision: D5622490
fbshipit-source-id: 4a8b2707df0300a37931c465bafb1ed54d6d4d25
Summary:
Adds method similar to https://doc.rust-lang.org/std/path/struct.Path.html#method.join.
Will be used in the next diff to prepend path prefix to the tree manifest entries
Reviewed By: jsgf
Differential Revision: D5563808
fbshipit-source-id: 3637275093c301bf159083cb9bfedaa0e490a75e
Summary:
A preparation step before blob importing of tree manifest repos to blobrepo.
`get_parents()` method of BlobEntry reads parents from the blobstore. It works fine for file entries because file entries can stores its parents in the blobstore. With tree manifests BlobEntry can contain also tree manifest entries, and that means that tree manifest entries parents should also be stored somewhere in the blobstore.
I suggest to use the same logic for the tree manifest entries as for the file entries. File and manifest entries have two blobstore entries - one stores hash of the content and parents, another stores the actual content.
To do this I moved `RawNodeBlob` and `get_node()` to the separate module and made fields public.
Reviewed By: jsgf
Differential Revision: D5622342
fbshipit-source-id: c9f0c446107d4697b042544ff8b37a159064f061
Summary:
Let's create a separate function `fetch_file_blob_from_blobstore` that will be
used to get content of the blob without creating file::BlobEntry.
file::BlobEntry requires Path and type parameters that won't be passed to eden
server.
Reviewed By: jsgf
Differential Revision: D5537647
fbshipit-source-id: 13fb2fbba7d3068b88aac2d842cc92b4da6ca80c
Summary: Will be used by eden server in the next diff
Reviewed By: jsgf
Differential Revision: D5553955
fbshipit-source-id: 196e2da597d0456b4337a6d9ada8fe188075e87e
Summary: Add helpers to allow an implementation of a Repo object to be boxed up into a type-erased trait object.
Reviewed By: sid0
Differential Revision: D5540673
fbshipit-source-id: c16332f6a548d053125dd3376e7e06d0ddffadf9
Summary: Add a `get_bookmarks()` method to the Repo trait and implement it for RevlogRepo and BlobRepo.
Reviewed By: sid0
Differential Revision: D5540667
fbshipit-source-id: 710849e50b1d358be1c4f95a1cbd61efb7c2cf6b
Summary: Blobrepo can take any Heads or Blobstore implementation, each of which may have their own error type. Error chain doesn't support type parameters, and can only have foreign links for specific concrete types. To handle this we have a couple of placeholder errors in Blobrepo's errors which are chained onto the real underlying error.
Reviewed By: sid0
Differential Revision: D5540676
fbshipit-source-id: 9543e204754f22dfd68a04aec5a484fa85d53ff8
Summary:
blobimport appends ".bincode" in the end of the file.
Let's do the same in blobrepo
Reviewed By: sid0
Differential Revision: D5527661
fbshipit-source-id: d54fe52c1d282f038a47ac2f2cd0aaf917a92227
