Summary: - Make `Bookmakrs` work with `ChangsetId` instead of `HgChangesetId`
Reviewed By: StanislavGlebik, farnz
Differential Revision: D9297139
fbshipit-source-id: e18661793d144669354e509271044410caa3502a
Summary:
This is a series of patches which adds Cargo.toml files to all the crates and tries to build them. There is individual patch for each crate which tells whether that crate build successfully right now using cargo or not, and if not, reason behind that.
Following are the reasons why the crates don't build:
* failure_ext and netstring crates which are internal
* error related to tokio_io, there might be an patched version of tokio_io internally
* actix-web depends on httparse which uses nightly features
All the build is done using rustc version `rustc 1.27.0-dev`.
Pull Request resolved: https://github.com/facebookexperimental/mononoke/pull/7
Differential Revision: D8778746
Pulled By: jsgf
fbshipit-source-id: 927a7a20b1d5c9643869b26c0eab09e90048443e
Summary:
Unify all uses of Sqlite and of Mysql
This superceded D8712926
Reviewed By: farnz
Differential Revision: D8732579
fbshipit-source-id: a02cd04055a915e5f97b540d6d98e2ff2d707875
Summary:
We need to be able to distinguish logical transaction failure from
infrastructure failure, so change the `Transaction::commit` future to
`Future<Item=bool, Error=Error>`, where `false` indicates a logical transaction
failure. This allows the caller to determine whether a retry or other recovery
logic is needed.
Reviewed By: lukaspiatkowski
Differential Revision: D8555727
fbshipit-source-id: 8ab64f3019f2644e7eaabc8d699d99aa8eb08fbb
Summary:
All sorts of weird can happen. The reason it didn't is because we had just one
repo
Reviewed By: lukaspiatkowski
Differential Revision: D8530459
fbshipit-source-id: aa3acd393f2dc96ea2141d09deb0209cbc29a740
Summary:
Now it is as it should be: mercurial_types have the types, mercurial has revlog related structures
burnbridge
Reviewed By: farnz
Differential Revision: D8319906
fbshipit-source-id: 256e73cdd1b1a304c957b812b227abfc142fd725
Summary: It only needs to borrow them.
Reviewed By: kulshrax
Differential Revision: D8244267
fbshipit-source-id: 2a24a3b7c6eb65177e4e26c57650dd7e096b4202
Summary:
This will make it easier to change the "real" bookmark type from AsciiString to
String if we decide to do that.
BookmarkPrefix is a separate type because we may want to change it from
AsciiString to String. Also we don't want to confuse a bookmark prefix with a
bookmark name.
Reviewed By: jsgf
Differential Revision: D7909992
fbshipit-source-id: 3d4d075c204ed5ef1114a743430982c2836bac04
Summary: For on-disk-rocksdb use cases we should persist bookmarks like any other table we use
Reviewed By: farnz
Differential Revision: D7728717
fbshipit-source-id: f63a6410f5ed254a719a16a7504d1b31da5a20a8
Summary:
The diff size is formidable, however it's not that bad really. This is just
about moving the code inside the macro and tests in a separate file.
This is similar to changesets or filenodes crates
Reviewed By: lukaspiatkowski
Differential Revision: D7639681
fbshipit-source-id: 4216652780bf99939245ae39e508b3b46cf96a03
Summary:
This is similar to what we've already done in filenodes and changesets. Sqlite
implementation just grabs the lock, future mysql implementation will get a
connection from pool.
Reviewed By: lukaspiatkowski
Differential Revision: D7585387
fbshipit-source-id: 95a046808f4d78d7776a26bdf3f7d939b1ee0451
Summary: mercurial_types::DChangesetId should be replaced by types from mononoke_types in most cases and by mercurial::HgChangesetId in others. This rename should help with tracking this
Reviewed By: sid0
Differential Revision: D7618897
fbshipit-source-id: 78904f57376606be99b56662164e0c110e632c64
Summary:
Bump bookmark size limit - we use 512 for infinitepush, let's use the same
here. Also add a primary key
Reviewed By: farnz
Differential Revision: D7534967
fbshipit-source-id: aeef926de910a3a9934fb1588778f8f503821071
Summary:
Implementation of the bookmarks trait using diesel. Currently it's quite slow
because it uses mutex to make the struct sync. Also it's not really async, even
though it returns future. The plan to make it async is to move the computation
on a separate thread pool.
Reviewed By: kulshrax
Differential Revision: D7255935
fbshipit-source-id: ce622a14af777ae470aecc039cef8c6cab9db41e
Summary:
Mononoke will introduce its own ChangesetId, ManifestId and BlobHash, and it
would be good to rename these before that lands.
Reviewed By: farnz
Differential Revision: D7293334
fbshipit-source-id: 7d9d5ddf1f1f45ad45f04194e4811b0f6decb3b0
Summary:
First draft of the new bookmarks trait. Note that it doesn't support Bookmarks
& Heads simultaneous transactions.
There are a few notable differences from old bookmarks trait:
1) New `Transaction` trait was added that handles all write operations.
2) It has two different set methods: `set()` and `set_unconditionally()`.
`set()` checks the current bookmark value before updating it,
and `set_unconditionally()` doesn't.
Reviewed By: farnz
Differential Revision: D7138038
fbshipit-source-id: 8cbec5caa972d847a30baa0a351df651306b4fd1
Summary:
We are going to significantly change Bookmark trait and implementation.
We are going to use Diesel, and new Bookmark trait won't use Version. Instead
of changing the current implementation, let's just move it to the separate
directory which will soon be deleted.
Reviewed By: lukaspiatkowski
Differential Revision: D7138039
fbshipit-source-id: 72b81bf7f6524052b3a360bac23ab56a978b08cb
Summary: Update the bookmarks module to use ChangesetId to represent bookmarks, rather than NodeHash.
Reviewed By: lukaspiatkowski
Differential Revision: D6774650
fbshipit-source-id: 1742e4e78798ad68a7f17ebd345eef14a7de2cec
Summary:
Don't use failure's bail!() and ensure!() macros.
Instead, failure_ext provides:
- bail_err!(err) - Converts its single parameter to the expected error and returns; ie `return Err(From::from(err));`
- bail_msg!(fmt, ...) - takes format string parameters and returns a `failure::err_msg()` error
- ensure_err!(), ensure_msg!() - corresponding changes
Also:
- remove all stray references to error-chain
- remove direct references to failure_derive (it's reexported via failure and failure_ext)
- replace uses of `Err(foo)?;` with `bail_err!()` (since `bail_err` unconditionally returns, but `Err(x)?` does not in principle, which can affect type inference)
Reviewed By: kulshrax
Differential Revision: D6507717
fbshipit-source-id: 635fb6f8c96d185b195dff171ea9c8db9e83af10
Summary:
Convert scm/mononoke to use failure, and update common/rust crates it depends on as well.
What it looks like is a lot of deleted code...
General strategy:
- common/rust/failure_ext adds some things that are in git failure that aren't yet in crates.io (`bail!` and `ensure!`, `Result<T, Error>`)
- everything returns `Result<T, failure::Error>`
- crates with real error get an error type, with a derived Fail implementation
- replicate error-chain by defining an `enum ErrorKind` where the fields match the declared errors in the error! macro
- crates with dummy error-chain (no local errors) lose it
- `.chain_err()` -> `.context()` or `.with_context()`
So far the only place I've needed to extract an error is in a unit test.
Having a single unified error type has simplified a lot of things, and removed a lot of error type parameters, error conversion, etc, etc.
Reviewed By: sid0
Differential Revision: D6446584
fbshipit-source-id: 744640ca2997d4a85513c4519017f2e2e78a73f5
Summary:
This diff does a few things:
- Change the rust versions in `third-party{2,-buck}/config.py` to 1.21.
- Update the tp2 symlinks for `rust` and `rust-crates-io`
- Fix build breakages due to new errors/warnings from rustc.
Reviewed By: jsgf, Imxset21
Differential Revision: D6319954
fbshipit-source-id: cd4fe9e0d6f26c1a6c9c3f1256d84cb002bb83d6
Summary:
As part of removing excess genericism, make Bookmarks a trait with no
associated types or type parameters. I will revisit error types here once I
have removed the Repo trait - these are not ideal, but enough to get things
going
Reviewed By: StanislavGlebik
Differential Revision: D6405315
fbshipit-source-id: 814c106612a061e1766f1ea9a9428a13a73bd007
Summary: Add DbBookmarks to the common bookmarks test crate, which runs tests that all bookmarks implementations must pass.
Reviewed By: farnz
Differential Revision: D6215852
fbshipit-source-id: 64238c584eb05cbccdef78a68cec6c8f93b14df1
Summary: Add a new implementation of the Bookmarks trait backed by a MySQL database.
Reviewed By: farnz
Differential Revision: D6215856
fbshipit-source-id: 4258a6195a946917a1878e87a53f9bbc6ca64400
Summary: In practice, Mononoke requires that the Value associated type of the Bookmarks trait be NodeHash. As a result, in the case of DbBookmarks, rather than making the implementation fully generic over values, I've made it specialized for NodeHashes for simplicity's sake. This means in order to use these tests, they need to store NodeHashes instead of Strings.
Reviewed By: farnz
Differential Revision: D6217789
fbshipit-source-id: 3f0f04180eb0869eb9de3da7be634d95c2b24d84
Summary:
The `Bookmarks` trait assumes that the underlying implementation is asynchronous (as evidenced by the fact that all methods return `Future`s). Most non-toy implementations of an asynchronous bookmark store will need to schedule arbitrary tasks on the Core executing the returned Futures. (In some cases, the constructor itself may need to schedule tasks on the Core.) As such, this test harness should expose a Core to each test in a first class way (rather than trying to shoehorn it in via the `state`, which doesn't actually quite work here).
Existing implementations can just ignore the Core.
Reviewed By: jsgf
Differential Revision: D6215855
fbshipit-source-id: ea3f28bdaec667111b384db29d0c7c49954398e5
Summary:
This field is optional. We need to ensure in linknodes that the
version is always set to a fixed value so that linknode blobs written out to
disk are deterministic.
Reviewed By: jsgf
Differential Revision: D6214032
fbshipit-source-id: c1eea7bf28bb096c1d23165e1f50ff2dc6d21274
Summary:
We need ownership of the buffer in all of these cases, and
`AsRef<Path>` could potentially create unnecessary copies.
Reviewed By: jsgf
Differential Revision: D6214034
fbshipit-source-id: 806a87bfe3b125febaaaaf26c8b8dcac407de145
Summary:
All bookmark implementations, including the upcoming MySQL-based one,
should work with these tests. Don't duplicate them in every test crate.
Reviewed By: StanislavGlebik
Differential Revision: D6075454
fbshipit-source-id: 547e3c2c98326a6fdca55a562811924ebec93796
Summary:
This code is generally useful and will also be used for linknodes.
Note that I marked `filekv` as a copy of `filebookmarks` to preserve blame
info. Git doesn't track renames so this unfortunately won't be transferred to
the open source repo.
Reviewed By: farnz
Differential Revision: D6015776
fbshipit-source-id: 2c17e2440e25e6179dd71d16a87095ad1b346b49
Summary:
This Version type is going to form the basis for other key-value
stores like linknodes, so it needs to be moved into a separate crate.
I've chosen `storage_types` as the name because it seems to be the most obvious
candidate.
Reviewed By: jsgf
Differential Revision: D6015772
fbshipit-source-id: 52de7866d68fdec2a4908626679a6f08c5f73402
Summary:
Will be making functional changes to these files separately and don't
want the two to interfere.
Reviewed By: jsgf
Differential Revision: D6015773
fbshipit-source-id: 26529ce4075ac47e5f0e80177319e1beb90c2076
Summary:
Realized that this didn't have `#![deny(warnings)]`, so deprecated
tokio APIs hadn't been cleaned up.
Reviewed By: farnz
Differential Revision: D6015774
fbshipit-source-id: 10f8e8b28b7d6e81ef88ca2382e331cde822ac66
Summary: Change usage of BoxStream from futures crate to futures-ext crate and replace stream::iter() with stream::iter_ok().
Reviewed By: jsgf
Differential Revision: D5938460
fbshipit-source-id: bfb90844e646079d72b35495f409591a65686978
Summary:
I'm going to factor out some of this code into a new crate, and I want
to get these changes out of the way first.
Reviewed By: lukaspiatkowski
Differential Revision: D5924471
fbshipit-source-id: 7007638ce688e62397637ff6f9a2e51247ec9312
Summary: membookmarks can't fail, so use `!` for its `Error` type
Reviewed By: sid0
Differential Revision: D5849875
fbshipit-source-id: 5dbb4edf9bb57dde55877d2c90936b51dcc5d7ff
Summary: All its users need this, and it makes sense.
Reviewed By: jsgf
Differential Revision: D5847753
fbshipit-source-id: 042c7b5637e5f83918fa9bc5d131c8745d7bbc3e
Summary:
Finally got an update working by removing the `mysql_async` crate.
Some notes:
* The `mysql_async` crate was responsible in this case: see https://github.com/rust-lang/cargo/issues/4066#issuecomment-328210204 for why.
* tokio/futures deprecated a bunch of stuff. I've filed a TODO for now.
* We finally pulled in error-chain 0.11, which has a bunch of nice improvements.
Reviewed By: kulshrax
Differential Revision: D5798282
fbshipit-source-id: a38a7b17ee0205428e2ea63334722aa408582493
Summary:
The url::percent_encoding module was split into its own crate. Use
that instead.
Reviewed By: jsgf
Differential Revision: D5769525
fbshipit-source-id: 4d95c5d791e486f56a16091ffa5114b3cfbfd47e
Summary:
Add BoxedBookmark to assist in boxing a Bookmark trait, and implement Bookmark
for Box<_> and Arc<_> Bookmark trait objects.
Reviewed By: sid0
Differential Revision: D5540675
fbshipit-source-id: 7582adbd2858f6814d57b7e54089b922886a4d54
Summary:
In practice, no users of Bookmarks will not want listing, and all
implementations need to be able to support it.
Reviewed By: StanislavGlebik
Differential Revision: D5540672
fbshipit-source-id: d64a3894c7cd14d329bd23809525935461289bc3
Summary: Allow the error type to be any type the internal errors can be converted to.
Reviewed By: sid0
Differential Revision: D5540671
fbshipit-source-id: 1077132d25bcc13ba2a3a509c916f2e07ac10dcf
