Summary:
`RepoPath::parent` returns the parent of the path. The empty path,
`RepoPath::empty()` does not have a parent so `None` is returned in that case.
(Note: this ignores all push blocking failures!)
Differential Revision: D14276641
fbshipit-source-id: fc60da9750dc76e4f598dd483a63adc180135cb4
Summary:
Tries to split the current `RepoPath` in a parent path and a component. If the
current path is empty then None is returned. If the current path contains only
one component then the pair that is returned is the empty repo path and a path
component that will match the contents `self`.
(Note: this ignores all push blocking failures!)
Differential Revision: D14276644
fbshipit-source-id: e85a22f65cc267e48f12af6bf6b40c7673b7eaaa
Summary:
An iterator for the parent directories of a path.
(Note: this ignores all push blocking failures!)
Differential Revision: D14276646
fbshipit-source-id: 2b580ee8d762db5113110ec9b09ec3a093a1063a
Summary:
This diff defines the empty `RepoPath`. This path is equivalent with the root
of the repository.
Also adding a method to check whether a `RepoPath` matches this special path.
(Note: this ignores all push blocking failures!)
Differential Revision: D14276647
fbshipit-source-id: 6e9ad5957ad39a711a1680bd084f448bb9d73f87
Summary: Now that `Key` has a `Display` implementation that produces useful output (namely the filenode and path as strings rather than bytes), we can use it to get better debug output.
Differential Revision: D14310544
fbshipit-source-id: ace5b76f07aa1216b5e9aae22dc7b6bd561e9560
Summary:
Since all the methods of an aync mutable pack are taking the ownership of self,
there can only be one accessor to self, and therefore the Mutex is not
required. For the same reasons, the Arc can be removed.
This saves a bunch of atomic operation when operating on the mutable packs.
Reviewed By: kulshrax
Differential Revision: D14287252
fbshipit-source-id: 792bc41aa0dc372e2114dfca4895cca7083f3a56
Summary:
Since most of the datastore/historystore types are already Sync, we don't
really need to wrap them in a Mutex. For the ones that aren't Sync, we can
easily wrap them in a Mutex and pass that to AsyncWrapper.
Reviewed By: kulshrax
Differential Revision: D14287253
fbshipit-source-id: 0f5c04c651592561403caa6e4627017ab1731d0a
Summary: Implement the `Display` trait for `Key`. This is important for human-readable output for debugging since both filenode and path inside the `Key` are stored as byte arrays, which are just printed out as numbers by the derived `Debug` implementation.
Differential Revision: D14294970
fbshipit-source-id: a601dc9a0c23ac40894ec27135171928f5635507
Summary:
We currently alias many of the imports from the store submodule. It is nicer
if we just import the submodule name and prefix with the submodule.
Differential Revision: D14276638
fbshipit-source-id: 6661df7f9cb5d976b11153003f653a1f66301c9a
Summary: Add a `get_history()` method to the `EdenApi` trait that queries the API server's `getfilehistory` endpoint and writes the resulting history entries to a historypack in the user's cache.
Differential Revision: D14223269
fbshipit-source-id: bf69c767f5a89177c36e755250330dbbbc219e4f
Summary: Add an `add_entry()` method, similar to D14162781 which did the same for non-async packs.
Differential Revision: D14246980
fbshipit-source-id: 8c8f507f2e0133d80d826d04345df7b41d6013a3
Summary: `AsRef<Path>` is more ergonomic than `&Path` since the former can accept `PathBuf`, `String`, etc.
Differential Revision: D14223167
fbshipit-source-id: 12d26adaa63855c339e04734c19d6697624f9c9e
Summary:
The order of imports we're trying to follow is: std, remote crates
(ie: crates.io), local crates (ie: fbcode crates), and crate local. A new line
in between each group can be used to prevent rustfmt from re-ordering them.
Reviewed By: singhsrb
Differential Revision: D14243163
fbshipit-source-id: fbbb07693af14b13ae6b3f4e788972d99193fd64
Summary:
The order of imports we're trying to follow is: std, remote crates
(ie: crates.io), local crates (ie: fbcode crates), and crate local. A new line
in between each group can be used to prevent rustfmt from re-ordering them.
Reviewed By: singhsrb
Differential Revision: D14243162
fbshipit-source-id: 6fc2cceb3d6834b602be20b8b8f74e0f61b227e1
Summary:
This diff focuses on addding deserialization. Because the most effective way
of testing deserialization is doing round-trip conversions we also implement
serialization.
`manifest::tree::store::Entry` is the structure that is in charge of perfroming
serialization and deserialization. We update the Store trait to interface with
this new object.
Differential Revision: D14178264
fbshipit-source-id: bb12262c181a518ba4111d40c079d6836ec44301
Summary:
The `once_cell` crate is more flexible than `lazy-init`. It has more types,
a richer api, recent updates and more features.
Differential Revision: D14232727
fbshipit-source-id: 14aeb34a96e094069bb8dc3fb5efcf5b5707ce8c
Summary:
The purpose of `FileMetadata` is to be a data container. Using the
`FileMetadata` implies accessing the underlying values. That means that the
underlying values need to be `public`.
Differential Revision: D14178260
fbshipit-source-id: 8b8e5ead23d47962498f991152c2a5fd94ac3c74
Summary: These implementations are going to used in quickcheck tests.
Differential Revision: D14178259
fbshipit-source-id: 0bded67deab3422b4aad53666c14cf195ea1b0d4
Summary:
`as_bytes` is useful for interacting with code that is written in reference
to a `slice` or when attempting to serialize them. It is fine to serialize
these paths using their underlying representation because we request that they
be normalized.
Reviewed By: quark-zju
Differential Revision: D14178263
fbshipit-source-id: 36529e2ae47580ae4014ae279df98b35eac5484a
Summary:
Implementing these traits allows easy use of `RepoPath` and friends in
quickcheck tests.
It is tricky to implement `PathComponentBuf`. I believe that most strings will
end up being a valid `PathComponent` so I think that it is reasonable to loop
until a valid string is found. To note generating Arbitrary Unicode `char`s is
implemented using a loop where random bytes are validated against the `char`
constructor.
Reviewed By: quark-zju
Differential Revision: D14178262
fbshipit-source-id: 759486a2053b851f8259d7cc03eee1cd69893f9f
Summary:
These constant can be useful for parsing byte streams that we expect to contain
`Node`s. These constant should replace all potential hardcoding of values that
intend to represent the byte lengths for `Node`.
Differential Revision: D14178261
fbshipit-source-id: 34471151b5d253504e32a9e8b039608c1d4943fe
Summary:
Instead of using the rand crate imported into quickcheck, we can use the
rand crate directly.
Reviewed By: quark-zju
Differential Revision: D14174653
fbshipit-source-id: c848f139765b9e458d374790227399f0ad836af6
Summary:
The watchman_client crate relied on unix-only crates for both
unix_socket_transport and command_line_transport. Let's not compile
these on windows.
Reviewed By: quark-zju
Differential Revision: D14174654
fbshipit-source-id: 67d26d1799e71a1bf20af1a57a687249f5dce227
Summary: Changing the permission on the packfile failed due to the file being opened.
Reviewed By: quark-zju
Differential Revision: D14174652
fbshipit-source-id: 356ac4748fd69e660a6cb9e63367a87489755e5e
Summary:
Rust tells us that Rng::choose and Rng::shuffle should be replaced by
SliceRandom::choose and SliceRandom::shuffle, so let's do it.
Reviewed By: singhsrb
Differential Revision: D14178565
fbshipit-source-id: 586eb2891f1c2cab0a3435c1b4ae8f870e7a3c25
Summary:
Tell serde to use an [untagged representation](https://serde.rs/enum-representations.html#untagged) of this enum. This means that `Parents::None` will map to a null value, `Parents::One` will map to an array representing a `Node`, and a `Parents::Two` will map to an array of 2 arrays representing `Node`.
Using CBOR serialization, this means that these variants are 1 byte, 21 bytes, and 43 bytes respectively.
Differential Revision: D14174309
fbshipit-source-id: 0217a23c4ee5409ab293525d7b6e5ae969b5504d
Summary: Add implementations for `FromIterator` and `IntoIterator` for the `Parents` type to make it more ergonomic to use.
Reviewed By: quark-zju
Differential Revision: D14172511
fbshipit-source-id: 5ba848c1dfbb8cc23ed19c9dc816616f5ed7af5f
Summary: Add a convenience function to `MutableHistoryPack` to add an entry from a `PackHistoryEntry` struct.
Differential Revision: D14162781
fbshipit-source-id: a0e07f34b9231011a339ce63adcef8ab55a0555e
Summary:
This changes the way we represent history entries for the Eden API by splitting them into two types and putting them into a new `historyentry` module.
- `PackHistoryEntry` is the same as the old `HistoryEntry`, containing the fields required to add this entry to a `MutableHistoryPack` (namely a `Key` and a `NodeInfo`).
- `LooseHistoryEntry` is a history entry containing the information that would normally be present in a line of the history text in the remotefilelog loose file format.
There are several reasons why it makes sense to have both of these types:
- The existing remotefilelog code in Mononoke uses a type very similar to `LooseHistoryEntry` internally, and as such having a similar type for API calls simplifies code on the server side.
- `PackHistoryEntry` contains redundant information (in particular, the file path may be duplicated up to 3 times). While it's structure is ideal for `revisionstore`'s in-memory data structures, for transmitting data, this redundancy is undesirable, especially since the client already has the file path (it is required to make the request in the first place).
- Conversions between these two representations include some subtle details that are tricky to get right. By putting the conversion in one canonical place, we can avoid having to duplicate this conversion logic in multiple places.
Differential Revision: D14162783
fbshipit-source-id: 63e0a060709916f21613442b75370f4d34a04f04
Summary:
Add a type representing a node's parents. Mercurial nodes can have zero, one, or two parents; this is normally represented as an array of two node hashes, with any absent parents denoted with a null hash. This representation is not particularly Rustic because it allows for invalid combinations (such as a null p1 and non-null p2) to be represented. By using an enum, invalid values are unrepresentable, making code using this type simpler (due to not requiring validation logic).
This type will be used later in the stack to represent parents in history entries.
Reviewed By: quark-zju
Differential Revision: D14162782
fbshipit-source-id: dfff3ecc76dea114e0044839216d080b7f34a506
Summary:
`Durable` nodes are inner nodes that come from storage. Their contents can be shared between multiple instances of Tree. They are lazily evaluated. Their children list will be read from storage only when it is accessed.
The inner structure of a durable link poses an interesting question related to handling failures: what do we do when we have a failure when reading from storage?
We can cache the failure or we don't cache it. Caching it is mostly fine if we had an error reading from local storage or when deserializing. It is not the best option if our storage is remote and we hit a network blip. On the other hand we would not want to always retry when there is a failure on remote storage, we'd want to have a least an exponential backoff on retries. Long story short is that caching the failure is a reasonable place to start from.
The lazy-init crate is useful for modeling the lazy initialization model that we have for durable node links. See docs at https://docs.rs/lazy-init/0.3.0/lazy_init/
Reviewed By: quark-zju
Differential Revision: D14142928
fbshipit-source-id: 077f708b38e2ace772f30b3392445326ce17f47c
Summary:
The store abstracts the data that we need from the store for TreeManifest. This is not a long term abstraction, mostly something that we can code against until we write the real parsing logic from the real data store.
Adding a TestStore implementation that we can use in tests and debug.
Reviewed By: quark-zju
Differential Revision: D14142930
fbshipit-source-id: 7f2d4f05a7b7e63758db9247cdbcd51541c88ec0
Summary: Preparing to add the store abstraction for the tree manifest. This store implementations are going to be tied to the tree implementation and should be made private from the rest of the crate. To do this we move the tree to a module.
Reviewed By: quark-zju
Differential Revision: D14142929
fbshipit-source-id: 588d597e1248fc2e632c9efe03f08ba3d491e8cd
Summary: manifest::tree::Link has several convenience functions. In the grand scheme of things this is something that we will want because they are common operations. The problems is that the node structure is currently changing rapidly and the additional layer of abstraction is hindering iteration. I don't know the exact shape that the nodes will have and trying different things out is easier when the Tree functions use the nodes directly.
Reviewed By: quark-zju
Differential Revision: D14142926
fbshipit-source-id: 5e4e8f4d3e1b19fd14dcc290a3dea4271b502d97
Summary: Add conversions between the `HgFileHistoryEntry` type produced by Mononoke and the `edenapi_types::HistoryEntry` type shared between Mercurial and Mononoke. The latter can be serialized and sent from the API server to the Mercurial client, and will be used for HTTP history fetching.
Reviewed By: quark-zju
Differential Revision: D14079338
fbshipit-source-id: 2123c88310f633f87d8c92405382d5046874dfee
Summary: `RepoPath` and `PathComponent` were recently added as path abstractions for source control internals. This diff makes use of these abstractions in the manifest crate.
Reviewed By: quark-zju
Differential Revision: D14142925
fbshipit-source-id: 1e0033f4f2aeb5c1d63899adbce198302086bfda
Summary:
While working on TreeManifest I found that the code does not allow creating an empty `RepoPath`. I did not realize that splitting an empty string always results in an empty string. This is an edge case and we need to handle it appropriately.
Currently adding a hack where the components iterator function will skip empty values. We are going to add more iterators in a future revision and we should revisit this function then.
Reviewed By: quark-zju
Differential Revision: D14142927
fbshipit-source-id: 1be43d61913138e3fa50e02976e81f9ca38a74a7
Summary:
With the Store trait, we can de-duplicate code between the datapack repack, and
the historypack repack.
Reviewed By: quark-zju
Differential Revision: D14091894
fbshipit-source-id: 5bf335414df2420b42ec45cce7097f3a97a49796
Summary: I am making this a new diff because the I am documenting failures which only make sense when validation was already added.
Differential Revision: D14097490
fbshipit-source-id: 7eb2c22ebbbc4365f8203d68bc29134176f326f6
Summary:
When introducing `Component` we said that it will not have any slashes (`/`). This diff adds enforcement for that along with enforcing that `Component` is not empty.
`Path` and by extension `Component` also have a set of invalid bytes: `\0`, `\1` and `\n`. I got these checks from the mononoke codebase (mononoke-types/path.rs)
Reviewed By: DurhamG
Differential Revision: D14001106
fbshipit-source-id: 5acbdf66214e54f1034fd89d90e88c3e904d7f9b
Summary:
`Component` and `ComponentBuf` are specializations of `RelativePath` and `RelativePathBuf` that do not have any separators: `/`. The main operation that is done on paths is iterating over its components. The components are generally directory names and occasionally file names.
For the path: `foo/bar/baz.txt` we have 3 components: `foo`, `bar` and `baz.txt`.
A lot of algorithms used in source control management operate on directories so having an abstraction for individual components is going to increase readability in the long run. A clear example for where we may want to use `ComponentBuf` is in the treemanifest logic where all indexing is done using components. The index in those cases must be able to own component. Writing it in terms of `RelativePathBuf` would probably be less readable that writing it in terms of `String`.
This diff also adds the `RelativePath::components` which acts as an iterator over `RelativePath` producing `Component`.
The name `Component` is inspired from `std::path::Component`.
Reviewed By: DurhamG
Differential Revision: D14001108
fbshipit-source-id: 30916d2b78fa89537fc5b30420b3b7c12d1f82c7
Summary:
`RelativePath` and `RelativePathBuf` are types for working with paths specialized for source control internals. They are akin to `str` and `String` in high level behavior. `RelativePath` is an unsized type wrapping a `str` so it can't be instantiated directly. `RelativePathBuf` represents the owned version of a `RelativePath` and wraps a `String`.
RelativePaths are going to be used in all contexts for SCM so flexible and efficient abstractions for dealing with common needs is important.
The inspiration for `RelativePath` and `RelativePathBuf` comes from the `std::path` crate however we know that the internal representation of a path is consistently an array of bytes where directories are delimited by `/` thus our types can have a simpler representation. It is because of the same reason that we can't use the abstractions in `std::path` for internal uses where we need to apply the same algorithm for blobs we get from the server across all systems.
This diff adds a new crate called `vfs`. The recommended use of the types in this crate are `use ...::vfs` followed by `vfs::RelativePath` and `vfs::RelativePathBuf`.
Alternatives for representing the path:
* We could use `String` and `&str` directly however these types are inexpressive and have few guarantees. Rust has a strong type system so we can leverage it to provide more safety.
* Mononoke use `PathElement(Vec<u8>)` and `MPath(Vec<PathElement>)`. One issue is that this type requires more allocations. Having read a buffer it is more difficult to provide views into it as &MRelativePath.
`RelativePath` is Dynamically Sized Type (DST). That means that `mem::transmute` is the practical way of constructing it in Rust right now:
* https://doc.rust-lang.org/nomicon/exotic-sizes.html
* https://github.com/rust-lang/rust/pull/39594#issuecomment-279471476
Basing these types on `String` and `&str` means that the paths that we deal with need to have UTF-8 encodings. In the grand scheme of things a lot of things are simplified when we can make this kind of assumption.
The code right now doesn't have much documentation. I'll add that if this direction makes sense to other people.
Reviewed By: DurhamG
Differential Revision: D14001109
fbshipit-source-id: 1ca399827b1284f32cd83219e7e090d8b2487cee
Summary:
A lot of code is duplicated between data stores, and history stores, and one
reason for it is the absence of common trait between these 2. By adding a new
Store trait it will make it easier to write generic code that works accross
data and history store.
Reviewed By: quark-zju
Differential Revision: D14091899
fbshipit-source-id: deef1d43a7d300cb3607c67554ad54f20c870e23
Summary:
Instead of manually implementing DataStore/HistoryStore for Box, Rc, Arc, and
future smart pointers, we can simply implement the trait for all the types that
can be deref into a DataStore/HistoryStore.
Reviewed By: quark-zju
Differential Revision: D14078072
fbshipit-source-id: 47a80ab0179b84aa08836b6e7c5c3c5f9c1a08ff
Summary:
As we are moving to smooth switching to Mononoke and backwards, we can start to deprecate
Mercurial specific optimizations to simplify the code.
Reviewed By: markbt
Differential Revision: D14131594
fbshipit-source-id: fa927011890ecdf0874a3a74b4910412b3c84b70
Summary:
make local / relase are broken on trunk, fix it
in other modules, this code is only for tests. In toml file it is enabled for tests.
Reviewed By: mitrandir77, ikostia
Differential Revision: D14123392
fbshipit-source-id: eceaf17478b8b7e75ee2ba56df69b28ca8374c64
Summary: Move the contents of `edenapi-types` into the `types` crate so all of Mercurial's Rust types are in one place.
Reviewed By: quark-zju
Differential Revision: D14114547
fbshipit-source-id: feb8f9c35f102d30bf00b230df81a86a3893a49b
Summary:
In order to move the types in `edenapi-types` (containing types shared between Mercurial and Mononoke) to the `types` crate, we need to move a few types from the `revisionstore` crate into this crate first, because `revisionstore` depends on `types`, which would create a circular dependency since `edenapi-types` uses types from `revisionstore`.
In particular, this diff moves the `Key` and `NodeInfo` types into their own modules in the `types` crate.
Reviewed By: quark-zju
Differential Revision: D14114166
fbshipit-source-id: 8f9e78d610425faec9dc89ecc9e450651d24177a
Summary:
For HTTP data fetching, it will be necessary to have the same Rust types in Mononoke and Mercurial, so that Mononoke can send down the serialized types and Mercurial can deserialize them. These types must live in the Mercurial codebase since Mercurial can't link to code outside of fbcode/scm/hg. As such, this diff adds a new crate to Mercurial that Mononoke can link to, containing these shared types.
Right now the only shared type is a `HistoryEntry`, designed to match the interface of `MutableDatapack::add`. This type will be used as part of the HTTP history fetching API.
In the longer term, it would probably make sense to use something like Thrift for defining the on-the-wire formats used between Mercurial and Mononoke (and eventually for RPC as well). However, given that using Thrift from Mercurial is currently nontrivial (since Mercurial is typically built with Cargo and needs to be compatible with open source tooling), defining the schema in this crate and using `serde` for serialization and HTTP/2 for transport should be sufficient for now.
Reviewed By: quark-zju
Differential Revision: D14079337
fbshipit-source-id: c7880919aeb3fd7e1cf70067a89a17341c1d973f
