Summary:
quark-zju noticed in code review that `Cursor` could get into an infinite loop when
it's results would be collected into a Vec<_>. That was the motive that I
needed to update `Cursor` to transition to `State::Done` when the cursor
encounters an error. Previously I felt that users of `Cursor` would only be
empowered by having the ability to retry the failure.
Reviewed By: quark-zju
Differential Revision: D14393590
fbshipit-source-id: b3e0974ac15d62f3f17790229121c0dec3a6149e
Summary:
Follow up from D14178264.
Two changes:
* tree manifest entries must end with a line feed
* `t` is the byte that flags a directory
Reviewed By: DurhamG
Differential Revision: D14368316
fbshipit-source-id: b0b46c876649b8f25bf0ecdb1266527dbeb33796
Summary:
`manifest::tree::diff()` returns an iterator over the differences between two
tree manifests.
I chose a function that takes two parameters over a method on Tree because it
felt more clear to write `left` and `right`. Also because I am not sure how
iterators would be abstracted on a trait.
Differential Revision: D14347656
fbshipit-source-id: 537574070cd18b08c77b3cd1cf4cff38d77fbf81
Summary:
Cursor is a utility for iterating over a manifest tree. In this diff it is used
to implement Files. In the future it will be used to do a diff between two tree
manifests.
I am not sure how to describe an iterator return value in the Manifest trait so
I kept the function on the tree only for now. Looking forward to hearing your
suggestions.
Differential Revision: D14347655
fbshipit-source-id: ffd856443d8abe3ebd0557a096bf7a5ec46312d3
Summary:
`Node` is marked as `Copy`. `FileMetadata` is not much more than `Node` so it
seems pretty clear that it should be marked `Copy`.
Reviewed By: DurhamG
Differential Revision: D14347657
fbshipit-source-id: 939abf88087bc8c6f942047a08d6a4a0d61e053f
Summary:
Cleaning up the `mod.rs` file so that it provides more signal.
`Link` is an internal implementation detail that other internal components may depend on so it is a great candidate to be moved to a dedicated file.
Differential Revision: D14347654
fbshipit-source-id: e5b5a42faf1e9f9c4a0591e5bd94182391ed511f
Summary:
Save, finalize, flush, they mean about the same thing.
The first thing to note is that this implementation is not complete because
the parents are not correctly passed into the hashing function.
The second thing is that store failures make the code a little more complex
than it would have been otherwise.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D14292713
fbshipit-source-id: 807d7a385a62cb5f4948f1781d3146eaa6502ca9
Summary:
This function is a bit on it's own with the removal of the pair conversion.
Since it is used in only one place it makes sense to inline it.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D14292712
fbshipit-source-id: abbf1dc70d61c0ad039f5bc5ed5277d0770e3899
Summary:
Working on the save mechanism I realized that links_to_store_entry is not that
useful because we can avoid the failure states where we would try to serialize
an ephemeral node. I am removing that function and converting the code that was
using that function to using the Entry constructor directly.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D14292714
fbshipit-source-id: 54ef46670319c27d90fc78511a1eb6abf47d3acf
Summary:
This is what Rust is telling us to do. The situation that triggers this update is
writing to the store. Particularly when the store is an in memory hashmap we
need to have a mutable borrow to the hashmap to insert into it. From a general
point of view this means that any sharing of the store between different
instances of a manifest will have to be handled by the struct that implements
the `Store` trait.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D14292716
fbshipit-source-id: 6e789527dbdf3cd3ffe967f4900251bf31f7d6b2
Summary:
Removes a file from the manifest. Nothing special for it.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D14276645
fbshipit-source-id: 85e8ffd6cffee426c73eb627484dfa5a866a364b
Summary:
It is going to be useful in tests to check how certain internal nodes change
so adding an api that allows fetching an internal node.
(Note: this ignores all push blocking failures!)
Differential Revision: D14276642
fbshipit-source-id: 9a3e488be6031f7b4727a8643f64970dcec8c400
Summary:
This removes the need for the local buffer for the parent.
(Note: this ignores all push blocking failures!)
Differential Revision: D14276648
fbshipit-source-id: a9378ea592d502ddf2dcdc35fe6ffa9ba213bc14
Summary:
Using the recently added path utilities so that we don't keep a secondary
parent buffer around.
Updating the file insert logic so that it is readable and intuitive.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D14276649
fbshipit-source-id: 8e7e835814f0039645601abbf1b701e8c1ed3697
Summary:
We currently alias many of the imports from the store submodule. It is nicer
if we just import the submodule name and prefix with the submodule.
Differential Revision: D14276638
fbshipit-source-id: 6661df7f9cb5d976b11153003f653a1f66301c9a
Summary:
The order of imports we're trying to follow is: std, remote crates
(ie: crates.io), local crates (ie: fbcode crates), and crate local. A new line
in between each group can be used to prevent rustfmt from re-ordering them.
Reviewed By: singhsrb
Differential Revision: D14243162
fbshipit-source-id: 6fc2cceb3d6834b602be20b8b8f74e0f61b227e1
Summary:
This diff focuses on addding deserialization. Because the most effective way
of testing deserialization is doing round-trip conversions we also implement
serialization.
`manifest::tree::store::Entry` is the structure that is in charge of perfroming
serialization and deserialization. We update the Store trait to interface with
this new object.
Differential Revision: D14178264
fbshipit-source-id: bb12262c181a518ba4111d40c079d6836ec44301
Summary:
The `once_cell` crate is more flexible than `lazy-init`. It has more types,
a richer api, recent updates and more features.
Differential Revision: D14232727
fbshipit-source-id: 14aeb34a96e094069bb8dc3fb5efcf5b5707ce8c
Summary:
The purpose of `FileMetadata` is to be a data container. Using the
`FileMetadata` implies accessing the underlying values. That means that the
underlying values need to be `public`.
Differential Revision: D14178260
fbshipit-source-id: 8b8e5ead23d47962498f991152c2a5fd94ac3c74
Summary: These implementations are going to used in quickcheck tests.
Differential Revision: D14178259
fbshipit-source-id: 0bded67deab3422b4aad53666c14cf195ea1b0d4
Summary:
`Durable` nodes are inner nodes that come from storage. Their contents can be shared between multiple instances of Tree. They are lazily evaluated. Their children list will be read from storage only when it is accessed.
The inner structure of a durable link poses an interesting question related to handling failures: what do we do when we have a failure when reading from storage?
We can cache the failure or we don't cache it. Caching it is mostly fine if we had an error reading from local storage or when deserializing. It is not the best option if our storage is remote and we hit a network blip. On the other hand we would not want to always retry when there is a failure on remote storage, we'd want to have a least an exponential backoff on retries. Long story short is that caching the failure is a reasonable place to start from.
The lazy-init crate is useful for modeling the lazy initialization model that we have for durable node links. See docs at https://docs.rs/lazy-init/0.3.0/lazy_init/
Reviewed By: quark-zju
Differential Revision: D14142928
fbshipit-source-id: 077f708b38e2ace772f30b3392445326ce17f47c
Summary:
The store abstracts the data that we need from the store for TreeManifest. This is not a long term abstraction, mostly something that we can code against until we write the real parsing logic from the real data store.
Adding a TestStore implementation that we can use in tests and debug.
Reviewed By: quark-zju
Differential Revision: D14142930
fbshipit-source-id: 7f2d4f05a7b7e63758db9247cdbcd51541c88ec0
Summary: Preparing to add the store abstraction for the tree manifest. This store implementations are going to be tied to the tree implementation and should be made private from the rest of the crate. To do this we move the tree to a module.
Reviewed By: quark-zju
Differential Revision: D14142929
fbshipit-source-id: 588d597e1248fc2e632c9efe03f08ba3d491e8cd
Summary: manifest::tree::Link has several convenience functions. In the grand scheme of things this is something that we will want because they are common operations. The problems is that the node structure is currently changing rapidly and the additional layer of abstraction is hindering iteration. I don't know the exact shape that the nodes will have and trying different things out is easier when the Tree functions use the nodes directly.
Reviewed By: quark-zju
Differential Revision: D14142926
fbshipit-source-id: 5e4e8f4d3e1b19fd14dcc290a3dea4271b502d97
Summary: `RepoPath` and `PathComponent` were recently added as path abstractions for source control internals. This diff makes use of these abstractions in the manifest crate.
Reviewed By: quark-zju
Differential Revision: D14142925
fbshipit-source-id: 1e0033f4f2aeb5c1d63899adbce198302086bfda
Summary:
Starting the implementation of tree manifest with the in memory nodes and implementing `get` and `insert`. The in memory nodes are called `Ephemeral` and the stored immutable nodes are going to be called `Durable`.
Using a `BTreeMap` for storing the children because we want to efficiently insert, fetch and remove path components. We also want iteration to be done in ordered fashion so BTreeMap is our collection in this case.
Removing elements from the tree is going to be implemented in a future update.
Reviewed By: DurhamG
Differential Revision: D14016273
fbshipit-source-id: d3bc22e5ddb21b689d07a7d74bd639b8c2b138ce
Summary:
The seed for the rust implementation of manifests.
We start with the most primitive API for manifests and maps a paths to a `Node`. At the basic level we need the same operations that a map implements so we start with `insert`, `get` and `remove`. We know that retrieving data for Manifests can fail so we encode that in our interface using `Fallible`.
I let for future iterations requiring iterator or returning manifest flags.
Reviewed By: DurhamG
Differential Revision: D14016274
fbshipit-source-id: 8f1f83610933b9e9a96f8c5ba2c6e50567c76e06