Summary:
When reading entries, we were using ok_or to read a slice and catch
errors, but this causes an unnecessary allocation for the error even if we don't
have an error. Let's use ok_or_else to avoid that.
Reviewed By: quark-zju
Differential Revision: D8897109
fbshipit-source-id: d308f64d54a58077d9ec2eb34dd1bef431ac1819
Summary:
Adds a trait that represents a store that is repackable. An implementor
only needs to be iterable, and expose some basic type and identifier information
and the trait provides the actual repack logic.
Reviewed By: quark-zju
Differential Revision: D8894756
fbshipit-source-id: 13053f8c7b6dca8b80ea819ef18949f3862cf367
Summary:
We need the ability to iterate over a datastore so we can implement
repack and cleanup. In a later diff we'll use this trait to implement repack
functionality in a way that it can apply to any store that implements
IterableStore.
Reviewed By: quark-zju
Differential Revision: D8885094
fbshipit-source-id: 0a2b1ab8cf524392d890302c33e386f1cd218d24
Summary:
The paths for each data pack are used in various situations (repack,
error reporting, etc) so let's store them and make them accessible via the
python api.
Reviewed By: quark-zju
Differential Revision: D8884773
fbshipit-source-id: 4108c98b4e303ba9bded1f264746fa4a84845c73
Summary:
Fix crate names for where the crate name doesn't match the package
name. This affected a few crates, but in practice only rust-crypto/crypto was
used.
Reviewed By: Imxset21
Differential Revision: D9002131
fbshipit-source-id: d9591e4b6da9a00029054785b319a6584958f043
Summary:
If the DataIndex didn't have a key, we were returning a DataIndexError
when we should've been returning a KeyError. This tells the higher level stores
to continue to the next store instead of raising the exception further.
Reviewed By: quark-zju
Differential Revision: D8806186
fbshipit-source-id: c40da96101494d5e3ea7910bf4b1a89674463a77
Summary:
The version only has a few valid values, so let's change it to be an
enum. This will be used in an upcoming diff to make the python tests pass.
Reviewed By: quark-zju
Differential Revision: D8775752
fbshipit-source-id: b1101c123b4802fbcb0f0a6fe5a45d741aec764f
Summary:
In the python code, end of a delta chain is marked with one value while
a missing delta is marked with another value. This isn't actually used anywhere,
but let's make the rust code mimic this for now.
Reviewed By: quark-zju
Differential Revision: D8775039
fbshipit-source-id: c9f81471bfd67e720938d6c5bbd10db029406686
Summary:
In a future diff we'll be changing the storage of IndexEntry to be
different from the API. So let's hide the actual format behind functions.
Reviewed By: quark-zju
Differential Revision: D8923005
fbshipit-source-id: 2f87b35315f8a7a5a8e67b6d0be2c73a1d9bccb4
Summary:
This function is present on the python data store api, so let's
replicate it here. Later we should come back and refactor this to be a special
case of the get_delta_chain result, but for now we'll maintain the custom API so
we can start using this code from python.
Reviewed By: quark-zju
Differential Revision: D8774474
fbshipit-source-id: aabcff3a43ae68859a1bf3b23f433214571b1a9d
Summary: This will let us build with buck.
Reviewed By: quark-zju
Differential Revision: D8980839
fbshipit-source-id: ea64328d32bc2c88984d0c861acefcc55b84ce02
Summary:
There were a bunch of unused code warnings because the mutabledatapack
module wasn't exposed as public. This then lead to us ignoring other warnings.
Let's fix all of them.
Reviewed By: quark-zju
Differential Revision: D8895468
fbshipit-source-id: 914c81026469382fcf28015b4a6bce13bad746c2
Summary:
Previously the rust dataindex would store the delta base location as an
offset relative to the start of the file. The python implementation stores it
relative to the start of the index though. So let's update the rust
implementation.
Reviewed By: quark-zju
Differential Revision: D8774206
fbshipit-source-id: d4317a95df353a7b635f1827fcfad7f3fb171afd
Summary:
The Node::random() function changed while this was landing. So we need
to update the tests.
Reviewed By: quark-zju
Differential Revision: D8774074
fbshipit-source-id: 6f3bcdeac069ef5ffdb2deb1970a1655cabcedaf
Summary: Create a simple rust reader for our loose file format. One of Mercurial’s simplest file formats is the loose file format. fbsource/fbcode/scm/hg/hgext/remotefilelog/remotefilelog.py:_createfileblob() is the python writing implementation.
Reviewed By: DurhamG
Differential Revision: D8731050
fbshipit-source-id: 80eb2abde2a2e5bb672d7e8ffa8ba58ed62184c1
Summary:
Instead of using random nodes, let's use ones based off a seeded
generator.
Reviewed By: quark-zju
Differential Revision: D8741139
fbshipit-source-id: a90e6f092adac6aef35149ee6c4bf2b47c469602
Summary: Implements the get_delta_chain function of the DataStore trait.
Reviewed By: quark-zju
Differential Revision: D8598658
fbshipit-source-id: 708bca63e2da3aae6064ed18076a9a1f1282a756
Summary:
Deltas may not have bases if they are a full text. Let's represent
that as an Option instead of as a magical null id value. This has the nice
effect of moving the decision to serialize a missing delta base down into the
serializer instead of up at the delta chain construction level.
Reviewed By: quark-zju
Differential Revision: D8739231
fbshipit-source-id: b58bd40dae45cb85890812db21e7eeff46aa6b4e
Summary:
Make `lib` a cargo workspace so building in subprojects would share a
`target` directory and `cargo doc` will build documentation for all
subprojects.
Reviewed By: DurhamG
Differential Revision: D8741175
fbshipit-source-id: 512325bcb23d51e866e764bdc76dddb22c59ef05
Summary:
Implements the get_meta function of the DataStore trait. This caught a
bug in how we record lengths as well.
Reviewed By: quark-zju
Differential Revision: D8598661
fbshipit-source-id: 566dca1770d6666e4215fa1fd8f33babdede2f90
Summary:
We want to be able to format error strings, so we can't return a static
str anymore.
Reviewed By: quark-zju
Differential Revision: D8598659
fbshipit-source-id: 44d7a73c06416efca51ca4d0f24a0c8911af8582
Summary:
A mutabledatapack also needs to be readable as a normal store. Let's
start implementing the DataStore trait, starting with get_missing
Reviewed By: quark-zju
Differential Revision: D8598657
fbshipit-source-id: 1f8bc89fae2be73fe789bc0ef1cdd922222019a2
Summary: Implements the last of the DataStore api, getdeltachain.
Reviewed By: quark-zju
Differential Revision: D8557950
fbshipit-source-id: 7f6530fe2064f0d035414b7920a126c6aab41beb
Summary:
In a future diff we'll be returning data read from a pack file out as a
Delta. To avoid copies, we need to be able to return an Rc from DataPack. This
seems like it will be a common pattern, so let's go ahead and make Delta contain
its data as an Rc.
Reviewed By: quark-zju
Differential Revision: D8557949
fbshipit-source-id: 276005360bfa48e9154143dedce579a21129e976
Summary:
Introduces the DataEntry structure which is able to parse data entries
from pack files. Uses it to implement getmetadata
Reviewed By: quark-zju
Differential Revision: D8556610
fbshipit-source-id: c25427c3c247970a879ad7d409b821f3695b97d9
Summary:
This adds the initial struct and opener for a datapack. Future diffs
will add actual functionality and tests.
Reviewed By: quark-zju
Differential Revision: D8553436
fbshipit-source-id: 3b17f995632e859019205f242a4cce389ac77407
Summary:
Actually write the index to disk when the mutabledatapack is
serializing.
Reviewed By: quark-zju
Differential Revision: D8552276
fbshipit-source-id: 354c7fdc3fe84b91d582f0e8cde8c6ae2494c559
Summary: This adds the logic for reading a DataIndex from disk.
Reviewed By: quark-zju
Differential Revision: D8552278
fbshipit-source-id: 611ff09c27716b8d8ff7424c1a27287b9fc42b78
Summary:
Soon we will be writing the index during pack file serialization, so
let's add the logic for serializing the index.
Reviewed By: quark-zju
Differential Revision: D8552277
fbshipit-source-id: 60829631eb060f62d266c16f6016f34080311f8e
Summary: A simple helper method for producing Node's from slices in a safe way.
Reviewed By: quark-zju
Differential Revision: D8547679
fbshipit-source-id: 85ae8fcd7749c662b1459af1d84ccf9695dd5f0b
Summary:
Let's build an inmemory hash table of the revisions that were added. A
future diff will serialize this index into a dataidx file.
Reviewed By: quark-zju
Differential Revision: D8309730
fbshipit-source-id: 9efc7f0f34129a63c52309b4d70179f2c10840b3
Summary:
Implements a fanout table trait that history pack and data pack will
use. It basically consists of logic to build and read a quick lookup table that
uses the first few bytes of a key to determine the bounding range of a binary
search.
Reviewed By: quark-zju
Differential Revision: D8309729
fbshipit-source-id: 71e398277dc8ae041447035f044e5d47ca41cf7e
Summary:
The mutabledatapack format has a one byte header containing the version
number.
Reviewed By: quark-zju
Differential Revision: D8305653
fbshipit-source-id: c4a96dc48e64acd2c5849034e5d90b87363fbc8d
Summary:
Implements the logic that builds a hash of the contents of the pack
file and uses it as the name.
Reviewed By: quark-zju
Differential Revision: D8305654
fbshipit-source-id: d1270e7519a7718aa5427f3be5cdc0cd0dee2fe2
Summary:
This is the start of a rust mutable datapack implementation. The first
diff adds a simple add function. Later diffs will add the logic that builds the
index, serializes the index, and computes the final hash name.
Reviewed By: quark-zju
Differential Revision: D8304036
fbshipit-source-id: db05c2b845e51a3552c039b7fc0b8f4cc0ff0852
Summary:
In a future diff we'll be serializing and deserializing metadata in
datapacks. Let's add the reader and writer functions for Metadata and some unit
tests.
Reviewed By: quark-zju
Differential Revision: D8303603
fbshipit-source-id: 7e7a7aa218c05179b205abf8b151b1488be674b3
Summary:
This commit adds very basic tests for the Union History Store. These
tests just test for expected output of operations on bad/empty stores.
Reviewed By: quark-zju
Differential Revision: D8553821
fbshipit-source-id: a0dfa47f10083c37901535e8a810a99693a28c82
Summary: This commit just introduces the Union History Store.
Reviewed By: DurhamG
Differential Revision: D8553822
fbshipit-source-id: 6c7ee0b5d33dae6d51b4179616d206f42eb0cd50
Summary: This commit just introduces the history store.
Reviewed By: DurhamG
Differential Revision: D8553823
fbshipit-source-id: 93af6059296d11c4fcc0dd306b4472c4f2168fa7
Summary: This commit just fixes the messaging for the errors.
Reviewed By: DurhamG
Differential Revision: D8553820
fbshipit-source-id: 73f2cd13e7538b6870b16a0e47e657a6d08af9e3
Summary: This is just a refactor to address the naming scheme.
Reviewed By: quark-zju
Differential Revision: D8269217
fbshipit-source-id: 8c52d2c67837550e0b7dc1a45b3faf9a80319b61
Summary:
Based on review for D8214151 by quark-zju, addressing the nit here as
well.
Reviewed By: quark-zju
Differential Revision: D8267140
fbshipit-source-id: 12c3355852a49859c2b0a243fa8666105c914c73
Summary:
Adding the tests for the case when the union store has only one data
store which always returns an `Err` as `Result`. This `Err` is not of the type
`KeyError` which the union store handles differently.
Reviewed By: quark-zju
Differential Revision: D8214156
fbshipit-source-id: bd077af343086c92f46ec6a6f1551d05dd9bda09
Summary:
Adding tests for the case when the union store only has a single data
store which is completely empty.
Reviewed By: quark-zju
Differential Revision: D8214151
fbshipit-source-id: 9d8f329548a1b7e105a5dc6219067a6e292fe97c
Summary:
This commit just renames the methods to be more specific. This is
useful for later changes.
Reviewed By: quark-zju
Differential Revision: D8214153
fbshipit-source-id: e8db9148334f7cd539aca626e3798e256b9b022f
Summary:
These tests just test the expected outcome when the union data store
is empty.
Reviewed By: quark-zju
Differential Revision: D8018975
fbshipit-source-id: a2cc4c87509b857dbf5f6af506f165ea62080db8
Summary:
This commit derives the common traits for the Key type just as we did
for the Node type in D7872300.
Reviewed By: quark-zju
Differential Revision: D8018973
fbshipit-source-id: 566a69be16d74529c6eb5f157b84de25835f780f
