Summary: While doing blob importing from smallish revlog repo to mononoke blobstore I noticed that most of the time is spent in waiting for mutexes. This diff is about removing the lock completely and prefilling all mutable datastructures in the `::new()` method.
Reviewed By: jsgf
Differential Revision: D5631055
fbshipit-source-id: 5776143014c92372b316dd8dba31785430f1f222
Summary: We'll need to implement blob importing from revlogrepo to blobrepo
Reviewed By: jsgf
Differential Revision: D5622462
fbshipit-source-id: c57f016711bcec7c0bd432d40881588ebdce6f7f
Summary: Main part is `get_stream_of_manifest_entries` that creates a stream of all tree manifest entries by recursively going through all of them.
Reviewed By: jsgf
Differential Revision: D5622490
fbshipit-source-id: 4a8b2707df0300a37931c465bafb1ed54d6d4d25
Summary:
Will be tested in the next diffs, but the fix is quite obvious - apply_deltas
doesn't modify it's parameters, it returns new versions.
Reviewed By: ryanmce
Differential Revision: D5649250
fbshipit-source-id: d2f1badb474911e7a5ac79dd59150444a52b5904
Summary: Repos with flat manifest and with tree manifest use different path. Let's open tree manifest file if it's present and flat manifest otherwise
Reviewed By: sid0
Differential Revision: D5583566
fbshipit-source-id: e35eef4b1f8067c2a91ebfc62718fec100f19e2e
Summary: Revlog Entry can be both for file and for tree manifest. Previously get_parents() worked correctly only for files. This diff fixes it.
Reviewed By: sid0
Differential Revision: D5583433
fbshipit-source-id: 13642f4de8310e67c1ef416fc83176bf4975f3b6
Summary: Implements get_content() for tree manifest entry. Note that we need to prepend current entry path to the child entries, because child entries paths are just basenames.
Reviewed By: sid0
Differential Revision: D5563970
fbshipit-source-id: 52e896a7a8892f4ffb5fc14b64dd7ebc5692471d
Summary:
Add function similar to get_file_revlog to get tree manifest revlog.
Will be used in the next diffs.
Reviewed By: jsgf
Differential Revision: D5563895
fbshipit-source-id: 0ff84c458eb071763cbdc6d98bcf92e9b8ccc1b8
Summary:
Previously `fsencode()` worked incorrectly if Path was a directory. We didn't notice it before because we've never used Path to store directories, but we will use it for TreeManifest.
I considered two options when implementing it.
1) Put some kind of flag `isDir` inside Path struct. But that would create complications with `join()` method. For example, you can't join anything to the file - what should we do in this case? panic? return result?
2) and another `fsencode_dir()` method. Clients need to know what kind of Path they have. I choose this option because it requires less changes and brings less complications compared to the option 1
Reviewed By: sid0
Differential Revision: D5574847
fbshipit-source-id: c4c476a7fc3b884de847c431a56ff5f313c1389f
Summary:
Adds method similar to https://doc.rust-lang.org/std/path/struct.Path.html#method.join.
Will be used in the next diff to prepend path prefix to the tree manifest entries
Reviewed By: jsgf
Differential Revision: D5563808
fbshipit-source-id: 3637275093c301bf159083cb9bfedaa0e490a75e
Summary:
Both changelog and manifest are revlogs and they are both protected by their
own mutexes. There is no need to protect by RevlogRepo mutex.
It requires making `Revlog::get_heads()` method accept immutable `&self`. This
is fine since all `Revlog` methods are protected by mutex.
Reviewed By: farnz
Differential Revision: D5618949
fbshipit-source-id: 0511d547360e9785cb6e2cefadf8c10626a433c4
Summary:
A preparation step before blob importing of tree manifest repos to blobrepo.
`get_parents()` method of BlobEntry reads parents from the blobstore. It works fine for file entries because file entries can stores its parents in the blobstore. With tree manifests BlobEntry can contain also tree manifest entries, and that means that tree manifest entries parents should also be stored somewhere in the blobstore.
I suggest to use the same logic for the tree manifest entries as for the file entries. File and manifest entries have two blobstore entries - one stores hash of the content and parents, another stores the actual content.
To do this I moved `RawNodeBlob` and `get_node()` to the separate module and made fields public.
Reviewed By: jsgf
Differential Revision: D5622342
fbshipit-source-id: c9f0c446107d4697b042544ff8b37a159064f061
Summary: The `boxfnonce` crate is a workaround for the lack of `FnBox` in stable Rust. Without it, it isn't possible to call a `Box<FnOnce>`.
Reviewed By: jsgf
Differential Revision: D5626687
fbshipit-source-id: 01ab37b98add5dae10f3c4834c2a330593639ab6
Summary: As per title. This includes fixes by kulshrax to make things not break.
Reviewed By: jsgf
Differential Revision: D5569907
fbshipit-source-id: f6a4ad88b33caf8c8de3ef8db7932d36993e4e95
Summary:
Instead of storing `Vec<u8>`, let's store `Vec<PathComponent>`, where PathComponent is Vec<u8> without b'\'.
To make sure len() is still `O(1)` let's store it too.
Reviewed By: sid0
Differential Revision: D5573721
fbshipit-source-id: 91967809284d79bf0fcdcabcae9fd787a37c318b
Summary:
Let's create a separate function `fetch_file_blob_from_blobstore` that will be
used to get content of the blob without creating file::BlobEntry.
file::BlobEntry requires Path and type parameters that won't be passed to eden
server.
Reviewed By: jsgf
Differential Revision: D5537647
fbshipit-source-id: 13fb2fbba7d3068b88aac2d842cc92b4da6ca80c
Summary: Method that returns content of the manifest entry in json format.
Reviewed By: sid0
Differential Revision: D5527659
fbshipit-source-id: 1832b645f69da40cbd620a6bff318e25594c5148
Summary: Will be used by eden server in the next diff
Reviewed By: jsgf
Differential Revision: D5553955
fbshipit-source-id: 196e2da597d0456b4337a6d9ada8fe188075e87e
Summary: We're going to use this code in another context very soon.
Reviewed By: StanislavGlebik
Differential Revision: D5581807
fbshipit-source-id: 6d43eb114b3d82fd97a9b9cce775f1f398853e84
Summary: I'm going to make significant changes to this crate in upcoming diffs.
Reviewed By: StanislavGlebik
Differential Revision: D5581790
fbshipit-source-id: 2a3a4b183b554ca46c3bc611452e55eb17238f20
Summary:
I don't think it's possible for linkrevs to be `-1` (not sure why it was
written as `!0` -- jsgf, any idea why?)
Reviewed By: StanislavGlebik
Differential Revision: D5581780
fbshipit-source-id: 69d685d863ee51adc846e889eb766f3fecc47681
Summary: We're going to reuse this code in an upcoming diff.
Reviewed By: StanislavGlebik
Differential Revision: D5581683
fbshipit-source-id: 08724333c2f637bbf2c2e534855c9802c15328ee
Summary:
`buck check` for all targets in `scm/mononoke` was failing because of
this.
Reviewed By: StanislavGlebik
Differential Revision: D5581555
fbshipit-source-id: 2039de4f3a69fac85cf7a8e503eaf59915c93c13
Summary:
Mercurial treats copying as its first class operation, not renaming.
Using 'renamed' instead of 'copied' is pretty confusing.
Reviewed By: StanislavGlebik
Differential Revision: D5579408
fbshipit-source-id: 4494aa4bf56bd2c00f4331da9edd88d756525dd2
Summary: These comments make it a bit clearer to me what's going on.
Reviewed By: StanislavGlebik
Differential Revision: D5579411
fbshipit-source-id: 84cbd650208f200a04462cb6dda5d05a9d8e6196
Summary:
Server can construct boxed Repo instances, and performs all client protocol operations in terms of trait methods. This allows the repo to be set dynamically.
This currently makes the repo type a single setting which applies to all repos; this can be generalized to a per-repo setting later.
Reviewed By: sid0
Differential Revision: D5540670
fbshipit-source-id: ac450b078849d9bd65d6e3dc91b73d652ca86ce7
Summary: Add placeholder error in hgproto for errors from Repo operations. This is intended to be chained onto the real error.
Reviewed By: sid0
Differential Revision: D5540669
fbshipit-source-id: dedf12c9c749c40ba77444fb0adb30c8ddc868ed
Summary: Add helpers to allow an implementation of a Repo object to be boxed up into a type-erased trait object.
Reviewed By: sid0
Differential Revision: D5540673
fbshipit-source-id: c16332f6a548d053125dd3376e7e06d0ddffadf9
Summary: Add a `get_bookmarks()` method to the Repo trait and implement it for RevlogRepo and BlobRepo.
Reviewed By: sid0
Differential Revision: D5540667
fbshipit-source-id: 710849e50b1d358be1c4f95a1cbd61efb7c2cf6b
Summary: Blobrepo can take any Heads or Blobstore implementation, each of which may have their own error type. Error chain doesn't support type parameters, and can only have foreign links for specific concrete types. To handle this we have a couple of placeholder errors in Blobrepo's errors which are chained onto the real underlying error.
Reviewed By: sid0
Differential Revision: D5540676
fbshipit-source-id: 9543e204754f22dfd68a04aec5a484fa85d53ff8
Summary:
Add BoxedBookmark to assist in boxing a Bookmark trait, and implement Bookmark
for Box<_> and Arc<_> Bookmark trait objects.
Reviewed By: sid0
Differential Revision: D5540675
fbshipit-source-id: 7582adbd2858f6814d57b7e54089b922886a4d54
Summary:
In practice, no users of Bookmarks will not want listing, and all
implementations need to be able to support it.
Reviewed By: StanislavGlebik
Differential Revision: D5540672
fbshipit-source-id: d64a3894c7cd14d329bd23809525935461289bc3
Summary: Allow the error type to be any type the internal errors can be converted to.
Reviewed By: sid0
Differential Revision: D5540671
fbshipit-source-id: 1077132d25bcc13ba2a3a509c916f2e07ac10dcf
Summary:
blobimport appends ".bincode" in the end of the file.
Let's do the same in blobrepo
Reviewed By: sid0
Differential Revision: D5527661
fbshipit-source-id: d54fe52c1d282f038a47ac2f2cd0aaf917a92227
Summary:
Soon I'm going to add more different url handlers. And current url handling is
neither robust nor extendable. Let's use regexes to do url handling
Reviewed By: jsgf
Differential Revision: D5489445
fbshipit-source-id: f418c3f40892903b573a33e57899b47e5719a5ea