Summary: Move upload logic for mercurial data out of blobrepo crate
Reviewed By: StanislavGlebik
Differential Revision: D17366158
fbshipit-source-id: 9f1cdf4fbe67552b12fd1ef94f9c7de1be632988
Summary:
This diff moves initFacebook calls that used to happen just before FFI calls to instead happen at the beginning of main.
The basic assumption of initFacebook is that it happens at the beginning of main before there are additional threads. It must be allowed to modify process-global state like env vars or gflags without the possibility of a data race from other code concurrently reading those things. As such, the previous approach of calling initFacebook through `*fbinit::FACEBOOK` near FFI calls was prone to race conditions.
The new approach is based on attribute macros added in D17245802.
---
The primary remaining situations that still require `*fbinit::FACEBOOK` are when we don't directly control the function arguments surrounding the call to C++, such as in lazy_static:
lazy_static! {
static ref S: Ty = {
let _ = *fbinit::FACEBOOK;
/* call C++ */
};
}
and quickcheck:
quickcheck! {
fn f(/* args that impl Arbitrary */) {
let _ = *fbinit::FACEBOOK;
/* call C++ */
}
}
I will revisit these in a separate diff. They are a small fraction of total uses of fbinit.
Reviewed By: Imxset21
Differential Revision: D17328504
fbshipit-source-id: f80edb763e7f42b3216552dd32f1ea0e6cc8fd12
Summary: I think these are left over from pre-2018 code where they may have been necessary. In 2018 edition, import paths in `use` always begin with a crate name or `crate`/`super`/`self`, so `use $ident;` always refers to a crate. Since extern crates are always in scope in every module, `use $ident` does nothing.
Reviewed By: Imxset21
Differential Revision: D17290473
fbshipit-source-id: 23d86e5d0dcd5c2d4e53c7a36b4267101dd4b45c
Summary: If some blob is missing it becomes very hard to debug what is missing. This diff extends `LoadableError::Missing` with string argument.
Reviewed By: krallin
Differential Revision: D17132049
fbshipit-source-id: df1f60b1f739c8a837d1b1798edbaca58b1587f9
Summary:
This is a part of an effort to move mercurial related stuff into mercurial crate:
- `BlobManifest|HgBlobEntry` -> `mercurial/types/blobs`
- helper `Id` type is no longer required and `BlobManifeset` can be loaded with `HgManifesetId::load`
- `BlobManifest` implements `HgManifest` and typed `Manifest` traits
Reviewed By: StanislavGlebik
Differential Revision: D16984745
fbshipit-source-id: 33d4eb9c33137258956917328efa1f5ec6737ee9
Summary:
This updates the `mononoke_admin` filenodes command to allow showing filenodes
by path + ID in addition to revision + path. This makes it possible to locate
the linknode given a filenode ID but no revision.
This updates the filenodes CLI command a little bit (so if you want the old
command you need to add `by-revision`), but this should be fine since this is
a command that a human will normally be calling.
Reviewed By: farnz
Differential Revision: D16936628
fbshipit-source-id: 1079ad580b235076739c92d0c1f3c9bfc80db4c1
Summary:
We now have two Manifest and two Entry traits and it makes it very confusing.
The old Manifest trait (and related EmptyManifest struct and Entry trait) are
mercurial-specific, while new Manifest trait is generic over all kinds of
manifests (unodes, mercurial etc).
In ideal state we should have only new Manifest trait, but it'd require quite a
big refactoring. In short term let's at least rename Manifest to HgManifest and
Entry to HgEntry to make distinction clearer.
Reviewed By: krallin
Differential Revision: D16886032
fbshipit-source-id: 80e3999fae3e89347448579caf7aafec56705500
Summary:
This is the first diff in the stack that removes ugly hack `struct Id` that we
used in derived data (see more details about that later in the stack).
This diff reverses the dependency between blobstore crate and mononoke_types
crate. Previously blobstore depended on mononoke_types, and blobstore also had
Loadable/Storable traits. That means that we can't implement Loadable/Storable
traits for types from mononoke_types - we can't even import them without
creating cyclic dependency between crates! And we'll use Loadable in
mononoke_types later in the stack.
This diff reverses the dependency and moves Loadable/Storable in mononoke_types
Reviewed By: krallin
Differential Revision: D16884649
fbshipit-source-id: 20753f3e62980be7bb6a04ab9d19980bc459b40e
Summary:
This updates Mononoke to support LFS metadata when serving data over getpackv2.
However, in doing so, I've also refactored the various ways in which we currently access file data to serve it to clients or to process client uploads (when we need to compute deltas). The motivation to do that is that we've had several issues recently where some protocols knew about some functionality, and others didn't. Notably, redaction and LFS were supported in getfiles, but neither of them were supported in getpack or eden_get_data.
This patch refactors all those callsites away from blobrepo and instead through repo_client/remotefilelog, which provides an internal common method to fetch a filenode and return its metadata and bytes (prepare_blob), and separate protocol specific implementations for getpackv1 (includes metadata + file content -- this is basically the existing fetch_raw_filenode_bytes function), getpackv2 (includes metadata + file contents + getpackv2 metadata), getfiles (includes just file content, and ties file history into its response) and eden_get_data (which uses getpackv1).
Here are a few notable changes here that are worth noting as you review this:
- The getfiles method used to get its filenode from get_maybe_draft_filenode, but all it needed was the copy info. However, the updated method gets its filenode from the envelope (which also has this data). This should be equivalent.
- I haven't been able to remove fetch_raw_filenode_bytes yet because there's a callsite that still uses it and it's not entirely clear to me whether this is used and why. I'll look into it, but for now I left it unchanged.
- I've used the Mercurial implementation of getpack metadata here. This feels like the better approach so we can reuse some of the code, but historically I don't think we've depended on many Mercurial crates. Let me know if there's a reason not to do that.
Finally, there are a couple things to be aware of as you review:
- I removed some more `Arc<BlobRepo>` in places where it made it more difficult to call the new remotefilelog methods.
- I updated the implementation to get copy metadata out of a file envelope to not require copying the metadata into a mercurial::file::File only to immediately discard it.
- I cleaned up an LFS integration test a little bit. There are few functional changes there, but it makes tests a little easier to work with.
Reviewed By: farnz
Differential Revision: D16784413
fbshipit-source-id: 5c045d001472fb338a009044ede1e22ccd34dc55
Summary:
Start moving mercurial related stuff to `mercurial` directory:
- rename `mercurial` to `mercurial_revlog` and moved to `/mercurial/revlog`
- move `mercurial_types` to `/mercurial/types`
- move `mercurial_bundles` to `/mercurial/bundels`
Reviewed By: farnz
Differential Revision: D16783728
fbshipit-source-id: 79cf1757bb7cc84a6273a4a3c486242b1ef4cd00
