Summary:
This returns a Result of a tuple, but:
- This never errs.
- Nothing ever reads the left side of the tuple.
So let's stop doing that.
Reviewed By: StanislavGlebik
Differential Revision: D25219887
fbshipit-source-id: f33dcf6f6e68cb17b40c4638470312afae0662e6
Summary: Convert `ChangesetFetcher` to new type futures
Reviewed By: StanislavGlebik
Differential Revision: D25244213
fbshipit-source-id: 4207386d81397a930a566db008019bb8f31bf602
Summary:
Create `BlobstoreMapping` as a trait with the common implementations for
derived data mappings that are stored in the blobstore.
Reviewed By: StanislavGlebik
Differential Revision: D25099915
fbshipit-source-id: 8a62fbb809918045336944c8cd3584b109811012
Summary:
Batch derivation was disabled as it caused problems for other derived data types.
This was because the default batch implementation was wrong: it attempted to derive
potentially linear stacks concurrently, which would cause O(n^2) derivations.
Fix the default implementation to be a naive sequential iteration, and re-enable
batch derivation for fsnodes and skeleton manifests.
Reviewed By: StanislavGlebik
Differential Revision: D25218195
fbshipit-source-id: 730555829f092cc36e1c81cf02c2b4962a3904da
Summary:
On the second and subsequent linear stack in a batch, the parent fsnode may
have been derived in the previous iteration of the loop. Since we haven't
completed this batch yet, the mappings have not been stored, and so the
attempt to derive the parent will result in the parent being derived again,
repeating the work of this batch sequentially.
Apply the optimization used in skeleton manifests and fetch the parent fsnodes
out of the result we are accumulating.
Reviewed By: StanislavGlebik
Differential Revision: D25218194
fbshipit-source-id: 5cc49204b53984f8aa73542f1a794a6251eb2b2e
Summary:
Similar to fsnodes, allow skeleton manifests to be derived in parallel in large
batches by splitting the changesets into linear stacks with no internal
conflicts, and deriving each changeset in that batch in parallel.
Reviewed By: StanislavGlebik
Differential Revision: D25218196
fbshipit-source-id: e578de9ffd472e732abb1e2ef9cd19c073280cd4
Summary: convert mercurial_derived_data to new type futures
Reviewed By: ahornby
Differential Revision: D25220329
fbshipit-source-id: c2532a12e915b315fe6eb72f122dbc37822bbb2a
Summary:
This diff prepares the Mononoke codebase for composition-based extendability of
`ScubaSampleBuilder`. Specifically, in the near future I will add:
- new methods for verbose scuba logging
- new data field (`ObservabilityContext`) to check if verbose logging should
be enabled or disabled
The higher-level goal here is to be able to enable/disable verbose Scuba
logging (either overall or for certain slices of logs, like for a certain
session id) in real time, without restarting Mononoke. To do so, I plan to
expose the aforementioned verbose logging methods, which will run a check
against the stored `ObservabilityContext` and make a decision of whether the
logging is enabled or not. `ObservabilityContext` will of course hide
implementation details from the renamed `ScubaSampleBuilderExt`, and just provide a yes/no
answer based on the current config and sample fields.
At the moment this should be a completely harmless change.
Reviewed By: krallin
Differential Revision: D25211089
fbshipit-source-id: ea03dda82fadb7fc91a2433e12e220582ede5fb8
Summary: remove dependency on old futures from derived data filenodes
Reviewed By: ahornby
Differential Revision: D25218521
fbshipit-source-id: 4d7eaf42c3ba15ea09276a7f3567128d5216e814
Summary: `derived_data:derived_data` had already been almost converted, I've cleaned up some test so it would be possible to completely remove old futures dependency
Reviewed By: StanislavGlebik
Differential Revision: D25197406
fbshipit-source-id: 064439f42a15f715befc019e5350dda0a2975f2b
Summary:
- convert save_bonsai_changesets to new type futures
- `blobrepo:blobrepo` is free from old futures deps
Reviewed By: StanislavGlebik
Differential Revision: D25197060
fbshipit-source-id: 910bd3f9674094b56e1133d7799cefea56c84123
Summary:
If files are only modified, and not added or deleted, the skeleton manifest
doesn't change. Avoid duplicate writes by skipping the write if the skeleton
manifest we derive has the same ID as one of its parents.
Reviewed By: ahornby
Differential Revision: D25099917
fbshipit-source-id: 62900406711becea88491e706a09c6032109c964
Summary: Use these in next diff from walker so we can start to scrub fastlog
Reviewed By: aslpavel
Differential Revision: D25187548
fbshipit-source-id: 4188a0c9bdb51a898817b6f5801fe473b6712385
Summary: convert `BlobRepo::get_bonsai_bookmark` to new type futures
Reviewed By: StanislavGlebik
Differential Revision: D25188577
fbshipit-source-id: fb6f2b592b9e9f76736bc1af5fa5a08d12744b5f
Summary:
We have `HgBlobChangeset`, `HgFileEnvelope`, `HgManifestEnvelope` ... but we
also have `BlobManifest`. Let's be a little consistent.
Reviewed By: markbt
Differential Revision: D25122288
fbshipit-source-id: 9ae0be49986dbcc31cee9a46bd30093b07795c62
Summary:
HgBlobEntry is kind of a problematic type right now:
- It has no typing to indicate if it's a file or a manifest
- It always has a file type, but we only sometimes care about it
This results in us using `HgBlobEntry` to sometimes represent
`Entry<HgManifestId, (FileType, HgFileNodeId)>`, and some other times to
represent `Entry<HgManifestId, HgFileNodeId>`.
This makes code a) confusing, b) harder to refactor because you might be having
to change unrelated code that cares for the use case you don't care about (i.e.
with or without the FileType), and c) results in us sometimes just throwing in
a `FileType::Normal` because we don't care about the type, which is prone to
breaking stuff.
So, this diff just removes it, and replaces it with the appropriate types.
Reviewed By: farnz
Differential Revision: D25122291
fbshipit-source-id: e9c060c509357321a8059d95daf22399177140f1
Summary:
I'd like to get rid of HgEntry altogether, as it's not a very useful
abstraction (`Entry` is better). This is one step towards this, by moving the
HgEntry -> Leaf conversion close to where the `HgEntry` is created (which is
where we upload filenodes).
Reviewed By: markbt
Differential Revision: D25122290
fbshipit-source-id: 0e3049392791153b9547091516e702fb04ad7094
Summary: Remove 'static requirement for async methods of Blobstore, propagate this change and fixup low hanging fruits where the code can become 'static free easily.
Reviewed By: ahornby, farnz
Differential Revision: D24839054
fbshipit-source-id: 5d5daa04c23c4c9ae902b669b0a71fe41ee6dee6
Summary:
This isn't actually being consulted anywhere save for a single test, so let's
just remove it (it's not like the test checks anything important — that field
might not as well exist given we never read it).
Reviewed By: farnz
Differential Revision: D25093494
fbshipit-source-id: 5f4a53f8666fc0e8a89ceade44baa96e71fb813f
Summary:
Add a method to `SkeletonManifest` which finds the paths to the first case
conflict, if there is one.
First means lexicographically first within directories, and with the shortest path.
Reviewed By: ahornby, StanislavGlebik
Differential Revision: D24990175
fbshipit-source-id: ec10f66582b81c40740823e32362ca489a6ebb4d
Summary: Now that `derive03` is the only version available, rename it to `derive`.
Reviewed By: krallin
Differential Revision: D24900106
fbshipit-source-id: c7fbf9a00baca7d52da64f2b5c17e3fe1ddc179e
Summary:
Now that all code is using `BonsaiDerived::derive03`, we can remove the old
futures version `BonsaiDerived::derive`.
Reviewed By: krallin
Differential Revision: D24900108
fbshipit-source-id: 885d903d4a45e639e4d44e19b5d70fac26bce279
Summary:
Allow users of `tests_utils` to create paths that are not `String`, by supporting any type
that can be converted into `MPath`.
Reviewed By: StanislavGlebik
Differential Revision: D24887002
fbshipit-source-id: 47ad567507185863c1cfa3c6738f30aa9266901a
Summary:
`manifest/test_utils` contains test utilities that are only used by derived data, and only one
of which relates to manifests. Its name (`test_utils`) is also confusing with `tests/utils`.
Move it to `derived_data_test_utils`, and update it to new futures.
Reviewed By: mitrandir77
Differential Revision: D24787536
fbshipit-source-id: 7a4a735132ccf81e3f75683c7f44c9ada11bc9d7
Summary:
This updates the external facing API of the filestore to use 0.3 streams.
Internally, there is still a bit of 0.3 streams, but as of this change, it's
all 0.3 outside.
This required a few changes here and there in places where it was simpler to
just update them to use 0.3 futures instead of `compat()`-ing everything.
Reviewed By: ikostia
Differential Revision: D24731298
fbshipit-source-id: 18a1dc58b27d129970a6aa2d0d23994d5c5de6aa
Summary:
Like it says in the title. This required quite a lot of changes at callsites,
as you'd expect.
Reviewed By: StanislavGlebik
Differential Revision: D24731299
fbshipit-source-id: e58447e88dcc3ba1ab3c951f87f7042e2b03eb2c
Summary:
This updates the metadata APIs in the filestore to futures 0.3 & async / await.
This changes the external API of the filestore, so there's quite a bit of churn
outside of that module.
Reviewed By: markbt
Differential Revision: D24727255
fbshipit-source-id: 59833f185abd6ab9c609c6bcc22ca88ada6f1b42
Summary: As part of the effort to deprecate futures 0.1 in favor of 0.3 I want to create a new futures_ext crate that will contain some of the extensions that are applicable from the futures_01_ext. But first I need to reclame this crate name by renaming the old futures_ext crate. This will also make it easier to track which parts of codebase still use the old futures.
Reviewed By: farnz
Differential Revision: D24725776
fbshipit-source-id: 3574d2a0790f8212f6fad4106655cd41836ff74d
Summary:
Now that fsnodes is async, convert more functions to use references, and tidy
up imports and type names.
Reviewed By: krallin
Differential Revision: D24726145
fbshipit-source-id: 75a619777f19754daf494a3743d26fa2e77aef54
Summary:
Update `fsnodes::derive_fsnode` and its immediate utility functions to use new style
futures and `async`/`.await` syntax.
Reviewed By: krallin
Differential Revision: D24726146
fbshipit-source-id: 0b0d5b1162a73568ef5c47db6e8252267e760e7f
Summary:
This changes the methods from ones that return old `BoxFuture`s to an async method
using `async_trait`.
Reviewed By: krallin
Differential Revision: D24689506
fbshipit-source-id: 7b13010924369f81681e6590898af703c5423385
Summary:
This changes the trait method from one that returns an old `BoxFuture` to an async
method using `async_trait`.
Reviewed By: krallin
Differential Revision: D24686888
fbshipit-source-id: 0ac231cdbb60d256b6d5ad5aafbe8779b96905f3
Summary:
Add BonsaiDerived::derive03 for new callsites that are 0.3/async aware.
Plan is to migrate callsites and then rename BonsaiDerived::derive03 to derive.
Reviewed By: StanislavGlebik
Differential Revision: D24389763
fbshipit-source-id: 4a6f351b9e0c5a2e9a9c47429764f88c47caff08
