Commit Graph

17 Commits

Author SHA1 Message Date
Arun Kulshreshtha
5f0181f48c Regenerate all Cargo.tomls after upgrade to futures 0.3.5
Summary: D22381744 updated the version of `futures` in third-party/rust to 0.3.5, but did not regenerate the autocargo-managed Cargo.toml files in the repo. Although this is a semver-compatible change (and therefore should not break anything), it means that affected projects would see changes to all of their Cargo.toml files the next time they ran `cargo autocargo`.

Reviewed By: dtolnay

Differential Revision: D22403809

fbshipit-source-id: eb1fdbaf69c99549309da0f67c9bebcb69c1131b
2020-07-06 20:49:43 -07:00
Simon Farnsworth
454de31134 Switch Loadable and Storable interfaces to new-style futures
Summary:
Eventually, we want everything to be `async`/`await`; as a stepping stone in that direction, switch some of the blobstore interfaces to new-style `BoxFuture` with a `'static` lifetime.

This does not enable any fixes at this point, but does mean that `.compat()` moves to the places that need old-style futures instead of new. It also means that the work needed to make the transition fully complete is changed from a full conversion to new futures, to simply changing the lifetimes involved and fixing the resulting compile failures.

Reviewed By: krallin

Differential Revision: D22164315

fbshipit-source-id: dc655c36db4711d84d42d1e81b76e5dddd16f59d
2020-06-25 08:45:37 -07:00
Pavel Aslanov
905c8b213e move Filenodes to BlobRepo::attributes
Summary: Move `Filenodes` to `BlobRepo::attributes` as it is mercurial specific.

Reviewed By: ikostia

Differential Revision: D21662418

fbshipit-source-id: 87648a3e6fd7382437424df3ee60e1e582b6b958
2020-06-22 07:29:19 -07:00
Stanislau Hlebik
fde6230ea2 RFC: introduce FilenodeResult
Summary:
The motivation for the whole stack:
At the moment if mysql is down then Mononoke is down as well, both for writes
and for reads. However we can relatively easily improve the situation.
During hg update client sends getpack() requests to fetch files, and currently
for each file fetch we also fetch file's linknode. However hg client knows how
to deal with null linknodes [1], so when mysql is unavailable we can disable
filenode fetching completely and just return null linknodes. So the goal of this stack is to
add a knob (i.e. a tunable) that can turn things filenode fetches on and off, and make
sure the rest of the code deals nicely with this situation.

Now, about this diff. In order to force callers to deal with the fact that
filenodes might unavailable I suggest to add a special type of result, which (in
later diffs) will be returned by every filenodes methods.

This diff just introduces the FilenodeResult and convert BlobRepo filenode
methods to return it. The reason why I converted BlobRepo methods first
is to limit the scope of changes but at the same time show how the callers' code will look
like after FilenodeResult is introduced, and get people's thoughts of whether
it's reasonable or not.

Another important change I'd like to introduce in the next diffs is modifying FilenodesOnlyPublic
derived data to return success if filenodes knob is off. If we don't do that
then any attempt to derive filenodes might fail which in turn would lead to the
same problem we have right now - people won't be able to do hg update/hg
pull/etc if mysql is down.

[1] null linknodes might make some client side operation slower (e.g. hg rebase/log/blame),
so we should use it only in sev-like situations

Reviewed By: krallin

Differential Revision: D21787848

fbshipit-source-id: ad48d5556e995af09295fa43118ff8e3c2b0e53e
2020-06-01 05:27:34 -07:00
Jeremy Fitzhardinge
04c7c17185 mononoke: remove unused deps for a couple of targets
Summary:
Just top-level mononoke and cache_warmup as a more or less random
selection.

Reviewed By: StanislavGlebik

Differential Revision: D21648852

fbshipit-source-id: 0e5137da9a7742377112e085e9d29182abdfbf41
2020-05-20 17:00:08 -07:00
Stefan Filip
60966c93e7 autocargo: regenerate
Summary: maintenance

Reviewed By: StanislavGlebik

Differential Revision: D21640322

fbshipit-source-id: d0b2ce604735c05d540d06835c8e4c8a940fbf5c
2020-05-19 16:08:40 -07:00
Lukas Piatkowski
2a1863c35d mononoke/server: make the main mononoke binary OSS buildable
Summary: Covering repo_listener and microwave plus some final touch and we have a buildable Mononoke binary.

Reviewed By: krallin

Differential Revision: D21379008

fbshipit-source-id: cca3fbb53b90ce6d2c3f3ced7717404d6b04dd51
2020-05-06 06:11:02 -07:00
Thomas Orozco
0dd85d1c15 mononoke/cache_warmup: add a bit more error context
Summary:
This is helpful. Also, while in there, I removed an error that wasn't used at
all.

Reviewed By: StanislavGlebik

Differential Revision: D21399489

fbshipit-source-id: 0e5ef20b842afa9ffc0bb8530c48eb48339c558e
2020-05-05 05:44:53 -07:00
Thomas Orozco
48d3c33411 mononoke/microwave: warmup from last derived state
Summary:
Microwave doesn't normally allow writes, which can cause cache warmup to fail
if master has underived commits. So, let's go back in bookmarks history to
whatever is most recent and derived. We can do so using the existing logic we
use in the warm bookmarks cache.

Reviewed By: farnz

Differential Revision: D21325485

fbshipit-source-id: 11e758cd512a22e02704ac34458fead18c284c20
2020-04-30 12:47:35 -07:00
Thomas Orozco
a381722ad1 mononoke/cache_warmup: asyncify
Summary:
This updates cache_warmup to new futures, since I'd like to do a little bit of
work in there. This also changes a bit of functionality: until now we were
roundtripping through a hg changeset id un-necessarily so that updates it to
just use the bcs id instead.

Reviewed By: mitrandir77

Differential Revision: D21325484

fbshipit-source-id: 4d296c5fbe7c21dda681aed4ed9c572a38246893
2020-04-30 12:47:35 -07:00
Thomas Orozco
dd38f1fdb2 mononoke/cache_warmup: conditionally use microwave for faster warmup
Summary:
This incorporates microwave into the cache warmup process. See earlier in this
stack for a description of what this does, how it works, and why it's useful.

Reviewed By: ahornby

Differential Revision: D20219904

fbshipit-source-id: 52db74dc83635c5673ffe97cd5ff3e06faba7621
2020-03-05 11:57:43 -08:00
Thomas Orozco
16d5ab5066 mononoke/cache_warmup: remove tracing
Summary:
We don't use it, and this tries to write to Manifold from tests, which is
undesirable. Let's remove it;

Reviewed By: farnz

Differential Revision: D20219902

fbshipit-source-id: 2e983bee54cadad257648cc9633695be825a1ef3
2020-03-04 04:02:19 -08:00
Stanislau Hlebik
e9f78e0601 mononoke: add context with repoid to cache_warmup error message
Summary: Makes it easier to understand which repo is failing

Reviewed By: krallin

Differential Revision: D20244630

fbshipit-source-id: ca32f7831c5ed4e701103020e9878c459ba6d573
2020-03-04 01:52:11 -08:00
Mark Thomas
a8f06f75c0 derived_data: add DeriveError for when derivation is disabled
Summary:
Currently if derivation of a particular derived data type is disabled, but a
client makes a request that requires that derived data type, we will fail with
an internal error.

This is not ideal, as internal errors should indicate something is wrong, but
in this case Mononoke is behaving correctly as configured.

Convert these errors to a new `DeriveError` type, and plumb this back up to
the SCS server.  The SCS server converts these to a new `RequestError`
variant: `NOT_AVAILABLE`.

Reviewed By: krallin

Differential Revision: D19943548

fbshipit-source-id: 964ad0aec3ab294e4bce789e6f38de224bed54fa
2020-02-19 09:28:09 -08:00
Pavel Aslanov
c902acbc3f remove the need to pass mapping to ::derive method
Summary: remove the need to pass mapping to `::derive` method

Reviewed By: StanislavGlebik

Differential Revision: D19856560

fbshipit-source-id: 219af827ea7e077a4c3e678a85c51dc0e3822d79
2020-02-12 10:22:39 -08:00
Lukasz Piatkowski
542d1f93d3 Manual synchronization of fbcode/eden and facebookexperimental/eden
Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.

Reviewed By: StanislavGlebik

Differential Revision: D19722832

fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
2020-02-11 11:42:43 +01:00
Lukasz Piatkowski
e8d62b64d5 mononoke: move the codebase under eden/ directory
fbshipit-source-id: 43a0252cb3ec42aa365f20d1b6faa4d24d74c9b8
2020-02-06 13:46:04 +01:00