Commit Graph

12 Commits

Author SHA1 Message Date
Thomas Orozco
16384599a8 mononoke (+ rust/shed/async_unit): update async_unit to expect async fn's
Summary:
This allows code that is being exercised under async_unit to call into code
that expects a Tokio 0.2 environment (e.g. 0.2 timers).

Unfortunately, this requires turning off LSAN for the async_unit tests, since
it looks like LSAN and Tokio 0.2 don't work very well together, resulting in
LSAN reporting leaked memory for some TLS structures that were initialized by
tokio-preview (regardless of whether the Runtime is being dropped):
https://fb.workplace.com/groups/rust.language/permalink/3249964938385432/

Considering async_unit is effectively only used in Mononoke, and Mononoke
already turns off LSAN in tests for precisely this reason ... it's probably
reasonable to do the same here.

The main body of changes here is also about updating the majority of our
changes to stop calling wait(), and use this new async unit everywhere. This is
effectively a pretty big batch conversion of all of our tests to use async fns
instead of the former approaches. I've also updated a substantial number of
utility functions to be async fns.

A few notable changes here:

- Some pushrebase tests were pretty flaky — the race they look for isn't
  deterministic. I added some actual waiting (using pushrebase hooks) to make
  it more deterministic.  This is kinda copy pasted from the globalrev hook
  (where I had introduced this first), but this will do for now.
- The multiplexblob tests don't work at all with new futures, because they call
  `poll()` all over the place. I've updated them to new futures, which required
  a bit of reworking.
- I took out a couple tests in async unit that were broken anyway.

Reviewed By: StanislavGlebik

Differential Revision: D19902539

fbshipit-source-id: 352b4a531ef5fa855114c1dd8bb4d70ed967dd55
2020-02-18 01:55:00 -08:00
Stanislau Hlebik
ee7c0a8d26 backfill_derived_data mononoke: fail if derived data disabled
Summary:
Now let's fail if we try to derive data that's not enabled in the config.
Note that this diff also adds `derive_unsafe()` method which should be used in backfiller.

Reviewed By: krallin

Differential Revision: D19807956

fbshipit-source-id: c39af8555164314cafa9691197629ab9ddb956f1
2020-02-16 04:56:34 -08:00
Stanislau Hlebik
975d226cf2 mononoke: disable filenodes generation during hg changeset generation, take 2
Reviewed By: aslpavel

Differential Revision: D19856419

fbshipit-source-id: 0d1bcecf9f700d0789bba1227ef8a58748f04bbf
2020-02-13 03:18:13 -08:00
Pavel Aslanov
c902acbc3f remove the need to pass mapping to ::derive method
Summary: remove the need to pass mapping to `::derive` method

Reviewed By: StanislavGlebik

Differential Revision: D19856560

fbshipit-source-id: 219af827ea7e077a4c3e678a85c51dc0e3822d79
2020-02-12 10:22:39 -08:00
Lukasz Piatkowski
542d1f93d3 Manual synchronization of fbcode/eden and facebookexperimental/eden
Summary:
This commit manually synchronizes the internal move of
fbcode/scm/mononoke under fbcode/eden/mononoke which couldn't be
performed by ShipIt automatically.

Reviewed By: StanislavGlebik

Differential Revision: D19722832

fbshipit-source-id: 52fbc8bc42a8940b39872dfb8b00ce9c0f6b0800
2020-02-11 11:42:43 +01:00
Stanislau Hlebik
2b7e7e5676 mononoke: log if derived data is not enabled
Summary:
Before we start blocking generation of derived data let's start with logging if
derived data is not specified.

Reviewed By: farnz

Differential Revision: D19791523

fbshipit-source-id: 15bed8463f8a021de76a2878f06ec95d9fef877f
2020-02-10 01:44:09 -08:00
Stanislau Hlebik
8abe1af621 mononoke: add DerivedDataConfig
Summary:
See D19787960 for more details why we need to do it.
This diff just adds a struct in BlobRepo

Reviewed By: HarveyHunt

Differential Revision: D19788395

fbshipit-source-id: d609638432db3061f17aaa6272315f0c2efe9328
2020-02-10 01:44:09 -08:00
Thomas Orozco
d39eea991b blobrepo: don't fetch Hg Changeset IDs sequentially
Summary:
Fetching things from MySQL sequentially in a buffered fashion is a bad
practice, since we might end up saturating the underlying MySQL pool, and
starving other MySQL  clients.

Instead, let's make fewer, bigger queries.

Reviewed By: ahornby

Differential Revision: D19766787

fbshipit-source-id: 1cf9102eaca8cc1ab55b7b85039ca99627a86b71
2020-02-06 12:11:22 -08:00
Thomas Orozco
ce8b9a0fbe getbundle_response: don't fetch Hg Changeset IDs sequentially
Summary:
Fetching things from MySQL sequentially in a buffered fashion is a bad
practice, since we might end up saturating the underlying MySQL pool with a lot
of requests. Doing so will result in other queries being delayed as they wait
behind our batch of queries, which results in higher dispatch latency.

Instead, let's make fewer, bigger queries. Also, while we're in here, let's
update blobrepo to have an up-to-date comment.

Reviewed By: StanislavGlebik

Differential Revision: D19766788

fbshipit-source-id: 318ec4778ca259b210d431fc2add8b327bfce99a
2020-02-06 12:11:21 -08:00
Liubov Dmitrieva
8228f84a60 Short hashes lookup: implement suggestions the same way as in Mercurial.
Summary:
Suggestions come in the error message as it is currently implemented in
Mercurial code. Format of suggestions also stays the same.

We give the hash, time, author and the title.

All suggestions are ordered (most recent go first).

We don't show them if there are two many.

Reviewed By: krallin

Differential Revision: D19732053

fbshipit-source-id: b94154cbc5a4f440a0053fc3fac2bca2ae0b7119
2020-02-06 07:43:51 -08:00
Stanislau Hlebik
c97ceda175 mononoke: update IncompleteFilenodes to make transition to FilenodesOnlyPublic
Summary:
Jump from "generating filenodes while generating hg changeset" to "generate
filenodes separately" is tricky to do without breaking production. This diff
adds additional logic in IncompleteFilenodes that should make this transition
smoother. See code comment for more details.

Reviewed By: krallin

Differential Revision: D19741913

fbshipit-source-id: 48987c15fc4144c50afcee7ae34072f6cd634271
2020-02-06 07:26:14 -08:00
Lukasz Piatkowski
e8d62b64d5 mononoke: move the codebase under eden/ directory
fbshipit-source-id: 43a0252cb3ec42aa365f20d1b6faa4d24d74c9b8
2020-02-06 13:46:04 +01:00