Commit Graph

58404 Commits

Author SHA1 Message Date
Mateusz Kwapich
592e696146 remove one of ancestry check implementations
Summary:
I'm not sure why but D14405696 added this `is_ancestor` method with a
duplicate ancestry check implementation which:
 * has different behaviour when `ancestor == descendant`
 * doesn't log to ODS

Let's just settle on using one implementation now.

Reviewed By: StanislavGlebik

Differential Revision: D21937312

fbshipit-source-id: 0f4e2390cb0b984f92cd4fe645d9313fa4bba70e
2020-06-15 02:55:35 -07:00
Mateusz Kwapich
a56afca442 migrate lazy_index_node to async/await syntax
Summary: incremental step on the path to modernization of this module

Reviewed By: StanislavGlebik

Differential Revision: D21937311

fbshipit-source-id: 9db84c7cb166084edfc5a0638c1c884964f9984c
2020-06-15 02:55:34 -07:00
Mateusz Kwapich
86d3e87b16 migrate find_nodes_to_index to async/await syntax
Summary: incremental step on the path to modernization of this module

Reviewed By: StanislavGlebik

Differential Revision: D21937309

fbshipit-source-id: 61b1b4302782cea35d3f65e6fe982f530293dd81
2020-06-15 02:55:34 -07:00
Durham Goode
5a83869eb1 thrift: add serde to generated thrift types
Summary:
This will let us serialize and deserialize thrift structures from
non-thrift sources. In my particular case I'll be using it to deserialize json
into hg client config thrift structures that are shared with the backend.

Reviewed By: quark-zju

Differential Revision: D21875371

fbshipit-source-id: 6a85f28e69f2eeb375f6639ed1dcd3c1cd957d99
2020-06-12 16:51:52 -07:00
Arun Kulshreshtha
a5d36bf798 edenapi_server: Remove 'pretty' param from repos endpoint
Summary: The intention of this parameter was to produce human-readable JSON for tests and for manual debugging. In practice, given that the JSON response here is just a (typically short) list, there isn't really much value in prettifying. We weren't using this anyway, so             let's remove it to simplify the code.

Reviewed By: xavierd

Differential Revision: D22020784

fbshipit-source-id: 09290cc667cfdb194693e6c05858af2f8905f88e
2020-06-12 16:41:30 -07:00
Arun Kulshreshtha
8f3c80fbe5 edenapi_server: Add explicit file and tree handler functions
Summary: Add wrapper functions around `data::data` for files and trees. The intention is to make the code more easily searchable, and also to establish a consistent naming pattern across all the handler functions.

Reviewed By: xavierd

Differential Revision: D22020783

fbshipit-source-id: 5e7c63ff8f366b0c6aa4412d5d04d17ca6cbc6b9
2020-06-12 16:41:30 -07:00
Arun Kulshreshtha
2110dd9c26 edenapi_server: use macro to reduce handler boilerplate
Summary: Reduce the amount of repeated code in the handlers module by defining a macro to wrap async handler functions.

Reviewed By: krallin

Differential Revision: D22015557

fbshipit-source-id: 276dad4112e7019f8d5f5fa74243747a0104e1ca
2020-06-12 13:01:20 -07:00
Xavier Deguillard
8b9401b0b2 pyerror: format the original error, not the downcasted one
Summary:
The anyhow error contains the context for the error which will lead to better
error message for the user (and us). What the code previously did was simply
using the Debug trait to print the error, and thus was missing context.

Reviewed By: DurhamG

Differential Revision: D21985745

fbshipit-source-id: 31c603d7f42e79a360541f39e4aaf0fcfbb9a14f
2020-06-12 12:38:06 -07:00
Ailin Zhang
ec906443ea print fsck to stdout
Summary: This diff passes a callback function `progressCallback` from `EdenServer` to make `OverlayChekcer` print fsck messages to stdout.

Reviewed By: fanzeyi

Differential Revision: D21966060

fbshipit-source-id: 317b2c8954c718b51b5295c9f8d7698eb203906e
2020-06-12 12:34:55 -07:00
Arun Kulshreshtha
48544b0233 edenapi_server: add ErrorKind
Summary: Add an `errors` module containing an `ErrorKind` implemented via `thiserror`. All errors in the server's HTTP handles been annotated with an appropriate `ErrorKind` via `anyhow::Context`. Notably, there is specialized handling of `MononokeError`s from the `mononoke_api` crate via a new `MononokeErrorExt` trait.

Reviewed By: krallin

Differential Revision: D22012237

fbshipit-source-id: 5f6948696e4741816636f1c6fee12069ee1a4b32
2020-06-12 12:16:52 -07:00
Arun Kulshreshtha
63ffc2d838 mononoke_api: add Send + Sync + 'static bounds to HgDataContext and HgDataId
Summary:
Add trait bounds to ensure that implementors of these trait can be
safely used across threads. This is particularly useful when using these traits
in generic async code -- async code typically requires these trait bounds on
data captured in combinator closures, etc.

Reviewed By: krallin

Differential Revision: D22008466

fbshipit-source-id: 48eac59ca62debc9bd293261f6079a4fa134f331
2020-06-12 12:16:51 -07:00
Arun Kulshreshtha
19b249d2b7 edenapi_server: refactor utils module
Summary: Move the `util` module out of `handlers` and split it up into multiple submodules.

Reviewed By: krallin

Differential Revision: D22008148

fbshipit-source-id: 11b286fdef8f9d93fcc10a283ec1b2f42e9c8c04
2020-06-12 12:16:51 -07:00
Arun Kulshreshtha
14846656f5 edenapi_types: restructure crate
Summary:
Note: Although this diff looks big, it's all just cutting and pasting code.

Restructure the crate so that the types relevant to each EdenAPI endpoint are grouped in their own module. This means that authors of new endpoints can just add a new module and put their types there, rather than having to make edits throughout the crate.

Reviewed By: quark-zju

Differential Revision: D21983470

fbshipit-source-id: 4901662dc6b5f0a1feca1ab8d914f394114ef233
2020-06-12 12:16:51 -07:00
Arun Kulshreshtha
b7aafa1df9 edenapi_types: add HistoryResponseChunk
Summary: Change `HistoryResponse` so that instead of repeatedly sending the file path alongside every history entry, we group the history entries by file and send the path once per group. This will prevent the server from transmitting redundant information.

Reviewed By: quark-zju

Differential Revision: D21982558

fbshipit-source-id: f1c5d2573c97940c7bf1645ed7fef6e1887c0d42
2020-06-12 12:16:51 -07:00
Chad Austin
6185a90dc6 fix warnings on macOS
Summary: Fix a couple warnings in the macOS build.

Reviewed By: genevievehelsel

Differential Revision: D21992085

fbshipit-source-id: 578adb3b95d29177740debbce4dc4626ba9eed64
2020-06-12 10:28:28 -07:00
Lukas Piatkowski
07e0427eb4 mononoke: make segmented_changelog OSS buildable
Reviewed By: krallin

Differential Revision: D21980974

fbshipit-source-id: 60e8f0e78b2bd654e5109b39c85f08ee5b3e9490
2020-06-12 06:54:32 -07:00
Thomas Orozco
51623fbdbd mononoke/tests: disable mutation.record everywhere
Summary:
This is consistent with what is being done for now in hg for tests that haven't been migrated
to modern configurations yet, and ensures we get stable commit hashes in our tests: D21899139. It's already explicitly turned on on tests that want it.

In the future, this should probably be updated to use "modern configs" like the
Mercurial tests do.

Reviewed By: ikostia

Differential Revision: D22016705

fbshipit-source-id: b27f6423bf4ec5244ef3ce2e7676306165a331a8
2020-06-12 04:17:18 -07:00
Thomas Orozco
304701f890 mononoke/tests: remove expecting a message that is gone
Summary: This message is gone we shouldn't expect it anymore: D21913608

Reviewed By: ikostia

Differential Revision: D22016684

fbshipit-source-id: 97d86e9750e775c1bb3a1e75939f506cd35851c0
2020-06-12 04:17:17 -07:00
Thomas Orozco
01a84fabe4 mononoke: turn off mutation in test-commitcloud
Summary:
This test broke when this got turned on for all tests (D21899139). It's not
enabled for other commit cloud tests there, so let's be consistent.

Reviewed By: ikostia

Differential Revision: D22016686

fbshipit-source-id: 5f4385b60fd31c89e335e971f262da1226f32254
2020-06-12 04:17:17 -07:00
Thomas Orozco
cb64897c70 infinitepush: only warn about mutation server support if entries are to be sent
Summary:
In the Mononoke integration tests (which are .t tests), we've started seeing
this waring show up.

We probably shouldn't have mutation enabled for most of those tests anyway
(save for the tests that actually exercise mutation), so I'm turning that off.
That said, we'd still see the warning. So, this diff refactors mutation a
little bit to only warn if we do have something to send to the server.

Reviewed By: ikostia

Differential Revision: D22016685

fbshipit-source-id: e9e8dbcc44a0fa048dca3ca7ce0961631058a2d6
2020-06-12 04:17:17 -07:00
Jeremy Fitzhardinge
d03da109c7 mononoke: fix some little lints
Reviewed By: StanislavGlebik

Differential Revision: D22004427

fbshipit-source-id: 1418f91a3322ef3a3ff620c4e17745fc765dc971
2020-06-12 01:33:20 -07:00
Arun Kulshreshtha
b462c0575c edenapi_server: remove unused futures-old dependency
Summary: Last usage was removed in D20342689.

Reviewed By: xavierd

Differential Revision: D22011195

fbshipit-source-id: 58455ad3dc77438704e80c69eb728a319ee71a39
2020-06-11 22:28:53 -07:00
Genevieve Helsel
ca1da512bf store force restart action as its own flag
Summary: In its current state, FORCE is a restart mode that cannot be combined with other modes (specifically `graceful`). Removing this coupling from `force` will allow us to at first attempt a graceful restart, but if the daemon is not healthy, then do a force restart, all within one call.

Reviewed By: wez

Differential Revision: D21873166

fbshipit-source-id: 079a777ff6e7b219198cbef15335244b9865f5f3
2020-06-11 17:40:51 -07:00
Arun Kulshreshtha
5be68125a7 gotham_ext: Add StreamBody
Summary:
Move the LFS server's `StreamBody` into `gotham_ext`, along with some changes to decouple it from the LFS server.

In particular, the `Content-Length` header and post-request `Sender` have been made optional fields that can be set via a builder-style interface. The LFS server's `StreamBody` has been renamed to `LfsStreamBody` and is now a thin wrapper around `gotham_ext`'s `StreamBody` that preserves the old behavior.

Reviewed By: krallin

Differential Revision: D21988855

fbshipit-source-id: a9bf9c04bb791388d761fc705ebc38472a713b65
2020-06-11 13:03:24 -07:00
Stefan Filip
f586f46ca5 mononoke: move replication_lag utilities from blobstore_healer to sql_ext
Summary: I would like to use these utilities when building segmented changelog.

Reviewed By: krallin

Differential Revision: D21876432

fbshipit-source-id: 9022627e224bfcb155b47d696371d24e538e6f39
2020-06-11 12:38:50 -07:00
Stefan Filip
c06de91150 mononoke: add tracking for SQL connection usage
Summary: Incrementing CoreContext::perf_counters().

Reviewed By: krallin

Differential Revision: D21875162

fbshipit-source-id: 0187034c3004efb86d74ce23ddf61a2b469efc61
2020-06-11 12:38:49 -07:00
Stefan Filip
754c1d00c0 mononoke: add counters to segmented changelog calls
Summary: Title.

Reviewed By: krallin

Differential Revision: D21875163

fbshipit-source-id: e7c50e30d10a4d192edb082c0cabe290c4e6d29a
2020-06-11 12:38:49 -07:00
Stefan Filip
0bb787d892 mononoke: add batch query capability to segmented changelog IdMap
Summary:
Useful for keeping the insert logic working nicely even when we double insert
or have other inconsistencies.

Reviewed By: krallin

Differential Revision: D21853048

fbshipit-source-id: 958cbc0435f330749c5aae1ea0f41ecd03b01468
2020-06-11 12:38:48 -07:00
Mark Thomas
76bff4c32d segmented_changelog: fix warnings
Reviewed By: sfilipco

Differential Revision: D21998893

fbshipit-source-id: f0b8d82ec4937263fbe0db2e626a3847118e5b08
2020-06-11 10:55:31 -07:00
Mark Thomas
d3a02aa301 scs_server: use service_identity for repo_create_commit
Summary:
If the caller provides a service identity in `repo_create_commit`, use that to obtain
write access to the repository.

Reviewed By: StanislavGlebik

Differential Revision: D21995724

fbshipit-source-id: ba218e392b979938d598a37494b0ffe496f28722
2020-06-11 10:44:52 -07:00
Mark Thomas
4b76a1564d mononoke_api: remove legacy functions
Summary: Now that apiserver is gone, we can remove the legacy parts of mononoke_api.

Reviewed By: krallin

Differential Revision: D21976615

fbshipit-source-id: 139d7ebd4a7185693f6a4fe4e5c234c143f18498
2020-06-11 10:44:52 -07:00
Mark Thomas
f261469a2f scs_server: implement repo_move_bookmark
Summary: Implement repo_move_bookmark by calling the appropriate method in mononoke_api.

Reviewed By: StanislavGlebik

Differential Revision: D21904980

fbshipit-source-id: 537abe2241688f9c18ad279c169e30d9cee51c66
2020-06-11 10:44:51 -07:00
Mark Thomas
b0509f5419 mononoke_api: add move_bookmark method
Summary:
Add the `move_bookmark` method to `mononoke_api`.

This attempts to re-use code from `repo_client`.  This code isn't really designed to be called from this API, so the fit is very poor.  In a future diff we will fix this up.

The `repo_client` code for force-moving a bookmark does not support running hooks for that bookmark.  For now we will prevent API users from moving bookmarks that have hooks configured.  This will also be addressed in a future diff.

Reviewed By: krallin

Differential Revision: D21904979

fbshipit-source-id: 42bf840489e5b04f463c69c752bcaa5174630c21
2020-06-11 10:44:51 -07:00
Jun Wu
7728e73e01 mutation: do not auto migrate if proxy-obsstore is enabled
Summary:
There is no need to migrate if the obsstore is a proxy to the mutation store.
This avoids a crash reading the attribute `_all`.

Reviewed By: DurhamG

Differential Revision: D21999245

fbshipit-source-id: bfa6439982834c6835412fc175132526525af548
2020-06-11 10:00:42 -07:00
Xavier Deguillard
e2bb3e23d7 tests: do not enable mutation for test-common-commands-fb.t
Summary: The test breaks, let's not enable mutation for it.

Reviewed By: DurhamG

Differential Revision: D21998151

fbshipit-source-id: 9c06485287d906837444f39faf90886933decc9b
2020-06-11 09:44:24 -07:00
Kostia Balytskyi
87970c4168 tests: fix broken test-infinitepush-bookmarks-disabled.t
Reviewed By: markbt

Differential Revision: D21997584

fbshipit-source-id: b2a1f0daf540911cdde9735a53f4d2b6d3d6984e
2020-06-11 09:22:28 -07:00
Stanislau Hlebik
f7a418e8a9 mononoke: add a warning for slow getbundle calls
Reviewed By: HarveyHunt

Differential Revision: D21997359

fbshipit-source-id: 61f428b6e9e1bd2fa5480c8ad9330f438ddc83f6
2020-06-11 08:57:18 -07:00
svcscm
70ab1446b6 Updating submodules
Summary:
GitHub commits:

b42503196a
49499305d6
4e05aa3f7f

Reviewed By: jurajh-fb

fbshipit-source-id: e3da4fd0cec42797607c8a3f476a9e07784c51e8
2020-06-11 04:41:11 -07:00
Viet Hung Nguyen
3de40b0acb mononoke/revset: refactored TestChangesetFetcher repo variable type
Summary: Replaced Arc<BlobRepo> with BlobRepo for TestChangesetFetcher's repo variable (mononoke/revset/src/tests.rs) and refactored variables depending on TestChangesetFetcher

Reviewed By: StanislavGlebik

Differential Revision: D21993963

fbshipit-source-id: f617d693ec85156292e35f201c097bb935524062
2020-06-11 04:28:47 -07:00
svcscm
dce839459c Updating submodules
Summary:
GitHub commits:

be53b4f7e0
5b3422af1d
4747e6910f
8a6c441589
20c2ee07f2
0bb7a26c7e

Reviewed By: jurajh-fb

fbshipit-source-id: 1af1e0953d95818d3b6caad7ff6e3c295db2f0a2
2020-06-11 03:05:04 -07:00
Kostia Balytskyi
d176764593 derived_data: add splay to lease retry backoff times
Reviewed By: StanislavGlebik

Differential Revision: D21975819

fbshipit-source-id: 8acb7d5d7795d19c036f1f0e232e6eff97711d9e
2020-06-11 02:52:00 -07:00
Stanislau Hlebik
6fd0284b72 mononoke: rename comments() to message()
Summary:
Some of the hg changeset structures use `comments` as the name for commit
message. But this is confusing - let's rename

Reviewed By: HarveyHunt

Differential Revision: D21974571

fbshipit-source-id: e7c5c5ad8db9b2f1343abe9101fc56a6d4287548
2020-06-11 01:24:47 -07:00
Chad Austin
597c209fa1 disallow warnings in integration tests
Summary:
Unawaited coroutines are pernicious and not errors by default. In
advance of enabling asynchronous tests, fail our tests when Python
warnings are shown.

We record warnings and report them after the test body instead of
enabling an "error" warning filter because exceptions thrown by
finalizers are silently swallowed, and most important warnings occur
during finalization.

Reviewed By: genevievehelsel

Differential Revision: D21955210

fbshipit-source-id: b1fc52ddfa9f9befbde6800f85f090862af0ef15
2020-06-10 23:23:34 -07:00
svcscm
fd5e2481c7 Updating submodules
Summary:
GitHub commits:

b3485d1793
588ffede6b
cade649a20
2c9b2aadbc
0841d96c76
79b45fca22
65fe3a03d4
bb06a1d81d

Reviewed By: jurajh-fb

fbshipit-source-id: d97b9e1dd29f19c7dee9f28eec60e133e8a5656a
2020-06-10 22:55:38 -07:00
Jun Wu
277933aad8 enable remotenames extension in cpp tests
Summary:
The remotenames extension was enabled everywhere in production. It is
unsupported if remotenames is disabled.

Ideally the cpp tests should use the in-repo configuration instead of
maintaining its own hardcoded hgrc. For now I just did the minimal change to
unblock tests.

Reviewed By: fanzeyi

Differential Revision: D21986274

fbshipit-source-id: b632da074baf4026f331381ae8f2f8479a34e1b3
2020-06-10 22:36:10 -07:00
Durham Goode
17eb13e7b5 configs: fix config generation when executed from outside the repo
Summary:
When kicking off the background process, we were depending on the
current working directory being inside the repo. This is not always true, like
when the current command was invoked like `hg -R xxx log -r .`. Let's invoke the
debugdynamicconfig command with the explicit path.

Reviewed By: quark-zju

Differential Revision: D21983297

fbshipit-source-id: 72981ea596e62dced3918f2fc6272953ab3d84cd
2020-06-10 22:16:39 -07:00
Xavier Deguillard
16777c3d81 merge: remove the rustworkers option
Summary: It's now the default.

Reviewed By: DurhamG

Differential Revision: D21981563

fbshipit-source-id: 9e27e65ea8e47019f1542a505f5b7726c2cbeba2
2020-06-10 20:10:29 -07:00
Xavier Deguillard
d4a20e4160 tests: disable treemanifest in test-fb-hgext-pushrebase-protection.t
Summary:
Treemanifest was recently enabled in this test, but it breaks the Windows
tests. For now, let's just go back to the previous state and we can investigate
later.

Differential Revision: D21989349

fbshipit-source-id: fb3d3d1315400f56a6166a01112affc0a4d96da6
2020-06-10 19:37:13 -07:00
Xavier Deguillard
879e485aa9 tests: fix test-requires-t.py
Summary:
Somehow clone-dst isn't empty which makes the second clone to fail due to it,
let's just clone it elsewhere.

Differential Revision: D21989189

fbshipit-source-id: 125f33f3f498d0c089785018e39ee7c5b795b472
2020-06-10 19:37:13 -07:00
Jun Wu
704fb871c6 tests: enable mutation for some Python tests
Summary: Those tests only need minor changes to be compatible with mutation.

Reviewed By: sfilipco

Differential Revision: D21913606

fbshipit-source-id: 0a2b0e08cb558960a7d0f938300f375c34457a62
2020-06-10 19:30:01 -07:00