Commit Graph

61603 Commits

Author SHA1 Message Date
Robin Håkanson
64523f39c1 Convinience access functions for BonsaiDiffFileChange
Summary: Convinience access functions for BonsaiDiffFileChange

Reviewed By: krallin

Differential Revision: D24715052

fbshipit-source-id: 6eac5dd8ff4ec21a59c82ebe5d3b39386f57baea
2020-11-10 09:39:29 -08:00
Egor Tkachenko
99643e0409 Added verification of generated bonsai changeset between backup and prod repos during blobimport
Summary: It is possible that hash of newly created bonsai_changeset will be different from what is in prod repo. In this case let's fetch bonsai from the prod, to make backup repo consistent with prod.

Reviewed By: StanislavGlebik

Differential Revision: D24593003

fbshipit-source-id: 70496c59927dae190a8508d67f0e3d5bf8d32e5c
2020-11-10 08:46:16 -08:00
Alex Hornby
7c33ddab42 mononoke: use create_graph to generate EdgeType enum in walker
Summary: Use create_graph to generate EdgeType enum in walker to reduce boiler plate needed when adding new derived node and edge types to the walker

Differential Revision: D24687827

fbshipit-source-id: 63337f4136c649948e0d3039529965c296c6d67e
2020-11-10 08:26:54 -08:00
Alex Hornby
72006da99d mononoke: add edge support to create_graph macro
Summary: Prepare for populating edge types programmatically from create_graph

Differential Revision: D24687830

fbshipit-source-id: be109b1dd45ef6cc398b0f7658d9168e8286699c
2020-11-10 08:26:54 -08:00
svcscm
e9495c19d4 Updating submodules
Summary:
GitHub commits:

d46040f952

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 62aa8e43a2f22bdcff99e19d8fefc5b422e626a5
2020-11-10 08:23:27 -08:00
Lukas Piatkowski
3b6d8c2c0a rust-shed/futures_ext: add .return_remainder similar to futures_01_ext
Summary: Also use the 0.3 compatible .return_remainder in unbundle.

Reviewed By: ikostia

Differential Revision: D24729464

fbshipit-source-id: ede5cc60f4b872a3b968cf14bb0e2c5d9b85c242
2020-11-10 07:40:14 -08:00
Mark Juggurnauth-Thomas
6b0c3cfef7 mononoke_types: don't allocate and format error string on good path
Summary:
When finishing a hash computation for a blob, we currently call `format!` to allocate
and format the error string before calling `.expect` on the `write_all` result.

In practice this will never fail, so this is wasted work.  From experimentation on
the playground, the Rust compiler does not appear to be smart enough to optimize this
away, either.

A small optimization, but let's get rid of this by calling panic directly, and
only in the failure path.

Reviewed By: farnz

Differential Revision: D24857833

fbshipit-source-id: e3e35b402ca3a9f6c9d8fbbd758cc486ef1c5566
2020-11-10 07:12:33 -08:00
svcscm
b833a85ba7 Updating submodules
Summary:
GitHub commits:

413dc72d1c

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 05ca4920da7a6b36baae89ba655d8f72456bcb0d
2020-11-10 06:56:41 -08:00
svcscm
d437a58cb2 Updating submodules
Summary:
GitHub commits:

65af2b8ccf

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 48fb722b5906c74c676e53d546e647bfe6346d66
2020-11-10 06:56:41 -08:00
svcscm
98d47e4355 Updating submodules
Summary:
GitHub commits:

2701806b5b

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 2db68be9e32ef1c48cd5fbf7432c52b3ee5ebeee
2020-11-10 04:59:51 -08:00
Mateusz Kwapich
10bc4843d3 new mode to reverse-fill bookmarks
Summary:
Adds `--reversefill` mode to bookmarks filler that fetches bookmark updates
from the queue and syncs them to infinitepush database.

Reviewed By: farnz

Differential Revision: D24538317

fbshipit-source-id: 5ac7ef601f2ff120c4efd8df08a416e00df0ceb9
2020-11-10 02:58:00 -08:00
Mateusz Kwapich
89fd335b1e populate bookmark filler queue on scratch bookmark updates
Summary:
Summarry:
This is the first part of syncing new scratch bookmark pushes from Mononoke to
Mercurial: on each bookmark movement we log this bookmark movement to filler's
queue.

Reviewed By: liubov-dmitrieva

Differential Revision: D24480546

fbshipit-source-id: 27103b4b4f8c4600aaf485826db2936eaffcc4a9
2020-11-10 02:58:00 -08:00
svcscm
b1f262490b Updating submodules
Summary:
GitHub commits:

eaab379904

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 70d31a0e568f1aef9a25de96b358401ed53ab201
2020-11-10 02:58:00 -08:00
svcscm
4f41e7683f Updating submodules
Summary:
GitHub commits:

86cb2b3b77
fec37de9a2
c3911f1a72
2cbef65587
ec3030c8cc

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 3b1648773239ee3e3efd8ec0b0a585c422620960
2020-11-09 12:32:31 -08:00
Alex Hornby
6d316abaf3 mononoke: fix naming for BonsaiToRootFsnode EdgeTypes in walker
Summary: Make the naming of bonsai fsnode edge variants consistent with the other edges in preparation for building them programmatically from a macro

Reviewed By: krallin

Differential Revision: D24687833

fbshipit-source-id: 8d46a53c023a4b8f95c0edc42df86e467c054ebb
2020-11-09 10:26:48 -08:00
Alex Hornby
39f27ef010 mononoke: fix naming for linknode EdgeTypes in walker
Summary: Make the naming of linknode edge variants consistent with the other edges in preparation for building them programmatically from a macro

Reviewed By: krallin

Differential Revision: D24687832

fbshipit-source-id: 46525d7bebd17723a130a70f566b24104cc39656
2020-11-09 10:26:47 -08:00
Alex Hornby
2f4bc233d4 mononoke: use macro to implement Node::get_type() in walker
Summary:
Use macro to implement Node::get_type() in walker.

Reduces the boiler plate when adding new types to the graph.

Reviewed By: farnz

Differential Revision: D24687826

fbshipit-source-id: 5f89c6fb69fd9df3fff25a2425a4d2035dbf5ed9
2020-11-09 10:26:47 -08:00
Alex Hornby
4b3bc9f285 mononoke: generate NodeType::root_edge_type() from macro in walker
Summary: Generate NodeType::root_edge_type() so less boiler plate when adding new types to the walk.

Reviewed By: farnz

Differential Revision: D24687825

fbshipit-source-id: 083fc57aee8fe01b29ad4a6f9ebe660cc057dfab
2020-11-09 10:26:47 -08:00
Alex Hornby
804dbe3a6c mononoke: define walker graph with macro to reduce repetition
Summary:
Define walker graph with macro to reduce repetition which should make adding new derived data types simpler.

Specifically, this removes the duplication between NodeType and Node

Reviewed By: farnz

Differential Revision: D24687831

fbshipit-source-id: 97d67faf02b2a88bb871dc0388d75d3dd3e8528d
2020-11-09 10:26:47 -08:00
Alex Hornby
8673a9387a mononoke: use strum EnumCount instead of own macros
Summary: use strum EnumCount instead of own macros,  can remove some code. This needed strum upgrade to 0.19

Reviewed By: krallin

Differential Revision: D24680441

fbshipit-source-id: 56e5b66f75c3d8ff949685c26f503571873c0cde
2020-11-09 10:26:47 -08:00
Alex Hornby
1d44f3a2b3 rust: update to strum 0.19
Summary: Update to strum 0.19 as it has improved EnumCount derivation

Reviewed By: mohanz

Differential Revision: D24680442

fbshipit-source-id: 2d3d2a84e994f09bf3b1c7ea748a80a67d100c13
2020-11-09 10:26:47 -08:00
svcscm
36c2b8ef04 Updating submodules
Summary:
GitHub commits:

a89ef43c97
43385dd09c

Reviewed By: bigfootjon

fbshipit-source-id: a21e712fd0477e2191e5a84ed204a0605de07460
2020-11-09 08:23:54 -08:00
Mark Juggurnauth-Thomas
4909562fe9 derived_data_test_utils: move test_utils to derived_data and update to new futures
Summary:
`manifest/test_utils` contains test utilities that are only used by derived data, and only one
of which relates to manifests.  Its name (`test_utils`) is also confusing with `tests/utils`.

Move it to `derived_data_test_utils`, and update it to new futures.

Reviewed By: mitrandir77

Differential Revision: D24787536

fbshipit-source-id: 7a4a735132ccf81e3f75683c7f44c9ada11bc9d7
2020-11-09 07:23:51 -08:00
svcscm
c493f18335 Updating submodules
Summary:
GitHub commits:

0fdaafd17b

Reviewed By: bigfootjon

fbshipit-source-id: cf42cb24259152c4830c8f4f8648946160db8b82
2020-11-09 06:23:44 -08:00
svcscm
f621137cd7 Updating submodules
Summary:
GitHub commits:

4ed170390b
2d1bc199de

Reviewed By: bigfootjon

fbshipit-source-id: 92b4e69a8afb82006447b9c5155d7fb94995112f
2020-11-09 06:23:43 -08:00
svcscm
c5d75b194f Updating submodules
Summary:
GitHub commits:

11440055b2

Reviewed By: bigfootjon

fbshipit-source-id: c414d7320ae0fdf710ff874ea6167de53fa87287
2020-11-09 03:24:23 -08:00
Alex Hornby
6091d25bee mononoke: reduce visibility of add_* functions that MononokeApp controls
Summary:
Reduce visibility of add_* functions that MononokeApp controls, no need for them to be public.

Updated a couple of binaries to use MononokApp.with_fb303_args() instead of calling the add_fb303 function directly.

Reviewed By: krallin

Differential Revision: D24757202

fbshipit-source-id: a068ca4fd976429e7c02c4049429553cc8acf3d4
2020-11-09 02:25:25 -08:00
Alex Hornby
08595c74a1 mononoke: update benchmark to be MononokeApp
Summary: benchmark is the last remaining user of args::add_cachelib_args outside of MononokeApp,  switch it to use MononokeApp instead.

Reviewed By: krallin

Differential Revision: D24755785

fbshipit-source-id: c105b4443394c88b6effdac382089e7eaca65bfe
2020-11-09 02:25:24 -08:00
Alex Hornby
f28a33a86e mononoke: make MononokeApp arguments more configurable
Summary: make MononokeApp arguments more configurable so binaries can opt out of them if a section does not apply,  making the --help more relevant.

Reviewed By: krallin

Differential Revision: D24757007

fbshipit-source-id: eed2f321bdbd04208567ef9a45cf861e56cdd07e
2020-11-09 02:25:24 -08:00
Stanislau Hlebik
d59b6d61e9 mononoke: make hash_validation_percentage a tunable
Summary:
Previously it was a config knob, but they are rather hard to change because
they require a restart of the service. Let's make it a tunable instead.

Reviewed By: farnz

Differential Revision: D24682129

fbshipit-source-id: 9832927a97b1ff9da49c69755c3fbdc1871b3c5d
2020-11-09 02:25:24 -08:00
svcscm
03586e21b5 Updating submodules
Summary:
GitHub commits:

de01724ca4
689e288241
cfd6702b03
e5a83651de
b480094424
0359f05808
d3351df447
64d711c01a
d9cb3fafb8

Reviewed By: bigfootjon

fbshipit-source-id: 0a2e538f24b962570d89c0b719e99d292a963887
2020-11-09 02:25:24 -08:00
svcscm
70aa96b4de Updating submodules
Summary:
GitHub commits:

c2a8dc4259
ad1b1ab030
0305594747

Reviewed By: bigfootjon

fbshipit-source-id: 0e152ce0ca29afa7006f3f5aae776220c71f024b
2020-11-08 04:23:18 -08:00
svcscm
136bbf10a5 Updating submodules
Summary:
GitHub commits:

fcd3cde609
896bf5c20d
716621ebc3
64dde261c7
9b55acf114
48e81ec67a
c438e6ebad
2f36dea07d

Reviewed By: bigfootjon

fbshipit-source-id: 9d4d91b00be078344f8c7bff0d03ca63e3214991
2020-11-07 16:23:28 -08:00
svcscm
f0e0092271 Updating submodules
Summary:
GitHub commits:

e5ea734651
f58ed86887
1ce105d0ea

Reviewed By: bigfootjon

fbshipit-source-id: 56defe3b119f66372adb605fd2df28c876674930
2020-11-06 16:25:00 -08:00
Jun Wu
d89311e478 Back out "log: backout D24293498 about prefetching commit data"
Summary:
The original problem was a fastlog bug, solved by D24513444 (c3bcc1ab88).

Restores prefetching for phabricator status so `hg ssl` and `hg fssl` become fast again.

Original commit changeset: b10c4caf8fda

Reviewed By: sfilipco

Differential Revision: D24749774

fbshipit-source-id: fa14f7dde9c922733525a7ff014efc32875426fa
2020-11-06 16:25:00 -08:00
Jun Wu
c6741e4c3a Back out "smartset: back out use Rust reentrant generator for generatorset"
Summary:
The original issue was a rust-cpython bug, solved by D24698226, or https://github.com/dgrunwald/rust-cpython/pull/244.

Original commit changeset: 08f598df0892

Reviewed By: sfilipco

Differential Revision: D24759765

fbshipit-source-id: f9a1359cfce68c8754ddd1bcb8bfc54bf75af7ff
2020-11-06 16:25:00 -08:00
Stanislau Hlebik
7b1746907e mononoke: initialize with_seq at the beginning of request handling
Reviewed By: ahornby

Differential Revision: D24788666

fbshipit-source-id: 25b2d854b96bc1ada42739fe0f94e20355a4d61b
2020-11-06 15:25:06 -08:00
svcscm
3a94509926 Updating submodules
Summary:
GitHub commits:

d3a32cf467

Reviewed By: bigfootjon

fbshipit-source-id: 0977b28ca0e07eb1a8a32fb5af060f2fc8cd3cdc
2020-11-06 15:25:06 -08:00
svcscm
e10909d5c2 Updating submodules
Summary:
GitHub commits:

e1e4001dd0

Reviewed By: bigfootjon

fbshipit-source-id: e869a0b5f7f9400973900ff95168c7a49efd2094
2020-11-06 13:27:51 -08:00
Stanislau Hlebik
e7fe9e4ea9 mononoke: do not allow too big commits to be landed
Reviewed By: krallin

Differential Revision: D24730106

fbshipit-source-id: e2b116c437a557f1b795c3b07eb124b8dd1a3ef8
2020-11-06 11:24:19 -08:00
svcscm
922d7eaa34 Updating submodules
Summary:
GitHub commits:

4794d4e1e3
48c67b1550
b82bdf5baf
4c2aef04bd

Reviewed By: bigfootjon

fbshipit-source-id: d508efb9963192ecae0a2584c5b57fb206d1defc
2020-11-06 11:24:19 -08:00
Thomas Orozco
447217c840 mononoke/filestore: update fetch to use 0.3 futures
Summary:
This updates the last bit of the Filestore that was using 0.1 futures to 0.3.
This used to use a weighted buffered stream (which we don't have for 0.3
futures at this point), but as I started working on one I realized we don't
even need it here, so I took this out.

Reviewed By: StanislavGlebik

Differential Revision: D24735907

fbshipit-source-id: 00a55c14864b09f9c353f95f2f8cbb895cf52791
2020-11-06 07:26:04 -08:00
Thomas Orozco
26e06ef1a0 mononoke/filestore: update fetch methods to return 0.3 stream
Summary:
This updates the external facing API of the filestore to use 0.3 streams.
Internally, there is still a bit of 0.3 streams, but as of this change, it's
all 0.3 outside.

This required a few changes here and there in places where it was simpler to
just update them to use 0.3 futures instead of `compat()`-ing everything.

Reviewed By: ikostia

Differential Revision: D24731298

fbshipit-source-id: 18a1dc58b27d129970a6aa2d0d23994d5c5de6aa
2020-11-06 07:26:04 -08:00
Thomas Orozco
8cad2ed3f2 mononoke/filestore: update exists() to futures 0.3
Summary: Like it says in the title.

Reviewed By: StanislavGlebik

Differential Revision: D24731300

fbshipit-source-id: b9c44fc1e4bd4cfe8655e1024a0547e40fb99424
2020-11-06 07:26:03 -08:00
Thomas Orozco
184310158b mononoke/filestore: update fetch external API to 0.3 futures
Summary:
Like it says in the title. This required quite a lot of changes at callsites,
as you'd expect.

Reviewed By: StanislavGlebik

Differential Revision: D24731299

fbshipit-source-id: e58447e88dcc3ba1ab3c951f87f7042e2b03eb2c
2020-11-06 07:26:03 -08:00
Thomas Orozco
b6949dbc26 mononoke/filestore: update store to futures 0.3
Summary: Like it says in the title. This updates `store()` and its (many) callsites.

Reviewed By: ahornby

Differential Revision: D24728658

fbshipit-source-id: 5fccf76d25e58eaf069f3f0cf5a31d2c397687ea
2020-11-06 07:26:03 -08:00
Thomas Orozco
a9b64b93c5 mononoke/filestore: update rechunk module to futures 0.3
Summary: Not much to be seen here.

Reviewed By: ahornby

Differential Revision: D24727257

fbshipit-source-id: d97b68544d53cfe32eb11caab73535d9be16ad11
2020-11-06 07:26:03 -08:00
Thomas Orozco
8e7bf512a5 mononoke/filestore: convert alias module to futures 0.3
Summary: Like it says in the title. Not much to be said here.

Reviewed By: ahornby

Differential Revision: D24727256

fbshipit-source-id: 1645339edf287ac7e59612589b308f08b708ae00
2020-11-06 07:26:03 -08:00
Thomas Orozco
40e8cab560 mononoke/filestore: update metadata to futures 0.3
Summary:
This updates the metadata APIs in the filestore to futures 0.3 & async / await.
This changes the external API of the filestore, so there's quite a bit of churn
outside of that module.

Reviewed By: markbt

Differential Revision: D24727255

fbshipit-source-id: 59833f185abd6ab9c609c6bcc22ca88ada6f1b42
2020-11-06 07:26:03 -08:00
Thomas Orozco
c2ad26f991 mononoke/filestore: convert finalize to futures 0.3
Summary:
Like it says in the title. This also lets us get rid of some macros we no
longer need.

Reviewed By: markbt

Differential Revision: D24727259

fbshipit-source-id: 5e3211bc08fa5376b4cfce4bea0428ab7bf3dc0f
2020-11-06 07:26:03 -08:00