Commit Graph

57961 Commits

Author SHA1 Message Date
Kostia Balytskyi
56eea5b6f6 commitcloud_backfiller: add reverse filler
Summary:
This diff finally introduce the continuous reverse filler. Specifically, this adds a cli (and underlying wiring) to operate the filler logic in the `reversefillerqueue` table.

To achieve this:
- the filler class is turned into a base class with two subclasses for the forward and reverse fillers
- the main file is renamed from `forwardfiller.py` into `filler.py`, to better reflect the independence of direction.

Reviewed By: krallin

Differential Revision: D21628259

fbshipit-source-id: 5676a162a62f0dc6fe80e6300b72d30370fc80b4
2020-05-26 07:00:12 -07:00
Stanislau Hlebik
7ccd28de14 scs_server: add track_history_across_deletions parameter
Reviewed By: krallin

Differential Revision: D21718738

fbshipit-source-id: b7dd7e05b1773ac1a442d89991549f0f97e1e55b
2020-05-26 05:38:55 -07:00
Alex Hornby
a7a4dcd046 mononoke: add devdb support to integration test runner
Summary: Add devdb support to integration test runner so that one can more easily inspect mysql test data, also makes it easier to run tests with locally modified schema.

Reviewed By: HarveyHunt

Differential Revision: D21645234

fbshipit-source-id: ec75d70ef59f04548c7346a122298567dd09c264
2020-05-26 02:36:12 -07:00
svcscm
3173bb25c8 Updating submodules
Summary:
GitHub commits:

e69b649453
7d97a671aa
bc44b0f07d
545f4fc1c1
85f9aafb64
5267abf80e
df2f1779db

Reviewed By: wittgenst

fbshipit-source-id: dc3b4dcf07c36545428356018ccd2ab769d0843e
2020-05-26 02:36:12 -07:00
svcscm
cc12206e49 Updating submodules
Summary:
GitHub commits:

213c4ed5b4
1d0b0d98cd
9043fd7559
d65ffba71e
a395ae3d31
bd68bfb41b
f187cd3ec0

Reviewed By: wittgenst

fbshipit-source-id: a122bc5dbcc57e018e2daf1ac19f53859a3424f4
2020-05-26 00:29:16 -07:00
Yedidya Feldblum
ed2c4a65ba Add missing includes of folly/synchronization/Baton.h
Summary: Add missing includes of `folly/synchronization/Baton.h`.

Differential Revision: D21716396

fbshipit-source-id: 8c8fb818f3083b4fe04fd1c62923d7ba698dcedd
2020-05-25 23:13:35 -07:00
svcscm
06f32fa94e Updating submodules
Summary:
GitHub commits:

71dbf454c9
7e4f99fc6f

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 45c5a4ea86e82eed15ebe79ac542b3a90183f4b9
2020-05-25 22:23:39 -07:00
svcscm
ab13dfc04a Updating submodules
Summary:
GitHub commits:

3139f52cb0
b408eaadae
4091de5e08

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: f14e0475ad4dc0e5555f0fbe37f5b36339cceef1
2020-05-25 07:24:32 -07:00
svcscm
f86cff6e22 Updating submodules
Summary:
GitHub commits:

9db970345d

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 48d989adf41319527b4480fbea081e2079b3cf14
2020-05-24 14:23:17 -07:00
svcscm
47a8bd05d7 Updating submodules
Summary:
GitHub commits:

ac5666bd7d
d5ce6e820b
558c49789f

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: a016ba866d2996169de881377af59ccf85cecb13
2020-05-23 15:33:15 -07:00
svcscm
d2484b4bb3 Updating submodules
Summary:
GitHub commits:

44ff9b5003

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: b8a6d0afa64ccdbb45362f7d61899d5cb0142e31
2020-05-23 01:23:54 -07:00
svcscm
6a871bb4d8 Updating submodules
Summary:
GitHub commits:

070df52dd0
c4f10fe40a
7945e61cf0
93d952a945
955e1bf5d0
f23ada231b
9504676f7f
78c6f87ce3
981e8f7255
058217dd8b
cfad3e375d

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 98d0b40dac46330a72d99038a6639b943f212e7b
2020-05-22 01:06:29 -07:00
Wez Furlong
72111106fd getdeps: use eden prefetch on windows
Summary:
Now that we've deployed the new eden build with globfiles
support, we can enable the use of eden prefetch in the getdeps
build when running inside an EdenFS mount on Windows.

Reviewed By: fanzeyi

Differential Revision: D21692689

fbshipit-source-id: b42e778901976cf0385ec31056c227b2049162dc
2020-05-21 22:55:34 -07:00
Sukwon Chung
abaa62ad33 Handle inaccessible files in redirections in eden du
Summary:
1. Add directory existence checking logic before checking "legacy_bind_mounts_dir"
2. Still print the disk consumption returned by the du command when it fails with an error invoking du -skxc

Reviewed By: chadaustin

Differential Revision: D21683421

fbshipit-source-id: 531b6b289e9ffa445ed691611c9cbf22f393e742
2020-05-21 22:20:17 -07:00
svcscm
1aeff22dfc Updating submodules
Summary:
GitHub commits:

be6ee4d528
4b5b7d787f
6a97f85bcb
fb48f55139
0ef92d81db

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: de7acb38ade33a05bf4f52dc1f24373745f11f57
2020-05-21 22:20:16 -07:00
svcscm
9e3bd69e01 Updating submodules
Summary:
GitHub commits:

0bc8ef0d72
647364858e
28c893e596
d74f1d479e
91f616e4cf
f6e3f7d3d2
bbfcf1717b
122f3d90ff

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 36b22e15ac2988d6cc8c68cf54ee4cd2da4b5abc
2020-05-21 20:50:29 -07:00
Stefan Filip
a2254714c3 mononoke: update bulkops::fetch_all_public_changesets to return commits in order
Summary:
At first glance people will assume that changesets are returned in the same
order that they were added in the database or that at least commits are
returned in a deterministic fashion. That didn't happen because the both
changeset ids and changeset entries were received without any order.
This diff updates the function to returns results in order they were added
to the database.

Reviewed By: krallin

Differential Revision: D21676663

fbshipit-source-id: 912e6bea0532796b1d8e44e47d832c0420d97bc1
2020-05-21 20:43:45 -07:00
generatedunixname89002005307016
f9358e566a suppress errors in eden - batch 1
Summary:
This diff is auto-generated to upgrade the Pyre version and suppress errors in eden. The upgrade will affect Pyre local configurations in the following directories:
```
eden
```

Differential Revision: D21687853

fbshipit-source-id: baf0d9bc33f86da63ea289690faca6cf4d566588
2020-05-21 19:32:35 -07:00
svcscm
de0d51cb86 Updating submodules
Summary:
GitHub commits:

811ce17b86

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 8b0d8d59cbea551448cab9442667f23822aa464d
2020-05-21 19:32:35 -07:00
svcscm
816ae7971c Updating submodules
Summary:
GitHub commits:

732d3ffcae

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 40fabfa4b326e193419780e2dbf4276da4ec58c8
2020-05-21 17:49:36 -07:00
Stefan Filip
cafbbcd9c8 mononoke: add MemIdMap
Summary:
This structure has similar functionality to the IdMap that is backed by SQL.
It is probably going to be useful for caching in the case of batch operations.

Reviewed By: quark-zju

Differential Revision: D21601820

fbshipit-source-id: 9c3ebc3e9dc92a59ce0908fc241bd2b97da88dca
2020-05-21 17:19:16 -07:00
Stefan Filip
b9a93d49e5 mononoke: bulk construction of the segmented changelog Dag
Summary:
`Dag::build_all_graph` will load the whole graph for a given repository and
construct the segmented changelog from it.

Reviewed By: StanislavGlebik

Differential Revision: D21538029

fbshipit-source-id: b4ba846bb2870ba73257bed6128b8e198a0aab3e
2020-05-21 17:19:15 -07:00
svcscm
8ffad53b14 Updating submodules
Summary:
GitHub commits:

7c832f1dbb
791aa79790
263ece6907
826295a5e9
eaaf2c2adf
e52980a9ee

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 88c8e28d2e31ba3ca58cf946b60adcb4e8f631c7
2020-05-21 17:01:30 -07:00
Zeyi (Rice) Fan
9e87d4da4b check if the file exist locally before pushing to the request queue
Summary: This function won't go ask remote server and is able to produce a result very quick. This avoids the waiting time in the queue.

Reviewed By: chadaustin

Differential Revision: D21586511

fbshipit-source-id: db3793cc4e4e493d1beb6837c0bd91358e7e845f
2020-05-21 16:32:42 -07:00
Zeyi (Rice) Fan
8b49918ce1 add a fake remotestore to enable memcache
Summary:
xavierd and I realized that memcache is not actually enabled in EdenFS as `ContentStoreBuilder` only sets it when there is are remote store. We believe turning on memcache store can help us improve some performance for importing blobs (80% hit rate).

This diff adds a noop remote store that does nothing to trick `ContentStoreBuilder` to include the memcache store. We decided to go this way instead of modifying `ContentStoreBuilder` is because EdenFS is going to have a real remotestore very soon, and changes to `ContentStoreBuilder` may have some impact on how Mercurial uses it.

Reviewed By: xavierd

Differential Revision: D21621234

fbshipit-source-id: 5502e001ab498134b6a927d83be823261e70e9e1
2020-05-21 16:32:42 -07:00
Zeyi (Rice) Fan
d2facf6118 always use hgcache for data fetching
Summary: Reading from hgcache directly has been rolled out for quite a while now. Let's remove the configuration and make it default behavior.

Reviewed By: chadaustin

Differential Revision: D21585482

fbshipit-source-id: 7f30e262642adf58388a0580aa5a63595fa89155
2020-05-21 16:32:41 -07:00
svcscm
ad8c1956a3 Updating submodules
Summary:
GitHub commits:

750fa0dede
8174190bdf
9614d4db5b
5946fb89f8
292bcf6227
da3eb41a78

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 552536eae60e0a95a0a10cb8b06a324da6bda1b2
2020-05-21 15:03:31 -07:00
Chad Austin
e687d9426c fix oss linux build
Summary:
gcc and clang disagree here, so explicitly construct a Future from the
unique_ptr.

Reviewed By: genevievehelsel

Differential Revision: D21687654

fbshipit-source-id: 338174f5d4d051f509faace6e48d1c0fb4d898e1
2020-05-21 12:04:45 -07:00
Durham Goode
1d770d8e17 config: fix dynamic config overriding non-subset final value
Summary:
There was a bug where if the dynamicconfig chose a value that matched
one of the ported rc files, but it did not match the final value of that config
(from a not-yet-ported rc file), it would chose the dynamicconfig value instead
of the original final value. Let's drop the dynamicconfig value in this case as
well.

Reviewed By: quark-zju

Differential Revision: D21674469

fbshipit-source-id: efcd36e9602e16210999ec8c88e86b1d7ee355e4
2020-05-21 10:41:11 -07:00
Genevieve Helsel
f799ca8981 suggest graceful restart in eden doctor version check
Summary: I've updated most messages to suggest a graceful restart, but I overlooked this instance and we got a user question about this. So lets update this message here as well.

Reviewed By: chadaustin

Differential Revision: D21681035

fbshipit-source-id: 32bf5e75b9ad1d98bcd70427101aabeee0be5046
2020-05-21 09:38:58 -07:00
Mateusz Kwapich
d5e4d169f3 tests: fix tweakdefault tests
Summary: I didn't update it on my last diff.

Reviewed By: xavierd

Differential Revision: D21687344

fbshipit-source-id: 01c9662b9fd8c4670c2af34a999c69e6b93ed7f7
2020-05-21 08:44:39 -07:00
Mateusz Kwapich
611edcf4d7 histedit: improve compatibility with mutation&visibility
Summary:
When markbt converted histedit to be compatible with new style mutation
he missed a few spots.

This diffs adds a large test checking the that all the scenarios that were
supported with obsmarkers are supported with mutation now.

Reviewed By: markbt

Differential Revision: D21665484

fbshipit-source-id: a8a289910adb068a382e32f9d106dedc1573d413
2020-05-21 04:25:06 -07:00
Mateusz Kwapich
6a1760ea32 amend: on --to provide the correct argument to histedit
Summary:
The histedit goes down to the latest branching point by default which is not
always the right choice for `amend --to`

Reviewed By: markbt

Differential Revision: D21665485

fbshipit-source-id: d981d2216b5384f61c0082684eafa1e84eaeada8
2020-05-21 04:25:05 -07:00
Mateusz Kwapich
ba176f6937 amend: show the bug in amend --to
Summary:
This test addtion showcases a bug in `amend --to` when
a side branch is present

Reviewed By: markbt

Differential Revision: D21665486

fbshipit-source-id: 8c21e7a8cccfc6c313d3b7aa09be8309a662b9cc
2020-05-21 04:25:05 -07:00
Kostia Balytskyi
471a6a71f6 clone: increase timeout to 4 hours
Reviewed By: krallin

Differential Revision: D21684250

fbshipit-source-id: 703adb039f839f5c04b592f3ae20eaf76c3f876e
2020-05-21 04:19:13 -07:00
Alex Hornby
1cf88010c2 mononoke: walker: switch to DashMap for walk state
Summary:
Change from CHashMap to DashMap for the walk state tracking.

DashMap is using slightly less memory in testing, and is slightly quicker in walk rate (number of graph nodes processed per second ).

Reviewed By: HarveyHunt

Differential Revision: D21662210

fbshipit-source-id: ea0601df17e0e596fd59b67d9d01d0dc4e90799b
2020-05-21 04:08:08 -07:00
Harvey Hunt
a2111e06e3 mononoke: Add cache hit and miss statistics to CacheBlob
Summary:
CacheBlob logs hit and miss stastics to Scuba. Let's add the same for
ODS.

Reviewed By: krallin

Differential Revision: D21640922

fbshipit-source-id: 8f7d17f048bf53bdc6cd8bda0384a51cae7b6a30
2020-05-21 03:19:12 -07:00
Harvey Hunt
0ee0131454 mononoke: Remove hit and miss metrics from CountedBlobstore
Summary:
CountedBlobstore is a Blobstore that wraps another blobstore in order
to report metrics about it, such as the number of gets or errors. It's commonly
used to wrap a CacheBlob blobstore, which itself is a caching wrapper over an
inner blobstore.

CountedBlobstore exposes metrics that are supposed to track the number of hits
or misses when fetching blobs. However, these metrics don't make sense as the
CountedBlobstore has no view into cache activity. These metrics actually report
the number of requests and the number of missing blobs rather than hits and
misses.

Remove these misleading counters.

Reviewed By: krallin

Differential Revision: D21640923

fbshipit-source-id: 07b9fc9864c70991415c2b84f35d631b702c17d1
2020-05-21 03:19:11 -07:00
Stanislau Hlebik
a5eaa551e8 mononoke: support history across deletions
Summary:
now fastlog will be able to show a history of file like that:

```
Added
|
Deleted
|
Added
```

We do it by using deleted file manifest to find where a file was deleted. We do
it in case we find a unode which doesn't have a parent.

Differential Revision: D21664248

fbshipit-source-id: 8e80fccc87361e66e4e00ae2da3671ed8327f1f8
2020-05-21 02:46:21 -07:00
Stanislau Hlebik
090d543bf7 mononoke: avoid one special case in fastlog
Summary:
... or rather make this special case a bit smaller.

In fastlog we have a `prefetch` which was already returned to the caller,
however its parents hasn't been processed yet. Previously we had two separate
places where these parents were processed, now we have only one.

This will make it easier to add support for history across deletions in the
next diff.

Differential Revision: D21664246

fbshipit-source-id: a5a109fafaeec28a0e7a36ffa44545aebc032b00
2020-05-21 02:46:21 -07:00
svcscm
e2c443e41d Updating submodules
Summary:
GitHub commits:

ca792c0142
28929e8245

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 09ae1a9da72243f761f40f951037c432f8a31de3
2020-05-21 02:46:20 -07:00
svcscm
42c96bf16c Updating submodules
Summary:
GitHub commits:

909926d1ee
8abb78f423
ac53e737cf
0b892bcbfb
eb04bb86c6
0c2c715235
5c5e7ad98c
450e1aaae6
60e318d48d
b4b0ff439e

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 11bfde5db57254d449c0d5fb4cea1a895432989c
2020-05-20 19:30:10 -07:00
Durham Goode
0ef51f2f5c config: fix test-debugdynamicconfig.t on Windows
Summary: I had hardcoded the path separator, which is wrong on Windows.

Reviewed By: singhsrb

Differential Revision: D21679423

fbshipit-source-id: 309c84b0698355654baee1dae7f201310e695c49
2020-05-20 18:48:58 -07:00
Jeremy Fitzhardinge
04c7c17185 mononoke: remove unused deps for a couple of targets
Summary:
Just top-level mononoke and cache_warmup as a more or less random
selection.

Reviewed By: StanislavGlebik

Differential Revision: D21648852

fbshipit-source-id: 0e5137da9a7742377112e085e9d29182abdfbf41
2020-05-20 17:00:08 -07:00
Chad Austin
b653d9cc86 fix gcc compilation issues
Summary: Fix some RVO warnings and a compilation error when compiling with gcc.

Reviewed By: xavierd

Differential Revision: D21481738

fbshipit-source-id: 0621f5886df40c24ef1a6a68ccd957e38f2f4122
2020-05-20 15:49:02 -07:00
Xavier Deguillard
ee81495b3d revisionstore: handle empty proxy
Summary:
If either the config, or the environment variable is empty, the URL parsing
would raise an error, while the intent is to not have proxy. Let's handle it
properly.

Reviewed By: DurhamG

Differential Revision: D21669922

fbshipit-source-id: 84c8d09242ac8d5571f7592d874f4954692110f5
2020-05-20 14:03:50 -07:00
Durham Goode
3c8eb75a26 tests: fix test-debugdynamicconfig.t on Windows
Summary:
Looks like echo prints a \r\n which messes up the output. Let's use
printf to force it to use \n

Reviewed By: xavierd

Differential Revision: D21670125

fbshipit-source-id: f991c9c9fb55720bb68a3f71e8ddd3b4f16b5375
2020-05-20 14:00:54 -07:00
Durham Goode
98d428929b config: fix config validation at clone time
Summary:
At clone time we apply dynamic configs in memory, instead of loading
them from disk. The validation logic operated on the config value's location
field, which isn't set for data that comes from in memory. So we need to update
the validation logic to also consider the value source if location is not set.

Reviewed By: quark-zju

Differential Revision: D21664205

fbshipit-source-id: 8460c58c6d654780048de51ada8178c70ff0a9e6
2020-05-20 13:35:28 -07:00
Durham Goode
75c96be5ff configs: don't write dynamic config if it hasn't changed
Summary:
We kick off a background process every 15 minutes or so to compute the
new dynamicconfig. If the config hasn't changed, we should just touch the file
instead of rewriting the whole thing.

Reviewed By: quark-zju

Differential Revision: D21653098

fbshipit-source-id: 9d53d2a636cff082cd048994255cc809ce1b0221
2020-05-20 13:35:28 -07:00
Durham Goode
9f6f200a08 configs: version dynamic configs
Summary:
If we release a new version of Mercurial, we want to ensure that it's
builtin configs are used immediately. To do so, let's write a version number
into the generated config file, and if the version number doesn't match, we
force a synchronous regeneration of the config file.

For now, if regeneration fails, we just log it. In the future we'll probably
throw an exception and block the user since we want to ensure people are running
with modern configuration.

Reviewed By: quark-zju

Differential Revision: D21651317

fbshipit-source-id: 3edbaf6777f4ca2363d8617fad03c21204b468a2
2020-05-20 13:35:28 -07:00