Commit Graph

57912 Commits

Author SHA1 Message Date
Durham Goode
9f6f200a08 configs: version dynamic configs
Summary:
If we release a new version of Mercurial, we want to ensure that it's
builtin configs are used immediately. To do so, let's write a version number
into the generated config file, and if the version number doesn't match, we
force a synchronous regeneration of the config file.

For now, if regeneration fails, we just log it. In the future we'll probably
throw an exception and block the user since we want to ensure people are running
with modern configuration.

Reviewed By: quark-zju

Differential Revision: D21651317

fbshipit-source-id: 3edbaf6777f4ca2363d8617fad03c21204b468a2
2020-05-20 13:35:28 -07:00
Xavier Deguillard
cd92c10363 integration: symlinks are not supported on Windows
Summary:
Now that regular users can create symlinks, a handful of tests were failing
due to the symlink call no longer raising an exception. For now, let's always
consider that symlinks aren't supported on Windows.

Reviewed By: fanzeyi

Differential Revision: D21664213

fbshipit-source-id: c55a99d1cb92e68b9861701b5517b1d5db2d40c6
2020-05-20 13:11:10 -07:00
Stanislau Hlebik
8505bf3772 mononoke: move functionality to fetch linknodes in a separate function
Summary: it will be used in the next diffs in the stack

Differential Revision: D21664247

fbshipit-source-id: 6ebdba72b1e8382940a91f5fbec414e7bcc8160b
2020-05-20 12:26:16 -07:00
Stanislau Hlebik
ca44297680 fastlog: create visited one level up
Summary:
Previously we created `visited` and immediately drop it.
Let's instead create it one level up. I don't think it'd matter in practice,
but it should be cleaner

Reviewed By: HarveyHunt

Differential Revision: D21662456

fbshipit-source-id: 36823309ea09448c5be10c2655ea0f317649f290
2020-05-20 12:26:16 -07:00
svcscm
685727181b Updating submodules
Summary:
GitHub commits:

cb71aa00fb

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 69c3c96e73bfdd338d1030f36e94805d8040ab6a
2020-05-20 11:29:22 -07:00
Stanislau Hlebik
d09e107e1d mononoke: add merge commit validation
Summary:
Follow up from D21596758 - current logic of pushrebasing a merge is very
complicated. To prevent repo corruptions let's do a very simple validation -
generate hg changeset from a rebased bonsai changeset.

Given that generating hg changeset is an expensive operation let's do it only
after the first rebase attempt - otherwise we might risk constantly losing the
pushrebase race.

Differential Revision: D21659187

fbshipit-source-id: f43a855cf0fbdbd11a40d3ec38283af344cde5e6
2020-05-20 11:17:21 -07:00
Mark Thomas
c1e91b47fc cmdutil: remove graph-renderer hint
Summary: This hint has served its purpose.  Remove it.

Reviewed By: farnz

Differential Revision: D21658927

fbshipit-source-id: 0395fa2cb898732b2c64154104c703a428f62864
2020-05-20 08:06:21 -07:00
Kostia Balytskyi
a174858239 reversefillerqueue: change bundle from varchar to varbinary
Summary: This is to bring it into sync with the `forwardfillerqueue` types.

Reviewed By: markbt

Differential Revision: D21660012

fbshipit-source-id: 5148023478c175cd49707d88251701a08fcbe0ce
2020-05-20 07:34:22 -07:00
Harvey Hunt
46d2b44c0e mononoke: Use MononokeApp for mononoke server
Summary:
Mononoke server is the only binary using plain `clap::App`, whilst all
other services use `MononokeApp`. The `MononokeApp` variant configures some
default arguments and provides consistent argument naming between binaries.

Update the server to use `MononokeApp` and add some logic to allow passing
either `--config_path` or `--mononoke-config-path` until we remove all uses of
`--config-path`.

Reviewed By: StanislavGlebik

Differential Revision: D21642461

fbshipit-source-id: f42e0b6625f3979ced0920db269bdb4528f99e49
2020-05-20 05:47:17 -07:00
Mark Thomas
5ae4554539 server: log when the server starts to listen for connections
Summary:
When starting up with many repos, it can take a while before the server is ready
to accept connections, and there is no log to indicate when that is the case.

Add one.

Reviewed By: farnz

Differential Revision: D21641189

fbshipit-source-id: 057ca06c1b1725799a8445841b6eea89bba783b0
2020-05-20 03:56:49 -07:00
Mark Thomas
85d00edafa tunables: only log tunables when they change
Summary:
Rather than logging `Refreshing tunables` all the time, log only when they
change, and include the values that they are changing from and to.

Reviewed By: farnz

Differential Revision: D21641191

fbshipit-source-id: b0b5edd687e2e41074ce1c8be13fd6918dff924d
2020-05-20 03:56:49 -07:00
Alex Hornby
83dddd4c6f mononoke: fix test-blobstore_healer.t log expectations with replication enabled
Summary: Fix test log expections when replication is enabled.

Reviewed By: ikostia

Differential Revision: D21644474

fbshipit-source-id: fe1968994da427e2810be1bdea8fa56387d3f00f
2020-05-20 03:20:33 -07:00
Simon Farnsworth
ef23e0b853 Record entry to resolve, so that we can identify quickly when we have too few successes.
Summary:
There are a few paths through `resolve` that return early with a failure, and thus never record what happened.

Make a record the moment we enter `resolve` - then, we can use `count` type ODS charts to determine the failure rate deterministically, and alarm if the failure rate is too high

Reviewed By: ahornby

Differential Revision: D21647575

fbshipit-source-id: 667787ec000a8cd8e715563df10dbb84832fefa1
2020-05-20 02:58:55 -07:00
svcscm
f7ba3510bd Updating submodules
Summary:
GitHub commits:

54bccd495b

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: 5009d574a8eff12fa080b9b78a6de9b7a689aad9
2020-05-20 02:58:55 -07:00
svcscm
14375cd837 Updating submodules
Summary:
GitHub commits:

45f233c178
a1523efcdf
b18ffd1b6a

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: e5f53bc48c15ce2b71febc068a3df6a93615252f
2020-05-19 20:27:53 -07:00
Durham Goode
f0d7044aff configs: apply dynamicconfig during clone
Summary:
During clone the hgrc.dynamic file doesn't exist and doesn't even have
a place for us to generate it to. Let's instead generate and apply the config in
memory.

In the future, if generate fetches data from the network, this will mean clone
would depend on the network, since if generate fails the clone would fail. In
some situations this is desirable, since users shouldn't be cloning without our
approved configs, but if it causes problems we could probably tweak generate to
support an offline mode.

Reviewed By: quark-zju

Differential Revision: D21643086

fbshipit-source-id: d9a758207738d5983213d95725061517e0aa17db
2020-05-19 19:51:27 -07:00
Jun Wu
428cfc7b6b tests: update test-extensions.t
Summary:
The Rust version command does not support `-T` and does not read
`__version__.py`. Remove tests about them. As we're here, also remove tests
about 3rd-party extension version checking as we bundle all extensions.

Reviewed By: xavierd

Differential Revision: D21648591

fbshipit-source-id: 97c9e2ff3d224ff12b34801a1af07532d9e9bff5
2020-05-19 19:09:56 -07:00
Meyer Jacobs
341fbdc1aa debugging: Implement "debugdetectissues" command for detecting signs of repository issues
Summary:
Implements a "debugdetectissues" command, which runs a series of checks on the repository meant to detect potential issues, logging the data to Scuba so that we can determine how common certain issues are, and under what conditions they appear.

Future extensions of this change may involve merging the functionality into hg doctor, and setting the command to run automatically in the background on some interval.

Reviewed By: DurhamG

Differential Revision: D21558170

fbshipit-source-id: a878ae1804d5f11c83a574e0dc3c802b564d2ced
2020-05-19 18:11:29 -07:00
svcscm
e31a7f7d61 Updating submodules
Summary:
GitHub commits:

f425a3aa28
14182bff3b
40cf9cf678
b964ccc5ed
301ce66128
13323915fa
55629dc11a

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: f0e856e7298952190a541a855d52766020389474
2020-05-19 17:23:34 -07:00
Stefan Filip
60966c93e7 autocargo: regenerate
Summary: maintenance

Reviewed By: StanislavGlebik

Differential Revision: D21640322

fbshipit-source-id: d0b2ce604735c05d540d06835c8e4c8a940fbf5c
2020-05-19 16:08:40 -07:00
svcscm
cfe17e9d73 Updating submodules
Summary:
GitHub commits:

2d2c15fcbc
41e40d7ba8
39b24432d4
345dcbca42
34901ee92f

Reviewed By: 2d2d2d2d2d

fbshipit-source-id: d1707c5d41cb21f65233616a39466f98b757cb76
2020-05-19 15:03:57 -07:00
Durham Goode
861f813f25 configs: convert facebook_overrides.rc
Summary: Converts facebook_overrides.rs to our dynamic config generator

Reviewed By: quark-zju

Differential Revision: D21625721

fbshipit-source-id: 2a374939d90f1fb7f9173268e2a7fa636d672393
2020-05-19 13:23:19 -07:00
Xavier Deguillard
6cdc2782b3 cli: add a startservice.ps1 script
Summary:
This script deals with obtaining the right privilege to start the EdenFS
service. This effectively enables `edenfsctl start`.

Reviewed By: wez

Differential Revision: D21585739

fbshipit-source-id: 7b835434d865fa4c4c8473e13665ae669fd86108
2020-05-19 13:00:54 -07:00
Chad Austin
c2f85710d3 build with Rust datapack support on all platforms
Summary: Now that Rust libraries can be linked into fbcode binaries with mode/mac, enable Rust datapack on all platforms.

Reviewed By: xavierd

Differential Revision: D21241075

fbshipit-source-id: ae2abee25a5ad7b9db9ac5e6b8687c5f79376926
2020-05-19 11:53:51 -07:00
Jun Wu
e685e64758 remotefilelog: skip uploadblobs for public revs
Summary:
There is no need to upload content referred by a public commit.

This affects cases like `debugstrip` (ex. for testing `pull` performance with a
lagged commit graph). Without this change, the uploadrevs code path scans
stripped public commits without efficient tree prefetching, which results in
1-by-1 tree fetches and is unusably slow.

Reviewed By: xavierd

Differential Revision: D21630096

fbshipit-source-id: 385edf76cb4eb913b2d64422910cdb46b603e6c0
2020-05-19 10:25:57 -07:00
Mark Thomas
513a4f8426 color: don't disable colors if HGPLAINEXCEPT=color
Summary:
With `HGPLAINEXCEPT=color`, colors should still be enabled if the terminal is
capable of supporting them and output is to the terminal.

Currently this doesn't work if `--color=auto` (the default), as
`color._modesetup` uses `ui.formatted` to check if the output is a terminal,
and thus has colors available, but this is `False` for all `HGPLAIN` modes.

Instead, add a new method to `ui` that checks whether  `fout` is a terminal, and
use that for color autodetection.

This function also allows us to add `HGPLAINEXCEPT=pager` as we can use
that in the same way.

Reviewed By: farnz

Differential Revision: D21617170

fbshipit-source-id: 7ee4eaa8963f3d6eb7ed8044a678a4804b9a98f0
2020-05-19 06:13:54 -07:00
Mark Thomas
9e3a321235 windows: fix test-pager.t
Summary:
Setting the fake pager in the pager test doesn't work on Windows.

For some reason, the `pager.pager=C:/path/to/python.exe C:/path/to/test/fakepager.py` triggers
some kind of PATH translation code that results in `setconfig.py` receiving
`pager.pager=C;C:\\path\\to\\python.exe C;C:\\path\\to\\test\\fakepager.py` as its
argument.  This is clearly invalid, so the test fails with messages like
`'C' is not recognized as an internal or external command`.

Workaround this for now by setting the pager by appending directly to HGRCPATH.

Reviewed By: ikostia

Differential Revision: D21638507

fbshipit-source-id: e8a6b0c281030b23302116a79ef3ba754d37f601
2020-05-19 06:13:53 -07:00
Harvey Hunt
45714159c5 mononoke: Fix sampling of scrub_get
Summary:
D21573455 accidentally removed the scuba sampling for scrub_get
operations. Add this back in.

Reviewed By: StanislavGlebik

Differential Revision: D21638972

fbshipit-source-id: eee66dbce161de69246f4da0a15dc2cf00e1ba01
2020-05-19 05:44:43 -07:00
Stanislau Hlebik
e30a12ce58 mononoke: remove getfiles from traffic replay
Reviewed By: farnz

Differential Revision: D21622533

fbshipit-source-id: 3225e287df42c1bac8ad8f67cdb05ec33f27dfdd
2020-05-19 04:43:01 -07:00
Stanislau Hlebik
37437ebe60 mononoke: remove getfiles wireproto parsing
Reviewed By: farnz

Differential Revision: D21623155

fbshipit-source-id: b1f763b653c47c42bc1d765cfa8985a767a63652
2020-05-19 04:43:00 -07:00
Stanislau Hlebik
a5dcef86b1 mononoke: remove getfiles serialization
Summary: No longer needed!

Reviewed By: farnz

Differential Revision: D21623157

fbshipit-source-id: 8d033a61b8dd4c3ec990928e2921f9b46cbbdb94
2020-05-19 04:43:00 -07:00
Stanislau Hlebik
daa1162e08 mononoke: remove getfiles from ratelimiter and perf counter
Summary:
See bottom diff in the stack for the motivation. Though you can probably guess
the motivation :)

Reviewed By: farnz

Differential Revision: D21623154

fbshipit-source-id: a0940d766a67080ddcb346c2e3313eb08699edad
2020-05-19 04:43:00 -07:00
Stanislau Hlebik
aadecbf087 mononoke: remove getfiles from repo_client
Summary: First diff in the stack that removes getfiles since it's no longer needed.

Reviewed By: farnz

Differential Revision: D21623156

fbshipit-source-id: 44f310ec4e4f34845cc5bf1738f1a8ece14e6694
2020-05-19 02:02:23 -07:00
svcscm
c44cc1e6e4 Updating submodules
Summary:
GitHub commits:

bdd6a3a5cc

Reviewed By: wittgenst

fbshipit-source-id: 810add3221725dc7e0f601409ff5bd7bb8a3acfb
2020-05-19 02:02:23 -07:00
svcscm
28ff9b83a5 Updating submodules
Summary:
GitHub commits:

509ab86b8a
24b8d3e01c
ada700b906

Reviewed By: wittgenst

fbshipit-source-id: 053c53b2a5afe3b09b9982da63358b292a88e8c8
2020-05-18 19:26:55 -07:00
Jun Wu
88a9982eb6 hgcommands: make version a native command
Reviewed By: DurhamG

Differential Revision: D19803761

fbshipit-source-id: 32f77cc667375e537e1ff70316329251359ae6ed
2020-05-18 18:50:42 -07:00
Jun Wu
c2869d6ca9 remotefilelog: call setuprepo unconditionally in clone_shallow -> pull_shallow
Summary:
Before this change, pull_shallow only calls setuprepo if the remotefilelog
requirement is not in repo. With D21011401, the remotefilelog requirement
will be added by clone.py and pull_shallow can skip calling setuprepo, causing
the pull code paths to write file logs (and fail).

Change the pull_shallow to always call `setuprepo` to solve the issue.

The final fix is probably moving more remotefilelog related clone logic to core.
Right now I just did the minimal change to fix things.

Reviewed By: sfilipco

Differential Revision: D21632429

fbshipit-source-id: 17775ac0df18cda10247419b40f9c27436b22606
2020-05-18 17:52:34 -07:00
svcscm
acfa7f4291 Updating submodules
Summary:
GitHub commits:

e05a737c18
d3b7867ab4
30bff94e85

Reviewed By: wittgenst

fbshipit-source-id: 7c89711e394c8cc9ee4ee934e1bf2af7fcbb705a
2020-05-18 17:52:34 -07:00
svcscm
8cb4f39bc5 Updating submodules
Summary:
GitHub commits:

d67d568565
6472821406
10a3d74cc4
10957464df
b2ab01c578
06943d59da
218b463647

Reviewed By: wittgenst

fbshipit-source-id: af0d8f75f6ee6ac9e67812fcd6206070f74f3f49
2020-05-18 15:02:08 -07:00
Xavier Deguillard
97a90e7de9 revisionstore: add doc comment
Summary:
We didn't have a high level overview of the types and traits and their
purposes, add that.

Reviewed By: DurhamG

Differential Revision: D21599759

fbshipit-source-id: 18e8d23ed00134f9d662778eddaee4a9451f7c2c
2020-05-18 12:11:52 -07:00
svcscm
a9734869fe Updating submodules
Summary:
GitHub commits:

08763d6583
b2c842281f
474a2bde70
f732967c40
b9d65f5aa6
64c5a2a73e
49f42386a9

Reviewed By: wittgenst

fbshipit-source-id: 1cb968b6411cb414b0ce38ff6ba4a7d5bf4b28cf
2020-05-18 11:24:37 -07:00
Genevieve Helsel
c8402345af redirect PrivHelper stdin to /dev/null to fully daemonize
Summary: EdenFS doesnt daemonize correctly due to the privhelper not closing fd 0 (see http://www.faqs.org/faqs/unix-faq/programmer/faq/). This redirects stdin to /dev/null/ in order to do so.

Reviewed By: xavierd

Differential Revision: D21602545

fbshipit-source-id: 0aeb589efbf214ef22c0db039fbb6a436a71e360
2020-05-18 10:11:18 -07:00
Katie Mancini
76204197c0 fix use of service identity in scs CAT authentication
Summary:
This updates a placeholder in the authentication process for requests
in the rust thrift services to allow authentication with CATs. This change
allows for passing in a configurable server identity to be used as the
verifier identity here.

Additionally this sets the verifer identity to be "scm_service_identity"
for the mononoke scs server (the service identity for this service).

This is needed to use the mononoke scs server from corp.

Reviewed By: markbt

Differential Revision: D21560204

fbshipit-source-id: 02e41718307fddab6f0ac2101306bb7f4a59cf7e
2020-05-18 10:07:43 -07:00
Jun Wu
2aa85dc4d7 version: a Rust crate providing version information
Summary: Otherwise the version information is only available in Python.

Reviewed By: DurhamG

Differential Revision: D19803762

fbshipit-source-id: 044c5da86efc8c657d0c422a2b1947086444895e
2020-05-18 09:00:40 -07:00
Harvey Hunt
4629da9f0a mononoke: Fix potential race in blobstore write_order logging
Summary:
Currently, we have a gap between updating and subsequently reading the
value of `write_order`. If another blobstore's put operation completes before
we have called `record_put_stats` then we may have an incorrect `write_order`
value.

Whilst this is only a minor issue, let's fix it anyway. :-)

Reviewed By: krallin

Differential Revision: D21619669

fbshipit-source-id: 1b8bacbcb4c195e6765ebdfaa68425f286f88c06
2020-05-18 08:41:25 -07:00
Alex Hornby
7296456a39 mononoke: walker: use array of AtomicUsize for visit_count
Summary:
After other optimizations the CHashMap version of visit_count was showing as a hot spot.

Given the number of possible NodeTypes is small we can store the visit_count in array indexed by ordinal instead.

Reviewed By: farnz

Differential Revision: D21618518

fbshipit-source-id: 84978778034df11a9a48452adf9269db2dc17145
2020-05-18 08:17:00 -07:00
Stanislau Hlebik
54c036e8b2 mononoke: record git mapping while doing a push
Summary:
Currently we record them only during pushrebase. Let's record during push as
well.

To simplify things a little bit let's allow only a very simple push case:
1) Single bookmark.
2) All pushed commits should be reachable by this bookmark.

Reviewed By: krallin

Differential Revision: D21451337

fbshipit-source-id: bf2f1e6025ac116fb8096824b7c4c6440d073874
2020-05-18 05:41:41 -07:00
Stanislau Hlebik
108b468af3 mononoke: fix unnecessary bonsai entries when pushrebasing a merge
Summary:
This is the fix that might be hard to explain.

First, why we need `generate_additional_bonsai_file_changes()` function at all.
TL;DR is to fix pushrebasing of a diamond merge commit. Merge bonsai changeset
must have a file change entry if p1 and p2 has the same file with different
content, and when merge commit is pushrebased we need to preserve this property.
more details are in D18065013 and in the comments.

Secondly, what was wrong with generate_additional_bonsai_file_changes. While it
was technically correct (i.e. it was producing valid bonsai changeset) it had
an unexpected side-effect for non-diamond merges (e.g. repo imports). It was
adding all changed files between root and onto commit (i.e. all changed files
from commits over which it was rebased). This in turn leads to unnecessary
push failures with "conflicts" errors.

Thirdly, why it was wrong. generate_additional_bonsai_file_changes
a) finds all files changed between onto and root (see comments for more details about onto
and root),
b)then finds those that needs to be added in the pushrebase merge bonsai changeset.

The problem was in step b). It needed to check which files exists in merge
parent but instead it checked which files exists in merge commit itself.
This is because merge commit itself cannot change any of the files that were changed between root
and onto - in that case we get a pushrebase conflict. So all the files come
either from root commit or from another parents. And the entry can become
"stale" only if it existed in one of the parents and later was changed between
root and onto.

Reviewed By: krallin

Differential Revision: D21596758

fbshipit-source-id: f2bea36fcc29fc736caaa7494696a1f6dc848b9e
2020-05-18 05:31:31 -07:00
Alex Hornby
878693e1cf mononoke: walker: add MPathHash memoization to WrappedPath
Summary: Add MPathHash memoization to WrappedPath as the paths are hashed multiple times

Reviewed By: farnz

Differential Revision: D21613720

fbshipit-source-id: b850cf5ea1668b5ff75b07a0489b54f86078677a
2020-05-18 04:54:59 -07:00
Harvey Hunt
c45b4541e2 mononoke: Add with_tunables function to allow overriding tunable values
Summary:
Currently, there isn't a way to override the value of a tunable during
a unit test. This is unfortunate and may push people to creating integration
tests when unit tests would be best.

Add `with_tunables`, which allows callers to override the tunables values that
a closure accesses. Further, add an example unit test that shows the behaviour.

Thanks to krallin for suggesting this and helping with the implementation
details.

Reviewed By: krallin

Differential Revision: D21595220

fbshipit-source-id: c31da88fb4fcfd38f761960ffe8025132025e6c9
2020-05-18 04:31:40 -07:00