Summary:
D21573455 accidentally removed the scuba sampling for scrub_get
operations. Add this back in.
Reviewed By: StanislavGlebik
Differential Revision: D21638972
fbshipit-source-id: eee66dbce161de69246f4da0a15dc2cf00e1ba01
Summary:
See bottom diff in the stack for the motivation. Though you can probably guess
the motivation :)
Reviewed By: farnz
Differential Revision: D21623154
fbshipit-source-id: a0940d766a67080ddcb346c2e3313eb08699edad
Summary: First diff in the stack that removes getfiles since it's no longer needed.
Reviewed By: farnz
Differential Revision: D21623156
fbshipit-source-id: 44f310ec4e4f34845cc5bf1738f1a8ece14e6694
Summary:
Before this change, pull_shallow only calls setuprepo if the remotefilelog
requirement is not in repo. With D21011401, the remotefilelog requirement
will be added by clone.py and pull_shallow can skip calling setuprepo, causing
the pull code paths to write file logs (and fail).
Change the pull_shallow to always call `setuprepo` to solve the issue.
The final fix is probably moving more remotefilelog related clone logic to core.
Right now I just did the minimal change to fix things.
Reviewed By: sfilipco
Differential Revision: D21632429
fbshipit-source-id: 17775ac0df18cda10247419b40f9c27436b22606
Summary:
We didn't have a high level overview of the types and traits and their
purposes, add that.
Reviewed By: DurhamG
Differential Revision: D21599759
fbshipit-source-id: 18e8d23ed00134f9d662778eddaee4a9451f7c2c
Summary: EdenFS doesnt daemonize correctly due to the privhelper not closing fd 0 (see http://www.faqs.org/faqs/unix-faq/programmer/faq/). This redirects stdin to /dev/null/ in order to do so.
Reviewed By: xavierd
Differential Revision: D21602545
fbshipit-source-id: 0aeb589efbf214ef22c0db039fbb6a436a71e360
Summary:
This updates a placeholder in the authentication process for requests
in the rust thrift services to allow authentication with CATs. This change
allows for passing in a configurable server identity to be used as the
verifier identity here.
Additionally this sets the verifer identity to be "scm_service_identity"
for the mononoke scs server (the service identity for this service).
This is needed to use the mononoke scs server from corp.
Reviewed By: markbt
Differential Revision: D21560204
fbshipit-source-id: 02e41718307fddab6f0ac2101306bb7f4a59cf7e
Summary: Otherwise the version information is only available in Python.
Reviewed By: DurhamG
Differential Revision: D19803762
fbshipit-source-id: 044c5da86efc8c657d0c422a2b1947086444895e
Summary:
Currently, we have a gap between updating and subsequently reading the
value of `write_order`. If another blobstore's put operation completes before
we have called `record_put_stats` then we may have an incorrect `write_order`
value.
Whilst this is only a minor issue, let's fix it anyway. :-)
Reviewed By: krallin
Differential Revision: D21619669
fbshipit-source-id: 1b8bacbcb4c195e6765ebdfaa68425f286f88c06
Summary:
After other optimizations the CHashMap version of visit_count was showing as a hot spot.
Given the number of possible NodeTypes is small we can store the visit_count in array indexed by ordinal instead.
Reviewed By: farnz
Differential Revision: D21618518
fbshipit-source-id: 84978778034df11a9a48452adf9269db2dc17145
Summary:
Currently we record them only during pushrebase. Let's record during push as
well.
To simplify things a little bit let's allow only a very simple push case:
1) Single bookmark.
2) All pushed commits should be reachable by this bookmark.
Reviewed By: krallin
Differential Revision: D21451337
fbshipit-source-id: bf2f1e6025ac116fb8096824b7c4c6440d073874
Summary:
This is the fix that might be hard to explain.
First, why we need `generate_additional_bonsai_file_changes()` function at all.
TL;DR is to fix pushrebasing of a diamond merge commit. Merge bonsai changeset
must have a file change entry if p1 and p2 has the same file with different
content, and when merge commit is pushrebased we need to preserve this property.
more details are in D18065013 and in the comments.
Secondly, what was wrong with generate_additional_bonsai_file_changes. While it
was technically correct (i.e. it was producing valid bonsai changeset) it had
an unexpected side-effect for non-diamond merges (e.g. repo imports). It was
adding all changed files between root and onto commit (i.e. all changed files
from commits over which it was rebased). This in turn leads to unnecessary
push failures with "conflicts" errors.
Thirdly, why it was wrong. generate_additional_bonsai_file_changes
a) finds all files changed between onto and root (see comments for more details about onto
and root),
b)then finds those that needs to be added in the pushrebase merge bonsai changeset.
The problem was in step b). It needed to check which files exists in merge
parent but instead it checked which files exists in merge commit itself.
This is because merge commit itself cannot change any of the files that were changed between root
and onto - in that case we get a pushrebase conflict. So all the files come
either from root commit or from another parents. And the entry can become
"stale" only if it existed in one of the parents and later was changed between
root and onto.
Reviewed By: krallin
Differential Revision: D21596758
fbshipit-source-id: f2bea36fcc29fc736caaa7494696a1f6dc848b9e
Summary: Add MPathHash memoization to WrappedPath as the paths are hashed multiple times
Reviewed By: farnz
Differential Revision: D21613720
fbshipit-source-id: b850cf5ea1668b5ff75b07a0489b54f86078677a
Summary:
Currently, there isn't a way to override the value of a tunable during
a unit test. This is unfortunate and may push people to creating integration
tests when unit tests would be best.
Add `with_tunables`, which allows callers to override the tunables values that
a closure accesses. Further, add an example unit test that shows the behaviour.
Thanks to krallin for suggesting this and helping with the implementation
details.
Reviewed By: krallin
Differential Revision: D21595220
fbshipit-source-id: c31da88fb4fcfd38f761960ffe8025132025e6c9
Summary:
Let's add an option to log how many files and trees were fetched in a
particular repo that start with a prefix.
Reviewed By: farnz
Differential Revision: D21617347
fbshipit-source-id: a57f74eadf32781e6c024e18da252c98af21996d
Summary:
Just as for ints and bools, let's add support for strings.
A few notes:
1) we don't have AtomicString, so I decided to use ArcSwap<String>.
However to make code generation simpler (see tunables-derive) I've added a type
alias for that
2) Return type is Arc<String> to avoid unnecessary copies.Another option that we might have here is to put the whole Tunables structure inside ArcSwap, and changing `tunables()` to return Arc<MononookeTunables>
instead of &'static MononokeTunables. However that's a bigger change to make.
Reviewed By: markbt
Differential Revision: D21592390
fbshipit-source-id: 6d3cf340b13f7aef9adb2b1b99ed2bf260033285
Summary:
An earlier change (D21394302) made discovery sampling use the most
recent commits. This ended up not being correct since if the requested sample
did not find any hits then the unknown set would not be reduced and the next
requested set would not change, resulting in an infinite loop.
The goal of the original diff was to ensure master was in the initial sample set
so we didn't have to inspect a bunch of really old commits. Let's go back to
random sampling, but manually insert master into the initial set.
In the long-term we should throw away all this code and use something similar to
discovery.fastdiscovery instead which just relies on the public heads.
Reviewed By: quark-zju
Differential Revision: D21577331
fbshipit-source-id: d738de255e292fc569eb27f250dcd3eaafaaae43
Summary:
This checks in some design documents based on the "EdenFS Internals"
presentation that I've given internally a few times.
Reviewed By: wez
Differential Revision: D21519530
fbshipit-source-id: 3f79d38e8ccf994b2ef303d491809a91fa5b6d95
Summary:
CMake description of `JOB_POOL` is:
"A pool is a named integer property and defines the maximum number of
concurrent jobs which can be started by a rule assigned to the pool."
Not very clear by it looks like putting all the Rust jobs in it prevent CMake
from spawning concurrent jobs when these are running. This should help in
reducing the strain on the system while compiling, while not increasing
compile time.
Reviewed By: wez
Differential Revision: D21595135
fbshipit-source-id: e718c92a237274a9edbc35417644a46bdfde5617
Summary:
Next step in unifying the mount path, let's make the initialize the same in
Windows and unices. The only difference is now limited to the .eden directory
which we will be able to implement once regular users can create symlinks.
For the takeover code, the #ifdef is pushed down to the actual code that does
it, this allows the rest of the code to not have to bother about Windows vs
other platforms.
Reviewed By: wez
Differential Revision: D21517478
fbshipit-source-id: d40ca2694d23031ff98e319071e610efa306008f
Summary:
The blobstore multiplexer contains logic to log blobstore operations to
scuba, as well as updating `PerfCounters`. There are some cases where we don't use the
multiplexed blobstore, which means that we're missing this important logging.
Factor out the logging to a separate crate and implement `LogBlob`, which wraps
another blobstore and performs logging to both scuba and PerfCounters.
Reviewed By: StanislavGlebik
Differential Revision: D21573455
fbshipit-source-id: 490ffd347f1ad19effc93b93f880836467b87651
Summary:
The configerator thrift file was updated to remove a path from hook
config. However, this change wasn't synced to fbsource.
Sync the change and fix up the tests that are broken by this change.
Reviewed By: krallin
Differential Revision: D21594221
fbshipit-source-id: 7b64180914f6c6802e4d70fcb1a5d6ec36eb2eac
Summary:
When we don't provide a repo name, scm_telem_log will run `hg config` to try
and get the path, and fall back to using the basename of the repo directory.
However, this is a bit undesirable, because if we ran this code we already
checked the path, so the repo URL isn't going to magically materialize once `hg
config` asks for it, which means we make a completely redundant call to hg from
scm_telem_log.
By just doing this in logginghelper, we avoid this extra roundtrip.
Reviewed By: StanislavGlebik
Differential Revision: D21572027
fbshipit-source-id: 58e5ab2e3e525edef1ecde039cd968eab8d89172
Summary:
My jq has 2 spaces after "Usage:", so this fails. Let's make the matching a bit
more lenient.
Reviewed By: StanislavGlebik
Differential Revision: D21572028
fbshipit-source-id: 1948af8bbee6486df78299c3cc08209ce3ef3e12
Summary: This might provide some useful insights for debugging hanging processes.
Reviewed By: kulshrax
Differential Revision: D21580307
fbshipit-source-id: 424992cdd9851bfd782a3333ed22ac1193c33fe3
Summary:
Make `prev` and `next` match the behavior of `update` and allow movement with pending changes. A new `--check` flag has been added to enforce a clean working copy (similar to `update`).
Note that these commands literally just call `update` under the hood, so this was just a matter of removing the existing dirtiness check. (The check was originally put there because `update` originally required a clean working copy by default.)
Reviewed By: quark-zju
Differential Revision: D21581805
fbshipit-source-id: 33d0ac4d36a795713054af7e8776d077fd353048
Summary: Previously, we would only allow BFS prefetching when HTTP was enabled (since prior to the `designatednodes` server capability, SSH had no way of supporting BFS). That has now changed, so we should allow BFS even if HTTP is disabled.
Reviewed By: quark-zju
Differential Revision: D21581215
fbshipit-source-id: b9d5945730e2c718681004f7d5e5f319449fcf6a
