Summary:
Currently a single LFS 503 during clone/checkout aborts the operation and leaves the user in a situation that is difficult to recover from. Now we treat a 503 as any other 5XX server error and retry.
The 5XX retry count/backoff can be configured via the lfs.backofftimes config knob. It defaults in the code to [1, 4, 8], which is 3 retries with rand(1), rand(4) and rand(8) sleep in between.
503 can be served from proxygen indicating there are no live lfs servers, or from x2pagentd in cases of network failure.
Reviewed By: quark-zju
Differential Revision: D33804601
fbshipit-source-id: cbb3514051843d3517bb676cf7c83dce057e42b9
Summary: Adds a Python binding for the Rust `repo` crate. For now it only exposes the static `Repo::init` method.
Reviewed By: DurhamG
Differential Revision: D33796094
fbshipit-source-id: 9cf49bcdd24bbabead79902e64a49ec8dc6fac55
Summary:
Adds a method that will be used in the future for replacing the repo initialization done in Python.
More specifically, this method creates and populates the files required for a new repo in the same way as we currently do via the constructor of `localrepo` with the extensions and configs we have enabled by default.
Reviewed By: DurhamG
Differential Revision: D33901712
fbshipit-source-id: 0d12732f5c9d36fb115a97e53b356a88fd609b6c
Summary: This command is used by jf to find commits in a bundle.
Reviewed By: DurhamG
Differential Revision: D33948840
fbshipit-source-id: 2b7b8c09cb2b30c192a00e83437d079010be5a41
Summary:
I am making it impossible to put "." or ".." as a path element in Mononoke's `MPath` abstraction, since Mononoke does not do path traversal, and canonicalization is a minefield.
However, `linkfile src="."` occurs in https://fburl.com/code/evq63pto as a special case. Treat it specially in our code, so that I can forbid path traversal type elements in an `MPath`.
Reviewed By: mojsarn
Differential Revision: D33952767
fbshipit-source-id: 0c4ce68014fb6877163035d1c7eb813616748dd2
Summary:
`eden doctor` does not know how to detect or fix broken NFS mounts.
Let's teach it to recognize NFS mounts, recognize when those mounts are stale,
and fix it.
Recognizing a stale mount is a bit different on NFS and FUSE. FUSE is kind and
responds with errors when you try to read a stale mount. NFS is not so kind.
It just hangs. So instead of certain errors indicating stale mounts, hangs
will have to indicate them.
Reviewed By: chadaustin
Differential Revision: D33283361
fbshipit-source-id: 939c212db1dcb1904fc8bbf3bbf32cd39970a101
Summary: In production we do currently see some cases of lfs_read_hash_mismatch. I realized it may be due to missing chunks rather than bad chunk integrity. The new "skip_hash_on_read" path is still able to detect size mismatch, so tweak things to only log mismatch message if the sizes match.
Reviewed By: quark-zju
Differential Revision: D33928640
fbshipit-source-id: 5b63db2b05bc7b53779cdb5ae0041a7d22bb3962
Summary:
## Context
See D33771940
## This diff
I move `pull.httpbookmarks` and `infinitepush.httpbookmarks` to True.
Reviewed By: quark-zju
Differential Revision: D33891956
fbshipit-source-id: cbc3d92aa7ac7ac9edf90b4f99ab9804e9d750be
Summary:
## Context
There are some config options that have been rolled out to 100% of users for some time, but they're still not used in tests by default, which makes our coverage worse.
The best way to change the behaviour of hg regarding config is to change the default value on `configitems.py`, and then (eventually, when this rolls out everywhere), delete them from the configerator overrides config.
The other option would be to leave them in configerator (or in eden/scm/lib/configparser/src/fb/mod.rs), but then they would also need to be replicated in all tests that want to use it, or at the very least duplicated in the base hgrc for integration tests, which is very prone to errors (or simply forgetting to do it, the state we are in today).
Notice that this still allows us to set them to false via configerator quickly, if necessary.
## This diff
This diff defaults `pull.httpmutations` to true: It was previously always set to true on configparser/src/fb/mod.rs
Reviewed By: quark-zju
Differential Revision: D33771940
fbshipit-source-id: 987db9cfaff720645eac4ad65edf560f875b3655
Summary:
This is already done in another `hg pull` in this test, but needed to be done in the clone as well, as the test is a bit flaky right now.
Ideally we need to fix this so we don't need to sleep anymore, but this will unblock us for now. I tried flushing WBC in another diff (D33771281) but it did not fix the whole issue, there might be more underneath.
Reviewed By: markbt
Differential Revision: D33917157
fbshipit-source-id: c78b3718cb4d32480d1dce8dc2c962928f3f79ad
Summary:
Add a command that lets us inject a dag of commits into a test repo.
This is based on the `drawdag` implementation for unit tests. We
extend the specification format so we can call `CreateCommitContext`
methods from the integration test code.
Reviewed By: yancouto
Differential Revision: D33854929
fbshipit-source-id: 8c903adfc2caa87d24ad89744a4415619aa5ebad
Summary:
Add a new tool `mononoke-testtool`, which will be used for commands that make
integration tests easier to write.
Since this tool is expected to only be used for tests, it will detect
production configurations and refuse to run.
Reviewed By: yancouto
Differential Revision: D33854930
fbshipit-source-id: c07cb73c726c8c60ef7a94a704f485fe9fc2576d
Summary:
Add `fetch`, which will match the features of `bonsai-fetch` and
`content-fetch` in the old admin tool.
Reviewed By: kris1319
Differential Revision: D33829451
fbshipit-source-id: dd123e60b3bf168ad6cdf1b5c3e3e2f929da5ebb
Summary:
During checkout, the overlay is updated from the leaf up to the root, and thus
halfway through the operation, the overlay contains hashes from both the old
commit and the new commit. When the operation finishes, this discrepency no
longer exist as the overlay has been fully updated to the destination commit.
However, if EdenFS either crashes, or is being killed during the checkout
operation, the overlay will be left in this half updated state. Restarting
EdenFS and running a status or checkout operation will walk the inodes
hierarchy and construct them from the overlay, forcing a large amount of blobs
and manifests to be fetched from the network. This can be disruptive and lead
to updates/status taking a very long time due to potentially having to fetch
the entirety of the repository.
Thankfully, EdenFS can recognize this situation by simply updating its SNAPSHOT
file at the beginning of the checkout operation. If EdenFS crashes or is
killed, Mercurial will send EdenFS its own parent commit, which EdenFS can
compare and disagree about, failing the operation instead of fetching large
parts of the repository.
Reviewed By: chadaustin
Differential Revision: D33591210
fbshipit-source-id: b724429bb848425d2b1684b160ad8e1bdfee252a
Summary:
If a checkout is interrupted midway, EdenFS is unable to recover without
fetching very large parts of the repository. This leads to a poor user
experience as Mercurial suddenly hangs when doing anything.
In order to solve this, let's introduce a new SNAPSHOT version that allows
tracking the ongoing checkout operation and the commit origin and destination.
Since there is no current way for EdenFS to recover from this, let's also ask
the user to reclone if EdenFS is killed/crashed during checkout.
Reviewed By: genevievehelsel
Differential Revision: D33859121
fbshipit-source-id: b7cc4d2bc7aade047a230f5b579d8461c8586451
Summary:
A long time ago, getdeps scheduled each build up to the number of hardware threads. For some builds, that was too heavy, so it got throttled back to only ever use half the hardware threads. This left parallelism on the table for CPU-bound projects that don't use much RAM per compilation.
This commit makes better use of the hardware with finer-grained logic that allows each manifest to specify a `job_weight_mib` estimate in MiB, and limit concurrency to `available_ram / job_weight`.
Reviewed By: ahornby
Differential Revision: D33754018
fbshipit-source-id: 785bed6c6cfe3c473244e0806a77cec1fc119e1f
Summary:
When write notifications are pending on Windows, waitForPendingNotification
would wait for some time. Unfortunately, this means that the mountPoint thrift
argument would get freed, and since the lambda only captured a piece of the
path, and not the path directly, the mount path would get used after the
underlying path has been freed. We need to copy the path to avoid this issue.
Reviewed By: MichaelCuevas
Differential Revision: D33927317
fbshipit-source-id: 790bcb26cb46cdac750826ebe13c56e0d1275c22
Summary: Delegate to git's own bundle implementation for bundle and unbundle.
Reviewed By: DurhamG
Differential Revision: D33895676
fbshipit-source-id: 3d17160c6f83c2f43ff41bcf3f74c00aecded630
