Summary: Currently the edenapi client crashes trying to convert a headname to a hex string, when the headname is a string that's not in hex form. Check if the string is in hex form before attempting to look it up as a prefix.
Reviewed By: DurhamG
Differential Revision: D32894078
fbshipit-source-id: 1d2d8145aa3136a27e4bbe674a07e2aa8bca81ef
Summary:
Currently a single LFS 503 during clone/checkout aborts the operation and leaves the user in a situation that is difficult to recover from. Now we treat a 503 as any other 5XX server error and retry.
The 5XX retry count/backoff can be configured via the lfs.backofftimes config knob. It defaults in the code to [1, 4, 8], which is 3 retries with rand(1), rand(4) and rand(8) sleep in between.
503 can be served from proxygen indicating there are no live lfs servers, or from x2pagentd in cases of network failure.
Reviewed By: quark-zju
Differential Revision: D33804601
fbshipit-source-id: cbb3514051843d3517bb676cf7c83dce057e42b9
Summary: Adds a Python binding for the Rust `repo` crate. For now it only exposes the static `Repo::init` method.
Reviewed By: DurhamG
Differential Revision: D33796094
fbshipit-source-id: 9cf49bcdd24bbabead79902e64a49ec8dc6fac55
Summary:
Adds a method that will be used in the future for replacing the repo initialization done in Python.
More specifically, this method creates and populates the files required for a new repo in the same way as we currently do via the constructor of `localrepo` with the extensions and configs we have enabled by default.
Reviewed By: DurhamG
Differential Revision: D33901712
fbshipit-source-id: 0d12732f5c9d36fb115a97e53b356a88fd609b6c
Summary: This command is used by jf to find commits in a bundle.
Reviewed By: DurhamG
Differential Revision: D33948840
fbshipit-source-id: 2b7b8c09cb2b30c192a00e83437d079010be5a41
Summary:
I am making it impossible to put "." or ".." as a path element in Mononoke's `MPath` abstraction, since Mononoke does not do path traversal, and canonicalization is a minefield.
However, `linkfile src="."` occurs in https://fburl.com/code/evq63pto as a special case. Treat it specially in our code, so that I can forbid path traversal type elements in an `MPath`.
Reviewed By: mojsarn
Differential Revision: D33952767
fbshipit-source-id: 0c4ce68014fb6877163035d1c7eb813616748dd2
Summary:
`eden doctor` does not know how to detect or fix broken NFS mounts.
Let's teach it to recognize NFS mounts, recognize when those mounts are stale,
and fix it.
Recognizing a stale mount is a bit different on NFS and FUSE. FUSE is kind and
responds with errors when you try to read a stale mount. NFS is not so kind.
It just hangs. So instead of certain errors indicating stale mounts, hangs
will have to indicate them.
Reviewed By: chadaustin
Differential Revision: D33283361
fbshipit-source-id: 939c212db1dcb1904fc8bbf3bbf32cd39970a101
Summary: In production we do currently see some cases of lfs_read_hash_mismatch. I realized it may be due to missing chunks rather than bad chunk integrity. The new "skip_hash_on_read" path is still able to detect size mismatch, so tweak things to only log mismatch message if the sizes match.
Reviewed By: quark-zju
Differential Revision: D33928640
fbshipit-source-id: 5b63db2b05bc7b53779cdb5ae0041a7d22bb3962
Summary:
## Context
See D33771940
## This diff
I move `pull.httpbookmarks` and `infinitepush.httpbookmarks` to True.
Reviewed By: quark-zju
Differential Revision: D33891956
fbshipit-source-id: cbc3d92aa7ac7ac9edf90b4f99ab9804e9d750be
Summary:
## Context
There are some config options that have been rolled out to 100% of users for some time, but they're still not used in tests by default, which makes our coverage worse.
The best way to change the behaviour of hg regarding config is to change the default value on `configitems.py`, and then (eventually, when this rolls out everywhere), delete them from the configerator overrides config.
The other option would be to leave them in configerator (or in eden/scm/lib/configparser/src/fb/mod.rs), but then they would also need to be replicated in all tests that want to use it, or at the very least duplicated in the base hgrc for integration tests, which is very prone to errors (or simply forgetting to do it, the state we are in today).
Notice that this still allows us to set them to false via configerator quickly, if necessary.
## This diff
This diff defaults `pull.httpmutations` to true: It was previously always set to true on configparser/src/fb/mod.rs
Reviewed By: quark-zju
Differential Revision: D33771940
fbshipit-source-id: 987db9cfaff720645eac4ad65edf560f875b3655
Summary:
This is already done in another `hg pull` in this test, but needed to be done in the clone as well, as the test is a bit flaky right now.
Ideally we need to fix this so we don't need to sleep anymore, but this will unblock us for now. I tried flushing WBC in another diff (D33771281) but it did not fix the whole issue, there might be more underneath.
Reviewed By: markbt
Differential Revision: D33917157
fbshipit-source-id: c78b3718cb4d32480d1dce8dc2c962928f3f79ad
Summary:
Add a command that lets us inject a dag of commits into a test repo.
This is based on the `drawdag` implementation for unit tests. We
extend the specification format so we can call `CreateCommitContext`
methods from the integration test code.
Reviewed By: yancouto
Differential Revision: D33854929
fbshipit-source-id: 8c903adfc2caa87d24ad89744a4415619aa5ebad
Summary:
Add a new tool `mononoke-testtool`, which will be used for commands that make
integration tests easier to write.
Since this tool is expected to only be used for tests, it will detect
production configurations and refuse to run.
Reviewed By: yancouto
Differential Revision: D33854930
fbshipit-source-id: c07cb73c726c8c60ef7a94a704f485fe9fc2576d
Summary:
Add `fetch`, which will match the features of `bonsai-fetch` and
`content-fetch` in the old admin tool.
Reviewed By: kris1319
Differential Revision: D33829451
fbshipit-source-id: dd123e60b3bf168ad6cdf1b5c3e3e2f929da5ebb
Summary:
During checkout, the overlay is updated from the leaf up to the root, and thus
halfway through the operation, the overlay contains hashes from both the old
commit and the new commit. When the operation finishes, this discrepency no
longer exist as the overlay has been fully updated to the destination commit.
However, if EdenFS either crashes, or is being killed during the checkout
operation, the overlay will be left in this half updated state. Restarting
EdenFS and running a status or checkout operation will walk the inodes
hierarchy and construct them from the overlay, forcing a large amount of blobs
and manifests to be fetched from the network. This can be disruptive and lead
to updates/status taking a very long time due to potentially having to fetch
the entirety of the repository.
Thankfully, EdenFS can recognize this situation by simply updating its SNAPSHOT
file at the beginning of the checkout operation. If EdenFS crashes or is
killed, Mercurial will send EdenFS its own parent commit, which EdenFS can
compare and disagree about, failing the operation instead of fetching large
parts of the repository.
Reviewed By: chadaustin
Differential Revision: D33591210
fbshipit-source-id: b724429bb848425d2b1684b160ad8e1bdfee252a
Summary:
If a checkout is interrupted midway, EdenFS is unable to recover without
fetching very large parts of the repository. This leads to a poor user
experience as Mercurial suddenly hangs when doing anything.
In order to solve this, let's introduce a new SNAPSHOT version that allows
tracking the ongoing checkout operation and the commit origin and destination.
Since there is no current way for EdenFS to recover from this, let's also ask
the user to reclone if EdenFS is killed/crashed during checkout.
Reviewed By: genevievehelsel
Differential Revision: D33859121
fbshipit-source-id: b7cc4d2bc7aade047a230f5b579d8461c8586451
Summary:
When write notifications are pending on Windows, waitForPendingNotification
would wait for some time. Unfortunately, this means that the mountPoint thrift
argument would get freed, and since the lambda only captured a piece of the
path, and not the path directly, the mount path would get used after the
underlying path has been freed. We need to copy the path to avoid this issue.
Reviewed By: MichaelCuevas
Differential Revision: D33927317
fbshipit-source-id: 790bcb26cb46cdac750826ebe13c56e0d1275c22
Summary: Delegate to git's own bundle implementation for bundle and unbundle.
Reviewed By: DurhamG
Differential Revision: D33895676
fbshipit-source-id: 3d17160c6f83c2f43ff41bcf3f74c00aecded630
Summary:
The flag is not supported by git older than 2.28. There are still places
running older git. Change the command from `git init -b x` to
`git -c init.defaultBranch=x init` so it becomes compatibile with older
git.
We don't use the default branch. So set it to `_unused_branch` to
make it more explicit.
Reviewed By: DurhamG
Differential Revision: D33851129
fbshipit-source-id: 93534362b02353728d72d0ebef31c6a25737398f
Summary:
With a local git store all trees are local. There is no need to prefetch.
Forcing prefetch trees would cause errors, since the protocol is incompatible.
Reviewed By: DurhamG
Differential Revision: D33837781
fbshipit-source-id: c41281ac96aff1501bd056901213b912a77f6c31
Summary: It is a left-over from D33518822 (b897a19073).
Reviewed By: DurhamG
Differential Revision: D33837780
fbshipit-source-id: f42fb7f9099ab4cb388c7bd8f038839fd6976cbf
Summary:
Previously `clone` has to use git URLs with `git+` prefix in scheme such as
`git+ssh://...`, `git+file://...`, `git+https://...`. That was the same
behavior as the `hg-git` extension but is inconvenient for scp-like URLs
like `git@github.com:git/git.git` (provided by GitHub). That URL needs to be
translated to ssh form `git+ssh://git@github.com/git/git.git` manually,
which can be annoying.
Let's use a `--git` flag so `clone --git` can use the scp-like URL directly.
The `--git` flag also makes `clone` more consistent with `init`.
Reviewed By: DurhamG
Differential Revision: D33837782
fbshipit-source-id: 7fe11be6cf63932bde1f400cdfc2fc8120f5067a
Summary:
Packs are long dead. Let's remove the logging for hg rage. This logic
also doesn't work on Windows, due to depending on cmd commands, resulting in
some errors in the hg rage since we now use powershell.
Differential Revision: D33925868
fbshipit-source-id: c071f8f5685dbf7a9174cad31a0a8cd99a84cf88
Summary: There was an edge case where native checkout "up --clean" wasn't clearing the merge state if the working tree was clean. Add a check so we use fallback non-native code path if there is an active merge state.
Reviewed By: quark-zju
Differential Revision: D33893832
fbshipit-source-id: a0e4a8d23818a1e40582cede0afee62ac53cb680
Summary: Since `Repo` will soon contain more logic on its own not related to `clidispatcher`, we are moving its own crate.
Reviewed By: DurhamG
Differential Revision: D33883867
fbshipit-source-id: 94414e1e2692962bf491cc2abb4e1d3799cbcc03
Summary:
Copy the `blobstore-unlink` command into the new admin tool.
The blobstore-unlink command only works on SQLBlob, so keep it as a separate
command for now. In the future we can integrate it with the main blobstore
command.
Reviewed By: kris1319
Differential Revision: D33791722
fbshipit-source-id: 83afc8d87dc1b96e91b7e96bdb1adf0106a822b9
Summary:
Implement the `blobstore` subcommand for examining and uploading to the
blobs in the blobstore.
This is an improvement over the existing `blobstore-fetch`, as we can now
interpret the data as more types, and hex-dump the data if we can't decode it
as a known type.
Not all types are implemented yet, but it should be easy to add them later.
Reviewed By: mitrandir77
Differential Revision: D33791723
fbshipit-source-id: bb2ea6c76986fcdd34b3d7be773c5c8168e41be8
Summary: Due to a bug in the x2pagent with websocket connection polling in h2 we had to force h1. Now it's fixed and the new version of the agent with the fix should be rolled out to (almost - it's corp) 100% of users.
Reviewed By: quark-zju
Differential Revision: D33842574
fbshipit-source-id: aaf5c66143806a900253120a9f750e37ebe55b80
Summary: On Windows we apparently can't send a USR1 to trigger a trace on an eden helper process. This commit makes it possible to touch a file to trigger a particular long running process to write out its sigtrace.
Reviewed By: DurhamG
Differential Revision: D33904056
fbshipit-source-id: d42f6c071bcfaa8dfc352f65df31bdc5ea201823
Summary: Updating the takeover documentation since I have made a lot of changes.
Reviewed By: genevievehelsel
Differential Revision: D33852007
fbshipit-source-id: 665dcbb0745edf2a3e91f04ddeb69be6efb46d6e
Summary: Make "sparse enable" normalize profile names. It is not clear it wants a canonical path from the root of the repo, so now it makes some attempts to convert the given path(s) to that form. It now works with paths relative from cwd, and normalizes the path separator.
Reviewed By: quark-zju
Differential Revision: D33851998
fbshipit-source-id: 91a3d59b91266bb8941408d23d5021fa1f9e7025
Summary:
Point the thrift-types at the rust target, no need for the checked in generated code anymore
Saves generation, gives quicker signal on thrift changes (no codegen step), and stops thrift lib vs generated thrift code mismatches
Reviewed By: yancouto
Differential Revision: D33843588
fbshipit-source-id: 856faeb16dc0a2a42f2f07fd3005079a01b6e7bd
Summary: we were always running with Priority::Default. On the http path it wasn't even possible to set it to something different. I guess we don't need it then.
Reviewed By: yancouto
Differential Revision: D33842396
fbshipit-source-id: f2b18b47149e1c15df2fba0010718dd56030a29f
Summary:
No hgcli requests: https://fburl.com/ods/vkr7tjs7
Hgcli was a small binary that was running on hg servers. Its purpose was to proxy users' ssh traffic landing on hg servers to mononoke. It was sending a custom Preamble to mononoke with users' info parsed on hg servers from ssh environment, and after that it was proxying wireproto traffic that mononoke understands.
Now no client connects to hg servers. Everyone connects to mononoke using http (either http with upgrade to wireproto (only push) or edenapi). Hgcli is not even installed on hg servers. There is on need to have that code path around.
Reviewed By: yancouto
Differential Revision: D33841713
fbshipit-source-id: 682594210789ba5669660c53634f2f8111f39e41
Summary: This is an attempt to mitigate T110506843. Tasks have been crashing with SQL errors, this diff adds some retries to those calls.
Reviewed By: farnz
Differential Revision: D33791829
fbshipit-source-id: 8c5d54e7513d3526f3878285d8a684b391ef8b0d
Summary:
This diff extracts the `retry` function to a more common place. This is used to retry arbitrary functions, and from a quick search you can find multiple implementations of this throughout mononoke ([1]() [2]() [3]()), with different level of features (logging, exponential backoffs, splaying).
I made it slightly more generic, by using `FnMut` and templating the error type, so it's more future proof.
This seems to be the most generic one, so I'm extracting it to reuse it in a future diff, and avoid adding yet another implementation.
Reviewed By: ahornby
Differential Revision: D33791605
fbshipit-source-id: 6a09093a3334d41491ecab779a90b23b459d7db3
Summary: I moved some stuff around and am having trouble getting everything to be happy. One thing I forgot was to actually pipe the "fb" feature down from the entry point crates (hgcommands and backingstore).
Reviewed By: xavierd
Differential Revision: D33852732
fbshipit-source-id: aabdd145452e66435e130fcbab0ba8a80f9d943a
Summary: Run autocargo. Much of this will probably rebase away
Reviewed By: farnz
Differential Revision: D33840309
fbshipit-source-id: d0b5f5b0290f2d9703c57a36921d95952550c80f
Summary: Writing `repo: {}` everywhere is quite annoying when we can just set a value in the logger instead.
Reviewed By: farnz
Differential Revision: D33845555
fbshipit-source-id: 0e6773549c3913df68719362fa1c926f205802fe
