Summary:
I spotted D23955629 recently, and it turns out I made the same mistake in LFS.
Let's fix it. In order to keep the same semantics (yield on every chunk from
the filestore), I lowered the threshold a bit from what it was intended to be.
Reviewed By: johansglock
Differential Revision: D23960152
fbshipit-source-id: 4d384752228fd125ade7e032a477648798e7fa44
Summary: Now that the `async_runtime` crate exists, use Mercurial's global `tokio::Runtime` instead of creating one for each EdenAPI store.
Reviewed By: quark-zju
Differential Revision: D23945569
fbshipit-source-id: 7d7ef6efbb554ca80131daeeb2467e57bbda6e72
Summary:
Pull Request resolved: https://github.com/facebookexperimental/rust-shed/pull/12
The OpenSSL version on Mac doesn't work well with EdenSCM and Mononoke integration, just use the one from getdeps/brew.
Also remove the now redundant "DEVELOPER_DIR" since the modern XCode version works.
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/63
Reviewed By: StanislavGlebik
Differential Revision: D23927022
Pulled By: lukaspiatkowski
fbshipit-source-id: 6b6b3baa33d49b567b9aa6178cbd20b7ae9edc89
Summary:
This is (I think) the last step required to make rust-partial-io be github-first.
The diff was created using:
* `zbgs partial-io` and remove all instances of it in fbsource
* `hg rm common/rust/partial-io`
* add `partial-io` to `third-party/rust/Cargo.toml`
* `common/rust/tools/reindeer/vendor` to vendor
* `buck build eden/mononoke/mercurial/...` to check that it builds correctly
* `buck run //common/rust/cargo_from_buck:cargo_from_buck` to run autocargo
Reviewed By: aslpavel
Differential Revision: D23849634
fbshipit-source-id: 339fc3976cc9a0b6f10a0538d643b87797e2bc3c
Summary:
`arc install-certificate` is going away, so let's reference the replacement
Created from Diffusion's 'Open in Editor' feature.
Differential Revision: D23948479
fbshipit-source-id: e629458ed2be1e1c09d9674935e3bcc890b05ad9
Summary:
See motivation for the change in D23845720 (5de500bb99).
We'll need to store version name even for commits that weren't rewritten, but that have an equivalent working copy in another repo.
Reviewed By: ikostia
Differential Revision: D23864571
fbshipit-source-id: 408b68c3b0aa9885a9cd248b0b4abc2b87cd4cca
Summary:
get_source_target_mover likely awaits the same fate as
get_current_mover_DEPRECATED functions i.e. get_source_target_mover will likely
be removed.
This diff just removes a few intances of this function.
Reviewed By: ikostia
Differential Revision: D23929748
fbshipit-source-id: 2ac09da164de3916a552757acf0c39387f6126e4
Summary:
get_mover() and get_reverse_mover() functions return the mover for the
"current" version of the commit sync config, which means these are movers for the version
of the config that's used to create the latest commits on master branch.
So this function returns correct mover only for the latest master commit, but
for all other commits it returns an incorrect mover! This is wrong and it
happened to work just by change, and that's why these functions are marked as deprecated
now, and later we'll add functions 'get_mover_by_version()' which could be used to
replace deprecated functions.
Note that the story for get_bookmark_renamer()/get_reverse_bookmark_renamer()
functions seems to be different. If we can always figure out what's the correct
mover for a commit by e.g. look at its parent we can't really do the same for
bookmarks. Because of that I suggest to keep using the current version for
get_bookmark_renamer() function.
Reviewed By: ikostia
Differential Revision: D23929582
fbshipit-source-id: 3e5e9b46224aca0b75cf2d981ea21c4f9a378ba9
Summary: Finally remove version_name from CommitSyncRepos. Note that this diff adds a few TODOs that we'd need to fix later.
Reviewed By: ikostia
Differential Revision: D23929010
fbshipit-source-id: c72130af548ac7b26bc20ddaac9a59562cc75e0b
Summary: Just as in the previous diff, but this time remove bookmark_renamers from CommitSyncRepos
Reviewed By: ikostia
Differential Revision: D23910295
fbshipit-source-id: 0c2d147057c8d3e0749d5b31ef98ab5022255d95
Summary: Just as the previous diff, but this time it removes reverse_mover
Reviewed By: ikostia
Differential Revision: D23879509
fbshipit-source-id: ed111ca2d106120229c4facc0bb2435913c27966
Summary:
This diff starts to use CommitSyncDataProvider introduced in the previous diff
and removes Mover from CommitSyncRepos struct.
Reviewed By: ikostia
Differential Revision: D23878683
fbshipit-source-id: 0d54f889781aebe4726b3388343a87df783c17d4
Summary:
As described in D23845720 (5de500bb99) we are doing a pretty significant change in the
CommitSyncer. Previously it stored static Movers and BookmarkRenamers, but it
needs to change and they would need to fetch config from LiveCommitSyncConfig.
Unfortunately we already have a bunch of tests that create weird movers that
would be very hard to model via LiveCommitSyncConfig. So we have 2 options:
1) delete or rewrite all these tests
2) Create a wrapper that would return a mover/bookmark renamers.
Option #2 is preferable, and that's what this diff does. In production it would
use LiveCommitSyncConfig, but it also let's tests specify whatever weird movers
they need.
Reviewed By: ikostia
Differential Revision: D23909432
fbshipit-source-id: 83fb627812f625e07f7e40044e2f69274cd2d768
Summary:
Now that public phases are separated from draft, hidden behind the
ispublic() api, and no consumers require the entire set anymore, let's make them
lazy by testing commits against a lazy ancestor collection.
Reviewed By: quark-zju
Differential Revision: D23921326
fbshipit-source-id: 7c53345f41ff89d2d0ec11b1c09bbb2f89529c21
Summary:
Previously, _genrevancestor applied it's cutfunc function (which is
usually used to bound an ancestor traversal from processing public nodes) when
producing the parents of the current rev. In the walkrevtree function this means
we have to invoke cutfunc before we enqueue new parents onto the heap. This can
mean we have to process very old commits, even though walkrevtree might never
get to them.
Let's move the cutfunc to be at the deque stage instead of enque. When we make
public phases lazy, this will let us avoid looking up really old commits and
speed things up a lot.
Reviewed By: quark-zju
Differential Revision: D23921324
fbshipit-source-id: 8b520e9184441b9bd7581cdb71414f406cf9b544
Summary:
Unfortunately the current add operator implementation for nameset sorts
the result, which causes lazy namesets to be evaluated. Let's change the phase
revset to avoid that operator for now.
In a future diff we'll make publicrevs lazy, which will require this change to
be performant.
Reviewed By: quark-zju
Differential Revision: D23921325
fbshipit-source-id: c3477c30c1c5061b641c14b8aa73d07da6f7f398
Summary:
In a future diff we'll be changing the way public phase computation
works. Let's start by moving the draft phase computation out to belong to
draftrevs(), so then we can change publicrevs() without worrying about draft. In
the process, let's simplify draftrevs to just be an only() query.
Reviewed By: quark-zju
Differential Revision: D23921327
fbshipit-source-id: 255eab4b796bf7bc4467544cf8231b3ed98c08c8
Summary:
In an upcoming diff we'll be splitting draft and public calculations
apart when in headbased repositories. To make this cleaner, let's stop using
loadphaserevs and start using draftrevs and publicrevs accessors.
Reviewed By: quark-zju
Differential Revision: D23921328
fbshipit-source-id: 26a2a6fdc1201416bf61c7d32e3ede62d9579546
Summary:
Our configerator configs store both "current" mapping version and also they
store all versions that were used before.
These integration tests were updating just current version, but weren't
updating "all" versions. This is incorrect but it worked by accident.
But it will stop working in the next diffs, so this diff fixes it
Reviewed By: ikostia
Differential Revision: D23908970
fbshipit-source-id: 10f96bd02987d9195aff4855241efbd9a065a761
Summary:
A higher-level goal is to provide an interface for the manual remediation of
various xrepo-sync blockages. The nature of a candidate selection is such that
it may fail if the hint is not sufficient to decide which remapping changeset
is the best candidate. This is especially true about the `Only` hint variant:
it is designed to fail when there's more than one candidate. But even with
bookmark ancestorship hints, there are corner cases when the algorithm cannot
make a decision (as well as there may just be bugs in the algorithm). These
cases should be **extremely** rare. Nevertheless, we want to be able to unblock
ourselves. To do so, it is proposed to acccept parent selection hints in the `xrepo-lookup` scs
method. By default, it will use `Only` as a hint and be semantically equivalent
to the current behavior. But we'll provide CLI options to select other hints.
In order to make this work, we need the `sync_commit` method of the
`CommitSyncer` to accept hints too.
Reviewed By: StanislavGlebik
Differential Revision: D23913216
fbshipit-source-id: 05e1ff99cd2c6522829a6e8569040b226600af60
Summary:
These tests rely on sleep to simulate race conditions. They are the
largest source of hgbuild breakages. Let's just disable them. Most of this code
is going away in a couple months anyway when Mononoke takes over and revlogs go
away.
Also fixes a failure in test-bookmarks-strip.t that showed up.
Reviewed By: singhsrb
Differential Revision: D23922660
fbshipit-source-id: 35cacc19de03c031c84c1ebf2124884f4b4248e6
Summary: Now that the EdenAPI server is using the `LoadMiddleware` from `gotham_ext`, each response will contain an `X-Load` header that contains the number of active requests that the server is currently handling.
Reviewed By: quark-zju
Differential Revision: D23922809
fbshipit-source-id: 973143de5ddccf074d28aa3ef38d73f9fc1501b6
Summary:
Network speeds are usually reported in megabits per second (Mb/s), whereas file sizes are usually reported in [mebibytes](https://en.wikipedia.org/wiki/Binary_prefix) per second (MiB/s). Previously, the HTTP client reported neither of those and instead reported megabytes per second (MB/s).
This diff changes the latter to the former so that the numbers are more immediately useful. As a bonus, the speeds are now directly comparable to those reported by `hg debugnetwork`.
Reviewed By: quark-zju
Differential Revision: D23920367
fbshipit-source-id: 46500a42681ab83fc7c4ead82980e8ed620a4d5a
Summary: Now that stats are logged to `tracing` by the `HttpClient` directly, we no longer need to log them here. This commit backs out D23858077 (613fbc858f) which added the logging.
Reviewed By: quark-zju
Differential Revision: D23919308
fbshipit-source-id: 23d3a12c5307bc4b84dd9ffd25bd376718e3cc91
Summary:
Improve the log output of the HTTP client to avoid spewing redundant debug messages.
As part of this change, logging now uses the `tracing` crate instead of the `log` crate for better integration with the rest of Mercurial's logging infrastructure. Right now, `tracing` is just being used as a drop-in replacement for `log`, but now that it's in use we can start using its full capabilities (such as defining tracing spans) in later diffs.
Reviewed By: quark-zju
Differential Revision: D23919310
fbshipit-source-id: 95555ad083ead805ceece39c6e30aaf879bdf2bc
Summary:
Fix the `hg mv` and `hg cp` commands to wait for the lock if the repository is
locked, instead of immediately failing.
The current behavior is quite annoying as `hg mv` can easily fail due to
commitcloud sync commands running in the background and holding the lock.
I can't see any good reason why this command should not wait for the lock to
be held. The current behavior of failing when the lock is held dates back to
2006, in rFBSf5b6e7d2f591, when locking was first added to this command.
Reviewed By: DurhamG
Differential Revision: D23896302
fbshipit-source-id: 29bfeb8fd356abd31b6e1eb30c8f211a31fb044b
Summary:
The remotefilelog.indexedlog{data,history}store configs are no longer in use,
so we shouldn't use them as a way to test whether the indexedlog are present.
Reviewed By: quark-zju
Differential Revision: D23918133
fbshipit-source-id: 902a369f5e1774b8e2fa1e4b3280fbb7cc377775
Summary:
We were using the timeout parameter on `Multi::wait` (equivalent to `curl_multi_wait` in C) incorrectly. Previously, we were passing in the timeout provided by `curl_multi_timeout`.
This is incorrect usage because the value returned by `curl_multi_timeout` is the current value of libcurl's internal timeout (based on the state of the transfers), which will always be respected. The actual intention of the timeout parameter is to allow the caller to specify a hard cap on curl's internal timeout, so we should just pass some reasonable default value here. ([See explanation here.](https://github.com/curl/curl/issues/2996))
The purpose of `curl_multi_timeout` is to allow libcurl to tell the application what its desired timeout is in situations where the application itself is waiting for socket activity (using something like `epoll`), which is not the case when using `curl_multi_wait`.
Reviewed By: DurhamG
Differential Revision: D23914093
fbshipit-source-id: 76a25d7c59a4b08437c8d7be3d24708fb37b9172
Summary: Add appropriate EdenAPI calls to allow for HTTP tree fetching in `treemanifest`. Enabling `remotefilelog.http` essentially reroutes `_prefetch` and `getdesignatednodes` to their HTTP equivalents.
Reviewed By: DurhamG
Differential Revision: D23877319
fbshipit-source-id: 8a71934b47d07d2655fa46c103a14fb99e2f7b1f
Summary: Use the functionality from D23910534 (721f5af278) to set a timeout for EdenAPI requests, configured via the `edenapi.timeout` option.
Reviewed By: DurhamG
Differential Revision: D23911552
fbshipit-source-id: 4a6e3de1094d0faa1daaf6fe4b9b7aafb37a25a8
Summary:
Cast `names` and `nodes` to lists. The reason that they are sets is deduplication, but otherwise the code doesn't rely on them being sets (and in fact casts them to lists at multiple points later).
The main motivation for this is to allow these to be passed to Rust code later. The Rust bindings make a distinction between sequence types and unordered types, so passing a set in place of a list would result in a type error.
Reviewed By: DurhamG
Differential Revision: D23893108
fbshipit-source-id: 9ce2addb824867bcb2d24ba14c589b8791a156e8
Summary: Add the ability to set a timeout on HTTP requests. Equivalent to [`CURLOPT_TIMEOUT_MS`](https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT_MS.html).
Reviewed By: DurhamG
Differential Revision: D23910534
fbshipit-source-id: a7aec792ec3c122a01aa44fcfe2e2df6e3a111fc
Summary:
This diff adds the use of candidate selection hints to `cross_repo_sync` code sights, which need to query `CommitSyncOutcome` in the small-to-large direction. Specifically: `unsafe_sync_commit` and `unsafe_sync_commit_pushrebase` are the two main functions.
One will now get `CandidateSelectionHint` from the callsight (most notably: `push_redirector`), the other one will build a bookmark-based hint itself.
Reviewed By: StanislavGlebik
Differential Revision: D23715259
fbshipit-source-id: 3f4924f1337b09f3762cc050c4017c5d2bd6cab6
Summary: These concatenated strings and bytes and needed to be fixed.
Reviewed By: kulshrax
Differential Revision: D23907301
fbshipit-source-id: 0008d7d54469266ecbae8ddaaa7625820f62cb7e
Summary:
There are several places in the HTTP client where we log and discard errors. (Typically, these are "this should never happen" type situations.)
Previously, these were logged at the `trace` log level, meaning that in practice no one would ever know if we did hit these errors.
Let's upgrade them to `error` so that they'll be printed out. (In theory, users should never see these error messages unless something has gone horribly wrong.)
Reviewed By: DurhamG
Differential Revision: D23888268
fbshipit-source-id: 9007205f946ebb0127238c76812cf62524878047
