Summary:
When a cert is expired, it can happen that the SSL error isn't raised till
after all our TLS handling, but in performing the HTTP request. Check the
exception type to display our certificate warning in these cases as well.
Reviewed By: krallin
Differential Revision: D26101231
fbshipit-source-id: 8342db5908e1b5fac9973fe48de432d591e9f54e
Summary:
Bytecode generation entry point `compile_ffi::compile_from_text_cpp_ffi(...)` and a driver to exercise it: `hh_single_compile_cpp`.
What we get here is a C FFI for generating HHAS from source text. We can tweak the interface as we need to going forward but this will do for a first version and I think has roughly feature parity with the OCaml FFI:
- `compile_ffi.rs` defines the function;
- `compile_ffi.h` defines the C interface;
- `hh_single_compile_cpp.cpp` provides a C++ CLI that calls the function.
Reviewed By: shiqicao
Differential Revision: D25979967
fbshipit-source-id: 4b46f9af23c61150dda6c33f9fa14e2c455c54c2
Summary:
This diff revives D25454687 (f98273063a), which was backed out by D25792491 (b52168c4c8) because it was causing Mercurial to crash in certain environments where certificates are configured incorrectly.
I've modified the code so that by default, certificates are not validated (maintaining the old behavior), but users of the API can opt-in to validation. In the case of EdenAPI (which is the only user that opts in), this is controlled via a new `edenapi.validate-certs` config option, which defaults to false. This allows enforcing validation on platforms where the configs should be correct (such as devservers) while maintaining the old behavior on other platforms by default.
Reviewed By: DurhamG
Differential Revision: D26009207
fbshipit-source-id: 904dee61fd12fdee4a0031d14adef7fdb4801139
Summary:
Previously the LazySet only supports non-async Iterator. This makes it more
flexible useful. It will be used in upcoming changes.
Reviewed By: sfilipco
Differential Revision: D25858800
fbshipit-source-id: 8c8e874f05cfab721bc0fa55160a9337ed7c2c27
Summary:
In the future we'd like to allow building the dag crate without the indexedlog
portion. This diff adds support for that.
Reviewed By: DurhamG
Differential Revision: D25769054
fbshipit-source-id: eb5a200841f878836a9f68e65e7d50be7e6b9a79
Summary:
In the future we want to build dag without indexedlog dep for Mononoke
use-case. One of the problem is the ToWire trait implemented on dag::Id by
edenapi-types. Within buck, the dag crate will have 2 targets: dag and dag-lite
(no indexedlog). They are incompatible meaning that edenapi-types depending on
dag-lite will not provide Id::to_wire for crates using dag, or vice-versa.
To solve that, we move the Id and other types to a separate crate that only has
one buck target so edenapi-types, and segmented_changelog from Mononoke can
depend on it without issues. This also makes edenapi-types more lightweight.
Reviewed By: sfilipco
Differential Revision: D25857917
fbshipit-source-id: d3e15a2b6638cc6e15171a1e9bc37362e03df583
Summary: In upcoming changes, we're moving Id to a separate crate. This makes that easier.
Reviewed By: sfilipco
Differential Revision: D25857918
fbshipit-source-id: 6e2163f6fa171d4cd3be4fc0c4c248fd87ba739c
Summary:
this is needed to remind users to use the commands related to the Commit Cloud Workspaces
This is also needed to avoid confusion - the current error is not very informative:
```
connected to twshared9522.24.frc3.facebook.com session Wfq9tGuoPSBJH8nc
abort: 'listkeyspatterns' command is not supported for the server ssh://hg.vip.facebook.com//data/scm/fbsource?infinitepush_write
```
Reviewed By: markbt
Differential Revision: D26073439
fbshipit-source-id: 58ee477bc021a0796e22cc18610225ee003e06be
Summary: fix unit test that was broken by D19803760 (8e35bcc49d) to unblock release
Reviewed By: krallin
Differential Revision: D26073002
fbshipit-source-id: 88da60f5806ef5cb2f832922e503309836542e09
Summary:
Lots of generated code in this diff. Only code change was in
`common/rust/cargo_from_buck/lib/cargo_generator.py`.
Path/git-only dependencies (ie `mydep = { path = "../foo/bar" }`) are not
publishable to crates.io. However, we are allowed to specify both a path/git
_and_ a version. When building locally, the path/git is chosen. When publishing,
the version on crates.io is chosen.
See https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#multiple-locations .
Note that I understand that not all autocargo projects are published on crates.io (yet).
The point of this diff is to allow projects to slowly start getting uploaded.
The end goal is autocargo generated `Cargo.toml`s that can be `cargo publish`ed
without further modification.
Reviewed By: lukaspiatkowski
Differential Revision: D26028982
fbshipit-source-id: f7b4c9d4f4dd004727202bd98ab10e201a21e88c
Summary:
When we tried to update to Tokio 0.2.14, we hit lots of hangs. Those were due
to incompatibilities between Tokio 0.2.14 and Futures 1.29. We fixed some of
the bugs (and others had been fixed and were pending a release), and Futures
1.30 have now been released, which unblocks our update.
This diff updates Tokio accordingly (the previous diff in the stack fixes an
incompatibility).
The underlying motivation here is to ease the transition to Tokio 1.0.
Ultimately we'll be pulling in those changes one or way or another, so let's
get started on this incremental first step.
Reviewed By: farnz
Differential Revision: D25952428
fbshipit-source-id: b753195a1ffb404e0b0975eb7002d6d67ba100c2
Summary:
Without checking response status codes, we don't really know whether our request got processed as expected.
I am also adding port to url because default path doesn't have it. HTTPS is 443 by default.
Reviewed By: johansglock
Differential Revision: D26046197
fbshipit-source-id: c6082cc221c44c6c06557bccaaebb53efd56b4bf
Summary: Using speedtest http endpoint on mononoke when client is connecting directly to mononoke.
Reviewed By: markbt
Differential Revision: D25899297
fbshipit-source-id: b9e1fb90080c42cb2bd242a1cb3612c63a8f93a4
Summary:
This new crate is part of the new telemetry / logging effort. Its input is
tracing data, and output is aggregated NoSQL table content.
This diff is only the start, setting up the direction.
Reviewed By: DurhamG
Differential Revision: D19797702
fbshipit-source-id: bdf34461c05b5eae5e59652bc82d8ee1857dbf1e
Summary: Suddenly prompt stopped appearing for me. Flush the stream to be sure that it's printed out.
Reviewed By: HarveyHunt
Differential Revision: D25956018
fbshipit-source-id: 83419037fa6ce672e203385b71f1403a738d0c90
Summary:
It's no longer true - we're doing metalog commit in transaction.py, not lock
release. Also rename the function to clarify.
Reviewed By: DurhamG
Differential Revision: D25984806
fbshipit-source-id: b17a3f635210be7855341fc8a47fed6411599164
Summary:
This setup is more extendable than the TracingData focused approach. We can
more easily add new functionality using the Subscriber list.
The approach taken here to introduce the new collector tries to maintain
existing functionality. We can then move various logic to their own
Subscribers.
Reviewed By: quark-zju
Differential Revision: D25988580
fbshipit-source-id: 045cd355dbd499109e554a29a1439c2d490b7c40
Summary:
Dot `.` is the common separator for the metrics aggregator that we use.
This adds some form of consistency.
Reviewed By: DurhamG
Differential Revision: D25968398
fbshipit-source-id: 194d2f33fe477fe5d768a9cd8f9f46f56445e3e8
Summary:
The both options have basically the same value.
This is my next step for resolving mismatches between CC dynamic config and the current configuration and generally improving CC configuration.
Reviewed By: DurhamG
Differential Revision: D25973556
fbshipit-source-id: aae21efcd5174ed58efcb9e5d8c85831d35777ea
Summary:
Make local build is broken with
```
building 'hgmain' binary
error: failed to select a version for the requirement `indicatif = "^0.14"`
candidate versions found which didn't match: 0.15.0
location searched: directory source `/data/users/liubovd/fbsource/fbcode/eden/scm/../../../third-party/rust/vendor` (which is replacing registry `https://github.com/rust-lang/crates.io-index`)
required by package `hgcommands v0.1.0 (/data/users/liubovd/fbsource/fbcode/eden/scm/lib/hgcommands)`
... which is depended on by `hgmain v0.1.0 (/data/users/liubovd/fbsource/fbcode/eden/scm/exec/hgmain)`
perhaps a crate was updated and forgotten to be re-vendored?
error: compilation of Rust target 'hgmain' failed
make: *** [Makefile:68: local] Error 1
```
Reviewed By: krallin
Differential Revision: D25973253
fbshipit-source-id: 42c66dcfe26dc5c7713d4bdedc2b353aa5eb20ed
Summary:
We had a user whose `hg sl` was completely hosed because they didn't have
master anymore (or hardly any remote bookmarks for that matter), which made
everything draft.
Let's make this easier to debug by logging subscriptions to hg rage.
Reviewed By: StanislavGlebik
Differential Revision: D25951745
fbshipit-source-id: d08b6bf7febe47923908b312f1881e3be15188c8
Summary:
I gathered information from various sources in this newdoc entry.
The goal is to have an easy link that gives a general summary of what
Segmented Changelog is and what it does.
Reviewed By: quark-zju
Differential Revision: D25908322
fbshipit-source-id: 8f3d2725afe8db5a9bc179189c287fd54aaca08b
Summary:
The exit condition for looking for the `.arcconfig` file will never be reached
on Windows, as the root directory is usually something like `C:\\`.
Instead, stop if we get to a point where `os.path.dirname` returns the same
thing, as that will probably also be a root.
Reviewed By: singhsrb
Differential Revision: D25947124
fbshipit-source-id: 30f4bf3df0b4119694194bcb1c8bcadca41556e4
Summary:
This feature is useful for testing time-dependent stuff (e.g. it
allows you to stop/forward time). It's already included in the buck build.
Reviewed By: SkyterX
Differential Revision: D25946732
fbshipit-source-id: 5e7b69967a45e6deaddaac34ba78b42d2f2ad90e
Summary:
quark-zju made it much safer to rollout configs recently by enforcing
time-sharding, but one little downside is that it becomes harder to test
them locally (or at diff-time, which I'm working on adding) because the time
shard will usually not pass.
This diffs add an environment variable to force all shards to pass.
Reviewed By: DurhamG
Differential Revision: D25925207
fbshipit-source-id: 343b90165ad2b6bae64b6821eae95c58f7d79698
Summary:
I've been seeing this hundreds of times / day in my Eden logs:
```
2021-01-07 09:19:19,702 Fetching failed, continuing as this may be spurious: Expected type that converts to PySequence but received set
```
Unfortunately, those logs are really not very helpful, because that error
doesn't tell you anything about where it occurred. I do however care about them
because they have correlated with slowness in Eden for me :)
At the same time, this error is a bit important: The last time we had an issue
here, it turned into a SEV. That was because the fallback was buggy, but it
does suggest that we do not expect to be routinely falling back.
So, let's log more.
Reviewed By: xavierd
Differential Revision: D25924077
fbshipit-source-id: f59d81cc51786f94f747142ec4baaca78f3072c2
Summary:
Use the new API to clean up stale logs at open time. This hopefully helps
releasing disk space on Windows if the normal rotation fails to remove
old files being mmap-ed by other processes.
Reviewed By: xavierd
Differential Revision: D25894282
fbshipit-source-id: a3d8247b737dd451ee68b58cc5a38fdd2822c0c3
Summary:
Previously rotation only happens at flush time and file deletion is a best
effort (it might fail on Windows). For use-cases that are sensitive about
space usage that's suboptimal. This diff adds an API to remove old files
manually so high level logic can choose to clean up old files after open().
Reviewed By: xavierd
Differential Revision: D25894283
fbshipit-source-id: fbffff426544b39349ddf3537d46954d3cab5d12
Summary:
Previously, the fsmonitor state update logic will skip updating treestate if the wlock
cannot be obtained. D17468790 (8d4d0a66a2) made it wait for wlock for the painful "watchman fresh
instance" case. But things can still suck if it's not a "fresh instance" but there are just
too many nonnormal files.
This diff makes it that exceeding a threshold of nonnormal files will trigger a fsmonitor
state write as an attempt to to reduce the number of nonnormal files. In additional,
`--debug` was changed to print more internal states for debugging.
This would hopefully address issues where people have a large "nonnormal"
treestate, suffers from the bad performance issue and cannot recover from it
automatically.
Reviewed By: DurhamG
Differential Revision: D25794083
fbshipit-source-id: 741426cf31484d9318f9cfcab11d38da33ab5067
Summary: We don't use obsstore. Its size is irrelevant.
Reviewed By: singhsrb
Differential Revision: D25876066
fbshipit-source-id: 5657c3ca08f5ed1cb5a3d1a5b3395ab74756b7e5
Summary: It's used for malloc / realloc which take size_t instead of unsigned int.
Reviewed By: StanislavGlebik, xavierd
Differential Revision: D25890562
fbshipit-source-id: e2787786e9b995431c50b411d77cbce438a82c98
Summary:
The `pull` commmand has a lot of tech debt (with issues like inefficiency, race
conditions, named branches, etc). The new `repo.pull` API isn't designed to
support all legacy usecases.
This diff switches a subset of `pull` command that the new API can support to
the new API. It should solve race condition or `visibility.add` inefficiency
issues (ex. 20s on addgroup, 187s on visibility.add - P154836357).
Logic related to remotenames was ported from the remotenames extension.
The selectivepull tests seem strong enough to check their behaviors.
The new pull API is used by commit cloud for many months. So I turned the
new code path on by default. It can be turned off by changing
`commands.new-pull`.
There are a few test changes. The new behavior seems more correct overall:
- test-commitcloud-switch-workspace.t
- "M/F" swap is caused by rev number: both are correct
- "S: public remote/stable" is more correct than "S: draft"
- test-commitcloud-sync-rb-deletion.t
- "draft1: draft remote/scratch/draft1" is more correct because
"remote/scratch/draft1" is listed in the "book --list-subs"
command above.
- test-commitcloud-sync-rb-enabling.t
- "public1: draft" not showing up is more correct.
- test-fb-hgext-remotefilelog-lfs-prefetch.t
- Difference seems to be caused by rev order.
Reviewed By: markbt
Differential Revision: D25562089
fbshipit-source-id: ac22b2f0492ab53517d580d706dfbc823fd0e0cc
Summary:
Make the `pull` API update visible heads even if all commits exist locally.
This is a more expected behavior, and will make the "pull" command using the
pull API simpler.
Reviewed By: markbt
Differential Revision: D25562091
fbshipit-source-id: 8a43cfe4afd31d3cb9ad5369a6081de120043168
Summary:
Filters out some obviously public heads at the end of repo.pull so they don't
get passed to `visibility.add`. Note: this just removes some obviously "public"
commits without considering the graph. A stronger guarantee would be removing
public heads at metalog commit time.
Reviewed By: markbt
Differential Revision: D25562092
fbshipit-source-id: cc6a19252fcfe305e3a14895b61ab0d6b84a007e
Summary:
We used to get those in the old (Python) LFS extension, but didn't have them in
the new one. However, this is helpful to correlate requests to LFS with data in
hg logs. It's also convenient to be able to identify whether a set of requests
are part of the same session or not.
This diffs threads the client correlator through to the LFS store from the
Python, similarly to how it's done in EdenAPI.
Reviewed By: DurhamG
Differential Revision: D25804930
fbshipit-source-id: a5d5508617fa4184344834bbd8e3423816aa7668
Summary: The dependency that we want to share is 0.14 instead of 0.15.
Reviewed By: singhsrb
Differential Revision: D25871110
fbshipit-source-id: 16e9f8a858ee04a47867c2916909edfc996f8bc4
Summary:
Add `manifest_node` to the crdump output, which is the root manifest node of the commit.
This is useful for detecting commits that have the same tree content but
different metadata (e.g. if only the commit message has been edited).
Reviewed By: singhsrb
Differential Revision: D25782674
fbshipit-source-id: dfdf426833533140b676eee82e123a0cba23c77a
Summary:
commit cloud commands shouldn't fail if the set is empty
the bug was introduced by using the function bookmarks.selectivepullbookmarknames that has a check at the end that the set is not empty, commit cloud doesn't need the check
the issue introduced D25802193 (713faa71cd) via code refactoring
Differential Revision: D25853480
fbshipit-source-id: 7d3f057dead097b86269e7b03d78f5523e8f1ec7
Summary:
Rough progress reporting. The progress bars are straight coming from the
`indicatif` crate. Integrating with the `IO` object is not trivial because
we only have a reference. It gets tricky. I think that it makes sense for
us to expand the IO object to something that is more than a `Box<dyn Write>`.
We have about 3 scenarios:
1. Write object that we need to implement interior mutability for to give out
clones.
2. Stdin/Stdout which have their own imlementation for interior mutability.
3. PyObject which has locking already implemented.
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D25840469
fbshipit-source-id: 87f466f06f2c5d4c63ccb3bbc5c009fae41ed002
Summary:
There is a little bug here. We produce a stream of futures of futures, then we
buffer it, which gives us a stream of futures, and then we await the futures
one by one, here:
```
while let Some(next) = stream.next().await {
next.await?
}
```
This is not really correct, because it means we don't actually do fetches
concurrently at all (we just instantiate futures concurrently, but that's not
really async work).
This fixes that by removing one layer of future-ing.
Reviewed By: singhsrb
Differential Revision: D25825895
fbshipit-source-id: 3ad3367f1eb802ce5b9b5288f04fd3705e172537
Summary:
This allows adding progress bars tracking downloads from the server.
We could be smarter in this instance if we were to deserialize on the fly.
The first part of the payload contains the number of idmap entries that we need
but it needs more work to make it clear. The progress object right now is
designed for general bytes.
Reviewed By: quark-zju
Differential Revision: D25840470
fbshipit-source-id: c466c8d606b44981fe63c95352db2d8f14d6071b
Summary:
The most common scenario where we see matcher errors is when we iterate through
a manifest and the user sends SIGTERM to the process. The matcher may be both
Rust and Python code. The Python code handles the interrupt and prevents future
function calls. The iterating Rust code will continue to call matcher functions
through this time so we get matcher errors from the terminated Python stack.
As long as we have Python matcher code, errors are valid.
It is unclear to me whether the matcher trait should have `Result` return
values when all implementations are Rust. It is easy to imagine implementations
that can fail in different circumstances but the ones that we use if we just
port the Python code wouldn't fail.
All in all, I think that this is a reasonable step forward.
Reviewed By: quark-zju
Differential Revision: D25697099
fbshipit-source-id: f61c80bd0a8caa58040a447ed02d48a1ae84ad60
Summary: These globs were lost as part of D25315954 (ec0b533381).
Reviewed By: quark-zju
Differential Revision: D25814934
fbshipit-source-id: b1896893e37e355a73eb136758f8966666e0ec05
Summary:
Unless we can't update to a public root, there is nothing wrong with having local changes and switching workspaces feature.
Those are not related. Uncommited changes shouldn't impact switching workspaces.
Reviewed By: mitrandir77
Differential Revision: D25802406
fbshipit-source-id: 3fcb70864002bed11ad32621947294f643ca1fc3
Summary:
This was a request from users. Repo could go into a disconnected state, for example, if rejoin in fbclone fails due to some reason.
In this case it was confusing that `hg cloud switch` command doesn't work. Users have to run `hg cloud join` command first.
If the repo is disconnected but doesn't contain any relevant local changes for commit cloud, it should be fine to switch workspace.
Reviewed By: mitrandir77
Differential Revision: D25802193
fbshipit-source-id: 3216a10c3438463773602b2dfd13740866fb5908
Summary:
The cert path isn't correctly set up on all platforms, so this can
cause Mercurial to throw an error complaining about missing certs, even when
edenapi isn't enabled.
Let's back this out for now until we can fix the cert paths or only hit this
path when we actually use edenapi.
Reviewed By: singhsrb
Differential Revision: D25792491
fbshipit-source-id: 022a89a089cabcc709a07934eb62b883082261c2
Summary:
Lots of things can look like CBOR data, such as ... strings representing
errors. Right now, if the data in our CBOR stream is actually an error message,
then we'll just ignore it (see details in T80406893).
This isn't how we normally handle invalid data on the stream (we'd raise an
error) — it only happens with trailing data. This fixes our decoding to raise
an error in this case.
Reviewed By: quark-zju
Differential Revision: D25759082
fbshipit-source-id: c3d8be5007112ec1d2e7f25a102d8caaf0dbba56
Summary:
enable switching from a draft commit possible for most of the cases
make it possible if the public root of the current commit is an ancestor of the main bookmark
this condition we need because the remote bookmarks can be different for different workspaces and they define phases
I think it will cover most of workflows
Reviewed By: mitrandir77
Differential Revision: D25780999
fbshipit-source-id: b1c25b29a7668d51244ca43d6b0c30fa2fc068d9
Summary: Skip some very long configs to make rage output cleaner.
Reviewed By: DurhamG
Differential Revision: D25625452
fbshipit-source-id: 44bf8b9f93d9cb06d065a89f5d0ffa53ad6d6286
Summary:
In the previous diff I had to make the same change in two places, this change
deduplicates the code so we can reuse the change. This isn't 100% equivalent,
since now we have 2 layers of boxing on the stream in `Fetch`.
That being said, that seems quite unlikely to matter considering that this is
ultimately handling responses that came to us over HTTP, so one pointer
traversal seems to be reasonable overhead (also, similar experience in Mononoke
suggests it really does not matter).
Reviewed By: quark-zju
Differential Revision: D25758652
fbshipit-source-id: 399ead1b67ffbb241597615a29129411580cf194
Summary:
This updates the edenapi fetch mechanism to check status codes from the server.
If the server responds with an error, we propagate the error up to the caller.
This is equivalent to what we would do if e.g. the server had just crashed.
Reviewed By: quark-zju
Differential Revision: D25758653
fbshipit-source-id: f44f6384be7944dce670c3825ccbb60b5fa2090a
Summary:
Make sure we give more explanation to users to they can self-fix any errors
related to certificates that might pop up.
Reviewed By: xavierd
Differential Revision: D25758517
fbshipit-source-id: 3b9929be3d1c0c44a5e13cc9c1e7b2a4f785abf4
Summary:
Like it says in the title. Note that I did *not* retry stuff like resolving
hosts or connecting, so this should only really temporary blips in
connectivity. We probably shouldn't go much beyond that at a low level like
this.
Reviewed By: HarveyHunt
Differential Revision: D25615915
fbshipit-source-id: 78c33eff2e9ce380a260708e9fbeb929eede383c
Summary:
This is the goal of this stack: retry errors that occur when Curl detects that
the transfer speed is too low. This should let us eventually set a much higher
timeout on overall request completion, thus ensuring that we don't uploads
that make progress, all the while aborting connections early and retrying them
if they are legitimately stuck.
Reviewed By: farnz
Differential Revision: D25615790
fbshipit-source-id: fe294aee090758b1a3aef138788ac2926c741b79
Summary:
Right now, the error handling in LFS doesn't handle e.g. transfer timeouts. I'd
like us to support that, notably so that we can have curl require a minimum
transfer speed and retry if we fail.
To do so, I need to be able to capture the errors and figure out if they're
retryable. Right now, everything is either a `FetchError` that includes a HTTP
status and URL, or just an `Error` that aborts.
This diff introduces a `TransferError` that explains why a transfer failed and
can be used for retry decisions. We later add the request details if we decide
to not retry and turn it into a `FetchError`.
Reviewed By: xavierd
Differential Revision: D25615789
fbshipit-source-id: e4a2f4f16a34ca2f86bd61491bb26e7f328dec63
Summary:
Like it says in the title. This adds support for setting a min-transfer-speed
in Curl. My goal with this is to fix two problems we have:
- a) Uploads that timeout on slow connections. Right now we set a transfer
timeout on requests, but given files to upload can be arbitrarily large, that
can fail. This happened earlier this week to a user (T81365552).
- b) Transfer timeouts in LFS. Right now, we have a very high timeout on
requests and we can't lower it due to this problem with uploads. Besides,
the reason for lowering the timeout would be to retry thing, but right now
we don't support this anyway.
Reviewed By: xavierd
Differential Revision: D25615788
fbshipit-source-id: 57d75ee8f522cf8524f9d12103e34b0765b6846a
Summary:
I'd like to make it a little easier to add more options without having to
thread them all the way through to the HTTP transfer callsite.
Reviewed By: xavierd
Differential Revision: D25615787
fbshipit-source-id: 4c6274dc2e6b5ba878e0027aae9a08b04f974463
Summary:
The intention was to sort entries by Dag Id entry. This was instead sorted
lexicographically.
Reviewed By: quark-zju
Differential Revision: D25684784
fbshipit-source-id: 0a3db6398aec7d8df080bbb2366e41660483608c
Summary:
`PartialOrd` was suggested by sfilipco. Note `Option<std::cmp::Ordering>` is
similar to `Side` in terms of expressiveness. `PartialOrd` can be written
using shorter symbols (`<=`, etc) so it's easier to understand.
The `compatible` family APIs were replaced by `partial_cmp` APIs.
There are some minor differences:
- Bitwise or used by union set is no longer supported. `Hints::union` was
added as a replacement.
- `Option<T>` implements full order. `Some(T) > None`. This is different
from `compatible_dag` and `compatible_id_map` APIs. Additional `> None`
checks were added for correctness.
Reviewed By: sfilipco
Differential Revision: D25652784
fbshipit-source-id: 51d88948fa556300678050088c06e9dda09cbf98
Summary:
```
warning: variable does not need to be mutable
--> eden/scm/lib/configparser/src/config.rs:448:21
|
448 | let mut values_copy = values.clone();
| ----^^^^^^^^^^^
| |
| help: remove this `mut`
|
= note: `#[warn(unused_mut)]` on by default
```
Reviewed By: sfilipco
Differential Revision: D25625453
fbshipit-source-id: 8475056a87095f9ba633282666e6d3fee864074b
Summary:
Some code paths use (expensive) snapshot to be compatible with `Arc::ptr_eq`
compatibility check. With `VerLink` it's more efficient to use `VerLink`
directly. This is potentially more efficient for `VerLink` too because the
`Arc` won't be cloned unnecessarily and `VerLink::bump()` is more likely to
use its optimized path.
Reviewed By: sfilipco
Differential Revision: D25608200
fbshipit-source-id: 1b3ecc5d7ec5d495bdda22d66025bb812f3d68a0
Summary:
Similar to the previous change. `VerLink` tracks compatibility more accurately.
- No false positives comparing to the current `map_id` approach.
- Less false negatives comparing to the previous `Arc::ptr_eq` approach.
The `map_id` is kept for debugging purpose.
Reviewed By: sfilipco
Differential Revision: D25607513
fbshipit-source-id: 7d7c7e3d49f707a584142aaaf0a98cfd3a9b5fe8
Summary:
Previously, snapshots need to be invalidated manually. That is error-prone.
For example, `import_clone_data` forgot to call `invalidate_snapshot`.
With `VerLink`, it's easy to check if snapshot is up-to-date. So let's just
use that and remove the need of invalidating manually.
`invalidate_snapshot` is still useful to drop `version` in `snapshot` so
`VerLink::bump` might be more efficient. Forgetting about it no longer affects
correctness.
Reviewed By: sfilipco
Differential Revision: D25607514
fbshipit-source-id: 5efb489cda1d4875bcd274c5a197948f67101dc1
Summary:
`VerLink` tracks compatibility more accurately.
- No false positives comparing to the current `dag_id` approach.
- Less false negatives comparing to the previous `Arc::ptr_eq` approach.
The `dag_id` is kept for debugging purpose.
Note: By the current implementation, `dag.flush()` will make `dag`
incompatible from its previous state. This is somewhat expected, as
`flush` might pick up any changes on the filesystem, reassign non-master. Those
can be actually incompatible. This might be improved in the future to detect
reload changes by using some extra information.
Reviewed By: sfilipco
Differential Revision: D25607511
fbshipit-source-id: 3cfc97610504813a3e5bb32ec19a90495551fd3a
Summary:
There are 2 kinds of changes:
- Append-only changes. It is backwards-compatible.
- Non-append-only changes. It is not backwards-compatible.
Previously,
- `Arc::ptr_eq` on snapshot is too fragile. It treats append-only compatible
changes as incompatible.
- Even worse, because of wrapper types (ex. `Arc::new(Arc::new(dag))` is
different from `dag`), even a same underlying struct can be treated as
incompatible.
- `(map|dag)_id` is too rough. It treats incompatible non-append-only changes
as compatible.
Add `VerLink` to track those 2 different kinds of changes. It basically keeps a
(cheap) tree so backwards compatible changes will be detected precisely.
`VerLink` will replace IdMap and Dag compatibility checks.
Reviewed By: sfilipco
Differential Revision: D25607512
fbshipit-source-id: 478f81deee4d2494b56491ec4a851154ab7ae52d
Summary: This makes it easier to investigate fast path issues.
Reviewed By: sfilipco
Differential Revision: D25598077
fbshipit-source-id: 27b7042fb9510321c25371f8c5d134e248b3d5d5
Summary:
This makes it easier to check if set operations are using fast paths or not by
setting `RUST_LOG=dag=debug`.
Reviewed By: sfilipco
Differential Revision: D25598075
fbshipit-source-id: 1503a195268c0989d5166596f2c8a66e15201372
Summary:
See the previous diff for context. The new API will be used to check if two
dags are compatible.
Note: It can cause false positive on compatibility checks, which need a
more complex solution. See D25607513 in this stack.
Reviewed By: sfilipco
Differential Revision: D25598079
fbshipit-source-id: f5fc9c03d73b42fadb931038fe2e078881be955f
Summary: The backend is designed to be used by the "debugsegmentclone" command, which does not write revlog.
Reviewed By: sfilipco
Differential Revision: D25624786
fbshipit-source-id: e145128c7b41d78fed495f8da540169f741b674d
Summary: This makes it possible to add new commits in a repo without revlog.
Reviewed By: sfilipco
Differential Revision: D25602527
fbshipit-source-id: 56c27a5f00307bcf35efa4517c7664a865c47a43
Summary:
When running `python3 run-tests.py test-run-tests.py`, some bytes were printed
with `b` prefix. Convert them to `str`.
Reviewed By: DurhamG
Differential Revision: D25642164
fbshipit-source-id: f1103b24ad88d0d024f6be546bf632141f06ebd1
Summary:
The warning will go to debug level logs if the delay is not reached.
The messages about the locks make profoundly bad effect on attitude to commit cloud even if the delay is just 1 second (that is a reasonable delay).
Reviewed By: quark-zju
Differential Revision: D25587459
fbshipit-source-id: 9a09484d590ba04d17a881e0c9c5d543686b934f
Summary:
configs.allowedlocations restricts what configs can be loaded to a
certain set of files. This will enable us to deprecate all old config locations.
This diff adds Python support and a high level test.
Reviewed By: quark-zju
Differential Revision: D25539736
fbshipit-source-id: fa2544379b65672227e0d9cf08dad7016d6bbac8
Summary:
We want to start disallowing non-approved config files from being
loaded. To do that, let's update the config verifier to accept an optional list
of allowed locations. If it's provided, we delete any values that came from a
disallowed location.
This will enable us to prune our config sources down to rust configs,
configerator configs, .hg/hgrc, and ~/.hgrc.
Reviewed By: quark-zju
Differential Revision: D25539738
fbshipit-source-id: 0ece1c7038e4a563c92140832edfa726e879e498
Summary:
Those messages like "pulling from ...", "added n commits ..." belong to stderr.
This makes it possible for us to turn on verbose output for auto pull, without
breaking tools that parses stdout.
Reviewed By: sfilipco
Differential Revision: D25315955
fbshipit-source-id: 933f631610840eb5f603ad817f7560c78b19e4ad
Summary:
It turns out `Arc::ptr_eq` is becoming unreliable, which will cause fast paths
to be not used, and extreme slowness in some cases (ex. `public & nodes`
iterating everything in `public`).
This diff adds an API for an IdMap to tell us its identity. That identity is
then used to replace the unreliable `Arc::ptr_eq`.
For an in-memory map, we just assign a unique number (per process) for its
identity on initialization. For an on-disk map, we use the type + path to
represent it.
Note: strictly speaking, this could cause false positives about
"maps are compatible", because two maps initially cloned from each other
can be mutated differently and their map_id do not change. That will
be addressed in upcoming diffs introducing a more complex but precise way to
track compatibility.
Reviewed By: sfilipco
Differential Revision: D25598076
fbshipit-source-id: 98c58f367770adaa14edcad20eeeed37420fbbaa
Summary:
With selectivepull we can tell visibility directly about which heads to add,
instead of adding all nodes in a changegroup.
Note: This does not affect the pull command, nor exclude public commits yet.
Reviewed By: markbt
Differential Revision: D25562090
fbshipit-source-id: aa5f346f33058dfdb3b2f23f175e35b5d3c30a1d
Summary: They will be used in core later.
Reviewed By: markbt
Differential Revision: D25562093
fbshipit-source-id: 4402a629a09920fd4c6f85cb8e777446bb218a37
Summary:
It always takes a bit of time to find the logs. Since we do have scm daemon onces in the output of `hg cloud status`,
it would be nice to have the onces from background backup as well.
Reviewed By: markbt
Differential Revision: D25560145
fbshipit-source-id: cdf5d76c7c3ebb1492559d32935f9301452a1cd5
Summary: The order has been incorrect and led to a confusing message
Reviewed By: krallin
Differential Revision: D25559963
fbshipit-source-id: 4fcb3e53cedcb08675b60b25cbb5da2ca52c08ed
Summary:
IPython is incompatible with Python 3 demandimport. Disable demandimport to
make it work.
Reviewed By: singhsrb
Differential Revision: D25542394
fbshipit-source-id: 293880dff62e98895bc1ae2d3328d4af25b8218f
Summary:
Debug output belongs to stderr.
This makes it possible to turn on debug output without breaking programs
parsing stdout.
Reviewed By: singhsrb
Differential Revision: D25315954
fbshipit-source-id: c7813a824fbf6640cb5b80b5ed2d947e7059d53e
Summary:
With `collapse-obsolete`, `.` can be obsoleted and in the middle of a stack and
not shown up. That can be confusing. Make the smartlog revset always show the
`heads` passed in. If `.` is in `heads` (the default), then show it.
Reviewed By: DurhamG
Differential Revision: D24696595
fbshipit-source-id: 7deab109d0e0ae5e703928252bc63312d936955f
Summary: * suggest connecting to the VPN if hg update doesn't work
Reviewed By: sfilipco
Differential Revision: D25551017
fbshipit-source-id: 575f29cce4ab2719f2faae86616fdd9aac739f5f
Summary:
If Rust LFS is in use, we currently don't upload LFS blobs to commit cloud.
This is problematic because if you're going to Mononoke that means you can't
upload, and if you're going to Mercurial that means you're silently not backing
up data.
Reviewed By: StanislavGlebik
Differential Revision: D25537672
fbshipit-source-id: fd61f5a69450c97a0bc0895193f67fd22c9773fb
Summary:
The config verifier would remove items from the values list if they
were disallowed. To do this, it iterated through the values list backwards,
removing bad items. In some cases it stored the index of a bad value for later
use, but because it was iterating backwards and removing things, the indexed it
stored might not be correct by the time the loop is done. To fix this, let's go
back to iterating forwards.
Reviewed By: quark-zju
Differential Revision: D25539737
fbshipit-source-id: 87663f3c162c690f3961b8075814f3467916cb4b
Summary:
Restore the behavior disabled by D25350916 (49c6f86325). This time it no longer runs Python
logic in background threads.
Reviewed By: sfilipco
Differential Revision: D25513054
fbshipit-source-id: 0220ccb37e658518d105bba04f45424c9fcfe142
Summary:
Make `get_commit_raw_text` aware of hg's hardcoded commit hashes: NULL_ID and
WDIR_ID. Previously, only `stream_commit_raw_text` is aware of it.
This makes it a bit more compatible when used in more places.
Reviewed By: sfilipco
Differential Revision: D25515006
fbshipit-source-id: 08708734a28f43acf662494df69694988a5b9ca0
Summary:
Unlike streamcommitrawtext, the new API does not put Python logic to a
background thread. This will make it easier to reason about Python logic as
they do not need to be thread-safe, and we don't need to think about Python GIL
deadlocks in the Rust async world.
Reviewed By: sfilipco
Differential Revision: D25513057
fbshipit-source-id: 4b30d7bab27070badd205ac1a9d54bae7f1f8cec
Summary:
Previously, only the batch fetching, or the stream fetching APIs will
actually fetch commit remotely. The 1-commit fetching API does not have
the network side effect, with the hope that we can migrate all usecases
to stream or batch fetching.
Practically it's quite difficult to migrate all use-cases, and the Python
layer has to have a fallback 1-by-1 fetching. Now let's just move that
fallback to Rust to simplify the code. The fallback in the Rust code
is by the default impl of get_commit_raw_text.
Reviewed By: sfilipco
Differential Revision: D25513056
fbshipit-source-id: b3c615397d33b8d35876dc23ca7b95173783ef80
Summary: The API will be used in Python bindings to avoid running Python in background threads.
Reviewed By: sfilipco
Differential Revision: D25513055
fbshipit-source-id: a108b55115271a256c0d43e0ff7b82c0b209be81
Summary:
Previously only `iterctx` does prefetch. Make `__iter__` do prefetch via `iterctx`.
The old `__iter__` without prefetching was renamed to `iterrev`.
Reviewed By: sfilipco
Differential Revision: D24365404
fbshipit-source-id: db5c687066794257719bb64c673dc384b5460ff1
Summary:
Now smartset has a reference to repo. It does not need `repo` from external
source.
Reviewed By: sfilipco
Differential Revision: D24365405
fbshipit-source-id: 8a43697b7b84a8a41691ed8f095c271107a90f16
Summary:
This will make `__iter__` to be able to do proper prefetch, or make it possible
for `__iter__` to return node instead of rev. To avoid cycles, weakref is used.
The smartset types are used widely. It's hard to migrate all callsites at once.
For now, `repo` is optional. Later, it will be required.
Reviewed By: DurhamG
Differential Revision: D24365400
fbshipit-source-id: 5dd40e3d930893c39f16da8f3169b026c8933bd2
Summary:
The test is one of the most fragile tests (ex. it has traceback that is
fragile). Since we don't support 3rd party extensions the test is less
meaningful. Let's remove it.
Reviewed By: singhsrb
Differential Revision: D25514727
fbshipit-source-id: cc3491597ba38ad84c4c2b69a275da6505ae2207
Summary:
It has a high chance of failing with:
```
$ hg --config extensions.b=b.py --config 'pager.pager=head -1' init foo2
+ close failed in file object destructor:
+ IOError: [Errno 9] Bad file descriptor
line1
```
when running with chg together with other tests.
Reviewed By: singhsrb
Differential Revision: D25514728
fbshipit-source-id: d9d2adea97bc6a436341be6f6d9aac43cb78c90a
Summary:
This is a bit tricky - look-ahead does not work because the content of commit
texts decide whether the commit is in the resulting set or not.
Add a special case for `ancestorsaged` so it does not end up with 1-by-1
fetching.
Reviewed By: sfilipco
Differential Revision: D24324797
fbshipit-source-id: 844ac6a6637e25900eea00d99704a59dfc7a0345
Summary:
Some revset function need commit text data to be efficient. Prefetch commit
text explicitly for them.
Reviewed By: DurhamG
Differential Revision: D24322227
fbshipit-source-id: 49c0b4f8cd9d9a744e10d169a1b1247faa8edd8e
Summary: Looks like the permissions are different there. Let's glob it out.
Reviewed By: singhsrb
Differential Revision: D25507359
fbshipit-source-id: 6a5c19e41879798b829d9b6e79eba3009249c20c
Summary:
At the moment "hg pull -B bookmark" always fetches from infinitepush path even
if we do something like "hg pull -B master".
Let's fetch from infinitepush only if a bookmark matches scratch mather
Reviewed By: markbt
Differential Revision: D25460577
fbshipit-source-id: 6563dcd3423c6a7a70ea1c1f7acdaf5db5e21875
Summary: Make the auth crate validate the user's certificate before returning it. This way we can catch invalid certs before trying to use them.
Reviewed By: sfilipco
Differential Revision: D25454687
fbshipit-source-id: ad253fb433310570c20f33dbd0d0bf11df21e966
Summary: Add a new module that can parse X.509 certificates and detect common issues (e.g., the certificate is missing, corrupt, or expired). This should allow us to provide better UX around certificate errors.
Reviewed By: sfilipco
Differential Revision: D25440548
fbshipit-source-id: b7785fd17fa85f812fd38de09e79420f4e256065
Summary: This makes it more flexible.
Reviewed By: kulshrax
Differential Revision: D24467604
fbshipit-source-id: 63023cf0dde2fb7eac592ac79008e4b7a62340c1
Summary:
When a blob is redacted server side, the http code 410 is returned.
Unfortunately, that HTTP code wasn't supported and caused Mercurial to crash.
To fix this, we just need to store a placeholder when receiving this HTTP code
and simply return it up the stack when reading it.
Reviewed By: DurhamG
Differential Revision: D25433001
fbshipit-source-id: 66ec365fa2643bcf29e38b114ae1fc92aeaf1a7b
Summary:
There was a bug with local-data indexedlog storage where it
wasn't applying the appropriate suffix, so tree data was being stored in
.hg/store/indexedlogdatastore just like file data. Let's fix that and add a
test.
Reviewed By: quark-zju
Differential Revision: D25469917
fbshipit-source-id: 731252f924f9a8014867fc077a7ef10ac9870170
Summary:
The integration tests rely on the ordering, but watchman doesn't
define an ordering in its file results, so the current implementation is
implicitly tied to the iteration order of the InMemoryView indices inside
watchman.
In D25414061 that implementation is changed to be derived from the order
in which entries are added to a change journal, making the results more
likely to match the readdir directory iteration order.
Since the blackbox fsmonitor integration tests have a defined ordering in their
output, we need to sort these results here.
Reviewed By: singhsrb
Differential Revision: D25468807
fbshipit-source-id: aa83d82cca0ecdbfd1d4bb1309b9f3c31f6c3136
Summary: Make the parent function used by various graph building functions async.
Reviewed By: sfilipco
Differential Revision: D25353612
fbshipit-source-id: 31f173dc82f0cce6022cc2caae78369fdc821c8f
Summary:
It is no longer needed for building segments (replaced by "prepared flat
segments"). Remove it.
Reviewed By: sfilipco
Differential Revision: D25353613
fbshipit-source-id: aede9e33c3217a61b5b14aae5b128d8953bc578e
Summary: Make IdConvert async and migrate all its users.
Reviewed By: sfilipco
Differential Revision: D25350915
fbshipit-source-id: f05c89a43418f1180bf0ffa573ae2cdb87162c76
Summary: This will make it easier to make IdConvert async.
Reviewed By: sfilipco
Differential Revision: D25350912
fbshipit-source-id: fbaf638b16a9cf468b7530b19d699b7996ddc4f1
Summary: This will make async migrating easier.
Reviewed By: sfilipco
Differential Revision: D25350913
fbshipit-source-id: f33bdc0023ae0cc49601504b811991ea6813ff9e
Summary: This will make it easier to make IdConvert async.
Reviewed By: sfilipco
Differential Revision: D25350914
fbshipit-source-id: 9f2957731f13a28fdfab834de19763b8afcf8ffa
Summary: This will make it easier to make IdConvert async.
Reviewed By: sfilipco
Differential Revision: D25345239
fbshipit-source-id: 684a0843ae32270aa9b537ef9a2b17a28c027e51
Summary: This will make it easier to make IdConvert async.
Reviewed By: sfilipco
Differential Revision: D25345232
fbshipit-source-id: b8967ea51a6141a95070006a289dd724522f8e18
Summary:
Update DagAlgorithm and all its users to async. This makes it easier to make
IdConvert async.
Reviewed By: sfilipco
Differential Revision: D25345236
fbshipit-source-id: d6cf76723356bd0eb81822843b2e581de1e3290a
Summary:
Make it possible to use async functions in MetaSet functions.
It will be used when DagAlgorithm becomes async.
Reviewed By: sfilipco
Differential Revision: D25345229
fbshipit-source-id: 0469d572b56df21fbdbdfae4178377e572adbcda
Summary: This will make it easier if the `Set` passed in requires async.
Reviewed By: sfilipco
Differential Revision: D25345230
fbshipit-source-id: a327d4e5d425b7eb5296b2fbe25c446492aa9ea7
Summary: This makes it easier to make DagAlgorithm async.
Reviewed By: sfilipco
Differential Revision: D25345234
fbshipit-source-id: 5ca4bac38f5aac4c6611146a87f423a244f1f5a2
Summary: This will make it easier to use `.await` if part of `dag` becomes async.
Reviewed By: sfilipco
Differential Revision: D25345237
fbshipit-source-id: 7f07cdaa9c2e0468667638066611fabe3a3f7f28
Summary: `impl Trait` does not work with `async_trait`.
Reviewed By: sfilipco
Differential Revision: D25345238
fbshipit-source-id: e7890dbaeb162d44e072ea4428d045004608719b
Summary: This makes it easier to migrate to async.
Reviewed By: sfilipco
Differential Revision: D25345228
fbshipit-source-id: e819f0de5f805377a6977325216ef11b14d68c1d
Summary:
Marking IdConvert Sync makes it possible to be used as a trait object with async-trait.
See https://docs.rs/async-trait/0.1.41/async_trait/#dyn-traits
`dag` uses a lot `dyn DagAlgorithm`. In the future when async is used more, the
trait object will be required to be Send or Sync. Just require it on the trait
to make our life easier.
Marking `IdDagStore` as Send + Sync makes async migration easier.
Reviewed By: sfilipco
Differential Revision: D25345231
fbshipit-source-id: 45b96057907cbe2a1d38fd424e7d4c963dd1b245
Summary: Use async function for the PrefixLookup trait.
Reviewed By: sfilipco
Differential Revision: D24840820
fbshipit-source-id: d22cac9f11b06e3127fa956e3f116cf232214125
Summary: This makes the trait objects slightly easier to use.
Reviewed By: sfilipco
Differential Revision: D24840821
fbshipit-source-id: 22fcdf13b62420302b562c309874e08360d02372
Summary: This makes `dyn IdConvert` include `PrefixLookup`.
Reviewed By: sfilipco
Differential Revision: D24840819
fbshipit-source-id: 8d4e25c534f6e4397ec6f643eb3aa116bff12a2c
Summary:
In the future, when async APIs are used, Python bindings will have lifetime
issues. Make it possible to clone the IdMap so the Python bindings can be made
to work.
Reviewed By: sfilipco
Differential Revision: D24840822
fbshipit-source-id: 6aa4e369c877c428ed39d2cbea79e6943836afa8
Summary: This makes NameSet more friendly for async use-cases, interface-wise.
Reviewed By: sfilipco
Differential Revision: D24806695
fbshipit-source-id: 6e640ba2666872a9128d6460e8b53d6a0e595e56
Summary:
Change the main API of NameSet to async. Use the `nonblocking` crate to bridge
the sync and async world for compatibility. Future changes will migrate
Iterator to async Stream.
Reviewed By: sfilipco
Differential Revision: D24806696
fbshipit-source-id: f72571407a5747a4eabe096dada288656c9d426e
Summary:
This will make it easier to incrementally migrate APIs to async, and make sync
programs able to use sync interface if they'd like to. For example, the dag
crate is likely going to be async-ize. Its async APIs are only useful for
places where the dag is lazy. It's still possible to have a non-lazy dag
that a sync interface is suitable.
Reviewed By: ikostia
Differential Revision: D24777747
fbshipit-source-id: 6d56149fbd1f9b29f5fc62387cbf194800755d12
Summary:
```
warning: attribute should be applied to a function or static
--> pyhgtime/src/lib.rs:70:13
|
70 | #[no_mangle]
| ^^^^^^^^^^^^
71 | static timezone: c_long;
| ------------------------ not a function or static
|
= note: `#[warn(unused_attributes)]` on by default
```
Reviewed By: ikostia
Differential Revision: D25345233
fbshipit-source-id: 4b6f753544af1e3e8479dceed908299f6dc57ad5
Summary:
We already have a config to write shared history to an indexedlog. Let's
add a similar config for local history.
Reviewed By: quark-zju
Differential Revision: D23910457
fbshipit-source-id: eca21db0350895f573a60e4e1a6b05514e70f1c7
Summary:
In a future diff we'll use the indexedlog stores for local history. We
want those to exist forever, so let's move IndexedLogHgIdHistoryStore to use a
Store under the hood, and add an enum for distinguishing between the two types
at creation time.
Reviewed By: quark-zju
Differential Revision: D25429675
fbshipit-source-id: 5f2dc494e1175d4c1dc74992d3311d2e55d784ca
Summary: Previously, the EdenAPI client (and Mercurial's `http-client`) required both a client certificate and private key to be specified individually in order to use TLS mutual authentication. However, libcurl supports having both the certificate and key concatenated together into a single PEM file. This diff makes it possible to simply specify the combined file as the `cert`, leaving the `key` unset.
Reviewed By: quark-zju
Differential Revision: D25323786
fbshipit-source-id: 1800be3ef82ec4dfa89d725f5860190172994c89
Summary: Treat empty paths as `None`, which allows certificate and key paths to be unset via a `--config` flag (e.g. `hg debughttp --config auth.edenapi.key=`). This would normally require adding an `%unset` line to the appropriate hgrc, which adds friction to ad-hoc command line usage.
Reviewed By: quark-zju
Differential Revision: D25416531
fbshipit-source-id: 15aae78d9b3a82227278f33def45fa960aa65a98
Summary:
The f64 compatibility added by D25079001 (e91552fefe) changes the `{rev}` template output.
However, the revset object is not aware of the rev mapping so the following
pattern will stop working:
ifcontains(rev, revset('.'), '@', 'o')
# rev: mapped, but not contains in `revset('.')`
Fix it by teaching `ifcontains(a, b, ...)` to disable f64 compatibility when
evaluating `a`, since `a` is not going to be printed out.
This fixes an issue in VSCode ISL that "You are here" is missing on non-master
commits.
Reviewed By: singhsrb
Differential Revision: D25404673
fbshipit-source-id: 3e53a2ce1f135f8825c195c5a3061dad0359c4b2
Summary:
The code still took a dependency on Mercurial's old manifest code to parse
manifests. It turns out the manifests have a very simple format that we could
parse directly.
This avoids various copies, conversions, std::list, removes ~1k lines of code,
at the expense of adding ~100 lines of code (some of them being C++
boilerplate).
Reviewed By: fanzeyi
Differential Revision: D25385018
fbshipit-source-id: 90d4cda2b7797584bc48c086d5592a7ecaa05dfc
Summary:
We already have a config to write shared data to an indexedlog. Let's
add a similar config for local data.
Reviewed By: xavierd
Differential Revision: D23909569
fbshipit-source-id: 87317554beb6bef8237e6a900403701662c3c0d0
Summary:
We temporarily dropped repair support when transitioning to using Store instead
of a raw RotateLog. Let's add that back now.
Reviewed By: xavierd
Differential Revision: D25371622
fbshipit-source-id: e28fc425a6ffb50c93540672b0df75a172ebbe9c
Summary:
In a future diff we'll use the indexedlog stores for local data. We
want those to exist forever, so let's move IndexedLogHgIdDataStore to use a
Store under the hood, and add an enum for distinguishing between the two types
at creation time.
Reviewed By: xavierd
Differential Revision: D23915622
fbshipit-source-id: 296cf6dfcd53e5cf1ae7624fdccedf0a60a77f22
Summary:
In a future diff we'll be moving the IndexedLogHgIdDataStore to use the Store
type (that hides the differences between IndexedLog and RotateLog). To do so, it
needs to support repairs. Let's do some minor refactoring to enable this.
Reviewed By: xavierd
Differential Revision: D25371623
fbshipit-source-id: 846cb5f8c21f1e6b550a45259cc8da24cc65b13b
Summary:
The end goal is to have clients using a sparse IdMap. There is still some work
to get there though. In the mean time we can test repositories that don't use
any revlogs. The current expections for those repositories are that they have
a full idmap locally.
Reviewed By: quark-zju
Differential Revision: D25075341
fbshipit-source-id: 52ab881fc9c64d0d13944e9619c087e0d4fb547c
Summary:
It uses the Python iterator -> Rust stream code path, which can might
cause deadlock with async_runtme::block_on_exclusive. Avoid running
main Python logic in multiple threads.
The prefetching is only useful for lazy commit text backend, which hasn't been
rolled out yet. So disabling it won't cause pref regressions.
Note: This should not affect Phabricator Status prefetching.
Reviewed By: sfilipco
Differential Revision: D25350916
fbshipit-source-id: 49e9c4fea5ff48e1f579aa602693d38c7dd96fc7
Summary:
This commit adds a new eden configuration option that
controls whether we try to load our edenfs.kext in preference to
an alternative fuse implementation on macOS.
The majority of this diff is plumbing to convey the configuration
value through to the privhelper, which is relatively restrictive
due to its root-ness.
I've also updated watchman and mercurial to be aware of the new
filesystem type that shows up in the mount table.
Reviewed By: genevievehelsel
Differential Revision: D25065462
fbshipit-source-id: 4f35b9440654298e2706a0d0613d97eb63451999
Summary: If response is not HTTP/1.1 or isn't Weboscket upgrade we should report error.
Reviewed By: HarveyHunt
Differential Revision: D25368729
fbshipit-source-id: 62dcd93240902924312f4de2965f58357c46c98b
Summary:
Further down the following check is executed:
priority > bestpriority
When priority are strings, this leads to wrong expectations:
$ python3
>>> a = "90"
>>> b = "100"
>>> a > b
True
Reviewed By: krallin
Differential Revision: D25334682
fbshipit-source-id: 48287d57ae6a938e9f2619babf49d073534f46d7
Summary:
Now that we don't ship Python 2 rpms, let's remove the build and test
targets.
Reviewed By: singhsrb, xavierd
Differential Revision: D25313325
fbshipit-source-id: 876385ccb6cb7675fef8d93978b372a3085764b0
Summary:
T53602763 is about hg shelve and other similar
move-off-then-back-on-a-commit situations to get reported incorrectly to
watchman on eden checkouts. This can cause issues with buck and other tools.
The bug is in Eden, but let's try to save people time debugging by telling them
of the risk.
Reviewed By: fanzeyi
Differential Revision: D25337932
fbshipit-source-id: e293c43b5c87bea26564e1efd45b7a983862a442
Summary: This allows us to be more flexible in choosing authentication and expands variables used in configuration.
Reviewed By: singhsrb
Differential Revision: D25304008
fbshipit-source-id: 636893a9eaec31ca5acfa02f72931d5e56b695d0
Summary:
Update pywatchman/bser.c to define `PY_SSIZE_T_CLEAN` before including
`Python.h` and then receive parsed string lenghts as `Py_ssize_t` rather than
`int`. This eliminates some runtime Python warnings about the old `int`
behavior being deprecated.
Reviewed By: chadaustin
Differential Revision: D25294778
fbshipit-source-id: 7db678ed918db3bf4745ba716585ed6475d1c910
Summary:
The `repo.names` namespace lookup should not return `[None]`. Detect and avoid
that to avoid crashes.
Reviewed By: singhsrb
Differential Revision: D25261462
fbshipit-source-id: c7a0bba557717ab292a05d753569c68112e7f0b0
Summary:
Introduce `edenapithin` crate, which offers C bindings to EdenAPI Client.
There are two top-level `owned` types, `EdenApiClient` and `TreeEntryFetch`, which represent `Box`ed results from calls to EdenAPI's blocking client. The C / C++ code is responsible for calling the associated `free` functions for these types, as well as `OwnedString`, which is used to represent error variants of a `Result` as a string.
Most functionality is provided through functions which operate on and return references into these top-level owned types, providing access into Rust `Result`s and `Vec`s (manually-monomorphized), and EdenApi's `TreeEntry` and `TreeChildEntry`.
Additional non-pointer types are defined in the `types` module, which do not require manual memory management.
C++ bindings are not included currently, but will be introduced soon.
Reviewed By: fanzeyi
Differential Revision: D24866065
fbshipit-source-id: bb15127b84cdbc6487b2d0e1798f37ef62e5b32d
Summary:
Introduce a new API type, `TreeAttributes`, corresponding to the existing type `WireTreeAttributesRequest`, which exposes which optional attributes are available for fetching. An `Option<TreeAttributes>` parameter is added to the `trees` API, and if set to `None`, the client will make a request with TreeAttributes::default().
The Python bindings accept a dictionary for the attributes parameter, and any fields present will overwrite the default settings from TreeAttributes::default(). Unrecognized attributes will be silently ignored.
Reviewed By: kulshrax
Differential Revision: D25041255
fbshipit-source-id: 5c581c20aac06eeb0428fff42bfd93f6aecbb629
Summary: Adding `full_idmap_clone` to edenapi and usign that in `debugsegmentclone`.
Reviewed By: quark-zju
Differential Revision: D25139730
fbshipit-source-id: 682055f7c30a94a941acd16e2b8e61b9ea1d0aef
Summary:
This method reconstructs a dag from clone data.
At the moment we only have a clone data construction method in Mononoke. It's
the Dags job to construct and import the clone_data. We'll consolidate that at
a later time.
Reviewed By: quark-zju
Differential Revision: D24954823
fbshipit-source-id: fe92179ec80f71234fc8f1cf7709f5104aabb4fb
Summary:
The server expects post requests. At this time we don't want to cache this data
so POST.
Reviewed By: singhsrb
Differential Revision: D24954824
fbshipit-source-id: 433672189ad97100eee7679894a894ab1d8cff7b
Summary:
The config is something that makes sense for all commands to have access to.
Commands that don't use a repo don't have access to the config that is prepared
by the dispatches. This change is a stop-gap to allow new commands that don't
require a repository to receive the config as an argument.
The construction of the config is something that we should iterate on. I see
the current implementaiton as a workaround.
Reviewed By: quark-zju
Differential Revision: D24954822
fbshipit-source-id: 42254bb201ba8838e7cc107394e8fab53a1a95c7
Summary:
On Python 3 we only support utf8 files. Python 3 has a way of
representing non-utf8 strings in utf8 format by utilizing surrogateescape, but
these strings cause issues in other parts of the codebase that don't expect
suorrageescaped strings (like the Rust code). Since we simply do not support
these paths, let's filter them out as soon as we get them from Watchman.
Reviewed By: quark-zju
Differential Revision: D25134079
fbshipit-source-id: 8be6893a6114b816097422f4469ac317fa3795d1
Summary: As HG<->Mononoke will go through proxygen, testing showed that Proxygen forces us to use 'Upgrade: websocket' and confirm with the Websocket handshake. Adjust accordingly to do so.
Reviewed By: ahornby
Differential Revision: D25197395
fbshipit-source-id: ca00ac31be92817c6f1a99d7d492469b17b46286
Summary: editmergeps.bat was separating a filename with spaces into args[0] and args[1] when calling editmergeps.ps1. Use proper quoting to send files with spaces as a single argument.
Reviewed By: ikostia
Differential Revision: D25194324
fbshipit-source-id: 065f677c9015681b310e1cfc46f52ff563a35f99
Summary:
While trying to repro a user report (https://fburl.com/jqvm320o), I ran into a
new hg error: P151186623, i.e.:
```
KeyError: 'Key not found HgId(Key { path: RepoPathBuf("fbcode/thrift/facebook/test/py/TARGETS"), hgid: HgId("55713728544d5955703d604299d77bb1ed50c62d") })'
```
After some investigation (and adding a lot of prints), I noticed that this
was trying to query the EdenAPI server for this filenode. That request should
succeed, given Mononoke knows about this filenode:
```
[torozco@devbig051]~/fbcode % mononoke_exec mononoke_admin fbsource --use-mysql-client filenodes by-id fbcode/thrift/facebook/test/py/TARGETS 55713728544d5955703d604299d77bb1ed50c62d
mononoke_exec: Using config stage prod (set MONONOKE_EXEC_STAGE= to override)
I1126 08:10:02.089614 3697890 [main] eden/mononoke/cmdlib/src/args/mod.rs:1097] using repo "fbsource" repoid RepositoryId(2100)
I1126 08:10:02.995172 3697890 [main] eden/mononoke/cmds/admin/filenodes.rs:137] Filenode: HgFileNodeId(HgNodeHash(Sha1(55713728544d5955703d604299d77bb1ed50c62d)))
I1126 08:10:02.995282 3697890 [main] eden/mononoke/cmds/admin/filenodes.rs:138] -- path: FilePath(MPath("fbcode/thrift/facebook/test/py/TARGETS"))
I1126 08:10:02.995341 3697890 [main] eden/mononoke/cmds/admin/filenodes.rs:139] -- p1: Some(HgFileNodeId(HgNodeHash(Sha1(ccb76adc7db0fc4a395be066fe5464873cdf57e7))))
I1126 08:10:02.995405 3697890 [main] eden/mononoke/cmds/admin/filenodes.rs:140] -- p2: None
I1126 08:10:02.995449 3697890 [main] eden/mononoke/cmds/admin/filenodes.rs:141] -- copyfrom: None
I1126 08:10:02.995486 3697890 [main] eden/mononoke/cmds/admin/filenodes.rs:142] -- linknode: HgChangesetId(HgNodeHash(Sha1(dfe46f7d6cd8bc9b03af8870aca521b1801126f0)))
```
Turns out, the success rate for this method is actually 0% — out of thousands of requests,
not a single one succeeded :(
https://fburl.com/scuba/edenapi_server/cma3c3j0
The root cause here is that the server side is not properly deserializing
requests (actually finding that was a problem of its own, I filed T80406893 for this).
If you manage to get it to print the errors, it says:
```
{"message":"Deserialization failed: missing field `path`","request_id":"f97e2c7c-a432-4696-9a4e-538ed0db0418"}
```
The reason for this is that the server side tries to deserialize the request
as if it was a `WireHistoryRequest`, but actually it's a `HistoryRequest`, so
all the fields have different names (we use numbers in `WireHistoryRequest`).
This diff fixes that. I also introduced a helper method to make this a little
less footgun-y and double-checked the other callsites. There is one callsite
right now that looks like it might be broken (the commit one), but I couldn't
find the client side interface for this (I'm guessing it's not implemented
yet), so for now I left it as-is.
Reviewed By: StanislavGlebik
Differential Revision: D25187639
fbshipit-source-id: fa993579666dda762c0d71ccb56a646c20aee606
Summary:
The write & read path use different back up state files, which means they can
go out of sync. If that happens, it's a bit annoying because:
- The output of `hg cloud check` is out of sync with that of `hg sl`
- `hg cloud backup -r` says the commit is backedup, and `hg cloud check -r`
says it isn't.
This diff fixes this by just using the `backedup()` revset, which under the
hood reads all state files.
Reviewed By: liubov-dmitrieva
Differential Revision: D25186071
fbshipit-source-id: 0ae2d43227d65a3564b2f796218b55982d5cc467
Summary:
Right now we just get a "deadline exceeded" error, which isn't very amenable to
helping us understand why we timed out. Let's add more logging. Notably,
I'd like to understnad what we've actually received at this point, if anything,
and how long we waited, as I'm starting to suspect this issue doesn't have much
to do with HTTP.
See https://fb.workplace.com/groups/scm/permalink/3361972140519049/ for
more context.
Reviewed By: quark-zju
Differential Revision: D25128159
fbshipit-source-id: b45d3415526fdf21aa80b7eeed98ee9206fbfd12
Summary:
It turns out the hggit tests weren't passing in Python 3, despite us
having removed them from our py3-broken list. Woops. This fixes them and enables
the tests.
Reviewed By: sfilipco
Differential Revision: D25095189
fbshipit-source-id: acffca34b0d5defa7575ede60621ca2ce0a2afe4
Summary:
Back in March we forced all extras to be strings. It turns out hggit
needs to write binary extras since it stores some binary patches in extras.
To support that, let's encode commit extras using surrogateescaped. That will
allow hggit to put binary data in extras in a later diff.
Reviewed By: sfilipco
Differential Revision: D25095190
fbshipit-source-id: 330bf06b15fc435f70119490557b8c22341b6ed9
Summary:
With segmented changelog, rev can exceed f64 safe range
(Number.MAX_SAFE_INTEGER in Javascript, 9007199254740991, 0x1fffffffffffff).
If rev is used in JSON, then the JSON cannot be parsed with precise rev
information.
This diff adds a compatibility mode so template will map the out-of-range revs
to safe range, and the mapped revs can be detected, and mapped back to be
resolved correctly.
Reviewed By: sfilipco
Differential Revision: D25079001
fbshipit-source-id: 52a5a2c8345a76355b8bf81435034d90c7c31610
Summary:
The `getdesignatednodes` function returns a boolean indicating whether the requested nodes were actually fetched.
In the case of SSH, this is needed because the server may not support the `designatednodes` capability (only supported by Mononoke). If the fetch fails, Mercurial will fall back to an expensive complete tree fetch.
The HTTP version of this function accidentally omitted `return True`, which meant it implicitly returned `None`, which triggered the fallback path.
Reviewed By: sfilipco
Differential Revision: D25074067
fbshipit-source-id: 089d5382dd566db89ee732cdcb82762c8d43e21a
Summary:
Skip the need to go through SSH/Mercurial servers. Directly connect to Mononoke
via TLS with X509 certificates.
Reviewed By: markbt
Differential Revision: D24756947
fbshipit-source-id: a782c894956d6a6be055eaa78287c040ceb57da7
Summary:
We've seen some hangs with http 2 in lfs. Switching to http 1.1 seems
to fix it. Let's make this configurable so we can tweak this if we see it in
edenapi. For now we continue to default to http 2.
Reviewed By: krallin
Differential Revision: D24901201
fbshipit-source-id: 9806e2c37fa299e4bd381ebdcb17d00800408de3
Summary:
We had a bug: if two files were reverted and then we try to amend one of them
mercurial will actually amend both of them.
Looks like the problem was caused by "Prune files reverted by amend" block.
Previously this block was considering every file that was changed in a commit we
are about to amend and comparing with working copy. If a file is the same in a
commit we are about to amend and in the working copy then it will amended as
well.
This diff attempts to fix it by considering only files that were selected for
amending.
Reviewed By: DurhamG
Differential Revision: D24949727
fbshipit-source-id: cf6cb95af3f67ec769e8a58db3b829945133b830
Summary:
We have an edge case - if we reverted two files and then we try to amend only a
single one then both of them will be amended.
This diff adds a test for this and other edge-cases. The next diff will fix it
Reviewed By: DurhamG
Differential Revision: D24949726
fbshipit-source-id: c5c53de1d67f161efa8564f89127e61ac2f28ac9
Summary:
When we streamclone, we snapshot the revlogs under a lock, then we start
sending. That works fine, unless we have a file whose size changes during
the sending phase. This can happen if it's promoted from a single `.i` to a
`.i` and a `.d`.
When that happens, the clone fails (the client reports it received a bunch of
corrupted data because it starts interpreting parts of files as inputs). Since
the breakage is also confusing client side, I updated the server side to also
assert that it's sending what it thinks it's sending.
Reviewed By: DurhamG
Differential Revision: D24958885
fbshipit-source-id: a0349c651b7cb63ab27546adf9944e7fba63a95d
Summary:
The wirepack sending code builds up the entire history blob in memory
before sending it. Previously we did this by appending to the string. In Python
2 this was fast, in Python 3 this is n^2 and n can be 100k+ in cases of long
history.
Let's switch to list+join.
Reviewed By: xavierd
Differential Revision: D24933183
fbshipit-source-id: 5c36d7868e7c64a2292bd68ec2ffb584d85dd98f
Summary:
osxfuse is rebranding as macfuse in 4.x.
That has ripple effects through how the filesystem is mounted and shows up in
the system.
This commit adjusts for the new opaque and undocumented mount procedure and
speculatively updates a couple of other code locations that were sensitive to
looking for "osxfuse" when examining filesystems.
Reviewed By: genevievehelsel
Differential Revision: D24769826
fbshipit-source-id: dab81256a31702587b0683079806558e891bd1d2
