Summary: In advance of peppering async everywhere, allow subcmd's run method to be async.
Reviewed By: genevievehelsel
Differential Revision: D21892187
fbshipit-source-id: f611faacf95649d8bb5588aeefc4546bd5f63984
Summary:
During expensive getbundle request traversing skiplist uses a lot of cpu, and
in fact it's blocking the whole CPU. krallin suggested to yield since it
should avoid blocking cpus, and that's what this diff is doing.
Reviewed By: krallin
Differential Revision: D22160477
fbshipit-source-id: 5bd881d5c50f4d8e64f2cc90389abc8568ec1df6
Summary:
Fix error "AttributeError: module 'concurrent.futures' has no attribute
'_base'" when loading asyncio.base_futures module. There very well may be a more appropriate fix, but this allows ipdb python3 debugging.
Reviewed By: quark-zju
Differential Revision: D22144225
fbshipit-source-id: eb6a96f79c97e3bd40c1efd9fc75660d6fc372ea
Summary: DangerousOverride is moved into a separate crate. Not only it is usually not needed but it was introducing dependencies on mercurial crate.
Reviewed By: StanislavGlebik
Differential Revision: D22115015
fbshipit-source-id: c9646896f906ea54d11aa83a8fbd8490a5b115ea
Summary: This change will ensure that cloning blobrepo is cheap, even if someone adds field that is expensive to clone. Plus it will result in just one arc-clone instead of cloning all the fields one by one.
Reviewed By: mitrandir77
Differential Revision: D22114066
fbshipit-source-id: ca0c3c78033b4c74872da314a32deb37c05b70ca
Summary: Globalrev does not have any dependencies on mercurial so it can be moved to mononoke_types since it is used in BlobRepo
Reviewed By: StanislavGlebik
Differential Revision: D22092491
fbshipit-source-id: 1dded88eb2ace08e8c6c3673e2d50ae1fbb9850d
Summary: Move all mercurial changeset generation logic to `blobrepo_hg`. This is preliminary step is required to decouples BlobRepo from mercurial, and in later stages it will be moved to derived data infra once blobrepo is free of mercurial.
Reviewed By: StanislavGlebik
Differential Revision: D22089677
fbshipit-source-id: bca28dedda499f80899e729e4142e373d8bec0b8
Summary: move HgMutationStore to attributes, and all related methods to BlobRepoHg
Reviewed By: StanislavGlebik
Differential Revision: D22089657
fbshipit-source-id: 8fe87418ccb8a7ad43828758844bdbd73dc0573d
Summary: Move `Filenodes` to `BlobRepo::attributes` as it is mercurial specific.
Reviewed By: ikostia
Differential Revision: D21662418
fbshipit-source-id: 87648a3e6fd7382437424df3ee60e1e582b6b958
Summary: This diff introduces `BlobRepoHg` extension trait for `BlobRepo` object. Which contains mercurial specific methods that were previously part of `BlobRepo`. This diff also stars moving some of the methods from BlobRepo to BlobRepoHg.
Reviewed By: ikostia
Differential Revision: D21659867
fbshipit-source-id: 1af992915a776f6f6e49b03e4156151741b2fca2
Summary:
This diff adds additional filed `BlobRepo::attributes` which can store attributes of arbitrary type. This will help store opaque types inside blobrepo without creating dependency on a crate which contains type definition for this attribute. This diff also moves `BonsaiHgMapping` inside attributes set.
- This work will allow to move mercurial changeset generation logic to derive data infrastructure
Reviewed By: ikostia
Differential Revision: D21640438
fbshipit-source-id: 3abd912e7227738a73ea9b17aabdda72a33059aa
Summary: Not all facebook-specific code was moved out of integration_runner_real.py, but removing part of the code that is left would made the code less readable, the rest of it will be removed while the integration_runner_real.py is made usable for OSS
Reviewed By: farnz
Differential Revision: D22114948
fbshipit-source-id: d9c532a6a9ea653de2b12cffc92fbf45826dad37
Summary: Add link support to CountedBlobstore
Reviewed By: StanislavGlebik
Differential Revision: D22090644
fbshipit-source-id: 36dc5454f1ca12c91d0eac6e5059f554ac5cb352
Summary:
Previously perform_move was able to only create a single commit that moves all
files. However this commit can be very large and it can cause problems for
different sync jobs. Let's add a method that all the files in the stack of
commits rather than in a single commit.
Reviewed By: ikostia
Differential Revision: D22067851
fbshipit-source-id: aff4d01cd547eb98dd6f0c45704371724acee9fc
Summary: If a value is above cachelib limit let's try to compress it.
Reviewed By: krallin
Differential Revision: D22139644
fbshipit-source-id: 9eb366e8ec94fe66529d27892a988b035989332a
Summary: This diff adds a PID-fetchCounts map to `ObjectStore` and makes `ObjectStore` update that map after every `didFetch`
Reviewed By: kmancini
Differential Revision: D22100413
fbshipit-source-id: 740342c7b4a453fe482344c2db9542381c3772e4
Summary:
In the case where an expensive network operation is involved, holding the GIL
would mean that Mercurial cannot display progress bars, or simply cannot be
interrupted. This is a less than ideal user experience.
To fix this, let's release the GIL whenever we enter into the revisionstore
code.
Reviewed By: DurhamG
Differential Revision: D22140399
fbshipit-source-id: 131c8cf81e39128810e0f20d1922b5681a33d95a
Summary: This will help in debugging potential performance issues.
Reviewed By: DurhamG
Differential Revision: D22140035
fbshipit-source-id: d7403897fbb843a4eca874c1da3788190d181bc0
Summary:
We've seen a handful of hangs in the LFS code, adding a timeout on the
connection request would help in surfacing issues when the client for some
reason doesn't get a reply from the server.
We unfortunately can't add a general timeout, as a user's connection speed
would need to be taken into account.
Reviewed By: krallin
Differential Revision: D22139560
fbshipit-source-id: 0118fb8a38af488e2f40bed0a09677bc68d666a8
Summary:
At least one mac user had a confirmed hang in the new LFS code that caused us
to rollback LFS for fbsource. One of the thing that was weird is that both
Sandcastle and devservers have several orders of magnitude higher request to
the LFS server and thus we would expect this issue to be widespread there. We
haven't seen this.
One of the difference between the package on the devservers/sandcastle and on
macs is how they are compiled. The former is with buck, while the later is with
cargo, and the package built with buck has all the tokio features enabled,
while cargo has no features enabled. What interest us here is the kind of
scheduler that is in use: on mac the basic scheduler, on devserver/sandcastle,
the threaded scheduler.
While I'm not entirely convinced that the difference in the scheduler is what
causes the hang, it's a good idea to make sure that both of our packages are
compiled the same way.
Reviewed By: krallin
Differential Revision: D22138593
fbshipit-source-id: ce9e64a6a930d2b0cccb634a1af9c3b5b8816a21
Summary: The word "disabled" better express the intent behind it.
Reviewed By: fanzeyi
Differential Revision: D21945736
fbshipit-source-id: a61a3df4f1b704224e7f4d4a7f1a43f4d2b5b2aa
Summary:
Removing a directory with unlink raises different errors on Windows: EACCES
instead of EISDIR.
Reviewed By: fanzeyi
Differential Revision: D21944455
fbshipit-source-id: 251ceb305a46c01ece78f45b0fa8000778c6c650
Summary:
On Windows, we can't rely on the .eden directory to not be present to decide if
the repository is mounted. Instead, we can just let edenfs decide and catch the
exception if it tells us that it's already mounted.
Reviewed By: fanzeyi
Differential Revision: D21934538
fbshipit-source-id: 60ed9f530456b627091f95f7387dc4b8f3a8dc5c
Summary:
All the unification of the mount code lead to this point, where supporting
unmount is merely just removing the various #ifdef.
Reviewed By: fanzeyi
Differential Revision: D21797078
fbshipit-source-id: 7602eb85e3fa276b72daebda36ae269ec38cde66
Summary:
On unmount, the code relies on the channel (fuse/prjfs) to set a promise, while
not strictly required on Windows as the interface is synchronous, this will
enable unmount to be the same between on all platforms.
Reviewed By: fanzeyi
Differential Revision: D21797081
fbshipit-source-id: c3eac448d2b13ef49db7a7d1f36bb084130ea63a
Summary:
Most of what fuseMount does can be shared between fuse and prjfs, so let's
rename it and start using it on Windows.
Reviewed By: fanzeyi
Differential Revision: D21797079
fbshipit-source-id: b92a36ecc702ee72df8ebdf9534add42bea9b97e
Summary:
While the code contained type information, some of them were not entirely
correct, path for instance should be str, command arguments are also passed in
as str, not bytes.
Reviewed By: ikostia
Differential Revision: D22104689
fbshipit-source-id: 50cbc9f81a3d9cc324f9105f0bbfbc71d9ca9e98
Summary: Per our ongoing effort to reduce the usage of potentially problematic terminology, rename the environment variables used to include and exclude tests.
Reviewed By: singhsrb
Differential Revision: D22143371
fbshipit-source-id: 97a0f73b21183f94d34af929e75cd531e4e24196
Summary:
This is required for migrating to python 3. Originally this broke
builds on non-fbcode centos7 platforms. Since we don't need hgsql in non-fbcode
platforms, I've just tweaked it to skip the hgsql tests for non-fbcode centos7.
Reviewed By: quark-zju
Differential Revision: D22140187
fbshipit-source-id: 01676db9d006375bd3b5ae9e8f13f5592e36b074
Summary: In Python3, b'a\n'[-1] is 10 whereas in Python2 it is "\n".
Reviewed By: quark-zju
Differential Revision: D22128582
fbshipit-source-id: 3b13a14efd81ea2de4fd7125597e3c7ec6371492
Summary:
When the scuba telemetry logger is unavailable, fall back on the
subprocess.
Reviewed By: genevievehelsel
Differential Revision: D21687855
fbshipit-source-id: c58a4bc1b34974add35d194c8aafdaac7ed47fdb
Summary: It had some in-test python that needed to be updated.
Reviewed By: quark-zju
Differential Revision: D22097852
fbshipit-source-id: ebe4ab19a2e4bed73badc1d34d6fcf2db6072559
Summary: Fixes test-dispatch.t and fixes statprof while we're at it.
Reviewed By: xavierd
Differential Revision: D22103518
fbshipit-source-id: 7d795c6f9e3e2c8958746c10e39f55922a923dc5
Summary:
It now works with python 3. I dropped some test output. In one case
it was confirming that various path encoding work, but we only support utf8 now.
I also dropped some bit that enabled debug output, since the output order was
not the same between python 2 and 3 and it seemed non-trivial to fix.
Reviewed By: xavierd
Differential Revision: D22101132
fbshipit-source-id: 682e7c61f1dd18cce34bdc014c85d560f883bd18
Summary:
We support unicode file paths, and in python 3 those get passed to
python libraries as unicode strings. The tests set LANG=C which mean the python
library tries to convert the path to ascii, but fails for any non-ascii
characters. Let's switch to LANG="en_US.UTF-8" to match our production
behavior and make tests about unicode paths work.
Reviewed By: xavierd
Differential Revision: D22098359
fbshipit-source-id: c3057edc66e6e32f7b8b49374e622d02bd05711f
Summary:
This module was using binascii.hex directly instead of using Mercurial hex
function.
Reviewed By: xavierd
Differential Revision: D22096490
fbshipit-source-id: 32c912c16415ffd94371ad945c4df2f5a82b92e5
