Summary:
The goal of this diff is to provide more visibility into how long the client
takes to create/upload an infinitepush bundle. This is done in two ways:
- by adding more `perftrace` calls (useful when invistigating individual slow
pushes)
- by adding `ui.timesection` scopes (useful for aggregation purposes)
Two main things that are measured:
- creation of the bundle purely on the client
- sending of the bundle over the wire
In addition, in the perftrace recording, this measures how long it takes to
process the reply handlers, how much bytes are sent over the wire, what are the
part names and sizes (when available). These changes mostly do not distinguish
whether the code is infinitepush push or not, but they are always related to
some sort of a wireproto scenario, which means that the performance impact is
negligible (writing things to thread-local storage is *much* cheaper than
sending them over the network).
Reviewed By: DurhamG
Differential Revision: D24683484
fbshipit-source-id: 53fdfb63dcdfcf38924237c59a1e8f5e24ff96c0
Summary: We're getting rid of old futures - remove them as a dep here
Reviewed By: StanislavGlebik
Differential Revision: D24705787
fbshipit-source-id: 83ae938be0c9f7f485c74d3e26d041e844e94a43
Summary:
We can have different bonsai changesets hash for the same hg changeset. Consider situation - we have hg repo:
```
o B (Add file "b")
│
o A (Add file "a")
```
The correct bonsai changeset for B will have only entry `(<Path_to_b>,Some(<hash_b>))` in `file_changes`. But we can also have bonsai changeset for B with 2 entries `(<Path_to_b>,Some(<hash_b>)), (<Path_to_a>,Some(<hash_a>))`. This diff provides the functionality to manually create such situation. And later it will be used for verification blobimport backups
Reviewed By: StanislavGlebik
Differential Revision: D24589387
fbshipit-source-id: 89c56fca935dffe3cbfb282995efb287726a3ca9
Summary: We were incorrectly marking reverts as landed during pullcreatemarkers.
Reviewed By: quark-zju
Differential Revision: D24608217
fbshipit-source-id: f919f49469d6933c17894b3b0926ba2430a5947a
Summary:
As part of getting buck build to work on OSX, we need procinfo to
include it's OSX specific library.
Reviewed By: sfilipco
Differential Revision: D24513234
fbshipit-source-id: 69d8dd546e28b4403718351ff7984ee6b2ed3d1d
Summary:
Improve ux for setting the token.
This extra bits would help to figure out prompt issues on Windows
Reviewed By: markbt
Differential Revision: D24706085
fbshipit-source-id: 6101b8d7b90aad2d687465a09cc69670ca4a46f6
Summary:
Eden can often overfetch files during checkout. There may be multiple things
going on here, but at least one of them is tied to aux data prefetching.
Responding to a FUSE request for aux data will result in the inode for that
file being loaded. During checkout if we don't short circuit the checkout
then we will create a CheckoutAction for each loaded inode. And previously
CheckoutActions will always load their inodes contents.
To fix this we have two options
1. avoid creating the CheckoutAction in the first place.
2. Avoid downloading in CheckoutAction.
1 Turns out to be more difficult. I did some thinking through this here if
you want to see: https://fb.quip.com/LeqSAb3OlpWb
2 We don't actually need to be downloading blob data here, we only really
need the eden hash and content sha1 of the blob in this code path (this is
all that is used to compare the old and new file). So we should really only
be fetching the metadata (which should be avaiable locally since the inode
was loaded unless the caches have been cleared).
This implements fix 2.
I think there is likely still over fetching having to do with trees in checkout.
And potentially we can avoid looking at metadata at all in some cases.
I will look at this next.
Reviewed By: chadaustin
Differential Revision: D23432810
fbshipit-source-id: 17c0503bf95eb360de902a370948338bf04c1887
Summary:
This changes the methods from ones that return old `BoxFuture`s to an async method
using `async_trait`.
Reviewed By: krallin
Differential Revision: D24689506
fbshipit-source-id: 7b13010924369f81681e6590898af703c5423385
Summary:
This changes the trait method from one that returns an old `BoxFuture` to an async
method using `async_trait`.
Reviewed By: krallin
Differential Revision: D24686888
fbshipit-source-id: 0ac231cdbb60d256b6d5ad5aafbe8779b96905f3
Summary: This is to be able to see in Scuba why a given sync function was called.
Reviewed By: StanislavGlebik
Differential Revision: D24689366
fbshipit-source-id: f868fc1b6fcbf6c692e1373cbe8da8cd4a230780
Summary: This is a step towards modernizing unbundle crate to use futures 0.3.
Reviewed By: farnz
Differential Revision: D24682963
fbshipit-source-id: 55c17fd699846a24647a23ea1c22888407643dfd
Summary:
On Windows, we never unload any inodes until EdenFS is restarted, thus,
checkout times go up over time as more and more inodes are being loaded. While
on Windows we don't keep track of what is referenced by the kernel, the
checkout code will use the "precise inodes" code path when deciding what to
update. This means that every inode that is in the overlay will get updated
properly, and since the overlay is a superset of what is hydrated, we are
guaranteed to always invalidate what we need to.
Due to the above, this shouldn't result in any changes as we never gc the
overlay, but that will come later, at which point checkout times will stop
being more and more expensive as time goes.
Reviewed By: chadaustin
Differential Revision: D24634253
fbshipit-source-id: c7b838edc20589bbf92ff4e2b3abd079b9a4443d
Summary:
Futures are by default run inline, meaning that when the previous future is
completed, the future will run in the same context as the previous one. In the
case where the previous one is completed by another thread setting up its
promise, the future will be completed in the context of that other thread.
In most cases, this is OK, in others, this can cause a deadlock. And this is
exactly what we're seeing here. When a file is renamed concurrently to an `hg
update`, the inode the rename operates on might not be loaded, and thus both
update and the rename callback will race to load that inode. When update wins
that race, the rename callback will wait on a promise that update will then
set. When that happens, the rest of the rename callback will be run in the
update context, but that will in turn cause update to try to re-acquire the
rename lock that it already holds...
To fix this, we need to make sure that the rename callback doesn't run inline.
Reviewed By: chadaustin
Differential Revision: D24657422
fbshipit-source-id: 23b08765afae7bda4a628f0c23675bff9f486b6b
Summary:
I'm not entirely sure why these started failing, but enabling ui.allowmerge
made these run again.
Reviewed By: chadaustin
Differential Revision: D24697462
fbshipit-source-id: ec5ca987e7116edb12658eb7b4d03f1cf0f876d3
Summary:
For logging and analytics, it's convenient for the
HgQueuedBackingStore to know the manifest hash and path early in the
import process. Since every object fetch requires looking up the
HgProxyHash anyway, fetch it immediately and thread it down to the
importer.
Reviewed By: kmancini
Differential Revision: D24524319
fbshipit-source-id: 0d91d55655e5ee25a010f7664e80125b7c50cf84
Summary:
Use the empty string to indicate the moved-from state, which makes
HgProxyHash moves noexcept. I plan to look up HgProxyHash early and
move it into HgImportRequest.
Reviewed By: kmancini
Differential Revision: D24522612
fbshipit-source-id: 037b4012ad6a51ad7ebd6a96de2e391cd570771b
Summary:
Stop pretending that HgBackingStore is a standalone backing store
implementation, and instead indicate it's an implementation class used
by HgQueuedBackingStore.
Reviewed By: kmancini
Differential Revision: D24514247
fbshipit-source-id: 90c3a442d01647fa6d1337cfd814f5bf4b480137
Summary:
We had some unnamed threads that made profiling performance on macOS a
bit harder. Give them a semblance of a useful name, at least.
Reviewed By: kmancini
Differential Revision: D24640223
fbshipit-source-id: 7dd74b30a081753006df681bf97ac96147b1896c
Summary:
Until fanzeyi gets InodeMetadata moved over to SQLite, remove some
excessive logging when the InodeMetadataTable contains zeroes because
its buffers weren't flushed to disk.
Reviewed By: genevievehelsel
Differential Revision: D24377026
fbshipit-source-id: e6ffa54244730388aaf66dc53cd29a0069fba685
Summary:
Add a very simple debug command that simply prints all materialized
inodes under the given path. It is a quick way to uncover unexpected
writes (or unexpected failed dematerializations after checkou).
Reviewed By: xavierd
Differential Revision: D24378759
fbshipit-source-id: dc393d65506c74dbc0779732cdefd915cbbf9948
Summary:
Replace the old implementation of debugInodeStatus with the more
general traverseObservedInodes functionality, and add the ability to
customize its results with flags.
Reviewed By: xavierd
Differential Revision: D24300122
fbshipit-source-id: 0fbd3aa02575faa515fd7852441547d7de13426d
Summary:
Apply automated Thrift formatter. Even if the format is a bit off in a
couple places, and the bikeshed is still being painted, this avoids
unrelated formatting changes later in the stack.
Reviewed By: xavierd
Differential Revision: D24511057
fbshipit-source-id: f1b23578733a8ecf788509e407bc419fa073428d
Summary:
In Mercurial we're finding use cases for conditioning on buck vs
non-buck builds. For instance, in some cases we specify #[link] directives for
non-buck builds, but don't need to for buck builds
(https://fb.workplace.com/groups/rust.language/permalink/4568487309866515/). In
other cases we want to depend on internal buck libraries when building with
buck, and not otherwise.
Let's add a fb_buck_build cfg so other projects may distinguish the same.
Reviewed By: jsgf
Differential Revision: D24493974
fbshipit-source-id: 1d558cbe0ae01ab4a7b4b5d6d4be75bb8ab0f41a
Summary: This method is an override of its parent class, thus the need for the override.
Reviewed By: chadaustin
Differential Revision: D24657424
fbshipit-source-id: 5ce7200eeb4ef28fb51fabbe0ebb38ded51ae34e
Summary:
Dealing with non-owned path in futures is mind bending and can lead to use
after free fairly easily if not careful. It turns out that this code is
actually a bit buggy and in some cases will attempt to use a path that is
already freed. Since the callsite doesn't need to hold onto the paths, let's
just move them, which resolves the issuue.
Reviewed By: chadaustin
Differential Revision: D24657423
fbshipit-source-id: 47bbaccf18cd86e53860491e3cbfeadb4363499c
Summary:
The rage local config output was getting polluted by dynamic config.
Let's filter them out.
Reviewed By: farnz
Differential Revision: D24626564
fbshipit-source-id: df5ac04cd549595ecdccc0b2438d4e7c72b80e88
