Summary:
Changes:
- Limit simultainous open git-repo objects by amount of CPUs.
- Put a semaphore limit so we wait inside tokio::task domain instead of tokio::blocking domain (later is more expensive and has a hard upper limit).
Reviewed By: mitrandir77
Differential Revision: D30976034
fbshipit-source-id: 3432983b5650bac6aa5178d98d8fd241398aa682
Summary:
This allows the mononoke_api user to choose whether the skiplists
should be used to spped up the ancestry checks or not.
The skiplists crate is already prepared for the situation where skiplist
entries are missing and traverses the graph then.
Reviewed By: yancouto
Differential Revision: D30958909
fbshipit-source-id: 7773487b78ac6641fa2a427c55f679b49f99ac8d
Summary:
Allow the mononoke_api user to choose whether they want
oprerations to be sped up using WBC or not.
Reviewed By: yancouto
Differential Revision: D30958908
fbshipit-source-id: 038cf77735e7c655f6801d714762e316b6817df5
Summary:
Some crates like mononoke_api depend on warm bookmark cache to speed up the
bookmark operations. This prevents them from being used in cases requiring
quick and low overhead startup like CLIs.
This diff makes it possible to swap out the warm bookmark cache to a
implementation that doesn't cache anything. (See next diffs to see how it's
used in mononoke_api crate).
Reviewed By: yancouto
Differential Revision: D30958910
fbshipit-source-id: 4d09367217a66f59539b566e48c8d271b8cc8c8e
Summary:
This method was added before the more generic list method was added. Let's get
rid of it for simplicity and to discourage listing all the bookmarks.
Reviewed By: yancouto
Differential Revision: D30958911
fbshipit-source-id: f4518da3f34591c313657161f69af96d15482e6c
Summary:
0.4.24 is incompatible with crates that use `deny(warnings)` on a compiler 1.55.0 or newer.
Example error:
```
error: unused borrow that must be used
--> common/rust/shed/futures_ext/src/stream/return_remainder.rs:22:1
|
22 | #[pin_project]
| ^^^^^^^^^^^^^^
|
= note: this error originates in the derive macro `::pin_project::__private::__PinProjectInternalDerive` (in Nightly builds, run with -Z macro-backtrace for more info)
```
The release notes for 0.4.28 call out this issue. https://github.com/taiki-e/pin-project/releases/tag/v0.4.28
Reviewed By: krallin
Differential Revision: D30858380
fbshipit-source-id: 98e98bcb5a6b795b93ed1efd706a1711f15c57db
Summary:
Move optional line handling logic into a separate function and simplify.
This diff is intended to be a pure refactoring with no observable changes in behavior. In particular, all the code dealing with the "optional" list appears to be dead code because if the line is optional, linematch will return "retry", so that branch is never reachable.
Reviewed By: DurhamG
Differential Revision: D30849757
fbshipit-source-id: 17283f9217466b3f85d913da66222b9a6779abe4
Summary:
This line was iterating over a list of files and looking in the
manifest for each one. This results in serial manifest reads which can result in
serial network requests.
Let's instead use manifest.matches() to test them all at once via the underlying
BFS, which does bulk fetching.
Differential Revision: D30938359
fbshipit-source-id: 1af7d417288b82efdd537a4afeaf93c1b55eaf49
Summary:
Demonstrate issues with the vertex to path resolution. Basically, the vertex to
path resolution logic did not check if the "parent of merge" being used is
actually valid (is an ancestor of provided heads) or not.
Reviewed By: DurhamG
Differential Revision: D30911150
fbshipit-source-id: 83d215910d5ba67ac0d5749927018a7aefcc6730
Summary:
The tree metadata fetching evolution goes as follow
(1) (commit, path) scs query
(2) tree manifest scs query [we are here]
(3) eden api manifest query [in development]
Option (1) is no longer used and is the only placed that required scs proxy hash.
Removing it will simplify transition from (2) to (3) and also cleans up bunch of unused code.
It also comes with minor performance improvement, saving about 5% on file access time.
To be precise, this is measured by running fsprobe [this is probably too little to measure in high noise benchmark like running arc focus]:
```
fsprobe.sh run cat.targets --parallel 24
```
Results:
```
W/ scshash:
P24: 0.1044 0.1007 0.1005 (hot) 0.1019 avg
W/o scshash:
P24: 0.0954 0.0964 0.1008 (hot) 0.0975 avg
```
This performance improvement comes from the fact, that even though scs hash was never created or used, we still attempted to load it from scs table, and even though this load always failed it contributed to execution time.
Reviewed By: xavierd
Differential Revision: D30942663
fbshipit-source-id: af84f1e5658e7d8d9fb6853cbb88f02b49cd050b
Summary: File access latency can actually be less then 1 ms, so it's good to show more digits
Reviewed By: DurhamG
Differential Revision: D30942905
fbshipit-source-id: 2fc8d48dbc08c55b89d829d1474ae11c2c3df1c3
Summary:
Since fsprobe itself requires a 'plan' to run, we need separate script to standartize list of plans we think are relevant
This scripts allows to generate fsprobe plans and run them
Reviewed By: DurhamG
Differential Revision: D30908892
fbshipit-source-id: eb722fe1f6d982e42b66614f08bc73345e04f9e6
Summary:
We got errors like:
error.IndexedLogError: "repo/.hg/store/lfs/pointers/meta": when reading LogMetadata
in log::OpenOptions::open(Filesystem("repo/.hg/store/lfs/pointers"))
Caused by 1 errors:
- failed to fill whole buffer
from Sandcastle. There seems no easy way to get a sample of the broken `meta`
file content. Let's include the file content to make progress on debugging.
Reviewed By: DurhamG
Differential Revision: D30939737
fbshipit-source-id: ccd77f6b67e4aaf75af2248118845fd5b3434ff1
Summary: This `allow` is no longer needed.
Reviewed By: yancouto
Differential Revision: D30859520
fbshipit-source-id: 36b810a72a28af25513404739bccf471e380cdf1
Summary: Update TreeStore to use CommonFetchState and update TreeStore and BackingStore to use the other utility types already in use for files (`StoreTree`, `FetchResults`, etc).
Reviewed By: andll
Differential Revision: D30739008
fbshipit-source-id: e210b8d76614c762ba127d5f2e26391681da004f
Summary: Adds a utility method for converting a `StoreTree` to a `manifest-tree::Entry`, which wraps an hg manifest blob and provides methods for parsing it as a tree manifest (and a `TryFrom` impl used to convert it to a pre-parsed `manifest::List`, which is used by BackingStore in the next change in this stack).
Reviewed By: andll
Differential Revision: D30859470
fbshipit-source-id: 411e80a14861e0739b0c398290055002b35e59d3
Summary: This change does not add aux data support, so for now the types are a bit useless.
Reviewed By: DurhamG
Differential Revision: D30313314
fbshipit-source-id: 11968199b12c4f870c58c7e939b5c8ed5cd9afea
Summary: More refactoring of scmstore `TreeStore`. Introducing a new `tree` submodule as we'll be adding tree-specific metrics, types, etc. soon (as currently exist for files).
Reviewed By: andll
Differential Revision: D30313460
fbshipit-source-id: f20d3ee62520b1d9ea34ad04eb1880ad9b5a00c3
Summary: Extract out `CommonFetchState` from `FileStore`'s `FetchState`. Currently, direct field access is still used for computing derivations and a few other places, but this will be changed in a later diff.
Reviewed By: DurhamG
Differential Revision: D30308289
fbshipit-source-id: 16d34904412572facc9f51cbd791e30413bfe634
Summary: Don't show progress bars for pending HTTP requests until they actually start running, so that the user always sees progress bars from active transfers.
Reviewed By: quark-zju
Differential Revision: D30914241
fbshipit-source-id: ca2f85af055dc9324123d0f9cc765f42d3b36ad4
