Summary:
Will be used in tests to check what data is inserted into IdMap during clone or
pull.
Reviewed By: andll
Differential Revision: D30314054
fbshipit-source-id: 8a58473ef3d85263985c1be8a6c57a9a10b3fdb4
Summary:
In rare cases, the local IdMap might miss parents of draft roots.
This provides a way to emulate that in tests.
Reviewed By: andll
Differential Revision: D30314050
fbshipit-source-id: 09983604bf3259e62a41224a579774265ae0b272
Summary: This would help investigate some errors.
Reviewed By: andll
Differential Revision: D30314057
fbshipit-source-id: 070e75cf0a39180c544b49c9ed1292d8536040fa
Summary:
- This diff adds validation so that changesets that are not snapshots cannot have untracked or missing files.
- It removes the THIS IS A SNAPSHOT commit message.
- It makes the snapshot created by `hg snapshot createremote` be an actual snapshot.
Differential Revision: D30159184
fbshipit-source-id: 976968c0c2222f950a4a937aa805b25dc07c9207
Summary:
This diff makes the snapshot command upload all types of files that were missing (added/untracked/missing/deleted), using the new types of file changes added on the previous diff.
Next steps:
- Add some indicator to Bonsai Changeset saying it is a snapshot. Verify only snapshots can have certain file changes (untracked/missing).
- Upload the files and the changeset inside an ephemeral bubble instead of in the main blobstore
- Start writing the `snapshot restore` command
Differential Revision: D30137673
fbshipit-source-id: 555238f1d64a5438cde35a843043884a939de4fe
Summary:
On Windows, it's hard to rename `.hg/store/segments/v1` away because the files
are (too easily) being currently used. Instead, let's write the new lazy
segments to `segments/v1next`, and try to rename it back later on follow-up
commands.
Reviewed By: DurhamG
Differential Revision: D30118952
fbshipit-source-id: e3edb588dccf1acb5f4ed106bbb979bcc8c6e67e
Summary:
OD was running `hg rage --preview` in the background periodically and that has caused edenfsctl creating paste on the user's behalf without them actually knowing.
This will make Mercurial to collect EdenFS rage in dry-run mode (i.e. do not create paste).
Reviewed By: quark-zju
Differential Revision: D30288390
fbshipit-source-id: 1e7f25648ca1b76f24264ee13ed98cf974148b0f
Summary: Allow `manifest-tree::Entry` to be constructed directly by other crates, and introduce a `TryFrom` conversion to `manifest::List::Directory`. This will be used by the scmstore `BackingStore` implementation to avoid having to rely on the `TreeStore` trait, which does not support batching.
Reviewed By: kulshrax
Differential Revision: D30282738
fbshipit-source-id: 590350dd53217fa8a181e91b194abca753a8adbe
Summary: Add a new results adapter for `FileStore` (which will probably become the primary method of consuming batch errors in the future), which iterates over all keys, and explicitly encodes not found with `Option`, separate from the outer `Result`.
Reviewed By: kulshrax
Differential Revision: D30254000
fbshipit-source-id: 074c6ac4c279fdf72078dca17231e58d0c704956
Summary:
Add a `local` method to `TreeStore` like the one on `FileStore` which returns a `TreeStore` with only the local subset of backends.
Required for `BackingStore` implementation.
Reviewed By: kulshrax
Differential Revision: D30253980
fbshipit-source-id: 142f2d88454826ff9cb9c34f30b7d21bf62b297c
Summary: Add a new method, `edenapi_override`, to the `TreeStore` and `FileStore` builders for use in the `BackingStore` implementation.
Reviewed By: kulshrax
Differential Revision: D30253837
fbshipit-source-id: 4a42e83621fb2634024e4ee8529c26aeae0256a4
Summary:
Instrument all the fetching-related legacy API implementations (ie, not `get_logged_fetches` and `get_shared_mutable`) to track the number of calls, keys, and single-key calls.
This introduces an additional lock acquisition to each of these implementations (and it'd be awkward to merge it with the one in `fetch`), but I think that's probably fine.
For APIs which do not support a variable number of keys, I use `.call(0)` so we simply track the total number of API calls.
Reviewed By: DurhamG
Differential Revision: D30003444
fbshipit-source-id: 8756d2669ca038b3f6a08e211e44e8ccb9251312
Summary:
Adds tracking of FileStore write errors.
Write batches will still abort after the first failure as written.
Reviewed By: DurhamG
Differential Revision: D29997203
fbshipit-source-id: e1cc2ffc4a8d97ca935a7fc9aab30bde3dc548b2
Summary: Refactor the `FileStore::write_batch` method, extracting the various write cases into other methods.
Reviewed By: DurhamG
Differential Revision: D29997096
fbshipit-source-id: 8efac4b2efa2e2225d39583a5c6893efc5096fec
Summary:
Previously errors from `prefetch` weren't counted. With this change I move calculating the pending keys out of the `inner` method, making it easier to handle all batch error metrics from `?` calls right beside the existing `other_errors` call.
This is probably how I'll structure the other methods too, though it's still a little subtle - currently the batch errors only come from `prefetch`, but if batch errors were returned after the `drain` call (which always drains the vector, even if you don't consume the whole iterator), the second `pending.len()` call could incorrectly report 0. To address this, I could separate out the "single key fetching" part of the method, or just let it be understood that you should repeatedly call `pop` in such cases.
Reviewed By: DurhamG
Differential Revision: D29995789
fbshipit-source-id: 09753081c5e7076b83cb26fdddb949205ad7cd43
Summary: Previously, contentstore metrics were tracked as a special case. Now that I'm tracking metrics on each backend, I've standardized the contentstore metrics to match.
Reviewed By: DurhamG
Differential Revision: D29995500
fbshipit-source-id: 3133dca5912d9cd6c26bd88d9aeb6a572f6eca40
Summary:
Increase [CURLOPT_LOW_SPEED_TIME](`https://curl.se/libcurl/c/CURLOPT_LOW_SPEED_TIME.html`) because users were hitting low speed timeouts during normal usage.
2 minutes was arbitrarily chosen, but it seems unlikely that we'd have situations where connections legitimately sit idle for more than 2 minutes. (If it turns out we do, that would be a bit concerning.)
This diff also updates the config option to specify the time in seconds instead of milliseconds because libcurl only supports second-level precision when specifying `CURLOPT_LOW_SPEED_TIME`. The config option name has been updated to explicitly include the unit to avoid confusion.
Differential Revision: D30272017
fbshipit-source-id: 0ae5e912c0dafc897fe19de3ea8e5aeb4ed8459c
Summary: Clean Up: Once we migrated to Mononoke we can remove the infinitepush path because it creates extra complexity in remotenames, infinitepush and commitcloud extensions
Reviewed By: yancouto
Differential Revision: D30277402
fbshipit-source-id: 1708a71eb33dc9a9b509bdc1b03bed08073b6152
Summary:
fix remote lookup for every bookmark
Some users have lots of local bookmarks and remote bookmarks, so the current logic would perform tons of sequential remote lookups via edenapi (due to segmented changelog) just to maintain correct omission state for local and remote bookmarks even if nothing has been changed.
Reviewed By: markbt
Differential Revision: D30247185
fbshipit-source-id: 7c7fd8491ff0bd67827c2a01eb852322ba15f994
Summary:
Client::fetch was async method for no reason(it does not contain any awaits)
This diff removes async and makes it clear that fetch only prepares futures but does not itself contain async code
Reviewed By: DurhamG
Differential Revision: D30137299
fbshipit-source-id: 87f80cf906bbea856f9f54b88a3609d0aad03a73
Summary: This allow to render metrics in the real time, similar to how we render network usage
Reviewed By: DurhamG
Differential Revision: D30137295
fbshipit-source-id: 8843b574971c1ccaaae0727be6f9d46d1d90625f
Summary: See docstring to init_from_env for the description
Reviewed By: quark-zju
Differential Revision: D30137304
fbshipit-source-id: ad3fb5e0cbb52ca405959eb0d394f5a2c9761ad9
Summary: This allows to count number of parallel operations, like parallel entrance to some fn or parallel requests
Reviewed By: quark-zju
Differential Revision: D30137297
fbshipit-source-id: 74585dcab353091619a0a1f217548121079b463c
Summary: Allow this to be used by more utilities.
Reviewed By: DurhamG
Differential Revision: D29958512
fbshipit-source-id: b6a1a7017102a4ff4ad252246d3252903bbb793f
Summary: Prepare before moving to a common directory
Reviewed By: StanislavGlebik
Differential Revision: D29958513
fbshipit-source-id: 48c04a9bf6e8c3800b3f187e7741d240ecff9e9c
Summary: This allows us to remove older files to free disk space.
Reviewed By: DurhamG
Differential Revision: D30092445
fbshipit-source-id: 20770968cfaeba4fb6f6c45b247cc1cdafb79a08
Summary: This makes upcoming changes a bit easier.
Reviewed By: yancouto
Differential Revision: D30092446
fbshipit-source-id: f62f1ced45c46dc48c3b1112af661d8fbcd80706
Summary: This removes some redundant lookups when using the `*_batch` APIs.
Reviewed By: andll
Differential Revision: D30094155
fbshipit-source-id: f64702a09f67b67a10a2d51545f3b6940e80022f
Summary: The test exposes issues that the local cache isn't used in some cases.
Reviewed By: andll
Differential Revision: D30094158
fbshipit-source-id: 0122154b8bd2f9d321e248720632e28b6ca0559e
Summary:
Similar to D29888937 (8cb3d3dd75). This helps getting the Rust traceback of why certain
code path gets triggered.
Reviewed By: andll
Differential Revision: D30094157
fbshipit-source-id: ef5bacbf4612ed115fd631056f6c004fe6d1d93f
Summary: This will be used to debug some over fetching issues.
Reviewed By: andll
Differential Revision: D30094159
fbshipit-source-id: cfbdff497fbf53f97f1c957bf77c25e26ee79286
Summary:
Prefetch vertex on checkout (which is more expected to require network to download
files) so committing on them does not trigger remote lookup.
Reviewed By: andll
Differential Revision: D30094156
fbshipit-source-id: 917df2bbad7cee51a06f2fa38172775423893c37
Summary:
See the added comments. In some cases we can avoid remote lookups. This would
help make commit/amend more offline friendly.
Reviewed By: DurhamG
Differential Revision: D30004908
fbshipit-source-id: 94fbc7934a1eb3ae1058d8c542211a885d5ad8e6
Summary:
Similar to D30094157. This is helpful to get a Rust backtrace on "not found"
errors. This is useful because the Python traceback might not have Rust
stack.
Ideally, it'd be better if we can integrate backtrace into the error types.
However, the stdlib backtrace is still unstable and the thirdparty backtrace
seems too heavyweight.
Reviewed By: DurhamG
Differential Revision: D30123355
fbshipit-source-id: 7af76422143340194d8499a75e8ed65e9bf1fcae
Summary:
We have seen slicing panics on NFS, like:
thread 'main' panicked at 'range end index 14079379 out of range for slice of length 0', eden/scm/lib/indexedlog/src/log/mod.rs:1678:14
Possibly when some files were stale or out of sync (ex. `log` got shorter than
expected somehow) due to NFS weirdness (bad locking?). Let's make them Rust
errors instead of panics for easier error reporting and investigation.
Reviewed By: DurhamG
Differential Revision: D30166177
fbshipit-source-id: 25cbd9c178b24580c176fe9469362fe0020ede95
Summary:
commitcloud CLEAN UP: remove last bits code related to an old interngraph commit cloud proxy
The configs are not used, the service_url in scm daemon is not used, the interngraph proxy is not used
Reviewed By: singhsrb
Differential Revision: D30160471
fbshipit-source-id: 07fee2b8733afa29659467cb296df3b47fe1b54b
Summary: These were accidentally left over from a previous diff.
Reviewed By: singhsrb
Differential Revision: D30133799
fbshipit-source-id: 311f746848fc823a6fbae9e58bafda6641104b1e
