Summary: I don't think we need file history when diffing commits, so skip it. This can be really slow for files with a lot of history.
Reviewed By: quark-zju
Differential Revision: D44175699
fbshipit-source-id: 945517f58ea9baf9d5e22ca04699d72e8dad72f7
Summary:
This makes `test-invalid-windows-filepath.t` pass with recent pathauditor error
type changes.
Reviewed By: sggutier
Differential Revision: D44153212
fbshipit-source-id: 81e71c48a42ef29eecf3a3a3e117141f43232efb
Summary:
Not sure what changed, but we got:
--- test-blackbox.t
+++ test-blackbox.t.err
@@ -31,6 +31,7 @@
[process_tree] (this process)
[command] [*, "confuse"] started by uid 0 as pid 0 with nice 0 (glob) (?)
[process_tree] (this process) (?)
+ [legacy][command_info]
[legacy][command_info]
[legacy][env_vars]
[legacy][command_info] (?)
The alias expansion seems broken. Update the test text to mention it.
Reviewed By: sggutier
Differential Revision: D44139337
fbshipit-source-id: b4012eb3ee3340c52e3514dd49c617bdf851a572
Summary:
Move macOS OPENSSL_DIR logic from onetime-setup.py to distutils_rust.
Hopefully this makes it work for hgbuild which does not run onetime-setup.
`HG_LINKER_FRAMEWORK` is removed since code search shows it is not used.
Reviewed By: sggutier
Differential Revision: D44142250
fbshipit-source-id: b7af87bbdd9eac48b7b00fbd14ca062bc953c727
Summary: This was never utilized beyond the "warn" stage, and I don't want to implement in Rust, so let's kill it.
Reviewed By: quark-zju
Differential Revision: D43970585
fbshipit-source-id: cea765197a5f547441865a055deb1f13ad9ea28a
Summary: Sparse profiles supported configuring a list of "unsafe" marker files that would cause an error if they accidentally were included in a sparse profile (presumably to avoid unneeded sparse profile growth). However, this feature is not in use anymore, and I don't want to implement it in Rust, so kill it.
Reviewed By: quark-zju
Differential Revision: D43970582
fbshipit-source-id: a6576b8004cc41b3e6f2f0e0040b54e54af2bdc4
Summary:
Fix rust status when dirstate does not contain executable/symlink flags for a tracked file, and the file's flags have diverged from the manifest.
To achieve this we pipe the filesystem metadata long to the content check since the content check already looks up all the files in the manifest. Before actually doing the content check, it compares the on-disk flags to the manifest flags.
Reviewed By: quark-zju
Differential Revision: D43970583
fbshipit-source-id: 544acfa355cf9d0d2acf59ad9143de6dac349235
Summary:
Handle case where a directory with tracked files is replaced with a symlink.
In paticular, during status if we get a access-file-through-symlink audit error trying to read the contents of a file for comparison, we now treat the file as deleted.
Reviewed By: quark-zju
Differential Revision: D43970589
fbshipit-source-id: e145d36275e4af18ec77f4fb3b018f85df8f39f6
Summary: I need to differentiate the symlink error in the status code path.
Reviewed By: quark-zju
Differential Revision: D43970579
fbshipit-source-id: 7931f83f828352a8f5ca8f90ff037702b0c64369
Summary:
Don't error out for files with negative or huge mtimes. We use the same masking approach as Python for now.
Note that Rust status does not write the mtime back to dirstate in general right now, so success is merely that "hg status" doesn't crash in this case.
Reviewed By: zzl0
Differential Revision: D43970588
fbshipit-source-id: f1ea50eaeeee2fc9b564934ec587c9f02abaf651
Summary: Previously this diff fixed eagerepo file store to strip copy metadata, but that was handled better by D43972516. Now this diff just enables rust status in affected tests.
Reviewed By: zzl0
Differential Revision: D43970584
fbshipit-source-id: 8d2f87128a06f33524078d476bd9fb68cf6b642e
Summary: These will help me figure out "status" discrepancies.
Reviewed By: zzl0
Differential Revision: D43970581
fbshipit-source-id: 63ef7578b6aec100b172da9516292aa549b8d318
Summary:
The new status doesn't work w/ legacy repos, so the tests need to be migrated. For each test, first I tried migrating to eagerepo, but in many cases that didn't work. Next I tried a remotefilelog repo (AKA "newclientrepo"). If I still couldn't get the test to work I just disabled rust status. This was mostly in tests that disabled treemanifest or were explicitly coupled with revlogs somehow.
Some tests had acute "hg status" discrepancies, so I left a FIXME(status) and disabled rust status.
I added a new "eagerepo" tinit.sh function that turns on eager format (for client repos). With this, tests doing "hg init" might just work with features that expect a modern repo.
Reviewed By: quark-zju
Differential Revision: D43839499
fbshipit-source-id: ce6d3b2747b90d31bd246c30551bdb55c446112f
Summary: Recycles our POSIX `checklink` implementation so that it can be used with Windows. This should be safe given that D43456961 gates the Windows symlink functionality.
Reviewed By: muirdm
Differential Revision: D43456959
fbshipit-source-id: eaf20b692b118642606cb59ffbc01c3052f05522
Summary: This should allow in later diffs to enable symlinks on Windows if users have `windowssymlinks` on their repo store requirements. Additionally, it adds the aforementioned requirement to new repos cloned with the `symlinks.force-enable` config.
Reviewed By: muirdm
Differential Revision: D43456961
fbshipit-source-id: bd2f5a1cb6c3b647ebccb0de556959758a4d7fe2
Summary:
`os.fork()` (used by pfcserver) only duplicates one thread. That is problematic
for `ctrlc` because the background thread that handles the Ctrl+C won't exist
for the forked processes. Fix it by moving the ctrlc setup to after os.fork.
Reviewed By: muirdm
Differential Revision: D44048693
fbshipit-source-id: 5d4391dabb7b4fc6e3cc1e015c7d0acdb017041e
Summary: Add minimal pathcopies() impl that only works for direct parent/child commits. This is for legacy test compat so more tests can be migrated directly to eagerepo.
Reviewed By: quark-zju, zzl0
Differential Revision: D43839498
fbshipit-source-id: 4cee903c9d9838ee1dd013e2fbc5748c3861c444
Summary:
Now when fsmonitor.track-ignore-files in the config differs from the value stored in the treestate metadata, we reset the watchman clock to trigger a fresh instance. This makes Watchman walk the repo and return all files. In turn we use this complete list to migrate the treestate to contain or not contain ignored files, depending on fsmonitor.track-ignore-files.
Note that I changed the Python logic to default to track-ignore-files=False, and to assume the absence of the flag in the treestate metadata means the treestate doesn't contain ignored files. I made this change because I didn't want Rust to default to assuming "enabled" because that would trigger a fresh instance the first time "status" is called. I don't think this will materially affect the Python behavior.
Reviewed By: quark-zju, zzl0
Differential Revision: D43789376
fbshipit-source-id: c97fcc83bb7ba2744c6be954246fa8bb499f0c32
Summary:
The main thing is to add methods to TreeState to get and set the entire deserialized metadata. The existing get_metadata_by_key would deserialize the metadata each time it was called.
I removed the Metadata type from most places. It is a wrapper type only used for serialization, so it probably shouldn't be public at all.
I also renamed set_metadata_by_keys to update_metadata, which goes better with the new set_metadata.
Reviewed By: zzl0
Differential Revision: D43970591
fbshipit-source-id: 203cc68acd6476e5e11608fb4858c9abb77ec03c
Summary: After clearing the NEED_CHECK flag, delete treestate entry if all flags are empty. This cleans out untracked/ignored files that no longer exist on disk. I don't think this had an impact on correctness or performance, but we should keep things tidy.
Reviewed By: quark-zju, zzl0
Differential Revision: D43789378
fbshipit-source-id: 60777a0a2e9cef2b0dc85f96891513256dc407e1
Summary:
Previously the Rust status processed all results from watchman which meant it would insert ignored files into the treestate (as NEED_CHECK). This had no correctness issues, but probably had some performance impact.
Fix by respecting fsmonitor.track-ignore-files as to whether we should track ignored files in treestate. If it isn't enabled, we skip ignored results from watchman.
Tracking ignored files is required for complete correctness so that the files show up as "?" if they are removed from the ignore rules. We don't have fsmonitor.track-ignore-files turned on due to performance issues tracking a huge number of ignored files, but those issues may be moot now with efficient state flag based treestate walking (needs more work to enable this by default).
Reviewed By: zzl0
Differential Revision: D43789377
fbshipit-source-id: 43630bd3e8bc6b1e43ea20b8c8bd75057bece30c
Summary: Now that normalized_get() is available on the TreeState, it's less important to let the FileChangeDetector query the treestate. Further, I want to add some logic based on the treestate entry to watchmanfs, and having the caller pass it in means the caller will have it in-hand to use without a redundant query.
Reviewed By: zzl0
Differential Revision: D43970587
fbshipit-source-id: 9b7e7cdeb5654f0fabd5854115af572c32ddd833
Summary: It saves some allocations and annoyance to have a single method that normalizes the path and fetches the treestate value.
Reviewed By: zzl0
Differential Revision: D43970580
fbshipit-source-id: 4ef7a53cb73736a43f674fa37ee5837d5e952e3c
Summary:
Added `find_renames`API, this method will find renames files between two commits (TreeManifest). This will be used in the `trace_rename_XXX` APIs to trace the rename history.
```
rename history → loop {
narrow down to one commit
find_renames for one commit # compare current commit and previous commit
}
```
Reviewed By: quark-zju
Differential Revision: D43933932
fbshipit-source-id: d83f1c594ebcf8283a3b0a3aa9d9290d0b95d5ed
Summary: Add `read_rename_metadata` method to `ReadFileContents` trait, so we can read the `rename` info from a file. This will be used in the `find_renames` API in following diffs.
Reviewed By: quark-zju
Differential Revision: D43919689
fbshipit-source-id: 00bc5bdd3563b62a6740aa3fbd5eabe2e8c7e29c
Summary: If a user cloned the upstream repo then added their fork as "default-push", we would still try pushing commits to the "default" upstream repo. With this change, we prefer "default-push" when pushing commits.
Reviewed By: zzl0
Differential Revision: D43919071
fbshipit-source-id: fc1ab73ba8ff0b5aee3e34fdf92f8969b9269650
Summary:
Having imbalanced or otherwise invalid "<" or ">" in your username messes things up when pushing commits to a Git server. Let's add some basic validation and trigger it before we create Git commits.
Updates #535
Reviewed By: zzl0
Differential Revision: D43926284
fbshipit-source-id: 9a8bc90af5dbff471fbc3083df2be9190c36a611
Summary:
Move 'strip_metadata' to a separate create to we can use it in other
crates, for example, we will need it in eagerepo in the following diffs: D43972516, D43919689.
Reviewed By: muirdm
Differential Revision: D43972373
fbshipit-source-id: d8a91eb21229967e37fdae5da9ef8f1baf050b2c
Summary: And ignore *.profraw files. There are a bunch of tests failing when run w/ test coverage profiling due to these droppings.
Reviewed By: quark-zju
Differential Revision: D43986480
fbshipit-source-id: 481f88e68c09c6ed3b2903d54f03431c4a04e1c2
Summary:
Add tests to make sure we mark files NEED_CHECK properly even if the given matcher doesn't match the files. Although we try to limit work based on the matcher, we need to at least record NEED_CHECK for files reported by watchman since we will update the watchman clock in the treestate (and won't get notified about the non-matching file next time).
Note that this isn't right for ignored files which should not be tracked in the treestate by default. Will address in following diffs.
Reviewed By: quark-zju
Differential Revision: D43760941
fbshipit-source-id: f445f9576323b730e72d9df1acce2a26b8a3ef50
Summary: Fill in status_detail when the native status falls back to Python due to needing the morestatus extension.
Reviewed By: quark-zju
Differential Revision: D43626599
fbshipit-source-id: 4d4826a9a178557a7a4f649eb9ec7817bf707151
Summary: Add a parallelized FileChangeDetectorTrait impl. It is enabled by default but can be disabled by setting workingcopy.watchman-worker-count=0.
Reviewed By: quark-zju
Differential Revision: D43626598
fbshipit-source-id: b250b5dfa5552b5ee52b993731a09842cc0639f6
Summary:
I want to parallelize the work this object does, but the trait interface is serial (and synchronous):
```
pub trait FileChangeDetectorTrait {
// Check if file size and mtime are unchanged (otherwise queue "maybe changed" result).
fn has_changed(&mut self, ts: &mut TreeState, path: &RepoPath) -> Result<FileChangeResult>;
// Resolve "maybe changed" files by checking contents.
fn resolve_maybes(&self) -> Box<dyn Iterator<Item = Result<ResolvedFileChangeResult>> + Send>;
}
```
It also was a bit annoying to use because it reported results w/ two different types, so the client needed duplicate handling. The new trait allows for asynchronous processing and only returns a single kind of result:
```
// Converting to an iterator signals you are done submitting paths.
pub trait FileChangeDetectorTrait: IntoIterator<Item = Result<ResolvedFileChangeResult>> {
fn submit(&mut self, ts: &mut TreeState, path: &RepoPath);
}
```
I converted the existing concrete implementation to the new trait. It requires the object to buffer more results, but not any file data, so it shouldn't have much of an impact memory wise.
Reviewed By: quark-zju
Differential Revision: D43626597
fbshipit-source-id: 4ffcd9b870ee8a4fba94aa3972cc4e1a299ebec5
Summary: I want to share this logic w/ a different file detector architecture, so pull it out and consolidate into a single free function. This has one concrete advantage of avoiding a duplicate treestate lookup for every file. This may not be the final form, but it's easiest for me right now.
Reviewed By: quark-zju
Differential Revision: D43626600
fbshipit-source-id: 6ee4355110fc76f2542bf78d8be348e894e73cbd
Summary: Add progress bars for work we do processing the watchman (and treestate NEED_CHECK) results.
Reviewed By: quark-zju
Differential Revision: D43626601
fbshipit-source-id: f7e56543955b12a7d89ff9073fa13d2265ba948f
Summary: I thought the `get` method would return the contents of a file path as well, which is not the case. So I added a doc to that function to make the assumption explicit.
Reviewed By: quark-zju
Differential Revision: D43919690
fbshipit-source-id: 10b499e16f1910ee3ac24ef34a95f18e123a95c0
Summary: The fold action collapsed commits up to and including the working copy, but it left the working copy dirty. When it subsequently updated to the new collapsed commit, it "merged" all the files. They merge cleanly since their contents are identical, but we still invoke the merge drivers (why are we invoking merge drivers if the contents are identical?). Fix by using ctx.markcommitted() which updates the dirstate based on what was committed. And, we no longer need to do the "hg.update()" anymore since the working copy will already be on the collapsed commit.
Reviewed By: quark-zju, zzl0
Differential Revision: D43718659
fbshipit-source-id: 26865cca11feb600e21933c60ed83796212da954
Summary:
`nullid` in manifest is invalid (upstream `hg verify` will complain).
The eagerepo `filelog.cmp(nullid, b"")` returns `False` (same content),
which makes `repo.commitctx` think that the file hasn't changed and reuses the
`nullid` in manifest incorrectly.
The original `filelog.cmp(nullid,b"")` would return `True` (different content)
since it calculates the real hash and nullid does not match the real hash of
empty content.
Reviewed By: zzl0
Differential Revision: D43920391
fbshipit-source-id: 7573647440a636ff92c42192a1cf2085c892e4dd
Summary: The `ContentMetadataV2` provides the seeded blake3 hash for each created file. This information needs to be passed to `buck` which interacts with `EdenAPI` through `EdenFS`. The relevant endpoints in `EdenAPI` are `trees` and `files` which are modified in this diff to support the new hash.
Reviewed By: markbt
Differential Revision: D43504459
fbshipit-source-id: 3980494108014d687529b77791012a3b57e84903
Summary:
This is intended to be used for ISLv2 stack editing. The command exports
related file contents.
Reviewed By: muirdm
Differential Revision: D43773252
fbshipit-source-id: c8a58c090474efb0e0a79759efc3c7c45e2a97c1
Summary:
As an [user pointed out in #544](https://github.com/facebook/sapling/issues/544), `sl` crashes when `gpg` returns an error code instead of returning a slightly more useful error message.
This diff also changes the gpg flags so that it doesn't error out due to lack of user input. Previously it would return an error 2 when it waited for user input.
Reviewed By: zzl0
Differential Revision: D43860003
fbshipit-source-id: ecfcb6f8cabb736365ea407211437c52f878f0c8
Summary: Using python native base85 encode/decode by removing old implemented reference.
Reviewed By: sggutier
Differential Revision: D43700467
fbshipit-source-id: 8bc223bd9cb0054c0f227557bea719d6f6f91165
Summary: This diff is a basic implementation of `RenameTracer` to pass the unit test added in a previous diff.
Reviewed By: quark-zju
Differential Revision: D43724672
fbshipit-source-id: 1c9bb48064371f97440b7453bb708b7c494f399c
Summary: The for-loop is only iterating the `missing_ids`, so we just need to check if `missing_ids`
Reviewed By: quark-zju
Differential Revision: D43735124
fbshipit-source-id: 557a083854899357f332b38b018fad8a973d70c5
Summary: Add a simple unit test to show how the API will be used and guide the API implementation.
Reviewed By: quark-zju
Differential Revision: D43724673
fbshipit-source-id: 6b5f126b356d7d320040b7e2d5bc61508cf064e7
Summary: In this case, the revlog could be opened, but crashed when trying to add something. Let's catch that error and nuke the undo revlogs.
Reviewed By: evangrayk, zzl0
Differential Revision: D43704364
fbshipit-source-id: 50ba67fa17b4a31676efe9b9925bdb1ff5c574ba
Summary:
Support binary content by prefixing `base85:`.
This makes it easier to produce non-utf-8 content for testing purpose.
Reviewed By: muirdm
Differential Revision: D43773249
fbshipit-source-id: a83706038bcc4865412b694966cfcb4d69c823f9
Summary: This makes the API a bit easier to use.
Reviewed By: muirdm
Differential Revision: D43773251
fbshipit-source-id: 0a6cfb1d4d8812dfda50237d0fca508a0e24a6b8
Summary: If the user hasn't configured their ui.username, I would rather the command error out instead of using <user>@<fqdn>. On Windows in particular, this default gives a very wrong user name that gets immortalized in the commit cloud workspace. It's much less toil for the user to configure the username early.
Reviewed By: quark-zju
Differential Revision: D43668690
fbshipit-source-id: 64c4e0d235b2fe2fa80c76da6e04917573989a9b
Summary:
We've had reports of users getting "cached" permission errors because the pfc/chg server process has a out-of-date list of groups cached.
Linux seems to cache groups for the lifetime of the process, but MacOS seems to update them dynamically. This check shouldn't hurt on MacOS, though.
Reviewed By: sggutier
Differential Revision: D43676809
fbshipit-source-id: 7e4439465410567d83ceeaa4ad58bd332a03a4f7
Summary:
Some time ago we started shipping Python alongside our Windows build, which means that we haven't really supported Cygwin for some time. Thus, we can remove bits of code associated with Cygwin.
The removed lines of code would never really be executed since `sys.platform` (which is what `pycompat.sysplatform` uses) always evaluates to `windows` with the Python build we ship along the Windows build. The value of `sys.platform` is determined when building the Python binary.
Reviewed By: quark-zju
Differential Revision: D43456960
fbshipit-source-id: 659eb8750602fb42f2325b7f16c5497c83ec5d32
Summary:
The revlog clone code path is incompatible with the commitgraph endpoints.
Disable the commitgraph endpoint during revlog clone to fix it.
Reviewed By: muirdm, singhsrb
Differential Revision: D43737424
fbshipit-source-id: 6f1b39faa42fd0e760c5f21682f64bb34a7a417e
Summary:
skip logging as "bad" requests for health checks
health checks are currently implemented like simple connections to the port:
```
def _test_service_is_running(ui):
port = ui.configint("commitcloud", "scm_daemon_tcp_port")
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
if s.connect_ex(("127.0.0.1", port)):
_warn_service_not_running(ui)
s.close()
```
so, skipping errors coming from json parsing for those type of checks
todo: implement a proper health_check request which can be part of migration to
json rpc
Reviewed By: quark-zju
Differential Revision: D43693235
fbshipit-source-id: 689cd3368c665fefd61a86d1125dc8d2aba1354d
Summary: A more correct version of D33745363 (d4d662b828). This makes the hgbuild output easier to look at.
Reviewed By: zzl0
Differential Revision: D43703395
fbshipit-source-id: d72180db85da51f34e0a48843e22237797146c98
Summary: This test started failing after recent Rust "chrono" crate update in D43683906 (69c6d10d30). Chrono now caches the TZ variable on unix, which defeats the test's attempt to change TZ. (The debugruntest test runner runs "hg" commands in-process for speed.) Add a "sleep 1", which is the threshold chrono uses to check for an updated TZ value. Note that the "sleep 1" is still much faster overall than the alternative of disabling the in-process hg execution.
Reviewed By: zertosh
Differential Revision: D43713144
fbshipit-source-id: 7ccedcf75db6b1b1d45dd526ee9018569e8e31f3
Summary:
This update is needed in order to update to routinator-0.12.1
(D43638969).
Deprecated methods have an `_opt` alternative. Before these panicked on
invalid values. Now they return `None` and the panic is made explicit
with `unwrap`. This diff fixes most deprecations, but not those that
require type changes.
https://github.com/chronotope/chrono/releases/tag/v0.4.23
Reviewed By: d16r
Differential Revision: D43683906
fbshipit-source-id: 07a101a148e013d98286e85d5821588a403f5e5d
Summary: The Rust status wasn't detecting files deleted while watchman wasn't running. On the following "sl status", Watchman reports "fresh instance" and gives a list of all the files on disk. We now infer deleted files by comparing the treestate and the list of files that still exist.
Reviewed By: quark-zju
Differential Revision: D43501745
fbshipit-source-id: 790ab8073e88b368e68c981afa8ca03a710d32ff
Summary: Add test demonstrating the native status failing to detect a file deleted while watchman isn't running.
Reviewed By: quark-zju
Differential Revision: D43501744
fbshipit-source-id: 717f212b58aca6c4d2f1bba2979cb1ee4f2919aa
Summary:
These tests stub out watchman and the filesystem and test NEED_CHECK bookkeeping and propagation of pending changedness.
Next I will expand the tests to cover the "fresh instance" case, which has some differences.
Reviewed By: quark-zju
Differential Revision: D43501752
fbshipit-source-id: 60031122d2ad7f4d76f4c478fbe9d426e425ec3b
Summary: Handle clock update in the outer method so we don't have to pass it around so many places.
Reviewed By: quark-zju, zzl0
Differential Revision: D43501747
fbshipit-source-id: 5e1dc7e08b1bb9d1fc4f91a23c4bd6c01ddb902a
Summary: These traits were previously used to abstract the treetate in tests, but the treestate is easy enough to work with. I think it's much simpler overall to use the concrete TreeState in code and tests (more tests to follow).
Reviewed By: zzl0
Differential Revision: D43501754
fbshipit-source-id: 6b106d0d814afbaba14b58e5f2f2fd23c519eeb6
Summary:
There isn't any parallelism, so this type doesn't make sense conceptually. It is used for convenience so various helper structs can hold a mutable TreeState without dealing with references. However, it leads to code such as the FileChangeDetector which needs to lock the mutex O(changed files) times.
Instead, let's just pass `&mut TreeState` as a parameter. The TreeState a fundamental input to most code in here, so conceptually this makes sense to me.
Reviewed By: quark-zju, zzl0
Differential Revision: D43501748
fbshipit-source-id: 3320b0dade2ffc3949e65b91b7adcb3368366d7b
Summary:
This was a container type that didn't have a cohesive purpose. Now most functionality lives in the detect_changes free method.
I will restore test coverage in a later diff once I'm done rearranging things.
Reviewed By: zzl0
Differential Revision: D43501750
fbshipit-source-id: 38904dbc996d7f64f8abc2246fe92d44e2e8cf64
Summary: And fix warnings to use stderr instead of stdout.
Reviewed By: evangrayk
Differential Revision: D43501753
fbshipit-source-id: ba44fd2aa872f1077a22dce0bf8a22d56f22ef1a
Summary:
The WatchmanState struct seems to add an unnecessary level of indirection during pending changes computation, so I want to eliminate it.
In this diff I separate out the core "compute changes given list of needs-check files" into a separate (easily testable) function.
I did not intend any behavior change except for the minor optimization where we no longer add files to "needs_mark" if they came from the treestate originally (i.e. they are already marked, don't need to mark again).
Reviewed By: quark-zju
Differential Revision: D43501751
fbshipit-source-id: e19573e62eb67d03f7ddeddfff703694951c70f6
Summary:
The watchman config fields were only used in one spot, so separate them out of the state.
I'm preparing for some more changes, probably culminating in the elimination of WatchmanState.
Reviewed By: evangrayk
Differential Revision: D43501749
fbshipit-source-id: 7e62ad10b314e1239f7fbe68fa084919bc76330f
Summary: Apparently 600ms is typically sufficient because no one complained!
Reviewed By: zzl0
Differential Revision: D43501746
fbshipit-source-id: e22cc2650fc0ca84a42090a863b924ac7a6b595e
Summary:
tidy up the logic for updating backup state
this is to fix that we call the update twice for cloud joins, also fixing
devel-warn: excess usage of repo.__contains__ at: /opt/fb/mercurial/edenscm/ext/commitcloud/sync.py:518 (<listcomp>)
warning
Reviewed By: quark-zju
Differential Revision: D43659249
fbshipit-source-id: 927e53b0c9f8640446a2a2173450e42ce5947e4c
Summary: The Rust `getdag` supports some options. Expose them in the Python `getdag`.
Reviewed By: evangrayk
Differential Revision: D43196294
fbshipit-source-id: b78153b8fd1b65f935aefd1608f7ec90891a4bc5
Summary:
Add a progress bar as we are committing files. This can take a long time with a large amount of files.
Also add a per-file progress bar to addremove (only for the "add" part).
Reviewed By: quark-zju
Differential Revision: D43664269
fbshipit-source-id: 7de3f82f142e3053c39ba1abd02bfdcab1a736ef
Summary:
Re-implement the BfsIter using a thread pool. This fixes two important problems with the async stream version:
1. With the old BfsIter, each level of search would wait 1+1=2ms if there wasn't enough data to fill the chunks (see chunks_timeout). This meant using an ExactMatcher to bulk fetch a short list of paths would have unavoidable overhead of 20ms+, depending on the depth of the paths.
2. The old BfsIter stream pipeline was not being torn down correctly. It was leaking threads that were keeping references to the store alive, which in turn caused the TreeStore to not flush its caches since it never got dropped. I also think it is responsible for some of the file descriptor leaks.
The new implementation seems to also be faster for big walks. I tested using mf.walk() with a large sparse profile matcher. The old BfsIter was ~27s, and the new is ~21s. Both take around 4s with caches hot.
For the ExactMatcher case, in my repo w/ caches hot the old BfsIter took ~25ms and the new takes ~1ms. This will shave off a good chunk from the native status time, which hits the BfsIter for files not found in the dirstate.
Reviewed By: quark-zju
Differential Revision: D43622477
fbshipit-source-id: 5abde6025405d06170b31ed65ccef4990d9f5474
Summary: Previously fold would do a full restack, rebasing all descendants of all predecessors. A full "fix everything" restack doesn't really make sense when the command itself just created the successor. Instead, now fold only rebases descendants of direct predecessors.
Reviewed By: zzl0
Differential Revision: D43660708
fbshipit-source-id: 6802343712ec6e8b2bebd587d7545a6667778947
Summary:
Currently, the `render` function always return a non empty Vector, even for an empty string (shown as below), this behavior triggered a deadlock bug in `hg continue` (potentially other places as well when trying to lock a Rust mutex while holding the GIL), D43605596 fixed the bug.
```
lib/hgcommans/src/run.rs:458] &changes = [
Text(
"",
),
]
lib/hgcommands/src/run.rs:459] &last_changes = []
```
The non-empty vector makes many comparison checks fail, for example, [this logic](https://fburl.com/code/75u6gvuj) will never be executed, since `changes.is_empty()` is always `true`.
```
// Fast path: empty progress, unchanged.
if changes.is_empty() && !inner.progress_has_content {
return Ok(());
}
```
This diff filters out empty strings from the Vector.
Reviewed By: sggutier
Differential Revision: D43611776
fbshipit-source-id: 67c07570dcd2176f05b8ab787a9f4860ea40a8c0
Summary:
The current `hg continue` works like below:
* `hg continue` call flow is like: Rust (`hg continue`) -> Python (convert to `hg rebase --continue`) -> Rust (`dispatch`) -> Python.
* when Python calling Rust with `hg rebase --continue`, it wraps `stdin`, `stdout`, `stderr` in `WrappedIO` as Python object. https://fburl.com/code/iq3rwndu
* `WrappedIO`'s methods require Python's GIL lock. https://fburl.com/code/zzbxffgg
We have two threads:
* Thread 1: background progress thread
* Thread 2: normal command execution thread
Deadlock happens when trying to lock a Rust mutex while holding the GIL:
* Thread 2 acquires the GIL
* Thread 1 locks a mutex (iostate)
* Thread 1 acquires the GIL, blocks (Thread 2 has the GIL lock)
* Thread 2 tries to locks the mutex, blocks (Thread 1 has the mutex)
This diff removes the reuse of Python object and reuses Rust's main io.
Reviewed By: quark-zju
Differential Revision: D43605596
fbshipit-source-id: ce9d8672a42635c57d00fe2b4e5eb9f50c885ec0
Summary:
remove redundant code
heart beats are now processed inside the library and the removed code is never
called
Reviewed By: zzl0
Differential Revision: D43624385
fbshipit-source-id: 15b5881a4ac0d35fbfbbb7bc3a3a3099d2f950cb
Summary:
Previously the Rust clone was disabled in Sapling mode since the Rust clone thought it could handle any URL scheme other than "file" and "ssh", but that wasn't true for https Git URLs, among others.
Invert Rust's logic to a supported list of schemes instead of unsupported schemes. In particular, support "mononoke" and eagerpo schemes "eager" and "test".
Reviewed By: quark-zju
Differential Revision: D43291978
fbshipit-source-id: 78656e0584c362aa9ae693535376f1026307e9ee
Summary:
Support resolving a given URL based on presence of custom "schemes" in config.
Kill special "fb" handling repo_name_from_url in favor of proper scheme support.
Reviewed By: quark-zju
Differential Revision: D43329356
fbshipit-source-id: 8f8cddcf2f77fdc9382e0d031ecb8d6cf4518d12
Summary:
use batch text fetching for fetching dates rather than single revlog
edenapi requests in a loop
follow up for D43572943 (008e88081a)
#shipit
Reviewed By: RajivTS
Differential Revision: D43619158
fbshipit-source-id: b22df9df2fb298ff6e4113a81c6b3e163cd88175
Summary: Added `RenameTracer` API, this will be similar to `PathHistory` excerpt that we will only care about file path (not the content id) when searching segmented changelog.
Reviewed By: quark-zju
Differential Revision: D43554314
fbshipit-source-id: ee4e7eb480750b982770631fc8d19e14c94d4c5b
Summary: there is no `first` parameter in the API, deleted the corresponding comment.
Reviewed By: quark-zju
Differential Revision: D43554315
fbshipit-source-id: 62b6de2a885705bef3e1d410bc98c42792dcd7f3
Summary:
reduce number of revlog requests as much as possible
basically, make it no more than number of commit cloud bookmarks that point to
the known public nodes in the repo
the idea behind this diff is to check phases before calling any ctx = unfi[cloudnode], where the revlog request is made
Reviewed By: markbt
Differential Revision: D43572943
fbshipit-source-id: 6e3a30d651b6de3ff83a335bc203ba1206ab27d1
Summary:
change order in revset
in my repo the current version takes 167.76 sec (missing head len is 1 commit)
vs almost instant if I reoder
it is not only extremely slow but also does some remote requests
I checked other places and it seems draft() goes first in several other places,
so reorder here as well
Reviewed By: mzr
Differential Revision: D43568675
fbshipit-source-id: 2397f90f9cb8156250fb951d948ce710071349b2
Summary:
log from what version to what version the local state has been changed
this is nice for debugging, currently we don't log what the previous local
version was
Reviewed By: markbt
Differential Revision: D43568053
fbshipit-source-id: fee2baf6c2c47bb01b3185a91a983cedd367c990
Summary: This diff adds a timeot to `reqwest-eventsource` `poll` so that we could exit running tasks when `interrupt` has been set to `true`.
Reviewed By: liubov-dmitrieva
Differential Revision: D43534730
fbshipit-source-id: 0f924a5c669f31540acab29d549f55091cd02485
Summary: This allows us to track how many tests are using legacy formats.
Reviewed By: sggutier
Differential Revision: D43546411
fbshipit-source-id: df0d711205ebc1d223c8f70cdbd65803bda4b3b9
Summary:
[Some OSS users](https://github.com/facebook/sapling/issues/521) had some issues launching `hg web` after following the instructions we provide in the website.
This was caused by the command trying to launch the `../../edenscm-isl/isl` script, which only exists if `sl` is installed. If one compiled sl using something like `make oss`, it would instead be located at `../../../addons/isl`. This commit changes the prefix based on whether `sl` was compiled in dev mode or not.
Additionally, this makes `sl web` launch `isl` directly via node instead of using some of the two wrapper scripts created when building ISL.
Reviewed By: evangrayk
Differential Revision: D43288130
fbshipit-source-id: 2ff5e6f25fefe80213e2315bea5676bd55f93dcf
Summary:
[sapling] Remove Cargo.lock on `make clean`
The help text for `make clean` says:
> remove files created by other targets
> (except installed files or dist source tarball)
So I'd expect it to remove any non-installed built state like Cargo.toml.
Concretely, I was surprised by this behavior when the build failed after
pulling in updates, even after running `gmake clean`. My Cargo.lock from a
previous build had pinned the watchman_client crate to an old version, but the
updated workingcopy crate relied on the addition of `BytesNameField` to
watchman_client@main.
I think it would be better if autocargo generated a Cargo.lock for us from the
Buck versions of dependencies, but as long as it's being generated as a build
artifact it should be removed by `make clean`.
Pull Request resolved: https://github.com/facebook/sapling/pull/534
Test Plan:
```
% cd eden/scm
% gmake oss
% ls Cargo.lock
Cargo.lock
% gmake clean
% ls Cargo.lock
% ls: Cargo.lock: No such file or directory
```
---
Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/facebook/sapling/pull/534).
* __->__ https://github.com/facebook/sapling/issues/534
Reviewed By: quark-zju
Differential Revision: D43478239
Pulled By: mshroyer
fbshipit-source-id: 2c472a0e1d86c85994191e242337dab8249bafaa
Summary:
`memmap` is unmaintained and hasn't been changed in over 4 years. `memmap2` is
a fork, which is regularly maintained.
Also bump dependency on `grep` as it removes the last third-party internal
reference to memmap.
Reviewed By: dtolnay
Differential Revision: D43486158
fbshipit-source-id: f763848bd295facf60ec601620afd68dfc655bea
Summary:
fix the unbatched location to hash/ hash to location calls within the cloud bookmarks loop
total number of bookmarks in the workspace can be large
Reviewed By: quark-zju
Differential Revision: D43356959
fbshipit-source-id: 4f01bf14fd42d31dcb51fa9a2f48603cc2b64736
Summary:
Partially restore D43263811 (21303b73c8). While `repo.edenapi` now goes through the Rust
code path. The Python code path relies on the extra handling to construct
`eagerpeer`. Restore the logic.
Reviewed By: sggutier
Differential Revision: D43516089
fbshipit-source-id: 9cabca67b1698bffbd51fb741408c2bd236653f6
Summary:
Previously, EagerRepo can only be used as a server for lazy/lazytext client
repos. This diff extends it to also support non-lazy changelogs so EagerRepo
can be used more widely.
Reviewed By: sggutier
Differential Revision: D43516090
fbshipit-source-id: 2adc43b1205b111393c53bfbf57f75dcc9ff8775
Summary:
Check that the eager repo store is used for eager repo. This prevents
mysterious errors like cannot convert filescmstore to Arc<dyn TreeStore>.
Reviewed By: muirdm
Differential Revision: D43516091
fbshipit-source-id: 7204177b48c8f6cab078d42f0092a45a3618e2f4
Summary:
So far we don't have a supported way to create a "remotefilelog" repo on
"init". They are done in a hacky way by appending "remotefilelog" to
"requires". That would cause issues with other requires, like eagerepo.
Detect them.
Reviewed By: muirdm
Differential Revision: D43516088
fbshipit-source-id: 8be0888f0742e310c7c2fc1e5d28fe4051c8056a
Summary: The ones which involve using `named_deps` are explained in D43511516. Rustdoc doesn't support libraries that have the same crate name as one of their dependencies.
Reviewed By: quark-zju
Differential Revision: D43517770
fbshipit-source-id: 4e543f8dd58aa8a1c5f3d73b92db12134661625e
Summary: This test broke recently due to some changes in the test headers and the way it outputs to a file, which this commit fixes.
Reviewed By: quark-zju
Differential Revision: D43515941
fbshipit-source-id: fdcbf0b890b28f9e868fe8cdc02e99d8331ed863
Summary: `test-check-code.t` was failing due to missing internationalization for `_("EXC\n")`. This diff fixes the issue.
Reviewed By: quark-zju
Differential Revision: D43512577
fbshipit-source-id: cdc6c0dc49d131f9a860bb110f47d5a411e6de41
Summary: `_rsrepo` is an object and not invocable. For some reason, our linters and pyre never caught this typo.
Reviewed By: quark-zju
Differential Revision: D43503666
fbshipit-source-id: 703cacfd3d881ee39f8b9909a7170afef321097b
Summary: Use `.gitignore` to achieve the same effect of slimming down the dep.
Reviewed By: diliop
Differential Revision: D43489210
fbshipit-source-id: de518882ed0bad26bbcefb2ffc8bd782e95af6bd
Summary:
The default hgrc has a different path (`.hgrc` vs `hgrc`).
Update the test to not require `.`.
Reviewed By: zzl0
Differential Revision: D43012614
fbshipit-source-id: 5e33c2dcdcba053c8540ae7ebc16d74e2a540a61
Summary:
The test wants to shell out to real bash scripts. Let's ensure that the bash is
ready for that.
Reviewed By: zzl0
Differential Revision: D42826350
fbshipit-source-id: 301d27fba98e72b8bb459946b15c322cfaf1f658
Summary:
It uses `tr` which is missing in `debugruntest` stdlib.
Let's just replace both `tr` and `jq` with a Python function to make it
debugruntest compatible.
Reviewed By: zzl0
Differential Revision: D42826353
fbshipit-source-id: e9e7d2da6d7bbf256d83050abb0ca2b58f5516f1
Summary:
If the commit is not a stack, the regular GitHub PR workflow works fine.
Skip reviewstack and stack boilerplate.
Reviewed By: sggutier
Differential Revision: D43481900
fbshipit-source-id: a7b6f93e1b6ac4de16469980b85c0c8261d5efee
Summary: This makes it auto-fix friendly.
Reviewed By: sggutier
Differential Revision: D43481899
fbshipit-source-id: e7c54980f1aac667f7e4cebb04ec420045db1655
Summary:
This is the last step in exposing aux data to EdenFS in trees. EdenFS still
doesn't do caching of these just yet, but that's coming in a subsequent diff.
Reviewed By: muirdm
Differential Revision: D43132762
fbshipit-source-id: 86488d0f071448e6e78d30e85e2963e001b1d67c
Summary:
This is unused, but we still allocate a Vec for it, remove it to save on the
allocation.
Reviewed By: genevievehelsel
Differential Revision: D43132767
fbshipit-source-id: 715e4b3e030887ccad464a2d881acf11802b1b45
Summary:
Now that aux data can be collected from tree fetches, let's move it closer to
EdenFS by having the backingstore collect them.
Reviewed By: muirdm
Differential Revision: D43132764
fbshipit-source-id: c74898a7886e927ed9ef81a28fead1ce6fca1df3
Summary:
On EdenFS, it is very common to get a Tree and then immediately after read its
aux data. This is so common that the lock on the aux data IndexedLog is
extremely contended, leading to most of the time being spent waiting on taking
it. To avoid this, we should expose the aux data already present in the EdenAPI
result and return it to EdenFS immediately.
For now, this merely introduce the API, future diffs will plug it deeper into
EdenFS.
Reviewed By: muirdm
Differential Revision: D43132763
fbshipit-source-id: 13d2158e6f2b6d52905ce684bc0f12096a95920c
Summary:
Hg specifies the "null" id to be empty context. Respect that.
This makes `debugexportrevlog` work in `debugdrawdag --no-files` repos.
Reviewed By: zzl0
Differential Revision: D43373719
fbshipit-source-id: 13bdcbc03480482eee812dc8c485debdeac43dc3
Summary:
This is now done in a lower layer in Rust. So it's no longer necessary in
Python.
Reviewed By: muirdm
Differential Revision: D43263811
fbshipit-source-id: db69a4da749a6b7a405ff388af0ed533721c3ec1
