Summary: noticed these in passing while working on previous diff
Reviewed By: mitrandir77
Differential Revision: D27227682
fbshipit-source-id: e7858c81951b780722b0836ecf6ee72aeb1ffa09
Summary:
The latest git prints extra hints:
$ git init repo
+ hint: Using 'master' as the name for the initial branch. This default branch name
+ hint: is subject to change. To configure the initial branch name to use in all
+ hint: of your new repositories, which will suppress this warning, call:
+ hint:
+ hint: git config --global init.defaultBranch <name>
+ hint:
+ hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
+ hint: 'development'. The just-created branch can be renamed via this command:
+ hint:
+ hint: git branch -m <name>
Initialized empty Git repository in $TESTTMP/repo/.git/
Silent it to make the test pass.
Reviewed By: akushner
Differential Revision: D27235547
fbshipit-source-id: 6f7c8da3ac1e01ee9f57730e1586fa2053f9ca98
Summary:
a workspace could be renamed from a different machine
educate users that they need to switch to a valid workspace
Reviewed By: markbt
Differential Revision: D27155544
fbshipit-source-id: eed066f2f3e6ebf99732499fdb355f8aebb4c1df
Summary:
this could happen if it has been removed from a different machine
In this case, we should educate the users about the '--force' option
Reviewed By: markbt
Differential Revision: D27155423
fbshipit-source-id: 41cc3ac769dfd4145031fef687e8069d0ef8f4c9
Summary:
provide better ux for switching workspace if the current has been renamed from another machine
sometimes users rename a workspace but it's unclear for them how to switch other machines to the new one
we should educate them about '--force' option
Reviewed By: markbt
Differential Revision: D27117194
fbshipit-source-id: faef1cf9ce64f054f715ef3683a133d3088ddc72
Summary:
We've been seeing flaky test output due to python2 and python3 stdout being different
Bring getdeps builds unambiguously onto python3.
Reviewed By: mitrandir77
Differential Revision: D27190200
fbshipit-source-id: e53327fc3293a0ccdf88d8a199b66eddb4b9b8e4
Summary: There were a few remaining python2 parts to remove and the error case from hgcommand.run() had wrong encoding
Reviewed By: mitrandir77
Differential Revision: D27194046
fbshipit-source-id: 8c154204991e5a587f70ae4b18c73c4f005880ab
Summary:
This is unused, and the Thrift file mentions that it should be fine to remove
in July 2020. It's now March 2021, time to kill it.
Reviewed By: chadaustin
Differential Revision: D26852134
fbshipit-source-id: 2872185edd834f889b78802210071d16b881e14c
Summary:
See the previous diff for context. Disable the Rust progress for external
pager.
Reviewed By: kulshrax
Differential Revision: D27149241
fbshipit-source-id: 4260a8be55bbfa648d8910f021195e9d11bdab73
Summary:
When testing "disable_progress" with chg (next diff) I found it was not
effective because there are 2 separate IO structs. The one we disable
from Python is different from the one the Rust progress thread uses.
I traced it down here. Since the Python IOs are just wrappers of
Rust IOs in the chg use-case. There is no need to recreate an IO
struct.
The "creating IO" struct is still useful, for things like "-t.py" testing where
the output needs to be captured into different Python variables per different
commands.
Reviewed By: DurhamG
Differential Revision: D27149243
fbshipit-source-id: 6e27adcc9f48b21fc24fba120be8c4a8fef1f909
Summary:
In some cases (ex. using an external pager). The IO states are changed outside
the IO struct's control. Ideally we should implement the external pager logic
on IO too but for now let's just add an API so the Python external pager logic
can disable progress output after starting an external pager.
Reviewed By: kulshrax
Differential Revision: D27149242
fbshipit-source-id: ff51fc153d3cc211cfa8ef697923d36f7c0f0d9b
Summary:
Detecting prod on Windows wasn't working because we used a posix path.
Let's add the Windows equivalent.
Also moves us to use the new hostcaps crate.
Reviewed By: chadaustin
Differential Revision: D27126497
fbshipit-source-id: 4035012fb7701378fb6e2e902c0efcd54ef42ea9
Summary:
We've been seeing issues where repositories end up with incorrect
dynamic configuration since there's a window of time after they're cloned where
they don't have %include /etc/mercurial/.../repo.rc and therefore generate an
incorrect dynamicconfig which gets used for 15 minutes until we regen the
dynamicconfig.
Let's change hg clone to write the %include as part of the initial hgrc, so we
remove that window of time and the repo will always be correctly configured.
Reviewed By: quark-zju
Differential Revision: D27093772
fbshipit-source-id: a9ca0ec54e06549546d532d1c49a80d49981decf
Summary:
I'd like to add support for event handlers in the Rust ServiceRouter client (I
need this in order to inject CATs in calls made by the SMC client). To do so, I
need to be able to instantiate a `ContextStack`, and to do so, I need a static
c-string representing the service name & function name, which is what this diff
does.
Note that we actually do the same thing for Rust servers already.
#forcetdhashing
Reviewed By: farnz
Differential Revision: D27088187
fbshipit-source-id: be2ad541d5123b31f0dab73da16b35dbfd308d6f
Summary:
In regular xcode this was warning and being ignores. Not the working is handled as an error.
This diff is only a workaround so we wont get those errors .
```
eden/scm/edenscm/mercurial/cext/osutil.c:745:49: error: passing 'const char *' to parameter of type 'char *' discards qualifiers [-Werror,-Wincompatible-pointer-types-discards-qualifiers]
ret = _listdir_batch(path, pathlen, keepstat, skip, &fallback);
^~~~
eden/scm/edenscm/mercurial/cext/osutil.c:586:11: note: passing argument to parameter 'skip' here
char* skip,
```
Reviewed By: mzlee
Differential Revision: D27136440
fbshipit-source-id: 00d61fd00e3ed8e23643ea69b5a82dbeb5e742ce
Summary:
With `sl -r OBSOLETED` the intention is see the obsoleted stack instead of just
a single commit. So filter the "::heads" with "- public()", not "& draft()".
The goal is to deprecate `--hidden`. See the linked post for more context.
Reviewed By: DurhamG
Differential Revision: D27093425
fbshipit-source-id: 76e9650a809c1d94da2341e2aca31d349487610d
Summary:
This creates .hg directories when there is no repository at all, which
breaks jf submit in git repos. D26801059 is the real fix, but it has some other
complications. Let's drop the creation here, since it really isn't necessary.
Reviewed By: quark-zju
Differential Revision: D27134087
fbshipit-source-id: d15048b2d1022d38393b62cc02ebf022e617ed4f
Summary:
When using `log -r REVS` with filtering flags like `-u`, `-d`, preserve the
prefetch information by using the `revs(subset=subset)` API.
Reviewed By: sfilipco
Differential Revision: D27119174
fbshipit-source-id: 8483d7113cfc819c6053d1429221588c3a917c12
Summary:
This allows specifying subsets. So we can rewrite:
revs & repo.revs(expr)
to:
repo.revs(expr, subset=revs)
The latter will apply prefetch tweaks on subset when evaluating the revset
expr, while the former will lose prefetch information.
There might be a way to make `revs & repo.revs(expr)` do the right thing
for pre-fetching, too. But that could be more complicated w.r.t `&` fast
paths, over fetching (?), etc. For now I just took the fix that looks more
obvious to me.
Reviewed By: sfilipco
Differential Revision: D27119175
fbshipit-source-id: 2629d21594cf97d7c0f63cf085a2c427d8782e58
Summary:
The filteredset can often be expensive filtering commits like `hg log -u foo`,
`hg log -d '2010-1-1'`. Add a progress bar to show what's going on.
Reviewed By: sfilipco
Differential Revision: D27119176
fbshipit-source-id: 458fbf331978b26e78e6a85fb194ae8b12b949d6
Summary:
`CommitHashToLocationRequestBatch` and `CommitRevlogDataRequest` implemented `FromJson` but not `ToJson`. The latter is required so that the EdenAPI client can log JSON representations of all request types.
In the process of adding tests for those two types, I discovered a bug in `CommitLocationToHashRequestBatch`'s `ToJson` implementation which prevented it from passing JSON roundtrip tests (which I added because this type didn't have test coverage here). The bug is now fixed and the roundtrip test passes.
Reviewed By: krallin
Differential Revision: D27105414
fbshipit-source-id: d2e23e21f996ac8062bca948ed9abe3779e4669d
Summary: Use a macro to define quickcheck tests that perform roundtrip conversions for EdenAPI types that implement `ToJson` and `FromJson`. This makes it easier to add additional roundtrip tests for new types.
Reviewed By: krallin
Differential Revision: D27105416
fbshipit-source-id: c6059f9527d57b88cd6e3d1647fa89d8f826b1e3
Summary:
editmerge did not follow the same editor fallback described in
https://www.mercurial-scm.org/wiki/HGEDITOR and `hg help
environment`. Make the fallback behavior (especially to `vi` if none
are set) consistent by defining HGEDITOR before invoking external
merge scripts.
Reviewed By: farnz
Differential Revision: D27104830
fbshipit-source-id: 0ca657578f010a682afb84649f2f18ff46046864
Summary:
Rework the progress. Always move the cursor to the top-left corner of the
progress output. So the clear progress instruction is just to erase till
the end of the screen. There is no need to track the height of the progress.
Update flush logic so we only do flush when crossing the progress/non-progress
boundary: When we write progress (to stderr), flush stdout, write progress,
then flush stderr.
Also, disable progress unconditionally if the current output line is incomplete.
Reviewed By: sfilipco
Differential Revision: D27109228
fbshipit-source-id: 717345e9c7eaeebeb378ce090f7b2f60957fd150
Summary: This makes write_err() consistent with error().write().
Reviewed By: sfilipco
Differential Revision: D27109227
fbshipit-source-id: 4e7ea1fe464c17548c34e5f9bf78085868a20256
Summary:
See the previous diff. This avoids complex logic in the signal handler so we
don't need to think about whether the complex logic is reentrant or not.
Reviewed By: DurhamG
Differential Revision: D27111183
fbshipit-source-id: 110ef5699d5fcd007f8399e90443bb406e2b173c
Summary: The zz could match temporary file names on Windows. Let's add the _ to make it more unlikely to match temp files.
Reviewed By: quark-zju
Differential Revision: D27100451
fbshipit-source-id: 5646bb2ad1e52b1f49c29729d424111601468c69
Summary:
Now that EdenFS and Watchman are compatible with hg shelve and other
rapid commit changes, this warning is no longer necessary.
This backs out D25337932 (c1659751c9).
Reviewed By: quark-zju
Differential Revision: D27095024
fbshipit-source-id: 31af1d2dafaa0f340a4ace10a16f9357ed84dbf0
Summary: This line just collects a vector into a vector. Probably a remnant of a refactor. Delete it.
Reviewed By: quark-zju
Differential Revision: D27091643
fbshipit-source-id: fb611aabea375b8495476401b2d9cdf7ba12fde1
Summary:
These appear to have been made effectively dead by cleanup in D25313325 (23daa7f90f).
This is part of unblocking the hashed buck-out rollout effort
(https://fb.prod.workplace.com/groups/fbcode.fyi/permalink/3694589863910121/),
as `get_build_rule_output_path()` relies on hard-coded buck-out paths.
Reviewed By: mzlee
Differential Revision: D27072131
fbshipit-source-id: 4fccee06a73c4afbf89cb737b25e1713a1afc55a
Summary: This makes the new software respects writes by older software.
Differential Revision: D27093942
fbshipit-source-id: 097b57c61b5ee1f0264babb88737306113fe356a
Summary:
`hg stauts` can be "indeterministic" because of the last second mtime fix
special rule (see pytreestate/src/lib.rs:invalidatemtime).
The test sometimes fails like:
test-sparse-fetch-t.py:140: [] != ['x', 'x/x']
Update it to support both `[]` and `['x', 'x/x']` case.
Reviewed By: sfilipco
Differential Revision: D27071225
fbshipit-source-id: c413906897b408c1e85912852afed1717a87ffc9
Summary:
The error was triggered but it's unclear what's wrong. Make the error
more detailed.
Reviewed By: xavierd
Differential Revision: D27058212
fbshipit-source-id: 3f6220e2d100d9118c05a8b4c75c5ba19c9181db
Summary: This will be used by `doctor` command.
Reviewed By: sfilipco
Differential Revision: D27053349
fbshipit-source-id: bc33e25997f30107f919a090ff68693bfdd7199d
Summary:
By implementing DefaultOpenOptions, indexedlog provides `repair()` for free.
Re-export the `Repair` trait so other crates can use `repair()` without
importing indexedlog.
Reviewed By: sfilipco
Differential Revision: D27053352
fbshipit-source-id: 8fa952f0e51e007b9d348bc12699ef1d65000c6b
Summary:
With the new log for MultiMeta. It's now possible to repair a MultiLog by:
- Repair all Logs
- Scanning through the MultiMeta Log and find a valid MultiMeta.
- Set the current MultiMeta to the picked MultiMeta.
Reviewed By: sfilipco
Differential Revision: D27053346
fbshipit-source-id: d60596fb00323b3bcadd5ade2e34cad29a37d64a
Summary:
We recently saw a few reports about "multimeta" being 0-sized. MultiLog cannot
be repaired like other logs because the logs (ex. IdDag and IdMap) have to be
in sync. To implement Repair for MultiLog, let's track MultiMeta in a Log so
we can check its previous entries and fix the multimeta.
Reviewed By: sfilipco
Differential Revision: D27053347
fbshipit-source-id: af99b13d658ee62bfe63973ab9d37338d14a7d4a
Summary:
The test failed sometimes on Linux:
--- test-doctor.t
+++ test-doctor.t.err
@@ -204,11 +204,11 @@
M A2
A A0
A
- A X
R A
R A1
? B
? C
+ ? X
? Y
? Z
The treestate fix appears to rollback to even a previous version, which is also
a valid fix. Let's accept that state too.
Reviewed By: DurhamG
Differential Revision: D27064825
fbshipit-source-id: 6aab04e66ad14ad651f93805c9652c7423178665
Summary:
The test failed sometimes on OSX:
--- test-fb-hgext-fastlog.t
+++ test-fb-hgext-fastlog.t.err
@@ -34,6 +34,7 @@
$ hg log dir -T '{desc}\n'
b
a
+ Exception in thread Thread-3 (most likely raised during interpreter shutdown): (no-eol)
$ hg log dir -T '{desc}\n' --all
b
a2
The threading usage in fastlog does seem kind of risky (especially with async
Rust involved). Race condition in Py_Finalize is not at all fun. Let's just
make the test more robust for now. In the future we probably want to avoid
threading in fastlog.
Reviewed By: DurhamG
Differential Revision: D27064618
fbshipit-source-id: a6c2ee5eda0fbd5120c8b5e5cfcc7af0f158f9b9
Summary:
The test is failing:
--- test-fb-hgext-remotefilelog-repack-remove-old.t
+++ test-fb-hgext-remotefilelog-repack-remove-old.t.err
@@ -63,7 +63,7 @@
-r--r--r-- 80 *.datapack (glob)
-r--r--r-- 80 *.datapack (glob)
-r--r--r-- 80 *.datapack (glob)
- -r--r--r-- 144 *.datapack (glob)
+ -r--r--r-- 80 ef52660a201e447b43868610b08c72e22067b8b2.datapack
We are migrating away from repack so I just made the test pass without
investigating what's going on exactly.
Reviewed By: sfilipco
Differential Revision: D27064249
fbshipit-source-id: 6bcd583b6ecbe0b373d9fec2b23269b0da6a27f3
Summary: This makes it easier to spot problematic segments.
Differential Revision: D27025039
fbshipit-source-id: 44ae9eaf03ca54751b64c0cf977b78737eaa58f0
Summary: The Rust commit graph does not hardcode nullid. This avoids crashes in some cases.
Differential Revision: D27022659
fbshipit-source-id: 0f61ffd9da89de7a477470384c6015d9edfaaaa0
Summary:
We saw the `unwrap()` panic at `iddag.rs:453`:
let (new_idx, low, high, parent_count, has_root) = candidate.unwrap();
Reading the code, it seems it can only happen if one of `parents` is `high` in a segment.
Parents in a segment should all be < low. Add some validations for that.
Reviewed By: sfilipco
Differential Revision: D27021394
fbshipit-source-id: 2c51713dfb8f07f2921229d8866f7e8a0bd9490f
Summary:
For example, `hg log > a.txt`, writing to `a.txt` does not need to clear the
progress bars.
Reviewed By: andll
Differential Revision: D26886281
fbshipit-source-id: 392c50e62a854e03ea0c400a1dac8bfb900815a2
Summary:
The Rust progress bars now integrates with hg-http and all requests get
progress bars for free. There is no need for a separate bar.
Reviewed By: andll
Differential Revision: D26970747
fbshipit-source-id: c5e59c745e75a93e5da0541baab21c3f23dc1322
Summary:
Render the progress bars in a Rust thread.
Also make it compatible with traditional Python progress bars (not the most efficient way because the Python still have its own progress thread, but the code change is minimal).
Reviewed By: andll
Differential Revision: D26886282
fbshipit-source-id: ad1f862ced278cc1cb90c37f576eb05f52a50b13
Summary: 100 and 5k seems too small - requests are too frequent.
Reviewed By: andll
Differential Revision: D26886276
fbshipit-source-id: cb5cce1111120f8a437fbc8c60906940588f38bc
Summary:
Split `dispatch()` into 2 parts: config parsing and command execution.
This allows callsite to do extra work based on the config before running the
command.
Reviewed By: andll
Differential Revision: D26853749
fbshipit-source-id: 459918b43341f7e14df6958056b6e07453bb691b
Summary: This method should be preferred to use from python binding. It is similar to block_on but allow to interrupt future on Ctrl+C
Reviewed By: quark-zju
Differential Revision: D26894586
fbshipit-source-id: 02d8fbb2f9c4d9e585f8fbee51bfc400c402d0d2
Summary:
During sparse profile change, there might be a case when file has changed between checkouts, and also is added due to sparse profile change
Currently this would lead to creating duplicated update action that would waste resources. This diff deduplicates such action
Reviewed By: quark-zju
Differential Revision: D26953657
fbshipit-source-id: ccd9084cc3984ac0692ee6178fce018a94cd56e7
Summary:
Current store implementation returns value that may contain matadata.
We need to strip it before writing out file to disk
This is not going to be a problem with new Eden API, but it is not erady yet and rolling it out may take time
So in order to start testing native checkout now, we are incorporating metadata removal in the checkout code
Reviewed By: quark-zju
Differential Revision: D26953658
fbshipit-source-id: 05d8be55bb96e7c3162594498ef0b9944b422c90
Summary:
Add a way to register an IO time series showing total network I/O, and register
download progress bars for requests.
Reviewed By: andll
Differential Revision: D26853750
fbshipit-source-id: 42c4ab9724c60db9612a2bc419ca3bceaa667647
Summary:
It turns out there are multiple users sending requests bypassing the
HttpClient, like the LFS in revisionstore, or the segmented changelog
clone.
Requests bypassing HttpClient means HttpClient event callbacks do not
have a chance to insert progress and bandwidth monitoring. So let's
add another callback that can capture what HttpClient misses. This would allow
us to get proper progress bars of revisionstore LFS and segmented clone without
changing their code.
Reviewed By: andll
Differential Revision: D26970748
fbshipit-source-id: 5133bc6f9eeb14a6d2944d253bc66cefd49c83c5
Summary: Expose Rust's progress model so Python can operate on them.
Reviewed By: andll
Differential Revision: D26886283
fbshipit-source-id: 843de4cd2d8d22aba014ee097b0fee2feeb6ee43
Summary: Add a simple way to render progress into a multi-line string.
Reviewed By: andll
Differential Revision: D26853751
fbshipit-source-id: 4f1de55e7bb03f607d683eff16e035aa5d1476c1
Summary:
Add a struct to "register" progress data for Rust to render them.
Note: The "registry" only tracks the data. Rendering will be
implemented separately.
Reviewed By: andll
Differential Revision: D26615606
fbshipit-source-id: affa663660bd52a2583d40e3b28f0b65480851a3
Summary: Add an API to render the time series samples into ASCII graph.
Reviewed By: andll
Differential Revision: D26836633
fbshipit-source-id: 9c7a757e46c28c810fb3e0671a775fde16318f54
Summary: This could be used for showing overall statistics
Reviewed By: andll
Differential Revision: D26886280
fbshipit-source-id: a52f057dbbebb441d0eee14860275f2690c7a3a3
Summary: Add an API to render the time series samples into an array that can be further turned into an ASCII graph.
Reviewed By: andll
Differential Revision: D26615170
fbshipit-source-id: 4a2f178869c235e7dda3edef7930a6e04bc4bda3
Summary: Continue with last change. Add a time series type that keeps samples.
Reviewed By: andll
Differential Revision: D26615171
fbshipit-source-id: 5dcc775820d782ac502d84b6af5e270f2a9072ee
Summary:
Going to add some kind of "time series" graph based on periodical sampling.
Add the sample structure for IO with some simple math helpers.
Reviewed By: kulshrax
Differential Revision: D26615173
fbshipit-source-id: 4d610f435a3c7913e8bb31a8de0738ffb3de1468
Summary:
Going to add some kind of data model definitions for pure Rust logic.
Use a dedicated crate so progress model and rendering are separate. If the
rendering logic becomes bloated somehow, it won't affect the data models.
The idea is that only application-level logic like `hgcommands` care about
rendering. Other use-cases only care about pure progress states.
Reviewed By: andll
Differential Revision: D26836634
fbshipit-source-id: 5222d2f9fb4894bd246c62ad48e1c5fc982e2fa6
Summary: A recent diff modified this test and accidentally introduced a mismatch with Windows. Let's glob out the different path separators.
Reviewed By: quark-zju
Differential Revision: D26997063
fbshipit-source-id: 3cbe1f1fa748f8282fe16ee3d38f23519c87f6b9
Summary: With `ui.pushbuffer`, `out` can be `ui`, and it does not have `isatty()`.
Reviewed By: DurhamG
Differential Revision: D26992865
fbshipit-source-id: 1725291ba5ab3b7ccd8702fbb9df7213a05e3350
Summary:
Previously we referred to the config location as the repo_path or
repo_dir. In an upcoming diff we'll support NoRepo configuration, so let's
rename this to config dir and support using cache_dir as a config dir if there
is no repo. That code path isn't actually used in this diff though.
Reviewed By: kulshrax
Differential Revision: D26801061
fbshipit-source-id: 72cd672f65ca10821882ee9124fd4a293ead7257
Summary:
Previously we assumed there was always a repository. In an upcoming
diff we'll support configuring Mercurial without any repository. To support that
let's add a representation of NoRepo to the dynamicconfig structure and update
all uses to use that.
Nothing will currently create a NoRepo, but that will come in a later diff.
Reviewed By: kulshrax
Differential Revision: D26801060
fbshipit-source-id: d8e9066c865921f45e068c799ac8acb8eec32acd
Summary:
Pure Rust commands weren't loading dynamicconfigs correctly because the
validation was happening in Python. This diff moves it down into Rust so that
all config loading gets the same validation.
Also removes the hard coded list of files to validate. I'll add this to
configerator to make it easier for us to edit.
Also removes the configs.validatedynamicconfig option. Validation is on
everywhere and should just be on by default.
Reviewed By: kulshrax
Differential Revision: D26680075
fbshipit-source-id: f206198154801f655b4b277cdb22a728d0004b5f
Summary:
Previously, only --config configs were pinned in the uiconfig. "pinned"
means that no matter what values were loaded underneath, those pinned values
would not be changed. This allows CLI specified configs to take precendence
regardless of what and when config files are loaded. --configfile configs were
not pinned in the same way, despite it seeming like they should be pinned. In
fact, during clone we "reload" the config, which causes the --configfile value
to be lost.
Let's make --configfile pin all of it's configs as well.
This has a bit of an unfortunate side effect. When we do clone via hg-clone we
pass --configfile /etc/mercurial/repo-specific/$REPO.rc so the clone has the
right configuration. This diff means that those values are now loaded in a
pinned state, so dynamicconfig cannot overwrite them and the values cannot be
deny-listed.
The longterm solution to this would be to stop using --configfile during clone,
and instead use --config remotefilelog.reponame=$REPO and allow dynamicconfig to
generate the rest. But for now it's probably fine to let them be pinned.
Reviewed By: quark-zju
Differential Revision: D26677599
fbshipit-source-id: 2837a5b5d14bb42b49eaf9d1d0019f2ea0620a9e
Summary:
We've had a few issues recently where clones failed because of
unexpected config load ordering. The main problem is that we can't load repo
configs until the repo is created and the .hg/hgrc created. This happened rather
late in the process, for instance it happened after the source connection was
created, so no repo-specific dynamicconfigs could affect the source connection.
This diff moves the destination repo creation earlier in the clone process,
which will load the appropriate dynamicconfigs before we connect to the clone
source. This removes the last usage of applydynamicconfig, which will be deleted
in the next diff.
Reviewed By: quark-zju
Differential Revision: D26676519
fbshipit-source-id: 9a0f0b2ac0d9f19b8bb6fbaf26b4a5e7fb803399
Summary:
We've had a few issues recently where clones failed because of
unexpected config load ordering. The main problem is that we can't load repo
configs until the repo is created and the .hg/hgrc created. This happened rather
late in the process, for instance it happened after the source connection was
created, so no repo-specific dynamicconfigs could affect the source connection.
This diff moves basic repo creation (the directory, the .hg, and the .hg/hgrc)
earlier in the repo creation process and loads dynamicconfigs before proceeding.
This eliminates the use of applydynamicconfig in the localrepo constructor.
The next diff moves repo creation earlier in the clone process to then benefit
from this, and removes the final use of applydynamicconfig, thus preventing
config load ordering issues.
Reviewed By: quark-zju
Differential Revision: D26676520
fbshipit-source-id: 8784f2483909a50e8be9eb1a7adf03e8807ef076
Summary: These were generally useful when debugging native checkout.
Reviewed By: quark-zju
Differential Revision: D26955400
fbshipit-source-id: 967a4fa2d831b8b060409a5f7be31a1416c1ba63
Summary:
Add the `focusedbranch` revset. This expands to the current draft branch, plus
any commits that are related to the branch via commit mutation and their
descendants.
This can be used as the revset for `hg smartlog` to give a more focused smartlog
that shows only the changesets relevant to the current stack.
Reviewed By: quark-zju
Differential Revision: D26852107
fbshipit-source-id: 9723935bf3bf900f8e109d4f507ae1db6627807e
Summary:
When generating hg changesets from git changesets we can run into a situation
where two git commits are almost completely identical with the exception of
committer field. Currently it results in us generating identical mercurial
changesets for different bonsai changesets, and this is something we do not
allow.
Let's do the same thing as hggit does i.e. record committer field in extras if
committer is different from author.
In case of ambiguities (i.e. committer extra is already set, or committer date is set but committer name is not) I opted for returning an error instead of trying to guess what the correct field should be.
Reviewed By: farnz
Differential Revision: D26867740
fbshipit-source-id: 6271e2f7ad421bec296c60ff211723c2162878c6
Summary:
Similar to `output()`, and `error()`, provide a `progress()` to support setting
progress without needing `&IO`.
Reviewed By: andll
Differential Revision: D26886278
fbshipit-source-id: a54563a9cf1d9d1cdb9dabe945aeb5ed1d84e8fb
Summary: Make sure the progress is not hidden because of buffering.
Reviewed By: singhsrb
Differential Revision: D26886277
fbshipit-source-id: d44e48e67b4529a181dd36c30e92608654d8fea6
Summary:
Written by Martin von Zweigbergk at https://phab.mercurial-scm.org/D9551,
or https://www.mercurial-scm.org/repo/hg/rev/bdc2bf68f19e.
He suggested it and it's a useful feature. I did minor compatibility changes
(encoding, avoid rev numbers, foo_bar -> foobar).
Original commit message:
I use 3-way conflict markers. Often when I resolve them, I manually
compare one the base with one side and apply the differences to the
other side. That can be hard when the conflict marker is large. This
patch introduces a new type of conflict marker, which I'm hoping will
make it easier to resolve conflicts.
The new format uses `<<<<<<<` and `>>>>>>>` to open and close the
markers, just like our existing 2-way and 3-way conflict
markers. Instead of having 2 or 3 snapshots (left+right or
left+base+right), it has a sequence of diffs. A diff looks like this:
```
------- base
+++++++ left
a
-b
+c
d
```
A diff that adds one side ("diff from nothing") has a `=======` header
instead and does not have have `+` prefixed on its lines. A regular
3-way merge can be viewed as adding one side plus a diff between the
base and the other side. It thus has two ways of being represented,
depending on which side is being diffed:
```
<<<<<<<
======= left
contents
on
left
------- base
+++++++ right
contents
on
-left
+right
>>>>>>>
```
or
```
<<<<<<<
------- base
+++++++ left
contents
on
-right
+left
======= right
contents
on
right
>>>>>>>
```
I've made it so the new merge tool tries to pick a version that has
the most common lines (no difference in the example above).
I've called the new tool "mergediff" to stick to the convention of
starting with "merge" if the tool tries a regular 3-way merge.
The idea came from my pet VCS (placeholder name `jj`), which has
support for octopus merges and other ways of ending up with merges of
more than 3 versions. I wanted to be able to represent such conflicts
in the working copy and therefore thought of this format (although I
have not yet implemented it in my VCS). I then attended a meeting with
Larry McVoy, who said BitKeeper has an option (`bk smerge -g`) for
showing a similar format, which reminded me to actually attempt this
in Mercurial.
Reviewed By: DurhamG
Differential Revision: D26947920
fbshipit-source-id: 8b4446862897ff9a6dfdf5a2e35617d4db09e883
Summary:
Add a check to ensure the `into_vec()` fast path do not take the full vec if
the `Bytes` is sliced.
Reviewed By: andll
Differential Revision: D26966453
fbshipit-source-id: 538dfb8ca2f01a46d1ede7b98b7f0a30fc7a786e
Summary:
VS Code is seeing issues where they are accidentally triggering
multiple hg status calls, which trigger multiple expensive watchman queries.
While they're trying to track down why this is happening, they'd like a config
that would enable hg to only run one of their status calls at a time.
Reviewed By: quark-zju
Differential Revision: D26931193
fbshipit-source-id: 3b36ac06217bb506110b8d708d4a74378245d5bb
Summary:
RepoError prints `!` at the end of `abort:`. This maintains the old behavior
and should fix test breakages like:
$ hg clone 'ssh://fakehost|touch%20owned/path'
destination directory: path
- abort: no suitable response from remote hg!
+ abort: no suitable response from remote hg
[255]
Reviewed By: DurhamG
Differential Revision: D26964146
fbshipit-source-id: 2e6d095b82ce1e2e23a353cf8f06ae844ee872d7
Summary:
There are a report of slow smartlog that tries to show 8M nodes in `draft()`.
That is an issue after the automatic bookmark clean up, and the visible heads
incorrectly make large amount of commits draft.
Reviewed By: kulshrax
Differential Revision: D26934644
fbshipit-source-id: 0c0890f8eaf1422dab9c03159a419800ae7247ca
Summary:
We're seeing issues where watchman is frequently having to recrawl the
repository due to fsevents getting dropped. We believe this is caused by large
amounts of IO, but we're not sure from what process (buck? hg? arc? ide?). Let's
add some logging to Mercurial to estimate when a checkout triggers a recrawl.
Reviewed By: xavierd
Differential Revision: D26931996
fbshipit-source-id: 0026c792f0ec216cb3e98424da819c4c6e925072
Summary:
Visible heads can overlap with scratch remote bookmarks, and in bad cases
(before D26792731 (7a759b6075)), overlap with public remotenames.
When we remove remotenames we need to remove referred nodes in visibleheads
too so we don't end up with massive draft commits.
Reviewed By: DurhamG
Differential Revision: D26954215
fbshipit-source-id: 91010e6608d0150ecf374ce31705e97712154b27
Summary:
Scratch remote bookmarks are considered draft() and were not cleaned up.
Practically there are users with 100+ scratch remote bookmarks. Let's
clean them up too.
Note the commit still remain visible, which will be fixed in upcoming
diffs.
Reviewed By: DurhamG
Differential Revision: D26954216
fbshipit-source-id: e84f99e4e914f0c5169583fc9f60084f23664e02
Summary:
Current `block_on_future` implementation is not safe, as it does not panic on nested calls. Nesting `block_on_future` calls causes starvation of tokio scheduler processes and eventually will lead to deadlock if nested too many times
This diff simply calls `block_on_exclusive` from `block_on_future`. When we see that this does not cause problems we can simply remove `block_on_future`
Reviewed By: sfilipco
Differential Revision: D26899522
fbshipit-source-id: 93794bf2c5908421691dfb094d1807266c9ecd8d
Summary:
Johan Schuijt-Li found they are broken on some CI hosts.
(But other store paths seem okay, which might indicate the fsync list is effective)
Reviewed By: DurhamG
Differential Revision: D26950101
fbshipit-source-id: e3bcd3f77636325be9e9ce8dfded8b17ec68f436
Summary: This is intended to be used in Sandcastle / OnDemand use-cases to provide a cleaner state.
Reviewed By: DurhamG
Differential Revision: D26924711
fbshipit-source-id: a37d24a10c99d953e2af842f729ef634cbb2e2f9
Summary:
Some merge-tools, like kdiff3, are configured to fix end of line stuff.
This code breaks in Python 3.
Reviewed By: quark-zju
Differential Revision: D26932815
fbshipit-source-id: 7601d53616e50961b89bbc4b0340a9fb672293ef
Summary: This API will be used to spawn tasks in the runtime.
Reviewed By: andll
Differential Revision: D26615607
fbshipit-source-id: 6b9892fad755bbe8feb775e9dad457697b2ea1b7
Summary: hgsql can only use traditional revlog. Disable changelog migration for it.
Reviewed By: kulshrax
Differential Revision: D26891252
fbshipit-source-id: 36c5a448d4fcad15b3415e4534448a945f6d0b4b
Summary: We weren't passing a repo path when initially loading the repo's config in `clidispatch`. This meant that the resulting `ConfigSet` didn't contain values from the shared-repo `.hgrc.dynamic`. Evidently, some other code path in Python would eventually add these values, but this meant that pure-Rust commands could not see config values set via dynamicconfig. Passing the path fixes the problem.
Reviewed By: DurhamG
Differential Revision: D26508980
fbshipit-source-id: 65f187d18098a08c81325e78cb02a8ed854c739a
Summary:
See the previous diff for motivation. This removes bookmarks that are
ancestor of master, too. This is important in practice.
Reviewed By: DurhamG
Differential Revision: D26889412
fbshipit-source-id: 255722ed5b486e88ef56e7e378fae3f1113d5fbe
Summary:
The auto cleanup was conservative. It keeps `::draft()`. But that means
ancestors of public commits are not cleaned up. Not all release branches
branch off the master branch.
Reviewed By: DurhamG
Differential Revision: D26889413
fbshipit-source-id: c6a8e3f32cf1f7d2ffe74b7ecd183f4e583949bb
Summary:
This allows for errors raised in these cases to be retried. Most notable is
the timeout error.
Reviewed By: johansglock
Differential Revision: D26855441
fbshipit-source-id: 6137ed1755072d43dbdd25fa092ddb21c8669aa3
Summary:
No timeout is set up by default so the process wait forever when reading bytes
in cases where the connection is lost somehow.
Reviewed By: johansglock
Differential Revision: D26855443
fbshipit-source-id: d741f73e7186fe862f3d78a806f3219c2cbe7e0a
Summary:
Abort one of the most general exceptions in Mercurial. In theory it should be
something that isn't handled. We can say at least that it shouldn't be retried.
For thing that may be transient it is better to use a different type of
exception. NetworkError is something is checked and retries in a few places so
that seems like a natural candidate.
Reviewed By: johansglock
Differential Revision: D26855444
fbshipit-source-id: f15c723293a416b5f44a6592927e3500f3b0b7d5
Summary: Timeouts are another class of errors that are relevant.
Reviewed By: johansglock
Differential Revision: D26855442
fbshipit-source-id: 8ebb83714fa3d7a2f4efcbed8bd512c98301b49d
Summary: The goal is to reduce load on tokio scheduler by using threads & channels instead of spawning new task every time
Reviewed By: DurhamG
Differential Revision: D26801249
fbshipit-source-id: a8d9accc721c7ffc981fd538c06ab8cbd908f715
Summary:
AsyncVfs provides async vfs interface.
It will be used in the native checkout instead of current use case that spawns blocking tokio tasks for VFS action
Reviewed By: quark-zju
Differential Revision: D26801250
fbshipit-source-id: bb26c4fc8acac82f4b55bb3f2f3964a6d0b64014
Summary: This function is now internal details of vfs, and is not used outside of it. Making it private so it won't be accessed outside
Reviewed By: quark-zju
Differential Revision: D26801251
fbshipit-source-id: 03434e235978fa0745128d90ea0c5975ea662ff1
Summary: This diff removes the split between manually managed and autocargo managed Cargo.toml files in `eden/scm/lib`, now all files are autogenerated.
Reviewed By: quark-zju
Differential Revision: D26830884
fbshipit-source-id: 3a5d8409a61347c7650cc7d8192fa426c03733dc
Summary:
The GitTreeDict throws when we're trying to remove the root directory but there
are valid situations where the Tree becomes briefly empty. One of such is a
commit where all the files in the repo are moved. We process the deletions
first. Then we proceed with additions.
Reviewed By: DurhamG
Differential Revision: D26778717
fbshipit-source-id: 6caa1709ff2c5e78d4745336b0527af4bb20ec60
Summary:
Native checkout avoids the initial large prefetch, and instead does
small batch prefetches and immediately writes them to disk. This will prevent
the issue where data in the cache is evicted before it's used and has to be
redownloaded in a non-batch fashion later.
In a future diff, using native checkout will also allow us to serialize and
deserialize it's CheckoutPlan, so we can make clones resumable.
Reviewed By: quark-zju
Differential Revision: D26825773
fbshipit-source-id: 6fde90cd7578f6af5fdd4ac78004f7f63af4f287
Summary: the list of configured default selective pull bookmarks are actually related to the hoist namespace I think when we consider what is protected and what is not.
Reviewed By: quark-zju
Differential Revision: D26814708
fbshipit-source-id: 566f363f1e6b88dd2ebc6e33cfb6eba596493988
Summary:
On POSIX systems it's a good practice to fsync directories to ensure metadata
about a file is properly written.
Reviewed By: DurhamG
Differential Revision: D26822211
fbshipit-source-id: fca10c702b480f22020ad184eb55c8879999f0ee
Summary:
Typical mistake of `reposetup` - the `repo` object might be remote and lack of
local repo methods.
Reviewed By: DurhamG
Differential Revision: D26822960
fbshipit-source-id: 54fb95c51b09ef5021b0a10a93b4509e7aa4a115
Summary:
I moved the source of the sphinx project to newdoc for newdoc/dev.
I updated the sphinx config for markdown to something that works for recent
versions.
PNG images rendered better than SVG for me.
I moved the TARGETS to newdoc.
Reviewed By: quark-zju
Differential Revision: D26801426
fbshipit-source-id: 3ae51e886d27f848f0f6a48c96056da607d8da45
Summary:
Mercurial uses Makefile here to build stuff. We want to track Makefile under
scm.
Reviewed By: quark-zju
Differential Revision: D26802165
fbshipit-source-id: 1fe8db13d50c07a6a0681180959eba22eaf8d486
Summary:
The previous fix D23357655 (d60e80796a) actually only fixes py2 absorb -i. On Python 3,
`b"-"[0]` is `45`, not `b"-"` like Python 2. Fix it again using `b"-"[0:1]`.
Reviewed By: singhsrb
Differential Revision: D26805315
fbshipit-source-id: 07ca850373a6bc49b561466ead478024631ce051
Summary:
Memcache is dogslow to initialize, taking >30s on debug build. As a
consequence, this slows down every single test by that amount time, with the
guarantee that no blobs will be found in memcache, ie: a total waste of time.
On release builds, Memcache is significantly faster to initialize, so let's
only disable initializing Memcache for debug builds only.
Reviewed By: fanzeyi
Differential Revision: D26800265
fbshipit-source-id: 8b393c603414de68268fdadb385de177e214a328
Summary:
Adds a new hg py3 windows package to hgbuild for publishing. Currently
the tests don't run. I'll do that next.
Reviewed By: quark-zju
Differential Revision: D26768336
fbshipit-source-id: bd4533abbbc1e1c861aa9995b7a3868a7f6a1a22
Summary: Trigger the cleanup logic automatically if there are too many remote bookmarks.
Reviewed By: sfilipco
Differential Revision: D26802251
fbshipit-source-id: 1ab40c7be98c507865ea17001dc9775b6edf4446
Summary: This is handy to make the `sl` output cleaner.
Reviewed By: sfilipco
Differential Revision: D26802250
fbshipit-source-id: 1b74f3d9a42ab6b0780f07bec59b03a5cd0ea6a9
Summary:
Previously remotenames pointing to unknown commits are just ignored.
If key remotename like master is ignored, it can cause very slow operations
in pull, etc. Let's just raise an exception in this case.
Reviewed By: DurhamG
Differential Revision: D26800236
fbshipit-source-id: 13be4af5499da1b9098b4ff1a6ef41c54092824a
Summary:
Remove public heads when using Rust changelog backends. This should address
some issues seen in commit cloud sync.
This is done at the metalog commit time so we get the latest "remotenames" data for
accurate "public()" set calculation.
Reviewed By: singhsrb
Differential Revision: D26792731
fbshipit-source-id: 00b894fee9804740d664dad0ac47be564820da33
Summary:
The output encoding is used to render the graph log edges. With D26612487 (62ba7447f6), we
switched to Rust IO. The Rust IO requires UTF-8 data. So let's set
outputencoding to UTF-8.
Reviewed By: sfilipco
Differential Revision: D26799551
fbshipit-source-id: aa3e6420067d7c75bef47448e12e48f4cef56a84
Summary:
We have configs that affect peer connections, like help.tlsauthhelp,
that are considered "repo-specific" configs now that they come from
dynamicconfig. Unfortunately repo-specific configs are removed from the ui when
copying it for use in a new remotepeer.
Let's add a few config sections to the allow list for what can go in a remote
peer ui. I have a task for making even repo-less commands load the standard
config, so in the future we can have these new peer objects use the standard
repo-less config, which will remove much of the need for maintaining this
allow-list.
Reviewed By: singhsrb
Differential Revision: D26784364
fbshipit-source-id: 30d9292e48b0f27ce7f4d4904ff6d5ff8dcaf069
Summary:
Provide a method that does pure serialization from (name, node) pairs to the
remotenames blob. This makes it reusable outside `vfs` or `repo` context.
Reviewed By: DurhamG
Differential Revision: D26707454
fbshipit-source-id: c45662922d337e31d17070e5f5828d47e23773b1
Summary:
sometimes in visible heads the hashes of the remote bookmarks are not the latest, so the filtration like it was initially implemented didn't work
if visible heads are polluted, it makes commit cloud operations slow but at least this correct check won't allow public heads to enter workspaces
moreover, after roll out it will fix already existing workspaces with public heads automatically by removing them from workspaces
also, this will allow our magic script to fix remote bookmarks in workspaces to work properly (debugfixcommitcloud), unfortunately now it can break a workspace if we remove bookmarks but not heads
cc quark-zju - could we fix the initial issue how the public heads can enter visible heads in some cases in pull logic? I have a repro if you are interested in.
Reviewed By: singhsrb
Differential Revision: D26778632
fbshipit-source-id: 05dbd4cd415911283ea66ae17772b8d3e458bbd7
Summary: EdenFS doesn't need the history, therefore let's not spend time prefetching it.
Reviewed By: fanzeyi
Differential Revision: D26767634
fbshipit-source-id: 7113f4ce79fdef5455a2bb238ab9d51b7339d8b6
Summary:
Parsing the remotenames (blob) into a list of name -> node pairs.
This makes it reusable outside `vfs` or `repo` context.
Reviewed By: DurhamG
Differential Revision: D26707457
fbshipit-source-id: e6c8bd9ff14d0fea9209c25b89fe733675da747e
Summary:
Use the new memctx.mirror and memctx.__setitem__ APIs. This simplifies the
code.
Reviewed By: DurhamG
Differential Revision: D26726474
fbshipit-source-id: 044616137b883ca250e6d84c0ecfcc70458ec07a
Summary:
Use the Rust tree matcher to rule out files that do not need dirsync quickly.
This should make codemod faster to commit.
I created 5000 files outside dirsync config in fbsource (with 494 lines of
dirsync config), and `hg add` them:
$ mkdir unused
$ cd unused
$ for f in `seq 5000`; do touch $f; done
$ hg add .
Baseline "status":
In [3]: %time repo.status();1
CPU times: user 111 ms, sys: 10.2 ms, total: 122 ms
Wall time: 148 ms
Before, dirsync overhead is ~8x "status":
In [1]: %time x.dirsync.dirsyncctx(repo[None])
CPU times: user 1.37 s, sys: 28.8 ms, total: 1.4 s
Wall time: 1.79 s
Out[1]: (<workingctx f23d7c84c5a7+>, set())
In [2]: %time x.dirsync.dirsyncctx(repo[None])
CPU times: user 1.07 s, sys: 8.41 ms, total: 1.08 s
Wall time: 1.11 s
Out[2]: (<workingctx f23d7c84c5a7+>, set())
After, dirsync overhead is ~1/2 of "status":
In [1]: %time x.dirsync.dirsyncctx(repo[None])
CPU times: user 203 ms, sys: 18.9 ms, total: 222 ms
Wall time: 245 ms
Out[1]: (<workingctx 8ff14e46c9d8+>, set())
In [2]: %time x.dirsync.dirsyncctx(repo[None])
CPU times: user 154 ms, sys: 24.1 ms, total: 178 ms
Wall time: 202 ms
Out[2]: (<workingctx 8ff14e46c9d8+>, set())
Reviewed By: DurhamG
Differential Revision: D26726476
fbshipit-source-id: e34218769c779c9a4ee64c654c75298b7c79f213
Summary: Now dirsync works with IMM rebase. Add a test for it.
Reviewed By: DurhamG
Differential Revision: D26726478
fbshipit-source-id: 6712538d7e903ddb0e3c3df44f7dde638276e99d
Summary: Now dirsync works with absorb. Add a test for it.
Reviewed By: DurhamG
Differential Revision: D26726477
fbshipit-source-id: 4505ad6c1e1fd03bfb2cf12b46bd07c98f2bcc2b
Summary:
Previously, dirsync wraps `repo.commit`, requires an on-disk working copy,
and dirstate to work properly. This diff updates dirsync to wrap
`repo.commitctx` instead, do commit edits purely in memory, then sync the
commit back to disk. It makes dirsync compatible with absorb and in-memory
rebase (and potentially other things like drawdag, if drawdag's context APIs
are improved).
To sync the changes made in-memory back to the filesystem, a dirstate callback
is added to write back mirrored files from commit to disk. This works for both
amend and absorb so the special wrapper about amend is dropped. It is
also optimal for absorb, because it only writes the mirrored files once for
the "final" commit, instead of writing the files for each commit in the stack.
Some `O(N^2)`s (N: len(status)) complexities were avoided:
- `applytomirrors` was called N times.
- `allchanges = set(status.modified + status.removed + status.added)` in
`applytomirrors` was O(N).
- `sourcepath in status.removed` in `applytomirrors` was O(N).
- `mirrorpath in status.removed` in `applytomirrors` was O(N).
Note there is still a suboptimal complexity of `getmirrors` called per changed
path, `O(N*M)` (N: len(status), M: len(dirsync_conig)). That will be addressed
in a later diff.
Reviewed By: DurhamG
Differential Revision: D26726479
fbshipit-source-id: 482c6c830ab65cc0d9cd569a51ec610a1dac49cc
Summary:
## Why this diff
we want hostname prefix to support targeting configs at clients in corp ("corp" means laptop, labs, and other machines that are not in "prod" datacenters), like FRL machines, that don't support our existing tier mechanism.
## Changes
* Extract hostname prefix in `dynamicconfig.rs` and add a getter function `hostname_prefix()` for it.
*A hostname prefix only consists of alphabetical letters and dashes, which is followed by one or more digits in the hostname. If no valid match, the prefix is set to the empty string.*
* Use `gen.hostname_prefix()` in the `evaluate()` fn inside `mod.rs` to check the generator's prefix against a list of given prefixes.
* Copy changes from `configerator/source/scm/hg/hgclientconf/hgclient.thrift` to `fbsource/fbcode/configerator/structs/scm/hg/hgclientconf/hgclient.thrift`.
* Rebuild in `eden/scm/`.
Reviewed By: DurhamG
Differential Revision: D26706686
fbshipit-source-id: 725506a1c1f0983e981b0b3f3993c7c14510b1db
Summary:
dynamicconfig has logic to avoid loading remote configs during tests,
by checking the TESTTMP variable. Unfortunately, the tests currently run system
hg with that environment set, which causes it to generate incorrect
configuration for the real repository (i.e. fbsource).
Let's get rid of all test specific environment variables when running the system
hg. There's a comment in the code saying this breaks some tests (from 2017
03a31be593db), but as far as I can see they still work.
Reviewed By: quark-zju
Differential Revision: D26758364
fbshipit-source-id: 945cbd36f7eccc98149772a42421cbc51b157c5b
Summary:
The new test case failed on Mac because it didn't want to copy over a
symlink or something. Didn't dig too deep because 'rm -rf' works around it.
Reviewed By: sfilipco
Differential Revision: D26758985
fbshipit-source-id: 22ad7e23813bd6bfe86b91535d46c21ee59dfcd1
Summary:
`nonfilelogfctx` means the `fctx` does not have a backing filelog, is in-memory.
Previously, one of the `fctx` is assumed to be backed by a concrete filelog,
it will crash if that is no satisfied.
Reviewed By: DurhamG
Differential Revision: D26731810
fbshipit-source-id: f758f43dc140dbf6cc9b6234681c4830482e767a
Summary:
In a corner case, `mctx[f]` is `None`, and `f` does not exist in both parents.
The `f` should not appear in te `added` status list.
Fix it and remove a duplicated call to `managing`.
Reviewed By: DurhamG
Differential Revision: D26726482
fbshipit-source-id: f386ad9eb27bf053fb671e0f187aaada4699e84e
Summary:
This is used by next diffs.
Test code:
In [3]: v=repo[None]['.gitignore']
In [4]: v.cmp(v)
Before:
RuntimeError: maximum recursion depth exceeded
After:
Out[4]: False
Reviewed By: DurhamG
Differential Revision: D26726481
fbshipit-source-id: c27a8f414313f262d0f596b058df580f9216e8fc
Summary:
Not all context objects support the `__setitem__` API. Instead of adding supports
for all of them, let's add an API to create a `memctx` that mirrors the changes
from another `ctx` so one can them do mutations on the `memctx`. This will be
used by dirsync.
This might also be useful in other "amend" cases, because the `mirror` API takes
various kinds of context (workingctx, changectx, memctx), and is straihtforward
to use:
In [2]: v=m.context.memctx.mirror(repo['.'])
In [3]: v.commit()
Out[3]: 'b\xdeF\xbb\xbb&\xe5\xdbvJ\x16\xf2\xc3\x92\x96Mr\xb7\x85\xbb'
In [4]: repo['.'].node()
Out[4]: 'b\xdeF\xbb\xbb&\xe5\xdbvJ\x16\xf2\xc3\x92\x96Mr\xb7\x85\xbb'
In [5]: v['x']=v['.gitignore']
In [6]: v.commit()
Out[6]: 'b\xdeF\xbb\xbb&\xe5\xdbvJ\x16\xf2\xc3\x92\x96Mr\xb7\x85\xbb'
Reviewed By: DurhamG
Differential Revision: D26726480
fbshipit-source-id: 629cfc721f79d29ef395d4e1d6b3381f0095d573
Summary:
Add `memctx.__setitem__(path, fctx)` API to change a file in memctx. This will
be used in upcoming dirsync changes.
Regarding on `memctx` vs `overlayworkingctx`. Both need some changes to fit the
dirsync use-case (overlayworkingctx needs "amend", not "commit on top of",
memctx needs to be mutable).
The main difference is `overlayworkingctx` uses a concrete `cache` dict for
path, data, and metadata. `memctx` uses a concrete list of file paths, but file
data, metadata is abstracted - it uses a function that returns fctx. I would
like to preserve the laziness of reading files and metadata (especially for LFS),
so I picked the `memctx` instead of `overlayworkingctx` as the starting point.
Reviewed By: DurhamG
Differential Revision: D26726475
fbshipit-source-id: 7c16980ffa1c83fe0ff6d70c60b1daae5e266683
Summary: This simplifies the code a bit.
Reviewed By: kulshrax
Differential Revision: D26681779
fbshipit-source-id: 393565790ab711dd09ae6cfa6f9c4b19c930eb93
Summary:
Similar to D26670318, use the EventListeners APIs to implement the progress
callback. Now HandlerExt only has RequestContext related "as_ref" logic.
Reviewed By: kulshrax
Differential Revision: D26681778
fbshipit-source-id: b7f6e07ced43e0ae043e859337c06b69bd5dfc95
Summary: This makes it useful in non-mut callbacks.
Reviewed By: kulshrax
Differential Revision: D26681784
fbshipit-source-id: 97312df8bf3f900a36cbeb27206a2946bb6c702c
Summary: This makes ProgressUpdater Send + Sync so it can be used in the new callback APIs once `mut` gets dropped.
Reviewed By: kulshrax
Differential Revision: D26681781
fbshipit-source-id: 9c622b1d78b4091e3359c28972b6624f0b53734d
Summary:
This removes more mutable fields. Note the new code is more correct because
curl can call the `progress` callback periodically even if no progress is made.
According to https://curl.se/libcurl/c/CURLOPT_PROGRESSFUNCTION.html:
This function gets called by libcurl instead of its internal equivalent with
a frequent interval. While data is being transferred it will be called very
frequently, and during slow periods like when nothing is being transferred it
can slow down to about one call per second.
Reviewed By: kulshrax
Differential Revision: D26681780
fbshipit-source-id: 19aa4bcb4c56623e3f0408b06041b3a894f197e7
Summary:
This makes the `total_progress` field use lock-free interior mutability. The
goal is to eventually drop Rc and RefCell.
Reviewed By: kulshrax
Differential Revision: D26681782
fbshipit-source-id: ec0a6abbb2115c17c674db2255d196aaec847705
Summary: This removes the need for `mut` for this field.
Reviewed By: kulshrax
Differential Revision: D26681783
fbshipit-source-id: 10ed9adfb62081b0e6839abd9534db92d4e056c5
Summary:
The ProgressInner only exposes APIs for total (aggregated) progress. There is
no API to read individual progress. Make it only track the total progress. This
would make it easier to implement interior, lock-free mutability on ProgressInner.
The updater now needs `mut` temporarily, which will be dropped later (D26681784).
A test case is tweaked so progress does not go backwards.
Reviewed By: kulshrax
Differential Revision: D26681777
fbshipit-source-id: 4ad1b9173d5a2c2326e00c030d51f77e9b9458f3
Summary: Test both the HttpClient and Request events.
Reviewed By: kulshrax
Differential Revision: D26670325
fbshipit-source-id: ffbc4268f7de698830411434a769c8b1a4acd863
Summary:
This simplifies the code a bit and makes it look consistent with other event
listeners.
Reviewed By: kulshrax
Differential Revision: D26670318
fbshipit-source-id: f6eda9403bb6eb09a074544e672a45c84f38e2b1
Summary:
Add `RequestContext.event_listeners()` to register callbacks, and trigger the callbacks
when related events happen.
Reviewed By: kulshrax
Differential Revision: D26670323
fbshipit-source-id: 9b92b715444e83957c06b06f1ce696d4de3c0023
Summary:
This simplifies the logic a bit. There is no need for
`HandlerExt::with_payload` that is similar to `Request::body`.
Reviewed By: kulshrax
Differential Revision: D26670326
fbshipit-source-id: 9fe755821062ad6f2a74d6d5ba345620669f0f63
Summary:
This diff rollouts V2 of autocargo in an atomic way so there are quite a few things done here.
Arc lint support:
V1 used to be part of the default fbsource `arc lint` engine, but since V2 calls buck it must live in a separate lint engine. So this diff:
- Adds running `autocargo` as part of `arc lint-rust`
Mergedriver update:
- Mergedriver used in resolving conflicts on commits is now pointing to V2
- It handles files in `public_autocargo/` directories in addition to the ones containig generation preamble
Including regeneration results of running `common/rust/cargo_from_buck/bin/autocargo`. All the differences are accounted for:
- Some sections and attributes are removed as they can be autodiscovered by Cargo (like `lib.path = "src/lib.rs"` or empty [lib] section)
- "readme" attribute is properly defined as relative to Cargo.toml location rather than as hardcoded string
- "unittest = false" on a Buck rule propagates as "test = false; doctest = false" to Cargo
- "rusqlite" is not special-cased anymore, so the "budled" feature will have to be enabled using custom configuration if required by the project (for rust-shed in order to not break windows builds a default feature section was added)
- Files generated from thrift_library rules that do not support "rust" language are removed
- Custom .bzl rules that create rust artifacts (like `rust_python_extension`) are no longer ignored
Others:
- Changed `bin/cargo-autocargo` to be a wrapper for calling V2 via `cargo autocargo`
- Updated following files to use V2:
- `common/rust/tools/reindeer/version-bump`
- `remote_execution/rust/setup.sh`
- Removed few files from V1 that would otherwise interfere with V2 automatic regeneration/linting/testing
Reviewed By: zertosh
Differential Revision: D26728789
fbshipit-source-id: d1454e7ce658a2d3194704f8d77b12d688ec3e64
Summary:
Add a way to define event listeners. Define events on HttpClient and
Request(Info). They will be used by upcoming changes.
The idea is similar to GUI programming. A "control" has a list of "events" that
handlers can be defined on them.
This diff defines lists of events on the "client" and "request" types.
Areas this diff tries to improve:
- Make it easier to add new events (by using macro_rules).
- Make it easier to visually see all possible events.
Reviewed By: kulshrax
Differential Revision: D26670324
fbshipit-source-id: 92f74779f8e546491d0e922db27a4b87f527a5e9
Summary:
This makes it easier to access states defined in `RequestContext` from
`curl::Easy2<H>` types.
Reviewed By: kulshrax
Differential Revision: D26670317
fbshipit-source-id: 27eca9dcc11b14b5d41c8327448f7748ebc62e10
Summary:
Upcoming diffs extend the trait with new methods unrelated to configuration.
Rename to clarify.
Reviewed By: kulshrax
Differential Revision: D26670314
fbshipit-source-id: 7d33ebe22b26f1a286ae40c78f51f31a1a64957e
Summary:
Similar to the Streaming curl handler, add a RequestContext field to the Buffered
handler so the curl callbacks on the Handler can provide the Request
information like urls.
Reviewed By: kulshrax
Differential Revision: D26670321
fbshipit-source-id: de7abecf162c4aaed03d927c35516b6f8ac523ce
Summary:
Make `RequestContext` available in the streaming request. The `clone` will
be removed by a later change.
`dead_code` is temporarily allowed so the following won't be an error.
error: field is never read: `request_info`
--> src/handler/streaming.rs:24:5
|
24 | request_info: RequestContext,
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
Reviewed By: kulshrax
Differential Revision: D26670319
fbshipit-source-id: 53a1deeece5a2059e7caa9d28ef00e083a27b722
Summary:
Add an "auto incremental" Id to uniquely identify requests. This allows external
logic to have a map from the Id to extra metadata not owned by this crate. Without
the Id, there is no way to tell if `RequestContext`s with a same url and method are
actually a same request or not.
Reviewed By: kulshrax
Differential Revision: D26670327
fbshipit-source-id: 60fa760432b23ab5334f22806e01304f9c160182
Summary:
The `RequestContext` is a subset of `Request` that are:
- Independent from curl types.
- Carry useful states, and make them available on Handler callbacks.
- For now, the "useful states" include url and method. They can be extended
later.
Reviewed By: kulshrax
Differential Revision: D26670320
fbshipit-source-id: 3d71d5fee8927dd57a52f51b212397710379e7fa
Summary: This was broken by D26702166 (373d9ade0a).
Reviewed By: sfilipco
Differential Revision: D26736392
fbshipit-source-id: 8bae1599922c05193eba529d7f74c585f20d6f91
Summary: This treats update_meta collection same way as update_content
Reviewed By: DurhamG
Differential Revision: D26694077
fbshipit-source-id: f2a7afb988eb99afa6e8f62ae6cda57a108a987d
Summary: This diff adds unit test for sparse checkout plan
Reviewed By: DurhamG
Differential Revision: D26673701
fbshipit-source-id: 8f810df317c3447f8445765e3a510559d4f1b378
Summary:
This diff integrates CheckoutPlan::with_sparse_profile_change into python code
It also updates test-update-over-sparse-profile-change.t to use native checkout, as this test now passes
Reviewed By: DurhamG
Differential Revision: D26670414
fbshipit-source-id: 73011217b4a0d80ef643b778cdaf75bbada5dcf2
Summary:
This method updates checkout plan to account for sparse profile change
The logic is somewhat similar to sparse.py, except it is smaller because it does not account for merges
Reviewed By: DurhamG
Differential Revision: D26670413
fbshipit-source-id: 54294ad7f1c61e39400450ff139f40ac8fffdd62
Summary:
This matcher combinator is useful when checking out across sparse profile change, since it allows to list files that has been affected by the sparse profile chane
Python has similar combinator xormatcher
Reviewed By: DurhamG
Differential Revision: D26670412
fbshipit-source-id: 0f84835aad16177e79e87d15b41b2caef4374605
Summary:
This diff introduces support for fixed sparse profile into native checkout.
Sparse profile change is not yet supported.
This also updates test-sparse.t to use native checkout
Reviewed By: DurhamG
Differential Revision: D26648490
fbshipit-source-id: 516308dba096189e4657715214ad2a5b61816c49
Summary:
There are some reports about truncated latest files on NFS hosts (where the
hgcache portion might be outside NFS).
Fsync files in hgcache too. With indexedlog the total number of files is
bounded so listing them is considered bounded time.
Reviewed By: kulshrax
Differential Revision: D26741889
fbshipit-source-id: a789396cc5802110eb55fa6b52b5124cd9a3dbf3
Summary: It seems there is no "tty" check. So let's disable colors for tests explicitly.
Reviewed By: DurhamG
Differential Revision: D26708620
fbshipit-source-id: 802537af0d3f580c39a746b6aa6a617c9fb4c9c1
Summary: A few new config options were added to `EdenApiBuilder` without corresponding setting methods, meaning that they could only be set via an hgrc config file. This diff adds the methods so that Rust code can manually configure these settings.
Differential Revision: D26709471
fbshipit-source-id: 6af645961ed50526ccc04990b82ee201211b6ad2
Summary: Add `Content-Length` to `ResponseMeta` struct. Will be used for logging purposes.
Differential Revision: D26709386
fbshipit-source-id: 4346627e3883d00350738cca1e71004f8d407c0d
Summary:
With Rust IO, `hg` is more willing to report "broken pipe" issues:
$ hg export 'desc(2)' | head -7 > ../a.patch
+ abort: The pipe is being closed. (os error 232)
$ hg export tip > out
Fix the test by using a temporary file.
Reviewed By: sfilipco
Differential Revision: D26730868
fbshipit-source-id: 8446528cad72713329b173eaa10ee109e8dfa956
Summary:
Te old _isstdout check is no longer effective with Rust IO. That breaks editors
like vim. Fix it by replacing the _isstdout check with istty check.
Reviewed By: sfilipco
Differential Revision: D26729684
fbshipit-source-id: fc79eda30c3a585ba6d5960e1f72bfc0fc9df254
Summary:
Currently running native checkout shows much higher sys time then regular checkout
One theory that this is due to high scheduler overhead due to large number small tasks spawned with spawn_blocking
This diff batches 100 operations before spawning blocking proc
Reviewed By: quark-zju
Differential Revision: D26620395
fbshipit-source-id: f5be84c3ee14ffc55e4332e714a48f55701d5f8f
Summary:
Recently there are reports about "remote/master" pointing to a non-existed
commit, missing tree data, and general hard-reboot issues. Attempt to reduce
the chance of losing data by adding the critical paths to the fsync list.
Reviewed By: kulshrax
Differential Revision: D26708225
fbshipit-source-id: ae52534fdbf8d9d35498d30c24bdce7f5d9a96a3
Summary: Currently HGPLAIN=1 will disable the -p flag, even though it's analogue, -n, works fine. This should allow either flag to work with or without HGPLAIN
Reviewed By: DurhamG
Differential Revision: D26440729
fbshipit-source-id: f7058a82c32229f5ff9867f299ed72fe167c7104
Summary: This should fix issues where master points to an unknown commit somehow.
Reviewed By: DurhamG
Differential Revision: D26648623
fbshipit-source-id: 63f7a4b834bf19a7849a1c3771921e6b1e5919d3
Summary:
hg status was throwing an error on Py3 OSX because a fallback path
returned bytes instead of string. This fixes that.
Reviewed By: sfilipco, singhsrb
Differential Revision: D26702166
fbshipit-source-id: fa32e5b312377a899b6af16f40bca051f44ed6c3
Summary:
We got a report about windows in vscode being significantly slower than on
macOs/Linux (https://fburl.com/nv9xwc09) to the point that it can't be
explained by "Windows is just slower". As the post suggests we had a suspicion
it might be related to an issue described in T33479254.
After some debugging I can confirm the following:
1) I can reliably (but not 100% of the time) reproduce the issue on my windows
laptop. I just need to modify a file and do "Quick commit" or "Quick amend" from vscode,
and note that command finished quite fast (just a few seconds, I verified it by
running "hg st" and checking that working copy is now clean, and by running "hg
log -r ." and checking that hash has changed), but vscode takes longer to
notice that (i.e. it keeps spinning and showing that amend is still running).
2) By staring at "Process explorer" I noticed that the problem seem to be in
"hg.exe cloud backup" i.e. when "cloud backup" finishes then vscode stops
spinning. So I suspected it to be a problem.
3) As the next step I started to suspend the process in "Process Explorer"
(note that I started it as admin and SYSTEM user, whatever it means). Then I
looked through file handles in Process explorer and was closing them 1 by 1.
Evetually I noticed that closing handles for named pipes like
"\Device\NamedPipe\uv\0000013E91115170-26220" make vscode stop spinning (it was
usually necessary to close it in hg.exe cloud backup process and all child processes as
well).
4) I also looked at handle value, and noticed that it's bigger than 2048 (0x848).
So currently my suspicion is that the problem is that we don't close enought
handles, and this diff bumps it as a temporary workaround and also walk over handles that multiple of 4 only. quark-zju has an
upstream improvement that would make this hack go away
(https://github.com/rust-lang/rust/pull/75551/commits) but it's not landed yet.
So for now let's try to bump the magic number.
Reviewed By: quark-zju
Differential Revision: D26668773
fbshipit-source-id: ed1c203260a52c3e5449b7b06cf4ecbe4dcf6477
Summary:
See D26671795. The hotfix was ineffective because repo config (and
dynamicconfig) is not loaded. This fix ensures newer versions of
hg do not depend on the hotifx.
Reviewed By: DurhamG
Differential Revision: D26679775
fbshipit-source-id: 5e78051cb7d9ac9ac6b327d895049dca2b025068
Summary: When building against Python 3.8 the MSVC compiler complains about how this macro splits inside parentheses. Not sure what changed, but looks easy enough to fix.
Reviewed By: quark-zju
Differential Revision: D26434060
fbshipit-source-id: b46059a6e676a52c99f8923b6739cffe578cc6f7
Summary: Supports Windows by adding mkscratch and scmdaemon and updating it to build python3libs.zip instead of python27.zip. Changed the name to include 'libs' on the end since the zip no longer contains Python itself.
Reviewed By: quark-zju
Differential Revision: D26381051
fbshipit-source-id: 0a2cc40df103525fdc581a4102458e82fda1f670
Summary:
In Python 3 the strings are no longer bytes, so we need to convert them before calling Windows native APIs.
I originally attempted to make all C APIs use the W() Windows APIs. This sorta worked and enabled some unicode support for Windows in Python 2, like hg status, but it meant that utf-8 path encodings were being returned and eventually passed to python functions like os.lstat() which weren't expecting utf8 encoding. I gave up and just left the Python 2 C code as is, and made a copy that uses W() APIs for Python 3. This enables unicode support on Windows where it didn't work before (at least in my testing).
Reviewed By: quark-zju
Differential Revision: D26381053
fbshipit-source-id: 69d4e18ba9fb0f3d17bad58fbcc5d0e6e61d4252
Summary:
Windows tests were failing to execute hg clone ssh://user@dummy/... lines because setconfig ui.ssh=C:/foo/bar was being translated to ui.ssh=c;c:\\foo\\bar by mingw, since it detected /foo/bar as a unix-style path. This seems to be caused by this one line reversing the slashes. I'm not sure why it exists, but deleting it makes the tests pass.
My guess is this has been broken for about a year.
Reviewed By: quark-zju
Differential Revision: D26639206
fbshipit-source-id: d89cae1ea3dd055b90ec6ee8f7cdbee2ae08b228
Summary: The first byte is the flag.
Reviewed By: sfilipco
Differential Revision: D26654640
fbshipit-source-id: 4d66f14c73fe40a154ca7c08f9a9dff3a54ae337
Summary:
Add case expressions to the templating language that allow selection of a value based on
a single determinator. The expression `case(expr, case1, then1, case2, then2, else)`
will expand to `then1` if `expr` matches `case1`, etc. If it matches none of the cases
then it expands to `else`.
This can be used to simplify long `ifeq` chains in templates.
Reviewed By: quark-zju
Differential Revision: D26631539
fbshipit-source-id: 7543e6f7baa5599c96cac75da17db73e03b918f9
Summary: Make the auth Python bindings match the behavior of `httpconnection.readauthforuri` and omit unset fields in the returned dict.
Reviewed By: quark-zju
Differential Revision: D26439292
fbshipit-source-id: 776d8d9bd59726f5ad8d28973e9c2fdc99a2995c
Summary:
Facebook
It looks like we have quite a few errors when people cloning fbsource on their
laptops (see https://fburl.com/odo8tii0 for more details).
"Couln't resolve host name" is one of the errors we see often, so let's retry
that. ssl errors seem to be retryable as well.
Reviewed By: ikostia
Differential Revision: D26635252
fbshipit-source-id: 2412c4f09aab3d5d45923c4b05fc350556bf9af6
Summary:
The progress suspension is implemented in Rust. Since we're writing to the Rust
"error" and "output" streams, they should handle progress suspension directly.
This also removes the "flakiness" (progress disappears for a while when a line
is printed) when progress is shown in streampager:
lhg debugprogress -s 100 --sleep 100 --with-output --pager=on --config pager.interface=fullscreen
Reviewed By: sfilipco
Differential Revision: D26612478
fbshipit-source-id: b75bfe726c61d89495b198670f777440506fcecf
Summary:
See D23966992 (2a2971a4c7) for context. This is a continuation of D23966995 (ab88771161)
which toggles the deserialization part.
Without this change, the default tuple serialization is inefficient
in CBOR and is a disaster in Python:
In [1]: s,f=api.commithashtolocation('fbsource', repo['master'].node(), [bin('375c7ec7442d2bab635f4ae399be6604c5386eb7')])
In [2]: list(s)
Out[2]:
[{'hgid': (55, 92, 126, 199, 68, 45, 43, 171, 99, 95, 74, 227, 153, 190, 102, 4, 197, 56, 110, 183),
'location': {'descendant': (37, 71, 244, 168, 243, 219, 161, 246, 36, 253, 0, 121, 134, 114, 241, 20, 136, 105, 226, 80),
'distance': 10000}}]
With this change, the Python serialization works as expected:
In [2]: list(s)
Out[2]:
[{'hgid': '7\\~\xc7D-+\xabc_J\xe3\x99\xbef\x04\xc58n\xb7',
'location': {'descendant': '%G\xf4\xa8\xf3\xdb\xa1\xf6$\xfd\x00y\x86r\xf1\x14\x88i\xe2P',
'distance': 10000}}]
Places using HgId serialization directly has opt-in explicit serialization
format: D23966986 (0bb45fcbc4) (zstore), D23966991 (f833f03ba2) (metalog), D24009039 (9c5d20904d) (revisionstore) so
they should not be affected.
EdenAPI protocols used in production should be using `WireHgId` types so they
are not affected.
Reviewed By: sfilipco
Differential Revision: D26562685
fbshipit-source-id: 819b794272ee23a135bbed5b61706944ed0acb9f
Summary:
My edenfs log is filled with the following backtrace:
Traceback (most recent call last):
File "/opt/fb/mercurial/edenscm/mercurial/commands/eden.py", line 421, in get_tree
self._fetch_tree_impl(path, manifest_node)
File "/opt/fb/mercurial/edenscm/mercurial/commands/eden.py", line 671, in _fetch_tree_impl
self.repo._prefetchtrees(path, mfnodes, [], [])
File "/opt/fb/mercurial/edenscm/hgext/treemanifest/__init__.py", line 565, in _prefetchtrees
return self._httpprefetchtrees(*args, **kwargs)
File "/opt/fb/mercurial/edenscm/hgext/treemanifest/__init__.py", line 678, in _httpprefetchtrees
self.edenapi.complete_trees(
TypeError: Expected type that converts to PySequence but received set
And indeed, the code passes a set instead of a plain list.
Reviewed By: kmancini
Differential Revision: D26545708
fbshipit-source-id: 4164154d9cc8991b596198d9324474f646cc194c
Summary:
The Rust IO handles progress and streampager stuff. Switch to it so we don't
need to changing the `fout`, `ferr` when handling streampager in Python.
The chgserver logic is updated to just set raw fd 0, 1, 2 to update stdio,
since `fileno` is no longer exposed from Rust.
Manually tested the following commands, both without chg and with chg:
- lhg log -r . (no pager)
- lhg log (with streampager)
- lhg log --config pager.pager=less (with less pager)
- lhg commit (spawns pager)
- lhg debugprogress -s 100 --sleep 100 --with-output --pager=off (progress in stderr)
- lhg debugprogress -s 100 --sleep 100 --with-output --pager=on --config pager.interface=fullscreen (progress in streampager)
- lhg debugprogress -s 100 --sleep 100 --with-output --pager=on --config pager.pager='LESS= less' (progress is disabled with external pager)
Reviewed By: sfilipco
Differential Revision: D26612487
fbshipit-source-id: 8b4e36b614a0c080b93e41474f9a8fc33f890083
Summary:
Previously those fds were obtained via `fp.fileno()`. We're going to replace
the `fp` with something backed by Rust that does not have `fileno()`. Update
chg to replace the raw fds (0, 1, 2) directly. This is enough to affect both
Python and Rust's stdio so there is no need to replace `ui.fout`, `ui.ferr`
again.
Reviewed By: sfilipco
Differential Revision: D26612484
fbshipit-source-id: 5cd89e5955a1dcaad3d3132730354ee67c016bf0
Summary:
Write "\n" to flush the lines. This makes the lines visible instead of being
rewritten by the progress bar.
Reviewed By: sfilipco
Differential Revision: D26612483
fbshipit-source-id: 00cdc4af4dbed479532cc93fe183a19244c8c7c6
Summary:
Add a new API to test if a stream is a tty. This is needed to replace the
Python `fin`, `fout` etc. to Rust objects, because the Python land requires the
`istty` API. It is also useful for properly implement color detection in the
pure Rust land.
Reviewed By: sfilipco
Differential Revision: D26612480
fbshipit-source-id: 5cf79447b1d74e0031a954788db342afd48dc288
Summary:
Move IO to a separate crate so it is easier to be used in other libraries.
Practically, I'm going to add `is_tty()` API, and make some types from
`cpython-ext` implement it. If `cpython-ext` depends on `clidispatch`,
that would be too heavyweight.
Reviewed By: sfilipco
Differential Revision: D26612488
fbshipit-source-id: 00b18dadce3157a357d189aaa7d985efa6b4c80a
Summary:
Rename `pager` to `IO` matching the Rust IO APIs. This allows us to use the
Rust IO APIs from Python.
Reviewed By: sfilipco
Differential Revision: D26612485
fbshipit-source-id: d78f1c8966429e563fe7d190bbcb3bad1b0710fd
Summary:
I'm going to expose more APIs from the Rust IO such as setting progress
content, etc. Rename the module to clarify.
Reviewed By: sfilipco
Differential Revision: D26612490
fbshipit-source-id: 136ea27a733b09557d02077e68ce51184125ade1
Summary:
Unlike stderr or stdout, writing to progress clears the previous progress text.
Update the function name to clarify.
Reviewed By: sfilipco
Differential Revision: D26612481
fbshipit-source-id: 382182fdcc713f92117d01f53b8da47d195b06cb
Summary:
Update `write_progress` to write the progress to stderr if streampager is not
used.
Reviewed By: sfilipco
Differential Revision: D26612482
fbshipit-source-id: 3034252765024df2a6bda7347eff35f07f3186cb
Summary:
streampager uses "\f" to indicate "the end of progress". Move that logic from
Python to Rust. This hides the streampager implementation detail and allows
us to implement `write_progress` for non-streampager cases.
Reviewed By: sfilipco
Differential Revision: D26612489
fbshipit-source-id: 7f41c3af78a0c098de510f86c86726c43931cb6d
Summary:
There seems no need for the progress bar text to be binary (it is not used to
redirect to binary files). Let's change the type to make future changes easier.
Reviewed By: sfilipco
Differential Revision: D26612479
fbshipit-source-id: 1da7950d5ab2b590963f1591e2f1dd5ae8e0116c
Summary:
This makes existing ui.debug appear in tracing debug output with correct
callsite information. They can then be filtered by the `EDENSCM_LOG`. For
example:
$ EDENSCM_LOG=edenscm=debug lhg dbsh
In [1]: ui.debug('foo')
Feb 19 20:03:31.795 DEBUG edenscm.hgext.debugshell: foo
Reviewed By: sfilipco
Differential Revision: D26582807
fbshipit-source-id: b3d97d43d8a31c7d42e3b5c86a05142206e2c9f7
Summary:
We shouldn't delete from a dictionary while iterating over it, instead we should iterate over a copy and then delete from the original.
`.keys()` returns a view of the dict, while wrapping it in `list` makes a deep copy.
Reviewed By: StanislavGlebik
Differential Revision: D26618782
fbshipit-source-id: 234503aab0a0a2151ae0818282651b547f414016
Summary:
The future has a "map" clause that takes GIL too. If they run in different
threads it could deadlock. For example, the following code might hang
forever:
In [1]: s,f=api.commitdata('fbsource',list(repo.nodes('master')))
In [2]: f.wait()
Reviewed By: sfilipco
Differential Revision: D26582806
fbshipit-source-id: e3259850c68b8d48a7a69ed9d47ef75f26179382
Summary:
Previously, one has to set both EDENSCM_LOG=trace and EDENSCM_TRACE_LEVEL=trace
to enable all logging. That is because the filtering is global - one subscriber
(or layer) filtering out a span or event, then the span or event is gone
forever.
For now, let's just set the TracingCollector Subscriber's filter level
(EDENSCM_TRACE_LEVEL) to TRACE so it solely depends on the EnvFilter
(EDENSCM_LOG) if EDENSCM_LOG is set. That's more friendly.
See `impl<L: Layer<S>, S: Subscriber> Subscriber for Layered<L, S>`:
fn enabled(&self, metadata: &Metadata<'_>) -> bool {
if self.layer.enabled(metadata, self.ctx()) {
// if the outer layer enables the callsite metadata, ask the subscriber.
self.inner.enabled(metadata)
} else {
// otherwise, the callsite is disabled by the layer
false
}
}
See also: https://github.com/tokio-rs/tracing/issues/302
Reviewed By: sfilipco
Differential Revision: D26518017
fbshipit-source-id: 9016b940621fc9a883e6ca3f00eaaeccc051ae74
Summary:
Now we use tracing_subscriber for equivalent features (by setting EDENSCM_LOG).
It's no longer necessary to forward to log eco-system.
Reviewed By: sfilipco
Differential Revision: D26518018
fbshipit-source-id: 7cc080faa734ca40eed1ce250df3135ca2626c1f
Summary: Now IO has internal locking, `mut` is no longer needed.
Reviewed By: sfilipco
Differential Revision: D26518025
fbshipit-source-id: d71e213f6fed24fdefb4c730d576f6cf2fb09d82
Summary:
Cloning IO would make it harder to run the clean-up logic in Drop (ex. flush,
stop the pager). Forbid it.
The weak reference output and error streams can still be cloned. They sastify
the main "clone" use-cases.
Reviewed By: sfilipco
Differential Revision: D26538451
fbshipit-source-id: 6cc59214ec5fe1340cb27c7ee1a09658bb83190a
Summary:
Similar to D26518021 (3aba8d89b0), provide a way to obtain the "output" stream
that implements `std::io::Write`.
The "output" stream is then used to replace the `std::io::Write`
implementation on the `IO` itself.
This has 2 benefits:
- Removes the `&mut` methods on `IO`. So all `&mut IO` can be replaced by `&IO` without exceptions.
- Make it possible to drop `Clone` on `IO` (strong ref is not clonable, but weak ref can). So we can control the lifetime of the streams more predicatably.
Reviewed By: sfilipco
Differential Revision: D26538452
fbshipit-source-id: b00c65ae0ef5ef4a609104111706365028f47ef7
Summary:
Use the global IO state instead of keeping a separate IO state.
This makes the Python pager logic able to affect the tracing logger's error
output. For example, the following command will write the tracing logs to
the streampager's stderr channel as expected:
EDENSCM_LOG=trace lhg log -r '.~1000::.' --config pager.pager=internal:streampager
Traditional pager also works:
EDENSCM_LOG=trace lhg log -r '.~1000::.' --config pager.pager=less
Reviewed By: sfilipco
Differential Revision: D26518024
fbshipit-source-id: c7020b7467f9392af8c2a58d064339d05bce35ce
Summary:
This makes the IOError stream used by tracing loggers use weak reference
so the main IO can still be dropped without being affected by potentially
lingering IOError values.
Reviewed By: sfilipco
Differential Revision: D26518022
fbshipit-source-id: df4c847a7a59fff39b7832190a8723c78ffae9ad
Summary:
Set the global IO reference to the IO created by hgmain. This allows
other crates like `pypager` to use the `IO` without cloning or passing
the reference across layers (Rust references do not actually work for
across the Python boundary).
Reviewed By: sfilipco
Differential Revision: D26518027
fbshipit-source-id: 6144b06035d0ef9384cc1a37245c306071e35a9d
Summary:
This makes it easier to figure out the "main" IO, similar to APIs like
`std::io::stdout`, without passing the IO around. It will be used by
the `pypager` module.
The global reference is a weak reference so it won't affect IO drop
behavior.
Reviewed By: sfilipco
Differential Revision: D26518015
fbshipit-source-id: ab75ec31cbb8f156b72c94a6fe5b2de6fe9e0bff
Summary: This will be used by pypager.
Reviewed By: sfilipco
Differential Revision: D26518026
fbshipit-source-id: ac8290501ea572cc5ce2067a52d62f4933cbe87b
Summary:
I encountered the error but it's unclear what happened. Print type_name to
clarify.
Now I get:
not implemented: converting non-stdio Write (alloc::vec::Vec<u8>) from Rust to Python is not implemented
This turns out to be an error in "clidispatch: make IO clonable" - The `Drop`
should be implemented on `Inner`, instead of `IO`. The fix has been folded
into that commit.
Reviewed By: sfilipco
Differential Revision: D26518016
fbshipit-source-id: ddb5f828c59cbffa9767694558113167bc96db4b
Summary: Done some reordering of fields in Cargo.toml, added test and doctest = false, name of the target that generated the Cargo.toml file and sorted the cratemap.
Reviewed By: ahornby
Differential Revision: D26581275
fbshipit-source-id: 4c363369438c72d43d8ccf4799f103ff092457cc
Summary:
Allow color effects to be joined with `+`, e.g. `blue+bold`. Unlike effects
separated by spaces, these effects must all be available in order for the
effect to apply. If any of the effects are not available, then the combined
effect is not valid.
This is useful in fallback chains, where some of the fallbacks are combined
effects, for example: `brightred:red+bold`.
Use these to improve the fallback chains for sparse profiles, so that the
profile names are not shown in black on 8 color terminals.
Reviewed By: quark-zju
Differential Revision: D26580365
fbshipit-source-id: 8e204c901566ac03fb4af066ab1a09142750a9fb
Summary:
Some features in debushell, like `%time` uses `locals()`. They currently cannot
use common variables like `cl` like:
In [1]: %time len(cl)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<timed eval> in <module>
NameError: name 'cl' is not defined
Fix it by updating `locals()`.
In [1]: %time len(cl)
CPU times: user 700 µs, sys: 130 µs, total: 830 µs
Wall time: 1.28 ms
Reviewed By: ikostia
Differential Revision: D26590943
fbshipit-source-id: 2b7fd3274f1ea41f7996edbbdae0af8e103a9c0a
Summary:
The EdenFS import helper's `_fetch_tree_impl` method works by either calling `repo._prefetchtrees` or `repo.prefetchtrees`, both of which are methods from `treemanifest` extension's `treerepository` class. Unfortunately, these methods are lower-level than their names would suggest, and will always perform gettreepack style fetching (see [1]).
If HTTP fetching is enabled, this means that EdenFS will always query EdenAPI's `/complete_trees` endpoint instead of `/trees`, which is needlessly expensive given that EdenFS just wants the exact set of trees specified. As a somewhat hacky workaround, this diff just checks whether HTTP fetching is enabled, and if so, directly calls the appropriate HTTP fetch method.
This solution isn't ideal; it would be better for the import helper to request the trees from Mercurial's `remotetreestore` instead of via methods from `treerepository`. Unfortunately, with the current structure of Mercurial's storage layer, the `remotetreestore` isn't readily accessible because it gets passed into the Rust API's storage hierarchy upon construction.
---
[1]: The various `*prefetchtrees` methods are usually called from Mercurial's `remotetreestore`, which is where the choice of tree fetching strategy is made (i.e., designated nodes vs gettreepack). The `remotetreestore` then calls `*prefetchtrees` for gettreepack-style fetching, or `*getdesignatednodes` for on-demand fetching. As such, a call to `*prefetchtrees` generally implies gettreepack-style fetching.
Reviewed By: quark-zju
Differential Revision: D26560451
fbshipit-source-id: 2eedf50a6e66fac78df77214b777544eb8049714
Summary: This code has been used to transfer an existing repo to remotenames. After D26460435 (84280e36c3) it doesn't work as designed and should be removed as well.
Reviewed By: quark-zju
Differential Revision: D26544739
fbshipit-source-id: 34c08d4b9997c0b1f298ee1ecd0e8af24f4d8a39
Summary:
Support for unsubscribing from a scratch remote bookmarks in `hg hide` command.
This is support if you hide via a revision. Hiding by its name will be another change.
Reviewed By: quark-zju
Differential Revision: D26544305
fbshipit-source-id: d10372513dda88903e2cc031ff16883a001c8e34
Summary:
The `commit` or `amend` command might not have been the first item in the
command list (e.g. there could have been a custom config option). Expand the
heuristics for detecting the `commit` and `amend` cases to account for this.
Reviewed By: quark-zju
Differential Revision: D26545339
fbshipit-source-id: a5b1fc8ccc87989e742fce1fa79273266892ed79
Summary: Dynamic config only gets applied once the local repo is instantiated, but source peer is created long before that. This causes it to have missing configuration when being instantiated. Apply dynamic config much earlier in case of a clone so that srcpeer can use dynamicconf as well.
Reviewed By: DurhamG
Differential Revision: D26543635
fbshipit-source-id: 88c16d345f5116bfaaf57c593eb09145df81b4fb
Summary:
This diff ensures no one can exploit mkscratch's scratch directory. In reality probably no one would do this but just to add it as a precaution.
And also the renaming, I didn't land the rename changes in my previous diff and this diff fix that so we don't roll the "nested" mode name out.
Reviewed By: xavierd
Differential Revision: D26526702
fbshipit-source-id: 04ec883e4d1c0c3e9b54b6274e5113c8b4845b3f
Summary:
This warning was printed by old checkout, keeping it in nativecheckout too
The warning tells user that current working directory was removed during checkout
This diff also makes test-update-names.t to use nativecheckout. Otherwise last test fails on remote repo, because pyworker does not emit the deleted cwd warning
Reviewed By: quark-zju
Differential Revision: D26558188
fbshipit-source-id: 1f6ea2ea1ac7358ce2f06fed25069656481b30e6
Summary: After this update, test-update-names.t tests are using native checkout when congfig is set
Reviewed By: quark-zju
Differential Revision: D26558186
fbshipit-source-id: f70f65344b5f2209f313e3edd5fd7f541318459a
Summary: Those shortcuts can be used to setup remote repo via single line
Reviewed By: quark-zju
Differential Revision: D26558187
fbshipit-source-id: c6fd48ed38cc4dbaad4db714c4dfd76ec26bf608
Summary: This is needed to correctly show status after checkout
Reviewed By: quark-zju
Differential Revision: D26551119
fbshipit-source-id: 3ab576df8132b1fb8cccf507717923ecf3084757
Summary:
This diff makes CheckoutPlan::apply take &self instead of self, so that CheckoutPlan can be preserved.
This result in cloning of some paths(but realistically, they probably get moved anyway in this code path, so this should not have perf impact)
This will simplify pycheckout as it will allow to keep plan and later use it to update dirstate
This diff also exposes accessors for dirstate update (e.g. removed_files / updated_content_files / updated_meta_files)
Reviewed By: quark-zju
Differential Revision: D26551118
fbshipit-source-id: 4e2d02da0a692345eae7d5e7124663c577e9029c
Summary:
Currently native checkout works only on simplest cases when there is no merge / rebase and working copy is clean
Additionally, all native checkout logic is gated with experimental.nativecheckout config
Reviewed By: quark-zju
Differential Revision: D26501903
fbshipit-source-id: 2c1b2fd303f1cb1f461d4854ea60a7274ae38732
Summary:
Make the status fast path print warnings about non-utf8 names and continue on
other valid utf-8 names. It matches the behavior of the Python status code
path.
Before this change, an invalid utf-8 name would just print:
abort: invalid utf-8 sequence of 1 bytes from index 7
without telling which file it is.
Reviewed By: markbt
Differential Revision: D26553514
fbshipit-source-id: 6e26a2a8d738e9001d878a80c0dc82ae6d18a97c
Summary:
Similar to D26142175 (324f47b1f0), initialize the logger that writes to the error stream of
IO.
Reviewed By: sfilipco
Differential Revision: D26518014
fbshipit-source-id: 2547f4a8f3dba1f624fd00798672f1148869e250
Summary:
Add an API to obtain an `io::Write` object for the error stream.
It will be used by next changes.
Reviewed By: sfilipco
Differential Revision: D26518021
fbshipit-source-id: bc39fa54f83be8ff2c8fbc803700c4fe57e814bd
Summary:
Make it possible for IO to be cloned. This allows, for example, tracing or
progress bar logic to keep a reference of IO.
Reviewed By: sfilipco
Differential Revision: D26518028
fbshipit-source-id: d5ca7f4347251c68f7491f784f647ee80c47d159
Summary:
We're going to tweak the internals of IO. Making its fields private to allow
that.
Reviewed By: sfilipco
Differential Revision: D26518023
fbshipit-source-id: a1eed46f4f87a3b4b63e56be733102058b4d72d6
Summary: Add parking_lot dep. It will be used later.
Reviewed By: sfilipco
Differential Revision: D26518019
fbshipit-source-id: 76124a2f70731b30017a6d7019eef9dc856fe54a
Summary:
The changes (and fixes) needed were:
- Ignore rules that are not rust_library or thrift_library (previously only ignore rust_bindgen_library, so that binary and test dependencies were incorrectly added to Cargo.toml)
- Thrift package name to match escaping logic of `tools/build_defs/fbcode_macros/build_defs/lib/thrift/rust.bzl`
- Rearrange some attributes, like features, authors, edition etc.
- Authors to use " instead of '
- Features to be sorted
- Sort all dependencies as one instead of grouping third party and fbcode dependencies together
- Manually format certain entries from third-party/rust/Cargo.toml, since V2 formats third party dependency entries and V1 just takes them as is.
Reviewed By: zertosh
Differential Revision: D26544150
fbshipit-source-id: 19d98985bd6c3ac901ad40cff38ee1ced547e8eb
Summary: Introduce `FetchError` type, with separates "not found" errors (which require a key), from other errors (which may or may not have an associated key). This allows us to easily fall back for only "not found" errors propagate other fetch errors to the caller. The current version of `FetchError` still does not eliminate redundantly storing the key for cases like EdenApi, where the key may be part of the error type itself. This optimization may not be worthwhile after we refactor these store implementations to use `HgId` instead of `Key`.
Reviewed By: kulshrax
Differential Revision: D26410215
fbshipit-source-id: e6198d54de64b41ff696cabd64affc8dbaa41cf9
Summary: Add a wrapper type, `LegacyDatastore`, along with a `ReadStore` implementation that reads via the `HgIdDataStore` trait, for any wrapped types which implement that trait.
Reviewed By: kulshrax
Differential Revision: D26381207
fbshipit-source-id: d28dc3095adf20403f8b993d5a939d1e41c77462
Summary:
Added a new `ReadStore` implementation for EdenApi `FileEntry`s. This is pretty much identical to the `TreeEntry` implementation, but calls the `files` method instead, which does not yet support attributes or per-entry errors (although they are both supported on the wire).
I re-use the type-agnostic indexedlog `Entry` adapter for file support for now, to keep things minimal. This unfortunately means the output type of the `Fallback` combinator is a bare `Entry` for both files and trees, losing us some type safety, but I think this is acceptable for now, until I eventually introduce type-safe constructors and value types. Making the file and tree value output types distinct will only be necessary with the introduction of attribute support, or when we'd like to move away from using an "opaque blob" representation for them.
Reviewed By: kulshrax
Differential Revision: D26348577
fbshipit-source-id: 1a930af6e4b5b4d8bacd266c3e51db96cee92dbd
Summary:
Added write support to the `FallbackStore` combinator, along with a flag to control writing to the write store. This enables the "preferred" store to act as a read-through cache for the "fallback" store, with an optional "read only" mode via the "write" flag.
Currently, errors in the write path are silently ignored.
Reviewed By: kulshrax
Differential Revision: D26233602
fbshipit-source-id: acce020a877b87bfee87763e984923c7c51b2b17
Summary: Introduces a minimal WriteStore trait, with a basic implementation for IndexedLog. Currently, there's no explicit key in the `WriteStream`. Instead, it is implicit that the value type to be written should contain a key. In the future, we may introduce an abstraction over value types which do and do not contain a key.
Reviewed By: kulshrax
Differential Revision: D26213028
fbshipit-source-id: 75867ff7229d594346e7e7624e8ca91f8205fed4
Summary: This helps with pipelining (once chunk is prefetched it will start rolling out to fs) and also helps with limiting number of keys prefetched at once
Reviewed By: quark-zju
Differential Revision: D26496789
fbshipit-source-id: 9aa6b08c1605ab2a06a02347897f5dcfa22297fd
Summary: Also removing todo: test from checkout plan as there are tests already
Reviewed By: singhsrb
Differential Revision: D26496790
fbshipit-source-id: eb99771cb0cda7a90477190875997027b25e2482
Summary:
Move the logic to a common method. A side effect is that
IndexedlogIdDag needs to calculate `last_span.high` again, and
InProcessIdDag needs to calculate `last_store_id` again. But it's
probably okay since the merge is rare and the InProcessIdDag operations should
be considered fast even if it's called frequently.
Reviewed By: sfilipco
Differential Revision: D26504915
fbshipit-source-id: e215ffe9fef7948b7b662dba3ed990e374be989a
Summary: Add a few more tests to the generic IdDag tests.
Reviewed By: sfilipco
Differential Revision: D26415514
fbshipit-source-id: 5ef2a1d4e03527b184a07d94de91c64a05427b90
Summary: It is a constant `true` now. Inline the logic and remove the field.
Reviewed By: sfilipco
Differential Revision: D26414838
fbshipit-source-id: ceace835ecfd263ba16c3dead41ce6ba95087e4f
Summary:
Similar to the previous diff. Implement the merge behavior for InProcess IdDag.
The feature is not turned on yet for easier review. It is turned on by the
next diff.
Reviewed By: sfilipco
Differential Revision: D26414839
fbshipit-source-id: 2a485e0dffcd9d8e2ad95380159d11342636f9aa
Summary:
Merge adjacent flat segments to reduce fragmentation.
The merging is done by a special entry that instructs:
- Remove the old segment from index.
- Insert the new segment.
Note: For the parent -> child index we record parent -> child id, not parent ->
segment. So that index does not need to be updated, because the merged segment
has the same "parent -> child id" information.
Note: The merging is a bit expensive (multiple lookups). But most segments are
not mergeable, because their parents are not "span.low - 1". So the slow path
is not hit often.
The feature is not enabled by default in this diff, because it will break IdDag
tests, which assume indexedlog and in-memory IdDags behave the same, and we
haven't updated in-memory IdDag.
Reviewed By: sfilipco
Differential Revision: D26414841
fbshipit-source-id: 6a268bef56b7a56c62bb8ad949f13e503a88b033
Summary:
With previous refactorings the `max_level` is now maintained by the underlying
IdDag stores. Remove the `max_level` on the wrapper IdDag to simplify the
logic.
Reviewed By: sfilipco
Differential Revision: D26414840
fbshipit-source-id: 084a7f6f75fd53103c45519190607607fef4e161
Summary: We're going to remove the fields. Those add noise to the serialized data.
Reviewed By: sfilipco
Differential Revision: D26504916
fbshipit-source-id: 7fd13bdab4511ceea50195595514fb89a4169419
Summary:
By default it spawns `nproc` threads, which is an overkill on hosts with many cores.
Data shows the max CPU busy % for tokio threads is 30%, avg is 0.1%.
Testing framework will spawn `nproc` tests, meaning it's `nproc ^ 2` threads, which
seems too many.
The number 8 is picked to match the checkout worker count (worker.numcpus), to
ensure checkout tasks won't have bottleneck on the count of tokio threads doing
work like parsing manifest etc.
Reviewed By: sfilipco
Differential Revision: D26437442
fbshipit-source-id: 4dccef83294e64c432ef5ab4be259d41dd890ece
Summary:
Fix for error:
can only concatenate str (not "NoneType") to str
Reviewed By: quark-zju
Differential Revision: D26523879
fbshipit-source-id: 3d15d72be3444df25fb705ce444d6c3621547551
Summary:
Remove selectivepullaccessedbookmarks feature because it contains bugs and causes many undesired issues.
This was added to migrate existing repos to selective pull and is not needed anymore.
Main effects are:
* if you enable selectivepull for an existing repo, it won't reduce number of subscribed bookmarks.
* some operations like `hg push` or `hg pull -r` in their underlying implementation update all subscribed bookmarks, not just accessed like before.
This drives changes to the tests. Reminder, the bookmark has been marked as "accessed" if the repo has been ever updated to that bookmark.
All tests fixed accordingly.
Reviewed By: markbt
Differential Revision: D26460435
fbshipit-source-id: f839b9f207bfc478a0336ec807b720d35a0bb12e
Summary:
This was added in D26387205 (ca1249c269) to make the treemanifest store usage look
more like the file store usage, but it appears to cause read-after-write
failures. I'm not sure exactly why, but I think it's because something takes a
reference to the old store and writes to it, then something else reads from the
new store. Or vice versa.
Either way, reverting these three lines fixes it.
Reviewed By: singhsrb
Differential Revision: D26497708
fbshipit-source-id: e9afabed8d5f800260e647d5ed1513eed0bc60cd
Summary: This diff introduces a Python checkoutplan, which will be used later for Python code to interact with the Rust checkout infrastructure
Reviewed By: DurhamG
Differential Revision: D26437080
fbshipit-source-id: fe3517f4e2f15244a7c2cd290a41394e904db51b
Summary: This diff exposes treemanifest for public use and introduce method to borrow underlyning manifest for usage in rust
Reviewed By: DurhamG
Differential Revision: D26437079
fbshipit-source-id: d2d6cd2a4d1f108881d985932a36d48fed4b238d
Summary: This is most universal/flexible interface
Reviewed By: quark-zju
Differential Revision: D26410641
fbshipit-source-id: cae74675f89c0c1c05c84331c3ce0e78fd0b0b2f
Summary: The latest libc has fixes for compiling on the M1 mac, let's update it.
Reviewed By: dtolnay
Differential Revision: D26476625
fbshipit-source-id: d9a997e69c428d53c51fc52353289a6510314c50
Summary:
Sort works differently on OSX. Let's set the locale in this case to get
consistent sorting across platforms.
Reviewed By: quark-zju
Differential Revision: D26438678
fbshipit-source-id: 80a5c758795fd9675b8b2b658521c0e9d6807366
Summary:
Autocargo V2 will use a more structured format for autocargo field
with the help of `cargo_toml` crate it will be easy to deserialize and handle
it.
Also the "include" field is apparently obsolete as it is used for cargo-publish (see https://doc.rust-lang.org/cargo/reference/manifest.html#the-exclude-and-include-fields). From what I know this might be often wrong, especially if someone tries to publish a package from fbcode, then the private facebook folders might be shipped. Lets just not set it and in the new system one will be able to set it explicitly via autocargo parameter on a rule.
Reviewed By: ahornby
Differential Revision: D26339606
fbshipit-source-id: 510a01a4dd80b3efe58a14553b752009d516d651
Summary:
Make the struct and method names in this crate more clearly reflective of what they do:
- `Auth` -> `AuthGroup`
- `Auth::try_from` -> `AuthGroup::new`
- `AuthConfig` -> `AuthSection`
- `AuthConfig::new` -> `AuthSection::from_config`
- `AuthConfig::auth_for_url` -> `AuthSection::best_match_for`
Reviewed By: singhsrb
Differential Revision: D26436095
fbshipit-source-id: a5ec5d9c48d3b75a0ee166b74e5340f9c529eeae
Summary:
In a non-segmented-chanelog repo, `last`, the revset function has suboptimal
path that makes the following query:
last(::b8f917d4c & public())
take a long time. Use `max` to fix it.
Reviewed By: singhsrb
Differential Revision: D26435439
fbshipit-source-id: dc854b4c6978d2925cab2cb8f217365ed9c7f146
Summary: Make `AuthConfig::auth_for_url` match the behavior of the Python `httpconnection.readauthforuri` function when an auth group has a URL prefix with a scheme. Previously, the Rust version did not allow schemes to be specified on the prefix, and instead required them to be specified in the "schemes" fields. The Python code allows a scheme in the prefix, which overrides the contents of the "schemes" field for that auth group if present.
Reviewed By: DurhamG
Differential Revision: D26419721
fbshipit-source-id: 909b52a2e37e2fc908d5eb56740dd41dda826033
Summary:
Move these conversion related function and trait out of `hg` module so EdenFS can use it too. Changes:
* Moved `get_opt`, `get_or` and `get_or_default` directly into `ConfigSet`.
* Moved `FromConfigValue` and `ByteCount` into `configparser::convert`.
Reviewed By: quark-zju
Differential Revision: D26355403
fbshipit-source-id: 9096b7b737bc4a0cccee1a3883e89a323f864fac
Summary:
The push-pull test was broken by my recent change to make indexedlog
the default. It wasn't caught because the test is disabled in Python 3. It now
passes with Python 3 so let's enable it.
The repack-remove fix is just some flakiness I've seen lately.
Reviewed By: singhsrb
Differential Revision: D26412998
fbshipit-source-id: be4b648f31bd6dfbf6a6e5d2e382acb084461974
Summary: Add Python bindings to the Rust `auth` crate, with the intention of replacing `httpconnection.readauthforuri`.
Reviewed By: quark-zju
Differential Revision: D26419447
fbshipit-source-id: dd13bea74961137790beb8c96120ebef99e3c313
Summary: The Python `readauthforuri` method will include *any* fields specified in the `[auth]` section for a given auth group, even if those fields aren't one of the expect ones (such as `cert`, `key`, etc). This is sometimes used in tests to attach additional information to an auth group. To support this in Rust, let's just collect all unknown fields into a `HashMap`.
Reviewed By: quark-zju
Differential Revision: D26416086
fbshipit-source-id: 0252e340e38850a54e24d54810e9abd77c566f63
Summary:
The auth crate is now able to check the presence and expiration of client certificates (D26009207 (9f7d4447fd)). When a problem is detected, it emits an `X509Error`, which specifies exactly what the problem is. Since this error always indicates a certificate issue, we can print out the message configured in `help.tlsauthhelp` (which is more specific than `help.tlshelp` from the previous diff).
Previously, Mercurial would attempt to use the certificate anyway, resulting in a difficult to understand error message. Although the previous diffs in this stack improved the error messages on any TLS failure, the `X509Error` messages are even more helpful.
Users can opt in to this certificate validation with `edenapi.validate-certs`. The functionality is gated on a config option to prevent Mercurial from crashing if certificates are misconfigured, but EdenAPI isn't being used.
Reviewed By: quark-zju
Differential Revision: D26385843
fbshipit-source-id: 9809f612f8aab3f2dd442d6dd8dc348f1af45296
Summary: Print out the help text configured in `help.tlshelp` upon hitting a `TlsError`. Note that in this case, we use `help.tlshelp` rather than `help.tlsauthhelp` since all we know in this case is that *some* kind of TLS error occurred.
Reviewed By: quark-zju
Differential Revision: D26385844
fbshipit-source-id: 1fb5280195de75107ecdfc9203ef8ddda2a04052
Summary: Add a new `TlsError` Python exception type corresponding to `HttpClientError::Tls`.
Reviewed By: quark-zju
Differential Revision: D26385846
fbshipit-source-id: c0df543032461de650a4d24c26c6b8aaab1abbb9
Summary:
Add a new `HttpClientError::Tls` variant specifically for TLS errors, separating them from other `curl::Error`s from libcurl.
To determine whether a particular `curl::Error` is a TLS error, we check both the error code and the contents of the error message.
Reviewed By: quark-zju
Differential Revision: D26385845
fbshipit-source-id: fd58f86a3a61fcfb845d19e262fdcb132dc7ec9f
Summary: This diff adds auto-generated test cases to checkout code. It generates partially overlapping trees and tests transitions between them
Reviewed By: quark-zju
Differential Revision: D26384962
fbshipit-source-id: 6140bbb7ff8b87843a2235f8325f57829cdd8cae
Summary: Currently PathComponentBuf::arbitrary generates any characters. Those characters are ok for unit tests on hg abstractions where they are currently used, but many of them do not work with real filesystems
Reviewed By: quark-zju
Differential Revision: D26384961
fbshipit-source-id: dde1e9276114b30262bc477a3e0f828645f1f32a
Summary:
Currently if VFS overwrites executable file with regular, it preserves exec bit[see added test].
This diff makes sure that file has correct permissions after overwrite
This diff also slightly optimizes write_executable, by calling set_mode on the file handle, instead of path
TODO - we can check if calling stats() before set_permissions will save some time
Reviewed By: quark-zju
Differential Revision: D26379824
fbshipit-source-id: 42d0b2fb79ed860ac37b2de077388002ade69449
Summary:
Before this diff VFS::write_regular did not handle correctly use case when file already existed as as symlink - it would write into symlink location, instead of replacing symlink with a regular file (see updated test_symlink_overwrite that is failing on old implementation)
This diff adds O_NOFOLLOW option on unix when overwriting the file. When destination is a symlink, attempt to write fails with E_LOOP and triggers clear_conflict that removes symlink and allows retry write to succeed.
This also allows one of test cases in checkout that previously did not work
Reviewed By: quark-zju
Differential Revision: D26378893
fbshipit-source-id: 28bcdaba78db283ac7a25bb232c198d3d8f73e5d
Summary: This diff contains basic test setup for checkout tests - we compare transition between two trees without dirty changes
Reviewed By: quark-zju
Differential Revision: D26359502
fbshipit-source-id: ef670c944200bae1652863c91ada92c6fecce4ac
Summary:
Update the dirsync code to allow mirror and exclude rules to match individual
files rather than just directory prefixes.
This simply appends `/` to all filenames when looking for rule matches. This
allows us to efficiently match individual filenames in addition to directory
prefixes.
Reviewed By: quark-zju
Differential Revision: D26294583
fbshipit-source-id: 83b283f344f6e0bc0fe53b9068e7e0170f53504b
Summary:
One of the primary use cases for hash_to_location is translating user provided
hashes. It is then perfectly valid for the hashes that are provided to not
exist. Where we would previously return an error for the full request if a
hash was invalid, we now omit the hash from the response.
Reviewed By: quark-zju
Differential Revision: D26389472
fbshipit-source-id: c59529d43f44bed7cdb2af0e9babc96160e0c4a7
Summary:
The approach is very similar to what commitrevlogdata does. You could say
that it's cargo culted.
I am not sure how appropriate it is to return CommitLocationToHashResponse
but I think that it's fine for now.
Reviewed By: quark-zju
Differential Revision: D26374219
fbshipit-source-id: 61d851d5a4fc4223c65078ef434a0c67314a90cd
Summary: This is defined else where in Python 3, so let's get rid of it here.
Reviewed By: quark-zju
Differential Revision: D26381054
fbshipit-source-id: 9746d2c53f83209d9c795ffdd5841d58ef1153ef
Summary: These are just a few one-liners to fix Python 3 on Windows.
Reviewed By: sfilipco
Differential Revision: D26381055
fbshipit-source-id: d9257f2cf35c05f931d74b7d26bdc79f5bf34185
Summary:
We've rolled both of these out to 100%. Let's make this the default so
we can delete those configs.
Reviewed By: quark-zju
Differential Revision: D26233645
fbshipit-source-id: cd7a08c404483f78ab714763870f5bf0fa801e7a
Summary:
We're seeing cases where Mercurial is creating unused .tmp pack files
and leaving them around. It looks like there are two places this can happen, 1)
in the treemanifest python code we manually instantiate some mutable packs, and
2) when doing a read from the rust data store, when it does a read against the
mutable pack store it will unnecessarily create the MutableDataPackInner struct,
which creates the temp file.
Let's fix those.
Reviewed By: quark-zju
Differential Revision: D26387205
fbshipit-source-id: 5a567c886849084bcc8121949dd2fb0f9e66d570
Summary:
In our upcoming migration away from chef/static rc files, we'll be
marking certain files as "allowed". Our hope is that that list only includes
things like .hg/hgrc, ~/.hgrc, etc.
There are cases however where it's convienent to continue to use chef, for
instance when we condition on machine type. To support this, let's add an
allowed_config option, which will allow configs from non-supported locations.
This will also be useful when remediating issues that come up when we start
enforcing allow_location, without rolling back the entire thing.
Reviewed By: quark-zju
Differential Revision: D26233451
fbshipit-source-id: 71789e0361923a6f80de4aef7f012afc0269440d
Summary:
Copied from D17312417 (e1f4dbeb3d), because that did what I needed done, but incremented by 1.
I would like to change the length of the displayed hash in scm-prompt
to 9. Why such an impactful diff? Because hg sl shows 9 characters, and I
always get confused when the hash in my prompt doesn't match hg sl
Reviewed By: kulshrax
Differential Revision: D25934253
fbshipit-source-id: 15f2bc8bc7d666de1a077d2bafd74ab3c9753341
Summary:
On macOS, EdenFS can be configured to be case insensitive, and will soon be
switching its default to be case insensitive. Mercurial however considers that
EdenFS is always case sensistive on unix (linux, macOS), so let's change it on
macOS so it manually checks if the working copy is.
Long term, EdenFS on macOS will always be case insensitive, at which point this
code will be changed again to indicate that and avoid the manual checks.
Reviewed By: chadaustin
Differential Revision: D26357816
fbshipit-source-id: 4022c097f2804da69cfcc176840683a6dca12ffd
Summary:
The test wasn't meant to be committed. The feature it wants to test was not
easily testable (requires streamclone and pull to have different master,
which is true on Mononoke but false on local hg servers).
Reviewed By: sfilipco
Differential Revision: D26380544
fbshipit-source-id: 6fa720058df8b88ace704d186caa4213d9cd62c9
Summary:
Previously we include tip in sample. But that is problematic for fast paths
like:
if set(commonsample).issuperset(set(localheads) - {nullid}):
ui.note(_("all local heads known remotely\n"))
return localheads, True, remoteheads
If `localheads` is empty, then the returned "common heads" are empty, causing
downloading the entire repo inefficiently.
Fix the issue by moving `tip` from `sample` to `localheads` so the common
heads will include tip in the fast path.
Reviewed By: DurhamG
Differential Revision: D26374303
fbshipit-source-id: 45a2a44e4db4c4ec2a341522a257d46a62b058d5
Summary:
The backingstore crate depends on mio which depends on ntdll. It's not entirely
clear to me why we need to do this manually and why cargo/cmake doesn't pick it
up automatically, but let's fix it this way for now.
Reviewed By: genevievehelsel
Differential Revision: D26376606
fbshipit-source-id: 26714b3f03aabefafdf48c7fb0f442cad501a058
Summary:
Previously the `IdDag` struct has max_level caching. With the previous diff the
cache was gone. Re-implement it on the indexedlog IdDag to maintain
performance. This has visible performance wins:
Before:
building segments 115.949 ms
ancestors 93.072 ms
children (spans) 495.732 ms
children (1 id) 10.384 ms
common_ancestors (spans) 3.567 s
descendants (small subset) 25.829 ms
gca_one (2 ids) 258.997 ms
gca_one (spans) 3.718 s
gca_all (2 ids) 440.764 ms
gca_all (spans) 3.732 s
heads 322.552 ms
heads_ancestors 67.567 ms
is_ancestor 165.046 ms
parents 304.392 ms
parent_ids 11.765 ms
range (2 ids) 21.466 ms
range (spans) 18.663 ms
roots 471.934 ms
After:
benchmarking dag::iddagstore::indexedlog_store::IndexedLogStore
building segments 103.177 ms
ancestors 68.485 ms
children (spans) 451.383 ms
children (1 id) 9.817 ms
common_ancestors (spans) 3.096 s
descendants (small subset) 24.845 ms
gca_one (2 ids) 201.458 ms
gca_one (spans) 3.185 s
gca_all (2 ids) 357.899 ms
gca_all (spans) 3.239 s
heads 295.462 ms
heads_ancestors 50.991 ms
is_ancestor 146.798 ms
parents 296.667 ms
parent_ids 12.305 ms
range (2 ids) 7.781 ms
range (spans) 17.630 ms
roots 478.574 ms
Reviewed By: sfilipco
Differential Revision: D26360564
fbshipit-source-id: 51f55a5bb4e69321515e02f45545618320c1bce5
Summary:
Previously, algorithms are defined on `IdDag<Store>`, which requires a type
parameter to use. Move them to be defined directly on `Store` so using the
algorithms no longer require a type parameter. This will make it easier
to write code that calls algorithms on different IdDagStores.
Reviewed By: sfilipco
Differential Revision: D26360561
fbshipit-source-id: 8e0faf741019c4ed4119ad8e754aea9057b31866
Summary: It is not used. The definitions were moved to dag-types.
Reviewed By: sfilipco
Differential Revision: D26360562
fbshipit-source-id: 35e672194918e3f35294d69cad9e6990d8921900
Summary:
This allows us to "fork" the iterator so we can "peek ahead" multiple items,
without affecting the original iterator.
Reviewed By: sfilipco
Differential Revision: D26360566
fbshipit-source-id: 4cba280e64338b20fb3e1584609be8fda9b3d616
Summary:
Implement Iterator::{size_hint,count,last} for potential fast paths (ex.
Collect into a Vec).
Reviewed By: sfilipco
Differential Revision: D26360565
fbshipit-source-id: 227d9c5e615c2a0a624ba88d6d4c3f81b10d7795
Summary:
Benchmark code is out of sync. Fix it.
Example run:
benchmarking dag::iddag::IdDag<dag::iddagstore::in_process_store::InProcessStore> serde
serializing inprocess iddag with mincode 0.664 ms
mincode serialized blob has 707565 bytes
deserializing inprocess iddag with mincode 42.477 ms
Reviewed By: sfilipco
Differential Revision: D26360563
fbshipit-source-id: f87e7ad53e6b6dadecaa0976e1c61f0399814104
Summary:
Benchmark code is out of sync. This is an important benchmark.
Example run:
benchmarking dag::iddagstore::indexedlog_store::IndexedLogStore
building segments 111.814 ms
ancestors 67.953 ms
children (spans) 484.954 ms
children (1 id) 12.599 ms
common_ancestors (spans) 3.337 s
descendants (small subset) 27.979 ms
gca_one (2 ids) 216.430 ms
gca_one (spans) 3.297 s
gca_all (2 ids) 371.049 ms
gca_all (spans) 3.348 s
heads 303.232 ms
heads_ancestors 52.821 ms
is_ancestor 149.525 ms
parents 294.633 ms
parent_ids 12.173 ms
range (2 ids) 7.612 ms
range (spans) 16.991 ms
roots 459.869 ms
benchmarking dag::iddagstore::in_process_store::InProcessStore
building segments 68.869 ms
ancestors 6.683 ms
children (spans) 175.711 ms
children (1 id) 2.061 ms
common_ancestors (spans) 408.220 ms
descendants (small subset) 6.990 ms
gca_one (2 ids) 16.983 ms
gca_one (spans) 411.237 ms
gca_all (2 ids) 27.921 ms
gca_all (spans) 415.704 ms
heads 110.486 ms
heads_ancestors 5.228 ms
is_ancestor 11.223 ms
parents 108.636 ms
parent_ids 0.746 ms
range (2 ids) 1.539 ms
range (spans) 5.885 ms
roots 172.910 ms
benchmarking NameDag with many heads
range (master::draft) 55.400 ms
range (recent_draft::drafts) 12.439 ms
Reviewed By: sfilipco
Differential Revision: D26360567
fbshipit-source-id: 6d3244e3f4655634c239f84a7304540860a7d34a
