Summary:
When pushing a treeonly commit to a tree+flat hybrid server, the server
needs to execute the hooks in treeonly mode so we never try to access the flat
manifests. Previously we did this for shell hooks by setting some environment
variables, but we didn't do it for python hooks that ran in process.
This diff makes the python hooks run against a bundle repo that is instantiated
in treeonly mode. No changes to any hooks are required, as the repo object they
are given is already in the correct mode.
Reviewed By: quark-zju
Differential Revision: D6840971
fbshipit-source-id: 9fcb97d972076911b35bccf3f79b60972bcafc33
Summary:
Since we have `debugbuilddag` as a builtin command, and `drawdag` is
pretty useful for writing clean test cases. Let's inline it. This would
simplify many tests since `drawdag=..." is no longer needed.
Reviewed By: DurhamG
Differential Revision: D6827925
fbshipit-source-id: b749f57fd5c55b2cb000dd2da2c2f9bc2386a519
Summary:
crecord locked out the amend functionality to older versions of
Mercurial. Since it's packaged in core now, let's always enable it.
Reviewed By: quark-zju
Differential Revision: D6837724
fbshipit-source-id: 2a0dae2e03346734f7d8298e5a073f776849005e
Summary:
smartlog, among other places, would benefit from being able to
strictly limit the size of text output to the terminal. Provide two functions:
one truncates long lines in a filled block, the other limits the number of
lines of output. Both can optionally add a truncation marker.
Reviewed By: markbt
Differential Revision: D6819153
fbshipit-source-id: d95b654f4a6fef49ad4840f26ef74a18c370bdd8
Summary:
There are a couple of ways that dirstate and fsmonitor interact with
treedirstate that are sub-optimal.
When iterating over all files in the dirstate map, use the iterator of keys
(`__iter__` rather than `iteritems`) to hit the treedirstate fastpath that
doesn't build the dirstate tuples.
Use `__getitem__` and catch the `KeyError` rather than `__contains__` followed
by `__getitem__`, as the latter involves two look-ups in the tree.
Skip the indirect call to `hastrackedfile` and `hasremovedfile` as these add
noticable overhead when called many times.
Also fix up the `test-check-code` failure for treedirstate.
Reviewed By: ryanmce
Differential Revision: D6818952
fbshipit-source-id: d49907c282a1b97c66a3d24257bdf54800d7da70
Summary:
Adds some basic building blocks to build hg using buck.
Header files are cleaned up, so they are relative to the project root.
Some minor changes to C code are made to remove clang build
warnings.
Rust dependencies, fb-hgext C/Python dependencies (ex. cstore,
mysql-connector), and 3rd-party dependencies like python-lz4
are not built yet. But the built hg binary should be able to run
most tests just fine.
Reviewed By: wez
Differential Revision: D6814686
fbshipit-source-id: 59eefd5a3ad86db2ad1c821ed824c9f1878c93e4
Summary:
Suggesting disabled extensions uses extensions.loadpath, which is no
longer supported for core extensions. Since this test is just testing that the
extension can be disabled, let's not worry about the exact output.
Reviewed By: phillco
Differential Revision: D6802719
fbshipit-source-id: 2a602e086f4809676ef8dd36ffc30f021b3186fc
Summary:
check-execute was failing because the filesets are too slow (about 45+s
each). This fixes the performance by optimizing some internal fileset logic, and
by fixing a fileset to be scoped to the current working directory instead of the
whole repo.
Unfortunately the fileset implementation optimization needs to be deployed
before the test can benefit from it (since the test runs the system hg), so we
need to leave the test disabled until the fix rolls out.
Reviewed By: quark-zju
Differential Revision: D6802078
fbshipit-source-id: 26767c989e15b6d0ae5f5d9560309dafa5c380ff
Summary:
Port of D6798134 to fbsource. It eliminates module-import failures as well as errors like this:
```
mercurial.error.ForeignImportError: hgext.extlib.treedirstate: /home/phillco/.local/lib/python2.7/site-packages/hgext/extlib/treedirstate.so lives outside /..../hg
```
....that block other tests, like test-help.t
(Note: this ignores all push blocking failures!)
Reviewed By: quark-zju
Differential Revision: D6799259
fbshipit-source-id: b77d1b565dbf52165e0847002be498648658e064
Summary: Otherwise this test will fill up with default-on tests that will be a pain to filter manually. Plus it's a sensible flag to have.
Differential Revision: D6760553
fbshipit-source-id: 083cb2d8a7cffe28eef812edad2f4c5d997db981
Summary:
`_peerorrepo` calls `extensions.extensions()` to figure out on which modules to run reposetup. This uses a slightly
different code path than that patched by D6716674 and double-checks if an extension is enabled. So we need to patch here too.
Reviewed By: quark-zju
Differential Revision: D6758486
fbshipit-source-id: b5bfe2d11e5e2aeb2d3a0ee7c9d6e3e2c213233d
Summary:
This was blocking us default-enabling `fbsparse`; however, this function is actually incredibly fragile and can break with a variety of extensions enabled (even blackbox).
E.g., try this on the latest release and you'll get an exception!
```
cd $(mktemp -d); hg init a; hg share a b; cd b; hg unshare
```
That particular breakage is called by blackbox trying to look at `repo[None]` to do some logging, but anything that tries to read `repo.dirstate` after this will print the exception. The root cause is this line which is trying to override `repo.root`:
```
repo.unfiltered().__init__(repo.baseui, repo.root)
```
It's trying to update the repo's path to indicate that it is independent and no longer shared. But, the initializer isn't really designed to be called twice, AFAICT, and doing so here leaves the property caches out of sync with properties of the repo (namely `_filecache`).
Durham's suggestion was just to nuke `hg unshare`. This patch works around it for now, though, in case we want to keep it alive.
Reviewed By: quark-zju
Differential Revision: D6758397
fbshipit-source-id: 90d3773d9340f2a5b2e6e900a2194d8b931f410d
Summary:
...instead of '(untested!)'.
Partly this is necessitated by the `hg debugextensions` call in test-extension.t (post
D6758486). But it also seems like the right thing to do.
Reviewed By: ryanmce
Differential Revision: D6758599
fbshipit-source-id: 3ecfa52b0c9c1a028201d2eda0eaa706d80225c0
Summary:
The isgooddelta tweak was introduced in D2693043 (perftweaks: change revlog
delta heuristic, 2015-11-24). Comparing with the existing version, the only
change is that we removed `dist > maxdist` check.
Note that the upstream commit 895ecec31 (revlog: add an experimental option
to mitigated delta issues (issue5480), 2017-06-23) also introduces a config
option to override `maxdist` to make the condition fail, which basically does
a same thing.
Instead of introducing new config options or adding more "if"s to the
codebase to make it more obscure, let's just simplify it by disabling the
check entirely, and removing the `dist` concept, removing two config
options: `experimental.maxdeltachainspan` and `perftweaks.preferdeltas`.
The `chainlen > self._maxchainlen` check should be enough for keeping
delta chain length bounded.
Reviewed By: DurhamG
Differential Revision: D6752529
fbshipit-source-id: e8fd8ec39240191db5fb274190fc661e97087a78
Summary:
This enables clindex for its nodemap. Verification is turned off by default
for the performance win since we have been running verification in
production for a while.
Reviewed By: phillco
Differential Revision: D6751412
fbshipit-source-id: bc3e87df86e86a758392bdd4aef3e282f397fe04
Summary: Also change the internal API so it no longer accepts the "heads" argument.
Reviewed By: ryanmce
Differential Revision: D6745865
fbshipit-source-id: 368742be49b192f7630421003552d0a10eb0b76d
Summary: This removes the effectflag logic from both core and perftweaks.
Reviewed By: ryanmce
Differential Revision: D6745769
fbshipit-source-id: 55ed1676e7117bca358471c256805ded7bc83f3c
Summary:
Previously it's `:. & follow(FILE)` and there is a Python loop iterating
`:.`. This patch makes it `follow(FILE) & :.`.
In the common case, `_logrevs` returns a large revset in desc order. So it
makes sense to put it at the right side of `&` operator. The only exception is
when `_logrevs` uses user-specified revset (`-r`), whose order needs to be
respected and size is unknown.
Test Plan:
Ran all tests.
Facebook:
This reduces `hg log FILE` from ~3 seconds to ~0.5 seconds.
```
# Before
% time hg log cmd_util.py --pager=off --config extensions.fastlog=! -T'{oneline}\n'
11e9d4 D5997870 mbolin Use --home-dir to compute the config dir, if specified. 2017-10-06
5041e2 D4888633 simpkins [eden] move eden/fs/cli to eden/cli 2017-04-14
7dc2bd D4760528 simpkins [eden] an "eden debug" CLI command and thrift APIs to support it 2017-04-03
hg log cmd_util.py --pager=off --config extensions.fastlog=! -T='{oneline}\n' 2.88s user 0.09s system 99% cpu 2.975 total
# After
% time hg log cmd_util.py --pager=off --config extensions.fastlog=! -T'{oneline}\n'
7dc2bd D4760528 simpkins [eden] an "eden debug" CLI command and thrift APIs to support it 2017-04-03
5041e2 D4888633 simpkins [eden] move eden/fs/cli to eden/cli 2017-04-14
11e9d4 D5997870 mbolin Use --home-dir to compute the config dir, if specified. 2017-10-06
hg log cmd_util.py --pager=off --config extensions.fastlog=! -T='{oneline}\n' 0.42s user 0.03s system 98% cpu 0.456 total
```
Fastlog is disabled since a network request makes the time less stable.
Logging a single file is answered locally by remotefilelog ancestor handling.
Reviewers: durham, #mercurial
Reviewed By: durham
Subscribers: durham
Differential Revision: https://phabricator.intern.facebook.com/D6735007
Signature: 6735007:1516163080:0d7ddae9f24933d795ba7aaa4461916c7fdd2309
Summary:
It's pretty handy. The implementation is simple and clean. So move it from
contrib and enable it by default. "import"s are adjusted to make the module
checker happy.
Security and UX wise, since we have `--debugger` already, adding another
REPL seems fine.
Test Plan: Ran all tests
Reviewers: phillco, #mercurial
Reviewed By: phillco
Differential Revision: https://phabricator.intern.facebook.com/D6741283
Signature: 6741283:1516225662:ddc19a663e7ecef2a1fdaa5041f308dc838a8471
Summary:
This if statement is testing whether we ever set any data, not whether the data is truthy.
This can cause a bug where rebasing an empty file can cause IMM to fail and restart.
Test Plan: ./run-tests.py
Reviewers: durham, #mercurial
Reviewed By: durham
Differential Revision: https://phabricator.intern.facebook.com/D6687459
Signature: 6687459:1515535581:c2018237f3ca9daed6a0cb2f5a1475d1dc8e4865
Summary:
Before, we would raise whenever the `usemergedriver` condition was set when merging in-memory,
which equated to "any merge with (cd, dc, or m) actions in a repo with a mergedriver script".
This was done to be as conservative as possible.
However, a better solution is to run the preprocess() script and only raise if any files are
marked to actually be driver-resolved. That way we only restart the merge if we absolutely need
to.
Since some of our preprocess() scripts aren't ready yet, I also added
experimental.inmemory.nomergedriver in a previous change so we can deploy this in a build before the preprocess scripts are good to go.
Test Plan: ./run-tests.py
Reviewers: quark, #sourcecontrol
Reviewed By: quark
Subscribers: durham
Differential Revision: https://phabricator.intern.facebook.com/D6668426
Signature: 6668426:1515185050:a640208454caf053f8213b831d0f8e645ebe682c
If the user specifies a ui.origbackuppath, we used to always copy the file
there, but if the source file is in memory we must write it instead of copying.
Differential Revision: https://phab.mercurial-scm.org/D1806
Summary: Log whichever paths were driver-resolved but not in experimental.inmemorydisallowedpaths, so we can update experimental.inmemorydisallowedpaths and keep the experience of rebasing with IMM and
mergedriver a pleasant one.
Test Plan: .
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D6656159
Summary: Fixing a bug that causes a fatal crash when a user attempts to delete a bookmark by inputting it's name twice, for example:
# hg bookmark --delete name name
Test Plan: The following test has been added in the test/test-bookmarks.t file
https://pxl.cl/bmTf
to test the new functionality:
~~~
kosullivan-mbp:hg-crew kosullivan$ hg diff tests/test-bookmarks.t
diff --git a/tests/test-bookmarks.t b/tests/test-bookmarks.t
--- a/tests/test-bookmarks.t
+++ b/tests/test-bookmarks.t
@@ -288,6 +288,10 @@
abort: bookmark 'A' does not exist
[255]
+ensure bookmark names are deduplicated before deleting
+ $ hg book delete-me
+ $ hg book -d delete-me delete-me
+
bookmark name with spaces should be stripped
$ hg bookmark ' x y '
~~~
Reviewers: rmcelroy,rafeca
Subscribers: suiting
Tags: python,bootcamp,source_control
Tasks: 22525999
Differential Revision: https://phabricator.intern.facebook.com/D6711774
Summary:
Demonstrate the full power of this batte station with the first extension to be default-enabled, `githelp.py`. This also lets us write a test for `DEFAULT_EXTENSIONS`
(`test-extensions-default.t`)
All other test changes are actual fallout from enabling the githelp extension.
Test Plan: Added a test.
Differential Revision: https://phabricator.intern.facebook.com/D6717098
Summary:
Based on in-person discussion in the warroom, we think this is the right path forward to "upstream" certain extensions without actually folding them into the mercurial/ codebase.
Enabling extensions by default achieves most of our goals (testing _everything_ in tests, reducing configuration permutations) without having to pay an up-front cost of fixing all the code today or accepting subpar code in mercurial/.
Test Plan: ./run-tests.py.
Reviewers: quark, #mercurial
Reviewed By: quark
Subscribers: quark
Differential Revision: https://phabricator.intern.facebook.com/D6716674
Signature: 6716674:1515803845:a209ec288341e4de8e79fd3759250e834a3573e3
Summary: Based on feedback to D6687860.
Test Plan: n/a
Reviewers: durham, #mercurial
Reviewed By: durham
Differential Revision: https://phabricator.intern.facebook.com/D6714211
Signature: 6714211:1515788399:386b8f7330f343349234d1f317e5ac0a594142cf
Summary: This avoids issues where a wrong (outside the repo) extension was used.
Test Plan: Ran all tests
Reviewers: durham, #mercurial
Reviewed By: durham
Subscribers: durham
Differential Revision: https://phabricator.intern.facebook.com/D6690534
Signature: 6690534:1515632034:1353c9272650cf925dd299d252592afd52970e98
Summary:
clindex is an hg extension, so moved to `hgext`.
linelog is not an hg extension, but is only used by hg extensions, not
`mercurial/`, so moved to `hgext/extlib`.
Test Plan: `make local` and `run-tests.py` without `-l` and with an empty `PYTHONPATH`.
Reviewers: durham, #mercurial
Reviewed By: durham
Subscribers: fried
Differential Revision: https://phabricator.intern.facebook.com/D6685080
Signature: 6685080:1515525106:88ebb275d0cac041911f243a3e82b82482b6cd34
This wasn't broken, but should mirror the non-in memory case to save an extra write.
Differential Revision: https://phab.mercurial-scm.org/D1807
(grafted from d592d5905ae972407fb6f9d649db1e2d41bf6c85)
(grafted from 9111386f1fe8316b9079669d0bd0e33704ba2a0c)
Summary:
`hg help` shows a list of all disabled extensions, which is gets by looking at
all submodules in the `hgext` modules. Exclude the new `extlib` submodule,
which is actually just a container for extension libraries.
Test Plan: Re-run `test-help.t`.
Reviewers: durham, #mercurial
Reviewed By: durham
Differential Revision: https://phabricator.intern.facebook.com/D6679750
Tasks: T24908724
Signature: 6679750:1515458834:8813e9ea9aaf6a4d55d24bc8a48ddc628f6070fa
Summary:
The `lib/linelog` directory contains pure C code that is unrelated from
either Mercurial or Python. The `mercurial/cyext` contains Cython extension
code (although for linelog's case, the Cython extension is unrelated from
Mercurial).
Cython is now a hard dependence to simplify the code.
Test Plan: `make local` and check `from mercurial.cyext import linelog` works.
Reviewers: durham, #mercurial
Reviewed By: durham
Subscribers: durham, fried
Differential Revision: https://phabricator.intern.facebook.com/D6678541
Signature: 6678541:1515455512:967266dc69c702dbff95fdea05671e11c32ebf28
The old code here was overly broad and would raise in cases when we didn't end
up calling `xmerge` and resolved using an internal tool (such as when
`premerge=True`).
Instead, let's swap out _xmerge if IMM is enabled and have the new tool raise
when called, which is the behavior we want.
Differential Revision: https://phab.mercurial-scm.org/D1739
This enables the SIGCHLD handler to work properly if some buggy program
started chg server with SIGCHLD blocked.
A test of this probably requires C code, but we don't have such kind of
tests already. Since this is a simple and clear fix, I'm leaving it as
"untested" but I did a manual test and there were no longer zombie workers.
Differential Revision: https://phab.mercurial-scm.org/D1737
Signals could be blocked by something like:
#include <unistd.h>
#include <signal.h>
int main(int argc, char * const argv[]) {
sigset_t set;
sigfillset(&set);
sigprocmask(SIG_BLOCK, &set, NULL);
execv("/bin/hg", argv);
return 0;
}
One of the problems is if SIGCHLD is blocked, chgserver would not reap
zombie workers since it depends on SIGCHLD handler entirely.
While it's the parent process to blame but it seems a good idea to just
unblock the signal from hg. FWIW git does that for SIGPIPE already [1].
Unfortunately Python 2 does not reset or provide APIs to change signal
masks. Therefore let's add one in osutil. Note: Python 3.3 introduced
`signal.pthread_sigmask` which solves the problem.
`sigprocmask` is part of POSIX [2] so there is no feature testing in
`setup.py`.
[1]: 7559a1be8a
[2]: http://pubs.opengroup.org/onlinepubs/7908799/xsh/sigprocmask.html
Differential Revision: https://phab.mercurial-scm.org/D1736
It's causing issues with pushrebase. See https://fburl.com/hd5yz5w6
(grafted from 431521982e6add35955b5b5e8b74d9e317d9ca0c)
(grafted from 8e34bb24dfc0436a65724f30e601efd08b6f2abd)
(grafted from 72e1214d63bd8de353fd1e9d1ffedc4d338fadde)
(grafted from 3588155212ad313a7abd52e44f7ebde5fedbfe06)
(grafted from de91755cc930607220d0926effc69506ca85eabc)
(grafted from 8a44eb22d38c9a60ec6102d43f5b7b5ba60100ab)
(grafted from 396b563d93e5645d4a4ec60ecf56bd91ab28f3be)
(grafted from 14a53b4cf2b352e88a3bcfda5075f9679d067b61)
(grafted from 457c63699089270e6658e1186c70a5bc3426f183)
(grafted from 5e525709296041b77cda5d9f04176a31784afbb7)
(grafted from 80926668f057e712eb11fdc580f65501830f4f8c)
(grafted from a350ecb9b0e94a332fdfebb7f0bba72d87a687ca)
This fixes treemanifest-infinitepush.
(grafted from 0364e7f0f9ef7eb92c7ca6f825cd252d1ecdac2e)
(grafted from c62ae0604a5124ae799f3ba7e309c23696fbd03d)
(grafted from e4b056930eb739c01d34a04e852643bf9c175d77)
Summary:
This separates some concerns that were introduced in
https://phab.mercurial-scm.org/D1341.
In particular, this makes it easier for `eden_dirstate` to provide its own
implementation, which incidentally, does not use `dirstatetuple`.
Test Plan:
Ran this against a complementary change in Eden and verified that all of Eden's
integration tests pass.
Reviewers: mbthomas, durham, #hg-reviewers!
Subscribers: mercurial-devel
Differential Revision: https://phab.mercurial-scm.org/D1354
(grafted from eb6590eec45176982198e1a91ccae5821f65958c)
(grafted from 43c991e04a2a1e56918040db8e7c0ea49d3d0abe)
(grafted from 4926c8af95b45593749ef96a194a1a065d916b3c)
(grafted from 5c77cbac5f578e3924a76f7017bf5d34b6718757)
(grafted from 14cf809e89209da973d17b38a8cde9fe6659e424)
(grafted from 7ddca31c3875fdcaf7a2a64198bbb321ec1cb018)
(grafted from 9e9ff559440e4733fda49ddeb1b9ce5e7c9660af)
(grafted from b14a9cd166ad6e48284ad887a7a47537ab4b1df4)
(grafted from 749be17f25a14a0a91d9e68e68df77a1b305eb7f)
(grafted from fd3caadd0a3f365551334f346875799964056354)
(grafted from ce1070bbc53c18d0afa9ba6fe000588c4fb3cf33)
(grafted from 834c5948156ceb0be31c40e95293277d9648f423)
(grafted from 24ec6b8d9402b53d38f9763b4706afaca4fb5b20)
(grafted from 0104f253d24f416e8034f302fd2bd57ec2cdbd3e)
(grafted from 893aadeea79010ca5e039a6bfaaca7471f7d9061)
(grafted from dc447bcdf1499da52b43723426aefd4b1cec4c01)
(grafted from 2db9cf55d1566b7273342b711073a57e075b11f3)
This should be effective to reduce the number of chg servers on OS X.
(grafted from 64cc416c50a4c54dfcebdbdcea73ea0a4dcc6b12)
(grafted from 4d35490df4c269132c43e5783d12697735ef1eae)
(grafted from 0f739f622491de9c6f7cfe3d8949843923291d95)
(grafted from ae11921088ae3ec06726719c7037ce7e9fb204a1)
(grafted from c1d14170449337f008d0f09e9c940309021236c8)
(grafted from e5d909b77f32f7d2e36c76cc318876e999d8cffe)
(grafted from 50a780fc7fb2ac422250d4aad426e0d010f5a290)
(grafted from f66d8cc7d1a72ece27e4ec682fa9e2401a50e2d2)
(grafted from a2fae201970502c50a7240a1fd8a597cc83f9db9)
(grafted from a102b2c1deb978bfc69be02730f9685062835691)
(grafted from dd3133eae2ba7268310115196c53135f0ccd3087)
(grafted from 3b15fb83e1d940a65b107422c4c8e34c9ecabf87)
(grafted from ed74ebc92814da5283e2cedbcfcaabb3f0c16f40)
(grafted from fbe58b0351a20121f1cfd68c80b37ded8f94e81d)
(grafted from 89e8affd3e204663e9ec57bfb2653c9f253553ce)
(grafted from 5b88b03fc62991939872516eb04dd35583225157)
(grafted from 3e4861b738936daa58b881e07af850cb9ffa336e)
(grafted from b05e674be709c24b61540eccf692bee5ba88536b)
(grafted from 28bd58eace4db7dc6ce739e9594974b3fe0d62ca)
(grafted from 8ec38a43c525ee62e8f46dd4be03fcbf63273760)
(grafted from f91b3053b66e4425d21bdbe7399832beeccd497e)
(grafted from 6658c07390dc0c27ec8f91effeeccaab34ca7d20)
Recently hg-ssh was changed to block writes via in-memory hook configuration
instead of by passing config hooks, and dispatch.py blocks any invocation of hg
serve --stdio that has options passed. We have infrastructure that sets up read
only serve processes without using hg-ssh, and it was broken by this change.
Let's add a --read-only option to hg serve so non-hg-ssh solutions can still
launch hg in read-only mode. This makes it also work with non-stdio serve
processes as well.
(grafted from 7a0ed9aad8526f689343a5a02aa4a66e5f3be1f2)
(grafted from bf33f750447d8b0dfeae2a311e1d1eb93e19d6a0)
(grafted from 9ada6a6e4ac2a92325592cd58edd9160e17c9e31)
(grafted from 50e676e99c3b7cc929ceaaebbd3c684a8a58f9d8)
(grafted from 01833a49fa4dca204dc0e606f21279530925307c)
(grafted from 301af2e1a42fe912acf90ae9a87ca6a20ce5cd5d)
(grafted from 6ae2eaad3edbfdfb04ac5880a86341e69980529c)
(grafted from fcdedd417b29d28797840fc2393e0ff846fa54c8)
(grafted from ddda3705adfb2ac103f506d694d2b30028dfaca9)
(grafted from 138e3cf3bc394c4ff507341a390e1876c7104042)
(grafted from f8073d595e87086213525dbb642962b84158ee9a)
(grafted from 96bac04dc722030250a53616f0fb55125829f25e)
(grafted from 2aeed48cc3b3324b564343d8515aed1ecec69b14)
(grafted from 4fb2e02a273c868febdae9530b0a07a53a0e92a7)
(grafted from c57b28835f0b880c075d5b8aa99ddb9da54b21b1)
(grafted from 35fd78b021bec96db63e8dd99f98efc3b2342380)
(grafted from 9ac14f96c9a82068f07a709374f359283c206791)
(grafted from 4b64c191a9aacd52ea58ae8ec943605667759398)
(grafted from 1db390a79e32db12dde7a225e26b86ed245f9473)
(grafted from 9a999ed1ce50af8e5fa03dc270488c37304d8c94)
(grafted from b6791f2eb83f176192c9df50c736bc4c54fab5a6)
(grafted from 4b6e87c5be38d9971399e4ad989e4261f283b93a)
(grafted from ae660c075b4af0849d1ff5d36404ef66aeea9933)
(grafted from ff0f3bf0834b38a527654495369cd538ca8744f4)
(grafted from bff8177767c9023295ff93bc520114bb909952a8)
(grafted from 05a833e4071b9da7b447669f6bd8a3f62c1d3c27)
(grafted from be8ab299731fb8295efbe10a014798c7a177d4a0)
(grafted from beec2bc2ebd9eaf7093bba5fca8fef07c669d970)
(grafted from 03d88ba3cd8795d17a99dc1b50ac55e1937d38e1)
(grafted from 92a33bc0d275a96c61553f8bccefcd32f1360931)
(grafted from 3d37833f54e37356f3e32db2ad8fb2ffe4fa14f2)
(grafted from 77fa3393787a9410e14afc26465abb5561253075)
(grafted from 9d908f86cc7986c167cef5cdccaabb565fd2bc04)
(grafted from 0dbc2023c42f72aea6b608e5111725163dcbb64b)
(grafted from 3acf4e9bb718617efaf31abcba583b9b7be2559d)
(grafted from c671696a06e418f5f040427efb3e51fe4c9fa6cd)
(grafted from 309f11e682eb3c6fa497bf767cbcbef3b0dbaf4b)
(grafted from 4f828ef4b70a6a16fe747d5f6393042bff204b5e)
(grafted from 71193e84a71d029dedc744882978285cbe5722e6)
(grafted from 2929832c61c9727bd884f94da5afa29e80334a96)
(grafted from 2ff8a9f1761f82ffa8ebd2a96d86f7de7c712d9c)
(grafted from b438cb1e6cff36e7e197da7669def8a5e528053c)
(grafted from a9ed103481b779af9e160d2b81a9bfe81cc7d173)
(grafted from d139e95d22dc811002dafef1ecaac5dac99825fe)
(grafted from b4e41a9f2c3a6328ada72810407686d11833347c)
(grafted from 2b3826c7b3bdf669b397f1ad31ae106a05d7b05b)
(grafted from 23737fb5e1d6874cf79a1eb841dd1614c0295a1b)
(grafted from 69588396b17d3dafeece8bc9e2101559d871d9fb)
(grafted from b3252a277a53b1657e6bcf31359b413d2becffcc)
(grafted from 12c8e6062d34d4d6cf0b0128084278800a1ed8f7)
(grafted from 8ce5c67748afbe6e82fc3f822e35ddc58cb03694)
(grafted from b0f656426efcf9a70386b1c781507f40ab95af49)
(grafted from 147ae96993dcffbed2f39f31795ef3d60631d43f)
(grafted from 2ecb34a565accf638e6004c59aa5b2d2361f9428)
(grafted from 6bfc53cd4c479b4e789d4086a2c7c2f4045a288f)
(grafted from bf73f92394a079928db6d4b0b3e7aa78448af91a)
(grafted from b69a654342339eb740527fc84af523eb53edeb71)
(grafted from 3f090f44e8a33cde8d5708454a5292d0976269e6)
(grafted from ac4432275d3b750405e53f67b1267579764f4fee)
(grafted from dbfd0bce0eded53dc7d824393f03ddd2f2e693fb)
(grafted from bf7087d072ca6c5d5dac2ddef4c43339d02f6133)
(grafted from b9d63feb8c90f83e74f3e9a89328419c81088082)
(grafted from 718e93a4e545f3e16d09c66f210a567427f1068a)
(grafted from e0ba57c8bf13ccc45b7eaa62d64e03038cd002ad)
(grafted from 5c849011421ad00ef190c2bf15c640656424f681)
(grafted from e833de714167fe6039b42f1cd1890b0470a32ea2)
(grafted from 480872890137130564910a29ed8ef3890810f0c4)
(grafted from 6224dc455a24542cf7d55721fceb14a08e92d391)
(grafted from 24ced5d2b0d6fb837a3994a80ef808e29f62ccc5)
(grafted from 452eb5c8624cc22867fafa692c6c7905e46da27a)
(grafted from cdc9f1b121878c26c986eca2233b5d03ea50ad74)
(grafted from 8b3a45fe3a612fdbee3a1f291f41bfaadfd16a6f)
(grafted from 2a07c0b3cb9785a9f8d5d669b885044e4d4544b1)
(grafted from 56d892df53cfdf3a13f38cd386a437ea59ef0d77)
(grafted from b63c65fad2d28a86a3bc3871d58e45019b11e6a1)
(grafted from 8618bd56f309543d8577000a4310fdf8648f1087)
(grafted from e04c7ddddc5cc40d6347d2336b81d5be2289243e)
(grafted from 5951fe6318d02a9b739f0174f3aecc3d5eead31c)
(grafted from 0f4d380f641a55791ca9eef13cd49da24cf40a7a)
(grafted from d7ecf3376e572d77b670cbe2184370b08d38dcf7)
(grafted from a75534c9e6d7a481303096e44e265593fb5b0b2f)
Summary: It surprising that this bug lasted for over a year.
Test Plan:
there's no tests for it. Do we have any tooling for simulating the inconsitent
state?
Reviewers: phillco, #mercurial
Reviewed By: phillco
Subscribers: baq
Differential Revision: https://phabricator.intern.facebook.com/D6651663
Signature: 6651663:1514938468:715869f443e24d7921c9a67b32d36c66fdf61a92
After applying suggestions from https://phab.mercurial-scm.org/D1564
to catch all exceptions in the same way I actually broke the handling of
KeyboardInterrupt on windows. The reason is that KeyboardInterrupt doesn't
dervie from Exception, but BaseException:
https://docs.python.org/2/library/exceptions.html starting from python 2.5
Test Plan:
Run hg on windows and ctrl-c during a large update. No random
exceptions from threads surface in the shell. Previously we'd nearly always get
stack traces from some of threads
Run tests ./run-tests.py
[...]
Failed test-convert-svn-encoding.t: output changed
# Ran 622 tests, 41 skipped, 1 failed.
python hash seed: 2962682116
The test failing seems to have nothing to do with the change and fails on base
revision as well
Differential Revision: https://phab.mercurial-scm.org/D1718
We could use patch.diffhunks() instead of patch.diff() to get filenames
without parsing patch content, but that isn't always possible because we
sometimes feed raw patch data to patch.diffstat().
In large repositories, updates involving the creation of many files check the
same directories repeatedly in the wctx manifest. Move these checks out to a
separate loop to avoid repeated checks hitting the manifest.
Differential Revision: https://phab.mercurial-scm.org/D1226
As mentioned in D1222, the recent pathconflicts change regresses update
performance in large repositories when many files are being updated.
To mitigate this, we introduce two caches of directories that have
already found to be either:
- unknown directories, but which are not aliased by files and
so don't need to be checked if they are files again; and
- missing directores, which cannot cause path conflicts, and
cannot contain a file that causes a path conflict.
When checking the paths of a file, testing against this caches means we can
skip tests that involve touching the filesystem.
Differential Revision: https://phab.mercurial-scm.org/D1224
If this feature is enabled, early options are parsed using the global options
table. As the parser stops processing options when non/unknown option is
encountered, it won't mistakenly take an option value as a new early option.
Still "--" can be injected to terminate the parsing (e.g. "hg -R -- log"), I
think it's unlikely to lead to an RCE.
To minimize a risk of this change, new fancyopts.earlygetopt() path is enabled
only when +strictflags is set. Also the strict parser doesn't support '--repo',
a short for '--repository' yet. This limitation will be removed later.
As this feature is backward incompatible, I decided to add a new opt-in
mechanism to HGPLAIN. I'm not pretty sure if this is the right choice, but
I'm thinking of adding +feature/-feature syntax to HGPLAIN. Alternatively,
we could add a new environment variable. Any bikeshedding is welcome.
Note that HGPLAIN=+strictflags doesn't work correctly in chg session since
command arguments are pre-processed in C. This wouldn't be easily fixed.
The next patch will add a flag for strict parsing of early options, where
we'll have to parse all early options at once instead of processing them
one-by-one by dispatch._earlygetopt(). That's why I decided to hook
fancyopts().
All dispatch._early*opt() functions is planned to be replaced with this
function. But in this stable series, only the strict mode will be handled
by fancyopts.earlygetopt().
The `hg lfconvert --to-normal` command uses the convert extension internally to
work its magic, but that produced devel-warn messages if the convert extension
wasn't loaded by the user. The test in 658e7a6d93e0 (modified here) wasn't
showing the warnings because the convert extension was loaded via $HGRCPATH.
Most of the config options default to None/False, but 'hg.usebranchnames' and
'hg.tagsbranch' are supposed to default to True and 'default' respectively.
The first iteration of this was to ui.setconfig() inside lfconvert, to force the
convert extension to load. But there really is no precedent for doing this, and
check-config complained that 'extensions.convert' isn't documented. Yuya
suggested this alternative.
This partially backs out 448e09d8859d.
Repoview can have a different life cycle, causing issue in some corner
cases. The particular instance that revealed this comes from localpeer. The
localpeer hold a reference to the unfiltered repository, but calling 'local()'
will create an on-demand 'visible' repoview. That repoview can be garbaged
collected any time. Here is a simplified step by step reproduction::
1) tr = peer.local().transaction('foo')
2) tr.close()
After (1), the repoview object is garbage collected, so weakref used in (2)
point to nothing.
Thanks to Sean Farley for helping raising and debugging this issue.
Before, early options were stripped from args, and because of this, some
kind of parsing errors weren't reported. For example,
$ hg ci -m -Ra file
would execute "hg ci -m file" in repository "a".
This patch fixes the issue by parsing early options again by real getopt-based
parser, and verifying the results. If the early parsing appears wrong, hg just
aborts. The current error message seems not nice, and should be improved, maybe
in V2 or follow-up.
Note that this isn't a security feature because we can still do anything by
using shell aliases.
So we can easily compare it with the corresponding getopt() result.
There's a minor behavior change. Before, "hg --cwd ''" failed with ENOENT.
But with this patch, an empty cwd is silently ignored. "hg -R ''" has always
worked as such, so -R has no BC.
This allows us to parse the original args later by full-blown getopt() in
order to verify the result of the faulty early parsing. Still we need the
'strip=True' behavior for shell aliases.
Note that this series is RFC because it seems to change too much to be
included in stable release.
Perhaps we'll need to restrict the parsing rules of --debugger and --profile,
where this patch will help us know why the --debugger option doesn't work.
I have another series to extend this feature to --config/--cwd/-R, but even
with that, shell aliases can be used to get around the restriction.
This is a minimal copy of localrepo.commit(). As the current amend() function
heavily depends on the wctx API, it wasn't easy to port it to use a separate
status tuple. So for now, wctx._status is updated in-place.
When origbackuppath is set, when looking to see if a file we are backing up
conflicts with a directory in the origbackuppath, we incorrectly match on
symlinks to directories. This means we try to call vfs.rmtree on the
symlink, which fails.
Differential Revision: https://phab.mercurial-scm.org/D1311
We change subrepos.allowed from a list of allowed subrepo types to
a combination of a master switch and per-type boolean flag.
If the master switch is set, subrepos can be disabled wholesale.
If subrepos are globally enabled, then per-type options are
consulted. Mercurial repos are enabled by default. Everything else
is disabled by default.
These config items control share behavior that is implemented in core.
Since the functionality is implemented in core, extensions may
leverage it.
Mozilla has one such extension. And, it needs to access share.pool.
Before this patch, a devel warning regarding accessing an unregistered
config option would be issued unless the share extension were loaded.
Moving the registration of the config options to core fixes this.
We have a security issue with git subrepos. I'm not sure if svn subrepo is
vulnerable, but it seems not 100% safe to allow writing arbitrary data into
a metadata directory. So for now, only hg subrepo is enabled by default.
Maybe we should improve the help to describe why git/svn subrepos are
disabled.
This allows us to minimize the behavior change introduced by the next patch.
I have no idea which config style is preferred in UX POV, but I decided to
get things done.
a) list: 'allowed = hg, git, svn'
b) sub option: 'allowed.hg = True' or 'allowed:hg = True'
c) per-type action: 'hg = allow', 'git = abort'
This is an alternative workaround for the issue5730.
Perhaps this is the simplest way of disabling subrepo operations. It does
nothing clever, but just aborts if Mercurial starts accessing to a subrepo.
I think Greg's patch is more useful since it allows us to at least check
out the parent repository. However, that would be confusing if the default
is flipped to checkout=False and subrepos are silently ignored.
I don't like the config name 'allowed', but I couldn't get any better name.
Previously, if there were unresolved files and the CWD drive was different from
the repo drive, `hg status -v` would page the normal status, followed by the
exception header. A stacktrace was waiting when the pager exited. The
underlying cause was the same as 12441ef4442f.
Unfortunately, I don't see any reasonable way to write a test this [1].
[1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-November/107401.html
This is a followup to 12441ef4442f. Since there's no way to ensure that more
drive letters than C: exist, this seems like the only way to test it. This is
enough to catch the 12441ef4442f scenario, as well as CWD outside of the repo
when the path isn't prefixed with path/to/repo.
It wasn't easy to extend the pathauditor to check symlink traversal across
subrepos because pathauditor._checkfs() rejects a directory having ".hg"
directory. That's why I added the explicit islink() check.
No idea if this patch is necessary after we've fixed the issue5730 by
splitting submerge() into planning and execution phases.
os.path.relpath() exploded if the 'root' and 'cwd' directories had different
drive letters. I noticed this in TortoiseHg when typing a fileset into the
filter, and it kept complaining until the closing '()' was typed. This was
reproducible on the command line with:
$ cd /d
$ hg -R /c/Users/Matt/Projects/hg files 'set:e'
Traceback (most recent call last):
...
File "mercurial\pathutil.pyc", line 182, in canonpath
File "ntpath.pyc", line 529, in relpath
ValueError: path is on drive c:, start on drive d:
The original condition was a bit harsh for extension authors since third-party
extensions need to preserve compatibility with older Mercurial versions, where
no defaults would be loaded from the configtable. So let's silence the warning
if the given default value matches, which should be harmless.
After a refactoring, _toolstr stopped having default="" as one of it's args,
therefore when called without a default it returns None and not "". This causes
concatenation to fail.
We've found a severe perf regression in `hg update` caused by the path conflict
checking code. The next patch will disable this by default.
Differential Revision: https://phab.mercurial-scm.org/D1222
> For some reason, I thought someone (Mads?) said we had basically given
> up on ui.slash, and some commands didn't support it.
(from https://bz.mercurial-scm.org/show_bug.cgi?id=5572#c1)
So the ui.slash option doesn't always work and is somewhat confusing. Let's
make it clearer we won't improve the situation.
As we were trying to transition off of the non production lfs-test-server for
further experimenting, one of the problems we ran into was interoperability. A
coworker setup gitbucket[1] to act as the blob server, tested with git, and
passed it off to me. But push failed with a message saying "abort: LFS server
returns invalid JSON:", and then proceeded to dump a huge HTML page to the
screen. It turns out that it is assuming that git is the only thing that wants
to do a blob transfer, and everything else is a web browser wanting HTML.
It's only a single data point, but I suspect other things may be doing this too.
RFC7231 gives an example [2] of listing multiple products in decreasing order of
significance. Since the standard provides for this, and since it works with the
one problematic server I found, I'm just enabling this by default for a better
UX.
There's nothing significant about the version of git chosen, other than it is
the current version.
[1] https://github.com/gitbucket/gitbucket/
[2] https://tools.ietf.org/html/rfc7231#page-46
Make 'hg outgoing' respect "paths.default:pushurl" in addition to
"paths.default-push".
'hg outgoing' has always meant "what will happen if I run 'hg push'?" and it's
still documented that way:
Show changesets not found in the specified destination repository or the
default push location. These are the changesets that would be pushed if a
push was requested.
If the user uses the now-deprecated "paths.default-push" path, it continues to
work that way. However, as described at
https://bz.mercurial-scm.org/show_bug.cgi?id=5365, it doesn't behave the same
with "paths.default:pushurl".
Why does it matter? Similar to the bugzilla reporter, I have a read-only mirror
of a non-Mercurial repository:
upstream -> imported mirror -> user clone
^-----------------------/
Users push directly to upstream, and that content is then imported into the
mirror. However, those repositories are not the same; it's possible that the
mirroring has either broken completely, or an import process is running and not
yet complete. In those cases, 'hg outgoing' will list changesets that have
already been pushed.
Mozilla's desired behavior described in bug 5365 can be accomplished through
other means (e.g. 'hg outgoing default'), preserving the consistency and
meaning of 'hg outgoing'.
We already have the ability to customize the ssh command line arguments, let's
add the ability to customize its environment as well.
Example use-case is ssh.exe from Git on Windows. If `HOME` enviroment variable
is present and has some non-empty value, ssh.exe will try to access that
location for some stuff (for example, it seems for resolving `~` in
`.ssh/config`). Git for Windows seems to sometimess set this variable to the
value of `/home/username` which probably works under Git Bash, but does not
work in a native `cmd.exe` or `powershell`. Whatever the root cause, setting
`HOME` to be an empty string heals things. Therefore, some distributors
might want to set `sshenv.HOME=` in the configuration (seems less intrusive
that forcing everyone to tweak their env).
Test Plan:
- rt
Differential Revision: https://phab.mercurial-scm.org/D1683
The windows workers weren't daemons and were not correctly killed when ctrl-c'd from the terminal. Withi this change when the main thread is killed, all daemons get killed as well.
I also reduced the time we give to workers to cleanup nicely to not have people ctrl-c'ing when they get inpatient.
The output when threads clened up nicely:
PS C:\<dir>> hg.exe sparse --disable-profile SparseProfiles/<profile>.sparse
interrupted!
The output when threads don't clenup in 1 sec:
PS C:\<dir> hg.exe sparse --enable-profile SparseProfiles/<profile>.sparse
failed to kill worker threads while handling an exception
interrupted!
Exception in thread Thread-4 (most likely raised during interpreter shutdown):
PS C:\<dir>>
Test Plan:
Run hg command on windows (pull/update/sparse). Ctrl-C'd sparse --enable-profile command that was using threads and observed in proces explorer that all threads got killed.
ran tests on CentOS
Differential Revision: https://phab.mercurial-scm.org/D1564
This adds config to disable/enable workers with default being enabled.
Test Plan:
enabled profile without updaing .hg/hgrc (the default should be to use workers) and ran
hg sprase --enable-profile <profile>.sparse
Watched in the proces explorer that hg started 12 new threads for materializing files (this is my worker.numcpus) value
Added
[worker]
enabled = False
to the .hg/hgrc and re ran the command. This time hg didn't spawn any new threads for matreializing of files
Differential Revision: https://phab.mercurial-scm.org/D1460
This adds handling of exceptions from worker threads and resurfaces them as if the function ran without workers.
If any of the threads throws, the main thread kills all running threads giving them 5 sec to handle the interruption and raises the first exception received.
We don't have to join threads if is_alive() is false
Test Plan:
Ran multiple updates/enable/disable sparse profile and things worked well
Ran test on CentOS- all tests passing on @ passed here
Added a forged exception into the worker code and got it properly resurfaced and the rest of workers killed: P58642088
PS C:\open\<repo>> ..\facebook-hg-rpms\build\hg\hg.exe --config extensions.fsmonitor=! sparse --enable-profile <profile>
updating [==> ] 1300/39166 1m57sException in thread Thread-3:
Traceback (most recent call last):
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\threading.py", line 801, in __bootstrap_inner
self.run()
File "C:\open\facebook-hg-rpms\build\hg\mercurial\worker.py", line 244, in run
raise e
Exception: Forged exception
Exception in thread Thread-2:
Traceback (most recent call last):
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\threading.py", line 801, in __bootstrap_inner
self.run()
File "C:\open\facebook-hg-rpms\build\hg\mercurial\worker.py", line 244, in run
raise e
Exception: Forged exception
<...>
Traceback (most recent call last):
File "C:\open\facebook-hg-rpms\build\hg\hgexe.py", line 41, in <module>
dispatch.run()
File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 85, in run
status = (dispatch(req) or 0) & 255
File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 173, in dispatch
ret = _runcatch(req)
File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 324, in _runcatch
return _callcatch(ui, _runcatchfunc)
File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 332, in _callcatch
return scmutil.callcatch(ui, func)
File "C:\open\facebook-hg-rpms\build\hg\mercurial\scmutil.py", line 154, in callcatch
return func()
File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 314, in _runcatchfunc
return _dispatch(req)
File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 951, in _dispatch
cmdpats, cmdoptions)
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\remotefilelog\__init__.py", line 415, in runcommand
return orig(lui, repo, *args, **kwargs)
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\undo.py", line 118, in _runcommandwrapper
result = orig(lui, repo, cmd, fullargs, *args)
File "C:\open\facebook-hg-rpms\build\hg\hgext\journal.py", line 84, in runcommand
return orig(lui, repo, cmd, fullargs, *args)
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\perftweaks.py", line 268, in _tracksparseprofiles
res = runcommand(lui, repo, *args)
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\perftweaks.py", line 256, in _trackdirstatesizes
res = runcommand(lui, repo, *args)
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\copytrace.py", line 144, in _runcommand
return orig(lui, repo, cmd, fullargs, ui, *args, **kwargs)
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\fbamend\hiddenoverride.py", line 119, in runcommand
result = orig(lui, repo, cmd, fullargs, *args)
File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 712, in runcommand
ret = _runcommand(ui, options, cmd, d)
File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 959, in _runcommand
return cmdfunc()
File "C:\open\facebook-hg-rpms\build\hg\mercurial\dispatch.py", line 948, in <lambda>
d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
File "C:\open\facebook-hg-rpms\build\hg\mercurial\util.py", line 1183, in check
return func(*args, **kwargs)
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\fbsparse.py", line 860, in sparse
disableprofile=disableprofile, force=force)
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\fbsparse.py", line 949, in _config
len, _refresh(ui, repo, oldstatus, oldsparsematch, force))
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\hgext3rd\fbsparse.py", line 1116, in _refresh
mergemod.applyupdates(repo, typeactions, repo[None], repo['.'], False)
File "C:\open\facebook-hg-rpms\build\hg\hg-python\lib\site-packages\remotefilelog\__init__.py", line 311, in applyupdates
return orig(repo, actions, wctx, mctx, overwrite, labels=labels)
File "C:\open\facebook-hg-rpms\build\hg\mercurial\merge.py", line 1464, in applyupdates
for i, item in prog:
File "C:\open\facebook-hg-rpms\build\hg\mercurial\worker.py", line 286, in _windowsworker
raise t.exception
Exception: Forged exception
PS C:\open\ovrsource>
Differential Revision: https://phab.mercurial-scm.org/D1459
This change implements thread based worker on windows.
The handling of exception from within threads will happen in separate diff.
The worker is for now used in mercurial/merge.py and in lfs extension
After multiple tests and milions of files materiealized, thousands lfs fetched
it seems that neither merge.py nor lfs/blobstore.py is thread unsafe. I also
looked through the code and besides the backgroundfilecloser (handled in base
of this) things look good.
The performance boost of this on windows is
~50% for sparse --enable-profile
* Speedup of hg up/rebase - not exactly measured
Test Plan:
Ran 10s of hg sparse --enable-profile and --disable-profile operations on large profiles and verified that workers are running. Used sysinternals suite to see that all threads are spawned and run as they should
Run various other operations on the repo including update and rebase
Ran tests on CentOS and all tests that pass on @ pass here
Differential Revision: https://phab.mercurial-scm.org/D1458
This disables background file closing when in not in main thread
Test Plan:
Ran pull, update, sparse commands and watched the closer threads created and destroyed in procexp.exe
ran test on CentOS. No tests broken compared to the base
Differential Revision: https://phab.mercurial-scm.org/D1457
This does a few things:
* Changes "-r" to "--rev", since "-r" is not a valid short form
* Removes non-existent "-l" and "-b" options
* Removes "..." after options, since we don't usually have that
Differential Revision: https://phab.mercurial-scm.org/D1706
I may add an alternative way of getting copy metadata (from changelog,
not filelog) but the chaining with the dirstate copy metadata will be
the same, so it will probably help to have this extracted. Even if
that doesn't happen, the next patch will show that we can simplify
this a bit after this refactoring, so it seems worth it regardless.
Differential Revision: https://phab.mercurial-scm.org/D1697
The function would ignore the matcher if the dirstate copies were
requested. It doesn't matter in practice because all callers used the
returned map only for looking up specific files from and those files
had already been filtered by the matcher (AFACT). Still, it's a little
confusing, so let's make it clearer by respecting the matcher in this
case too.
Differential Revision: https://phab.mercurial-scm.org/D1695
It seems like it didn't even exist when debugdiscovery was introduced
in 43f4c1113c8d (discovery: add new set-based discovery, 2011-05-02).
Differential Revision: https://phab.mercurial-scm.org/D1693
It seems like it didn't even exist when debugdiscovery was introduced
in 43f4c1113c8d (discovery: add new set-based discovery, 2011-05-02).
Differential Revision: https://phab.mercurial-scm.org/D1692
It seems like it didn't even exist when debugdiscovery was introduced
in 43f4c1113c8d (discovery: add new set-based discovery, 2011-05-02).
Differential Revision: https://phab.mercurial-scm.org/D1691
Once upon a time, in 1995, there were browsers that didn't understand <script>
tags and they would simply show the code inside as text. This started a
tradition of wrapping everything inside <script> in <!-- HTML comments -->.
Nowadays, it's not only not needed, but can be considered harmful[1]:
- within XHTML documents, the source will actually be hidden from all browsers
and rendered useless
- `--` is not allowed within HTML comments, so any decrement operations in
script are invalid
[1]: http://www.javascripttoolbox.com/bestpractices/#comments
When a dirstate backup is restored, it is possible that no actual changes to
the dirstate have been made. In this case, the backup is still a hardlink
to the original dirstate.
Unfortunately, `os.rename` silently fails (nothing happens, and no error
occurs) when `src` and `dst` are hardlinks to the same file. As a result,
the backup is left lying around. Over time, these files accumulate.
When restoring dirstate backups, check if the backup and the dirstate are
the same file, and if so, just delete the backup.
Differential Revision: https://phab.mercurial-scm.org/D1201
A recent refactor added a layer of abstraction to the dirstate which makes doing
things like 'foo in dirstate' now require some extra Python attribute lookups.
This is causing a 100ms slow down in hg status for mozilla-central.
The fix is to hoist the inner dict's functions onto the main class once the lazy
loading it complete, as well as store the actual functions before doing the
status loop (as is done for other such functions).
In my testing, it seems to address the performance regression, but we'll
need to see the perf run results to know for sure.
Differential Revision: https://phab.mercurial-scm.org/D1257
Before the recent refactor, we would not load the entire map until it was
accessed. As part of the refactor, that got lost and even just trying to load
the dirstate parents would load the whole map. This caused a perf regression
(issue5713) and a regression with static http serving (issue5717).
Making it lazy loaded again fixes both.
Differential Revision: https://phab.mercurial-scm.org/D1253
A future diff will move the lazy loading aspect of dirstate to the dirstatemap
class. This means it requires a slightly different strategy of clearing than
just reinstantiating the object (since just reinstantiating the object will
lazily load the on disk data again later instead of remaining permanently
empty).
So let's give it it's own clear function.
Differential Revision: https://phab.mercurial-scm.org/D1252
The same applies to '?' if --quiet is used (or any of the other states if some
of -marduic is specified), but I couldn't figure out how to express that
clearly.
This seems a bit hacky, but works well. There should be no reason that
static-http repo had to load dirstate.
Initially I tried to proxy os.stat() call through vfs so that statichttpvfs
could hook it, but there wasn't a good error value which the statichttpvfs
could return to get around the util.filestat issue.
The short options "-c" and "-C" may be confusing for a novice reading the
documentation. Let's try to be more explicit, also mentioning the equivalent
long options ("--check" and "--clean") in the comments.
Per the last discussion, this is more reliable and consistent way than
suppressing an error. For grep, erroring out might be inconvenient, but
for curl, non-zero exit status should be detected. The latter wouldn't be
possible if non-zero status is ignored.
https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-October/105727.html
Previously, when user scrolled down to see the next page on /graph, all hgweb
did was re-render everything that would be visible (by simply incrementing
revcount). It was not efficient at all, and this patch makes /graph page behave
similarly to the regular /log: every new page only consists of new changesets,
no duplication, and only jsdata is based on the full set of changesets required
to build accurate graph.
This is achieved by adding "?graphtop=<node>" to the next page URL template,
effectively remembering where the graph started, and using that value to create
the new `tree` that covers the whole visible graph. That variable is then used
to produce jsdata for redrawing graph client-side.
nextentry is used for the same purpose as on /log page (to format the next page
URL), but it's not a part of the graph.
hgweb determines and passes to templates some variables related to graph
appearance, like bg_height, canvaswidth and canvasheight. bg_height was and
still is used for graph.scale() call in graph.tmpl, and the two latter
variables were used in <canvas> element as width and height properties, and
they were set before JS code got to run. Setting these properties server-side
doesn't make a lot of sense, because a graph that has been scaled should
calculate things like width and height on its own when being rendered.
Let's move (re)sizing <canvas> to JavaScript (to Graph.render function) and
stop parsing HTML with regular expressions just to know new width and height.
That extra loop that only counts cols is required because <canvas> can't
be resized after or in the process of rendering (or it gets cleared).
Incidentally, SVG doesn't have this problem and I'm hoping to switch graph to
using it in future.
There also was truecanvasheight, but according to hg grep --all it was never
used, see f5506d2a674c.
Consumers in this function use output of graphmod.colored(), but only want
items with type == CHANGESET, so let's filter it early.
This is primarily just a refactoring, but it also fixes a potential small bug
with `rows = len(tree)` (this variable is used for "Rows shown" line in
raw-graph) if there are items of other types.
Similar to map, the return value of zip was also turned into a generator on
Python 3. We have multiple occurences in our codebase where we retrieve the
values of zip multiple times, so let's introduce pycompat.ziplist similar to
pycompat.maplist.
Differential Revision: https://phab.mercurial-scm.org/D1675
