Summary:
The manifestlog has some mutable packs that it persists and manages the
lifetime of. As long as every tree writer uses these, we'll only get one new
pack per process/transaction.
The prefetch code was not using these though, so things like 'hg log -p' would
produce many packs over the course of many fetches. This change fixes it to
always write to the same pack.
Reviewed By: phillco
Differential Revision: D8670684
fbshipit-source-id: e56590c0ced84a99a06a94028d6a4e14bcc22ab6
Summary:
A future diff will make us depend on repo.close() being called to
finalize the writing of some tree data. It turns out we didn't call close on the
destination repo during a clone, which caused some tests to change. Let's call
close on that repo to prevent a test change in the next diff.
Reviewed By: phillco
Differential Revision: D8670680
fbshipit-source-id: ad63ccfc51ec4c3de2dad494fdb1ae8f6ba8e40a
Summary:
An upcoming diff changes the way we use the hgcache and this test is
mildly affected because it has two local peers using the same cache. Let's
change the test so the two peers use different caches, that way the test
requires no changes for the next diff.
Reviewed By: phillco
Differential Revision: D8670681
fbshipit-source-id: acd0051aa07974fa6dc2850f69957ff9ae701c83
Summary:
We already had mutable local packs, which were used to allow all local
writes to go to the same pack. Let's add an equivalent mutable shared pack. A
future diff will change shared writes to go to these packs.
Reviewed By: phillco
Differential Revision: D8670682
fbshipit-source-id: cbe3ab40b30f5ff53ece23914a95ffeba8b03ec3
Summary:
ManifestLog._mutabledata/historypack is used to persist a single set of
mutable packs for all local writes in the process. In a future diff we'll be
addeding the same concept for shared packs as well, so let's refactor the
current logic to explicitly be named _mutablelocalpacks.
Reviewed By: quark-zju
Differential Revision: D8670679
fbshipit-source-id: 8b1a04490c71525eca9466a7288dd7fd23991089
Summary:
Previously we were encoding the full filepath into a single filename to
use as a backup file. If the full path was over 255 characters, then we couldn't
create a file with the name.
Since the user never sees this name unless they run --debug, let's just get rid
of the encoded name and use a purely temporary name.
Reviewed By: farnz
Differential Revision: D8695749
fbshipit-source-id: 18b6d00e5f4c856b7dd6073e9d8c9677c4baea69
Summary: See the commit content itself for the details on the testing.
Reviewed By: quark-zju
Differential Revision: D8695360
fbshipit-source-id: c7d4bb85a1c5ec3648a00e171b3ad592066d5daf
Summary:
this will reduce cloud sync errors and unnecessary cloud sync calls
the daemon triggers cloud sync on service start/restart
it is not always the time when the machine online (and connected to correct network), so we get cloud sync errors
Reviewed By: markbt
Differential Revision: D8692972
fbshipit-source-id: 59033fd4c3e7c30100d82b908442bbf1ebea9322
Summary:
Previously, drawdag can only create commits which add or modify files.
In some cases, removing files is needed. This diff makes it support removing
files by special comments like "(removed)" and "(renamed from ...)".
Reviewed By: phillco
Differential Revision: D8698324
fbshipit-source-id: 97da946332fe64275cef40ba9ba6ac60846d7de9
Summary:
This commits just adds some tests for the `globalrevs` extension. See
the commit content itself for the details on the testing.
Reviewed By: DurhamG
Differential Revision: D8364382
fbshipit-source-id: f32b9fedd70f5f8ad40a034ce0d28867f9d1bdb4
Summary:
D8464848 added code to invalidate merge drivers after running them, for reasons described in that diff.
But it turns out deleting from sys.modules is not enough, because Python will occasionally write a `.pyc` of the original hook, and re-use it when loading again, thus bypassing the invalidation.
This doesn't happen at all on my Mac, but happens approximately every other time when running on CentOS. Thanks to DurhamG who discovered the flakiness.
This fixes the flakiness of the test, and AFAIK this would be a problem in the wild, too, though we haven't heard of it yet.
Reviewed By: singhsrb
Differential Revision: D8675720
fbshipit-source-id: bec8fec6af9a362db1f0dd5d262932cbda6137ff
Summary: We initially split out lots of different causes of slowness into their own unique tags, so that we could identify the source of slowness. Now that we've identified all the big causes, wrap them all back up into summary categories - we can change back to detailed tags at callsites if we want to separate out phases.
Reviewed By: ikostia
Differential Revision: D8664314
fbshipit-source-id: 6e141382479ca044847db1611f885e08f24f986b
Summary: Break the cycle by using local variable.
Reviewed By: chadaustin
Differential Revision: D8686397
fbshipit-source-id: e1846d33472066466814ab2c0c666f72b8393ff3
Summary:
A reference of `repo` is kept in the `finalize` closure. That forms a cycle:
- finalize needs repo
- repo.transaction needs finalize
Break the cycle by making `finalize` use `self` passed to `repo.transaction`
instead.
Reviewed By: chadaustin
Differential Revision: D8676286
fbshipit-source-id: e27bed996883e2bf22cdeec41fcf1ab3e771b208
Summary: When wrapping the repo object, use local variable if possible.
Reviewed By: chadaustin
Differential Revision: D8676284
fbshipit-source-id: d35d9b7b9357a8e16d4eb10032911ecbdc34b907
Summary: When wrapping the repo object, use local variable if possible.
Reviewed By: chadaustin
Differential Revision: D8676283
fbshipit-source-id: 627e5fa4cc126efff42f29cf463fa90b97d9a0e0
Summary:
fastmanifest uses a `uiproxy` semi-global variable to workaround a chg issue
where `uisetup` received an out-of-date `ui` object.
That's no longer necessary D7840237. Removing the `uiproxy` global variable
fixes the memory leak.
Reviewed By: chadaustin
Differential Revision: D8676285
fbshipit-source-id: a5e042e4f61460ce53d43a231f07d70f27a52f19
Summary:
The blackbox extension uses a global variable for `lastui` in case certain ui
objects do not have an associated repo. That `lastui` reference makes the `ui`
(and `repo`) harder to be dropped. It also sets `ui.repo` which could form a
cycle if `repo.ui` is that `ui`. Fix it by using weakref.
Reviewed By: chadaustin
Differential Revision: D8675908
fbshipit-source-id: 7d0c083cc5c7a3ac134379732de1e0c408a1d5fc
Summary:
Continue with the sampling fix. hgsubversion is the second trouble marker.
It uses similar patterns (reference to non-local variables in closure).
Fix it by using local variables.
Reviewed By: chadaustin
Differential Revision: D8675439
fbshipit-source-id: bff7a827608fbf4a2662e6e9956ddb733c6a3bb5
Summary:
The latest fbcode dev build environment reports several memory leak issues from
the Rust extensions. It's caused by CPython not calling `tp_dealloc` of the
native object, which is mostly caused by `repo` not released for various
reasons. Add a test demonstrating the issue to verify the upcoming fix and
prevent future regression.
The most noticable problematic extensions are listed in the test.
Reviewed By: chadaustin
Differential Revision: D8675484
fbshipit-source-id: 152aff1e8ee574151d8a655e73cfa93dec5b91cb
Summary:
When debugging T30931730, I created a samll repo with almost all extensions
disabled. Then there is no leak report for the eden importer. Then I start
bisecting the extensions, and found the sampling extension is one of the
troublemakers. This is similar to a fix in hgsubverison [1].
This does not reproduce outside the eden importer because the atexit handlers
are consumed by `dispatch.py`. Regarding on whether eden importer should also
process atexit handlers or not remains a question. Anyway, avoiding cyclic
references shouldn't make things worse.
[1]: 3c4fbf59e5
Reviewed By: chadaustin
Differential Revision: D8675401
fbshipit-source-id: cf87869d2006d5166c697c2a86b65ece6cbe8c06
Summary:
This code is no longer needed now that the buck built binary uses the same
entry point as the setup.py built code.
Reviewed By: quark-zju
Differential Revision: D8675727
fbshipit-source-id: 1b3a44371453aa16c530f73b48611958082ee080
Summary:
The windows compiler doesn't support this, so let's just hard code our
default value.
Reviewed By: singhsrb
Differential Revision: D8679518
fbshipit-source-id: 031a6e06d12c4980756e60cdeb76b6828ed8f775
Summary:
Once we have the `globalrevs` extension enabled, it should always be
enabled for proper operation of the repository. This test checks that disabling
the `globalrevs` extension would cause a Mercurial error provided we the
`.hg/requires` is valid.
Reviewed By: DurhamG
Differential Revision: D8364385
fbshipit-source-id: 9c512dd2f9fa27c69f366f15dab040286a195ae1
Summary:
With the `globalrevs` extension enabled, Mercurial starts adding a
strictly increasing revision number to each commit which is accessible through
the `globalrev` template.
Reviewed By: DurhamG
Differential Revision: D8364387
fbshipit-source-id: 84bd43f56b9cb1c775cffb875501dd64d10ffa42
Summary:
This commit adds very basic tests for the Union History Store. These
tests just test for expected output of operations on bad/empty stores.
Reviewed By: quark-zju
Differential Revision: D8553821
fbshipit-source-id: a0dfa47f10083c37901535e8a810a99693a28c82
Summary: This commit just introduces the Union History Store.
Reviewed By: DurhamG
Differential Revision: D8553822
fbshipit-source-id: 6c7ee0b5d33dae6d51b4179616d206f42eb0cd50
Summary: This commit just introduces the history store.
Reviewed By: DurhamG
Differential Revision: D8553823
fbshipit-source-id: 93af6059296d11c4fcc0dd306b4472c4f2168fa7
Summary:
Adjust link node needs to read the node for a given file in a manifest,
and it can inspect manifests far back in time. The tree fetching code attempts
to download the entire tree, which makes each lookup take 40+ seconds in some
situations. Let's tweak adjust linknodes so downloads during it will download
trees one-by-one as needed.
Let's also record how long adjust link node takes in the scuba table.
Reviewed By: quark-zju
Differential Revision: D8629725
fbshipit-source-id: 4f1f9a3694c45a10a53f23ee081ba5f3c9d296b6
Summary:
We're experiencing cases where users are pulling down far more trees
than they need. Now that we can download just portions of a tree, let's add a
depth limit to the fetch code path. This will let us avoid downloading trees
that are never used.
The default is still to fetch all trees, but future diffs will make certain code
paths download only portions of trees.
Reviewed By: quark-zju
Differential Revision: D8523217
fbshipit-source-id: 066c5aef14b00ea0a366cf43e8a89ee053f82d22
Summary:
We're experiencing cases where users are pulling down far more trees
than they need. Let's enable the ability to download a certain depth of trees.
In this diff we just implement the actual iteration limiting in native code. In
a future diff we'll make the appropriate python changes and add a test.
Reviewed By: quark-zju
Differential Revision: D8523215
fbshipit-source-id: ccfeadee1525b86b1a5578631dbafe666de41f4c
Summary:
Previously, we could only ask `logginghelper` to log some `section.name`
config option via `ui.log` under the `name` name. This might be not enough
when section keys are generic and only make sense when context is available.
The entire list of config options to log was also supposed to be provided
under a single config option, `logging.configoptions`, therefore it was
impossible to extend it in the included config files, just completely
overwrite.
Let's solve this by dedicating the entire `[logging]` config section to
the purpose of defining other config options to log, where each key would
be a name to use and each value would be a `section.key` of the desired option
to log.
Reviewed By: DurhamG
Differential Revision: D8639436
fbshipit-source-id: 442792c2ea11cb4c9e7191b92cd4a8c0c67e9f27
Summary:
Update the buck build to use the "hg" script as its main module, rather than
using its own custom mercurial/main.py file.
This also changes how the extensions code decides whether to perform strict
checks or not. Previously this was disabled for buck builds by checking the
entrypoint argument. Now that buck builds use the same main module the
entrypoint is the same. Therefore we now distinguish buck-based builds based
on the presence of the `__manifest__` module.
Reviewed By: ryanmce
Differential Revision: D8601651
fbshipit-source-id: abfea4c6d91596149c240baa1bead740dffe8750
Summary:
Remove the code from the highlights extension that tries to eagerly load the
pygments lexers. This prevents commands that load the highlight extension
from failing in some circumstances.
This code was added in https://phab.mercurial-scm.org/D1619 since it avoided
"really weird traceback problems caused by demandimport messing with some of
the lexer plugins." It also claimed that eagerly loading these lexers
shouldn't affect most users since the highlight extension is only relevant for
servers. However the highlight module is loaded by other client-side commands
in several cases (for instance, "hg help"). When combined with the strict
foreign module import checking this causes this extension to get loaded
triggering this code.
This `find_plugin_lexers()` is using pkg_resources internally to find its
dependencies. This fails if some of the dependencies were installed as plain
python packages rather than as setuptools packages or eggs.
Reviewed By: quark-zju
Differential Revision: D8656400
fbshipit-source-id: 0b16852208e56823c5f8ab055b69020ac338447b
Summary:
When we apply draft commits to the repository, through
pull/unbundler/etc, we want to prefetch their public parent trees. If we don't,
when the draft commit is accessed it has the root node, but not some of the
children, and therefore attempts to download all the necessary trees ondemand
without any base trees. This is very expensive.
Let's prefetch the public parents, which should be relatively cheap since it can
delta against a nearby tree, and which is probably required anyway since
applying draft commits usually indicates they will be accessed soon.
Reviewed By: quark-zju
Differential Revision: D8654617
fbshipit-source-id: 63ee1e958fc0a0abdb8fa7bdb8c5d9e01181b6a7
Summary:
A future diff will want to use the base wirepack.receivepack with some
existing data and history packs, so let's refactor it so it can be used this
way.
Reviewed By: phillco
Differential Revision: D8404524
fbshipit-source-id: b1ab053870042933a2e2125f9bd58175479da255
Summary:
This commit just moves out the updates to the revision_references
table to a separate method so that it can be easily wrapped around.
Reviewed By: DurhamG, phillco
Differential Revision: D8406981
fbshipit-source-id: 8046a3e5b41dcf6c2f569699ac03de78611ba0bd
Summary:
p4fastimport has been replaced by p4seqimport months ago, on top of that previous assumptions have changed that might make p4fastimport not work (e.g. allowing import to connect to LFS).
This change cleans up a lot of unused code and a bunch of tests that no longer make sense.
Reviewed By: zhh95
Differential Revision: D8525286
fbshipit-source-id: 91d33e7530bf6df1e1ec92fae3acde0345230247
Summary: Otherwise, tests that use seq.py to create files, like test-rebase-inmemory-abort-tool.t, will end up with different content, and thus commit hashes, on Windows.
Reviewed By: quark-zju
Differential Revision: D8643553
fbshipit-source-id: 8bb533765f3f6bd794aa1f7ccaf754cfbe5c3bfb
Summary:
this command will be use in fbclone
it does sync and join if a person has been used already the commit cloud workspace in that repo
Reviewed By: markbt
Differential Revision: D8640667
fbshipit-source-id: 9c8d96497cf7930b995a49c8bdaebe7048e0c1e8
Summary: show commit cloud workspace state and infinitepush state as pretty jsons
Reviewed By: farnz
Differential Revision: D8611360
fbshipit-source-id: 97c0f38cd25bc0cc73f9a41d474b94234e463d3c
Summary: This commit just fixes the messaging for the errors.
Reviewed By: DurhamG
Differential Revision: D8553820
fbshipit-source-id: 73f2cd13e7538b6870b16a0e47e657a6d08af9e3
Summary:
The scenario we are handling is as follows:
1. the user runs a long-running Mercurial command with a lot of output, like
`hg log`
2. Mercurial runs pager, which is a subprocess, like `less` and changes its
stdout/stderr to point to pager's stdin.
3. The user exits from the pager (by pressing `q`, for example)
4. The pipe connecting Mercurial stdout/stderr with pager's stdin is now dead.
5. Mercurial tries to write something into its stdout, a failure occurs
6. The finally section restores stdout/stderr to be a tty
7. A failure gets caught up in the stack and printed to stderr, now a tty
A failure is vague here. On Linux, it means a `SIGPIPE`, on Windows, just and
exception that is manifested in Python as `IOError` with `errno=EINVAL`.
This is how Linux deals with it:
1. registed a signal handler for `SIGPIPE`, make this handler raise a
Mercurial-defined `error.SingalInterrupt`, which is a child exception class to
`KerboardInterrupt`
2. catch `KeyboardInterrupt` in `dispatch.dispatch`, handle the broken pipe
case there
For Windows this does not work, since there's no `SIGPIPE`, so
`SignalInterrupt` never gets raised, so we never handle the `IOError`.
This diff just explicitly handles the `IOError` in `dispatch.dispatch` and
silently consumes the `errno.EINVAL` case.
Reviewed By: quark-zju
Differential Revision: D8610555
fbshipit-source-id: 6cadba6a9ec296f03b1b616b763e56ec55da3a70
Summary:
Use the registrar to register template keywords and revset predicates, and
remove code for running the extension in old versions of Mercurial.
Reviewed By: farnz
Differential Revision: D8633454
fbshipit-source-id: aa0ebf86ff644072a3e78df8142dd8da7a188309
Summary:
Assume the stack:
```
o - C
|
o - B
|
o - A
|
o - Root
```
then if we need to metaedit commits A and B the stack becomes:
```
o - C' (metaedited)
| o - B'
| |
| o - A' (metaedited)
| /
o - Root
```
This bug affects also Jellyfish while submitting the stack of commits where some commit in the middle is already assotiated with Phab diff.
The bug is caused by the logic where we firstly metaedit and rebase the commits to metaedit in their topological order and only after all their descendants. So for now the given commits are processed with their descendants in topological order to track new parents for each commit.
Reviewed By: mitrandir77
Differential Revision: D8591897
fbshipit-source-id: 330a598e084af60cff42255510aca420bfe389de
Summary:
Add new revsets for finding remote bookmarks and remote branches by their exact
name, or by regular expression.
Reviewed By: quark-zju
Differential Revision: D8588923
fbshipit-source-id: 11e6d3636a0cccc5145c7dbd8824135b51dcbab3
Summary:
Add a config option `fsmonitor.detectrace` and environment variable
`HGDETECTRACE`. Once set, perform an additional query to check if there are
writes to the working copy while `wctx._buildstatus()` is being calculated.
Also enable the race detector for all fsmonitor tests.
Reviewed By: wez
Differential Revision: D8597957
fbshipit-source-id: 5d83d529641325dc8d5e72ad059eed8db27d1f2a
