The branch information was properly preserved in the changeset, but the
"active" branch of the working copy could be lost (the branch of the base
being used).
Histedit used to behave properly in this regard but the case was not tested
and regressed 4 years ago in c038baa4b6f0.
Summary: For now pushrebase only supports the legacy exchnage mode.
Test Plan: - fewer tests fail after this change
Reviewers: #fbhgext
Differential Revision: https://phab.mercurial-scm.org/D1660
Summary:
A 'bookmarks' capability was recently introduced in the core. Let's make
tests aware of it.
Test Plan: - rt
Reviewers: #fbhgext
Differential Revision: https://phab.mercurial-scm.org/D1659
Summary:
[1] adds a new check which moves bookmarks race check earlier, and this conflicts
with `pushrebase`. In reality `pushrebase`-enabled repos don't need this check
anyway, so we can just remove this generator.
[1] https://www.mercurial-scm.org/repo/hg/rev/f135b42f3faf
Test Plan: - run pushrebase tests, see them change their status from failing to passing
Reviewers: #fbhgext
Differential Revision: https://phab.mercurial-scm.org/D1655
A dirstate update that appends data to the treedirstate tree file, followed by
a hard reboot before the filesystem cache is flushed, can result in a dirstate
tree root that is referred to by the dirstate file, but does not contain the
correct data. Ensure the appended data is synced to disk before returning from
`Store.flush()`.
Differential Revision: https://phab.mercurial-scm.org/D1654
Sometimes users end up with tens of thousands of shared packfiles, which
prevents Mercurial from working. Dropping them merely degrades offline support,
so let's choose the lesser of two evils.
Differential Revision: https://phab.mercurial-scm.org/D1582
Summary:
The background prefetch uses the incorrect executable while running
the prefetch command due to which there are no logs for the background
prefetch. This commit fixes that by specifying the correct executable.
Test Plan:
- Ran all the tests.
- Tested the output of util.hgexecutable() with system hg and chg, and
verified that it returns the correct path. As expected, util.hgcmd() returns
the corresponding path for hg and chg in this case.
Reviewers: #fbhgext, durham
Reviewed By: #fbhgext, durham
Differential Revision: https://phab.mercurial-scm.org/D1600
Summary:
The incremental repacking currently repacks the 2 biggest files if it
doesn't find anything else to do. This was perhaps done to converge to a single
pack file as soon as possible. However, the 2 biggest files can be huge and
this leads to a large repack. Moreover, the resulting pack file after this
repacking is then considered for a subsequent incremental repacking which is
fairly huge as well. Therefore, with the current setup, we can potentially end
up doing 2 big incremental repacks. After this change, we will have just one
incremental repack which is almost equally costly to get to the same state.
Test Plan: Ran all the tests.
Reviewers: #fbhgext, durham
Reviewed By: #fbhgext, durham
Differential Revision: https://phab.mercurial-scm.org/D1596
Summary: Make sure that we can build Rust extensions on Windows.
Test Plan:
- `set USERUST=1`
- run FB build script, which invokes `fb-hgext`'s `setup.py`
- see it not failing
Reviewers: mbthomas, #fbhgext
Differential Revision: https://phab.mercurial-scm.org/D1590
Summary:
Incrementally repacking too big files results in more problems than
solving them like the repack taking a long time to finish and eventually
culminating in a one huge pack file. Therefore, it is preferable to ignore
files over a certain size during the incremental repack.
Test Plan: Added a test and ran all the tests.
Reviewers: #fbhgext, durham
Reviewed By: #fbhgext, durham
Subscribers: durham
Differential Revision: https://phab.mercurial-scm.org/D1588
Summary:
Adding any repacking option during the incremental repack results in
the _computeincrementalrepack signature to change. This should help avoid that
problem.
Test Plan: Ran all the tests.
Reviewers: #fbhgext, durham
Reviewed By: #fbhgext, durham
Subscribers: durham
Differential Revision: https://phab.mercurial-scm.org/D1587
Summary: Some options seem convoluted enough to warrant documentation.
Test Plan: Ran all the tests.
Reviewers: #fbhgext, durham
Reviewed By: #fbhgext, durham
Differential Revision: https://phab.mercurial-scm.org/D1586
It saves 0.15s per transaction in fbsource. With other implemented
optimization (treemanifest, treedirstate, and clindex and perftweaks).
Before:
(hg-dev) $ echo 1 >> README ; time hg commit -m bar
postclose warms-cache: User 0.140 Sys 0.020 Real 0.160
real 0m0.613s
user 0m0.419s
sys 0m0.186s
After:
(hg-dev) $ echo 1 >> README ; time hg commit -m bar
postclose warms-cache: User 0.000 Sys 0.000 Real 0.000
real 0m0.461s
user 0m0.303s
sys 0m0.149s
Differential Revision: https://phab.mercurial-scm.org/D1516
The commit code path reads branchheads just to show whether the new commit
is creating a new head or not.
439 | _docommit commands.py:1505
111 \ branchheads localrepo.py:2169
109 | branchmap localrepo.py:953
109 | _branchmapupdatecache perftweaks.py:146
109 | closure extensions.py:343
109 | _branchmapupdate perftweaks.py:107
Disabling the branch calculation saves about 100ms during commit in a large
repo.
Test Plan:
Added a test.
Differential Revision: https://phab.mercurial-scm.org/D1514
The branchmap code is actually quite expensive (O(changelog)). So let's try
to avoid them as much as possible.
First, reading or writing on-disk caches are unnecessary since even with the
on-disk cache, it still has to pay O(changelog) headrevs cost to
incrementally update the cache, which is the same cost of rebuilding the
cache from memory.
Secondly, `repo.branchmap()` calls update unconditionally so the update
function should be smart to not update unnecessarily. That's the `validfor`
check.
Thirdly, `repo.branchmap()` calls `branchmap.updatecache` which constructs
the "served", "immutable", "base" repoviews as an attempt to "incrementally"
build the branch cache for the "visible" repoview. There is no need to have
that many repoviews in this case. Therefore `branchmap.updatecache` is
replaced to a simplified version that does not construct other repoviews.
Note: there is a simple cache in index.headrevs() but that cache gets
invalidated if filteredrevs has a different address - that means different
repoviews will not benefit from index.headrevs() cache if they pass
different filteredrevs objects.
The change is gated by a new config option `perftweaks.disablebranchcache2`
so we can rollback if anything went wrong.
Test Plan:
Run core hg tests which do not use branches (about 371 tests):
ruby -e 'puts Dir["*.t"].reject{|x|File.read(x)[/(hg.* branch(es)?|branch\(|--branch|branch:\s*[^d]|cache\/branch|updating to branch\s*[^d])/]}' > /tmp/tests
./run-tests.py -l --extra-config-opt=extensions.perftweaks=$HOME/fb-hgext/hgext3rd/perftweaks.py --extra-config-opt=perftweaks.disablebranchcache=1 --extra-config-opt=perftweaks.disablebranchcache2=1 -j `nproc` `cat /tmp/tests`
Categorize them and analyse failures:
"abort: push creates new remote head X":
Failed test-exchange-obsmarkers-case-A2.t: output changed
Failed test-exchange-obsmarkers-case-A3.t: output changed
Failed test-exchange-obsmarkers-case-A5.t: output changed
Failed test-exchange-obsmarkers-case-B3.t: output changed
Failed test-exchange-obsmarkers-case-D4.t: output changed
Failed test-obsolete-checkheads.t: output changed
Failed test-push-checkheads-unpushed-D3.t: output changed
Failed test-push-checkheads-unpushed-D4.t: output changed
Failed test-push-checkheads-unpushed-D5.t: output changed
Failed test-push-checkheads-unpushed-D7.t: output changed
Failed test-bundle2-exchange.t: output changed
Failed test-phases-exchange.t: output changed
This is already broken before this change because we ignored `revgen`
parameter passed to `branchcache.update` from `checkheads` code path.
Harmless: config/extension/help/ui.log output changes:
Failed test-basic.t: output changed
Failed test-debugextensions.t: output changed
Failed test-help.t: output changed
Failed test-devel-warnings.t: output changed
Failed test-blackbox.t: output changed
Harmless: Change because .hg/cache does not get created:
Failed test-parseindex.t: output changed
Harmless: Branch related tests that are not filtered by regex:
Failed test-convert-cvs-branch.t: output changed
Failed test-convert-cvs.t: output changed
Failed test-convert-tagsbranch-topology.t: output changed
Harmless "Unable to find a working hg binary to extract the version from the repository tags":
Failed test-run-tests.t: output changed
Run jf integration test which catches an error caused by previous
problematic patches.
Differential Revision: https://phab.mercurial-scm.org/D1501
The current method to check whether the bundle is a bundle2 push or not
introduces a new bundle2 part and we can only store changegroup using that part.
This patch introduces logic which will add a stream level parameter to bundle2,
reading whose value while unbundling, we can easily decide whether this bundle
is infinitepush bundle or not.
The stream level parameters are read before any of the parts and hence we can be
sured that we get to know whether this push if infinitepush or not before
processing any part.
This will also helps us in storing the whole bundle i.e. changegroup, bookmarks,
obsmarker etc. in the store.
The logic is currently hidden behind a config option.
Differential Revision: https://phab.mercurial-scm.org/D1585
See hg change 0980979a0d48. Since the interface has changed
more than one time, switch to explicit keywords arguments to avoid
surprises.
Since hgsubversion targets hg >= 3.2.4, drop support for hg 3.0 and 3.1
here.
Upstream change 3d707d50cb3d and d27d1bf3399f now requires certain
`req.earlyoptions` items to be set before entering `dispatch._dispatch`.
Update the test code to be compatible with it. This should be compatible
with upstream changeset 8fbb26.
This effectively conditionalizes 7ac081713920. Some Linux distributions (like
CentOS 7) use really old versions, and the change referenced was causing
exceptions to be thrown.
Even though the deprecation warning says 'since 2.5.0', it wasn't marked as such
in 2.5.1, but is by 2.6.0. This was tested with 2.4.2 and 2.6.0 with
PYTHONWARNINGS=::DeprecationWarning, and both paths were exercized.
The upstream has made the bundle parts non-seekable to reduce memory usage.
So we need to copy a bundle part if we need to process it later.
Also, fix an apparent issue where `part` should be `scratchbookpart`.
Differential Revision: https://phab.mercurial-scm.org/D1572
Upstream 02845f74 enhanced command line parsing, with a side effect of
exposing `--config` flags to hooks. Let's change the test to not use
`update --config ...` to be compatible.
Summary:
There is no logic in place right now to delete the files with suffix
'_old' which are created solely for debugging purposes in addremotefilelognode
method. There is no need to keep these files around if a corresponding file
without the '_old' doesn't exist. Therefore, this commit removes such files if
they are no longer required. This also allows for more robust deletion of the
directories during the clean up because sometimes they contain only the files
with '_old' suffix.
Test Plan:
- Ran all the tests.
- Tested that the files with '_old' suffix are removed during `hg repack
--incremental` on a sample repo.
Reviewers: #fbhgext, quark
Reviewed By: #fbhgext, quark
Subscribers: quark
Differential Revision: https://phab.mercurial-scm.org/D1569
Summary:
This commit just refactors to use a method meant for the same
purpose. Thanks to @quark for discovering this method!
Test Plan: Ran all the tests.
Reviewers: #fbhgext, quark
Reviewed By: #fbhgext, quark
Subscribers: quark
Differential Revision: https://phab.mercurial-scm.org/D1571
Instead of wrapping changectx constructor, there is a cleaner way to change
the hidden error message. changectx is constructed frequently so this change
also has perf benefit. It saves about 5ms when running `sl` on fbsource and
it also makes traceprof able to expand changectx.__init__ call graph.
Differential Revision: https://phab.mercurial-scm.org/D1445
We don't actually need to store fileflags='', which uses a lot of memory
because there are many files. Instead, just treat missing fileflags as empty.
Also, no need to call manifest.setflag() for every file -- call
manifest.flags() for every file, then setflag() only if needed.
Differential Revision: https://phab.mercurial-scm.org/D1315
The flock function supports windows and is cleaner. Let's delete the fcntllock
class and migrate consumers to the flock function.
Differential Revision: https://phab.mercurial-scm.org/D1567
We want to use this flock function in another place, so let's support non
blocking waits. We also want it to throw the appropriate LockHeld in that
situation so the behavior is consistent with a normal Mercurial lock.
Differential Revision: https://phab.mercurial-scm.org/D1565
Summary:
The `cleanup` method right now does not clean up all the directories that are
empty. This leads to a situation where we can have a large number of empty
directories for the next repack which is unnecessary. This commit removes all
the empty directories in the cache directory during the clean up.
Test Plan:
- Ran all the tests.
- Removed about 220k empty directories from the local cache. Running time of
`hg repack --incremental` went from 3-4s every time to around 5s for the
first time (because of deleting the empty directories) and ~200 milliseconds
every subsequent time. Note that there was nothing to repack during the
repack operation.
Reviewers: #fbhgext, quark
Reviewed By: #fbhgext, quark
Subscribers: quark
Differential Revision: https://phab.mercurial-scm.org/D1561
The patch that tried to reuse a connection during pull was actually fairly
broken. For one, if multiple peers were created during a pull it would reuse the
same peer, even if the other peer was still in use. Another issue is that hg
pull closes its peer when it's finished, which would return a bad connection
into the pool.
Let's just revert this entirely for now, since there's not an easy way to do
this without refactoring in core.
Abnormally huge packfiles (20 GB+) are unwanted and can cause more problems
than they solve. Let's add a config to simply delete them at the beginning of
the repack proces.
Differential Revision: https://phab.mercurial-scm.org/D1523
