Summary:
This hardcodes several perftweaks configs that have been enabled for major FB
repos for months:
[perftweaks]
disablebranchcache = True
disablebranchcache2 = True
disableresolvingbranches = True
disableupdatebranchcacheoncommit = True
Practically, this means the branchmap is now just `{'default': heads}`. (i.e.
there are no named branches other than `default`), and branchcache is removed
(i.e. `.hg/cache` does not exist without clindex or in-repo tags).
This diff only makes easy-to-verify logic changes by assuming the configs and
removing dead code. Things can be further cleaned up. They will be done by
upcoming changes.
Most test changes are due to the fact that `.hg/cache` is no longer created.
Reviewed By: singhsrb
Differential Revision: D14179858
fbshipit-source-id: 479f7427168eb1d9614a973e273a229e50f5620a
Summary:
This makes tests closer to production setup and removes a bunch of "saved
backup bundle to ..." messages.
With D9236657, this should not hurt server-side performance.
Unfortunately a lot tests cannot be migrated easily, mostly because revision
numbers are used. They are left with a TODO.
Reviewed By: DurhamG
Differential Revision: D9237694
fbshipit-source-id: c993fce18f07aba09f6d70964e248af8d501575a
Summary:
When you get an error, let's print the number of conflicts in each file. This will give the user some sense of how much work they have to do.
The code change is entirely in `filemerge.py`, and `tests/test-merge-conflict-count.t` adds a new test.
Reviewed By: quark-zju
Differential Revision: D9815243
fbshipit-source-id: 1b73a1db293902ac7242997a7d6ae09478344068
Summary:
Like D9323267, hg tests commonly reinvent common aliases to render the DAG, and they often differ very slightly. This makes adding a test require more boilerplate, and reading a test in a foreign new area slightly more overhead.
Let's standardize these to reduce the copypasta.
It's necessary to define this as a shell function instead of an hgrc alias to prevent tests that list aliases from printing it. Plus that enforces a nice separation of test/stdlib logic.
Bookmarks and branches are easy enough to add since they're empty if not used. A good number added `{phase}` -- I renamed this to `tglogp`.
Reviewed By: quark-zju
Differential Revision: D9347072
fbshipit-source-id: 6aac7de3e65d2295a7ebecd2ab30901709af3ff1
Summary:
A future diff will make us depend on repo.close() being called to
finalize the writing of some tree data. It turns out we didn't call close on the
destination repo during a clone, which caused some tests to change. Let's call
close on that repo to prevent a test change in the next diff.
Reviewed By: phillco
Differential Revision: D8670680
fbshipit-source-id: ad63ccfc51ec4c3de2dad494fdb1ae8f6ba8e40a
Summary:
Since we have `debugbuilddag` as a builtin command, and `drawdag` is
pretty useful for writing clean test cases. Let's inline it. This would
simplify many tests since `drawdag=..." is no longer needed.
Reviewed By: DurhamG
Differential Revision: D6827925
fbshipit-source-id: b749f57fd5c55b2cb000dd2da2c2f9bc2386a519
Summary: Also change the internal API so it no longer accepts the "heads" argument.
Reviewed By: ryanmce
Differential Revision: D6745865
fbshipit-source-id: 368742be49b192f7630421003552d0a10eb0b76d
# skip-blame because this was mechanically rewritten the following script. I
ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t
ones appear to be, and they run without addition failures on both Windows and
Linux.
import argparse
import os
import re
ap = argparse.ArgumentParser()
ap.add_argument('path', nargs='+')
opts = ap.parse_args()
globre = re.compile(r'^(.*) \(glob\)(.*)$')
for p in opts.path:
tmp = p + '.tmp'
with open(p, 'rb') as src, open(tmp, 'wb') as dst:
for line in src:
m = globre.match(line)
if not m or '$LOCALIP' in line or '*' in line:
dst.write(line)
continue
if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'):
dst.write(line)
continue
dst.write(m.group(1) + m.group(2) + '\n')
os.unlink(p)
os.rename(tmp, p)
This is just a very simple start, but verifies some of the basic cases of an
in-memory rebase.
Differential Revision: https://phab.mercurial-scm.org/D1652
We replace 'experimental.stabilization=all' by 'experimental.evolution=true'
as we will extract individual config in their own config in later patches.
Differential Revision: https://phab.mercurial-scm.org/D1148
Upon pull or unbundle, we display a message with the range of new revisions
fetched. This revision range could readily be used after a pull to look out
what's new with 'hg log'. The algorithm takes care of filtering "obsolete"
revisions that might be present in transaction's "changes" but should not be
displayed to the end user.
These states will be removed to make the code cleaner and more robust.
Remove their messages first to make review easier.
Differential Revision: https://phab.mercurial-scm.org/D25
evolution* config has been rewritten in stabilization* in the previous patch,
update tests file to use the new names.
Differential Revision: https://phab.mercurial-scm.org/D249
morestatus extension in fbext use to show more context about the state of the
repo like the repository is in a unfinished merge state, or a rebase is going
on, or histedit is going on, listing the files which need to be resolved and
also suggesting ways to handle the situation.
This patch moves the extension directly to core by plugging it into the
--verbose flag of the status command. So now if you are in any unfinished state
and you do hg status -v, it will show you details and help related to the state.
The extension in fbext also shows context about unfinished update state
which is not ported to core as that plug in hooks to update command which need
to be tackled somewhat differently.
The following configuration will turn the behaviour on by default
[commands]
status.verbose = 1
You can also skip considering some states like bisect as follows:
[commands]
status.skipstates=bisect
This patch also adds test for the feature.
.. feature::
``hg status -v`` can now show unfinished state. For example, when in
an unfinished rebase state, ``hg status -v`` might show::
# The repository is in an unfinished *rebase* state.
# No unresolved merge conflicts.
# To continue: hg rebase --continue
# To abort: hg rebase --abort
Differential Revision: https://phab.mercurial-scm.org/D219
This patch migrates rebase to use scmutil.cleanupnodes API. It simplifies
the code and makes rebase code reusable inside a transaction.
This is a BC because the backup file is no longer strip-backup/*-backup.hg,
but strip-backup/*-rebase.hg. The latter looks more reasonable since the
directory name is "strip-backup" so there is no need to repeat "backup".
I think the backup file name change is probably fine as a BC, since we have
changed it before (2e51c9a7a08f) and didn't get complains. The end result
of this series will be a much more consistent and unified backup names:
command | old backup file suffix | new backup file suffix
-------------------------------------------------------------------
amend | amend-backup.hg | amend.hg
histedit | backup.hg (could be 2 files) | histedit.hg (single file)
rebase | backup.hg | rebase.hg
strip | backup.hg | backup.hg
(note: backup files are under .hg/strip-backup)
It also fixes issue5606 as a side effect because the new "delayedstrip" code
path will carefully examine nodes (safestriproots) to make sure orphaned
changesets won't get stripped by accident.
Some warning messages are changed to the new "warning: orphaned descendants
detected, not stripping HASHES", which provides more information about
exactly what changesets are left behind.
Another minor behavior change is when there is an obsoleted changeset with a
successor in the destination branch, bookmarks pointing to that obsoleted
changeset will not be moved. I have commented in test-rebase-obsolete.t
explaining why that is more desirable.
Having a single transaction for rebase means the whole transaction gets rolled back
on error. To work around this a small hack has been added to detect merge
conflict and commit the work done so far before exiting. This hack works because
there is nothing transaction related going on during the merge phase.
However, if a hook blocks the rebase to create a changeset, it is too late to commit the
work done in the transaction before the problematic changeset was created. This
leads to the whole rebase so far being rolled back. Losing merge resolution and
other work in the process. (note: rebase state will be fully lost too).
Since issue5610 is a pretty serious regression and the next stable release is a
couple day away, we are taking the backout route until we can figure out
something better to do.
Turning obsstore and allowunstable on, rebase will skip the
"can't remove original changesets with unrebased descendants" check. Then
rebase could be interrupted (merge conflict), and the user has a chance to
turn off obsstore. If rebase continues, the current code may strip
irrelevant commits (in the test case added, "C" got stripped unexpectedly).
The test case reproduces issue5606. It will be fixed by the "multidest"
rebase refactoring being reviewed. The test case itself is relatively
separate from the rebase refactoring, therefore sent separately hoping to
reduce the number of patches of the main rebase series.
Before this patch, unbundling a stripped changeset would make it a
draft (unless the parent was secret). This meant that one would lose
phase information when stripping and unbundling secret changesets. The
same thing was true for public changesets. While stripping public
changesets is generally rare, it's done frequently by e.g. the
narrowhg extension.
We also include the phases in the temporary bundle, just in case
stripping were to fail after that point, so the user can still restore
the repo including phase information. Before this patch, the phases
were left untouched during the bundling and unbundling of the
temporary bundle. Only at the end of the transaction would
phasecache.filterunknown() be called to remove phase roots that were
no longer valid. We now need to call that also after the first
stripping, i.e. before applying the temporary bundle. Otherwise
unbundling the temporary bundle will cause a read of the phase cache
which has stripped changesets in the cache and that fails.
Like with obsmarkers, we unconditionally include the phases in the
bundle when stripping (when using bundle2, such as when generaldelta
is enabled). The reason for doing that for strip but not for bundle is
that strip bundles are not meant to be shared outside the repo, so we
don't care as much about compatibility.
Now that we garantee that branchmap cache is updated at the end of the
transaction we can drop this update. This removes a problematic case with
nested transaction where the new cache could be written on disk before the
transaction is finished (and even roll-backed)
Such premature cache write was visible in the following test:
* tests/test-acl.t
* tests/test-rebase-conflicts.t
In addition, running the cache update later means having more date about the
state of the repository (in particular: phases). So we can generate caches with
more information. This creates harmless changes to the following tests:
* tests/test-hardlinks-whitelisted.t
* tests/test-hardlinks.t
* tests/test-phases.t
* tests/test-tags.t
* tests/test-inherit-mode.t
We are about to remove the branchmap cache update in changegroup application.
There is a debug message alongside this update that we do not want to loose. We
move the message beforehand to simplify the test update in the next changeset.
The message move is quite noisy and isolating that noise is useful.
Most tests update are just line reordering since the message is issued at a
later point during the transaction.
After this changes, the message is displayed in more case since local commit
creation also issue it.
Previously, rebasing would open several transaction over the course of rebasing
several commits. Opening a transaction can have notable overhead (like copying
the dirstate) which can add up when rebasing many commits.
This patch adds a single large transaction around the actual commit rebase
operation, with a catch for intervention which serializes the current state if
we need to drop back to the terminal for user intervention. Amazingly, almost
all the tests seem to pass.
On large repos with large working copies, this can speed up rebasing 7 commits
by 25%. I'd expect the percentage to be a bit larger for rebasing even more
commits.
There are minor test changes because we're rolling back the entire transaction
during unexpected exceptions instead of just stopping mid-rebase, so there's no
more backup bundle. It also leave an unknown file in the working copy, since our
clean up 'hg update' doesn't delete unknown files.
Previously, if .hg/rebasestate existed but .hg/last-message.txt was missing, 'hg
rebase --abort' would say there's no rebase in progress but 'hg checkout foo'
would say 'abort: rebase in progress'. It turns out loading the collapse message
will throw a "no rebase in progress" error if the file doesn't exist, even
though .hg/rebasestate obviously indicates a rebase is in progress.
The fix is to only throw an exception if we're trying to --continue, and to just
eat the issues if we're doing --abort.
This issue is exposed by us writing the rebase state earlier in the process.
This will be used by later patches to ensure the user can appropriately 'hg
rebase --abort' if there's a crash before the first the first commit has
finished rebasing. Tests cover all of this. The only negative affect is we now
require a hg rebase --abort in a very specific exception case, as shown in the
test.
The way default marker template was defined before this patch,
the spacing before dash in conflict markes was dependent on
whether changeset is a tip one or not. This is a relevant part
of template:
'{ifeq(tags, "tip", "", "{tags} "}'
If revision is a tip revision with no other tags, this would
resolve to an empty string, but for revisions which are not tip
and don't have any other tags, this would resolve to a single
space string. In the end this causes weirdnesses like the ones
you can see in the affected tests.
This is a not a big deal, but double spacing may be visually
less pleasant.
Please note that test changes where commit hashes change are
the result of marking files as resolved without removing markers.
When there are unresolved merge conflicts, there is no reason
to make the user wait for rebase to process all of the
already rebased commits just to complain that it cannot do
anything. Abort early.
Revlog can now be configured to store full snapshot only. This is used on the
changelog. However, the changegroup packing was still recomputing deltas to be
sent over the wire.
We now just reuse the full snapshot directly in this case, skipping delta
computation. This provides use with a large speed up(-30%):
# perfchangegroupchangelog on mercurial
! wall 2.010326 comb 2.020000 user 2.000000 sys 0.020000 (best of 5)
! wall 1.382039 comb 1.380000 user 1.370000 sys 0.010000 (best of 8)
# perfchangegroupchangelog on pypy
! wall 5.792589 comb 5.780000 user 5.780000 sys 0.000000 (best of 3)
! wall 3.911158 comb 3.920000 user 3.900000 sys 0.020000 (best of 3)
# perfchangegroupchangelog on mozilla central
! wall 20.683727 comb 20.680000 user 20.630000 sys 0.050000 (best of 3)
! wall 14.190204 comb 14.190000 user 14.150000 sys 0.040000 (best of 3)
Many tests have to be updated because of the change in bundle content. All
theses update have been verified. Because diffing changelog was not very
valuable, the resulting bundle have similar size (often a bit smaller):
# full bundle of mozilla central
with delta: 1142740533B
without delta: 1142173300B
So this is a win all over the board.
The algorithm of _checkcopies can only walk backwards in the DAG, never
forward. Because of this, the two _checkcopies patches need to run from
their respective endpoints to the TCA to cover the entire subgraph where
the merge is being performed. However, detection of files new in both
endpoints, as well as directory rename detection, need to run with respect
to the merge base, so we need lists of new files both from the TCA's and
the merge base's viewpoint to correctly detect renames in a graft-like
merge scenario.
(Series reworked by Pierre-Yves David)
The code used self._rbcnamescount as if it was the length of self._names ...
but actually it is just the number of good entries on disk. This caused the
cache to be populated inefficiently. In some cases very inefficiently.
Instead of checking the length before lookup, just try a lookup in self._names
- that is also in most cases faster.
Comments and debug messages are tweaked to help understanding the issue
and the fix.
The bundle2 changegroup part has an advisory param saying how many
changesets are in the part. Before this patch, we were setting
this part when generating bundle2 parts via the wire protocol but
not when generating local bundle2 files.
A side effect of not setting the changeset count part is that progress
bars don't work when applying changesets. As the tests show, this
impacted clone bundles, shelve, backup bundles, `hg unbundle`, and
anything touching bundle2 files.
This patch adds a backdoor to allow us to pass state from
changegroup generation into the unbundler. We store the number
of changesets in the changegroup in this state and use it to
populate the aforementioned advisory part parameter when generating
the bundle2 bundle.
I concede that I'm not thrilled by how state is being passed in
changegroup.py (it feels a bit hacky). I would love to overhaul the
rather confusing set of functions in changegroup.py with something that
passes rich objects around instead of e.g. low-level generators.
However, given the code freeze for 3.9 is imminent, I'd rather not
undertake this endeavor right now. This feels like the easiest way
to get the parameter added to the changegroup part.
Previously, two changes that were nearly, but not quite, identical would result
in large merge conflict regions that looked very similar, and were thus very
confusing to users, and lead people used to other source control systems to
claim that "mercurial's merge algorithms suck". In the relatively common case
of a new file being introduced in two branches with very slight modifications,
the old behavior would show the entire file as a conflict, and it would be very
difficult for a user to determine what was going on.
In the past, mercurial attempted to solve this with a "very smart" algorithm
that would find all common lines, but this has significant problems as
described in 3d22d20aa950.
Instead, we use a "very dumb" algorithm introduced in the previous patch that
simply matches lines at the periphery of conflict regions. This minimizes most
conflict regions well, though there may still be some degenerate edge cases,
like small modification to the beginning and end of a large file.
The current output for a failed merge with conflict markers looks something like:
merging foo
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
merging bar
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
We're going to change the way merges are done to perform all premerges before
all merges, so that the output above would look like:
merging foo
merging bar
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
The 'warning: conflicts during merge' line has no context, so is pretty
confusing.
This patch will change the future output to:
merging foo
merging bar
warning: conflicts while merging foo! (edit, then use 'hg resolve --mark')
warning: conflicts while merging bar! (edit, then use 'hg resolve --mark')
The hint on how to resolve the conflicts makes this a bit unwieldy, but solving
that is tricky because we already hint that people run 'hg resolve' to retry
unresolved merges. The 'hg resolve --mark' mostly applies to conflict marker
based resolution.
The computation of roots was buggy, any ancestor of a bundled merge which was
also a descendant of the parents of a bundled revision were included as part of
the bundle. We fix it and add a test for strip (which revealed the problem).
Check the test for a practical usecase.
Previously we would only actually write the revbranchcache to disk if we were in
the middle of a write operation (like commit). Now we will also write it during
any read operation. The cache knows how to invalidate itself, so it shouldn't
become corrupt if multiple writers try at once (and the write-on-read
behavior/risk is the same as all our other caches).
The cache files are usually append only but will automatically be truncated and
recover in exceptional situations. Add a debug notice when such exceptional
situations are encountered.
Previously, a backup bundle could overwrite an existing bundle and cause user
data loss. For instance, if you have A<-B<-C and strip B, it produces backup
bundle B-backup.hg. If you then hg pull -r B B-backup.hg and strip it again, it
overwrites the existing B-backup.hg and C is lost.
The fix is to add a hash of all the nodes inside that bundle to the filename.
Fixed up existing tests and added a new test in test-strip.t
commitctx already showed notes with filenames but didn't provide any context.
It is just as relevant to know when manifest or changelog is committed.
So, in addition to filenames, also show headlines 'committing files:',
'committing manifest' and 'committing changelog'.
This patch makes bundrepo retract the phase boundary for new commits to 'draft'
status, which is consistent with the behavior of 'hg unbundle'. The old
behavior was for commits to appear with the same phase as their nearest
ancestor in the base repository.
This affects several classes of operation:
* Inspecting a bundle with the -B flag
* Treating a bundle file as a peer (old: everything public, new: everything draft)
* Incoming command (neither old or new behavior is sensible -- fixed in next patch)
When the progress extension is not enabled, each call to 'ui.progress' used to
issue a debug message. This results is a very verbose output and often redundant
in tests. Dropping it makes tests less volatile to factor they do not meant to
test.
We had to alter the sed trick in 'test-rename-merge2.t'. Sed is used to drop all
output from a certain point and hidding the progress output remove its anchor.
So we anchor on something else.
Show status messages while rebasing, similar to what graft do:
rebasing 12:2647734878ef "fork" (tip)
This gives more context for the user when resolving conflicts.
Globbing the hash made it harder to maintain tests with run-tests -i when it
was so far by the generated test output.
The hashes are stable and we just need to add a (glob).
