Previously, rebasing would open several transaction over the course of rebasing
several commits. Opening a transaction can have notable overhead (like copying
the dirstate) which can add up when rebasing many commits.
This patch adds a single large transaction around the actual commit rebase
operation, with a catch for intervention which serializes the current state if
we need to drop back to the terminal for user intervention. Amazingly, almost
all the tests seem to pass.
On large repos with large working copies, this can speed up rebasing 7 commits
by 25%. I'd expect the percentage to be a bit larger for rebasing even more
commits.
There are minor test changes because we're rolling back the entire transaction
during unexpected exceptions instead of just stopping mid-rebase, so there's no
more backup bundle. It also leave an unknown file in the working copy, since our
clean up 'hg update' doesn't delete unknown files.
Previously, if .hg/rebasestate existed but .hg/last-message.txt was missing, 'hg
rebase --abort' would say there's no rebase in progress but 'hg checkout foo'
would say 'abort: rebase in progress'. It turns out loading the collapse message
will throw a "no rebase in progress" error if the file doesn't exist, even
though .hg/rebasestate obviously indicates a rebase is in progress.
The fix is to only throw an exception if we're trying to --continue, and to just
eat the issues if we're doing --abort.
This issue is exposed by us writing the rebase state earlier in the process.
This will be used by later patches to ensure the user can appropriately 'hg
rebase --abort' if there's a crash before the first the first commit has
finished rebasing. Tests cover all of this. The only negative affect is we now
require a hg rebase --abort in a very specific exception case, as shown in the
test.
The way default marker template was defined before this patch,
the spacing before dash in conflict markes was dependent on
whether changeset is a tip one or not. This is a relevant part
of template:
'{ifeq(tags, "tip", "", "{tags} "}'
If revision is a tip revision with no other tags, this would
resolve to an empty string, but for revisions which are not tip
and don't have any other tags, this would resolve to a single
space string. In the end this causes weirdnesses like the ones
you can see in the affected tests.
This is a not a big deal, but double spacing may be visually
less pleasant.
Please note that test changes where commit hashes change are
the result of marking files as resolved without removing markers.
When there are unresolved merge conflicts, there is no reason
to make the user wait for rebase to process all of the
already rebased commits just to complain that it cannot do
anything. Abort early.
Revlog can now be configured to store full snapshot only. This is used on the
changelog. However, the changegroup packing was still recomputing deltas to be
sent over the wire.
We now just reuse the full snapshot directly in this case, skipping delta
computation. This provides use with a large speed up(-30%):
# perfchangegroupchangelog on mercurial
! wall 2.010326 comb 2.020000 user 2.000000 sys 0.020000 (best of 5)
! wall 1.382039 comb 1.380000 user 1.370000 sys 0.010000 (best of 8)
# perfchangegroupchangelog on pypy
! wall 5.792589 comb 5.780000 user 5.780000 sys 0.000000 (best of 3)
! wall 3.911158 comb 3.920000 user 3.900000 sys 0.020000 (best of 3)
# perfchangegroupchangelog on mozilla central
! wall 20.683727 comb 20.680000 user 20.630000 sys 0.050000 (best of 3)
! wall 14.190204 comb 14.190000 user 14.150000 sys 0.040000 (best of 3)
Many tests have to be updated because of the change in bundle content. All
theses update have been verified. Because diffing changelog was not very
valuable, the resulting bundle have similar size (often a bit smaller):
# full bundle of mozilla central
with delta: 1142740533B
without delta: 1142173300B
So this is a win all over the board.
The algorithm of _checkcopies can only walk backwards in the DAG, never
forward. Because of this, the two _checkcopies patches need to run from
their respective endpoints to the TCA to cover the entire subgraph where
the merge is being performed. However, detection of files new in both
endpoints, as well as directory rename detection, need to run with respect
to the merge base, so we need lists of new files both from the TCA's and
the merge base's viewpoint to correctly detect renames in a graft-like
merge scenario.
(Series reworked by Pierre-Yves David)
The code used self._rbcnamescount as if it was the length of self._names ...
but actually it is just the number of good entries on disk. This caused the
cache to be populated inefficiently. In some cases very inefficiently.
Instead of checking the length before lookup, just try a lookup in self._names
- that is also in most cases faster.
Comments and debug messages are tweaked to help understanding the issue
and the fix.
The bundle2 changegroup part has an advisory param saying how many
changesets are in the part. Before this patch, we were setting
this part when generating bundle2 parts via the wire protocol but
not when generating local bundle2 files.
A side effect of not setting the changeset count part is that progress
bars don't work when applying changesets. As the tests show, this
impacted clone bundles, shelve, backup bundles, `hg unbundle`, and
anything touching bundle2 files.
This patch adds a backdoor to allow us to pass state from
changegroup generation into the unbundler. We store the number
of changesets in the changegroup in this state and use it to
populate the aforementioned advisory part parameter when generating
the bundle2 bundle.
I concede that I'm not thrilled by how state is being passed in
changegroup.py (it feels a bit hacky). I would love to overhaul the
rather confusing set of functions in changegroup.py with something that
passes rich objects around instead of e.g. low-level generators.
However, given the code freeze for 3.9 is imminent, I'd rather not
undertake this endeavor right now. This feels like the easiest way
to get the parameter added to the changegroup part.
Previously, two changes that were nearly, but not quite, identical would result
in large merge conflict regions that looked very similar, and were thus very
confusing to users, and lead people used to other source control systems to
claim that "mercurial's merge algorithms suck". In the relatively common case
of a new file being introduced in two branches with very slight modifications,
the old behavior would show the entire file as a conflict, and it would be very
difficult for a user to determine what was going on.
In the past, mercurial attempted to solve this with a "very smart" algorithm
that would find all common lines, but this has significant problems as
described in 3d22d20aa950.
Instead, we use a "very dumb" algorithm introduced in the previous patch that
simply matches lines at the periphery of conflict regions. This minimizes most
conflict regions well, though there may still be some degenerate edge cases,
like small modification to the beginning and end of a large file.
The current output for a failed merge with conflict markers looks something like:
merging foo
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
merging bar
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
We're going to change the way merges are done to perform all premerges before
all merges, so that the output above would look like:
merging foo
merging bar
warning: conflicts during merge.
merging foo incomplete! (edit conflicts, then use 'hg resolve --mark')
warning: conflicts during merge.
merging bar incomplete! (edit conflicts, then use 'hg resolve --mark')
The 'warning: conflicts during merge' line has no context, so is pretty
confusing.
This patch will change the future output to:
merging foo
merging bar
warning: conflicts while merging foo! (edit, then use 'hg resolve --mark')
warning: conflicts while merging bar! (edit, then use 'hg resolve --mark')
The hint on how to resolve the conflicts makes this a bit unwieldy, but solving
that is tricky because we already hint that people run 'hg resolve' to retry
unresolved merges. The 'hg resolve --mark' mostly applies to conflict marker
based resolution.
The computation of roots was buggy, any ancestor of a bundled merge which was
also a descendant of the parents of a bundled revision were included as part of
the bundle. We fix it and add a test for strip (which revealed the problem).
Check the test for a practical usecase.
Previously we would only actually write the revbranchcache to disk if we were in
the middle of a write operation (like commit). Now we will also write it during
any read operation. The cache knows how to invalidate itself, so it shouldn't
become corrupt if multiple writers try at once (and the write-on-read
behavior/risk is the same as all our other caches).
The cache files are usually append only but will automatically be truncated and
recover in exceptional situations. Add a debug notice when such exceptional
situations are encountered.
Previously, a backup bundle could overwrite an existing bundle and cause user
data loss. For instance, if you have A<-B<-C and strip B, it produces backup
bundle B-backup.hg. If you then hg pull -r B B-backup.hg and strip it again, it
overwrites the existing B-backup.hg and C is lost.
The fix is to add a hash of all the nodes inside that bundle to the filename.
Fixed up existing tests and added a new test in test-strip.t
commitctx already showed notes with filenames but didn't provide any context.
It is just as relevant to know when manifest or changelog is committed.
So, in addition to filenames, also show headlines 'committing files:',
'committing manifest' and 'committing changelog'.
This patch makes bundrepo retract the phase boundary for new commits to 'draft'
status, which is consistent with the behavior of 'hg unbundle'. The old
behavior was for commits to appear with the same phase as their nearest
ancestor in the base repository.
This affects several classes of operation:
* Inspecting a bundle with the -B flag
* Treating a bundle file as a peer (old: everything public, new: everything draft)
* Incoming command (neither old or new behavior is sensible -- fixed in next patch)
When the progress extension is not enabled, each call to 'ui.progress' used to
issue a debug message. This results is a very verbose output and often redundant
in tests. Dropping it makes tests less volatile to factor they do not meant to
test.
We had to alter the sed trick in 'test-rename-merge2.t'. Sed is used to drop all
output from a certain point and hidding the progress output remove its anchor.
So we anchor on something else.
Show status messages while rebasing, similar to what graft do:
rebasing 12:2647734878ef "fork" (tip)
This gives more context for the user when resolving conflicts.
Globbing the hash made it harder to maintain tests with run-tests -i when it
was so far by the generated test output.
The hashes are stable and we just need to add a (glob).
Preparing for action list split-up, making sure the final change don't have any
test changes.
The patch moves debug statements around without really changing anything.
Arguably, it temporarily makes the code worse. The only justification is that
it makes it easier to review the test changes ... and in the end the big change
will not change test output at all.
The changes to test output are due to changes in the ordering of debug output.
That is mainly because we now do the debug logging for files when we actually
process them. Files are also processed in a slightly different but still
correct order. It is now primarily ordered by action type, secondarily by
filename.
The patch introduces some redundancy. Some of it will be removed again, some of
it will in the end help code readability and efficiency. It is possible that we
later on could introduce a "process this action list and do some logging and
progress reporting and apply this function".
The "preserving X for resolve" debug statements will only have single space
indentation. It will no longer have a leading single space indented "f: msg ->
m" message. Having this message double indented would thus no longer make
sense.
The bid actions will temporarily be sorted using a custom sort key that happens
to match the sort order the simplified code will have in the end.
The recently introduced message was:
no unresolved files; you may continue your unfinished operation
This had three problems:
- looks a bit like an error message because it's not saying "we've
just resolved the last file"
- refers to "unfinished operation", which won't be the case with
"update" or "merge"
- introduces semicolons to error messages, which is stylistically
questionable
I've simplified this to:
no more unresolved files
In the future, if we want to prompt someone to continue a particular operation, we should use
a hint style:
no more unresolved files
(use 'hg graft --continue' to finish grafting)
When using resolve, users often have to consult with the output of |hg
resolve -l| to see if any unresolved files remain. This step is tedious
and adds overhead to resolving.
This patch will notify a user if there are no unresolved files remaining
after executing |hg resolve|::
no unresolved files; you may continue your unfinished operation
The patch stops short of telling the user exactly what command should be
executed to continue the unfinished operation. That is because this
information is not currently captured anywhere. This would make a
compelling follow-up feature.
Prior this changeset, rebasing a merge whose first parent was not in
the rebase lead to wrong and highly conflicting merge. See the in-line
comment for details.
Test have been updated with the data provided by the reported.
When a rebase has conflicts and the user uses rebase --continue, the previously
active bookmark was not being made active once again. With this change that
bookmark is made active again, just as if the rebase had never been interrupted.
This changes the rebasestate file format, but should handle old formats correctly.
Since the file is transient, this is even less of a problem.
Adds a test to verify the new behavior. I manually tested continuing rebases
with and without an active bookmark, and with and without being on the bookmark
being rebased.
Many tests didn't change back from subdirectories at the end of the tests ...
and they don't have to. The missing 'cd ..' could always be added when another
test case is added to the test file.
This change do that tests (99.5%) consistently end up in $TESTDIR where they
started, thus making it simpler to extend them or move them around.
