Now that all internal callers pre-compute and set a destination at a higher level
it feels like we can kill this API. This will allow us to simplify this
function. However I feel like this is a bit too central and critical to break
now. I'm adding a devel warning to let extension make catch this in the next
cycle.
File/directory case folding collisions cannot be represented on case folding
systems and have to fail.
To detect this and abort early, utilize that for file/directory collisions, a
sorted list of case folded manifest names will have the colliding directory
right after the file.
(This could perhaps be optimized, but this way of doing it also has
directory/directory case folding in mind ... which however not is handled yet.)
A driver-resolved file is a file that's handled specially by the driver. A
common use case for this state would be autogenerated files, the generation of
which should happen only after all source conflicts are resolved.
This is done with an uppercase letter because older versions of Mercurial will
not know how to treat such files at all.
A 'merge driver' is a coordinator for the overall merge process. It will be
able to control:
- tools for individual files, much like the merge-patterns configuration does
today
- tools that can work across groups of files
- the ordering of file resolution
- resolution of automatically generated files
- adding and removing additional files to and from the dirstate
Since it is a critical part of the merge process, it really is part of the
merge state.
This is a lowercase character (i.e. optional) because ignoring this is fine for
older versions of Mercurial -- however, if there are any files that are
specially treated by the driver, we should abort. That will happen in upcoming
patches.
There is a potential security issue with storing the merge driver in the merge
state. See the inline comments for more details.
For the same reason, we move the bookmark related update logic into the
'destupdate' function. This requires to extend the returns of the function to
include the bookmark that needs to move (more or less) and the bookmark to
activate at the end of the function. See function documentation for details on
this returns.
We perform all that we can non-interactively before prompting the user for input
via their merge tool. This allows for a maximally consistent state when the user
is first prompted.
The test output changes indicate the actual behavior change happening.
The section of code that writes out the version of the file cached in the merge
state should only be run at preresolve time. This is so that if the premerge
keeps around conflict markers, those don't get overwritten before the main
merge.
This means that in ms.resolve we must call merge after calling premerge. This
doesn't yet mean that all premerges happen before any merges -- however, this
does get us closer to our goal.
The output differences are because we recompute the merge tool. The only
user-visible difference caused by this patch is that if the tool is missing
we'll print the warning twice. Not a huge deal, though.
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
We can safely drop this because the very same assignment is enforcement later in
the function. Dropping it will make it simpler to extract the default
destination logic in its own function.
This is another step toward having "default" destination more clear and unified.
Not all the logic is there because some bookmark related computation happened
elsewhere. It will be moved later.
The function is private because as for the other ones, cleanup is needed before
we can proceed.
Resolving other conflicts before merge ones is better because the state before
the merge is as consistent as possible. It will also help with future work
involving automatic resolution of merge conflicts with an external merge
driver.
There are no ordering issues here because it is easy to verify that the same
file is never in both the dg/dm and the m sets.
Explicit 'dirstate.normallookup()' invocation via 'dirtysubstate()' in
'applyupdates()' is useless now, because previous patch fixed the
relevant issue by writing in-memory dirstate changes out at the end of
dirty check.
'dirstate.normallookup()' invocation was introduced by 13fc4cf249d9 to
avoid occasional test failure. This is partial backout of it (added
tests are still left).
Python 2.6 introduced the "except type as instance" syntax, replacing
the "except type, instance" syntax that came before. Python 3 dropped
support for the latter syntax. Since we no longer support Python 2.4 or
2.5, we have no need to continue supporting the "except type, instance".
This patch mass rewrites the exception syntax to be Python 2.6+ and
Python 3 compatible.
This patch was produced by running `2to3 -f except -w -n .`.
There were 2 test failures in 3.4-rc when running test-hook.t with the
largefiles extension enabled. For context, the first is a commit hook:
@@ -618,9 +621,9 @@
$ echo 'update = hg id' >> .hg/hgrc
$ echo bb > a
$ hg ci -ma
- 223eafe2750c tip
+ d3354c4310ed+
$ hg up 0
- cb9a9f314b8b
+ 223eafe2750c+ tip
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
make sure --verbose (and --quiet/--debug etc.) are propagated to the local ui
In both cases, largefiles acquires the wlock before calling into core, which
also acquires the wlock. The first case was fixed in 4100e338a886 by ensuring
the hook only runs after the lock has been fully released. The full release is
important, because that is what writes dirstate to the disk, allowing external
hooks to see the result of the update. This simply changes how the update hook
is called, so that it too is deferred until the lock is finally released.
There are many uses of mergemod.update(), but in terms of commands, it looks
like the following commands take wlock while calling mergemod.update(), and
therefore will now have their hook fired at a later time:
backout, fetch, histedit, qpush, rebase, shelve, transplant
Unlike the others, fetch immediately unlocks after calling update(), so for all
intents and purposes, its hook invocation is not deferred (but the external hook
still sees the proper state).
Previously it was impossible to graft a commit onto it's own parent (i.e. create
a copy of the commit). This is useful when wanting to create a backup of the
commit before continuing to amend it. This patch enables that behavior.
The change to the histedit test is because histedit uses graft to apply commits.
The test in question moves a commit backwards onto an ancestor. Since the graft
logic now more explicitly supports this, it knows to simply accept the incoming
changes (since they are more recent), instead of prompting.
Before this patch, failure of updating subrepos may cause inconsistent
".hgsubstate". For example:
1. dirstate entry for ".hgsubstate" of the parent repo is filled
with valid size/date (via "hg state" or so)
2. "hg update" is invoked at the parent repo
3. ".hgsubstate" of the parent repo is updated on the filesystem as
a part of "g"(et) action in "merge.applyupdates"
4. it is assumed that size/date of ".hgsubstate" on the filesystem
aren't changed from ones at (1)
this is not so difficult condition, because just changing hash
ids (every ids are same in length) in ".hgsubstate" doesn't
change the file size of it
5. "subrepo.submerge()" is invoked to update subrepos
6. failure of updating in one of subrepos raises exception
(e.g. "untracked file differs")
7. "hg update" is aborted without updating dirstate of the parent repo
dirstate entry for ".hgsubstate" still holds size/date at (1)
Then, ".hgsubstate" of the parent repo is treated as "CLEAN"
unexpectedly, because updating ".hgsubstate" at (3) doesn't change
size/date of it on the filesystem: see assumption at (4).
This inconsistent ".hgsubstate" status causes unexpected behavior, for
example:
- "hg revert" forgets to revert ".hgsubstate"
- "hg update" misunderstands that (not yet updated) subrepos diverge
(then, it shows the prompt to confirm user's decision)
To avoid inconsistent ".hgsubstate" status above, this patch marks
".hgsubstate" as possibly dirty before "submerge" invocation.
"normallookup"-ed (= dirty) dirstate should be written out, even if
processing is aborted by failure.
This patch marks ".hgsubstate" as possibly dirty before "submerge",
also when it is removed or merged while merging, for safety. This
should prevent Mercurial from misunderstanding inconsistent
".hgsubstate" as clean.
To satisfy conditions at (1) and (4) above, this patch uses "hg status
--config debug.dirstate.delaywrite=2" (to fill valid size/date into
dirstate) and "touch" (to fix date of the file).
This change touches every module in which repository.wopener was being used, and
changes it for the equivalent repository.wvfs.
It should now be possible to remove localrepo.wopener.
This change touches every module in which repository.opener was being used, and
changes it for the equivalent repository.vfs. This is meant to make it easier
to split the repository.vfs into several separate vfs.
It should now be possible to remove localrepo.opener.
This moves most reading of filelogs out of manifestmerge, making it
easy for a narrow clone extension to filter out or translate unwanted
actions before any filelogs are read. The only call left is inside of
copies.mergecopies(), which can be overridden separately at a lower
level.
We still have one case of a call to _checkunknownfile() in
manifestmerge(): when force=True and branchmerge=True and the remote
side has a file that the local side doesn't. This combination of
arguments is used by 'hg merge --force', but also by rebase and
unshelve. In this scenario, we try to create the file from the
contents from the remote, but if there is already a local untracked
file in place, we merge it instead.
When a directory was renamed and a new untracked file was added in the
new directory and the remote directory added a file by the same name
in the old directory, the local untracked file gets overwritten, as
demonstrated by the broken test case in test-rename-dir-merge.
Fix by checking for unknown files for 'dg' actions too. Since
_checkunknownfile() currently expects the same filename in both
contexts, we need to add a new parameter for the remote filename to
it.
The 'c' and 'dc' actions include creating a file on disk and we need
to check that no conflicting file exists unless force=True. Move two
of the calls to _checkunknownfile() to a single place at the end of
manifestmerge(). This removes some of the reading of filelogs from the
heart of manifestmerge() and collects it in one place close to where
its output (entries in the 'aborts' list) is used.
Note that this removes the unnecessary call to _checkunknownfile()
when force=True in one of the code paths.
_checkunknownfile() reads the filelog of the remote side's file. For
narrow clones, the filelog will not exist for all files and we need a
way to avoid reading them. While it would be easier for the narrow
extension to just override _checkunknownfile() and make it ignore
files outside the narrow clone, it seems cleaner to have
manifestmerge() not care about filelogs (considering its
name).
In order to move the calls to _checkunknownfile() out, we need to be
able to tell in which cases we should check for unknown files. Let's
start by introducing a new action distinct from 'g' for this
purpose. Specifically, the new action will be just like 'g' except
that it will check that for conflicting unknown files first. For now,
just add the new action type and convert it to 'g'.
This simplifies largefiles' overridecalculateupdates(), which no
longer has to do the conversion it started doing in 478d610ca1b0
(largefiles: rewrite merge code using dictionary with entry per file,
2014-12-09).
To keep this patch small, we'll leave the name 'actionbyfile' in
overrides.py. It will be renamed in the next patch.
By moving the conversion from the file->action dict after
_forgetremoved(), we make that method shorter by removing the need for
the confusing 'xactions' variable.
By moving the conversion from the file->action dict after the bid
merge code, bid merge can be simplified a little.
A few tests are affected by this change. Where we used to iterate over
the actions first in order of the action type ('g', 'r', etc.) [1], we
now iterate in order of filename. This difference affects the order of
debug log statements.
[1] And then in the non-deterministic order of files in the manifest
dictionary (the order returned from manifest.diff()).
In the same vein as 478d610ca1b0 (largefiles: rewrite merge code using
dictionary with entry per file, 2014-12-09), rewrite manifestmerge()
itself as dictionary with the filename as key. This will let us
simplify some of the other code in merge.py and eventually drop the
conversion in the largefiles code.
No difference in speed could be detected (well within the noise level
when run in Mozilla repo).
When there are multiple common ancestors, we should check for case
collisions only on the resulting actions after bid merge has run. To
do this, move the code until after bid merge.
Move it past _resolvetrivial() too, since that might update
actions. If the remote changed a file and then reverted the change,
while the local side deleted the file and created a new file with a
name that case-folds like the old file, we should fail before this
patch but not after.
Although the changes to the actions caused by _forgetremoved() should
have no effect on case collisions, move it after that, too, so the
next person reading the code won't have to think about it.
Moving it past these blocks of code takes it to the end of
calculateupdates(), so let's even move it outside of the method, so we
also check collisions in actions produced by extensions overriding the
method.
By moving the cd/dc prompts out of calculateupdates(), we let
largefiles' overridecalculateupdates() so the unresolved values
(i.e. 'cd' or 'dc' rather than 'g', 'r', 'a' and missing). This allows
overridecalculateupdates() to ask the user whether to keep the normal
file or the largefile before the user gets the cd/dc prompt. Whichever
answer the user gives, we make overridecalculateupdates() replace 'cd'
or 'dc' action, saving the user one annoying (and less clear)
question.
We would eventually like to move the resolution of modify/delete and
delete/modify conflicts to the resolve phase. However, we don't want
to move the checks for identical content that were added in
99b29d2bd5ed (merge: before cd/dc prompt, check that changed side
really changed, 2014-12-01). Let's instead move these out to a new
_resolvetrivial() function that processes the actions from
manifestmerge() and replaces any false cd/dc conflicts. The function
will also provide a natural place for us to later add code for
resolving false 'm' conflicts.
As preparation for making 'dr' and 'rd' actions no longer actions,
move the reporting from applyupdates() to its caller update(). This
way we won't have to pass additonal arguments to applyupdates() when
they are no longer actions. Also, the warnings are equally unrelated
to applyupdates() as they are to recordupdates(), as they don't result
in any changes to either the working copy or the dirstate.
See earlier patch for additional motivation.
