Before, a --clean update with largefiles would use the "optimization" that it
didn't read hashes from standin files before and after the update. Instead of
trusting the content of the standin files, it would rehash all the actual
largefiles that lfdirstate reported clean and update the standins that didn't
have the expected content. It could thus in some "impossible" situations
automatically recover from some "largefile got out sync with its standin"
issues (even there apparently still were weird corner cases where it could
fail). This extra checking is similar to what core --clean intentionally do
not do, and it made update --clean unbearable slow.
Usually in core Mercurial, --clean will rely on the dirstate to find the files
it should update. (It is thus intentionally possible (when trying to trick the
system or if there should be bugs) to end up in situations where --clean not
will restore the working directory content correctly.) Checking every file when
we "know" it is ok is however not an option - that would be too slow.
Instead, trust the content of the standin files. Use the same logic for --clean
as for linear updates and trust the dirstate and that our "logic" will keep
them in sync. It is much cheaper to just rehash the largefiles reported dirty
by a status walk and read all standins than to hash largefiles.
Most of the changes are just a change of indentation now when the different
kinds of updates no longer are handled that differently. Standins for added
files are however only written when doing a normal update, while deleted and
removed files only will be updated for --clean updates.
By moving the cd/dc prompts out of calculateupdates(), we let
largefiles' overridecalculateupdates() so the unresolved values
(i.e. 'cd' or 'dc' rather than 'g', 'r', 'a' and missing). This allows
overridecalculateupdates() to ask the user whether to keep the normal
file or the largefile before the user gets the cd/dc prompt. Whichever
answer the user gives, we make overridecalculateupdates() replace 'cd'
or 'dc' action, saving the user one annoying (and less clear)
question.
When merging and the remote has turned a normal file into a largefile
and the user chooses to keep the local largefile, we use the 'r'
action for the remote largefile standin. This is wrong, since that
file does not exist in the parent of the working copy. Use 'k', which
does nothing but debug logging, instead.
When merging and the remote has turned a largefile into a normal file
and the user chooses to keep the local largefile, we use the 'r'
action for the remote normal file. This is wrong, since that file does
not exist in the parent of the working copy. Use 'k', which does
nothing but debug logging, instead.
Before, merging would in some cases ask "wrong" questions about
"changed/deleted" conflicts ... and even do it before the resolve phase where
they can be postponed, re"resolved" or answered in bulk operations.
Instead, check that the content of the changed file really did change.
Reading and comparing file content is expensive and should be avoided before
the resolve phase. Prompting the user is however even more expensive. Checking
the content here is thus better.
The 'f in ancestors[0]' should not be necessary but is included to be extra
safe.
Use suffix -same for cases where file changed but content is the same - that is
the case where manifestmerge doesn't detect that a file is unchanged.
(The suffix -id is already used for cases where the file didn't change - that
is the trivial case where manifestmerge detects that the file is unchanged.)
These new tests are good but the results are bad. There shouldn't be any merge
conflicts or prompts when one side didn't change.
The tests often set ui.interactive to control normally interactive prompts from
stdin. That gave an output where it was non-obvious what prompts got which
which response, and the output lacked the newline users would see after input.
Instead, if the input not is a tty, write the selection and a newline.
Before this patch, largefiles in the working directory aren't updated
correctly, if rebase is aborted by conflict. This prevents users from
viewing appropriate largefiles while resolving conflicts.
While rebase, largefiles in the working directory are updated only at
successful committing in the special code path of
"lfilesrepo.commit()".
To update largefiles even if rebase is aborted by conflict, this patch
centralizes the logic of updating largefiles in the working directory
into the "mergeupdate" wrapping "merge.update".
This is a temporary way to fix with less changes. For fundamental
resolution of this kind of problems in the future, largefiles in the
working directory should be updated with other (normal) files
simultaneously while "merge.update" execution: maybe by hooking
"applyupdates".
"Action list based updating" introduced by hooking "applyupdates" will
also improve performance of updating, because it automatically
decreases target files to be checked.
Just after this patch, there are some improper things in "Case 0" code
path of "lfilesrepo.commit()":
- "updatelfiles" invocation is redundant for rebase
- detailed comment doesn't meet to rebase behavior
These will be resolved after the subsequent patch for transplant,
because this code path is shared with transplant.
Even though replacing "merge.update" in rebase extension by "hg.merge"
can also avoid this problem, this patch chooses centralizing the logic
into "mergeupdate", because:
- "merge.update" invocation in rebase extension can't be directly
replaced by "hg.merge", because:
- rebase requires some extra arguments, which "hg.merge" doesn't
take (e.g. "ancestor")
- rebase doesn't require statistics information forcibly displayed
in "hg.merge"
- introducing "mergeupdate" can resolve also problem of some other
code paths directly using "merge.update"
largefiles in the working directory aren't updated regardless of
the result of commands below, before this patch:
- backout (for revisions other than the parent revision of the
working directory without "--merge")
- graft
- histedit (for revisions other than the parent of the working
directory
When "partial" is specified, "merge.update" doesn't update dirstate
entries for standins, even though standins themselves are updated.
In this case, "normallookup" should be used to mark largefiles as
"possibly dirty" forcibly, because applying "normal" on lfdirstate
treats them as "clean" unexpectedly.
This is reason why "normallookup=partial" is specified for
"lfcommands.updatelfiles".
This patch doesn't test "hg rebase --continue", because it doesn't
work correctly if largefiles in the working directory are modified
manually while resolving conflicts. This will be fixed in the next
step of refactoring for largefiles.
All changes of tests/*.t files other than test-largefiles-update.t in
this patch come from invoking "updatelfiles" not after but before
statistics output of "hg.update", "hg.clean" and "hg.merge".
Previously, the directory '.hg/largefiles' would always be created if it didn't
exist when the lfdirstate was opened. If there were no standin files, no
dirstate file would be created in the directory. The end result was that
enabling the largefiles extension globally, but not explicitly adding a
largefile would result in the repository eventually sprouting this directory.
Creation of this directory effectively changes readonly operations like summary
and status into operations that require write access. Without write access,
commands that would succeed without the extension loaded would abort with a
surprising error when the extension is loaded, but not actively used:
$ hg sum -R /tmp/thg --config extensions.largefiles=
parent: 16541:00dc703d5aed
repowidget: specify incoming bundle by plain file path to avoid url parsing
branch: default
abort: Permission denied: '/tmp/thg/.hg/largefiles'
This change is simpler than changing the callers of openlfdirstate() to use the
'create' parameter that was introduced in 74522122b97d, and probably how that
should have been implemented in the first place.
Prompts like
foo has been turned into a largefile
use (l)argefile or keep as (n)ormal file?
was not as clear as the usual prompts that use 'remote' or 'local' to explain
what happened on which side ... especially not when used to the normal prompts.
"as" could also indicate that it would be possible to take the content of the
largefile and somehow put it into the normal file. It could make it more clear
that it was a choice between one side or the other.
For consistency we will now phrase it like:
remote turned local normal file f into a largefile
use (l)argefile or keep (n)ormal file?
bc94b1afd3c9 fixed one problem with update and added a test case for it. The
test coverage was thus insufficient before that.
To make sure we have good test coverage in this area we add systematic testing
of all cases of merges that may or may not change normal files to largefiles or
vice versa.
The tests shows some annoying extra merge prompts in some cases, but these
prompts are hard to avoid and they are now "safe" - they do not leave the
system in a confused inconsistent state.
We used to get like:
$ hg up -r 2
foo has been turned into a normal file
keep as (l)argefile or use (n)ormal file? l
getting changed largefiles
0 largefiles updated, 0 removed
0 files updated, 0 files merged, 2 files removed, 0 files unresolved
$ cat foo
cat: foo: No such file or directory
[1]
- which both asked the wrong question and did the wrong thing.
Instead, skip this conflict resolution when the local conflicting file has been
scheduled for removal and there thus is no conflict.
Looking for a (potentially empty) directory was not reliable - both because it
is a reasonable assumption that empty directories can be removed and because it
wasn't created in all cases ... such as when pulling to an existing repository.
Override updaterepo() instead of individual methods that may not be called for
each subrepo. Add test.
Based on patch from Matt Harbison.
Changes the order of update-related messages (now largefiles comes before the
global status).
Many tests didn't change back from subdirectories at the end of the tests ...
and they don't have to. The missing 'cd ..' could always be added when another
test case is added to the test file.
This change do that tests (99.5%) consistently end up in $TESTDIR where they
started, thus making it simpler to extend them or move them around.
The largefiles extension prevents users from adding a normal file
named 'foo' if there is already a largefile with the same name.
However, there was a loop-hole: when merging, it was possible to bring
in a normal file named 'foo' while also having a '.hglf/foo' file.
This patch fixes this by extending the manifest merge to deal with
these kinds of conflicts. If there is a normal file 'foo' in the
working copy, and the other parent brings in a '.hglf/foo' file, then
the user will be prompted to keep the normal file or the largefile.
Likewise for the symmetric case where a normal file is brought in via
the second parent. The prompt looks like this:
$ hg merge
foo has been turned into a largefile
use (l)argefile or keep as (n)ormal file?
After the merge, either the '.hglf/foo' file or the 'foo' file will
have been deleted. This would cause status to return output like:
$ hg status
M foo
R foo
To fix this, the lfiles_repo.status method is changed so that a
removed normal file isn't shown if there is largefile with the same
name, and vice versa for largefiles.
