Before this patch, if both "--message" and "--collapse" are specified
for "hg rebase", "rebaes.normal" is used as "editform" unexpectedly.
Unlike patches before and after in this series for improvement, this
is bug fix patch.
In all the remaining cases the comprehension variable is used for the same
thing as a previous loop variable.
This will mute some pyflakes "list comprehension redefines" warnings.
_ is usually used for i18n markup but we also used it for I-don't-care
variables.
Instead, name don't-care variables in a slightly descriptive way but use the _
prefix to designate unused variable.
This will mute some pyflakes "import '_' ... shadowed by loop variable"
warnings.
Before this patch, linear merging of modified largefiles causes
an unexpected result, if (1) largefile collides with same-name normal one
in the target revision and (2) "local" largefile is chosen, even
though branch merging between such revisions works correctly.
Expected result of such linear merging is marking the largefile as
(re-)"added", but the actual result is marking it as "modified".
The standin of modified "local largefile" is not changed by linear
merging, and updating/merging update lfdirstate entries only for
largefiles of which standins are changed.
This patch adds the code path to update lfdirstate only for largefiles
of which standins are not changed.
In this case, "synclfdirstate" should be invoked with True as
"normallookup" argument always to force using "normallookup" on
dirstate for "n" files, because "normal" may mark target files as
"clean" unexpectedly.
To reduce cost of "lfile not in filelist", this patch converts
"filelist" to a "set" object: "filelist" is used only in (1) the newly
added code path and (2) the next line of "filelist = set(filelist)".
This is a temporary way to fix with less changes. For fundamental
resolution of this kind of problems in the future, "lfdirstate" should
be updated with "dirstate" simultaneously during "merge.update"
execution: maybe by hooking "recordupdates" (+ total refactoring
around lfdirstate handling)
Before this patch, linear merging of modified or newly added largefile
causes unexpected result, if (1) largefile collides with same name
normal one in the target revision and (2) "local" largefile is chosen,
even though branch merging between such revisions doesn't.
Expected result of such linear merging is:
(1) (not yet recorded) largefile is kept in the working directory
(2) largefile is marked as (re-)"added"
(3) colliding normal file is marked as "removed"
But actual result is:
(1) largefile in the working directory is unlinked
(2) largefile is marked as "normal" (so treated as "missing")
(3) the dirstate entry for colliding normal file is just dropped
(1) is very serious, because there is no way to restore temporarily
modified largefiles.
(3) prevents the next commit from adding the manifest with correct
"removal of (normal) file" information for newly created changeset.
The root cause of this problem is putting "lfile" into "actions['r']"
in linear-merging case. At liner merging, "actions['r']" causes:
- unlinking "target file" in the working directory, but "lfile" as
"target file" is also largefile itself in this case
- dropping the dirstate entry for target file
"actions['f']" (= "forget") does only the latter, and this is reason
why this patch doesn't choose putting "lfile" into it instead of
"actions['r']".
This patch newly introduces action "lfmr" (LargeFiles: Mark as
Removed) to mark colliding normal file as "removed" without unlinking
it.
This patch uses "hg debugdirstate" instead of "hg status" in test,
because:
- choosing "local largefile" hides "removed" status of "remote
normal file" in "hg status" output, and
- "hg status" for "large2" in this case has another problem fixed in
the subsequent patch
Before this patch, there are two distinct "wlock" scopes below in
"hgmerge":
1. "merge.update" via original "hg.merge" function
2. "updatelfiles" specific "wlock" scope (to synchronize largefile
dirstate)
But these should be executed in the same "wlock" scope for
consistency, because users of "hg.merge" don't get "wlock" explicitly
before invocation of it.
- merge in commands
This patch puts almost all of the original "hgmerge" implementation into
"_hgmerge" to reduce changes.
Before this patch, there are two distinct "wlock" scopes below in
"hgupdaterepo":
1. "merge.update" via original "hg.updaterepo" function
2. "updatelfiles" specific "wlock" scope (to synchronize largefile
dirstate)
In addition to them, "dirstate.walk" is executed between these "wlock"
scopes.
But these should be executed in the same "wlock" scope for
consistency, because many (indirect) users of "hg.updaterepo" don't
get "wlock" explicitly before invocation of it.
"hg.clean" is invoked without "wlock" from:
- mqrepo.restore in mq
- bisect in commands
- update in commands
"hg.update" is invoked without "wlock" from:
- clone in mq
- pullrebase in rebase
- postincoming in commands (used in "hg pull -u", "hg unbundle")
- update in commands
This patch puts almost all original "hgupdaterepo" implementation into
"_hgupdaterepo" to reduce changes.
This makes hg log --follow --patch work, since in cmdutil._makelogrevset we
use the non-follow matcher for hg log --follow --patch with no file arguments.
This has actually been broken since at least Mercurial 2.8 -- hg log --patch
with largefiles only used to work when no largefiles existed. Rev 658ce4a0a0a9
exposed this bug for all cases.
lfstatus should only be True for operations where we want standins to be
printed out. We explicitly do not want that for historical operations like log.
Other historical operations like hg diff -r A -r B don't print out standins
either.
This is required to fix issue4334, but doesn't fix anything by itself. That's
why there aren't any tests accompanying this patch.
When the authorship of the changeset folded in does not match that of
the base changeset, we currently use the configured ui.username
instead. This is especially surprising when the user is not the author
of either of the changesets. In such cases, the resulting authorship
(the user's) is clearly incorrect. Even when the user is folding in a
patch they authored themselves, it's not clear whether they should
take over the authorship. Let's instead keep it simple and always
preserve the base changeset's authorship. This is also how
"git rebase -i" handles folding/squashing.
This is a quick fix for some consoles on windows (consoles that are not mingw
based) so that the debugcolor command doesn't throw a KeyError when effects
aren't supported (e.g. italic).
This patch fixes argument mismatch at formatting the abort message,
introduced by dcf325a55d85: the last '%s' doesn't have corresponded
argument.
This patch uses "unexpected size" in the abort message, to distinguish
the reason of failure from "unexpected type" failure checked in the
prior code path below:
if info[1] != type:
raise util.Abort(_('cannot read %r object at %s') % (type, rev))
This patch changes help text for "--edit" option of commands below:
- fetch
- qnew
- qrefresh
- qfold
- commit
- tag
This unification reduces translation cost, too.
This patch chooses not "further edit commit message already specified"
(of "hg commit") but "invoke editor on commit messages" as unified
help text for "--edit" option, because the latter is much older than
the former.
This patch changes help text for "--message" option of commands below
for unification.
- sign (of gpg)
- tag
This unification reduces translation cost, too.
This patch doesn't change the description for "--message" of "hg
rebase" below, because this should contain "collapse" word to explain
its purpose (only for "--collapse") clearly.
use text as collapse commit message
Before this patch, largefiles gotten from revisions other than the
parent of the working directory at "hg revert" become "clean"
unexpectedly in steps below:
1. "repo.status()" is invoked (for status check before reverting)
1-1 "dirstate" entry for standinfile SF is "normal"-ed
(1-2 "lfdirstate" entry of largefile LF (for SF) is "normal"-ed)
2. "cmdutil.revert()" is invoked
2-1 standinfile SF is updated in the working directory
2-2 "dirstate" entry for SF is NOT updated
3. "lfcommands.updatelfiles()" is invoked (by "overrides.overriderevert()")
3-1 largefile LF (for SF) is updated in the working directory
3-2 "dirstate" returns "n" and valid timestamp for SF (by 1-1 and 2-2)
3-3 "lfdirstate" entry for LF is "normal"-ed
3-4 "lfdirstate" is written into ".hg/largefiles/dirstate", and
timestamp of LF is stored into "lfdirstate" file (by 3-3)
(ASSUMPTION: timestamp of LF differs from one of "lfdirstate" file)
Then, "hs status" treats LF as "clean", even though LF is updated by
"other" revision (by 3-1), because "lfilesrepo.status()" always treats
"normal"-ed files (by 3-3 and 3-4) as "clean".
When largefiles are reverted, they should be "normallookup"-ed
forcibly.
This patch uses "normallookup" on "lfdirstate" while reverting, by
passing "True" to newly added argument "normallookup".
Forcible "normallookup"-ing is not so expensive, because list of
target largefiles is explicitly specified in this case.
This patch uses "[debug] dirstate.delaywrite" feature in the test, to
ensure that timestamp of the largefile gotten from "other" revision is
stored into ".hg/largefiles/dirstate" (for ASSUMPTION at 3-4)
Before this patch, largefiles gotten from "other" revision (with
conflict) at "hg merge" become "clean" unexpectedly in steps below:
1. "repo.status()" is invoked (for status check before merging)
1-1 "dirstate" entry for standinfile SF is "normal"-ed
1-2 "lfdirstate" entry of largefile LF (for SF) is "normal"-ed
2. "merge.update()" is invoked
2-1 SF is updated in the working directory
(ASSUMPTION: user choice "other" at conflict)
2-2 "dirstate" entry for SF is "merge"-ed
3. "lfcommands.updatelfiles()" is invoked (by "overrides.hgmerge()")
3-1 largefile LF (for SF) is updated in the working directory
3-2 "dirstate" returns "m" for SF (by 2-2)
3-3 "lfdirstate" entry for LF is left as it is
3-4 "lfdirstate" is written into ".hg/largefiles/dirstate", and
timestamp of LF is stored into "lfdirstate" file (by 1-2)
(ASSUMPTION: timestamp of LF differs from one of "lfdirstate" file)
Then, "hs status" treats LF as "clean", even though LF is updated by
"other" revision (by 3-1), because "lfilesrepo.status()" always treats
"normal"-ed files (by 1-2 and 3-4) as "clean".
When state of standinfile in "dirstate" is "m", largefile should be
"normallookup"-ed.
This patch invokes "normallookup" on "lfdirstate" for merged files.
This patch uses "[debug] dirstate.delaywrite" feature in the test, to
ensure that timestamp of the largefile gotten from "other" revision is
stored into ".hg/largefiles/dirstate". (for ASSUMPTION at 3-4)
Before this patch, largefiles gotten from "other" revision (without
conflict) at "hg merge" become "clean" unexpectedly in steps below:
1. "merge.update()" is invoked
1-1 standinfile SF is updated in the working directory
1-2 "dirstate" entry for SF is "normallookup"-ed
2. "lfcommands.updatelfiles()" is invoked (by "overrides.hgmerge()")
2-1 largefile LF (for SF) is updated in the working directory
2-2 "dirstate" returns "n" for SF (by 1-2)
2-3 "lfdirstate" entry for LF is "normal"-ed
2-4 "lfdirstate" is written into ".hg/largefiles/dirstate", and
timestamp of LF is stored into "lfdirstate" file
(ASSUMPTION: timestamp of LF differs from one of "lfdirstate" file)
Then, "hs status" treats LF as "clean", even though LF is updated by
"other" revision (by 2-1), because "lfilesrepo.status()" always treats
"normal"-ed files (by 2-3 and 2-4) as "clean".
When timestamp is not set (= negative value) for standinfile in
"dirstate", largefile should be "normallookup"-ed regardless of
rebasing or not, because "n" state in "dirstate" doesn't ensure
"clean"-ness of a standinfile at that time.
This patch uses "normallookup" instead of "normal", if "mtime" of
standin is unset
This is a temporary way to fix with less changes. For fundamental
resolution of this kind of problems in the future, "lfdirstate" should
be updated with "dirstate" simultaneously while "merge.update"
execution: maybe by hooking "recordupdates"
It is also why this patch (temporarily) uses internal field "_map" of
"dirstate" directly.
This patch uses "[debug] dirstate.delaywrite" feature in the test, to
ensure that timestamp of the largefile gotten from "other" revision is
stored into ".hg/largefiles/dirstate". (for ASSUMPTION at 2-4)
This patch newly adds "test-largefiles-update.t", to avoid increasing
cost to run other tests for largefiles by subsequent patches
(especially, "[debug] dirstate.delaywrite" causes so).
Previously, the directory '.hg/largefiles' would always be created if it didn't
exist when the lfdirstate was opened. If there were no standin files, no
dirstate file would be created in the directory. The end result was that
enabling the largefiles extension globally, but not explicitly adding a
largefile would result in the repository eventually sprouting this directory.
Creation of this directory effectively changes readonly operations like summary
and status into operations that require write access. Without write access,
commands that would succeed without the extension loaded would abort with a
surprising error when the extension is loaded, but not actively used:
$ hg sum -R /tmp/thg --config extensions.largefiles=
parent: 16541:00dc703d5aed
repowidget: specify incoming bundle by plain file path to avoid url parsing
branch: default
abort: Permission denied: '/tmp/thg/.hg/largefiles'
This change is simpler than changing the callers of openlfdirstate() to use the
'create' parameter that was introduced in 74522122b97d, and probably how that
should have been implemented in the first place.
This new histedit command (short for "rollup") is a variant of "fold" akin to
"hg amend" for working copy: it accumulates changes without interrupting
the user and asking for an updated commit message.
Before this patch, after successful "hg rebase" of the revision
removing largefiles, "hg status" may still show ""R" for such
largefiles unexpectedly.
"lfilesrepo.commit" executes the special code path for automated
committing while rebase/transplant, and lfdirstate entries for removed
files aren't updated in this code path, even after successful
committing.
Then, "R" entries still existing in lfdirstate cause unexpected "hg
status" output.
This patch synchronizes lfdirstate with dirstate after automated
committing.
This patch passes False as "normallookup" to "synclfdirstate", because
modified files in "files()" of the recent (= just committed) context
should be "normal"-ed.
This is a temporary way to fix with less changes. For fundamental
resolution of this kind of problems in the future, lfdirstate should
be updated with dirstate simultaneously. Hooking "markcommitted" of
ctx in "localrepository.commitctx" may achieve this.
This problem occurs, only when (1) the parent of the working directory
is rebased and (2) it removes largefiles, because:
- if the parent of the working directory isn't rebased, returning to
the initial revision (= update) after rebase hides this problem
- files added on "other" branch (= rebase target) are treated not as
"added" but as "modified" (= "normal" status and "unset"
timestamp) at merging
This patch tests also the status of added largefile, but it is only
for avoiding regression.
In addition to conditions above, "hg status" must not take existing
files to reproduce this problem, because existing files make
"match._files" not empty in "lfilesrepo.status" code path below:
def sfindirstate(f):
sf = lfutil.standin(f)
dirstate = self.dirstate
return sf in dirstate or sf in dirstate.dirs()
match._files = [f for f in match._files
if sfindirstate(f)]
Not empty "match._files" prevents "status" on lfdirstate from
returning the result containing problematic "R" files.
This is reason why "large1" (removed) and "largeX" (added) are checked
separately in this patch.
Problematic code path in "lfilesrepo.commit" is used also by "hg
transplant", but this problem doesn't occur at "hg transplant",
because invocation of "updatelfiles" after transplant-ing in
"overridetransplant" causes cleaning lfdirstate up.
This patch tests also "hg transplant" as same as "hg rebase", but it
is only for avoiding regression.
Before this patch, newly added (but not yet committed) largefiles
aren't treated as unknown ("?") after "hg rollback".
After "hg rollback", lfdirstate still contains "A" status entries for
such largefiles, even though corresponding entries for standins are
already dropped from dirstate.
Such "orphan" entries in lfdirstate prevent unknown (large)files in
the working directory from being listed up in "unknown" list. The code
path in "if working" route of "lfilesrepo.status" below drops
largefiles tracked in lfdirstate from "unknown" list:
lfiles = set(lfdirstate._map)
# Unknown files
result[4] = set(result[4]).difference(lfiles)
This patch drops orphan entries from lfdristate at "hg rollback".
This is a temporary way to fix with less changes. For fundamental
resolution of this kind of problems in the future, lfdirstate should
be rollback-ed as a part of transaction, as same as dirstate.
Before this patch, removed or forgotten largefiles aren't treated as
removed ("R") after "hg rollback". Removed ones are treated as missing
("!") and forgotten ones are treated as clean ("C") unexpectedly.
"overriderollback" uses "normallookup" to restore status in lfdirstate
for largefiles other than ones not added in rollback-ed revision, but
this isn't correct for removed (or forgotten) largefiles.
This patch uses "lfutil.synclfdirstate" to restore "R" status of
removed (or forgotten) largefiles correctly at "hg rollback".
This is a temporary way to fix with less changes. For fundamental
resolution of this kind of problems in the future, lfdirstate should
be rollback-ed as a part of transaction, as same as dirstate.
Before this patch, there are three distinct "wlock" scopes in
"overriderollback":
1. "localrepository.rollback" via original "rollback" command,
2. "merge.update" for reverting standin files only, and
3. "overriderollback" specific "wlock" scope (to synchronize
largefile dirstate)
But these should be executed in the same "wlock" scope for
consistency.
We now pass a transaction option to this phase movement function. The
object is currently not used by the function, but it will be in the
future.
All call sites have been updated. Most call sites were already enclosed in a
transaction for a long time. The handful of others have been recently
updated in previous commit.
We now pass a transaction option to this phase movement function. The object
is currently not used by the function, but it will be in the future.
All call sites have been updated. Most call sites were already enclosed in a
transaction for a long time. The handful of others have been recently
updated in previous commit.
The retractboundary function remains to be upgraded.
cset 21b4faf3787e has removed this option. This commit just tidies the
code that was associated to it. It also fixes the internal calls to
the strip() function.
Before this change, any function that thought it would want as a final
safety to keep a partial backup bundle (bundling changes not linearly
related to the current change being stripped), had to explicitly pass
a backup="strip" option. With this change, these backups are always
kept in case of an exception and always removed if there is no
exception. Only full backups can be specified with backup=True or no
full backups with backup=False.
We rely on the internal mechanism to commit the changeset in the right state.
This is similar to what the mq extension is doing.
This is an important change as we plan to move phase movement with the
transaction. Avoiding phase movement from high level code will avoid them the
burden of transaction handling. It is also important to limit the need for
transaction handling as this limits the odds of people messing up. Most common
expected mess-up is to use a different transaction for changesets creation and
phase adjustment.
We rely on the internal mechanism to commit the changeset in the right phase.
This similar to what the mq extension is doing.
This is an important change as we plan to includes phase movement within the
transaction. Avoiding phase movement from high-level code will avoid the
burden of transaction handling. It is also important to limit the need for
transaction handling as this limits the odds of people messing up. Most common
expected mess-up is code using a different transaction for changeset creation
and phase adjustment.
This patch passes 'editform' argument according to the format below:
EXTENSION[.COMMAND][.ROUTE]
- EXTENSION: name of extension
- COMMAND: name of command, if there are two or more commands in EXTENSION
- ROUTE: name of route, if there are two or more routes in COMMAND
In this patch, COMMAND and ROUTE are omitted.
This patch passes 'editform' argument according to the format below:
EXTENSION[.COMMAND][.ROUTE]
- EXTENSION: name of extension
- COMMAND: name of command, if there are two or more commands in EXTENSION
- ROUTE: name of route, if there are two or more routes in COMMAND
In this patch:
- 'shelve' is used as COMMAND
- ROUTE is omitted
This patch passes 'editform' argument according to the format below:
EXTENSION[.COMMAND][.ROUTE]
- EXTENSION: name of extension
- COMMAND: name of command, if there are two or more commands in EXTENSION
- ROUTE: name of route, if there are two or more routes in COMMAND
In this patch:
- COMMAND is omitted
- 'normal' and 'collapse' are used as ROUTE
This patch passes 'editform' argument according to the format below:
EXTENSION[.COMMAND][.ROUTE]
- EXTENSION: name of extension
- COMMAND: name of command, if there are two or more commands in EXTENSION
- ROUTE: name of route, if there are two or more routes in COMMAND
In this patch:
- MQ command names (qnew/qrefresh/qfold) are used as COMMAND
- ROUTE is omitted
This patch passes 'editform' argument according to the format below:
EXTENSION[.COMMAND][.ROUTE]
- EXTENSION: name of extension
- COMMAND: name of command, if there are two or more commands in EXTENSION
- ROUTE: name of route, if there are two or more routes for COMMAND
In this patch:
- 'edit', 'fold', 'mess' and 'pick' are used as COMMAND
- ROUTE is omitted
'histedit.pick' case is very rare, but possible if:
- target revision causes conflict at merging (= requires '--continue'), and
- description of it is empty ('hg commit -m " "' can create such one)
In the code path for 'histedit --continue' (the last patch hunk),
'canonaction' doesn't contain the entry for 'fold', because 'fold'
action causes:
- using temporary commit message forcibly, and
- making 'editopt' False always (= omit editor invocation if commit
message is specified)
This patch passes 'editform' argument according to the format below:
EXTENSION[.COMMAND][.ROUTE]
- EXTENSION: name of extension
- COMMAND: name of command, if there are two or more commands in EXTENSION
- ROUTE: name of route, if there are two or more routes in COMMAND
In this patch, 'sign' is used as COMMAND, and ROUTE is omitted.
This patch passes 'editform' argument according to the format below:
EXTENSION[.COMMAND][.ROUTE]
- EXTENSION: name of extension
- COMMAND: name of command, if there are two or more commands in EXTENSION
- ROUTE: name of route, if there are two or more routes in COMMAND
In this patch, COMMAND and ROUTE are omitted.
