Commit Graph

8836 Commits

Author SHA1 Message Date
Yuya Nishihara
a85993b95a revset: port parsing rule of old-style ranges from scmutil.revrange()
The old-style parser will be removed soon.
2015-07-18 23:30:17 +09:00
Yuya Nishihara
137f249748 debugrevspec: pass lookup function to visualize fallback mechanism
The next patch will move the exceptional parsing of old-style ranges
to revset.tokenize(). This patch will allow us to see the result tree.

Note that the parsing result of '-a-b-c-' is incorrect at this changeset.
It will be fixed soon.
2015-07-18 23:02:18 +09:00
Pierre-Yves David
d41570d94c merge with stable 2015-08-03 14:05:42 -07:00
Pierre-Yves David
6b2961e47a histedit: backout 3e883e7ec57b
The faulty changeset use obsolescence marker to roll the repository back on
--abort. This is a problematic approach because --abort should be as close as an
actually transaction rollback as possible stripping all created data from the
repository (cf `hg rebase --abort` stripping all created changesets). Instead
3e883e7ec57b made all content created during the aborted histedit still
available in the repository adding obsolescence marker to make them hidden. This
will cause trouble to evolution user as a re-run of the same histedit (with
success) will likely result in the very same node to be "recreated" while
obsolescence marker would be in place for them. And canceling an obsoletion is
still a fairly complicated process.

This also rollback using obsmarkers instead of strip to clean up temporary node
on successful histedit run because the two change were not split in separated
changeset. Rolling that part back does not have significant consequence a will
have to be resubmitted independently
2015-07-31 15:11:07 -07:00
Eugene Baranov
322848b810 convert: when converting from Perforce use original local encoding by default
On Windows Perforce command line client uses default system locale to encode
output. Using 'latin_1' causes locale-specific characters to be replaced with
question marks. With this patch we will use default locale by default whilst
allowing to specify it explicity with 'convert.p4.encoding' config option.

This is a potentially breaking change for any scripts relying on output treated
as in 'latin_1' encoding.

Also because hgext.convert.convcmd overwrites detected default system locale
with UTF-8 we had to introduce an import cycle in hgext.convert.p4 to retrieve
originally detected encoding from hgext.convert.convcmd.
2015-07-22 16:57:11 +01:00
Siddharth Agarwal
20e8951b71 test-bookmarks.t: avoid nested repo
This is (a) pretty unnecessary and (b) breaks tests for the third-party
hgwatchman extension, which doesn't support nested repos.
2015-08-02 21:56:38 -07:00
Gregory Szorc
db719a7a87 help: scripting help topic
There are a lot of non-human consumers of Mercurial. And the challenges
and considerations for machines consuming Mercurial is significantly
different from what humans face.

I think there are enough special considerations around how machines
consume Mercurial that a dedicated help topic is warranted. I concede
the audience for this topic is probably small compared to the general
audience. However, lots of normal Mercurial users do things like create
one-off shell scripts for common workflows that I think this is useful
enough to be in the install (as opposed to, say, a wiki page - which
most users will likely never find).

This text is by no means perfect. But you have to start somewhere. I
think I did cover the important parts, though.
2015-07-18 17:10:28 -07:00
Matt Harbison
5dece5b905 convert: document convert.hg.startrev 2015-07-29 21:31:56 -04:00
FUJIWARA Katsunori
2fb0db56cb transplant: restore dirstate correctly at unexpected failure
Before this patch, transplant can't restore dirstate as expected at
failure other than one while patching. This causes:

  - unexpected file status

  - dirstate refers already rollback-ed parent
    (only at failure of transplanting the 2nd or later revision)

To restore dirstate correctly also at unexpected failure, this patch
encloses scope of store lock and transaction by 'dirstateguard'.

This is temporary fixing for stable branch. See
DirstateTransactionPlan wiki page for detail about the future plan to
treat dirstate consistently around scope boundary of transaction.

    https://mercurial.selenic.com/wiki/DirstateTransactionPlan

This patch also adds 'if lock' examination for safety
'lock.release()', because creating 'dirstateguard' object may fail
unexpectedly (e.g. IOError for saving dirstate).

BTW, in the test script, putting section header '[extensions]' into
'.hg/hgrc' is needed to fix incomplete disabling 'abort' extension at
d0d06f4ca862.
2015-07-30 06:22:09 +09:00
FUJIWARA Katsunori
4865b28d1a localrepo: make journal.dirstate contain in-memory changes before transaction
Before this patch, in-memory dirstate changes aren't written out at
opening transaction, even though 'journal.dirstate' is created
directly from '.hg/dirstate'.

Therefore, subsequent 'hg rollback' uses incomplete 'undo.dirstate' to
restore dirstate, if dirstate is changed and isn't written out before
opening transaction.

In cases below, the condition "dirstate is changed and isn't written
out before opening transaction" isn't satisfied and this problem
doesn't appear:

  - "wlock scope" and "transaction scope" are almost equivalent

    e.g. 'commit --amend', 'import' and so on

  - dirstate changes are written out before opening transaction

    e.g. 'rebase' (via 'dirstateguard') and 'commit -A' (by separated
    wlock scopes)

On the other hand, 'backout' may satisfy the condition above.

To make 'journal.dirstate' contain in-memory changes before opening
transaction, this patch explicitly invokes 'dirstate.write()' in
'localrepository.transaction()'.

'dirstate.write()' is placed before not "writing journal files out"
but "invoking pretxnopen hooks" for visibility of dirstate changes to
external hook processes.

BTW, in the test script, 'touch -t 200001010000' and 'hg status' are
invoked to make file 'c' surely clean in dirstate, because "clean but
unsure" files indirectly cause 'dirstate.write()' at 'repo.status()'
in 'repo.commit()' (see e1d123a2ee1f for detail) and prevents from
certainly reproducing the issue.
2015-07-30 06:16:12 +09:00
Matt Harbison
e8c6c3b443 dirstate: ensure mv source is marked deleted when walking icasefs (issue4760)
Previously, importing a case-only rename patch on a case insensitive filesystem
caused the original file to be marked as '!' in status.  The source was being
forgotten properly in patch.workingbackend.close(), but the call it makes to
scmutil.marktouched() then put the file back into the 'n' state (but it was
still missing from the filesystem).

The cause of this was scmutil._interestingfiles() would walk dirstate,
and since dirstate was able to lstat() the old file via the new name,
was treating this as a forgotten file, not a removed file.
scmutil.marktouched() re-adds forgotten files, so dirstate got out of
sync with the filesystem.

This could be handled with less code in the "kind == regkind or kind
== lnkkind" branch of dirstate._walkexplicit(), but this avoids
filesystem accesses unless case collisions occur. _discoverpath() is
used instead of normalize(), since the dirstate case is given first
precedence, and the old file is still in it. What matters is the
actual case in the filesystem.
2015-07-27 21:27:24 -04:00
Wagner Bruna
f6ce8ccc47 repair: fix typo in warning message 2015-07-26 09:28:52 -03:00
Durham Goode
677efb706e ignore: fix include: rules depending on current directory (issue4759)
When reading pattern files, we just call open(path), which is relative to the
current directory.  Let's fix this by resolving the paths before attempting to
read the file.
2015-07-24 16:44:52 -07:00
Durham Goode
c9da9870f4 test: move ignore test run into a subdirectory
Previously the hgignore test just called hg init in the test directory. A future
patch needs to test hgignore stuff from outside of the repo, so let's move the
entire test repo into a subdirectory.
2015-07-24 16:43:21 -07:00
Anton Shestakov
403857c028 highlight: produce correct markup when there's a blank line just before EOF
Due to how the colorized output from pygments was stripped of <pre> elements,
when there was an empty line at the end of a file, highlight extension produced
an incorrect markup (no closing tags from the fileline/annotateline template).
It wasn't usually noticeable, because browsers were smart enough to see where
the missing tags should've been, but in monoblue style it resulted in the last
line having twice the normal height.

Instead of awkwardly trying to strip outer <pre></pre> tags, let's make the
formatter with nowrap=True, which should do what we need in pygments since at
least 0.5 (2006-10-30).

Example from monoblue style:

Before:

    <div class="source">

<div style="font-family:monospace" class="parity0">
<pre><a class="linenr" href="#l1" id="l1">     1</a> </pre>
</div>
<div style="font-family:monospace" class="parity1">
<pre><a class="linenr" href="#l2" id="l2">     2</a>
    </div>

Now:

    <div class="source">

<div style="font-family:monospace" class="parity0">
<pre><a class="linenr" href="#l1" id="l1">     1</a> </pre>
</div>
<div style="font-family:monospace" class="parity1">
<pre><a class="linenr" href="#l2" id="l2">     2</a> </pre>
</div>
    </div>

(Notice the missing </pre></div> now in place)
2015-07-22 10:19:17 +08:00
Matt Harbison
1fb45c7e46 test-check-config: convert directory separators to '/' for MSYS
The output of the files command uses native separator.  MSYS then seems to drop
the '\' on Windows when invoking python:

 --- c:/Users/Matt/Projects/hg/tests/test-check-config-hg.t
 +++ c:/Users/Matt/Projects/hg/tests/test-check-config-hg.t.err
 @@ -6,22 +6,10 @@

    $ hg files "set:(**.py or **.txt) - tests/**" |
    >   xargs python contrib/check-config.py  Traceback (most recent call last):
      File "contrib/check-config.py", line 93, in <module>
        sys.exit(main(sys.argv[1:]))
      File "contrib/check-config.py", line 24, in main
        for l in open(f):
    IOError: [Errno 2] No such file or directory: 'contriball-revsets.txt'
    [123]
2015-07-20 23:33:57 -04:00
Matt Harbison
3e67d17d81 test-subrepo: fix globs for Windows
It looks like these changes originated with 23b60798c736.  I'm not sure that it
was intentional, but is seems harmless enough for an error message.
2015-07-20 22:48:42 -04:00
Matt Harbison
9af3be2f3b test-clone: fix globs for Windows 2015-07-20 22:44:53 -04:00
Danek Duvall
256c943981 tests: diff -r on Solaris emits "Common subdirectories:"
When using the -r option to Solaris diff, any directores that compare
identically are mentioned in the output.  We don't really care about these
directories for the purposes of this test, so ignore them.
2015-07-20 15:42:39 -07:00
Danek Duvall
ccade6afcc tests: work around differing hunk headers between GNU and Solaris diff
The hunk headers specifying what lines the hunk apply to differ.  They're
irrelevant to the test, so just eliminate them from the output.
2015-07-20 15:39:15 -07:00
Danek Duvall
6c0991c070 tests: use egrep with regular expression branches for compatibility
GNU grep allows you to use "a\|b" in a regular expression to match either
"a" or "b", but at least Solaris grep does not; only egrep allows for that.
And egrep considers "a+" to be "a{1,}" instead of an "a" and a literal plus
sign, so escape that as well.
2015-07-20 15:33:20 -07:00
Yuya Nishihara
d7d0977d82 tests: disable test of buffer overflow in parsers.c if --pure
It fails with AttributeError and there's no benefit to make it runnable
with pure Python code.
2015-07-19 15:27:28 +09:00
Matt Mackall
9630b6a23f tests: add a check-config pass 2015-06-26 18:45:29 -05:00
Yuya Nishihara
f7a6661b37 revset: parse nullary ":" operator as "0:tip"
This is necessary for compatibility with the old-style parser that will be
removed by future patches.
2015-07-05 12:15:54 +09:00
Matt Harbison
400240f7c1 extdiff: add support for subrepos
Git and svn subrepo support is incomplete, because they don't support archiving
the working copy.
2012-07-15 12:43:10 -04:00
Matt Harbison
6579c8c6d9 extdiff: use archiver to take snapshots of committed revisions
This is the last step before supporting extdiff -S.  It maintains the existing
behavior of diffing the largefile standins instead of the largefiles themselves.
Note however that the standins are not updated immediately upon modification, so
uncommitted largefile changes are ignored, as they previously were, even with
the diff command.
2012-07-11 20:48:51 -04:00
Yuya Nishihara
83b3c48a90 parsers: fix buffer overflow by invalid parent revision read from revlog
If revlog file is corrupted, it can have parent pointing to invalid revision.
So we should validate it before updating nothead[], phases[], seen[], etc.
Otherwise it would cause segfault at best.

We could use "rev" instead of "maxrev" as upper bound, but I think the explicit
"maxrev" can clarify that we just want to avoid possible buffer overflow
vulnerability.
2015-07-16 23:36:08 +09:00
Laurent Charignon
e2ba5b2bbc histedit: mark temporary commits as obsolete when allowed to
Before this patch, we were stripping temporary commits at the end of a histedit
whether it was successful or not. If we can create obs markers, we should
create them instead of stripping because it is faster and safer.
2015-07-16 11:12:15 -07:00
FUJIWARA Katsunori
a8288b7afa censor: make various path forms available like other Mercurial commands
Before this patch, censored file should be exactly "a path relative to
repository root" regardless of current working directory, because "hg
censor" passes "path" to "repo.file()" directly without any
preparations.

To make various path forms available like other Mercurial commands,
this patch gets a target file path in the way of "hg parents FILE".

Getting "wctx" is relocated to reuse "wctx" for efficiency.
2015-07-17 00:22:16 +09:00
Eugene Baranov
afafa68827 convert: use 'default' for specifying branch name in branchmap (issue4753)
A fix for issue2653 with f5abbf51a76e introduced a discrepancy how default
branch should be denoted when converting with branchmap from different SCM.
E.g. for Git and Mercurial you need to use 'default' whilst for Perforce and
SVN you had to use 'None'. This changeset unifies 'default' for such purposes
whilst falling back to 'None' when no 'default' mapping specified.
2015-07-14 14:40:56 +01:00
FUJIWARA Katsunori
3894a2e107 shelve: omit incorrect 'commit' suggestion at 'hg shelve -i'
Before this patch, 'hg shelve -i' under non-interactive mode suggests
'use commit instead', and it obviously incorrect, because what user
wants to do isn't 'commit' but 'shelve'.

To omit incorrect 'commit' suggestion at 'hg shelve -i', this patch
specifies 'None' for 'cmdsuggest' argument of 'cmdutil.dorecord()'.
2015-07-15 04:45:58 +09:00
FUJIWARA Katsunori
a26b3feac9 record: omit meaningless 'qrefresh' suggestion at 'hg qrefresh -i'
Before this patch, 'hg qrefresh -i' under non-interactive mode
suggests 'use qrefresh instead', and it obviously meaningless.

To omit meaningless 'qrefresh' suggestion at 'hg qrefresh -i', this
patch specifies 'None' for 'cmdsuggest' argument of 'cmdutil.dorecord()'.
2015-07-15 04:45:58 +09:00
FUJIWARA Katsunori
b6717eba7b record: omit meaningless 'qnew' suggestion at 'hg qnew -i'
Before this patch, 'hg qnew -i' under non-interactive mode suggests
'use qnew instead', and it obviously meaningless.

To omit meaningless 'qnew' suggestion at 'hg qnew -i', this patch adds
internal function '_qrecord()' and specifies 'cmdsuggest' for each of
'qrecord' and 'qnew' separately.
2015-07-15 04:45:58 +09:00
FUJIWARA Katsunori
b3a7702a57 record: omit meaningless 'commit' suggestion at 'hg commit -i'
Before this patch, 'hg commit -i' under non-interactive mode suggests
'use commit instead', and it obviously meaningless.

This patch makes 'record.record'()' examine 'ui.interactive()' and
show suggestion by itself before calling 'commands.commit()'.

This allows 'commands.commit()' to specify 'None' for 'cmdsuggest'
argument of 'cmdutil.dorecord()' to omit meaningless 'commit'
suggestion at 'hg commit -i'.
2015-07-15 04:45:58 +09:00
Anton Shestakov
f8c9284f5c spartan: don't drop current revision in log/graph links
Just to be consistent with log and shortlog links, graph links should have a
revision context too. And the same goes for the graph page, where it's log and
shortlog links that should have context.
2015-07-12 01:51:01 +08:00
Matt Mackall
9573c557ad bookmarks: clear active bookmark on non-linear update 2015-07-14 18:50:20 -05:00
Eugene Baranov
39327b5b56 convert: unescape Perforce-escaped special characters in filenames 2015-07-14 16:23:57 +01:00
Durham Goode
ecce172bba convert: allow customizing git remote prefix
Previously all git remotes were created as "remote/foo". This patch adds a
configuration option for deciding what the prefix should be. This is useful if
you want the bookmarks to be "origin/foo" like they are in git, or if you're
integrating with the remotenames extension and don't want the local remote/foo
bookmarks to overlap with the remote foo bookmarks.
2015-07-13 21:37:46 -07:00
Yuya Nishihara
4444465461 templater: introduce one-pass parsing of nested template strings
Instead of re-parsing quoted strings as templates, the tokenizer can delegate
the parsing of nested template strings to the parser. It has two benefits:

 1. syntax errors can be reported with absolute positions
 2. nested template can use quotes just like shell: "{"{rev}"}"

It doesn't sound nice that the tokenizer recurses into the parser. We could
instead make the tokenize itself recursive, but it would be much more
complicated because we would have to adjust binding strengths carefully and
put dummy infix operators to concatenate template fragments.

Now "string" token without r"" never appears. It will be removed by the next
patch.
2015-06-15 23:11:35 +09:00
Yuya Nishihara
c95926ebb3 templater: check existence of closing brace of template string 2015-06-15 23:03:30 +09:00
Anton Shestakov
dc749de111 hgweb: provide links to branches, tags and bookmarks by name (paper and coal)
It's sometimes handy to, say, have a url always point to branch head, not just
at the current branch head by node hash. Previously, this was only possible by
manually editing url and replacing node hash with branch/tag/bookmark name. It
wasn't very convenient, or easy - in case the name contained special
characters that needed to be urlencoded.

Let's have /branches, /tags and /bookmarks pages in paper and coal style
provide links both to symbolic revisions and to node hashes.

This feature was wished for in issue3594.
2015-07-12 18:04:48 +08:00
Anton Shestakov
59f182075a hgweb: allow symbolic revisions with forward slashes in urls
It's possible to have a branch/tag/bookmark with all kinds of special
characters, such as {}/\!?. While not very conveniently, symbolic revisions
with such characters work from command line if user correctly quotes the
characters. These characters also work in hgweb, when they are properly
encoded, with one exception: '/' (forward slash, urlencoded as '%2F'), which
was getting decoded before hgweb could parse it as a part of PATH_INFO.
Because of that, hgweb was seeing it as any other forward slash, that is, as
just another url parts separator.

For example, if user wanted to see the content of dir/file at bookmark
'feature/eggs', url could be: '/file/feature%2Feggs/dir/file'. But hgweb tried
to find a revision 'feature' and get contents of 'eggs/dir/file'.

To fix this, let's assume forward slashes are doubly-urlencoded (%252F), so
CGI/WSGI server decodes it into %2F. Then we can decode %2F in the revision
part of the url into an actual '/' character.

Making hgweb produce such urls will be done in the next 2 patches.
2015-07-12 16:06:57 +08:00
FUJIWARA Katsunori
ec38bf98b8 shelve: keep old backups if timestamp can't decide exact order of them
Before this patch, backups to be discarded are decided by steps below
at 'hg unshelve' or so:

  1. list '(st_mtime, filename)' tuples of each backups up
  2. sort list of these tuples, and
  3. discard backups other than 'maxbackups' ones at the end of list

This doesn't work well in the case below:

  - "sort by name" order differs from actual backup-ing order, and
  - some of backups have same timestamp

For example, 'test-shelve.t' satisfies the former condition:

  - 'default-01' < 'default-1' in "sort by name" order
  - 'default-1'  < 'default-01' in actual backup-ing order

Then, 'default-01' is discarded instead of 'default-1' unexpectedly,
if they have same timestamp. This failure appears occasionally,
because the most important condition "same timestamp" is timing
critical.

To avoid such unexpected discarding, this patch keeps old backups if
timestamp can't decide exact order of them.

Timestamp of the border backup (= the oldest one of recent
'maxbackups' ones) as 'bordermtime' is used to examine whether
timestamp can decide exact order of backups.
2015-07-13 23:34:12 +09:00
FUJIWARA Katsunori
78eff9a5a8 subrepo: use repo.pathto instead of util.pathto to simplify invocation
This centralization into 'repo.pathto()' should reduce the cost of vfs
migration around 'getcwd()' and so on in the future.
2015-07-10 00:59:51 +09:00
Yuya Nishihara
33dcb19532 revset: work around x:y range where x or y is wdir()
All revisions must be contiguous in spanset, so we need the special case
for the wdir revision.
2015-06-28 16:08:07 +09:00
Yuya Nishihara
3732960ab3 revset: use integer representation of wdir() in revset
This is the simplest way to handle wdir() revision in revset. None didn't
work well because revset heavily depends on integer operations such as min(),
max(), sorted(), x:y, etc.

One downside is that we cannot do "wctx.rev() in set" because wctx.rev() is
still None. We could wrap the result set by wdirproxyset that translates None
to wdirrev, but it seems overengineered at this point.

    result = getset(repo, subset, tree)
    if 'wdir' in funcsused(tree):
        result = wdirproxyset(result)

Test cases need the '(all() + wdir()) &' hack because we have yet to fix the
bootstrapping issue of null and wdir.
2015-03-16 16:17:06 +09:00
Yuya Nishihara
0cf602f409 changeset_printer: display wdirrev/wdirnode values for workingctx
Because we want to eliminate "if"s in the default template, it makes sense to
display wdirrev/wdirnode values for now. wdir() is still experimental, so the
output of "log -r'wdir()'" may change in future.
2015-07-04 17:19:49 +09:00
Gregory Szorc
c9f7c94c30 hg: support for auto sharing stores when cloning
Many 3rd party consumers of Mercurial have created wrappers to
essentially perform clone+share as a single operation. This is
especially popular in automated processes like continuous integration
systems. The Jenkins CI software and Mozilla's Firefox release
automation infrastructure have both implemented custom code that
effectively perform clone+share. The common use case here is that
clients want to obtain N>1 checkouts while minimizing disk space and
network requirements. Furthermore, they often don't care that a clone
is an exact mirror of a remote: they are simply looking to obtain
checkouts of specific revisions.

When multiple third parties implement a similar feature, it's a good
sign that the feature is worth adding to the core product. This patch
adds support for an easy-to-use clone+share feature.

The internal "clone" function now accepts options to control auto
sharing during clone. When the auto share mode is active, a store will
be created/updated under the base directory specified and a new
repository pointing to the shared store will be created at the path
specified by the user.

The share extension has grown the ability to pass these options into
the clone command/function.

No command line options for this feature are added because we don't
feel the feature will be popular enough to warrant their existence.

There are two modes for auto share mode. In the default mode, the shared
repo is derived from the first changeset (rev 0) in the remote
repository. This enables related repositories existing at different URLs
to automatically use the same storage. In environments that operate
several repositories (separate repo for branch/head/bookmark or separate
repo per user), this has the potential to drastically reduce storage
and network requirements. In the other mode, the name is derived from the
remote's path/URL.
2015-07-08 16:19:09 -07:00
FUJIWARA Katsunori
ec0ec0f1d2 cmdutil: apply dirstate.normallookup on (maybe partially) committed files
To detect change of a file without redundant comparison of file
content, dirstate recognizes a file as certainly clean, if:

  (1) it is already known as "normal",
  (2) dirstate entry for it has valid (= not "-1") timestamp, and
  (3) mode, size and timestamp of it on the filesystem are as same as
      ones expected in dirstate

This works as expected in many cases, but doesn't in the corner case
that changing a file keeps mode, size and timestamp of it on the
filesystem.

The timetable below shows steps in one of typical such situations:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
  N                                              ***   ***
       - change "f"                                     N

       - execute 'hg commit -i'
         - backup "f" with timestamp N
         - revert "f" by 'merge.update()'               N
           with 'partially'
         - apply selected hunks                         N
           by 'patch.patch()'

         - 'repo.commit()'
           - 'dirstate.normal("f")'         N
  N+1
           - 'dirstate.write()'             N    N

         - restore "f"                                  N+1
         - restore timestamp of "f"                     N

       - 'hg status' shows "f" as "clean"   N    N      N
  ---- ----------------------------------- ---- ----- -----

The most important point is that 'dirstate.write()' is executed at N+1
or later. This causes writing dirstate timestamp N of "f" out
successfully. If it is executed at N, 'parsers.pack_dirstate()'
replaces timestamp N with "-1" before actual writing dirstate out.

This issue can occur when 'hg commit -i' satisfies conditions below:

  - the file is committed partially, and
  - mode and size of the file aren't changed before and after committing

The root cause of this issue is that (maybe partially changed) files
are restored with original timestamp but dirstate isn't updated for
them.

To detect changes of files correctly, this patch applies
'dirstate.normallookup()' on restored files. Status check is needed
before 'dirstate.normallookup()', because status other than "n(ormal)"
should be kept at failure of committing.

This patch doesn't examine whether each files are committed fully or
partially, because interactive hunk selection makes it difficult.

After this change, timetable is changed as below:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
  N                                              ***   ***
       - change "f"                                     N

       - execute 'hg commit -i'
         - backup "f" with timestamp N
         - revert "f" by 'merge.update()'               N
           with 'partially'
         - apply selected hunks                         N
           by 'patch.internalpatch()'

         - 'repo.commit()'
           - 'dirstate.normal("f")'         N
  N+1
           - 'dirstate.write()'             N    N

         - restore "f"                                  N+1
         - restore timestamp of "f"                     N
       ----------------------------------- ---- ----- -----
         - normallookup("f")                -1
         - release wlock
           - 'dirstate.write()'             -1   -1     N
       ----------------------------------- ---- ----- -----

       - 'hg status' shows "f" as "clean"   -1   -1     N
  ---- ----------------------------------- ---- ----- -----

To reproduce this issue in tests certainly, this patch emulates some
timing critical actions as below:

  - change "f" at N

    'touch -t 200001010000' before command invocation changes mtime of
    "f" to "2000-01-01 00:00" (= N).

  - apply selected hunks at N

    'patch.internalpatch()' with 'fakepatchtime.py' explicitly changes
    mtime of patched files to "2000-01-01 00:00" (= N).

  - 'dirstate.write()' at N+1 (or "not at N")

    'pack_dirstate()' uses actual timestamp at runtime as "now", and
    it should be different from the "2000-01-01 00:00" of "f".

BTW, in 'test-commit-interactive.t', files are sometimes treated as
modified , even though they are just committed fully via 'hg commit
-i' and 'hg diff' shows nothing for them.

Enabling win32text causes EOL style mismatching below:

  - files are changed in LF style EOL

    => files restored after committing uses LF style EOL (1)

  - 'merge.update()' reverts files in CRLF style EOL
  - 'patch.internalpatch()' changes files in CRLF style EOL

    => 'dirstate.normal()' via 'repo.commit()' uses the size of files
       in CRLF style EOL (2)

Therefore, fully committed files are treated as "modified", because
'lstat()' returns size of (1) restored files in LF style EOL, but
dirstate expects size of (2) committed files in CRLF style EOL.

After this patch, 'dirstate.normallookup()' on committed files forces
subsequent 'hg status' to examine changes exactly, and fully committed
files are treated as clean as expected.

This is reason why this patch also does:

  - add some 'hg status' checking status of fully committed files
  - clear win32text configuration before size/timestamp sensitive examination
2015-07-08 17:07:45 +09:00
FUJIWARA Katsunori
1dcc27a649 context: write dirstate out explicitly at the end of markcommitted
To detect change of a file without redundant comparison of file
content, dirstate recognizes a file as certainly clean, if:

  (1) it is already known as "normal",
  (2) dirstate entry for it has valid (= not "-1") timestamp, and
  (3) mode, size and timestamp of it on the filesystem are as same as
      ones expected in dirstate

This works as expected in many cases, but doesn't in the corner case
that changing a file keeps mode, size and timestamp of it on the
filesystem.

The timetable below shows steps in one of typical such situations:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
   *                                             ***    ***
       - 'hg transplant REV1 REV2 ...'
         - transplanting REV1
           ....
   N
           - change "f", but keep size                   N
             (via 'patch.patch()')
           - 'dirstate.normal("f")'          N   ***
             (via 'repo.commit()')

         - transplanting REV2
           - change "f", but keep size                   N
             (via 'patch.patch()')
           - aborted while patching
  N+1
         - release wlock
           - 'dirstate.write()'              N    N      N

       - 'hg status' shows "r1" as "clean"   N    N      N
  ---- ----------------------------------- ---- ----- -----

The most important point is that 'dirstate.write()' is executed at N+1
or later. This causes writing dirstate timestamp N of "f" out
successfully. If it is executed at N, 'parsers.pack_dirstate()'
replaces timestamp N with "-1" before actual writing dirstate out.

This issue can occur when 'hg transplant' satisfies conditions below:

  - multiple revisions to be transplanted change the same file
  - those revisions don't change mode and size of the file, and
  - the 2nd or later revision of them fails after changing the file

The root cause of this issue is that files are changed without
flushing in-memory dirstate changes via 'repo.commit()' (even though
omitting 'dirstate.normallookup()' on files changed by 'patch.patch()'
for efficiency also causes this issue).

To detect changes of files correctly, this patch writes in-memory
dirstate changes out explicitly after marking files as clean in
'committablectx.markcommitted()', which is invoked via
'repo.commit()'.

After this change, timetable is changed as below:

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
   *                                             ***    ***
       - 'hg transplant REV1 REV2 ...'
         - transplanting REV1
           ....
   N
           - change "f", but keep size                   N
             (via 'patch.patch()')
           - 'dirstate.normal("f")'          N    ***
             (via 'repo.commit()')
       ----------------------------------- ---- ----- -----
           - 'dirsttate.write()'            -1    -1
       ----------------------------------- ---- ----- -----
         - transplanting REV2
           - change "f", but keep size                   N
             (via 'patch.patch()')
           - aborted while patching
  N+1
         - release wlock
           - 'dirstate.write()'              -1   -1     N

       - 'hg status' shows "r1" as "clean"   -1   -1     N
  ---- ----------------------------------- ---- ----- -----

To reproduce this issue in tests certainly, this patch emulates some
timing critical actions as below:

  - change "f" at N

    'patch.patch()' with 'fakepatchtime.py' explicitly changes mtime
    of patched files to "2000-01-01 00:00" (= N).

  - 'dirstate.write()' via 'repo.commit()' at N

    'fakedirstatewritetime.py' forces 'pack_dirstate()' to use
    "2000-01-01 00:00" as "now", only if 'pack_dirstate()' is invoked
    via 'committablectx.markcommitted()'.

  - 'dirstate.write()' via releasing wlock at N+1 (or "not at N")

    'pack_dirstate()' via releasing wlock uses actual timestamp at
    runtime as "now", and it should be different from the "2000-01-01
    00:00" of "f".

BTW, this patch doesn't test cases below, even though 'patch.patch()'
is used similarly in these cases:

  1. failure of 'hg import' or 'hg qpush'

  2. success of 'hg import', 'hg qpush' or 'hg transplant'

Case (1) above doesn't cause this kind of issue, because:

  - if patching is aborted by conflicts, changed files are committed

    changed files are marked as CLEAN, even though they are partially
    patched.

  - otherwise, dirstate are fully restored by 'dirstateguard'

    For example in timetable above, timestamp of "f" in .hg/dirstate
    is restored to -1 (or less than N), and subsequent 'hg status' can
    detect changes correctly.

Case (2) always causes 'repo.status()' invocation via 'repo.commit()'
just after changing files inside same wlock scope.

  ---- ----------------------------------- ----------------
                                           timestamp of "f"
                                           ----------------
                                           dirstate   file-
  time          action                     mem  file  system
  ---- ----------------------------------- ---- ----- -----
  N                                              ***   ***
       - make file "f" clean                            N

       - execute 'hg foobar'
         ....
         - 'dirstate.normal("f")'           N    ***
           (e.g. via dirty check
            or previous 'repo.commit()')

         - change "f", but keep size                    N
         - 'repo.status()' (*1)
           (via 'repo.commit()')
  ---- ----------------------------------- ---- ----- -----

At a glance, 'repo.status()' at (*1) seems to cause similar issue (=
"changed files are treated as clean"), but actually doesn't.

'dirstate._lastnormaltime' should be N at (*1) above, because
'dirstate.normal()' via dirty check is finished at N.

Therefore, "f" changed at N (= 'dirstate._lastnormaltime') is forcibly
treated as "unsure" at (*1), and changes are detected as expected (see
'dirstate.status()' for detail).

If 'hg import' is executed with '--no-commit', 'repo.status()' isn't
invoked just after changing files inside same wlock scope.

But preceding 'dirstate.normal()' is invoked inside another wlock
scope via 'cmdutil.bailifchanged()', and in-memory changes should be
flushed at the end of that scope.

Therefore, timestamp N of clean "f" should be replaced by -1, if
'dirstate.write()' is invoked at N. It means that condition of this
issue isn't satisfied.
2015-07-08 17:01:09 +09:00