file in nested directory causes unexpected abort.
problems below should be fixed for recursive normalization route in
dirstate._normalize():
1. rsplit() may cause unpacking into more than 2 elements.
it should be called with 'maxsplit' argument to unpack
into 'd, f'
2. 'd' is replaced by normalized value prefixed with
'self._root', but this makes 'folded' as absolute path,
and it is unexpected one for caller of recursive
normalization
on icasefs, "hg qnew" fails to import changing letter case of filename
already occurred in working directory, for example:
$ hg rename a tmp
$ hg rename tmp A
$ hg qnew casechange
$ hg status
R a
$
"hg qnew" invokes 'dirstate.walk()' via 'localrepository.commit()'
with 'exact match' matching object having exact filenames of targets
in ones 'files()'.
current implementation of 'dirstate.walk()' always normalizes letter
case of filenames from 'match.files()' on icasefs, even though exact
matching is required.
then, files only different in letter case are treated as one file.
this patch prevents 'dirstate.walk()' from normalizing, if exact
matching is required, even on icasefs.
filenames for 'exact matching' are given not from user command line,
but from dirstate walk result, manifest of changecontext, patch files
or fixed list for specific system files (e.g.: '.hgtags').
in such case, case normalization should not be done, so this patch
works well.
path to repo root may contains case sensitive part, even though repo
is located in case insensitive filesystem: e.g. repo in FAT32 device
mounted on Unix.
so, case normalized root causes failure of stat(2).
this patch uses case preserved root for 'util.fspath()' invocation to
avoid this problem.
case preserved root for 'util.fspath()' may decrease efficiency of
fspath cache, but 'util.fspath()' is currently called only from
dirstate, so this fix has less impact.
this patch adds 'dirs()' to changectx/workingctx, which returns map of
all directories deduced from manifest, to examine whether specified
pattern is related to the context as directory or not quickly.
'workingctx.dirs()' uses 'dirstate.dirs()' rather than building
another copy of it.
'dirstate._normalize()', the only caller of 'util.fspath()', has
already normcase()-ed path before invocation of it.
normcase()-ed root can be cached on dirstate side, too.
so, this patch changes 'util.fspath()' API specification to avoid
normcase()-ing in it.
at first of dirstate.walk() on case insensitive filesystem,
normalization of '.' causes util.fspath() invocation, but '.' is not
cached in it.
this invocation is not only useless, but also harmful: initial "hg
tag" causes creation of ".hgtags" file after dirstate.walk(), and
looking up ".hgtags" in cache will fail, because directory contents of
root is already cached at util.fspath() invocation for '.'.
Complex merges with divergent renames can cause a file to be 'moved'
twice, causing dirstate.drop() to be called twice. Rather than try to
ensure there are no unexpected corner cases where this can happen, we
simply ignore drops of files that aren't tracked.
Before this patch, Windows always did the wrong thing with exec bits
when committing a merge: consult the flags in first parent.
Now we manually recompute the result of merging flags at commit time,
which almost always does the right thing (except when there are
conflicts between symlink and exec flags).
To do this, we:
- pull flag synthesis out into its own function
- delay building this function unless it's needed
- add a merge case that compares flags in local and other against the ancestor
This has been tested in multiple ways on Linux:
- running the whole test suite with both old and new code in place,
checking for differences in each flags() result
- running the whole test suite while comparing real on-disk flags
against synthetic ones for merges
- test-issue1802 (from Martin Geisler) which disables exec bit
checking on Unix
The usual contract is that close() makes your writes permanent, so
atomictempfile's use of close() to *discard* writes (and rename() to
keep them) is rather unexpected. Thus, change it so close() makes
things permanent and add a new discard() method to throw them away.
discard() is only used internally, in __del__(), to ensure that writes
are discarded when an atomictempfile object goes out of scope.
I audited mercurial.*, hgext.*, and ~80 third-party extensions, and
found no one using the existing semantics of close() to discard
writes, so this should be safe.
It has substantially different semantics from forget at the command
layer, so change it to avoid confusion.
We can't simply combine it with remove because we need to explicitly
drop non-added files in some cases like commit.
These leaks may occur in environments that don't employ a reference
counting GC, i.e. PyPy.
This implies:
- changing opener(...).read() calls to opener.read(...)
- changing opener(...).write() calls to opener.write(...)
- changing open(...).read(...) to util.readfile(...)
- changing open(...).write(...) to util.writefile(...)
We can get rid of the _lastnormal set by using the filesystem mtimes to
identify the problematic "lastnormal" files on status(), forcing a file
content-comparison if the file's mtime timeslot is equal to _lastnormaltime.
- consistently use mtime as mapped to dirstate granularity (needed for
filesystems like NTFS, which have sub-second resolution)
- no need to add files with mtime < _lastnormaltime
- improve comments
(issue2264, issue2516)
The race happens when two commits in a row change the same file
without changing its size, *if* those two commits happen in the same
second in the same process while holding the same repo lock. For
example:
commit 1:
M a
M b
commit 2: # same process, same second, same repo lock
M b # modify b without changing its size
M c
This first manifested in transplant, which is the most common way to
do multiple commits in the same process. But it can manifest in any
script or extension that does multiple commits under the same repo
lock. (Thus, the test script tests both transplant and a custom script.)
The problem was that dirstate.status() failed to notice the change to
b when localrepo is about to do the second commit, meaning that change
gets left in the working directory. In the context of transplant, that
means either a crash ("RuntimeError: nothing committed after
transplant") or a silently inaccurate transplant, depending on whether
any other files were modified by the second transplanted changeset.
The fix is to make status() work a little harder when we have
previously marked files as clean (state 'normal') in the same process.
Specifically, dirstate.normal() adds files to self._lastnormal, and
other state-changing methods remove them. Then dirstate.status() puts
any files in self._lastnormal into state 'lookup', which will make
localrepository.status() read file contents to see if it has really
changed. So we pay a small performance penalty for the second (and
subsequent) commits in the same process, without affecting the common
case. Anything that does lots of status updates and checks in the
same process could suffer a performance hit.
Incidentally, there is a simpler fix: call dirstate.normallookup() on
every file updated by commit() at the end of the commit. The trouble
with that solution is that it imposes a performance penalty on the
common case: it means the next status-dependent hg command after every
"hg commit" will be a little bit slower. The patch here is more
complex, but only affects performance for the uncommon case.
Add missing calls to close() to many places where files are
opened. Relying on reference counting to catch them soon-ish is not
portable and fails in environments with a proper GC, such as PyPy.
split can be more readable for longer lists like the list in
dirstate.invalidate. As dirstate.invalidate is used in wlock() and therefoe
used heavily, I think it's worth avoiding a split there too.
Previously, branch names were ideally manipulated as UTF-8 strings,
because they were stored as UTF-8 in the dirstate and the changelog
and could not be safely converted to the local encoding and back.
However, only about 80% of branch name code was actually using the
right encoding conventions. This patch uses the localstr addition to
allow working on branch names as local strings, which simplifies
handling so that the previously incorrect code becomes correct.
This gives the repository control over which nested repository paths
that should be allowed via the custom path auditor.
Since paths into subrepositories are now allowed, dirstate.walk must
now filter away more paths than before.
When the filesystem cannot handle the executable bit, we currently
ignore it completely when looking for modified files. Similarly, it is
impossible to set or clear the bit when the filesystem ignores it.
This patch makes Mercurial treat symbolic links the same way.
Symlinks are a little different since they manifest themselves as
small files containing a filename (the symlink target). On Windows,
these files show up as regular files, and on Linux and Mac they show
up as real symlinks.
Issue1888 presents a case where the symlink files are better ignored
from the Windows side. A Linux client creates symlinks in a working
copy which is shared over a network between Linux and Windows clients.
The Samba server is helpful and defererences the symlink when the
Windows client looks at it. This means that Mercurial on the Windows
side sees file content instead of a file name in the symlink, and
hence flags the link as modified. Ignoring the change would be much
more helpful, similarly to how Mercurial does not report any changes
when executable bits are ignored in a checkout on Windows.
An initial checkout of a symbolic link on a file system that cannot
handle symbolic links will still result in a regular file containing
the target file name as its content. Sharing such a checkout with a
Linux client will not turn the file into a symlink automatically, but
'hg revert' can fix that. After the revert, the Windows client will
see the correct file content (provided by the Samba server when it
follows the link on the Linux side) and otherwise ignore the change.
Running 'hg perfstatus' 10 times gives these results:
Before: After:
min: 0.544703 min: 0.546549
med: 0.547592 med: 0.548881
avg: 0.549146 avg: 0.548549
max: 0.564112 max: 0.551504
The median time is increased about 0.24%.
