Garbage collection behave pathologically when creating a lot of containers. As
we do that more than once it become sensible to have a decorator for it. See
inline documentation for details.
This patch uses "False" as default value of "notindexed" argument,
even though "vfs.makedir()" uses "True" for it, because "os.mkdir()"
doesn't set "_FILE_ATTRIBUTE_NOT_CONTENT_INDEXED" attribute to newly
created directories.
Future patches will allow extensions to choose which order a namespace should
output in the log, so we add a way for sortdict to insert to a specific
location.
Future patches will start using sortdict for log operations where order is
important. Adding iteritems removes the headache of having to remember to use
items() if the object is a sortdict.
Previously, we'd scan through the entire directory listing looking for a
normalized match. This is O(N) in the number of files in the directory. If we
decide to call util.fspath on each file in it, the overall complexity works out
to O(N^2). This becomes a problem with directories a few thousand files or
larger.
Switch to using a dictionary instead. There is a slightly higher upfront cost
to pay, but for cases like the above this is amortized O(1). Plus there is a
lower constant factor because generator comprehensions are faster than for
loops, so overall it works out to be a very small loss in performance for 1
file, and a huge gain when there's more.
For a large repo with around 200k files in it on a case-insensitive file
system, for a large directory with over 30,000 files in it, the following
command was tested:
ls | shuf -n $COUNT | xargs hg status
This command leads to util.fspath being called on $COUNT files in the
directory.
COUNT before after
1 0.77s 0.78s
100 1.42s 0.80s
1000 6.3s 0.96s
I also tested with COUNT=10000, but before took too long so I gave up.
util.system() copies subprocess' output through pipe if output file is not
stdout. Because a file iterator has internal buffering, output won't be
flushed until enough data is available. Therefore, it could easily miss
important messages such as "waiting for lock".
This effectively backs out changeset 7582042d6cce.
The API change is done so that both util.sha1 and util.md5 can be called the
same way. The function is moved in order to use it for md5 checksumming for
an upcoming bundle2 feature.
templates, help and locale data is normally stored as sub folders in the
directory containing the source of the mercurial module. In a frozen build they
live as sub folders next to 'hg.exe' and 'library.zip'.
These different kind of data were handled in different ways. Unify that by
introducing util.datapath. The value is computed from the environment and is
always used, so we just calculate the value on module load.
Reading all available data from a pipe has a platform-dependent
implementation.
This patch establishes platform.readpipe() by copying the
inline implementation in sshpeer.readerr(). The implementations
for POSIX and Windows are currently identical. The POSIX
implementation will be changed in a subsequent patch.
The escape method in at least one of the modules called 're2' is in C. This
means it is significantly faster than the Python code written in 're'.
An upcoming patch will have benchmarks.
Before this patch, 'util.ellipsis' tried to avoid splitting at
intermediate multi-byte sequence, but its implementation was incorrect.
Internal function '_ellipsis' trims specified unicode sequence not at
most maxlength 'columns in display', but at most maxlength number of
'unicode characters'.
def _ellipsis(text, maxlength):
if len(text) <= maxlength:
return text, False
else:
return "%s..." % (text[:maxlength - 3]), True
In many encodings, number of unicode characters can be different from
columns in display.
This patch replaces 'ellipsis' implementation by 'encoding.trim',
which can trim string at most maxlength columns in display correctly,
even though specified string contains multi-byte characters.
'_ellipsis' is removed in this patch, because it is referred only from
'ellipsis'.
Before this patch, "util.cachefunc()" caches the value returned by the
specified function into dictionary "cache", even if the specified
function takes no arguments.
In such case, "cache" has at most one entry, and distinction between
entries in "cache" is meaningless.
This patch adds the code path to "cachefunc()" for the function taking
no arguments for efficiency: to store only one cached value, using
list "cache" is a little faster than using dictionary "cache".
Close another stream (default stdout, which often is buffered) before writing
to the primary stream (default stderr, which often is unbuffered). The primary
stream is also flushed after writing (in case it is buffered).
This fixes non-deterministic output order, especially on windows.
This is often very handy when hacking/debugging.
Calling util.debugstacktrace('hey') from a place in hg will give something like:
hey at:
./hg:38 in <module>
/home/user/hgsrc/mercurial/dispatch.py:28 in run
/home/user/hgsrc/mercurial/dispatch.py:65 in dispatch
/home/user/hgsrc/mercurial/dispatch.py:88 in _runcatch
/home/user/hgsrc/mercurial/dispatch.py:740 in _dispatch
/home/user/hgsrc/mercurial/dispatch.py:514 in runcommand
/home/user/hgsrc/mercurial/dispatch.py:830 in _runcommand
/home/user/hgsrc/mercurial/dispatch.py:801 in checkargs
/home/user/hgsrc/mercurial/dispatch.py:737 in <lambda>
/home/user/hgsrc/mercurial/util.py:472 in check
...
Backslashes (\) in paths were encoded to %C5 when converting from url to
string. This does not look nice for windows paths. And it introduces many
problems when running tests on windows.
Paths ending with \ will fail the verification introduced in 0bc0c17d663e when
checking out on Windows ... and if it didn't fail it would probably not do what
the user expected.
Propertycache used standard attribute assignment. In the repoview case, this
assignment was forwarded to the unfiltered repo. This result in:
(1) unfiltered repo got a potentially wrong cache value,
(2) repoview never reused the cached value.
This patch replaces the standard attribute assignment by an assignment to
`objc.__dict__` which will bypass the `repoview.__setattr__`. This will not
affects other `propertycache` users and it is actually closer to the semantic we
need.
The interaction of `propertycache` and `repoview` are now tested in a python
test file.
When running 'hg pull --rebase', I was seeing this exception 100% of the
time as the python process was closing down:
Exception TypeError: TypeError("'NoneType' object is not callable",) in
<bound method Popen.__del__ of <subprocess.Popen object at 0x937c10>> ignored
By storing the subprocess on the sshpeer, the subprocess seems to clean up
correctly, and I no longer see the exception. I have no idea why this actually
works, but I get a 0% repro if I store the subprocess in self.subprocess,
and a 100% repro if I store None in self.subprocess.
Possibly related to issue 2240.
I often want to measure the cost of a function call before/after
an optimization, where using top level "hg --time" timing introduces
enough other noise that I can't tell if my efforts are having an
effect.
This decorator allows a developer to measure a function's cost with
finer granularity.
With the new parallel update code, it is possible for multiple
workers to try to create a hierarchy of directories at the same
time. This is hard to trigger in general, but most likely during
initial checkout.
To deal with these races, we introduce a new ensuredirs function
whose contract is to ensure that a directory hierarchy exists - it
will ignore a failure that implies that the desired directory already
exists.
In certain cases we would like to have a cache of the last N results of a
given computation, where N is small. This will be used in an upcoming patch to
increase the size of the manifest cache from 1 to 3.
Adding support to parsedate in util module to understand the more idiomatic
dates 'today' and 'yesterday'.
Added unified tests and docstring tests for added functionality.
This makes a big difference to performance.
In a clean working directory containing 170,000 files, performance of
"hg --time diff" improves from 2.38 seconds to 1.69.
Some of the localrepo property caches must be computed unfiltered and
stored globally. Some others must see the filtered version and store data
relative to the current filtering.
This changeset introduces two classes `unfilteredpropertycache`
and `filteredpropertycache` for this purpose. A new function
`hasunfilteredcache` is introduced for unambiguous checking for cached
values on unfiltered repos.
A few tweaks are made to the property cache class to allow overriding
the way the computed value is stored on the object.
Some logic relative to _tagcaches is cleaned up in the process.
The old str-based += collector performed very nicely on Linux, but
turns out to be quadratically expensive on Windows, causing
chunkbuffer to dominate in profiles.
This list-based version has been measured to significantly improve
performance with large chunks on Windows, with negligible overall
overhead on Linux (though microbenchmarks show it to be about 50% slower).
This may increase memory overhead where += didn't behave quadratically. If we
want to gather up 1G of data to join, we temporarily have 1G in our
list and 1G in our string.
bffd8f8dfc85 claims this was needed "to avoid cyclic dependency", but there is
no cyclic dependency.
windows.py already imports encoding, posix.py can import it too, so we can
simply use encoding.upper in windows.py and in posix.py.
(this is a partial backout of bffd8f8dfc85)
There are two sets of Python re2 bindings available on the internet;
this code works with both.
Using re2 can greatly improve "hg status" performance when a .hgignore
file becomes even modestly complex.
Example: "hg status" on a clean tree with 134K files, where "hg
debugignore" reports a regexp 4256 bytes in size.
no .hgignore: 1.76 sec
Python re: 2.79
re2: 1.82
The overhead of regexp matching drops from 1.03 seconds with stock
re to 0.06 with re2.
(For comparison, a git repo with the same contents and .gitignore
file runs "git status -s" in 1.71 seconds, i.e. only slightly faster
than hg with re2.)
There have been quite a few places where we pop elements off the
front of a list. This can turn O(n) algorithms into something more
like O(n**2). Python has provided a deque type that can do this
efficiently since at least 2.4.
As an example of the difference a deque can make, it improves
perfancestors performance on a Linux repo from 0.50 seconds to 0.36.
This patch contains support for Plan 9 from Bell Labs. A README is
provided in contrib/plan9 which describes the port in greater detail.
A new extension is also provided named factotum which permits the
factotum(4) authentication agent to provide credentials for HTTP
repositories. This extension is also applicable to other POSIX
platforms which make use of Plan 9 from User Space (aka plan9ports).
Currently, the 'user' filter is using util.shortuser(text) (which clearly
doesn't extract only the user portion of an email address, even though the
help text says it does).
The new 'emailuser' filter uses the new util.emailuser(text) function which,
instead, does exactly that.
The help text on the 'user' filter has been modified accordingly.
this patch disuses length check against un-normcase()-ed filenames
gotten by "os.listdir()", because there is no assurance that
filesystem stores filenames normalized except in letter case, even
though some case insensitive filesystems (in some environment, for
some language setting) store them in such manner.
'dirstate._normalize()', the only caller of 'util.fspath()', has
already confirmed exsistance of specified file as relative to root.
so, this patch omits path-absoluteness/existance check from
'util.fspath()'.
some hg operation (e.g.: qpush) create new files after first
dirstate.walk()-ing, and it invalidates _fspathcache for fspath().
then, fspath() will fail to look up specified name in _fspathcache.
this causes case preservation breaking, because parts of already
normcase()-ed path are used as result at that time.
in this case, file creation and writing out should be done before
fspath() invocation, so the second invocation of os.listdir() has not
so much impact on runtime performance.
this patch uses encoding.lower/upper for case folding, because ones of
str can not fold case of non ascii characters correctly.
to avoid cyclic dependency and to encapsulate logic of normcase in
each platforms, this patch introduces encodinglower/encodingupper in
both posix/windows specific files.
this patch does not change implementation of normcase() in posix.py,
because we do not know the encoding of filenames on POSIX.
some "normcase()" are excluded from function wrap list in
hgext/win32mbcs.py, because they become encoding aware by this patch.
'dirstate._normalize()', the only caller of 'util.fspath()', has
already normcase()-ed path before invocation of it.
normcase()-ed root can be cached on dirstate side, too.
so, this patch changes 'util.fspath()' API specification to avoid
normcase()-ing in it.
Before:
>>> str(url('file:///c:/tmp/foo/bar'))
'file:c%3C/tmp/foo/bar'
After:
>>> str(url('file:///c:/tmp/foo/bar'))
'file:///c%3C/tmp/foo/bar'
The previous behaviour had no effect on mercurial itself (clone command for
instance) because we fortunately called .localpath() on the parsed URL.
hgsubversion was not so lucky and cloning a local subversion repository on
Windows no longer worked on the default branch (it works on stable because
2b62605189dc defeats the hasdriveletter() test in url class).
I do not know if the %3C is correct or not but svn accepts file:// URLs
containing it. Mads fixed it in 2b62605189dc, so we can always backport should
the need arise.
Python's time module sets timezone and altzone based on UTC offsets of
two dates: first and middle day of the current year. This approach
doesn't work on a year when DST rules change.
For example Russia abandoned winter time this year, so the correct UTC
offset should be +4 now, but time.timezone returns 3 hours difference
because that's what it was on 01.01.2011.
Related python issue: http://bugs.python.org/issue1647654
It was very elegant that httpsendfile implemented __len__ like a string. It was
however also dangerous because that protocol can't handle sizes bigger than 2 GB.
Mercurial tried to work around that, but it turned out to be too easy to
introduce new errors in this area.
With this change __len__ is no longer implemented at all and the code will work
the same way for short and long posts.
neither number of 'bytes' in any encoding nor 'characters' is
appropriate to calculate terminal columns for specified string.
this patch modifies MBTextWrapper for:
- overriding '_wrap_chunks()' to make it use not built-in 'len()'
but 'encoding.colwidth()' for columns of string
- fixing '_cutdown()' to make it use 'encoding.colwidth()' instead
of local, similar but incorrect implementation
this patch also modifies 'encoding.py':
- dividing 'colwith()' into 2 pieces: one for calculation columns of
specified UNICODE string, and another for rest part of original
one. the former is used from MBTextWrapper in 'util.py'.
- preventing 'colwidth()' from evaluating HGENCODINGAMBIGUOUS
configuration per each invocation: 'unicodedata.east_asian_width'
checking is kept intact for reducing startup cost.
This re-introduces the unicode conversion what was lost in e5976ee55f4b 5 years
ago and had the comment:
To avoid corrupting multi-byte characters in line, we must wrap
a Unicode string instead of a bytestring.
urllib2 password manager does not strip credentials from URIs registered with
add_password() and compare them with stripped URIs in find_password(). Remove
credentials from URIs returned by util.url.authinfo(). It sometimes works when
no port was specified as the URI host is registered too.
8264e5172141 made sure that paths that seemed to start with a windows drive
letter would not get an extra leading slash.
localpath should thus not try to handle this case by removing a leading slash,
and this special handling is thus removed.
(The localpath handling of this case was wrong anyway, because paths that look
like they start with a windows drive letter can't have a leading slash.)
A quick verification of this is to run 'hg id file:///c:/foo/bar/'.
The usual contract is that close() makes your writes permanent, so
atomictempfile's use of close() to *discard* writes (and rename() to
keep them) is rather unexpected. Thus, change it so close() makes
things permanent and add a new discard() method to throw them away.
discard() is only used internally, in __del__(), to ensure that writes
are discarded when an atomictempfile object goes out of scope.
I audited mercurial.*, hgext.*, and ~80 third-party extensions, and
found no one using the existing semantics of close() to discard
writes, so this should be safe.
Before, makedirs could call itself recursively with the same path name it was
given, relying on sane file system behavior to terminate the recursion. That
could cause infinite recursion on insane file systems.
Instead we now call mkdir explicitly after having created parent directory
recursively. Exceptions from this mkdir is not swallowed.
This re-introduces the unicode conversion what was lost in e5976ee55f4b 5 years
ago and had the comment:
To avoid corrupting multi-byte characters in line, we must wrap
a Unicode string instead of a bytestring.
reducing it to a NOP on Windows.
This eliminates a pointless stat call on Windows and reduces the risk of
interferring with other processes (e.g. AV-scanners, file change watchers).
See also http://mercurial.selenic.com/wiki/UnlinkingFilesOnWindows, item 2d
util is never imported by any other name than util, so this is mostly just a
simple search and replace from util.localpath to util.urllocalpath (assuming
other uses of util.localpath already has been renamed).
str(url) was recently changed to return only file:/. However, the
canonical way to represent absolute local paths is file:/// [1], which
is also expected by at least hgsubversion.
Relative paths are returned as file:the/relative/path.
[1] http://en.wikipedia.org/wiki/File_URI_scheme
This class contains a stat result, and possibly other file info to reliably
determine between two points in time whether a file has changed.
Uniquely identifying a file gives us that reliability because we either
atomic rename or append. So one of two will happen: the file 'id' will change,
or the size of the file will change.
posix implements it simply by calling os.stat() and checking if the result
has st_ino.
For now on Windows we always assume the path is uncacheable. This can be
improved on NTFS due to file IDs: http://msdn.microsoft.com/en-us/library/aa363788(v=vs.85).aspx
So we need to find out if a file path is on an NTFS drive, for that we have:
- GetVolumeInformation, which unfortunately only works with a root path (but is available on XP)
- GetVolumeInformationByHandleW, works on a full file path but requires Vista or higher
The ui class uses util.hasscheme() in a couple of places, causing hg
to import urllib even when it doesn't need to. This copies
urllib.unquote() to avoid that import.
perfstartup time before the URL refactoring (707e4b1e8064):
! wall 0.050692 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
before this change:
! wall 0.064742 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
after this change:
! wall 0.052126 comb 0.000000 user 0.000000 sys 0.000000 (best of 100
The introduction of the new URL parsing code has created a startup
time regression. This is mainly due to the use of url.hasscheme() in
the ui class. It ends up importing many libraries that the url module
requires.
This fix helps marginally, but if we can get rid of the urllib import
in the URL parser all together, startup time will go back to normal.
perfstartup time before the URL refactoring (707e4b1e8064):
! wall 0.050692 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
current startup time (9ad1dce9e7f4):
! wall 0.070685 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
after this change:
! wall 0.064667 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
The problem is that a programmer using atomictempfile directly can
make an innocent everyday mistake -- not enough args to the
constructor -- which escalates badly. You would expect a simple
TypeError crash in that case, but you actually get an infinite
recursion that is surprisingly difficult to kill: it happens between
__del__() and __getattr__(), and Python does not handle infinite
recursion from __del__() well.
The fix is to not implement __getattr__(), but instead assign instance
attributes for the methods we wish to delegate to the builtin file
type: write() and fileno(). I've audited mercurial.* and hgext.* and
found no users of atomictempfile using methods other than write() and
rename(). I audited third-party extensions and found one (snap)
passing an atomictempfile to util.fstat(), so I also threw in
fileno().
The last time I submitted a similar patch, Matt proposed that we make
atomictempfile a subclass of file instead of wrapping it. Rejected on
grounds of unnecessary complexity: for one thing, it would make the
Windows implementation of posixfile quite a bit more complex. It would
have to become a subclass of file rather than a simple function -- but
since it's written in C, this is non-obvious and non-trivial.
Furthermore, there's nothing wrong with wrapping objects and
delegating methods: it's a well-established pattern that works just
fine in many cases. Subclassing is not the answer to all of life's
problems.
path_auditor is used for checking patterns too, but a pattern is not a valid
filename.
This patch fixes fb1792e89e34, which introduced the bug:
$ hg log -l3 glob:**.py
abort: filename contains '*', which is reserved on Windows: mercurial\**.py
Example (on Windows):
$ hg parents
$ hg manifest tip
con.xml
$ hg update
abort: filename contains 'con', which is reserved on Windows: con.xml
Before this patch, update produced (as explained in issue2755):
$ hg update
abort: No usable temporary filename found
I've added the new function checkwinfilename to util.py and not to windows.py,
so that we can later call it when running on posix platforms too, for when we
decide to implement a (configurable) warning message on 'hg add'.
As per this patch, checkwinfilename is currently only used when running
on Windwows.
path_auditor calls checkosfilename, which is a NOP on posix and an alias for
checkwinfilename on Windows.
Python added support for Windows 6.0 (Vista) symbolic links in 3.2 [1], but
even these symbolic links aren't what we can expect from a canonical
symbolic link, since creation requires SeCreateSymbolicLinkPrivilege,
which typically only admins have.
So we can safely assume that we don't have symbolic links on Windows.
[1] http://docs.python.org/py3k/library/os.html#os.symlink
The use of "{datetime}" was unfortunate since I as a user never knew
if I was expected to do
hg log -d '>{2011-04-01}'
or
hg log -d '>2011-04-01'
The word "datetime" is also confusing -- calling it a date it much
simpler.
This replaces util.drop_scheme() with url.localpath(), using url.url for
parsing instead of doing it on its own. The function is moved from
util to url to avoid an import cycle.
hg.localpath() is removed in favor of using url.localpath(). This
provides more consistent behavior between "hg clone" and other
commands.
To preserve backwards compatibility, URLs like bundle://../foo still
refer to ../foo, not /foo.
If a URL contains a scheme, percent-encoded entities are decoded. When
there's no scheme, all characters are left untouched.
Comparison of old and new behaviors:
URL drop_scheme() hg.localpath() url.localpath()
=== ============= ============== ===============
file://foo/foo /foo foo/foo /foo
file://localhost:80/foo /foo localhost:80/foo /foo
file://localhost:/foo /foo localhost:/foo /foo
file://localhost/foo /foo /foo /foo
file:///foo /foo /foo /foo
file://foo (empty string) foo /
file:/foo /foo /foo /foo
file:foo foo foo foo
file:foo%23bar foo%23bar foo%23bar foo#bar
foo%23bar foo%23bar foo%23bar foo%23bar
/foo /foo /foo /foo
Windows-related paths on Windows:
URL drop_scheme() hg.localpath() url.localpath()
=== ============= ============== ===============
file:///C:/foo C:/C:/foo /C:/foo C:/foo
file:///D:/foo C:/D:/foo /D:/foo D:/foo
file://C:/foo C:/foo C:/foo C:/foo
file://D:/foo C:/foo D:/foo D:/foo
file:////foo/bar //foo/bar //foo/bar //foo/bar
//foo/bar //foo/bar //foo/bar //foo/bar
\\foo\bar //foo/bar //foo/bar \\foo\bar
Windows-related paths on other platforms:
file:///C:/foo C:/C:/foo /C:/foo C:/foo
file:///D:/foo C:/D:/foo /D:/foo D:/foo
file://C:/foo C:/foo C:/foo C:/foo
file://D:/foo C:/foo D:/foo D:/foo
file:////foo/bar //foo/bar //foo/bar //foo/bar
//foo/bar //foo/bar //foo/bar //foo/bar
\\foo\bar //foo/bar //foo/bar \\foo\bar
For more information about file:// URL handling, see:
http://www-archive.mozilla.org/quality/networking/testing/filetests.html
Related issues:
- issue1153: File URIs aren't handled correctly in windows
This patch should preserve the fix implemented in
5c92d05b064e. However, it goes a step further and "promotes"
Windows-style drive letters from being interpreted as host names to
being part of the path.
- issue2154: Cannot escape '#' in Mercurial URLs (#1172 in THG)
The fragment is still interpreted as a revision or a branch, even in
paths to bundles. However, when file: is used, percent-encoded
entities are decoded, so file:test%23bundle.hg can refer to
test#bundle.hg ond isk.
The previous test assumed that 'os.name' was "mac" on Mac OS X. This
is not the case; 'mac' was classic Mac OS, whereas Mac OS X has 'os.name'
be 'posix'.
Please note that this change will break Mercurial on hypothetical
non-Mac OS X deployments of Darwin.
Credit to Brodie Rao for thinking of CGSessionCopyCurrentDictionary()
and Kevin Bullock for testing.
Add missing calls to close() to many places where files are
opened. Relying on reference counting to catch them soon-ish is not
portable and fails in environments with a proper GC, such as PyPy.
The pywin32 package is no longer needed.
ctypes is now required for running Mercurial on Windows.
ctypes is included in Python since version 2.5. For Python 2.4, ctypes is
available as an extra installer package for Windows.
Moved spawndetached() from windows.py to win32.py and fixed it, using
ctypes as well. spawndetached was defunct with Python 2.6.6 because Python
removed their undocumented subprocess.CreateProcess. This fixes
'hg serve -d' on Windows.
If pywin32 is not installed, 'os.lstat(pathname).st_nlink' is used for
nlinks(), which is always zero for all files on Windows.
To make sure we break up hardlinks if pywin32 is missing, we force
nlink = 2 if nlinks() returns < 1.
(this completely fixes issue1922)
It tries to convert localstr to unicode before truncating.
Because we cannot assume that the given text is encoded in local encoding,
it falls back to raw string in case of unicode error.
In a date like 10:30, there are two underspecified ends: the specific
end (seconds) and the broad end (day, month, year). When matching
"10:30", we need to allow the specific end to go from 0 to 59 seconds,
while the broad end is assumed to be today's date.
Similar handling applies for a date range like "Mar 1": year is fixed
to today, any time matches.
Preventing file loss repository corruption (e.g. vanished changelog.i) when
Mercurial pushes to repositories on Windows shares served by Samba.
This is a workaround for Samba bug 7863, which is present in current latest
stable Samba 3.5.6 and various prior versions down to 3.0.26a (the oldest one
I tested).
Of course this should be fixed in Samba, but there probably aren't that many
other applications who use hardlinks that extensively and keep files open like
Mercurial, so the pressure to fix this on Samba is probably not that high. And
even if the Samba project should be able to fix their bug within a month or
two, it will take quite some time until users upgrade their Samba installs.
split can be more readable for longer lists like the list in
dirstate.invalidate. As dirstate.invalidate is used in wlock() and therefoe
used heavily, I think it's worth avoiding a split there too.
If Linux is asked to open a filename with a trailing directory separator,
e.g. "foo/", the open fails with EISDIR. On AIX, the open succeeds, opening
file "foo". This causes test-mq-qnew to fail on AIX.
Fix by adding 'ends with directory separator' to the conditions checked
by the path auditor. Change test to expect auditor fail message.
Python's time.gmtime(lt) fails on Windows, producing a traceback with
ValueError: (22, 'Invalid argument')
if lt < -43200.
We get a local time boundary value of -43200 if we take "the epoch"
Thu Jan 01 00:00:00 1970 = time.gmtime(0)
from timezone 'UTC+0' into timezone 'UTC-12'. All other timezones will have
larger local time values for that point in time.
Aborting with a traceback on 'hg log' for revisions with a timestamp value
< -43200 is clearly not acceptable.
Returning "invalid timestamp" or similar as string representation is not an
option either, since that may crash other tools which parse the output of
'hg log'.
Instead, we teach util.datestr() to return the epoch in timezone UTC+0 on
*all platforms*, represented by the string
Thu Jan 01 00:00:00 1970 +0000
if the timestamp's unix time value is negative.
(based on a patch originally proposed by Benjamin Pollack)
If os_link fails on Windows, errno is always errno.EINVAL,
so we can't really say if the testlink could not be created
because (a) the FS doesn't support hardlinks or (b) there
is a leaked .hgtmp file lying around from a previous crashed
run.
So let's err on the safe side, keep the code simple and assume
we can't detect hardlinks in both cases.
The Linux CIFS kernel driver (even in 2.6.36) suffers from a hardlink
count blindness bug (lstat() returning 1 in st_nlink when it is expected
to return >1), which causes repository corruption if Mercurial running
on Linux pushes or commits to a hardlinked repository stored on a Windows
share, if that share is mounted using the CIFS driver.
This patch works around issue1866 and improves the workaround done in
65e082ae3076 to fix issue761, by teaching the opener to lazily execute a
runtime check (new function checknlink) to see if the hardlink count
reported by nlinks() can be trusted.
Since nlinks() is also known to return varying count values (1 or >1)
depending on whether the file is open or not and depending on what client
and server software combination is being used for accessing and serving
the Windows share, we deliberately open the file before calling nlinks() in
order to have a stable precondition. Trying to depend on the precondition
"file closed" would be fragile, as the file could have been opened very
easily somewhere else in the program.
- Don't call atomictempfile or nlinks() if the path is malformed
(no basename). Let posixfile() raise IOError directly.
- atomictempfile already breaks up hardlinks, no need to poke
at the file with nlinks() if atomictemp.
- No need to copy the file contents to break hardlinks for 'w'rite
modes (w, wb, w+, w+b). Unlinking and recreating the file is faster.
226847bf9cab updated copyfile to also copy over atimes and
mtimes. That behavior is specifically to trick editors into thinking
files that hg record has modified haven't changed. We don't really
care about preserving times in the general case.
The canonpath function will default to creating its own path auditor,
but in some cases it will be useful to use a specialized auditor,
e.g., one that wont abort if a path lies within a subrepository.
This adds util.getport(port) which tries to parse port as an int, and
failing that, looks it up using socket.getservbyname(). Thus, the
following will work:
[smtp]
port = submission
[web]
port = http
This does not apply to ports in URLs used in clone, pull, etc.
The following patch allows the use of python2.4 with a standalone
hashlib rather than assuming that python2.5 is in use when hashlib is
imported successfully.
util.interpolate can be used to replace multiple items in a string all at once
(and optionally apply a function to the replacement), without worrying about
recursing:
>>> import util
>>> s = '$foo, $spam'
>>> util.interpolate(r'\$', { 'foo': 'bar', 'spam': 'eggs' }, s)
'bar, eggs'
>>> util.interpolate(r'\$', { 'foo': 'spam', 'spam': 'foo' }, s)
'spam, foo'
>>> util.interpolate(r'\$', { 'foo': 'spam', 'spam': 'foo' }, s, lambda s: s.upper())
'SPAM, FOO'
The patch also changes filemerge.py to use this new function.
The following patch allows the use of python2.4 with a standalone
hashlib rather than assuming that python2.5 is in use when hashlib is
imported successfully.
