Emulating POSIX unlink() behavior with os.rename() and os.unlink() is often slow
especially over CIFS from Windows clients due to its protocol overhead. This
patch changes win32.unlink() to try first an exclusive open with the Win32
delete-on-close flag, and if a sharing violation is detected, to fall back to
the original emulation.
This patch also removes a test with os.path.isdir() since we expect opening a
directory shall fail as os.unlink() would.
Example measurements (repeated 3-times after 1-time calibration):
(Without this patch: hg update from null to default)
127 files updated, 0 files merged, 0 files removed, 0 files unresolved
time: real 19.871 secs (user 0.328+0.000 sys 1.794+0.000)
time: real 19.622 secs (user 0.312+0.000 sys 2.044+0.000)
time: real 19.138 secs (user 0.250+0.000 sys 1.872+0.000)
(Without this patch: hg update from default to null)
0 files updated, 0 files merged, 127 files removed, 0 files unresolved
time: real 35.158 secs (user 0.156+0.000 sys 2.512+0.000)
time: real 35.272 secs (user 0.250+0.000 sys 2.512+0.000)
time: real 36.569 secs (user 0.203+0.000 sys 2.387+0.000)
(With this patch: hg update from null to default)
127 files updated, 0 files merged, 0 files removed, 0 files unresolved
time: real 17.893 secs (user 0.328+0.000 sys 1.700+0.000)
time: real 18.512 secs (user 0.265+0.000 sys 1.529+0.000)
time: real 20.238 secs (user 0.312+0.000 sys 1.685+0.000)
(With this patch: hg update from default to null)
0 files updated, 0 files merged, 127 files removed, 0 files unresolved
time: real 12.312 secs (user 0.250+0.000 sys 0.811+0.000)
time: real 12.471 secs (user 0.156+0.000 sys 0.889+0.000)
time: real 9.727 secs (user 0.125+0.000 sys 0.858+0.000)
win32.spawndetached starts the detached process by `cmd.exe` (or COMSPEC). The
pid it returned was the one of cmd.exe and not the one of the detached process.
When this pid is used to kill the process, the detached process is not killed,
but only cmd.exe.
With this patch the pid of the detached process is written to the pid file.
Killing the process works as expected.
The pid is only evaluated on writing the pid file. It is unnecessary to search
the pid when it is not needed. And more important, it probably does not yet
exist right after the cmd.exe process was started. When the pid is written to
the file, waiting for the start of the detached process has already happened.
Use this functionality instead of writing a 2nd wait function.
Many tests on windows will not fail anymore, all those with the first failing
line "abort: child process failed to start". (The processes still hanging
around from previous test runs have to be killed first. They still block a
tcp port.)
A good test for the functionality of this patch is test-treediscovery.t,
because it starts and kills `hg serve -d` several times.
Creation and writing into target file via vfs (a.k.a opener) is done
after "unlink()" target file, if it exists.
For example, it is assumed that the revision X consists of file 'A',
and the revision Y consists of file 'A/B'. Merging revision X into Y
tries to "unlink()" on directory 'A' of 'A/B', before creation of file
'A'.
On POSIX environment, directories should be removed by "rmdir(2)", and
"unlink(2)" on directories fails. "unlink()" of Mercurial (and Python)
uses "unlink(2)" directly, so unlinking in the merge case above would
fail.
In the other hand, on Windows environment, "unlink()" of Mercurial
tries to rename before actual unlinking, to follow POSIX semantics:
already opened file can be unlinked safely.
This causes unexpected success in unlinking in the merge case above,
even though directory 'A' is renamed to another. This confuses users.
This patch checks whether target is directory or not before renaming,
and raises IOError(errno.EPERM) if so, to follow POSIX semantics.
spawndetached() was the only user of _STARTF_USESHOWWINDOW and it creates the
process with _CREATE_NO_WINDOW anyway. If the process has no window, then
there is nothing to hide.
Before this change, a console window briefly popped up on "hg serve -d" and
disappeared again, stealing the focus window (which was very annyoing when
running tests).
Specifying _CREATE_NO_WINDOW instead of _DETACHED_PROCESS fixes this (as tested
on Windows 7 x64).
This is a partial backout of 7efea3b5db4c.
7efea3b5db4c switched win32.py to using ctypes with the intention to get rid
of the dependency on the pywin32 package.
But 7efea3b5db4c replaced the usage of the Python standard module _winreg in
lookup_reg as well, which was uneeded (note that lookup_reg was later renamed
into lookupreg).
Basically, we're switching back to the previous _winreg-based implementation,
which uses _winreg.QueryValueEx(). QueryValueEx returns a unicode code string.
See also: issue3467
This is a feature of ctypes. Without these, pypy complains with
RuntimeWarning: C function without declared arguments called
RuntimeWarning: C function without declared return type called
As a side effect of specifying restypes, the return value of e.g. CreateFileA
is now implicitly converted to an instance of _HANDLE, so we also need to
change the definition
_INVALID_HANDLE_VALUE = -1
to
_INVALID_HANDLE_VALUE = _HANDLE(-1).value
Otherwise, tests for equality to _INVALID_HANDLE_VALUE in code like
def _getfileinfo(name):
fh = _kernel32.CreateFileA(name, 0,
_FILE_SHARE_READ | _FILE_SHARE_WRITE | _FILE_SHARE_DELETE,
None, _OPEN_EXISTING, 0, None)
if fh == _INVALID_HANDLE_VALUE:
_raiseoserror(name)
would now fail to detect an invalid handle, which in turn would lead to
exceptions raised with wrong errno values, like e.g.
>>> nlinks('missing.txt')
Traceback (most recent call last):
...
OSError: [Errno 9] missing.txt: The handle is invalid.
instead of the correct (as per this patch and before it)
>>> nlinks('missing.txt')
Traceback (most recent call last):
...
OSError: [Errno 2] missing.txt: The system cannot find the file specified.
The pywin32 package is no longer needed.
ctypes is now required for running Mercurial on Windows.
ctypes is included in Python since version 2.5. For Python 2.4, ctypes is
available as an extra installer package for Windows.
Moved spawndetached() from windows.py to win32.py and fixed it, using
ctypes as well. spawndetached was defunct with Python 2.6.6 because Python
removed their undocumented subprocess.CreateProcess. This fixes
'hg serve -d' on Windows.
Set dwDesiredAccess to 0 instead of GENERIC_READ.
Zero is enough for querying the file metadata. We don't even need to
access the -contents- of the file.
Set dwShareMode to FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE
instead of the overly restrictive FILE_SHARE_READ.
There is no need to cause write or delete accesses by other processes to
fail while we are querying file metadata.
See http://msdn.microsoft.com/en-us/library/aa363858(v=vs.85).aspx
According to the API document, GetModuleFileName is the preferred way to
retrieve the filename of the current process. So we shouldn't try
GetModuleFileName'Ex' first.
Previously system_rcpath_win32() happened to return unicode paths due to
GetModuleFileNameEx (issue2480). This problem is fixed as GetModuleFileName
never return unicode.
The Linux CIFS kernel driver (even in 2.6.36) suffers from a hardlink
count blindness bug (lstat() returning 1 in st_nlink when it is expected
to return >1), which causes repository corruption if Mercurial running
on Linux pushes or commits to a hardlinked repository stored on a Windows
share, if that share is mounted using the CIFS driver.
This patch works around issue1866 and improves the workaround done in
65e082ae3076 to fix issue761, by teaching the opener to lazily execute a
runtime check (new function checknlink) to see if the hardlink count
reported by nlinks() can be trusted.
Since nlinks() is also known to return varying count values (1 or >1)
depending on whether the file is open or not and depending on what client
and server software combination is being used for accessing and serving
the Windows share, we deliberately open the file before calling nlinks() in
order to have a stable precondition. Trying to depend on the precondition
"file closed" would be fragile, as the file could have been opened very
easily somewhere else in the program.
The fallback was introduced by 33415b0b4e64 at the same time than
nlinks(). Apparently it only handles the case where target path
does not exist. Just raise IOError directly.
win32.nlinks() was often returning 1 instead of the correct
hardlinks count when reading from network drives. This made
commit or push to a repository on a network share to fail
breaking the hardlinks in the datastore, possibly causing
integrity errors in repositories linked locally on the remote
side.
Here is what the MSDN says about GetFileInformationByHandle():
Depending on the underlying network features of the operating
system and the type of server connected to, the
GetFileInformationByHandle function may fail, return partial
information, or full information for the given file.
In practice, we never got the correct hardlinks count when
reading from and to many combinations of Window XP, 2003, Vista
and 7, via network drives or RDP shares. It always returned 1
instead. The only setup returning an accurate links count was a
samba on Debian.
To avoid this, Mercurial now breaks the hardlinks unconditionally
when writing to a network drive.
Hiding the child process window is not strictly necessary but it avoids opening
an empty shell window when running hg serve as well as a task in the task bar.
The window is hidden after the process is already started causing a single
flicker.
This fixes the implied reliance on pywin32 and the win32 module. This
also fixes a regression in fe7c89838a64 that made Mercurial unusable
without pywin32.
The posixfile_nt class has been superseded by posixfile in osutils.c,
which works on Windows NT and above. All other systems get the regular
python file class which is assigned to posixfile in posix.py (for POSIX)
and in the pure python version of osutils.py (for Win 9x or Windows NT
in pure mode).
