Inspired by the dirstate fix in 39954a8760cd, this should fix any race
conditions with the fsmonitor state changing from underneath.
Since we now grab the wlock for any non-invalidate writes, the only situation
this appears to happen in is with a concurrent invalidation. Test that.
Since 1d07d9da84a0, pycompat.bytestr() wrapped by win32mbcs returns
unicode object, if an argument is not byte-str object. And this causes
unexpected failure at colorization.
pycompat.bytestr() is used to convert from color effect "int" value to
byte-str object in color module. Wrapped pycompat.bytestr() returns
unicode object for such "int" value, because it isn't byte-str.
If this returned unicode object is used to colorize non-ASCII byte-str
in cases below, UnicodeDecodeError is raised at an operation between
them.
- colorization uses "ansi" color mode, or
Even though this isn't default on Windows, user might use this
color mode for third party pager.
- ui.write() is buffered with labeled=True
Buffering causes "ansi" color mode internally, regardless of
actual color mode. With "win32" color mode, extra escape sequences
are omitted at writing data out.
For example, with "win32" color mode, "hg status" doesn't fail for
non-ASCII filenames, but "hg log" does for non-ASCII text, because
the latter implies buffered formatter.
There are many "color effect" value lines in color.py, and making them
byte-str objects isn't suitable for fixing on stable. In addition to
it, pycompat.bytestr will be used to get byte-str object from any
types other than int, too.
To resolve this issue, this patch does:
- replace pycompat.bytestr in checkwinfilename() with newly added
hook point util._filenamebytestr, and
- make win32mbcs reverse-wrap util._filenamebytestr
(this is a replacement of 1d07d9da84a0)
This patch does two things above at same time, because separately
applying the former change adds broken revision (from point of view of
win32mbcs) to stable branch.
"_" prefix is added to "filenamebytestr", because it is win32mbcs
specific hook point.
Our current deprecation warning mechanism relies on ui object. They are case
where we cannot have access to the UI object. On a general basis we avoid using
the python mechanism for deprecation warning because up to Python 2.6 it is
exposing warning to unsuspecting user who cannot do anything to deal with them.
So we build a "safe" strategy to hide this warnings behind a flag in an
environment variable. The test runner set this flag so that tests show these
warning. This will help us marker API as deprecated for extensions to update
their code.
We have to do the % formatting over the sysstr, since the things we're
going to splat into it are themselves sysstrs. This is probably
technically wrong-ish, since bt is probably actually a bytestr here,
but this fixes the immediate issue, which was that hg was broken on
Python 3.
An upcoming patch will add support for documenting bundle
specifications in more detail. As part of this, we'd like to
enumerate available bundle compression formats. In order to do
this, we need to provide the help mechanism a dict of names
and objects with docstrings.
This patch adds docstrings to compengine.bundletype and adds
a function for retrieving a dict of them. The code is not yet
used.
Previously, when copying a file, copyfiles will compare src's st_dev with
dirname(dst)'s st_dev, to decide whether to enable hardlink or not.
That could have issues on Linux's overlayfs, where stating directories could
result in different st_dev from st_dev of stating files, even if both the
directories and the files exist in the overlay's upperdir.
This patch fixes it by checking dirname(src) instead. It's more consistent
because we are checking directories for both src and dest.
That fixes test-hardlinks.t running on common Docker setups.
This patch removes the global variable "allowhardlinks" that disables
hardlink in all cases, so hardlink gets enabled if the filesystem type is
whitelisted.
Third party extensions wanting to enable hardlink support unconditionally
can replace "_hardlinkfswhitelist.__contains__".
Since osutil.getfstype is available, use it to detect filesystem types. The
whitelist currently includes common local filesystems on Linux where they
should have good hardlink support. We may add new filesystems for other
platforms later.
pycompat.urlreq.unquote and pycompat.urlunquote effectively alias the
same thing. pycompat.urlunquote is only used once in the code base.
So let's switch to urlreq.unquote.
"Effectively" in the above paragraph is because pycompat.urlreq.unquote
aliases urllib.unquote and pycompat.urlunquote aliases urlparse.unquote
on Python 2. You might think one of urllib.unquote and urlparse.unquote
is an alias to the other, but you would be incorrect. In fact, these
functions are copies of each other. There is even a comment in the
CPython source code saying to keep them in sync. You can't make this
up.
Throughout mercurial cdoe, there is a common pattern of attempting to remove
a file and ignoring ENOENT errors. Let's move this into a common function to
allow for cleaner code.
Previously, there were two slightly different versions of unlinkpath between
windows and posix, but these differences were eliminated in previous patches.
Now we can unify these two code paths inside of the util module.
Changeset 3b9cdb72931f removed the mutable default value, but did not explicitly
tested for None. Such implicit checking can introduce semantic and performance
issue. We move to an explicit check for None as recommended by PEP8:
https://www.python.org/dev/peps/pep-0008/#programming-recommendations
string_escape doesn't exist on Python 3, but fortunately the undocumented
codecs.escape_encode() function exists on CPython 2.6, 2.7, 3.5 and PyPy 5.6.
So let's use it for now.
http://stackoverflow.com/a/23151714
It was specified to be an empty list in 3d8abfdaa08a in 2007.
It was correct at the time. But when the function was
refactored in 7bca0f2718ab (2010), it started expecting a dict.
I guess this code path is untested?
Thanks to Yuya for spotting this.
I don't think this is any tight loops and we'd need to worry about
PyObject creation overhead. Also, I'm pretty sure strptime()
will be much slower than PyObject creation (date parsing is
surprisingly slow).
d46a7814be5f refactored util.parsedate in order to raise ValueError instead
of Abort for using with ui.configwith. It causes several problems, putting
arbitrary bytes in ValueError can cause issues with Python 3. Moreover, we
added a function to convert ValueError exceptions back to Abort.
A better approach would be to make parsedate raises ParseError, removing
the convert function and update configwith to also catch ParseError.
The side-effect is that error message when giving an invalid date in CLI
change from:
abort: invalid date: 'foo bar'
to:
hg: parse error: invalid date: 'foo bar'
I'm not sure if it's an acceptable change, I found personally the error
message more clear but more verbose too.
