Since 1d07d9da84a0, pycompat.bytestr() wrapped by win32mbcs returns
unicode object, if an argument is not byte-str object. And this causes
unexpected failure at colorization.
pycompat.bytestr() is used to convert from color effect "int" value to
byte-str object in color module. Wrapped pycompat.bytestr() returns
unicode object for such "int" value, because it isn't byte-str.
If this returned unicode object is used to colorize non-ASCII byte-str
in cases below, UnicodeDecodeError is raised at an operation between
them.
- colorization uses "ansi" color mode, or
Even though this isn't default on Windows, user might use this
color mode for third party pager.
- ui.write() is buffered with labeled=True
Buffering causes "ansi" color mode internally, regardless of
actual color mode. With "win32" color mode, extra escape sequences
are omitted at writing data out.
For example, with "win32" color mode, "hg status" doesn't fail for
non-ASCII filenames, but "hg log" does for non-ASCII text, because
the latter implies buffered formatter.
There are many "color effect" value lines in color.py, and making them
byte-str objects isn't suitable for fixing on stable. In addition to
it, pycompat.bytestr will be used to get byte-str object from any
types other than int, too.
To resolve this issue, this patch does:
- replace pycompat.bytestr in checkwinfilename() with newly added
hook point util._filenamebytestr, and
- make win32mbcs reverse-wrap util._filenamebytestr
(this is a replacement of 1d07d9da84a0)
This patch does two things above at same time, because separately
applying the former change adds broken revision (from point of view of
win32mbcs) to stable branch.
"_" prefix is added to "filenamebytestr", because it is win32mbcs
specific hook point.
When trying to create a zstd bundle, the MSI based install said:
abort: compression engine zstd could not be loaded
The Inno installer is unaffected. The name will need to be updated to include
'cext' when merging into default.
This commit essentially reverts 62ad9c1dbce9.
urllib2.URLError receives a "reason" argument. It isn't always a
tuple. Mozilla has experienced at least IndexError failures due
to the reason[1] access.
https://bugzilla.mozilla.org/show_bug.cgi?id=1364687
Before this patch, "hg CMD --pager on" on Windows shows output
unintentionally decorated with ANSI color escape sequences, if color
mode is "auto". This issue occurs in steps below.
1. dispatch() invokes ui.pager() at detection of "--pager on"
2. stdout of hg process is redirected into stdin of pager process
3. "ui.formatted" = True, because isatty(stdout) is so before (2)
4. color module is loaded for colorization
5. color.w32effects = None, because GetConsoleScreenBufferInfo()
fails on stdout redirected at (2)
6. "ansi" color mode is chosen, because of "not w32effects"
7. output is colorized in "ansi" mode because of "ui.formatted" = True
Even if "ansi" color mode is chosen, ordinarily redirected stdout
makes ui.formatted() return False, and colorization is avoided. But in
this issue case, "ui.formatted" = True at (3) forces output to be
colorized.
For correct console information on win32, it is needed to ensure that
color module is loaded before redirection of stdout for pagination.
BTW, if any of enabled extensions has "colortable" attribute, this
issue is avoided even before this patch, because color module is
imported as a part of loading such extension, and extension loading
occurs before setting up pager. For example, mq and keyword have
"colortable".
Adding markers to the repository might affect the set of obsolete changesets. So we
most remove the "volatile" set who rely in that data. We add two missing
invalidations after merging markers. This was caught by code change in the evolve
extensions tests.
This issues highlight that the current way to do things is a bit fragile,
however we keep things simple for stable.
win32mbcs wraps some functions, to prevent them from unintentionally
treating backslash (0x5c), which is used as the second or later byte
of multi bytes characters by problematic encodings, as a path
component delimiter on Windows platform.
This wrapping assumes that wrapped functions can safely accept unicode
string arguments.
Unfortunately, 440868900036 broke this assumption by introducing
pycompat.bytestr() into util.checkwinfilename() for py3 support. After
that, wrapped checkwinfilename() always fails for non-ASCII filename
at pycompat.bytestr() invocation.
This patch wraps underlying pycompat.bytestr() function to use
util.checkwinfilename() safely.
To avoid similar regression in the future, another patch series will
add smoke testing on default branch.
Graft used findmissingrevs to find the candidates for graft duplicates in the
destination. That function operates with the constraint:
1. N is an ancestor of some node in 'heads'
2. N is not an ancestor of any node in 'common'
For our purpose, we do however have to work correctly in cases where the graft
set has multiple roots or where merges between graft ranges are skipped. The
only changesets we can be sure doesn't have ancestors that are grafts of any
changeset in the graftset, are the ones that are common ancestors of *all*
changesets in the graftset. We thus need:
2. N is not an ancestor of all nodes in 'common'
This change will graft more correctly, but it will also in some cases make
graft slower by making it search through a bigger and unnecessary large sets of
changes to find duplicates. In the general case of grafting individual or
linear sets, we do the same amount of work as before.
This demonstrates unfortunate behaviour: extending the graft range cause the
graft to behave differently. When the graft range includes a merge, we fail to
detect duplicates that are ancestors of the merge.
"merge-tools" help topic has described that the merge of the file
fails if no tool is found to merge binary or symlink, since
9da9bced2226 (or Mercurial 1.7), which based on (already removed)
MergeProgram wiki page.
But even at that revision, and of course now, merge of the file
doesn't fail automatically for binary/symlink. ":prompt" (or
equivalent logic) is used, if there is no appropriate tool
configuration for binary/symlink.
As an aside, I'm having trouble parsing the help text meaning for HG when it is
unset or empty. How can it be the frozen name or searched if it is empty?
This may be too subtle of a change to get the point across, but when I first
read the original text, I thought maybe the pager would only be invoked if
writing more than a screenful. The distinction between this and a pager that
simply exits after printing less than a screenful is important on Windows, given
the inability of `more` to color output.
Even though I figured this out a few weeks ago, I was initially puzzled where
the color went when I upgraded to 4.2 on a different Windows machine. Let's
point users reading the help into the right direction.
I wonder if we should be even more explicit about cmd.exe/MSYS/pager/color
interplay, but at least all of the breadcrumbs are here (I think).
Windows uses `more.com`, which unhelpfully adds an extra trailing line
consisting only of '\r'. It also converts tab characters to spaces, which
throws off the last two tests.
Setting the 'ui.formatted' option is what allowed the pager to be used by these
tests in the first place.
This option was never released except for a release candidate. Dropping
compatibility with this option will free the 'pager.enable' config option for
other usage in the future.
This aligns with what we do for color (see cea7a760c58d). Pager is a central
enough notion that having the master config in the [ui] section makes senses. It
will helps with consistency, discoverability. It will also help having a simple
and clear example hgrc mentioning pager.
The previous form of the option had never been released in a non-rc version but
we keep it around for convenience. If both are set, 'ui.pager' take priority.
Previously, 'ui.color=yes' meant "always show color", While
"ui.color=auto" meant "use color automatically when it appears
sensible".
This feels problematic to some people because if an administrator has
disabled color with "ui.color=off", and a user turn it back on using
"color=on", it will get surprised (because it breaks their output when
redirected to a file.) This patch changes ui.color=true to only move the
default value of --color from "never" to "auto".
I'm not really in favor of this changes as I suspect the above case will
be pretty rare and I would rather keep the logic simpler. However, I'm
providing this patch to help the 4.2 release in the case were others
decide to make this changes.
Users that want to force colors without specifying --color on the
command line can use the 'ui.formatted' config knob, which had to be
enabled in a handful of tests for this patch.
Nice summary table (credit: Augie Fackler)
That is, before this patch:
+--------------------+--------------------+--------------------+
| | not a tty | a tty |
| | --color not set | --color not set |
| | | |
+--------------------+--------------------+--------------------+
| [ui] | | |
| color (not set) | no color | no color |
| | | |
+--------------------+--------------------+--------------------+
| [ui] | | |
| color = auto | no color | color |
| | | |
+--------------------+--------------------+--------------------+
| [ui] | | |
| color = yes | *color* | color |
| | | |
+--------------------+--------------------+--------------------+
| [ui] | | |
| color = no | no color | no color |
| | | |
+--------------------+--------------------+--------------------+
(if --color is specified, it always clobbers the setting in [ui])
and after this patch:
+--------------------+--------------------+--------------------+
| | not a tty | a tty |
| | --color not set | --color not set |
| | | |
+--------------------+--------------------+--------------------+
| [ui] | | |
| color (not set) | no color | no color |
| | | |
+--------------------+--------------------+--------------------+
| [ui] | | |
| color = auto | no color | color |
| | | |
+--------------------+--------------------+--------------------+
| [ui] | | |
| color = yes | *no color* | color |
| | | |
+--------------------+--------------------+--------------------+
| [ui] | | |
| color = no | no color | no color |
| | | |
+--------------------+--------------------+--------------------+
(if --color is specified, it always clobbers the setting in [ui])
"Churn" is not the useful example we have, but this is the one used in
'hg help config.extensions'. As we need something to replace the deprecated
'pager' extension in the example config, we are adding 'churn'.
If a marker has no parent information, parents field is set to None, which
caused TypeError. I think this shouldn't normally happen, but somehow
I have such marker in my repo.
We point out at the help of the config option for user who wants to learn more
about the possible config value.
The suggested command returns some other extra (color related) results, but this
is bug to fix outside of the freeze.
The --color option is described to accept "boolean, always, auto,
never, or debug". Let's use a similar description for ui.color. Also
fix the formatting to use double quotes as we seem to use for about
half the values in config.txt (the other half uses double
backticks). Also use uppercase Boolean for consistency within the
file.
On system where HGTEST_PORT is configured to a value an order a magnitude lower
or higher than the default. The number of digit in the HGPORT changes and this
test breaks.
Acquiring lock by vfs.makelock() and getting lock holder (aka
"locker") information by vfs.readlock() aren't atomic action.
Therefore, failure of the former doesn't ensure success of the latter.
Before this patch, lock is unintentionally acquired, if ENOENT
causes failure of vfs.readlock() while 5 times retrying, because
lock._trylock() returns to caller silently after retrying, and
lock.lock() assumes that lock._trylock() returns successfully only if
lock is acquired.
In this case, lock symlink (or file) isn't created, even though lock
is treated as acquired in memory.
To avoid this issue, this patch makes lock._trylock() raise
LockHeld(EAGAIN) at the end of it, if lock isn't acquired while
retrying.
An empty "locker" meaning "busy for frequent lock/unlock by many
processes" might appear in an abortion message, if lock acquisition
fails. Therefore, this patch also does:
- use '%r' to increase visibility of "locker", even if it is empty
- show hint message to explain what empty "locker" means
Acquiring lock by vfs.makelock() and getting lock holder (aka
"locker") information by vfs.readlock() aren't atomic action.
Therefore, failure of the former doesn't ensure success of the latter.
Before this patch, lock is unintentionally acquired, if
self.parentlock is None (this is default value), and lock._readlock()
returns None for ENOENT at vfs.readlock(), because these None are
recognized as equal to each other.
In this case, lock symlink (or file) isn't created, even though lock
is treated as acquired in memory.
To avoid this issue, this patch retries lock acquisition immediately,
if lock._readlock() returns None "locker".
This issue will be covered by a test added in subsequent patch,
because simple emulation of ENOENT at vfs.readlock() easily causes
another issue. "raising ENOENT only at the first vfs.readlock()
invocation" is too complicated for unit test, IMHO.
Another raising PeerTransportError for "incomplete response" in
httppeer.py uses this (changed) hint message. This unification reduces
cost of translation.
There are some paragraphs, which aren't rendered in online help as
expected because of indentation and literal blocking issues.
- hgext/rebase.py
- paragraph before example code ends with ":", which treats
subsequent indented paragraphs as normal block
=> replace ":" with "::" to treat subsequent paragraphs as literal block
- help/pager.txt
- paragraph before a list of --pager option values ends with "::",
which treats subsequent indented paragraphs as literal block
=> replace "::" with ":" to treat subsequent paragraphs as normal block
- the second line of explanation for no/off --pager option value is
indented incorrectly (this also causes failure of "make" in doc)
=> indent correctly
- help/revisions.txt
- explanation following example code of "[revsetalias]" section
isn't suitable for literal block
=> un-indent explanation paragraph to treat it as normal block
- indentation of "For example" before example of tag() revset
predicate matching is meaningless
- descriptive text for tag() revset predicate matching isn't
suitable for literal block
=> un-indent concatenated two paragraphs to treat them as normal block