As suggested by Ryan in D1019, it's cleaner if we use defined constants
instead of `osname == 'nt'` everywhere.
Differential Revision: https://phab.mercurial-scm.org/D1033
`platform.system()` may have a side effect spawning a shell executing
`uname -p`, which may print a warning when the current directory is removed:
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
This patch changes selectors2 to test the `sys.platform` string, which is a
much safer way to detect Jython.
Jython's `sys.platform` looks like this:
Jython 2.7.1 (default:0df7adb1b397, Jun 30 2017, 19:02:43)
[OpenJDK 64-Bit Server VM (Oracle Corporation)] on java1.8.0_144
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.platform
'java1.8.0_144 ( ==linux2 for targets )'
Differential Revision: https://phab.mercurial-scm.org/D1018
These are all the httpserver and urllib.* aliases. They seem to make
more sense in the slightly-more-specific urllibcompat package than the
general-purpose pycompat.
Differential Revision: https://phab.mercurial-scm.org/D935
pycompat.py is unorganized and looks ugly. Next few patches will try to make it
look more cleaner so that adding more code is easy and reading code also.
This patch moves the multiline comments above functions to function docs. While
moving, I improved the comments and make them better suitable for func doc.
While I was here I drop a unrequired and misplaced comment.
urllib.parse.urlencode() returns unicodes on Python 3. This commit adds a
method which will take its output and encode it to bytes so that we can use
bytes consistently.
Mixing bytes and unicode creates a mess. Do things in bytes as possible.
New sysbytes() helper only takes care of ASCII characters, but avoids raising
nasty unicode exception. This is the same design principle as sysstr().
Currently, we export urlparse via util.urlparse then
call util.urlparse.urlparse() and util.urlparse.urlunparse()
in a few places. This is the only url* module exported from
pycompat, making it a one-off. So let's transition to urlreq
to match everything else.
Yes, we double import "urlparse" now on Python 2. This will
be cleaned up in a subsequent patch.
Also, the Python 3 functions trade in str/unicode not bytes.
So we'll likely need to write a custom implementation that
speaks bytes. But moving everyone to an abstracted API
is a good first step.
It is duplicated by urlreq.unquote and is unused. Kill it.
We retain the imports because it is re-exported via util.urlparse,
which is used elsewhere.
Since we no longer access attributes of urlparse at module load time,
this change /should/ result in that module reverting to a lazy module.
Previously, urlreq.unquote aliased to urllib.parse.unquote,
which returned a str/unicode. We like bytes, so switch urlreq.unquote
to dispatch to urllib.parse.unquote_to_bytes.
This required a minor helper function to register an alias under a
different name from which it points. If this turns into a common
pattern, we could likely teach _registeralias to accept tuple
values defining the mapping. Until then, I didn't feel like
adding complexity to _registeralias.
This allows us to handle bytes in mostly the same manner as Python 2 str,
so we can get rid of ugly s[i:i + 1] hacks:
s = bytestr(s)
while i < len(s):
c = s[i]
...
This is the simpler version of the previous RFC patch which tried to preserve
the bytestr type if possible. New version simply drops the bytestr wrapping
so we aren't likely to pass a bytestr to a function that expects Python 3
bytes.
With Python 3.4.3, timeit says 0.437 usec -> 0.0685 usec. With Python
3.6, timeit says 0.157 usec -> 0.0907 usec. So it's faster on both
versions, but the speedup varies a lot.
Thanks to Gregory Szorc for the suggestion.
urllib.parse.quote() accepts either str or bytes and returns str.
There exists a urllib.parse.quote_from_bytes() which only accepts
bytes. We should probably use that to retain strong typing and
avoid surprises.
In addition, since nearly all strings in Mercurial are bytes, we
probably don't want quote() returning unicode.
So, this patch implements a custom quote() that only accepts bytes
and returns bytes. The quoted URL should only contain URL safe
characters which is a strict subset of ASCII. So
`.encode('ascii', 'strict')` should be safe.
urllib.request imports a bunch of symbols from other urllib
modules. We should map to the original symbols not the
re-exported ones because this is more correct. Also, it
will prevent an import of urllib.request if only one of
the lower-level symbols/modules is needed.
pycompat.getenv returns os.getenvb on py3 which is not available on Windows.
This patch replaces them with encoding.environ.get and checks to ensure no
new instances of os.getenv or os.setenv are introduced.
shlex.split() only accepts unicodes on Python 3. After this patch we will be
using pycompat.shlexsplit(). This patch also replaces existing occurences of
shlex.split with pycompat.shlexsplit.
sys.executable on Python 3 returns unicodes and we want bytes. So this patch
adds a new pycompat.sysexecutable which returns bytes by encoding using
os.fsencode() since it is path variable.
os.getenv() on python 3 deals with unicodes. If we want to pass bytes. we have
os.getenvb() which deals with bytes. This patch adds up a pycompat.osgetenv
which deals with bytes on both python 2 and 3.
Keys of keyword arguments need to be str(unicodes) on Python 3. We have a lot
of function where we pass keyword arguments. Having utility functions to help
converting keys to unicodes before passing and convert back them to bytes once
passed into the function will be helpful. We now have functions named
pycompat.strkwargs(dic) and pycompat.byteskwargs(dic) to help us.
getopt.getopt() deals with unicodes on Python 3 internally and if bytes
arguments are passed, then it will return TypeError. So we have now
pycompat.getoptb() which takes bytes arguments, convert them to unicode, call
getopt.getopt() and then convert the returned value back to bytes and then
return those value.
All the instances of getopt.getopt() are replaced with pycompat.getoptb().
Following the behaviour of Python 3, os.getcwd() return unicodes. We need
bytes version as path variables are bytes in UNIX. Python 3 has os.getcwdb()
which returns current working directory in bytes.
Like rest of the things there in pycompat, like osname, ossep, we need to
rewrite every instance of os.getcwd to pycompat.getcwd to make them work
correctly on Python 3.
Since standard streams are TextIO on Python 3, we can't use sys.stdin/out/err
directly. Fortunately we can get the underlying BytesIO via .buffer as long as
the streams aren't replaced by e.g. StringIO.
stdin/out/err are provided through util so we can wrap them by platform API.
open() requires mode argument as unicodes on Python 3. This patch introduces
pycompat.open() which is inserted to files using transformer and replaces
builtins.open() calls.
sys.argv returns unicodes on Python 3. We need a bytes version for us.
There was also a python bug/feature request which wanted then to implement
one. They rejected and it is quoted in one of the comments that we can use
fsencode() to get a bytes version of sys.argv. Though not sure about its
correctness.
Link to the comment: http://bugs.python.org/issue8776#msg217416
After this patch we will have pycompat.sysargv which will return us bytes
version of sys.argv. If this patch goes in, i will like to make transformer
rewrite sys.argv with pycompat.argv because there are lot of occurences.
