This test helps us to keep track on the commands which runs to Python 3.
The full traceback is hidden. Thanks to Augie and Martijn to wrap it up
in four lines.
We can't predict where those will show up and they're not
super-important for the contents of this particular test, so just drop
them. Further reduces the flakiness of the test to zero.
Healthy output (one log file mentioning "existing pooled" and one
mentioning "new pooled") will now print in a stable order, but
unhealthy output will print some sort of error.
This reduces the flakiness of the test from 55% to 38%. My next patch
makes it completely stable.
This fixes open(filename, 'r'), open(filename, 'w'), etc. calls. In Python
3, that second argument *must* be a string, you can't use bytes.
The fix is the same as used with getattr() (where the second argument must
also always be a string); in the tokenizer, where we detect calls, if there
is something that looks like a call to open (and is not an attribute, so
the previous token is not a "." dot) then make sure that that second
argument is not converted to a `bytes` object instead.
There is some remaining issue where the current transformer will also rewrite
open(f('foo')).
However this also affect function for which we perform similar rewrite
('getattr', 'setattr', 'hasattr', 'safehasattr') and will be dealt with in a
follow up.
The termwidth template keyword is of limited use without some way to ensure
that margins are respected.
Provide a full set of arithmetic operators (four basic operations plus the
mod function, defined to match Python's // for division), so that you can
create termwidth based layouts that match the user's terminal size
PyIntObject doesn't exist in Python 3. While PyIntObject is preferred
on Python 2 because it is a fixed capacity and faster, the difference
between PyIntObject and PyLongObject for scenarios where performance
isn't critical or the caller isn't performing type checking shouldn't
be relevant.
So change recvfds to return a list of longs instead of ints on Python
2.
PyIntObject no longer exists in Python 3. Instead, there is
PyLongObject.
Furthermore, PyInt_AS_LONG is a macro referencing a struct member.
PyInt_AS_LONG doesn't exist in Python 3 and PyLong_AS_LONG is a
#define for PyLong_AsLong, which is a function. So assigning to the
return value of PyLong_AS_LONG doesn't work.
This patch introduces a macro for obtaining the value of an
integer-like type that works on Python 2 and Python 3. On
Python 3, we access the struct field of the underlying
PyLongObjet directly, without overflow checking. This is
essentially the same as what Python 2 was doing except using a
PyLong instead of a PyInt.
PyStringObject was renamed to PyBytes in Python 3 along with the
corresponding PyString* functions and macros. PyString* doesn't
exist in Python 3. But PyBytes* is an alias to PyString in Python 2.
So rewrite PyString* to PyBytes* for Python 2/3 dual compatibility.
The old code happened to work because of how the macro was defined.
This no longer works in Python 3. Furthermore, assigning to a macro
just feels weird. So just inline the macro.
The macro changed slightly in Python 3, introducing curly brackets
that somehow confuse Clang into issuing a ton of compiler warnings.
Using PyVarObject_HEAD_INIT makes these go away.
It's worth noting that the code is identical: the 2nd argument to
PyVarObject_HEAD_INIT is assigned to the ob_size field and is
inserted immediately after "PyObject_HEAD_INIT(type)" is generated.
Compilers are weird.
On Python 2, PyBytes_GET_SIZE is the same as PyString_GET_SIZE which
is the same as Py_SIZE which resolves to a struct member.
On Python 3, PyBytes_GET_SIZE is
"(assert(PyBytes_Check(op)),Py_SIZE(op))". The compiler barfs when
assigning to this version.
This patch simply changes PyBytes_GET_SIZE to Py_SIZE. On Python 2,
there is no effective change in behavior. On Python 3, we drop the
PyBytes_Check(). However, in all cases we have explicitly created
a PyBytesObject in the same function, so the PyBytes_Check() is
guaranteed to be true. Despite this, code changes over time, so
I've added added assert() in all callers so we can catch this in
debug builds.
With this patch, all mercurial.* C extensions now compile on Python 3
on my OS X machine. There are several compiler warnings and I'm sure
there are incompatibilities with Python 3, including possibly
segfaults. But it is a milestone.
We no longer have any users of the legacy PyString* functions. We no
longer need these redefinitions.
After this change, the only reference to "PyString" in the repo is in
watchman's C extension. That isn't our code and porting Mercurial
extensions to Python 3 is not a high priority at the moment. watchman's
C extension will be dealt with later.
Python 2.6 introduced PyBytesObject and PyBytes* as aliases for
PyStringObject and PyString*. So on Python 2.6+, PyBytes* and PyString*
are identical and this patch should be a no-op.
On Python 3, PyStringObject is effectively renamed to PyUnicodeObject
and PyBytesObject becomes the main type for byte strings.
This patch begins the process of mass converting PyString* to PyBytes*
so the C extensions use the correct type on Python 3.
Otherwise no code transformation would be applied to the modules which are
imported only by imp.load_module().
This change means modules are imported from PYTHONPATH, not from the paths
given by command arguments. This isn't always correct, but seems acceptable.
I think remapping Python C API types and functions is not a great
approach. I'd prefer this whole #ifdef disappeared. Add a comment
so we don't forget about it.
util.h attempts to wallpaper over C API differences between Python 2
and 3. This is not the correct approach where performance is critical.
But it is good enough for the current state of the Python 3 port.
The custom module importer was making these bytes, so when we poked
values into self.__dict__ we had bytes instead of unicode on py3 and
it didn't work.
This also includes what I consider to be the minimum set of steps
someone should be able to perform even if they can't run the
testsuite. Hopefully this will help new contributors know to at least
run the two checkers that find most things that (in my experience)
require manual cleanup.