Most test scripts use "hg" to interact with a temporary test repository.
However a few tests also want to run hg commands to interact with the local
repository containing the mercurial source code. Notably, many of the
test-check-* tests want to check local files and commit messages.
These tests were previously using the version of hg being tested to query the
source repository. However, this will fail if the source repository requires
extensions or other settings not supported by the version of mercurial being
tested. The source repository was typically initially cloned using the system
hg installation, so we should use the system hg installation to query it.
There was already a helpers-testrepo.sh script designed to help cope with
different requirements for the source repository versus the test repositories.
However, it only handled the evolve extension. This new behavior works with
any extensions that are different between the system installation and the test
installation.
The useful pudb debugger can be used with Mercurial, but its import of urwid
fails when demandimport is enabled. Add urwid.command_map to the ignore list so
pudb can be used with hg without disabling all of demandimport.
I've spent several hours over the past few weeks investigating
networking failures involving hg.mozilla.org. As part of this, it
has become clear that the Mercurial client's error handling when
it encounters network failures is far from robust.
To prove this is true, I've devised a battery of tests simulating
various network failures, notably premature connection closes. To
achieve this, I've implemented an extension that monkeypatches the
built-in HTTP server and hooks in at the socket level and allows
various events to occur based on config options. For example, you
can refuse to accept() a client socket or you can close() the socket
after N bytes have been sent or received. The latter effectively
simulates an unexpected connection drop (and these occur all the
time in the real world).
The new test file launches servers exhibiting various "bad" behaviors
and points a client at them. As the many TODO comments in the test
call attention to, Mercurial often displays unhelpful errors when
network-related failures occur. This makes it difficult for users
to understand what's going on and difficult for server administrators
to pinpoint root causes without packet tracing.
Upcoming patches will attempt to fix these error handling
deficiencies.
Module 'appdirs' tries to import win32com.shell (and catch ImportError as an
indication of failure) to check whether some further functionality should
be implemented one or another way [1]. Of course, demandimport lets it down, so
if we want appdirs to work we have to add it to demandimport's ignore list.
The reason we want appdirs to work is becuase it is used by setuptools [2] to
determine egg cache location. Only fairly recent versions of setuptools depend
on this so people don't see this often.
[1] https://github.com/ActiveState/appdirs/blob/master/appdirs.py#L560
[2] aae0a92811/pkg_resources/__init__.py (L1369)
revlog.revision takes either node or rev, but taking a rev is more
efficient, because converting rev to node is just a seek and read.
That's cheaper than converting node to rev, which may require O(n) walk in
revlog index for the first times, and then triggering building the radix
tree index. Even with the radix tree built, rev -> node is still faster than
node -> rev because the radix tree requires more jumps in memory.
So r.revision(r.node(rev)) should be changed to r.revision(rev). This patch
adds a check-code rule to detect that.
String literals without explicit prefix in __init__.py and policy.py
are treated as unicode object on Python3, because these modules are
loaded before setup of our specific code transformation (the later
module is imported at the beginning of __init__.py).
BTW, "modulepolicy" in __init__.py is initialized by "policy.policy".
This causes issues below;
- checking "policy" value in other modules causes unintentional result
For example, "b'py' not in (u'c', u'py')" returns True
unintentionally on Python3.
- writing "policy" out fails at conversion from unicode to bytes
db1ebf457295 fixed this issue for default code path, but "policy"
can be overridden by HGMODULEPOLICY environment variable (it should
be rare case for developer using Python3, though).
This patch does:
- add "b" prefix to all string literals, which are related to module
policy, in modules above.
- check existence of HGMODULEPOLICY, and overwrite "policy" only if
it exists
For simplicity, this patch omits checking "supports_bytes_environ",
switching os.environ/os.environb, and so on (Yuya agreed this in
personal talking)
Earlier this was left thinking that its part of pywatchman package.
This patch replaces variables os.sep, sys.platform and os.envrion with their
py3 compatible ones.
We have our own bytes versions of things like, getopt.getopt, os.sep, os.name,
sys.executable, os.environ and few more for python 3 portability. Its better
to come up with warnings if someone breaks the things which we have fixed.
After this patch, check-code will warn us to use our bytes version.
These checks run on mercurial/ and hgext/ and pycompat.py is excluded.
pywatchman is imported from upstream and therefore fails to pass
linting. We have added 'no-check-code' manually to every file in the
past. This is cumbersome and modifies upstream sources.
I felt like inline Python in test-check-code.t was the most
appropriate place for this, as other linters in contrib/ seem to
be source file agnostic.
The test currently fails.
As the commit message for the previous changeset says, we wish
for zstd to be a 1st class citizen in Mercurial. To make that
happen, we need to enable Python to talk to the zstd C API. And
that requires bindings.
This commit vendors a copy of existing Python bindings. Why do we
need to vendor? As the commit message of the previous commit says,
relying on systems in the wild to have the bindings or zstd present
is a losing proposition. By distributing the zstd and bindings with
Mercurial, we significantly increase our chances that zstd will
work. Since zstd will deliver a better end-user experience by
achieving better performance, this benefits our users. Another
reason is that the Python bindings still aren't stable and the
API is somewhat fluid. While Mercurial could be coded to target
multiple versions of the Python bindings, it is safer to bundle
an explicit, known working version.
The added Python bindings are mostly a fully-featured interface
to the zstd C API. They allow one-shot operations, streaming,
reading and writing from objects implements the file object
protocol, dictionary compression, control over low-level compression
parameters, and more. The Python bindings work on Python 2.6,
2.7, and 3.3+ and have been tested on Linux and Windows. There are
CFFI bindings, but they are lacking compared to the C extension.
Upstream work will be needed before we can support zstd with PyPy.
But it will be possible.
The files added in this commit come from Git commit
e637c1b214d5f869cf8116c550dcae23ec13b677 from
https://github.com/indygreg/python-zstandard and are added without
modifications. Some files from the upstream repository have been
omitted, namely files related to continuous integration.
In the spirit of full disclosure, I'm the maintainer of the
"python-zstandard" project and have authored 100% of the code
added in this commit. Unfortunately, the Python bindings have
not been formally code reviewed by anyone. While I've tested
much of the code thoroughly (I even have tests that fuzz APIs),
there's a good chance there are bugs, memory leaks, not well
thought out APIs, etc. If someone wants to review the code and
send feedback to the GitHub project, it would be greatly
appreciated.
Despite my involvement with both projects, my opinions of code
style differ from Mercurial's. The code in this commit introduces
numerous code style violations in Mercurial's linters. So, the code
is excluded from most lints. However, some violations I agree with.
These have been added to the known violations ignore list for now.
Vendored from https://bitbucket.org/facebook/hg-experimental
changeset 73f9db47ae5a1a9fa29a98dfe92d557ad51234c3 without
modification.
This introduces a number of code style violations. The file
already has the magic words to skip test-check-code.t. I'll
make additional changes to clean up the test-check-py3-compat.t
warnings and to change some behavior in the code that isn't
suitable for general use.
test-check-commit.t also complains about numerous things. But
there's nothing we can do if we're importing as-is.
In preparation for the filesystem monitor extension, include the pywatchman
library. The fbmonitor extension relies on this library to communicate with
the Watchman service. The library is BSD licensed and is taken from
https://github.com/facebook/watchman/tree/master/python.
This package has not been updated to mercurial code standards.
We had them on 'test-check-code-hg.t' to avoid collision with the test checking
'check-code' itself. Now that this one have been rename, we can safely remove
this suffix for all of them. This get them in line with 'check-pyflakes.t'.
This test (making sure the 'check-code' script run as intended) have been
confused with the test making that the mercurial code base comply with our
coding still by multiple generations of contributors.
We are moving it out of the way so that all tests starting with
'test-check' are now doing compliance testing.
The check-code tool now expects the "desc" keyword to be followed by the
"websub" filter, with the following exceptions:
a) It has no filters at all, e.g. a changeset description in the raw style
templates or the repository description in the summary page.
b) It is followed by the "firstline" filter, e.g. the first line of the
changeset description is displayed as a summary or title.
Before this patch, "contrib/check-code.py" can't detect "% inside _()"
correctly, when there are leading whitespaces before the format
string, like below:
_(
"format string %s" % v)
This patch adds regexp pattern "[ \t\n]*" before the pattern matching
against the format string.
"[\s\n]" can't be used in this purpose, because "\s" is automatically
replaced with "[ \t]" by "_preparepats()" and "\s" in "[]" causes
nested "[]" unexpectedly.
The code adding the prefix is now run once per pattern. It was run once per
file (after the change 17484f4c54fb).
Demonstrate that it is working now by extending the test. Raise two different
warnings, one of them twice.
