We do this so that any linters installed via pip install --user don't
break. See https://docs.python.org/2/library/site.html#site.USER_BASE
for a description of what this nonsense is all about.
An alternative would be to not set HOME, but that'll cause other
problems (see issue2707), or to forward every single path entry from
sys.path in PYTHONPATH (which seems sketchy in its own way).
Some systems don't have a 127/8 address for localhost (I noticed this
on a FreeBSD jail). In order to work around this, use 127.0.0.1 as a
glob pattern. A future commit will update needed output lines and add
a requirement to check-code.py.
This patch includes addition of absolute_import and print_function to the
files where they are missing. The modern importing conventions are also followed.
In py2, json.dumps includes a trailing space after a comma at the
end of lines. The py3 behavior which omits the trailing space is
preferable, so we're going to strip it.
Without this, my python 2.6 virtualenv test run with --pure and
--local fails with:
+ ImportError: Python minor version mismatch: The Mercurial extension modules were compiled with Python 2.7.8, but Mercurial is currently using Python with sys.hexversion=33950192: Python 2.6.9 (unknown, Apr 13 2016, 12:40:12)
+ [GCC 4.9.2 20141101 (Red Hat 4.9.2-1)]
+ at: ~/hg/py26/bin/python
Several tests fail with chg for several reasons such as loaded chgserver
extension, running uisetup() per server instead of per runcommand, etc.
Since these tests can't/shouldn't be changed to be chg friendly, we need
a flag to skip them.
This patch explicitly drops CHGHG environment if chg isn't involved. This
way, hghave can just check if CHGHG exists.
Without this, sometimes installerrs generated errors
about no such file. It also did not work well when you
had multiple tests runners running around.
It also did not make sense to pollute the repository test
directory with the log file.
The regular expression in use passed tests because the test repo only
has single-digit changesets present. When I tried to use this for real
today, it broke, because the regular expression would only match a
single digit.
https://xkcd.com/1171/, or something like that.
72b40f92c680 enabled lines that were not matched to be found later in cases of jitter.
Unfortunately, in this model an optional line would always jitter to the end when
it is not present. That is not ideal.
It would be possible to do better, by queuing all writes until the end in case
an optional line jitters, but for now, it is simpler to assume optional lines
have a fixed place in the stream.
Before this patch, if --chg or --with-chg is specified, all tests are using
the same chgserver socket. Since the chg client holds a lock when it starts a
new server, and every test needs at least a new chg server due to different
HGRCPATH affecting the confighash, the result is a lot of tests will be
timed out if -j is large (for example, 50 or 100).
This patch solves the issue by using different chg socket directories for
different tests.
Currently, very few parts of Mercurial run under Python 3, notably the
test harness.
We want to write tests that run Python 3. For example, we want to
extend test-check-py3-compat.t to parse and load Python files.
However, we have a problem: finding appropriate files requires
running `hg files` and this requires Python 2 until `hg` works
with Python 3.
As a temporary workaround, we add --with-python3 to the test harness
to allow us to define the path to a Python 3 interpreter. This
interpreter is made available to the test environment via $PYTHON3 so
tests can run things with Python 3 while the test harness and `hg`
invocations continue to run from Python 2. To round out the feature,
a "py3exe" hghave check has been added.
Instead of treating expected output as happening in a precise order,
and assuming that if a line is missing it will never happen,
assume that expected output is a prioritized list of likely matching
lines.
This means that if:
foo/bar (glob)
baz/bad (glob)
changes to:
baz/bad
foo/bar
instead of generating:
baz/bad
foo/bar
For which we've lost both (glob) markers,
we will match both lines and generate:
baz/bad (glob)
foo/bar (glob)
This retains any special annotations we have for lines.
Current pidfile logic will only keep the pid of the newest server, which is
not very useful if we want to kill all servers, and will become outdated if
the server auto exits after being idle for too long.
Besides, the server-side pidfile writing logic runs before chgserver gets
confighash so it's not trivial to append confighash to pidfile basename like
we did for socket file.
This patch removes --pidfile from the command starting chgserver and switches
to an alternative way (unlink socket file) to stop the server.
Previously, after matching a single line, any contiguous subsequent lines ending
with (?) would be added to the output and removed from the expected output.
This is a problem if the subsequent test output would have matched the consumed
(?) line, because it kept the optional line and then added a duplicate without
the (?) [1]. Instead, wait until there is nothing more to match before handling
the leftovers.
[1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-February/080197.html
At one point run-tests.py and test-run-tests.t worked and passed
under Python 3.5. Various changes to run-tests.py over the past
several months appear to have broken Python 3.5 compatibility.
This patch implements various fixes (all related to str/bytes type
coercion) to make run-tests.py and test-run-tests.t mostly work
again. There are still a few failures in test-run-tests.t due to
issues importing mercurial.* modules. But at least run-tests.py
seems to work under 3.5 again.
When reloading tests, run-tests.py was assuming that it could look
up the test by the basename, which only works if you are running
tests which are in the current directory.
This patch changes that lookup to use the full path. This is all
that was needed, and does not appear to cause any problems for
any of the existing testing work flows based on running the
suggested commands at the top of run-tests.py.
Motivation: In order to test Mercurial with Hypothesis (according
to https://www.mercurial-scm.org/wiki/HypothesisPlan) it is
useful to be able to generate temporary test files and execute
them. Generating temporary files in the tests/ directory leads to
a lot of suboptimal clutter.
The only consumer was test-treemanifest.t, which has been fixed.
In general, you should be able to use killdaemons.py to recycle
ports instead of going over 3 ports (HGPORT, HGPORT1, HGPORT2).
In the future, if you want to add a port, be sure to change
portneeded in _getport.
Adding a port reservation was too hard and someone did it wrong.
By refactoring, such reservations can be managed more safely.
This also adds documentation so that the next person who tries
is more likely to update all the places correctly.
Note that in this commit the reservation and consumers do not
match, that will be fixed in the next commit.
Because the temporary installation directory is shared between hg and chg,
--chg is not allowed if --with-hg option is specified. Also, --chg option
does not work on FreeBSD because "make" command is hard-coded. These
limitations can be improved later.
Almost all tests will fail with chg right now.
Unlike --with-hg=/path/to/chg, this option allows us to start and clean up
command servers in isolated environment. And we can specify the hg command
as well as the chg command.
If the executable is not named as "hg", TTest runner inserts alias. This
way, we can run tests with chg. But it is still warned because the alias
does not always work. We do "$BINDIR"/hg in a few places.
Similar to the previous patch, the .hg/store/meta/ directory does not
get copied when when using "hg clone --uncompressed". Fix by including
"meta/" in store.datafiles(). This seems safe to do, as there are only
a few users of this method. "hg manifest" already filters the paths by
"data/" prefix. The calls from largefiles also seem safe. The use in
verify needs updating to prevent it from mistaking dirlogs for
orphaned filelogs. That change is included in this patch.
Since the dirlogs will now be in the fncache when using fncachestore,
let's also update debugrebuildfncache(). That will also allow any
existing treemanifest repos to get their dirlogs into the fncache.
Also update test-treemanifest.t to use an a directory name that
requires dot-encoding and uppercase-encoding so we test that the path
encoding works.
Laurent's commit 56cdfddbd2ed still suffers from a race: by the
time the "job" function tries to assign to channels[channel], that
list has been truncated to empty. The result is that every job
thread raises an IndexError.
Earlier, I tried an approach of correctly locking channels, but
that caused run-tests to hang on KeyboardInterrupt sometimes.
This approach is strictly hackier, but seems to actually work
reliably.
This patch fixes a crash when both --json and --blacklist were given as
arguments of run-tests.py. Now, instead of crashing, we add an entry for
blacklisted tests in the json output to show that the tests were skipped.
Before this patch, it was possible for run-tests to crash on a race condition.
The race condition happens in the following case:
- the last test finishes and calls: done.put(None)
- the context switches to the main thread that clears the channels list
- the context switches to the last test mentioned above, it tries to access
channels[channel] and crashes
This happened to me while running run-tests.
This patch fixes the issue by clearing the channel before considering that the
test is done.
Threading is incompatible with most Python debuggers,
which makes debugging run-tests.py a real pain.
If there is only one test to run, skip using a thread for it.
Note that --debug is not compatible with debugging tests,
since it bypasses the output handling, which is where
much of the excitement is.
This patch adds to the json report the "diff" between expected and observed
result. This diff can be useful for automatically filing bug report on failing
tests.
