47985606a0ae (run-tests: check if stream is a tty before using color,
2017-07-18) made the check redundant but forgot to remove it.
Differential Revision: https://phab.mercurial-scm.org/D116
At first I updated the color field of the 'options' object (from the
CLI parser), but then I decided to put it directly on the test case
object itself to avoid mutating the shared object (even though all
tests would have the same value).
Differential Revision: https://phab.mercurial-scm.org/D114
Previous implementation (ccf66c9bf5af) checked only if sys.stderr was a tty
which was less general. Also makes sure that colors is never used if pygments is
not available, irrespective of --color flag value.
e80041832e introduced support to color the output of tests but used pygments
without checking whether it's installed or not. That breaks test-run-tests.t for
machines which don't have pygments installed. This patch conditionalize the
color test in test-run-tests.t and also add a check to make sure pygments is
installed before using that.
More Windows sadness. Maybe someone can figure out how to make win32 color
work, but I think we avoid importing stuff from the mercurial package in this
module. On the plus side, this conditionalizes away a test failure.
The output of run-tests has no formatting by default, which hampers readability.
This patch colors the diff output when pygments is available. To avoid coloring
even when pygments is available, use --color never.
Since os.environ may be overridden in run-tests.py, several important
variables such as PATH weren't restored.
I don't like the idea of using the system hg *by default* because the
executable and the configs are out of our control. But I don't mind as
long as the tests pass.
Update the code to correctly anchor the expression on the end of the name, to
require that the entire name match this expression. It was already anchored at
the start by using re.match(), but this does not anchor it at the end.
Update the syshgenv function to attempt to completely restore the original
environment, rather than only updating a few specific variables. run_tests.py
now generates a shell script that can be used to restore the original
environment, and syshgenv sources it.
This is a bit more complicated than the previous code, but should do a better
job of running the system hg in the correct environment.
I've tested it on Linux using python 2.x, but let me know if it causes issues
in other environments. I'm not terribly familiar with how the tests get run on
Windows, for instance, and how the environment needs to be updated there.
When running the tests, define ORIG_PATH and ORIG_PYTHONPATH environment
variables that contain the original contents of PATH and PYTHONPATH, before
they were modified by run-tests.py
This will make it possible for tests to refer to the original contents of these
variables if necessary. In particular, this is necessary for invoking the
correct version of hg for examining the local repository (the mercurial
repository itself, not the temporary test repositories). Various tests examine
the local repository to check the file lists and contents of commit messages.
The "#testcases" feature introduced by 250afd791085 has issues with "-i"
because "-i" uses "test.name.endswith('.t')" to test if a test is .t or not.
test.name could now be something like "test-foo.t (caseA)" so the above
endswith test is no longer valid.
This patch changes the test to use "self.path" which won't have the issue.
The .t file is both test input and reference output. They should always
match. However we have different code paths to read reference output
(Test.__init__ -> Test.readrefout) and test input (TTest._run) so they might
be inconsistent if somethings change the file between those two functions.
This patch assigns "lines" read by "_run" back to "_refout" if "_refout" is
not None (with --debug, see Test.readrefout) so reference output and test
input will always match.
The race condition is like:
1. run-tests.py reads test-a.t as reference output, content A
2. run-tests.py runs the test (which could be content B, another race
condition fixed by the next patch, but assume it's content A here)
3. something changes test-a.t to content C
4. run-tests.py compares test output (content D) with content A
5. with "-i", run-tests.py prompts diff(A, D), while the file has content
C instead of A at this time
This patch detects the above case and tell the user to rerun the test if
they want to apply test changes.
The only call site called addFailure before raising, which is
exactly what the failure exception handler does. So this
complexity is not needed.
We have test coverage of this "server failed to start" scenario
and nothing appeared to change.
The previous changeset removed the last caller of addWarn(). So,
we rip out that method and all the code related to tracking warned
tests in the results system.
There was even a comment saying we may want to fold warned tests into
the "failed" state, which is what the previous changeset did.
We would raise this if a test didn't return a result code. AFAICT
this can only occur if there is a logic error in the test harness
itself.
I don't think it is worth the code complexity to distinguish this
failure scenario from a regular test failure.
When hghave testing goes awry, the output order was changing on Windows.
diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -920,10 +920,10 @@
> EOF
> done
$ rt -j 2
- ....
+ ....skipped: unknown feature: notarealhghavefeature\r (esc)
+
+
# Ran 5 tests, 0 skipped, 0 warned, 0 failed.
- skipped: unknown feature: notarealhghavefeature
-
$ cd ..
$ rm -rf broken
Since 'skipped: unknown feature: notarealhghavefeature\n\n' is printed to stdout
and the rest to stderr, it seems like maybe stdio isn't line buffered on
Windows. When a program exits, stdout is flushed before stderr[1].
[1] https://blogs.msdn.microsoft.com/oldnewthing/20060519-09/?p=31133
In practice this doesn't appear to have been true for some time - we
reference Python using the $PYTHON variable in all the tests now
(which we have to for PyPy and Python 3), and I've been using
~/.../python.exe to test with tip of the cpython 3.6 release branch
while working on manifest tests in Python 3 and everything seems to be
just fine. The only real observable difference from this change is
that I stop getting a warning about python.exe not being a thing on
$PATH, which seems like an improvement.
Some test runners are interested in listing tests, so they can do their own
filtering on top (usually based on attributes like historically observed
runtime). Add support for that.
That has (and still does) caused the test to be skipped, but without
this fix it was possible to exit this block of code without clearing
the output channel, which poisoned the channel list for later test
method runs. Fix this by always clearing the channel in a finally.
The test for this is somewhat unfortunate. Sadly, I couldn't get a way
to reproduce this with less than 2n+1 test cases, nor could I get it
to reproduce reliably without the sleep statements. It's also crucial
that the test with the broken #if be smaller (in terms of byte count)
than the sleeping tests, so that it runs first and would poison the
channel list prior to another test needing that entry from the list.
I hit a weird corner case in run-tests where a test that caused an
exception to be raised was breaking everything with an unbound
variable error a few lines down because channel was never getting set
in this for loop. By adding an `else` clause to this for loop, we can
explode right away if we can't find a channel and give the developer a
better chance at figuring out what's going on.
Now that we have the default-date by default and all code have been updated,
remove the old commands alias that forced the date as they are not longer
useful.
Writing tests now should be easier for everyone now that all dates should be
stable.
These are the obvious ones. There is tons of code in this file
implementing features from unittest that weren't present in
Python 2.6. But that's for other patches.
When running tests on Windows (via msys), user sometimes does not want to run
them against source hg, but against compiled hg.exe. For that purpose,
--with-hg option can be used, but currently run-tests.py prints a warning if
the value of this argument is not a file with basename 'hg'. This patch allows
such file to be 'hg.exe'.
Sometimes we want to run similar tests with slightly different
configurations. Previously we duplicate the test files. This patch
introduces special "#testcases" syntax that allows a single .t file to
contain multiple test cases.
Defined cases could be tested using "#if".
For example, if a test should behave the same with or without an
experimental flag, we can add the following to the .t header:
#testcases default experimental-a
#if experimental-a
$ cat >> $HGRCPATH << EOF
> [experimental]
> feature=a
> EOF
#endif
The "experimental-a" block won't be executed when running the "default" test
case.
Previously the word "test" was used for both a Test instance and a path or
test dict. This patch renames them so it's clear that "testdesc" is the
dict, and "test" is the instance.
Previously, we use path to identify a test. A later patch adds more
information so a path is not enough to identify a test. So we change it to a
dictionary.
Our current deprecation warning mechanism relies on ui object. They are case
where we cannot have access to the UI object. On a general basis we avoid using
the python mechanism for deprecation warning because up to Python 2.6 it is
exposing warning to unsuspecting user who cannot do anything to deal with them.
So we build a "safe" strategy to hide this warnings behind a flag in an
environment variable. The test runner set this flag so that tests show these
warning. This will help us marker API as deprecated for extensions to update
their code.
Duplicating entire tests just because the output is different is both error
prone and can make the tests harder to read. This harnesses the existing '(?)'
infrastructure, both to improve readability, and because it seemed like the path
of least resistance.
The form is:
$ test_cmd
output (hghave-feature !) # required if hghave.has_feature(), else optional
out2 (no-hghave-feature2 !) # req if not hghave.has_feature2(), else optional
I originally extended the '(?)' syntax. For example, this:
2 r4/.hg/cache/checkisexec (execbit ?)
pretty naturally reads as "checkisexec, if execbit". In some ways though, this
inverts the meaning of '?'. For '(?)', the line is purely optional. In the
example, it is mandatory iff execbit. Otherwise, it is carried forward as
optional, to preserve the test output. I tried it the other way, (listing
'no-exec' in the example), but that is too confusing to read. Kostia suggested
using '!', and that seems fine.
Previously, if a series of optional output lines marked with '(?)' had a (glob)
in one of the first lines, the output would be reordered such that it came last
if none of the lines were output. The (re) declaration wasn't affected, which
was helpful in figuring this out. There were no tests for '(re) (?)' so add
that to make sure everything plays nice.
Hooks related to the transaction are aware of the transaction id. By definition
this txn-id is unique and different for each transaction. As a result it can
never be predicted in test and always needs matching. As a result, touching any
like with this data is annoying. We solve the problem once and for all by
installing an automatic replacement. In test, this will now show as:
TXNID=TXN:$ID$
This is similar to what 9704c8e70d2d does. Since 1363aaf74791 has changed
"127.0.0.1" to "$LOCALIP". The glob pattern needs update accordingly. It is
expected to fix tests running in some BSD jails.
Plenty of tests break when "make tests" is run while environment
variables "HGPLAIN" or "HGPLAINEXCEPT" are set (test "test-obsolete-
checkheads.t" is just a single example).
This patch causes script "run-tests.py" to also remove these two
variables from the environment the tests are executed in.
Previously, we only set web.ipv6 if IPv6 is used, but not on the IPv4 case.
Since we already have set web.address, it makes sense to move "web.ipv6" out
from "extra config options".
Previously, "hg serve" will listen on "", which is not clear which interface
it will actually listen on - it could listen on all interfaces (ex. 0.0.0.0
on IPv4).
The run-tests.py script only checks "localhost" for available ports. So
let's make it the same for "hg serve" by explicitly setting "web.address" to
"localhost".
This resolves some IPv6 EADDRINUSE errors.
This patch replaces hardcoded 127.0.0.1 with $LOCALIP in all tests.
Till now, the IPv6 series should make tests pass on common IPv6 systems
where the local device has the address "::1" and the hostname "localhost"
resolves to "::1".
Previously, tests hard-code local IP address as "127.0.0.1". That won't work
for IPv6.
This patch exports the $LOCALIP environment variable, which is set to "::1"
if we decide to use IPv6.
Previously, run-tests.py only exports HGPORT, and scripts in tests do not
know if IPv6 should be used. And that breaks scripts like dumbhttp.py which
always uses IPv4.
This patch makes run-tests.py export HGIPV6, which can help test scripts
like dumbhttp.py and tinyproxy.py to decide whether to use IPv6 or not.
To make IPv6 work, there are multiple areas that need to fix. Before they
all get fixed, use IPv4 by default.
This should fix tests caused on IPv6 systems.
As explained by the previous patch, we need to set "web.ipv6=True" if we
decide to use IPv6. Otherwise "hg serve" will still try to listen on IPv4.
This patch makes it so by appending web.ipv6 to "extra configs".
This patch was tested in a Linux system with IPv6, by the following steps:
1. Change hgweb/server.py temporarily to write a file if
IPv6HTTPServer.__init__ is called.
2. run-tests.py -l --keep-tmpdir test-serve.t
3. Check the generated .hgrc, make sure it sets web.ipv6=1.
4. Check the log file to make sure IPv6HTTPServer.__init__ is called.
As explained by the previous patch, checkportisavailable() should only check
the preferred family - either IPv4 or IPv6, not both.
This patch makes it so.
Previously, checkportisavailable returns True if the port is free either on
IPv4 or IPv6, but the hg server only uses IPv4 by default. That leads to
issues when IPv4 port is not free but the IPv6 one is.
To address that, run-tests should stick with either IPv4 or IPv6. This patch
adds a function similar to checkportisavailable to test if IPv6 is
available, and assigns the result to a variable.
The new function was tested in a Linux system script with the following
steps:
1. Run "ip addr del ::1/128 dev lo" to delete lo's IPv6 address,
Confirm checkipv6available() returns False.
2. Run "ip addr add ::1/128 dev lo" to add back lo's IPv6 address.
Confirm checkipv6available() returns True.
3. Start a web server taking the 8000 port.
Confirm checkipv6available(8000) is still True.
This is a follow-up of "runtests: check ports on IPv6 address". On some
platforms, "socket.AF_INET6" exists while that does not necessarily mean the
platform support IPv6 - when initializing a socket using "socket.socket", it
could fail with EPROTONOSUPPORT. So treat that as "Port unavailable".
Previously, checkportisavailable only checks ports on the IPv4 address. This
patch makes it check IPv6 as well. It'll be useful if "localhost" does not
have an IPv4 address, or its IPv4 address does not exist somehow.
We do this so that any linters installed via pip install --user don't
break. See https://docs.python.org/2/library/site.html#site.USER_BASE
for a description of what this nonsense is all about.
An alternative would be to not set HOME, but that'll cause other
problems (see issue2707), or to forward every single path entry from
sys.path in PYTHONPATH (which seems sketchy in its own way).
Some systems don't have a 127/8 address for localhost (I noticed this
on a FreeBSD jail). In order to work around this, use 127.0.0.1 as a
glob pattern. A future commit will update needed output lines and add
a requirement to check-code.py.
This patch includes addition of absolute_import and print_function to the
files where they are missing. The modern importing conventions are also followed.
In py2, json.dumps includes a trailing space after a comma at the
end of lines. The py3 behavior which omits the trailing space is
preferable, so we're going to strip it.
Without this, my python 2.6 virtualenv test run with --pure and
--local fails with:
+ ImportError: Python minor version mismatch: The Mercurial extension modules were compiled with Python 2.7.8, but Mercurial is currently using Python with sys.hexversion=33950192: Python 2.6.9 (unknown, Apr 13 2016, 12:40:12)
+ [GCC 4.9.2 20141101 (Red Hat 4.9.2-1)]
+ at: ~/hg/py26/bin/python
Several tests fail with chg for several reasons such as loaded chgserver
extension, running uisetup() per server instead of per runcommand, etc.
Since these tests can't/shouldn't be changed to be chg friendly, we need
a flag to skip them.
This patch explicitly drops CHGHG environment if chg isn't involved. This
way, hghave can just check if CHGHG exists.
Without this, sometimes installerrs generated errors
about no such file. It also did not work well when you
had multiple tests runners running around.
It also did not make sense to pollute the repository test
directory with the log file.
The regular expression in use passed tests because the test repo only
has single-digit changesets present. When I tried to use this for real
today, it broke, because the regular expression would only match a
single digit.
https://xkcd.com/1171/, or something like that.
72b40f92c680 enabled lines that were not matched to be found later in cases of jitter.
Unfortunately, in this model an optional line would always jitter to the end when
it is not present. That is not ideal.
It would be possible to do better, by queuing all writes until the end in case
an optional line jitters, but for now, it is simpler to assume optional lines
have a fixed place in the stream.
Before this patch, if --chg or --with-chg is specified, all tests are using
the same chgserver socket. Since the chg client holds a lock when it starts a
new server, and every test needs at least a new chg server due to different
HGRCPATH affecting the confighash, the result is a lot of tests will be
timed out if -j is large (for example, 50 or 100).
This patch solves the issue by using different chg socket directories for
different tests.
Currently, very few parts of Mercurial run under Python 3, notably the
test harness.
We want to write tests that run Python 3. For example, we want to
extend test-check-py3-compat.t to parse and load Python files.
However, we have a problem: finding appropriate files requires
running `hg files` and this requires Python 2 until `hg` works
with Python 3.
As a temporary workaround, we add --with-python3 to the test harness
to allow us to define the path to a Python 3 interpreter. This
interpreter is made available to the test environment via $PYTHON3 so
tests can run things with Python 3 while the test harness and `hg`
invocations continue to run from Python 2. To round out the feature,
a "py3exe" hghave check has been added.
Instead of treating expected output as happening in a precise order,
and assuming that if a line is missing it will never happen,
assume that expected output is a prioritized list of likely matching
lines.
This means that if:
foo/bar (glob)
baz/bad (glob)
changes to:
baz/bad
foo/bar
instead of generating:
baz/bad
foo/bar
For which we've lost both (glob) markers,
we will match both lines and generate:
baz/bad (glob)
foo/bar (glob)
This retains any special annotations we have for lines.
Current pidfile logic will only keep the pid of the newest server, which is
not very useful if we want to kill all servers, and will become outdated if
the server auto exits after being idle for too long.
Besides, the server-side pidfile writing logic runs before chgserver gets
confighash so it's not trivial to append confighash to pidfile basename like
we did for socket file.
This patch removes --pidfile from the command starting chgserver and switches
to an alternative way (unlink socket file) to stop the server.
Previously, after matching a single line, any contiguous subsequent lines ending
with (?) would be added to the output and removed from the expected output.
This is a problem if the subsequent test output would have matched the consumed
(?) line, because it kept the optional line and then added a duplicate without
the (?) [1]. Instead, wait until there is nothing more to match before handling
the leftovers.
[1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-February/080197.html
At one point run-tests.py and test-run-tests.t worked and passed
under Python 3.5. Various changes to run-tests.py over the past
several months appear to have broken Python 3.5 compatibility.
This patch implements various fixes (all related to str/bytes type
coercion) to make run-tests.py and test-run-tests.t mostly work
again. There are still a few failures in test-run-tests.t due to
issues importing mercurial.* modules. But at least run-tests.py
seems to work under 3.5 again.
When reloading tests, run-tests.py was assuming that it could look
up the test by the basename, which only works if you are running
tests which are in the current directory.
This patch changes that lookup to use the full path. This is all
that was needed, and does not appear to cause any problems for
any of the existing testing work flows based on running the
suggested commands at the top of run-tests.py.
Motivation: In order to test Mercurial with Hypothesis (according
to https://www.mercurial-scm.org/wiki/HypothesisPlan) it is
useful to be able to generate temporary test files and execute
them. Generating temporary files in the tests/ directory leads to
a lot of suboptimal clutter.
The only consumer was test-treemanifest.t, which has been fixed.
In general, you should be able to use killdaemons.py to recycle
ports instead of going over 3 ports (HGPORT, HGPORT1, HGPORT2).
In the future, if you want to add a port, be sure to change
portneeded in _getport.
Adding a port reservation was too hard and someone did it wrong.
By refactoring, such reservations can be managed more safely.
This also adds documentation so that the next person who tries
is more likely to update all the places correctly.
Note that in this commit the reservation and consumers do not
match, that will be fixed in the next commit.
Because the temporary installation directory is shared between hg and chg,
--chg is not allowed if --with-hg option is specified. Also, --chg option
does not work on FreeBSD because "make" command is hard-coded. These
limitations can be improved later.
Almost all tests will fail with chg right now.
Unlike --with-hg=/path/to/chg, this option allows us to start and clean up
command servers in isolated environment. And we can specify the hg command
as well as the chg command.
If the executable is not named as "hg", TTest runner inserts alias. This
way, we can run tests with chg. But it is still warned because the alias
does not always work. We do "$BINDIR"/hg in a few places.
Similar to the previous patch, the .hg/store/meta/ directory does not
get copied when when using "hg clone --uncompressed". Fix by including
"meta/" in store.datafiles(). This seems safe to do, as there are only
a few users of this method. "hg manifest" already filters the paths by
"data/" prefix. The calls from largefiles also seem safe. The use in
verify needs updating to prevent it from mistaking dirlogs for
orphaned filelogs. That change is included in this patch.
Since the dirlogs will now be in the fncache when using fncachestore,
let's also update debugrebuildfncache(). That will also allow any
existing treemanifest repos to get their dirlogs into the fncache.
Also update test-treemanifest.t to use an a directory name that
requires dot-encoding and uppercase-encoding so we test that the path
encoding works.
Laurent's commit 56cdfddbd2ed still suffers from a race: by the
time the "job" function tries to assign to channels[channel], that
list has been truncated to empty. The result is that every job
thread raises an IndexError.
Earlier, I tried an approach of correctly locking channels, but
that caused run-tests to hang on KeyboardInterrupt sometimes.
This approach is strictly hackier, but seems to actually work
reliably.
This patch fixes a crash when both --json and --blacklist were given as
arguments of run-tests.py. Now, instead of crashing, we add an entry for
blacklisted tests in the json output to show that the tests were skipped.
Before this patch, it was possible for run-tests to crash on a race condition.
The race condition happens in the following case:
- the last test finishes and calls: done.put(None)
- the context switches to the main thread that clears the channels list
- the context switches to the last test mentioned above, it tries to access
channels[channel] and crashes
This happened to me while running run-tests.
This patch fixes the issue by clearing the channel before considering that the
test is done.
Threading is incompatible with most Python debuggers,
which makes debugging run-tests.py a real pain.
If there is only one test to run, skip using a thread for it.
Note that --debug is not compatible with debugging tests,
since it bypasses the output handling, which is where
much of the excitement is.
This patch adds to the json report the "diff" between expected and observed
result. This diff can be useful for automatically filing bug report on failing
tests.
Vaguely empirical observations:
* ".py" tests are about an order of magnitude faster than ".t" tests
* dividing size by 1000 gives an approximation to wall-clock
run time (in seconds) that is not completely ridiculous.
This gives one line of output per second with one column per -j level
that allows analyzing test scheduling problems. First 24 seconds of
output at -j 30 looks like this:
0 .
1 = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = s.
2 c c o c r l g r s s = c p = c h c a h c g c h c b c c l l c ss
3 h o b o e a e u u u c o a h o e o c g o l h g h u o = a o = s
4 e n s n b r n n b b m t g n l n l w n o e w e n n e r g i .
5 c t o = a g d - r r = m c w v p v . e v g c e c d v x g . m
6 k r l r s e o t e e b a h e e . e . b e . k b k l e t e . p
7 - i e e e f c e p p u n b b r . r . - r . - - - e r e f . o .
8 p b t v - i . s o o n d o d t . t . c t . c s = 2 t n i . r
9 y - e s c l . t - . d - m i - . - . o - . o y r - - s l . t
10 3 p - e h e . s s . l t b r s . s . m s . d m e f s i e . .
11 - e c t e s . . v . e e . . v . v . m v . e r n o v o s . .
12 c r h . c - . . n . 2 m . . n . n . a n . . e a r n n . . .
13 o f e . k u . . . . - p . . - . - . n - . . v m m - . . . .
14 m . c . - p . . . . e l . . s . m . d s . . . e a e . . . .
15 p . k . r d . . . . x a . . i . o . s o . . . - t n . . . .
16 a . h . e a . . . . c t . . n . v . . u . . . m . c . . . .
17 t . e . s t . . . . h e . . k . e . . r . . . e . o . . . .
18 . . a . t e . . . . a . . . . . . . . c . . . r . d . . . .
19 . . d . o . . . . . n . . . . . . . . e . . . g . i . . . .
20 . . s . r . . . . . g . . . . . . . . . . . . e . n . . . .
21 . . . . e . . . . . e . . . . . . . . . . . . 2 . g . . . .
22 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
24 . . . . . . . . . . . . . . . . . . . . . . . . . = . . . . ^C
Test names read off vertically, beginning with '='. Idle time (not
shown) appears as blank space.
The scheduler would like to order test execution by expected run-time,
but doesn't know much about how long a test will run. It thus uses
test size as a proxy for run-time. By tweaking these weights we can
keep CPUs more evenly busy and thus finish sooner.
In particular, this change pushes the three currently longest-running
tests closer to the beginning:
test-largefiles-update.t
test-run-tests.t
test-gendoc.t
As the largefiles test is currently the long pole of the test suite
with higher -j factors, the sooner it's started, the sooner the tests
can end.
We also up the weight on some shorter but long-running tests that
could have previously delayed completion with low -j factors by
running very close to the end.
test-run-tests and test-hghave call run-tests;
if you don't have a working build environment, and you are trying
to use --pure, then if they don't use --pure or --with-hg,
they'll break.
I want to add tests for our packaging rules, but those necessarily run
a whole build, or possibly two if both native packaging and docker are
available. This lets us flag such tests with a `#require slow` so that
they don't unnecessarily slow down normal test runs.
When running tests with -j100 or so on a large machine, I see this
os.remove call failing semi-regularly. Since it's not really a problem
when the file is already gone, just suppress the error in that case.
Before this patch, `RUNTESTDIR` is added to `PATH`, but `TESTDIR`
isn't.
This doesn't cause any problems, if `run-tests.py` runs in `tests`
directory of Mercurial source tree. In this case, `RUNTESTDIR` should
be equal to `TESTDIR`.
On the other hand, if `run-tests.py` runs in `tests` of third party
tools, commands in that directory should be executed with explicit
`$TESTDIR/` prefix in `*.t` test scripts. This isn't suitable for the
policy "drop explicit $TESTDIR from executables" of Mercurial itself
(see fcb1c7d8c36e).
BTW, fcb1c7d8c36e describes that "$TESTDIR is added to the path" even
though `TESTDIR` isn't added to `PATH` exactly speaking, because
`TESTDIR` and `RUNTESTDIR` weren't yet distinguished from each other
at that time.
This is a one of preparations for issue4677.
Before this patch, there is no way to refer files under `tests` or so
of Mercurial source tree, when `run-tests.py` runs in `tests` of third
party tools. In this case, `TESTDIR` refers the latter `tests`.
This prevents third party tools from using useful tools in Mercurial
source tree (e.g. `contrib/check-code.py`).
This patch adds `RUNTESTDIR` environment variable to refer `tests` of
Mercurial source tree, in which `run-tests.py` now running is
placed. For example, tests of third party tools can refer
`contrib/check-code.py` in Mercurial source tree as
`$RUNTESTDIR/../contrib/check-code.py`.
BTW, for similarity with `TESTDIR` referring `test*s*` directory,
newly added environment variable isn't named as `RUNTEST*S*DIR`. In
addition to it, the corresponded local variable is also named as
`runtestdir`.
This is a one of preparations for issue4677.
Before this patch, `run-tests.py` executes `hghave` by the path
relative to `TESTDIR` (= cwd of `run-tests.py` running).
This prevents third party tools for Mercurial from running
`run-tests.py`, which is placed in `tests` of Mercurial source tree,
in `tests` of own source tree. In such cases, `TESTDIR` refers the
latter `tests`, and `hghave` doesn't exist in it.
This is a one of preparations for issue4677.
When the test engine fails to match output on a line marked with (?),
it will simply continue to the next expected line and try again. This
allows simplifying tests that have either version-specific or
non-fixed behavior, for instance:
$ coin-flip
heads (?)
tails (?)
(There's no form of back-tracking attempted, so optional matches
should be specific.)
We have started to isolate extra usecases for developer-only output
that is not a warning. As the section has the fairly generic name
'devel' it makes sense to tuck them there. As a result, 'all' becomes
a bit misleading so we rename it to 'all-warnings'. This will break
some developer setups but the tests are still fine and developers will
likely spot this change.
Now that http://bugs.python.org/issue24230 is fixed (thanks to Gregory
Smith for that quick response!) we can drop one more ugly hack around
path handling. Tests still pass in 3.5 with this cleaner version, as
well as in 2.6.
This fixes the python3.5 build. We'll presumably want to build our own
helper function or use 2to3 for this on the main source tree, but for
run-tests we want a single-source version that works in 2.6 and 3.5.
Python 2.7.3 on Windows doesn't have os.WIFEXITED, and the test output looked
like this before I interrupted it.
$ ./run-tests.py --local -j2 -t700
EEEEEEEEEEEEEEEEEEEEEE
This also cleans up the mkdtemp code mentioned in the previous patch.
At this point, the remaining callsites of .{en,de)code() are in the
following categories:
Handling escaped lines in .t files
-----------------------------------
It seems eminently reasonable to me for us to declare that .t files
are valid utf-8, and that any escape sequences we see in .t files
should be valid unicode_escape sequences.
Making error text safe for cdata blocks for xml error reports
-------------------------------------------------------------
This is a point where we're already basically screwed, and we're
simply trying to do something "good enough" that the xml output will
be vaguely useful to the user. Punting here seems fine, and we should
probably stick to the same encoding here that we used in the previous
section.
This doesn't fix the probably-wrong utf-8 encoding choice, it just
starts the process of encapsulating all the path handling in run-tests
in a single place.
One known-path use of .encode() remains: it's related to use of
mkdtemp, and it will be fixed in a followup patch once we have a
companion _strpath() helper function to go from bytes to a str, as we
need to file a bug about mkdtemp upstream.
We depend on both stdlib functionality (difflib.diff_bytes) and
language behavior (bytes formatting) introduced in 3.5, so let's try
and prevent some useless bug reports before they happen.
While in the throes of a recent run-tests adventure, I found it useful
to have profiler output for the testrunner itself. Adding it was
simple enough and seems worth keeping around.
