Before this patch, `RUNTESTDIR` is added to `PATH`, but `TESTDIR`
isn't.
This doesn't cause any problems, if `run-tests.py` runs in `tests`
directory of Mercurial source tree. In this case, `RUNTESTDIR` should
be equal to `TESTDIR`.
On the other hand, if `run-tests.py` runs in `tests` of third party
tools, commands in that directory should be executed with explicit
`$TESTDIR/` prefix in `*.t` test scripts. This isn't suitable for the
policy "drop explicit $TESTDIR from executables" of Mercurial itself
(see fcb1c7d8c36e).
BTW, fcb1c7d8c36e describes that "$TESTDIR is added to the path" even
though `TESTDIR` isn't added to `PATH` exactly speaking, because
`TESTDIR` and `RUNTESTDIR` weren't yet distinguished from each other
at that time.
This is a one of preparations for issue4677.
Before this patch, there is no way to refer files under `tests` or so
of Mercurial source tree, when `run-tests.py` runs in `tests` of third
party tools. In this case, `TESTDIR` refers the latter `tests`.
This prevents third party tools from using useful tools in Mercurial
source tree (e.g. `contrib/check-code.py`).
This patch adds `RUNTESTDIR` environment variable to refer `tests` of
Mercurial source tree, in which `run-tests.py` now running is
placed. For example, tests of third party tools can refer
`contrib/check-code.py` in Mercurial source tree as
`$RUNTESTDIR/../contrib/check-code.py`.
BTW, for similarity with `TESTDIR` referring `test*s*` directory,
newly added environment variable isn't named as `RUNTEST*S*DIR`. In
addition to it, the corresponded local variable is also named as
`runtestdir`.
This is a one of preparations for issue4677.
Before this patch, `run-tests.py` executes `hghave` by the path
relative to `TESTDIR` (= cwd of `run-tests.py` running).
This prevents third party tools for Mercurial from running
`run-tests.py`, which is placed in `tests` of Mercurial source tree,
in `tests` of own source tree. In such cases, `TESTDIR` refers the
latter `tests`, and `hghave` doesn't exist in it.
This is a one of preparations for issue4677.
When the test engine fails to match output on a line marked with (?),
it will simply continue to the next expected line and try again. This
allows simplifying tests that have either version-specific or
non-fixed behavior, for instance:
$ coin-flip
heads (?)
tails (?)
(There's no form of back-tracking attempted, so optional matches
should be specific.)
We have started to isolate extra usecases for developer-only output
that is not a warning. As the section has the fairly generic name
'devel' it makes sense to tuck them there. As a result, 'all' becomes
a bit misleading so we rename it to 'all-warnings'. This will break
some developer setups but the tests are still fine and developers will
likely spot this change.
Now that http://bugs.python.org/issue24230 is fixed (thanks to Gregory
Smith for that quick response!) we can drop one more ugly hack around
path handling. Tests still pass in 3.5 with this cleaner version, as
well as in 2.6.
This fixes the python3.5 build. We'll presumably want to build our own
helper function or use 2to3 for this on the main source tree, but for
run-tests we want a single-source version that works in 2.6 and 3.5.
Python 2.7.3 on Windows doesn't have os.WIFEXITED, and the test output looked
like this before I interrupted it.
$ ./run-tests.py --local -j2 -t700
EEEEEEEEEEEEEEEEEEEEEE
This also cleans up the mkdtemp code mentioned in the previous patch.
At this point, the remaining callsites of .{en,de)code() are in the
following categories:
Handling escaped lines in .t files
-----------------------------------
It seems eminently reasonable to me for us to declare that .t files
are valid utf-8, and that any escape sequences we see in .t files
should be valid unicode_escape sequences.
Making error text safe for cdata blocks for xml error reports
-------------------------------------------------------------
This is a point where we're already basically screwed, and we're
simply trying to do something "good enough" that the xml output will
be vaguely useful to the user. Punting here seems fine, and we should
probably stick to the same encoding here that we used in the previous
section.
This doesn't fix the probably-wrong utf-8 encoding choice, it just
starts the process of encapsulating all the path handling in run-tests
in a single place.
One known-path use of .encode() remains: it's related to use of
mkdtemp, and it will be fixed in a followup patch once we have a
companion _strpath() helper function to go from bytes to a str, as we
need to file a bug about mkdtemp upstream.
We depend on both stdlib functionality (difflib.diff_bytes) and
language behavior (bytes formatting) introduced in 3.5, so let's try
and prevent some useless bug reports before they happen.
While in the throes of a recent run-tests adventure, I found it useful
to have profiler output for the testrunner itself. Adding it was
simple enough and seems worth keeping around.
This is useful information to understand what is taking time in tests. Both are
included because I can see myself sorting this output using shell script. Having
both data makes it much easier than extracting 'start'+'real'.
We currently have information about how long each test took, but we have no data
about their actual scheduling. So we now track when a test started and stopped
(in the referential of the whole tests run) to expose this information.
The data is currently in the json only because the json output is meant to be
extensible. Later changeset will includes this data in the text output and we
will be happy to bikeshed its formating there.
Yes, "end" is actually just "start" + "time", but computing it an including it
in the output is simple, cheap and convenient.
As check-code is actually quite long to run for its file size, this prevent them
to be scheduled too late during the test run.
This reduces my typical test run from 107 seconds to 90 seconds
The 'test-check-code-hg.t' file is not big enough to be prioritized properly.
As a result my tests run often spend about 15 seconds running only it at the
end of its tests run. We make the "slow" mechanism a bit smarter to adjust the
extra weight of each category independently in a future patch.
This introduces a bname attribute for when we need to use the test
name in a path, and retains name for when we need it as a
string. Sadly, we seem to need both of these.
test-run-tests.t still passes fine on Python 2.6. run-tests.py --local
no longer fails with syntax errors, and now fails looking for xrange.
Most changes done with
2to3 -w -f numliterals -f except -f print tests/run-tests.py tests/killdaemons.py
after which one import was fixed in run-tests and a __future__ import
was added.
