Before this patch, 'detailed' is used as the default of '[ui]
mergemarkers'. This embeds non-ASCII characters in tags, branches,
bookmarks, author and/or commit descriptions into merged files in the
encoding specified by '--encoding' global option, 'HGENCODING' or
other locale setting environment variables.
But, if files to be merged use another encoding, this behavior breaks
consistency of encoding in merged files.
For example, ISO-2022-JP or EUC-JP are sometimes used as the file
encoding for Japanese characters, because of historical and/or
environmental reasons, even though UTF-8 or Shift-JIS are ordinarily
used as the terminal encoding.
This can't be resolved automatically, because Mercurial doesn't aware
encoding of managed files.
This patch uses 'basic' as the default of '[ui] mergemarkers' to avoid
embedding encoding sensitive characters for safety.
This patch puts '[ui] mergemarkers = detailed' into default hgrc file
for tests, to reduce changes for tests in this patch.
Otherwise diff may get separated from the corresponding prompt by
other threads. This required moving the interactive prompting from one
helper method to another.
This patch fixes a regression recently introduced by a refactoring.
Previously when failure occurs while testing with '--interactive' was enable,
it didn't prompt user by asking whether he wants to accept this failure
changes or not.
This was happening beacuse of the 'if' condition
if ret or not self._options.interactive or \
not os.path.exists(test.errpath):
Everytime failure occurs, this condition gets true and returns back even
when '--interactive' is enabled. This condition don't led the function to
execute further, which consist the '--interactive' functionality.
Now, on failure with '--interactive' enabled, it prompts user whether he wants
to accepts failure changes or not.
If yes then test gets passed and returns true, else test gets failed.
On every failure, results gets stored in "self.failures.append((test, reason))"
But if failure changes accepted by user then test must get "pop out" from
failed test list.
This patch fixes a regression recently introduced by a refactoring. (see
a2d588d82fa2 and about 200 previous changesets from Gregory Szorc)
While retesting, that is when '--retest' is enabled, only failure tests run
and others either skipped or ignored.
During retesting, "result.testsRun" holds the count of failure test that has
run. But as while printing output, we have subtracted the skipped and ignored
count from "result.testsRun". Therefore, to make the count remain
the same, we need to add skipped and ignored count before printing.
When '--retest' option is enabled then skipped test should not produce 'i' mark.
This fixes a regression introduced by a2d588d82fa2 and about 200 previous
changesets from Gregory Szorc.
This patch fixes a regression recently introduced by a refactoring (see
a2d588d82fa2 and about 200 previous changesets from Gregory Szorc). '!' mark
denotes that the test gets failed while testing.
This patch fixes a regression recently introduced by a refactoring (see
a2d588d82fa2 and about 200 previous changesets from Gregory Szorc). It produce
an error message everytime with a test filename which gets fail while testing
except at one condition when '--nodiff' option is enabled.
Earlier refactoring of run-tests.py accidentally broke --interactive
and external diff generation by not having .err files written before
they are consulted. This patch fixes that.
TestResult is the thing that captures all our test results. It's logical
for diff viewing to be handled there and not inside Test.
While writing this patch, it was discovered that Test.fail() was
printing redundant test result output. This has been removed.
Arguments controlling diffs have been removed from Test.__init__.
All options are now passed as arguments and we no longer need options.
This enables us to instantiate Test from "plain old data" types. Since
options must be given as arguments, it also makes people think harder
about adding things that may not belong in Test. This will help ensure a
proper separation of responsibility going forward.
Instead of making port numbers derived from count and a global initial
port, we now pass a start port to Test.__init__ and do the calculation
at a higher level.
Upcoming patches will move the execution of tests to separate processes.
To facilitate this, it makes sense to move logic out of Test.
Furthermore, test filtering is logically the domain of the test runner,
not the test itself.
This patch interrupts our mini series of factoring options into named
arguments because filtering consults many options and it is easier to
move this logic out of Test sooner so we don't have to introduce
arguments at all.
We no longer access any attributes on TestRunner besides options, so we
stop passing a TestRunner to Test.__init__ and now pass the options
data structure instead.
Subsequent patches will move accessed options attributes into named
arguments.
This patch starts a mini series of moving arguments to Test.__init__
from semi-complex data structures (such as the command options) to named
arguments. This will allow Test instances to be more easily instantiated
from other contexts. This improves the ability to run Mercurial tests in
new and different environments.
Previously, a Test's path came from the base directory of all tests and
a filename leaf. There is not a strong reason why an absolute test path
can not be specified.
This change isn't strictly necessary. But it does enable scenarios such
as more easily running tests from multiple, non-sibling directories.
TestResult has facilities for recording when tests start and stop. It
makes sense to move execution time recording into TestResult.
In addition, output generation is being moved into TestResult, a class
that has a concept of an output buffer (and a lock).
Now that we execute all tests via unittest, the MercurialTest wrapper is
not necessary. This patches moves the logic from MercurialTest into Test
and makes Test a child of unittest.TestCase.
Now that unittest mode is functionally equivalent to the default mode,
we switch the default execution mode to unittest and remove the choice
of different execution modes.
Unlike unittest's defaults, our result formatter does not print stack
traces. Here, we change TestResult.addFailure() to be compatible with
the existing/default execution mode.
rmtree() may fail under certain conditions. We ignore failures at the
individual test level because they can interfere with test execution.
Furthermore, we'll reattempt deletion in the high-level test runner
after all tests have finished.
Note that this patch is not a code refactor like most of the patches
before it. This change logically makes sense given the execution
behavior of the tests.
There is an execution mode on run-tests.py that stops after the first
failure. unittest mode was previously not obeying this option. This
patch fixes that.
testtmp is now a member variable of our test class. It's value is
computed during instance creation and the directory is managed via the
lifetime of the test object.
The unittest way of recording a skipped test is to raise an exception
(at least with modern unittest implementation). We change Test to raise
a SkipTest when operating in unittest mode.
This does prevent some "tear down" activities from running in unittest
mode. This will be fixed in subsequent patches. Since unittest mode
is experimental, this should be OK.
Simply wrapping TestCase.run() is not sufficient for robust results
reporting because unittest in Python 2.4 does not know about things
like skipped tests and reports them as success or failures instead of
skips.
We will reimplement TestCase.run() with knowledge and semantics present
in modern Python releases.
unittest does its own printing of progress indicators as part of
TestResult. So, there is no need to print them when running in unittest
mode.
This will fix the double output of progress indicators that had been
occurring in unittest mode.
We now have a custom unittest.TestSuite implementation that uses
_executetests() and thus knows how to execute tests concurrently.
Running tests in --unittest mode will use this TestSuite.
Since the TestSuite handles concurrency, the warnings around --jobs and
--loop have been removed.
Test instances (not paths) are passed into _executetests(). This means
the logic for instantiating Test instances has been moved outside of
_executetests(). This was done because an upcoming patch will reuse the
logic in _executetests().
As part of this change, test counts are no longer managed by
_executetests().
Tests executing in unittest mode behave a little differently. For
example, they report their results to a TestResult rather than just
printing. This patch paves the way for better integration of Test into
the unittest framework.
Previously, our unittest wrapper didn't report results properly. We now
properly report failures.
We had to rename the local variable to prevent "t" from being
overwritten in the local scope.
The unittest package in Python's standard library provides an almost
universal mechanism for defining and running tests. This patch starts
the process of making run-tests.py talk to it.
The main benefit of speaking unittest is that this will enable
Mercurial's tests to be more easily consumed by other testing tools,
like nose. This is useful for 3rd party extensions having their own
test infrastructure, for example.
Running tests in unittest mode will not result in completely sane
behavior until the unittest mode is made the default execution mode.
Expect things like double printing of output until support stabilizes.
One must choose between ``"y yes".split()`` and ``('y', 'yes')``. I choose the
later.
The feature still crash when you answer "yes" to use it. But at least, the
prompt itself works.
The `--interactive` flag workis by overwriting the original test file by its
`.err` version. So we need to write it before calling `self.fail`. Otherwise the
`.err` file does not exists and `--interactive` is ignored.
We can move that block code around because it is dedicated to write changed
output and we moves it in the try-except dedicated to handling changed output.
Note that the flog is still badly broken after this change. But at least it crash
instead of being ignored.
This patch starts the process of removing global variables from
run-tests.py. The goal of this is to make it easier to run tests
differently without having to concern yourself with global state.
As a safety precaution, we kill daemons in Test.cleanup(). This is
necessary for a subsequent patch that will reraise KeyboardInterrupt
before killdaemons() runs as part of run().
Reference output should be constant and doesn't need to be computed at
test execution time. We calculate it earlier.
This patch is the first in a mini series that will change how the
TestResult object works.
Test parsing is somewhat complicated. This patch extracts it into its
own function.
The impetus of this patch is folding tsttest() into the TTest class.
Subsequent patches will continue this work until tsttest() no longer
exists.
threadtmp is an implementation detail. We move the cleanup of this
directory to Test.cleanup() and make the variable internal. The
cleanup function will eventually disappear into unittest machinery.
Test.run() can now be executed multiple times on the same Test instance.
This feature is currently unused and there are no plans to implement it.
The main reason for this work was to refactor testtmp, replacements, and
env to be run-time specific as opposed to Test instance specific.
Some implementation details of test execution still live outside of
Test. These include determining what a result means and cleaning up
after the test.
To move to the world where more of this logic can live inside Test or a
derived object, the logic for test execution needs to be refactored.
Specifically, exception trapping and opportunities for result processing
need to be moved into Test.
This patch starts the process by establishing a TestResult class for
holding the results of a test execution. In order to actually use this
class, exception trapping and execution time recording needed to be
moved into Test.run().
testtmp is an implementation detail. It didn't need to be exposed to the
world.
threadtmp is derived from count. It is now created as part of the
constructor and mostly hidden from the outside world.
Environment variables are an implementation detail of how tests are
executed. This patch moves environment variable logic into Test and
completely hides it from the outside.
With this patch, a Test can be executed with two lines: init + run().
Tests are still single-use and take a more arguments to the constructor
than likely necessary. These will get addressed in subsequent patches.
createhgrc() is an implementation detail of how tests are run. It makes
sense to move it into Test.run().
Note that this will cause the test execution time to include the
creation of hgrc. The author does not believe this is a significant
change worth worrying about.
This patch starts the process of moving test-specific variables into the
Test class. The ultimate goal is to be able to instantiate a Test with
minimal arguments and to call run() on it without too much thinking.
This will make it much easier to run tests from other contexts. It will
also enable things like running a test multiple times.
Currently, the state for an individual test is scattered across a number
of functions and variables. This patch begins a process of isolating a
single test's state into instances of a class. It does this by
establishing a new Test base class and child classes for Python tests
and T tests. The class currently has a run() API that proxies into the
existing "runner" functions. Upcoming patches will move the logic for
each test type into the class.
Failing to start a server happens regularly, at least on windows buildbot.
Such a failure often has nothing to do with the test, but with the environment.
But half the test output can change because some data is missing. Therefore this
is worth an extended error message.
Detect the server failure in the diff output because it is most reliable
there. Checking the output only does not show if the server failure was
expected.
Old failure message when server start failed:
Failed test-serve.t: output changed
New message:
Failed test-serve.t: serve failed and output changed
Previously, test paths were assumed to be in the same directory and
wouldn't have a directory component. If a path with a directory
component was specified, it would be filtered out. This change allow
paths to contain directories. This in turn allows tests from other
directories to be executed.
Executing tests in other directories may break assumptions elsewhere in
the testing code. However, on initial glance, things appear to "just
work." This approach of running tests from other directories is
successfully being used at
https://hg.mozilla.org/hgcustom/version-control-tools/file/7085790ff3af/run-mercurial-tests.py
This patch moves the OptionParser population into its own function so
consumers may modify the OptionParser before arguments are evaluated.
This will allow consumers to add custom options, set different defaults,
etc.
Before, arguments were not passed into the optparse.OptionParser
instance and were coming from sys.argv. This patch enables consumers to
define the list of arguments to parse without having to adjust sys.argv.
Convenient when polishing patches and changing details of how they change test
output.
This will probably break in weird ways for revsets with special quoting ... but
it is good enough for run-tests.
Usage example:
yes | ./run-tests.py -li --changed qparent
