Commit Graph

560 Commits

Author SHA1 Message Date
Gregory Szorc
c2d6530853 run-tests: remove options from Test.__init__
All options are now passed as arguments and we no longer need options.

This enables us to instantiate Test from "plain old data" types. Since
options must be given as arguments, it also makes people think harder
about adding things that may not belong in Test. This will help ensure a
proper separation of responsibility going forward.
2014-04-22 12:20:10 -07:00
Gregory Szorc
5656f4586d run-tests: move shell to Test.__init__ 2014-04-22 12:17:23 -07:00
Gregory Szorc
285b810dae run-tests: move py3kwarnings to Test.__init__ 2014-04-22 12:15:55 -07:00
Gregory Szorc
7a6aa4b289 run-tests: move extra config options to Test.__init__ 2014-04-22 12:13:40 -07:00
Gregory Szorc
2bd392374e run-tests: refactor port number declaration
Instead of making port numbers derived from count and a global initial
port, we now pass a start port to Test.__init__ and do the calculation
at a higher level.
2014-04-22 12:10:25 -07:00
Gregory Szorc
e29eaf20cf run-tests: move timeout into Test.__init__ 2014-04-22 12:04:15 -07:00
Gregory Szorc
08548ac839 run-tests: move interactive to Test.__init__ 2014-04-22 12:01:31 -07:00
Gregory Szorc
7427586dc4 run-tests: move diff options into arguments of Test.__init__ 2014-04-22 11:58:32 -07:00
Gregory Szorc
d83a06e76b run-tests: move debug into an argument to Test.__init__ 2014-04-22 11:44:34 -07:00
Gregory Szorc
48b8b52024 run-tests: factor options.keep_tmpdir into an argument to Test.__init__ 2014-04-22 11:41:10 -07:00
Gregory Szorc
9edaec1d62 run-tests: remove redundant test name filter
TestSuite.run() doesn't need to filter tests by name because that
already happens elsewhere. Garbage in garbage out applies.
2014-04-22 12:29:33 -07:00
Gregory Szorc
d516d7b5e7 run-tests: move test filtering into TestSuite.run()
Upcoming patches will move the execution of tests to separate processes.
To facilitate this, it makes sense to move logic out of Test.
Furthermore, test filtering is logically the domain of the test runner,
not the test itself.

This patch interrupts our mini series of factoring options into named
arguments because filtering consults many options and it is easier to
move this logic out of Test sooner so we don't have to introduce
arguments at all.
2014-04-22 11:38:14 -07:00
Gregory Szorc
18e8487bdf run-tests: no longer pass a TestRunner into Test.__init__
We no longer access any attributes on TestRunner besides options, so we
stop passing a TestRunner to Test.__init__ and now pass the options
data structure instead.

Subsequent patches will move accessed options attributes into named
arguments.
2014-04-22 10:13:41 -07:00
Gregory Szorc
706551657a run-tests: pass abort into Test.__init__ 2014-04-22 10:12:19 -07:00
Gregory Szorc
71ca8805db run-tests: pass temp dir into Test.__init__
This patch starts a mini series of moving arguments to Test.__init__
from semi-complex data structures (such as the command options) to named
arguments. This will allow Test instances to be more easily instantiated
from other contexts. This improves the ability to run Mercurial tests in
new and different environments.
2014-04-22 10:05:32 -07:00
Gregory Szorc
b1c9f22ffa run-tests: pass a full test path into Test.__init__
Previously, a Test's path came from the base directory of all tests and
a filename leaf. There is not a strong reason why an absolute test path
can not be specified.

This change isn't strictly necessary. But it does enable scenarios such
as more easily running tests from multiple, non-sibling directories.
2014-04-22 10:01:22 -07:00
Gregory Szorc
50b1663f63 run-tests: factor refpath into Test classes
refpath is a side-effect of the test type. We therefore move it to the
Test class implementations.
2014-04-21 16:43:36 -07:00
Gregory Szorc
42d448ded4 run-tests: don't trap exceptions in Test.runTest()
Exceptions are all handled in Test.run(). This code was redundant since
the unittest test wrapper class was merged into Test.
2014-04-20 18:23:48 -07:00
Gregory Szorc
803865798c run-tests: factor options out of run()
We were only utilizing 2 parts of options. These have been converted to
named arguments.
2014-04-20 18:20:24 -07:00
Gregory Szorc
8e80b0bd22 run-tests: refactor temporary directories in Test
We no longer need cleanup() because setUp() and tearDown() cover it. We
move tempdir creation into setUp() and always delete in tearDown()
2014-04-20 18:15:38 -07:00
Gregory Szorc
0024c28a10 run-tests: move _executetests into TestSuite 2014-04-20 17:42:31 -07:00
Gregory Szorc
fd2322f58b run-tests: capture execution times in TestResult
TestResult has facilities for recording when tests start and stop. It
makes sense to move execution time recording into TestResult.

In addition, output generation is being moved into TestResult, a class
that has a concept of an output buffer (and a lock).
2014-04-20 17:39:26 -07:00
Gregory Szorc
b583c9eacb run-tests: move outputtimes() into unittest runner class
The unittest runner now does all the printing. Move outputtimes() there.
2014-04-20 17:28:24 -07:00
Gregory Szorc
d9c2ef0a49 run-tests: refactor how times are recorded
We now record times in Test.run(), which is a more appropriate place for
it.
2014-04-20 17:21:24 -07:00
Gregory Szorc
3c3efa484d run-tests: eliminate Test._result
The result of test execution is now reported on a TestResult. We no
longer need to store a tuple of the result.
2014-04-20 17:13:10 -07:00
Gregory Szorc
0f26e878dd run-tests: replace Test.ignore() with raise IgnoreTest 2014-04-20 17:10:49 -07:00
Gregory Szorc
352b2437a3 run-tests: replace Test.skip() with raise SkipTest
All the result functions now raise an Exception. We no longer need these
functions and we can inline the raise.
2014-04-20 17:09:52 -07:00
Gregory Szorc
2d619ddf2b run-tests: remove some legacy code around result handling
unittest handles all results now. We no longer need to manually keep
track of results.
2014-04-20 17:06:57 -07:00
Gregory Szorc
e9f9523063 run-tests: merge MercurialTest into Test
Now that we execute all tests via unittest, the MercurialTest wrapper is
not necessary. This patches moves the logic from MercurialTest into Test
and makes Test a child of unittest.TestCase.
2014-04-20 17:04:42 -07:00
Gregory Szorc
5b136e830c run-tests: execute tests via unittest
Now that unittest mode is functionally equivalent to the default mode,
we switch the default execution mode to unittest and remove the choice
of different execution modes.
2014-04-20 16:44:45 -07:00
Gregory Szorc
3a52928608 run-tests: implement Test.__str__
unittest sometimes does a str(test) and the default implementation
doesn't do the right thing. So we override it.
2014-04-22 13:37:27 -07:00
Gregory Szorc
1da160e9f3 run-tests: make failure reporting in unittest mode equivalent to default
Unlike unittest's defaults, our result formatter does not print stack
traces. Here, we change TestResult.addFailure() to be compatible with
the existing/default execution mode.
2014-04-21 12:15:55 -07:00
Gregory Szorc
9b379f8c82 run-tests: ignore failures from rmtree
rmtree() may fail under certain conditions. We ignore failures at the
individual test level because they can interfere with test execution.
Furthermore, we'll reattempt deletion in the high-level test runner
after all tests have finished.

Note that this patch is not a code refactor like most of the patches
before it. This change logically makes sense given the execution
behavior of the tests.
2014-04-20 16:54:51 -07:00
Gregory Szorc
1275a4e261 run-tests: abort tests after first failure in unittest mode
There is an execution mode on run-tests.py that stops after the first
failure. unittest mode was previously not obeying this option. This
patch fixes that.
2014-04-20 16:53:49 -07:00
Gregory Szorc
e6a894b09e run-tests: print compatible output from TextTestRunner
Our unittest TextTestRunner now prints output similar to the output
from the default execution mode - the output people expect from
run-tests.py.
2014-04-20 16:39:05 -07:00
Gregory Szorc
6eb1131c5c run-tests: don't print results in unittest mode
A subsequent patch will have unittest print tests for us. Don't perform
old-style printing when in unittest mode.
2014-04-20 16:17:18 -07:00
Gregory Szorc
15b613915a run-tests: move errpath deletion to setUp() 2014-04-20 15:53:25 -07:00
Gregory Szorc
1cba8e1ca6 run-tests: kill daemons during tearDown() 2014-04-20 15:46:34 -07:00
Gregory Szorc
960b628f9a run-tests: move err file writing to tearDown() 2014-04-21 09:30:11 -07:00
Gregory Szorc
8cee2c5a3b run-tests: refactor testtmp
testtmp is now a member variable of our test class. It's value is
computed during instance creation and the directory is managed via the
lifetime of the test object.
2014-04-20 15:40:22 -07:00
Gregory Szorc
5f83d4f3ae run-tests: store skipped state on Test
As code is being moved between functions, we need a member variable to
hold skipped state.
2014-04-20 15:19:21 -07:00
Gregory Szorc
57a48ec516 run-tests: move some functionality to Test.tearDown() 2014-04-20 15:16:29 -07:00
Gregory Szorc
6571ca3b17 run-tests: store last result in Test._result
Subsequent patches move post-test actions into tearDown(). This requires
tearDown() to have access to the last result.
2014-04-20 15:08:05 -07:00
Gregory Szorc
9692484ea6 run-tests: report test duration in Test.tearDown() 2014-04-20 15:00:13 -07:00
Gregory Szorc
33fe327c22 run-tests: store test return code and output in Test instance
A subsequent patch that will introduce tearDown() requires test
execution state be stored in the instance.
2014-04-20 14:58:36 -07:00
Gregory Szorc
879667e23b run-tests: keep track of test start and stop in MercurialTest.run()
This brings run() more compatible with unittest.TestCase.run()
2014-04-20 14:55:33 -07:00
Gregory Szorc
3e73bf2558 run-tests: keep track of test execution state in Test
This patch starts a mini series of moving functionality into the
newly-established setUp() and tearDown() methods.
2014-04-20 14:52:57 -07:00
Gregory Szorc
ecba404b77 run-tests: support setUp() and tearDown() in TestCase wrapper
unittest.TestCase.run() calls setUp() and tearDown() during run(). We
emulate that implementation.
2014-04-20 14:41:11 -07:00
Gregory Szorc
ed53d7b15f run-tests: fail tests by raising an exception
When in unittest mode, Test.run() will now raise for all non-success
cases. This makes it behave like TestCase.run().
2014-04-20 14:34:03 -07:00
Gregory Szorc
213e98d278 run-tests: record warnings by raising WarnTest
We continue the conversion of recording test results by raising
exceptions.
2014-04-20 14:32:03 -07:00