Commit Graph

383 Commits

Author SHA1 Message Date
Simon Heimberg
b2a439bb78 run-tests: unnecessary globs also count as warnings
When a glob is unnecessary, now there's a diff output and 'run-tests.py -i'
works for accepting the output.
On windows, some tests which have "passed" currently (with some info lines
printed) will now be reported as "warned". (I recommend to recognize "warned"
by buildbot before applying this patch.)
2014-02-13 08:29:44 +01:00
Simon Heimberg
1e51f26c6e run-tests: report a test as warned when only missing globs
On windows, some test reported as failed will now be reported as "warned".
2014-02-13 08:27:53 +01:00
Simon Heimberg
434d0f2c98 run-tests: add possibility for test-runners to report a "warned" test result
A test result is recognized as "warned" when the test runner returns the exit
code False. (False is similar to 0, which is reporting a command has run
sucessfully.)

The only difference in display is that the failure message while running writes
"Warning:" instead of "ERROR:". The diff output is the same as when the test
fails. Runing "run-tests.py -i" asks to accept the changed result also for
tests reported as "warned".

When running tests, a "warned" test would look like this:
..
 --- xxxx\tests\test-something.t
 +++ xxxx\tests\test-something.t.err
@@ -1279,7 +1279,7 @@
   $ echo anything
   $ hg commit -S -m whatever
   committing subrepository s
-  committing subrepository s/sbs
+  committing subrepository s/sbs (glob)
   warning: something happened
   committing subrepository t
   $ echo something

Warning: xxxx\tests\test-sOMETHING.t output changed
~.s...s...s..


Reporting a test result as "warned" will be used in following patches.
2014-02-13 08:26:13 +01:00
Simon Heimberg
11e4ef2216 run-tests: separate exit code when no failed tests but warned ones
This can be used by buildbot to set the result "warnings" instead of "failed".
2014-02-13 08:17:39 +01:00
Simon Heimberg
8864ac070e run-tests: introduce 'warned' as test result
While running, a test resulting in 'warned' is shown as '~'.
Test results with state warned are listed between the skipped and the failed
tests. Example:
Skipped test-revert-flags.t: missing feature: executable bit
Skipped test-inotify-lookup.t: missing feature: inotify extension support
Warned test-something.t: output changed
Failed test-largefiles.t: output changed
Failed test-subrepo.t: output changed
# Ran 11 tests, 2 skipped, 1 warned, 2 failed.


The test result "warned" will be used in later patches.
2014-02-13 08:14:54 +01:00
Simon Heimberg
9463ba6189 run-tests: handle sys.exit in main function 2014-02-13 08:02:06 +01:00
Simon Heimberg
073dd453ab run-tests: remove unnecessary variable initialisation
ret is always set before it is used, so remove ret = 0
2014-02-13 07:29:28 +01:00
Simon Heimberg
9d8139803a run-tests: fix typo and print out failing line
Instead of stripping the newline and printing the line, only the newline was
printed.
The output on buildbot will make more sense now.
2014-02-11 01:15:07 +01:00
Simon Heimberg
14f515afb4 run-tests: print more information on unnecessary glob matching
Extend the message with the test name and the approximate line number. (The
line number is the one of the command producing the output.)
Finding the line to fix is easier now.

old message:
......
Info, unnecessary glob: at a/b/c (glob)
..

new message:
......
Info, unnecessary glob in test-example.t (after line 9): at a/b/c (glob)
..


The test result is still pass as before.
2014-01-16 12:08:57 +01:00
Simon Heimberg
2967b15687 run-tests: suggest to append glob when only path sep does not match
When the line does not match because of \ instead of / (on windows), append
(glob) in the expected output.
This allows to rename test-bla.t.err to test-bla.t for getting a correct
output. This worked for other failures like missing (esc), but not here.

  Output example (only +- lines of diff):
Before:
-  path/with/local/sep
+  path\\with\\local/sep
Now:
-  path/with/local/sep
+  path/with/local/sep (glob)
2014-01-16 12:08:29 +01:00
Simon Heimberg
ee6f6f78e3 run-tests: test each line matching function on its own
This has several advantages.
 * Each match function can return some information to the caller runone (used in
   the next patch).
 * It is not checked that the line ends in " (glob)" when rematch() returns
   false.
 * And it looks more readable.
2014-01-16 12:06:49 +01:00
Simon Heimberg
4367a66877 run-tests: set the thread name to the test name for info on error
This does not happen when running normal. But when fiddling around with
the test infrastructure, this helps a lot.

  Old traceback messge
Exception in thread Thread-7:
Traceback (most recent call last):
...

  new traceback message
Exception in thread test-something.t:
Traceback (most recent call last):
...
2014-01-16 11:26:54 +01:00
Simon Heimberg
eff53c5d86 run-tests: report tests as failed when run-test raises an error
Before no message was returned to the main thread. No result was registered
and no new thread was started.
This does not happen when running normal. But when fiddling around with
the test infrastructure, this helps a lot.
2014-01-16 18:55:35 +01:00
Simon Heimberg
d4d6543b3a run-tests: better check for python version
Compare version by using pythons tuple comparison. So we do not match on python
3.0 or newer.
2013-11-25 22:00:46 +01:00
Mads Kiilerich
ca46558bdd tests: python hash seed is only relevant after failures - otherwise keep quiet 2013-11-17 10:59:35 -05:00
Sean Farley
2ced3aebc5 run-tests: remove files that we potentially create outside of $TMPDIR 2013-11-15 22:57:11 -05:00
Sean Farley
1c3131bb1b run-tests: remove code that creates a dummy 'diffstat'
This was introduced in 23816898bed5 for an old test that is no longer the same.
2013-11-15 22:56:08 -05:00
Mads Kiilerich
8c414ba152 run-tests: place the .t shell script next to $TESTTMP and with a useful name
--keep can thus keep it around - very convenient for debugging.
2013-10-03 14:50:47 +02:00
David Soria Parra
72600cc4d7 shelve: add a shelve extension to save/restore working changes
This extension saves shelved changes using a temporary draft commit,
and bundles the temporary commit and its draft ancestors, then
strips them.

This strategy makes it possible to use Mercurial's bundle and merge
machinery to resolve conflicts if necessary when unshelving, even
when the destination commit or its ancestors have been amended,
squashed, or evolved. (Once a change has been unshelved, its
associated unbundled commits are either rolled back or stripped.)

Storing the shelved change as a bundle also avoids the difficulty
that hidden commits would cause, of making it impossible to amend
the parent if it is a draft commits (a common scenario).

Although this extension shares its name and some functionality with
the third party hgshelve extension, it has little else in common.
Notably, the hgshelve extension shelves changes as unified diffs,
which makes conflict resolution a matter of finding .rej files and
conflict markers, and cleaning up the mess by hand.

We do not yet allow hunk-level choosing of changes to record.
Compared to the hgshelve extension, this is a small regression in
usability, but we hope to integrate that at a later point, once the
record machinery becomes more reusable and robust.
2013-08-29 09:22:13 -07:00
Augie Fackler
3f08f4aa32 run-tests: find mercurial path with syntax valid on both py2 and py3 2013-09-19 15:29:28 -04:00
Augie Fackler
a4c5cf3750 run-tests: if run by python3, execute setup.py with --c2to3 2013-09-19 15:29:04 -04:00
Simon Heimberg
96b55afc5a run-tests: only check the common criteria once per test 2013-07-26 21:18:40 -04:00
Matt Mackall
d64dd7b443 run-tests: backout a71b92f5b8d5 line endings change
It made the windows buildbot sad.
2013-07-18 23:22:59 -05:00
Brendan Cully
b22788d1aa run-tests: revert previous commit, run() waits after a timeout 2013-07-18 09:42:44 -07:00
Brendan Cully
ec9f9a7a0f run-tests: reap timed-out zombies 2013-07-18 09:39:01 -07:00
Simon Heimberg
690fb7f82d run-tests: generally handle line endings on windows by re
Recently this regexp was only appended when running a python test. When running
a tsttest there was a separate handling for each line type. Simplify and unify
this.
2013-07-13 23:58:01 +02:00
Simon Heimberg
2fc2f7ad2e run-tests: test for os.altsep instead of os.name when checking \ for /
This tells more explicitly what it is about
2013-07-13 23:57:55 +02:00
Brendan Cully
4e99027450 run-tests: replace popen locking with a noop _cleanup handler on py24
This also avoids the race condition, and isn't detrimental to job scheduling.
2013-07-17 13:42:08 -07:00
Brendan Cully
f3860d2150 run-tests: only lock Popen wait/poll on python 2.4
It can cause scheduling bubbles and is not necessary on newer pythons.
2013-07-17 13:02:12 -07:00
Brendan Cully
90d44a096b run-tests: lock popen wait/poll
In python2.4, any call to Popen() may attempt to wait on any active
process, and wait is not thread-safe. Make it thread-safe.
See http://bugs.python.org/issue1731717 for details.
2013-07-17 12:45:12 -07:00
Matt Mackall
14f333f579 run-tests: patch over py2.4 proc.wait() race 2013-07-16 12:44:11 -05:00
Simon Heimberg
8a32578a00 cleanup: remove unused imports
detected by pyflakes
2013-06-13 01:36:58 +02:00
simon@laptop-tosh
e5622bbc0e run-tests: sort missing files first instead of raising an error
sort first for early telling typos in test names
2013-06-11 23:39:48 +02:00
simon@laptop-tosh
975f5dea4c run-tests: call Threads constructor with keyword arguments
This is suggested in the python documentation
(on http://docs.python.org/2/library/threading.html#thread-objects)
2013-06-09 16:49:08 +02:00
simon@laptop-tosh
400b5ef2b9 run-tests: ignoring tests works again
running with --retest did not work anymore because runone still returned None
for this case. But this is not allowed since a204317f039b.
2013-06-09 16:42:55 +02:00
Matt Mackall
2a02cc28af run-tests: simplify interrupted message 2013-06-07 16:04:11 -05:00
Matt Mackall
df0aab998a run-tests: remove spurious disable of --time with --debug 2013-06-07 15:58:14 -05:00
Matt Mackall
5a359facfe run-tests: always gather runtimes 2013-06-07 15:56:16 -05:00
Simon Heimberg
cbe95e49e5 run-tests: when interrupted report time in failure message
When --time is specified, the interruption message of an interrupted test is
extended with the time the test has run

INTERRUPTED: /path/to/tests/test-example.t (after 513 seconds)
2013-06-05 22:05:29 +02:00
Simon Heimberg
3d444f656b run-tests: report interrupted tests
When the test run is aborted, a message is printed for each interrupted test.
This is helpful when a test is hanging.

example failure message:
INTERRUPTED: /path/to/tests/test-example.t

The message can appear before or after the line with the number of tests
2013-06-05 22:05:02 +02:00
Simon Heimberg
43548443c6 run-tests: open hgrc file only for writing
The file is not read here. Opening with "w+" is unnecessary.
2013-06-05 22:42:43 +02:00
Matt Mackall
bf221e220d run-tests: make --noskips work 2013-06-06 14:40:26 -05:00
Matt Mackall
4ee271c45d run-tests: add --loop support
This makes it easy to run tests repeatedly at high loads to look for test failures.
2013-06-02 18:09:06 -05:00
Matt Mackall
177f6f2cb0 run-tests: --interactive and --jobs no longer conflict 2013-06-02 17:58:51 -05:00
Matt Mackall
c5fd75f600 run-tests: sort certain slow tests earlier by keyword
There are a few tests that evade the size heuristic
2013-06-02 17:58:49 -05:00
Matt Mackall
8481b7ff76 run-tests: drop unused resultslock 2013-06-02 17:57:37 -05:00
Matt Mackall
438cc52b80 run-tests: drop options.child and users 2013-06-02 17:57:36 -05:00
Matt Mackall
66a2eb9cb9 run-tests: remove runchildren, now unused 2013-06-02 17:56:53 -05:00
Matt Mackall
9f730ea353 run-tests: schedule largest tests first 2013-06-02 17:56:51 -05:00
Matt Mackall
3a93b9c7d3 run-tests: introduce thread scheduler 2013-06-02 16:58:22 -05:00