test-verify-repo-operations.py currently starts way too late and
extends the running time with -j50 on my machine from around 3:48 min
to 6:30 min. We could of course make it run earlier, but the test case
seems unlikely to find bugs not covered by other tests, so let's mark
it "slow" instead. I think this type of test is better suited to
running separately in a long-running job.
Doing this required the introduction of a mechanism for keeping
track of more general config in the test. At present this is only
used for extensions but it could be used more widely (e.g. to
control specific extension behaviour)
This greatly simplifies the extension management logic by introducing
a general notion of config, which we maintain ourselves and pass to
HG on every invocation.
This results in significantly less error prone test generation, and
also allows us to turn extensions off as well as on.
The logic that used an environment variable to rerun the tests with
an extension disabled now just edits the test file (in a fresh copy)
to remove these --config command line flags.
This gives a good way of letting Hypothesis run until it
finds an error, save that error, and be restarted without it
picking up on the old bug. This lets you run long-running
Hypothesis processes and then perform a manual deduplication
task on the bugs found at the end.
It's not an entirely satisfying way of using this, but anything
much better would require extensive changes to Hypothesis itself.
This adds support for testing extensions, including both tests
that extensions don't change behaviour and test for specific
commands.
We use the precondition system to determine what commands are
available to us. If we never use any commands enabled by an
extension then that extension is *skippable* and should not
have changed the behaviour of the test. We thus rerun the test
with an environment variable which is designed to turn off the
extension.
This expands the Hypothesis based stateful testing so that
rather than having a single repository under test, Hypothesis
manages a family of repositories. Some of these are freshly
created, some are clones of others.
The idea of this patch is to expand the use of Hypothesis
within Mercurial to use its concept of "stateful testing".
The result is a test which runs a sequence of operations
against a Mercurial repository. Each operation is given a
set of allowed ways it can fail. Any other non-zero exit
code is a test failure.
At the end, the whole sequence is then reverified by
generating a .t test and testing it again in pure
mode (this is also useful for catching non-determinism
bugs).
This has proven reasonably effective at finding bugs,
and has identified two problems in the shelve extension
already (issue5113 and issue5112).
