Summary:
The tests for run-tests expect there to be no linewrapping control characters
output when there are no progress lines to print. Make sure they only get
printed if there are lines to print.
Reviewed By: DurhamG
Differential Revision: D14851271
fbshipit-source-id: 4004f848e2e205d36bd0dd107ced2fe9d69e31d8
Summary:
The progress indicator for run-tests.py disables terminal line wrapping at the
start of run-tests, and re-enables it at the end. This causes two problems:
* Diff output is not wrapped, so only the start of changed lines are visible.
* Sometimes run-tests.py exits without restoring line wrapping. In this case
the shell session is adversely affected.
Disabled line wrapping is only necessary when outputting the progress, so
restrict it to there.
Reviewed By: quark-zju
Differential Revision: D14834719
fbshipit-source-id: 2c0380e7e1a1d5be195b025c252c80208e83a545
Summary:
Add a `--update-output` flag to answer "yes" for all `-i` prompts.
Also add an "all" option for the `-i` prompt.
This is useful to apply changes to a large range of tests.
Reviewed By: DurhamG
Differential Revision: D14233452
fbshipit-source-id: ebfc5c9a897a7fc7fea779aefef538e420e1e197
Summary:
previously, unexpected output would cause the harness to
stack trace and not show the actual test failure.
This catches the error and treats it as a failed run, which causes
the test output to be displayed.
No more `ValueError: invalid literal for int() with base 10: '0\xe2\x90\x8a'`
Reviewed By: DurhamG
Differential Revision: D13995000
fbshipit-source-id: 5c1a9e8d6f8e81a924cdbf760fa75cdc0d33b74d
Summary:
D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`.
That has nasty problems if both `import mercurial` and
`import edenscm.mercurial` are used, because Python would think `mercurial.foo`
and `edenscm.mercurial.foo` are different modules so code like
`try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)`
would fail to handle the `edenscm.mercurial` version. There are also some
module-level states (ex. `extensions._extensions`) that would cause trouble if
they have multiple versions in a single process.
Change imports to use the `edenscm` so ideally the `mercurial` is no longer
imported at all. Add checks in extensions.py to catch unexpected extensions
importing modules from the old (wrong) locations when running tests.
Reviewed By: phillco
Differential Revision: D13868981
fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
Summary:
Tests don't close the child process stdout. On newer versions of Python, this
can lead to ResourceWarnings when the test runner thread terminates.
Reviewed By: HarveyHunt
Differential Revision: D13517425
fbshipit-source-id: 6cedf4f39efe1299c41dbde784daf8c159309640
Summary:
Some of the escape sequences in run-tests.py are invalid. These cause
DeprecationWarnings on newer versions of Python.
In both cases, there are `\` characters that need to be escaped as `\\`.
Reviewed By: HarveyHunt
Differential Revision: D13517137
fbshipit-source-id: a899c3c28d55210f5972a515474a2fa69d051671
Summary: Since we are using `hg.rust` with everything, we should also test it.
Differential Revision: D13121588
fbshipit-source-id: 3a4e07c1d70c279f008812d026bf9a7c74aa3b2e
Summary: Now that `chg` is embedded into `hg.rust`, we need to explicitly disable it while tests run.
Reviewed By: quark-zju
Differential Revision: D13021560
fbshipit-source-id: 843045f3bc9d9866e421f0bf0aa132849e5b338f
Summary:
The Rust implemenation has a smaller chance to have deadlock issues when
interruptted.
Reviewed By: markbt
Differential Revision: D10843400
fbshipit-source-id: af5614b9992577bde74cf54c5760d15d050aafbf
Summary: Reformat all opted-in python code with version `18.9b0` of Black.
Reviewed By: ambv
Differential Revision: D10126605
fbshipit-source-id: 82af0d645dd411ce8ae6b8d239e151b3730cd789
Summary:
Write a test that hangs forever, like:
$ hg init
$ hg serve -p 0
`run-tests.py --timeout 3` cannot handle it (on both Windows and Linux). It
only tries to kill the bash process, and the `hg serve` process stays running.
For some reason, existing tests are more likely hang on Windows. Therefore we
need a way to reliably kill processes and their descendants. Windows provides
Job Object [1] which reliably solves the problem.
On POSIX, process group is used. It's less reliable because processes can
escape from the group by using setpgid, etc. But it's already better than
before.
[1]: https://docs.microsoft.com/en-us/windows/desktop/procthread/job-objects#managing-processes-in-jobs
Reviewed By: phillco
Differential Revision: D9516042
fbshipit-source-id: 970fce7ba24d0e9bafd66438354de2e6df761edf
Summary:
Similar to buck. Show progress for running tests.
This also improves Ctrl+C handling. It's now more
responsive since we no longer wait for the entire
output.
Reviewed By: phillco
Differential Revision: D9516040
fbshipit-source-id: 98b29228177510855e29da341198697da940884c
Summary:
`.t` tests are stateful. This means if some early parts do not execute
as the test expect, the the remaining parts can break in funny ways,
and generate lots of uninteresting output.
Limit the length of the diff output so the "likely" boring parts won't show
up by default. The change can be overridden by `--maxdifflines`.
Reviewed By: phillco
Differential Revision: D9516041
fbshipit-source-id: 6c998cdc73ae4a45e2f4167c22c04df51392da29
Summary:
MSYS provides binaries that tests need so we no longer
depend on system binaries, which might be broken.
MSYS can provide `make.exe` to make `make local` working,
so we don't depend on other things to provide `make`.
As we're here, change `run-tests.py` to use the right Python
interpreter and escape Windows paths.
Reviewed By: phillco
Differential Revision: D9505926
fbshipit-source-id: f17e77dfc599069d4fe849057b7a198758c2dd42
Summary:
Switch `ui.load` and `ui.readconfig` to use the Rust config parser.
`ui` now no longer depends on `config.config` or `rcutil`.
Pest's error messages are fancier, thus most test changes.
For the fbsource repo, debugshell shows the new code is 10+x faster:
On laptop:
Before:
In [1]: %timeit m.ui.ui.load()
10 loops, best of 3: 27.8 ms per loop
After:
In [1]: %timeit m.ui.ui.load()
100 loops, best of 3: 1.85 ms per loop
On devserver:
Before:
In [1]: %timeit m.ui.ui.load()
100 loops, best of 3: 16.8 ms per loop
After:
In [1]: %timeit m.ui.ui.load()
1000 loops, best of 3: 1.28 ms per loop
Since `ui._rcfg` is no longer copy-on-write, there is concern about `ui.copy()`
performance. It is faster too (on devserver):
Before:
In [1]: %timeit ui.copy()
1000 loops, best of 3: 198 µs per loop
After:
In [1]: %timeit ui.copy()
10000 loops, best of 3: 157 µs per loop
The old `ui.py` was copied to `legacyui.py` and can replace the new `ui.py` if
a config file exists on the system. This provides a way to switch back to the
old config parser in case of emergency.
Reviewed By: mitrandir77
Differential Revision: D8887375
fbshipit-source-id: 2951ca622c77bf41187ad5c5cab3445cda0dc519
Summary:
Generally, it seems the `mercurial/entrypoint.py` approach might need some
more thinking, but in this particular case I'd prefer to just fix the tests.
Here's what's going wrong:
- D9407009 intoduced a logic to expect `mercurial/entrypoint.py` to always
exist under the `hg` installation on a *run-tests* level
- `run-tests.py test-run-tests.t` however runs `run-tests.py` (yo dawg), while
`hg` is a symlink to `$(which true)`
- while the `mercurial/entrypoint.py` script cannot be found by the test
suite in this case, it's not important becuase no real Mercurial is run.
Therefore, I propose to change `SystemExit` to `log`. On one hand, the absense
of `mercurial/entrypoint.py` is a big deal, so we should know about it. On the
other, this case can run completely without it. In the dominant majority of the
other test cases, we'll learn about the absense of `mercurial/entrypoint.py`
by means of the failing Mercurial call.
Generally, this migth require some more thought, but for now let's fix tests.
Reviewed By: lukaspiatkowski
Differential Revision: D9421433
fbshipit-source-id: e39628015ff96f27368b99ec1f99145c0cb1ae7b
Summary:
The run-tests script symlinks hg to a place and sets `PATH` so `hg` can be
called from anywhere without requiring the local shell settings, including
"dummy-ssh" server side.
The Rust `hgmain` uses
`env::current_exe().parent().join("mercurial/entrypoint.py")` to locate
the `entrypoint.py` file. However, `env::current_exe()` does not follow
symlink on OSX, unlike Linux. Therefore the entrypoint.py cannot be located on
OSX.
`hgmain` also reads `HGPYENTRYPOINT`. So let's set it explicitly from
run-tests.py.
Reviewed By: singhsrb
Differential Revision: D9407009
fbshipit-source-id: db9a55e11b424625a6ab58afdbebf0944c7012f5
Summary:
Setting `lfs.usercache` means every LFS object is available globally
for the test. Some tests would like to test pushing LFS objects from
one repo to another. This config hides real issues.
Reviewed By: DurhamG, zhh95
Differential Revision: D8986784
fbshipit-source-id: 3f8d925b28acfe1c6b8ebcb02bd3815642747629
Summary:
Previously, the watchman forground server could inherit other fds (ex. the one
that waits for "watchman --version --no-spawn" output) and cause the read of
"watchman --version" to take forever. This patch closes related fds to solve
the issue.
Reviewed By: wez
Differential Revision: D8846430
fbshipit-source-id: d65f3377ac3b2277d1be7149a4ceb590f9a184a2
Summary:
When running all tests with watchman, I noticed there are other tests that fail
with the sock path length limit. So let's just pick a shorter path in
run-tests.py.
Remove `#require no-fsmonitor` that is no longer necessary.
Reviewed By: phillco
Differential Revision: D8844485
fbshipit-source-id: 2d3c40bf33868c65517adf98d6c1e8b4cb46697a
Summary: This prefix makes this file harder because it isn't valid JSON. Per discussion with quark-zju, remove it.
Reviewed By: singhsrb
Differential Revision: D8769139
fbshipit-source-id: f45703c963898202775e90d536dde2894e5029bb
Summary:
Add a config option `fsmonitor.detectrace` and environment variable
`HGDETECTRACE`. Once set, perform an additional query to check if there are
writes to the working copy while `wctx._buildstatus()` is being calculated.
Also enable the race detector for all fsmonitor tests.
Reviewed By: wez
Differential Revision: D8597957
fbshipit-source-id: 5d83d529641325dc8d5e72ad059eed8db27d1f2a
Summary: It turned out that a waiting loop is necessary, at least on my laptop.
Reviewed By: DurhamG
Differential Revision: D8560023
fbshipit-source-id: 8efd8dca4c630a30e5ccb140eb4f2da8a0cc5717
Summary:
While writing the test, I noticed the test didn't error out as expected:
```
% ./run-tests.py test-treestate-downgrade.t -i --watchman
E
Errored test-treestate-downgrade.t: Traceback (most recent call last):
File "./run-tests.py", line 1064, in run
self.setUp()
File "./run-tests.py", line 1053, in setUp
raise RuntimeError("timed out waiting for watchman")
RuntimeError: timed out waiting for watchman
(exit code 0)
```
This means buck test which relies on exit code will treat those tests as
"successful". That's incorrect. So let's change the exit code.
Reviewed By: DurhamG
Differential Revision: D8560021
fbshipit-source-id: 4f98772099c559e59288e42165386f1069086377
Summary:
After spawning the watchman process, there is a small window before the
watchman process listening on the specified socket file. If hg tests run
during that window, they might fail with watchman unavailable problems.
Therefore, change run-tests.py to wait for watchman by testing
`watchman version`.
Reviewed By: DurhamG
Differential Revision: D8523989
fbshipit-source-id: c5cb2366f938c0617fdd14d232f879b7cdc46b30
Summary:
Similar to chg. Add flags to run tests with watchman. This is mostly moving
features from `fsmonitor-run-tests.py`. The blacklist is converted to
`#require no-fsmonitor`.
Reviewed By: phillco
Differential Revision: D8434518
fbshipit-source-id: a8512cd71c1171e9037f36dbef195f1e6210f27e
Summary:
The current buck test outputs failures like:
--- /xxx/xxxxx/xxxxx-xx-xxxxxx-xxxxxxx-529-1528508531/xxxxxx/xxx/hg/tests/test-fb-hgext-sparse-extensions.t
+++ /xxx/xxxxx/xxxxx-xx-xxxxxx-xxxxxxx-529-1528508531/xxxxxx/xxx/hg/tests/test-fb-hgext-sparse-extensions.t.err
....
The file name is too long. It's nicer to just get to the point and show the
basename:
--- test-fb-hgext-sparse-extensions.t
+++ test-fb-hgext-sparse-extensions.t.err
So this patch does that.
Reviewed By: phillco
Differential Revision: D8360547
fbshipit-source-id: 103929747944e819441a428d30dd6c6516439a71
Summary: Mostly empty lines removed and added. A few bugfixes on excessive line splitting.
Reviewed By: quark-zju
Differential Revision: D8199128
fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
Summary:
Our tests use features that are provided by the bash shell to function
properly. I ran into issues with our tests on Ubuntu because the default shell
was `dash` which did not even support the `source` command. Therefore, this
commit changes the default hg shell for the tests to `bash`. Clients can always
override the default hg shell for the tests to another shell.
Reviewed By: quark-zju
Differential Revision: D7563733
fbshipit-source-id: e9c16c002793a919a49292a9aa0a876ba232c293
Summary:
- Support 8 color mode, since some terminals (emacs) do not support 16
colors (i.e. "bright*" colors have no effects).
- Detect emacs and use 8 color mode automatically.
- Read terminfo colors. Respect it if it says the terminal supports more
colors. Suggested by wez.
- Read HGCOLORS. Respect it unconditionally. Change run-tests.py to set it.
- Change diff related colors to also fallback to 8 colors since "bright"
colors are not guaranteed available.
Differential Revision: D7432965
fbshipit-source-id: da75829f856b4de737b946af72d24ff5351026cb
Summary:
Allow color config to be something like `color132:red` that is to use 256
color if supported, but fallback to `red`.
To get a consistent test output. `run-tests.py` is changed to disable 256
color support explicitly.
Reviewed By: singhsrb
Differential Revision: D7388277
fbshipit-source-id: da74ae8fc70c971901d56a6985976db44cbec0d9
Summary:
`.t` tests have some highly repeatitive logics, like creating a repo, etc.
This patch adds a common shell file that defines frequently used functions.
For now, `newrepo` and `enable` are added. The latter can be used to
enable a feature (ex. obsstore), or an extension.
`test-fb-hgext-fbamend-next.t` and `test-fb-hgext-absorb.t` are migrated
to use the new shell functions.
Reviewed By: phillco
Differential Revision: D7121485
fbshipit-source-id: 167fcc20e4e30864199b6c5af0958b80bfb68817
Summary:
Update run-tests.py to cache the results of hghave checks. Some of the
linematch code can end up calling _hghave() hundreds of times in a row with the
same check.
This is particularly likely to happen on test failures, where a line with an
hghave check does not match the expected output. In this case the same hghave
check will be re-run on each subsequent line of test output.
Reviewed By: quark-zju, farnz
Differential Revision: D7077677
fbshipit-source-id: 768a11df78924b75cde4c8b48c801b79194f7969
Summary:
- Add support for RocksDB engine (developed as a drop in replacement for innodb) to hgsql to allow new xdb.hgsql.1-10 shards to host hg repos
- Prefer MySQL test DBs in same region
- Run all hgsql unit tests also for RocksDB engine
- Allow for nested ifs to make that possible (downside if you switch off rockdb tests, innodb tests are run twice)
Reviewed By: quark-zju
Differential Revision: D7014064
fbshipit-source-id: 073c36176aa7eaf74252ef33c3f47da594920b28
Summary:
Previously `hg server` uses `HGPORT` that might be in use. This patch uses
`-p 0 --port-file ...` so `hg server` always gets assigned a free port.
The change was first made by the following Ruby script:
```
re = /^ \$ hg serve(.*) -p \$(HGPORT[12]?) (.*[^\\])$\n \$/
Dir['*.t'].each do |path|
old = File.read(path)
new = old.lines.map do |l|
next l if l[/\(glob\)/] or not l['$HGPORT'] or l[/^ [$>]/]
"#{l.chomp} (glob)\n"
end.join.gsub re, <<-'EOS'.chomp
$ hg serve\1 -p 0 --port-file $TESTTMP/.port \3
$ \2=`cat $TESTTMP/.port`
$
EOS
File.write(path, new) if old != new
end
```
Then there are some manual changes:
run-tests.py: It now treats `$HGPORT` in output as glob pattern `*`, since
it does not know the assigned value in tests.
test-bookmarks-pushpull.t, test-https.t: Some `hg pull`s were changed to use
explicit paths instead of relying on `.hgrc` since the test restarts the
server and `.hg/hgrc` having an outdated URL.
test-schemes.t: The test writes `$HGPORT` to `.hgrc` before assigning it.
Changed the order so the correct `$HGPORT` is written.
test-patchbomb-tls.t: Changed `(?) (glob)` to `(glob) (?)`.
Reviewed By: DurhamG
Differential Revision: D6925398
fbshipit-source-id: d5c10476f43ce23f9e99618807580cf8ba92595c
Summary:
This can be used by a wrapper script running a single test
to detect whether the test is skipped or not.
Reviewed By: DurhamG, phillco
Differential Revision: D6879864
fbshipit-source-id: 8bc535fccdbc6ad32eec870bd9e503b6f84f694b
Summary:
run-tests.py creates the `python` symlink. Let's do the same for `hg`.
This means tests can just use `hg` and do not worry about it being a
wrong one (like, the system hg). And there is no need to create fragile
shell alias, or set `ui.remotecmd`.
The check to ensure `--with-hg` provides either `hg` or `hg.exe` was
removed since it's no longer necessary.
Reviewed By: DurhamG
Differential Revision: D6879879
fbshipit-source-id: 59255102ea3c4c646afc7900696403242bcfb4db
Summary:
With the upcoming `buck test` change, the default timeout is
not enough because of the python wrapper overhead. Let's double
the limits.
Reviewed By: DurhamG
Differential Revision: D6879874
fbshipit-source-id: 19175be8c4f48dda1fc8917d75798d840d4082da
Summary:
When `--with-hg` and `-l` are both provided. `-l` will override
`--with-hg`, which is unfriendly. Let's do the reverse.
Reviewed By: DurhamG
Differential Revision: D6879863
fbshipit-source-id: 00ea9805431cc7afa6dd521cfb9059c898ed6d1f
Summary:
This makes it easier to run integration tests where we want to load
additional sets of config files.
Reviewed By: DurhamG
Differential Revision: D6755017
fbshipit-source-id: af062fcc465eee969e1315f9430333908ef38303
Summary:
`run-tests.py` should set up `sys.path` correctly so `import mercurial` in
the hgsubversion tests should just work.
Also add a comment in `run-tests.py` to clarify this.
The hgsubversion module check is removed since it assumes the module lives in a
same directory as the test, which is not true with the `run-test.py --rebuild`
case.
Reviewed By: phillco
Differential Revision: D6758588
fbshipit-source-id: ba34253991e671dc972bcb1900243eab6b69fd92
Summary: Added --rebuild for the old behavior.
Reviewed By: phillco
Differential Revision: D6753296
fbshipit-source-id: 84f26af285662f91bc59db8b2a3145212118a9b6
Summary:
Given the multi-core era, it seems to be a good default to make use of all
available cores. Use `-j 1` to get the old behavior.
Reviewed By: phillco
Differential Revision: D6752704
fbshipit-source-id: 842ddea1d89160eed030fc71ba08024859db9268
Summary:
`lfs` is the better large file solution. `largefiles` is rarely used, and
its implementation is less clean. So let's remove it.
Test Plan:
Ran all tests. A subrepo test was removed instead of cleaned up since the
longer term plan is to also drop subrepo support.
Reviewers: phillco, #mercurial
Reviewed By: phillco
Differential Revision: https://phabricator.intern.facebook.com/D6740361
Signature: 6740361:1516225594:555e3803571ad05e0434021897a2823ac99347ae
Summary:
To incorporate hgsubversion tests into our test suite, we need to turn them into
silentrunner tests, which run python unittests, but do check for the output to match
some expected file. hgsubversion tests were not written with checking output in mind
(and even if they were, .t tests need globbing/regex support as well). So, this
simple support for (re) suffixes on .py.out files allow us to match unstable output.
Test Plan:
- perform some manual experiments
- following diffs in the stack make use of it
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D6719877
Summary:
Previously there is a race condition because things happen in this order:
1. Check shouldStop
2. If shouldStop is false, print the diff
3. Call fail() -> set shouldStop
The check and set should really happen in a same critical section.
This patch adds a lock to address the issue.
The patch was also sent as https://phab.mercurial-scm.org/D1830
Test Plan: Run `run-tests.py -l test-run-tests.t` many times and it no longer fails.
Reviewers: durham, #mercurial
Reviewed By: durham
Subscribers: durham
Differential Revision: https://phabricator.intern.facebook.com/D6680300
Signature: 6680300:1515524998:260c3e198330e7e6c94dcb6cf155f14a055b760a
I've seen the following error a few times recently when running the tests with
`yes | ./run-tests.py --local -j9 -i`:
Errored test-add.t: Traceback (most recent call last):
File "./run-tests.py", line 821, in run
self.runTest()
File "./run-tests.py", line 910, in runTest
if self._result.addOutputMismatch(self, ret, out, self._refout):
File "./run-tests.py", line 1774, in addOutputMismatch
rename(test.errpath, test.path)
File "./run-tests.py", line 571, in rename
os.remove(src)
WindowsError: [Error 32] The process cannot access the file because it is being
used by another process: 'c:\\Users\\Matt\\projects\\hg\\tests\\test-add.t.err'
This change doesn't fix the problem, but it seems like a simple enough
improvement.
