Summary:
The zstore for commit messages are now handled by Rust entirely. There is no
need to keep the Python zstore around, except for migration and doctor
use-cases.
Reviewed By: andll
Differential Revision: D27897893
fbshipit-source-id: 21b10596af28c45425f6f102fd13f0421d1e8373
Summary: This seems to have broken the EdenFS HgPrefetch test.
Reviewed By: xavierd
Differential Revision: D27795192
fbshipit-source-id: 80a748036961aa6a5750182bf65637fb76825341
Summary:
Previously we'd skip dynamicconfigs when there wasn't a repo available.
Now that dynamicconfig can represent the NoRepo state, let's load dynamicconfigs
in that situation.
This also supports the case where there is no user name.
Reviewed By: kulshrax
Differential Revision: D26801059
fbshipit-source-id: 377cfffb6695a7fbe31303868a88862259ebf8c4
Summary: WSAEACCES is included in Windows only. Only check it if the os is windows.
Reviewed By: quark-zju
Differential Revision: D27434606
fbshipit-source-id: 25eb8036363b42629fbd010f7637a404dccff236
Summary: This fixes various tests on Windows. Many of these fixes were ported from upstream.
Reviewed By: markbt
Differential Revision: D27174617
fbshipit-source-id: b9f36ad0714793f2b76db32c1d840284b744a841
Summary:
dynamicconfig has logic to avoid loading remote configs during tests,
by checking the TESTTMP variable. Unfortunately, the tests currently run system
hg with that environment set, which causes it to generate incorrect
configuration for the real repository (i.e. fbsource).
Let's get rid of all test specific environment variables when running the system
hg. There's a comment in the code saying this breaks some tests (from 2017
03a31be593db), but as far as I can see they still work.
Reviewed By: quark-zju
Differential Revision: D26758364
fbshipit-source-id: 945cbd36f7eccc98149772a42421cbc51b157c5b
Summary:
Windows tests were failing to execute hg clone ssh://user@dummy/... lines because setconfig ui.ssh=C:/foo/bar was being translated to ui.ssh=c;c:\\foo\\bar by mingw, since it detected /foo/bar as a unix-style path. This seems to be caused by this one line reversing the slashes. I'm not sure why it exists, but deleting it makes the tests pass.
My guess is this has been broken for about a year.
Reviewed By: quark-zju
Differential Revision: D26639206
fbshipit-source-id: d89cae1ea3dd055b90ec6ee8f7cdbee2ae08b228
Summary:
The Rust IO handles progress and streampager stuff. Switch to it so we don't
need to changing the `fout`, `ferr` when handling streampager in Python.
The chgserver logic is updated to just set raw fd 0, 1, 2 to update stdio,
since `fileno` is no longer exposed from Rust.
Manually tested the following commands, both without chg and with chg:
- lhg log -r . (no pager)
- lhg log (with streampager)
- lhg log --config pager.pager=less (with less pager)
- lhg commit (spawns pager)
- lhg debugprogress -s 100 --sleep 100 --with-output --pager=off (progress in stderr)
- lhg debugprogress -s 100 --sleep 100 --with-output --pager=on --config pager.interface=fullscreen (progress in streampager)
- lhg debugprogress -s 100 --sleep 100 --with-output --pager=on --config pager.pager='LESS= less' (progress is disabled with external pager)
Reviewed By: sfilipco
Differential Revision: D26612487
fbshipit-source-id: 8b4e36b614a0c080b93e41474f9a8fc33f890083
Summary:
We shouldn't delete from a dictionary while iterating over it, instead we should iterate over a copy and then delete from the original.
`.keys()` returns a view of the dict, while wrapping it in `list` makes a deep copy.
Reviewed By: StanislavGlebik
Differential Revision: D26618782
fbshipit-source-id: 234503aab0a0a2151ae0818282651b547f414016
Summary:
For `repo.transaction("tr-name")`, this records `Transaction: tr-name` to
metalog commit description.
It can be helpful to narrow things down for commands with multiple
transactions.
In the future we might want to attach more data to the logging (ex. what the
commit cloud local, remote states at the time of syncing). However I didn't
do it now since metalog is designed to hold repository data, not too much
logging data. With a better logging infra we might want to move `config` out
from metalog, associated with metalog root ids.
Reviewed By: DurhamG
Differential Revision: D25984805
fbshipit-source-id: 59c074272cff555c6ff11dd755f7e3ce9a292eb6
Summary: Suddenly prompt stopped appearing for me. Flush the stream to be sure that it's printed out.
Reviewed By: HarveyHunt
Differential Revision: D25956018
fbshipit-source-id: 83419037fa6ce672e203385b71f1403a738d0c90
Summary:
When running `python3 run-tests.py test-run-tests.py`, some bytes were printed
with `b` prefix. Convert them to `str`.
Reviewed By: DurhamG
Differential Revision: D25642164
fbshipit-source-id: f1103b24ad88d0d024f6be546bf632141f06ebd1
Summary:
configs.allowedlocations restricts what configs can be loaded to a
certain set of files. This will enable us to deprecate all old config locations.
This diff adds Python support and a high level test.
Reviewed By: quark-zju
Differential Revision: D25539736
fbshipit-source-id: fa2544379b65672227e0d9cf08dad7016d6bbac8
Summary:
The logic inside `eden/scm/tests/features.py` script enables certain
features of HG on tests based on the name of the test. Mononoke's integration
tests suit reuses eden/scm's tests suit and as a consequence it triggers the
same `features.py` logic. It was fine until D24201934 (f5988c415c) introduced a feature for
test that is named in the same way as one of the Mononoke tests - the
`test-clienttelemetry.t`.
In order to fix this problem Mononoke will pass a `--nofeatures` flag to the
`eden/scm/run-tests.py` invocations that will turn off the usage of
`features.py`.
Reviewed By: farnz
Differential Revision: D24276294
fbshipit-source-id: eb28ed55a05de3b012e37407603c2370adaaad16
Summary:
With the new tracing-core (0.1.10 -> 0.1.16), it's no longer effective
to set EDENSCM_TRACE_LEVEL to more verbose within the test.
Let's set it in run-tests.py which spawns the test process instead.
Reviewed By: kulshrax
Differential Revision: D24004327
fbshipit-source-id: db2cadc7334eb59b25a1e0517e3d1a513e31e0fe
Summary:
The remaining test failures are mostly around bundle support, which
I'll fix in a later diff.
Reviewed By: quark-zju
Differential Revision: D23664037
fbshipit-source-id: 2bdde3cb4fcded6e0cf3afdc23269662544821df
Summary:
extdiff uses shutil.rmtree which calls os.rmdir with new python 3
options. Since we pathc os.rmdir, we need to support those options.
Reviewed By: quark-zju
Differential Revision: D23350968
fbshipit-source-id: 081d179dcd67b51ffdeb6b85899adf4e574a8d0f
Summary: Rather than using a Python program, simulate having no editor during tests using `internal:none`.
Reviewed By: quark-zju, ikostia
Differential Revision: D23152446
fbshipit-source-id: 5560f58885ee5959c62f0ac8bcf0483b9c3072f6
Summary:
Pull Request resolved: https://github.com/facebookexperimental/eden/pull/41
As of D22098359 (7f1588131b) the default locale used by integration tests is en_US.UTF-8, but as the comment in code mentiones:
```
The en_US.UTF-8 locale doesn't behave the same on all systems and trying to run
commands like "sed" or "tr" on non-utf8 data will result in "Illegal byte
sequence" error.
That is why we are forcing the "C" locale.
```
Additionally I've changed the test-walker-throttle.t test to use "/bin/date" directly. Previously it was using "/usr/bin/date", but the "/bin/date" is a more standard path as it works on MacOS.
Reviewed By: krallin
Differential Revision: D22865007
fbshipit-source-id: afd1346e1753df84bcfc4cf88651813c06933f79
Summary:
The zstore-commit-data code paths are in Python. We want to move them to behind
the Rust HgCommits abstractions. So stop making Python interact with the
low-level details.
Reviewed By: DurhamG
Differential Revision: D22638457
fbshipit-source-id: 435db8425a29ce4eae24a6202ad928f85a5f5ee2
Summary: This makes tests depend less on revision numbers.
Reviewed By: DurhamG
Differential Revision: D22468669
fbshipit-source-id: 74a06930faa3e6ee9d246ecc718c2a3740f57a54
Summary:
Some tests use too many revision numbers (ex. test-rebase-scenario-global.t).
It's time consuming to fix it manually. Write a script to help with that.
This will be used by the next diff.
Reviewed By: DurhamG
Differential Revision: D22174956
fbshipit-source-id: a873ad326ecbd9cdfa2df58839b0ef21626e1506
Summary:
Sometimes `Ctrl+C` the test runner does not fully stop it. From gdb it seems
the test runner is waiting for a thread which might have deadlocked. The
progress thread does not have anything critical that need to sync back to
the main program. Avoid waiting for it to make Ctrl+C work better.
Reviewed By: kulshrax
Differential Revision: D22290453
fbshipit-source-id: bdc5260cbd339cc392728834330609343c0048d3
Summary:
We support unicode file paths, and in python 3 those get passed to
python libraries as unicode strings. The tests set LANG=C which mean the python
library tries to convert the path to ascii, but fails for any non-ascii
characters. Let's switch to LANG="en_US.UTF-8" to match our production
behavior and make tests about unicode paths work.
Reviewed By: xavierd
Differential Revision: D22098359
fbshipit-source-id: c3057edc66e6e32f7b8b49374e622d02bd05711f
Summary:
Add a `--record` flag, which uses `git` to track changes in `$TESTTMP`.
I use it to track changes with C revlog vs the Rust revlog.
Reviewed By: farnz
Differential Revision: D22007501
fbshipit-source-id: a8027aa4b2b3b97c3cf5f642434451826fcbded9
Summary:
Incompatible tests are blacklisted.
The affects non-tests environment: `proxy-obsstore` is turned on.
That is in theory not very different from having an empty obsstore.
If it becomes an issue we can easily turn it off.
Reviewed By: sfilipco
Differential Revision: D21899139
fbshipit-source-id: e4a87ecced0542e7852b48b1a807775e523b3cc9
Summary:
I found myself friendly editing the output of run-tests.py:
Failed test-failure.t: output changed
Failed test-failure-unicode.t: output changed
to extract file names:
test-failure-unicode.t
test-failure.t
Then use the file names as parameters to other commands, like `run-tests.py`
again, or `hg revert`, or `hg shelve`.
Change the output format to print the filenames directly to make it easier to
use.
Reviewed By: sfilipco
Differential Revision: D21913609
fbshipit-source-id: 4b1d2265f384ad9afbe478b7ff0d6e543eff00df
Summary:
Enables treemanifest for about 100 more tests. To make them pass I had
to expose the gettreepack capability from peers, which just works, and enable
treeonly and sendtrees by default in the tests, which I probably should've done
before anyway.
Reviewed By: quark-zju
Differential Revision: D21894955
fbshipit-source-id: 7a6a4d453824fb8c81a797a5487bf2ecc2b67761
Summary:
Update `contrib/check-code.py` to Python 3.
Mostly it was already compatible, however stricter regular expression parsing
revealed a case where one of our tests wasn't working, and as a result lots of
instances of `open(file).read()` existed that this test should have caught.
I have fixed up most of the instances in the code, although there are many
in the test suite that I have ignored for now.
Reviewed By: quark-zju
Differential Revision: D21427212
fbshipit-source-id: 7461a7c391e0ade947f779a2b476ca937fd24a8d
Summary: Print out a command that can be copied and executed to make `--keeptmp` more handy.
Reviewed By: sfilipco
Differential Revision: D20829140
fbshipit-source-id: 7976e3f64fd423425ec29634a53a34f7b5e091d0
Summary: Let's enable it for tests. We'll slow roll it in production.
Reviewed By: quark-zju
Differential Revision: D19543790
fbshipit-source-id: be7d18dd8ffe51615a27c39ebf4247ec405b4097
Summary:
If hghave fails to check a feature because the feature name is unknown, treat
it as a test failure instead of skipping the entire test. This is especially
useful since `#if feature-name` only affects part of the test and failing to
test the feature should not skip the entire test. It also allows us to capture
issues about mis-spelled feature names or stale feature tests.
This has bitten us twice in the past:
- D18819680 removed `pure` and accidently disabled tests including
`test-install.t`, `test-annotate.t` and `test-issue4074.t`. Those tests got
re-enabled as part of D20155399, while they pass Python 2 tests,
the Python 3 tests were failing.
- D18088850 removed svn related feature checks, which has caused some issues
that got fixed by D18713921 and D18713922.<Paste>
Reviewed By: xavierd
Differential Revision: D20231782
fbshipit-source-id: 6adf99bd79b2a295d4e84ce4da5f9425a100936a
Summary:
The ssh output order issue is a large contributor to test flakiness.
Example test failures are:
```
--- test-unbundlereplay.t
+++ test-unbundlereplay.t.respondfully.err
@@ -154,9 +154,9 @@
remote: [ReplayVerification] Expected: (master_bookmark, c2e526aacb5100b7c1ddb9b711d2e012e6c
69cda). Actual: (master_bookmark, 893d83f11bf81ce2b895a93d51638d4049d56ce2)
remote: pushkey-abort: prepushkey hook exited with status 1
remote: transaction abort!
+ replay failed: error:pushkey
+ unbundle replay batch item #0 failed
remote: rollback completed
- replay failed: error:pushkey
- unbundle replay batch item #0 failed
[1]
$ cat $TESTTMP/reports.txt
unbundle replay batch item #0 failed
--- test-commitcloud-backup-all.t
+++ test-commitcloud-backup-all.t.err
@@ -59,9 +59,9 @@
remote: pushing 1 commit:
remote: eccc11f58a56 D3
backing up stack rooted at 42952ab62cec
+ backing up stack rooted at 4903fdffd9c6
remote: pushing 1 commit:
remote: 42952ab62cec E1
- backing up stack rooted at 4903fdffd9c6
remote: pushing 1 commit:
remote: 4903fdffd9c6 E2
commitcloud: backed up 8 commits
test-fb-hgext-lfspushrebase-verify-blobs.t
--- test-fb-hgext-treemanifest-pushrebase.t
+++ test-fb-hgext-treemanifest-pushrebase.t.err
@@ -127,9 +127,9 @@
$ hg push --to master -B master --config treemanifest.sendtrees=True
pushing to ssh://user@dummy/master
searching for changes
- remote: baz
remote: prepushrebase.cat hook exited with status 1
abort: push failed on remote
+ remote: baz
[255]
- Disable the hook
```
The order is nondeterministic because the stderr reading thread can read the
content before or after ui.write or ui.write_err in the main thread.
This diff introduces an optional feature in dummyssh that buffers all stderr
output and only write them after the wrapped hg serve process has exited, at
which time the hg client should also have completed its operations and has no
reason to ui.write or ui.write_err anything nondeterministically. Then the
dummyssh wrapper writes out the buffered stderr so the output order becomes
well defined.
Reviewed By: xavierd
Differential Revision: D19872612
fbshipit-source-id: 84710f98a8e6b4a1c283ffecf008585cca12be0a
Summary:
On Windows, there are *two* 8-bit encodings for each process.
* The ANSI code page is used for all `...A` system calls, and this is what
Mercurial uses internally. It can be overridden using the `--encoding`
command line option.
* The OEM code page is used when outputing to the console. Mercurial has no
concept of this, and instead renders to the console using the ANSI code page,
which results in mojibake like "Θ" instead of "é".
Add the concept of an `outputencoding`. If this differs from `encoding`, we
convert from the local encoding to the output encoding before writing to the
console.
On non-Windows platforms, this defaults to the same encoding as the local encoding,
so this is a no-op unless `--outputencoding` is manually specified.
On Windows, this defaults to the codepage given by `GetOEMCP`, causing output
to be converted to the OEM codepage before being printed.
For ordinary strings, the local encoded version is wrapped by `localstr` if the
encoding does not round-trip cleanly. This means the output encoding works
even if the character is not represented in the local encoding.
Unfortunately, the templater is not localstr-clean, which means strings can get
flattened down to the local encoding and the original code points are lost. In
this case we can only output characters which are in the intersection of the
encoding and the output encoding.
Most US English Windows systems use cp1252 for the ANSI code page and cp437 for
the OEM code page. These both contain many accented characters, so users with
accented characters in their names will now see them correctly rendered.
All of this only applies to Python 2.7. In Python 3, everything is Unicode,
the `--encoding` and `--outputencoding` options do nothing, and it just works.
Reviewed By: quark-zju, ikostia
Differential Revision: D19951381
fbshipit-source-id: d5cb8b5bfe2bc131b2e6c3b892137a48b2139ca9
Summary:
See later in this stack for motivation. This seems to work fine, and it allows
characters that don't fit latin1 when rendering diffs.
Reviewed By: markbt
Differential Revision: D19969743
fbshipit-source-id: 79c4afce5a19822d9b075d23ff4c88aa76ce2f42
Summary:
Some Mononoke tests test about the backtrace. Having the environment variables
set break those tests. Clear them.
Reviewed By: xavierd
Differential Revision: D19887219
fbshipit-source-id: 529a17282c40730ee95bffbed00070edd1f0823a
Summary:
In D19581174 I made `--debug` trigger ipdb session for `.t` tests.
This diff adds similar feature for `.py` tests.
Reviewed By: DurhamG
Differential Revision: D19784795
fbshipit-source-id: 9118c74ea465320f15aa6ad5e1e04d8bc1a49966
Summary:
On python 2, os.environ is expected to be bytes, so let's remove the
forced decoding. This broke run-tests.py on Windows.
Reviewed By: singhsrb
Differential Revision: D19737218
fbshipit-source-id: d4782eea279275f57cc91bc412a2d74857002cc1
Summary:
We generally use 'utf-8' but are using 'utf8' at some places. This
diff changes to using 'utf-8' consistently everywhere.
Reviewed By: farnz
Differential Revision: D19674111
fbshipit-source-id: 3be42859a180ded1c234b805259f086f9531ba18
Summary: This fixes the Windows build failure we are seeing. Apparently Python 2 on Windows does not like unicode type in `os.environ`
Reviewed By: markbt
Differential Revision: D19672308
fbshipit-source-id: b45c1ccf47bdd46dc31d6c980d0e70dd2e084c8f
Summary:
Fix some common errors when run `run-tests` in Python 3.
shlib would be using `str` for stdin/stdout instead of bytes as most of the code in Python 2 operates with that assumption. Getting it to `bytes` will take some work.
Reviewed By: DurhamG, quark-zju
Differential Revision: D19649475
fbshipit-source-id: 1f0190fce4df9a530246774fd17cdfc2d7974a65
Summary: This diff makes `run-test.py` compatible with Python 3, so we can run this tests both in Python 3 and 2.
Differential Revision: D19611680
fbshipit-source-id: 97571cca7e1310e482ac8c03af73eb345ab825c1
