Commit Graph

128 Commits

Author SHA1 Message Date
Jun Wu
484939a75d hgmain: make bindings a builtin module
Summary:
Global states (For example, the global blackbox instance, potentially some
logging / tracing libraries) are separate in the Rust and Python worlds.

That is because related code gets compiled separately:

  bindings.so (top-level)
   \_ blackbox

  hgmain (top-level)
   \_ blackbox (have a different global instance than the above blackbox)

To address it, make `bindings` a builtin module in `hgmain`.

The builtin module was renamed from `edenscmnative.bindings` to `bindings` so
it does not require importing anything else (For example, `edenscmnative`).

This unfortunately makes `hg` 100+ MB. Fortunately it can be compressed well
(gzip: 31MB).

Reviewed By: singhsrb

Differential Revision: D17429688

fbshipit-source-id: bf16910d7a260ca58db0d272fc95d8071d47bbc6
2019-09-20 18:32:36 -07:00
Jun Wu
4e8707262f tests: change some 'python' to 'hg debugpython'
Summary:
In the future `python` will no longer be able to import `bindings`.
Change them to `hg debugpython` so they stay compatible.

I dropped the "custom hghave" feature. It breaks and I don't think there
are users of it.

Reviewed By: singhsrb

Differential Revision: D17429689

fbshipit-source-id: 96e55ef25a027bd4ad33fc279f27c1d5cbed6861
2019-09-20 18:32:35 -07:00
Jun Wu
2232a26c42 testutil: add "eq" assertion with autofix ability
Summary: The "eq" API is similar to `assert_eq!` in Rust, with the autofix ability.

Reviewed By: xavierd

Differential Revision: D16168821

fbshipit-source-id: bf7dcfd24c42dee30c54aeeaa71eccab78a8f0f6
2019-07-17 21:11:27 -07:00
Jun Wu
a17a87bf4f setup: move native extensions to edenscmnative
Summary:
This just moves things around. So native and pure Python modules are split to
different Python packages. This makes it possible to use the standard zip
importer without hacks (ex. `hgdemandimport/embeddedimport`).

This diff is mostly about moving things. While `make local` still works,
it does break nupkg build, which will be fixed in a later diff.

Reviewed By: kulshrax

Differential Revision: D15798642

fbshipit-source-id: 5d83f17099aa198df0acd5b7a99667e2f35fe7b4
2019-06-19 17:55:49 -07:00
Mark Thomas
ae0a81f2c2 rust: move bindings to a single python extension
Summary:
Move all Rust bindings to a single python extension, `bindings`.  This should
improve compilation time and make things simpler.

Reviewed By: quark-zju

Differential Revision: D13923866

fbshipit-source-id: 560592b5a6c0c4f1b836c755ef123666a1059164
2019-02-01 17:53:22 -08:00
Jun Wu
9dc21f8d0b codemod: import from the edenscm package
Summary:
D13853115 adds `edenscm/` to `sys.path` and code still uses `import mercurial`.
That has nasty problems if both `import mercurial` and
`import edenscm.mercurial` are used, because Python would think `mercurial.foo`
and `edenscm.mercurial.foo` are different modules so code like
`try: ... except mercurial.error.Foo: ...`, or `isinstance(x, mercurial.foo.Bar)`
would fail to handle the `edenscm.mercurial` version. There are also some
module-level states (ex. `extensions._extensions`) that would cause trouble if
they have multiple versions in a single process.

Change imports to use the `edenscm` so ideally the `mercurial` is no longer
imported at all. Add checks in extensions.py to catch unexpected extensions
importing modules from the old (wrong) locations when running tests.

Reviewed By: phillco

Differential Revision: D13868981

fbshipit-source-id: f4e2513766957fd81d85407994f7521a08e4de48
2019-01-29 17:25:32 -08:00
Jun Wu
c12e300bb8 codemod: move Python packages to edenscm
Summary:
Move top-level Python packages `mercurial`, `hgext` and `hgdemandimport` to
a new top-level package `edenscm`. This allows the Python packages provided by
the upstream Mercurial to be installed side-by-side.

To maintain compatibility, `edenscm/` gets added to `sys.path` in
`mercurial/__init__.py`.

Reviewed By: phillco, ikostia

Differential Revision: D13853115

fbshipit-source-id: b296b0673dc54c61ef6a591ebc687057ff53b22e
2019-01-28 18:35:41 -08:00
Aida Getoeva
1c7a7e8eb2 debugcrdump: add --nobinary flag to avoid dumping binaries
Summary: Jellyfish uses `hg debugcrdump` to get commit data, including binary files. Now we want in jf to be able avoid dumping binary files, so I added `--nobinary` option to crdump

Reviewed By: quark-zju

Differential Revision: D12981968

fbshipit-source-id: 669ef4cf8f6225b911f5085c1a47fb7242f259bf
2018-11-09 08:50:37 -08:00
Jun Wu
02e2830919 treemanifest: fix filecache for manifestlog
Summary:
Manifestlog needs to use `00manifesttree.i` for filecache if treeonly is turned
on.

Teach `filecache` to also cache non-existed files and add `00manifesttree.i`
to `repo.manifestlog` check list.

Assuming inode cannot be 0, `cacheable` is a constant `True`. Drop code
dealing with `cacheable = False` accordingly.

Reviewed By: DurhamG

Differential Revision: D10417795

fbshipit-source-id: c8aedd36dc39592c86847bf4327ed9c46736bab0
2018-10-17 20:07:47 -07:00
Jun Wu
f3718fc48d util: skip checkexec or checklink for wider file systems
Summary:
D9328199 made them skip eden mounts. This diff expands the whitelist to other
common filesystems.

Reviewed By: wez

Differential Revision: D9416599

fbshipit-source-id: 5f1dddf5a181833a2fa6e0954c2579bb5adcd170
2018-08-21 12:21:07 -07:00
Kostia Balytskyi
65f35cf9bd hghabve: fix how the presence of zstd is detected
Summary:
D8342421 added `zstd` as a rust crate dependency, effectively moving `zstd` importable module from `mercurial` to `mercurial.rust`. Also, the new module now does not have `__version__`.
This broke `test-http-protocol`, since `hghave` reported `zstd` as unavailable, but some other output suggested it was there.

Reviewed By: mitrandir77

Differential Revision: D8395518

fbshipit-source-id: f5d14ff0d246d7de2b1d786b14650e1c3f80ecfd
2018-06-13 08:19:03 -07:00
Lukasz Langa
dfda82e492 Upgrade to 18.5b1
Summary: Mostly empty lines removed and added.  A few bugfixes on excessive line splitting.

Reviewed By: quark-zju

Differential Revision: D8199128

fbshipit-source-id: 90c1616061bfd7cfbba0b75f03f89683340374d5
2018-05-30 02:23:58 -07:00
Jun Wu
584656dff3 codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).

Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.

Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.

An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.

As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.

Reviewed By: wez, phillco, simpkins, pkaush, singhsrb

Differential Revision: D8173629

fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-25 22:17:29 -07:00
Mark Thomas
e453f31b7d hghave: remove hasattr test
Summary:
Remove the test for the `search` attribute on the passed-in regexp in
`hghave.matchoutput`.  It's not necessary, and causes new check-code errors.

Reviewed By: quark-zju

Differential Revision: D7098248

fbshipit-source-id: b5f18c0db3cfbb37c9f7e23e2cdfdef9cedd3f49
2018-04-13 21:51:16 -07:00
Adam Simpkins
0e49f78053 help: update the test to exclude eden help
Summary:
Some of our internal hg builds include the Eden extension now.  This was
causing the test-help.t test to fail since it included help output for the Eden
extension, but the test code did not expect this.

We unfortunately cannot update the test output to always expect help output for
Eden since the Eden extension is only included in buck-based builds.

Reviewed By: quark-zju, farnz

Differential Revision: D7063937

fbshipit-source-id: e503ddc6889e546b5333a8d9e3555097d689e24c
2018-04-13 21:51:16 -07:00
Jun Wu
b58ff2f085 tests: run pyflakes tests in buck test
Summary:
Within buck test envirionment there is no guarnatee that `pyflakes` binary is
in `PATH`.  This patch builds pyflakes as a test dependency so buck test will
run `test-check-pyflakes.t` instead of skipping it.

Reviewed By: markbt

Differential Revision: D7008100

fbshipit-source-id: eca6f0174dbc9c12d45eca2f17f70a9805f13094
2018-04-13 21:51:12 -07:00
Jun Wu
f5924da1d3 tests: improve test compatibility with different zlib
Summary:
When running with a Python runtime with a slightly different zlib module,
some `zlib.compress` outputs are different. Some tests are testing the
length, or the content of `zlib.compress` output, directly or indirectly.
That's causing issues.

This patch adds a `common-zlib` hghave test so it can be used to gate tests
checking zlib output. Some lengths are also changed to glob patterns to be
compatible.

Reviewed By: ryanmce

Differential Revision: D6937735

fbshipit-source-id: 2328a39d7f2022f16d51f61b6178568b26dfe2fb
2018-04-13 21:51:09 -07:00
Jun Wu
f9ba760920 lz4: fix "import lz4.block" compatibility with demandimport
Summary:
`import lz4.block` is incompatible with demandimport. Let's use
`from lz4 import block` instead.

Reviewed By: DurhamG

Differential Revision: D6908162

fbshipit-source-id: 37119e21f7b289f89e41ad04fbb7f1ec81181259
2018-04-13 21:51:04 -07:00
Jun Wu
e92a601d9b hghave: add normal-layout feature
Summary:
With `buck build`, the single hg binary won't be guarnateed to have
access to i18n messages because directories like `mercurial/locale`
do not exist on filesystem. It could also mess up with `PYTHONPATH`
somehow because the python binary wrapper sometimes ignores
`PYTHONPATH`.

So let's add a hghave feature for it. And gate troublesome tests
with `#if normal-layout`.

Reviewed By: DurhamG, phillco

Differential Revision: D6879876

fbshipit-source-id: 3d63605b55c8f7096093b89be824add2ec491f81
2018-04-13 21:51:02 -07:00
Jun Wu
cc33c003c2 lz4: import lz4.block
Summary:
`lz4.block` needs to be imported explicitly before being able to
use `lz4.block.compress`.

We didn't notice this because we're using an old version of
`python-lz4`.

Reviewed By: DurhamG

Differential Revision: D6879877

fbshipit-source-id: 37e8fdc00386bef3733753f925ad308f42e5a740
2018-04-13 21:51:01 -07:00
Phil Cohen
e45033bb3c hghave: make the test-repo feature always return True
Summary:
Enables running `check-*` tests from fbsource. This used to check for the existence of a `.hg` folder to detect a repo (so
it can run `hg files`). Internally, we'll always have a repo, and externally we'll likely use git to publish so we'll need another
solution there anyway.

(Note: this ignores all push blocking failures!)

Reviewed By: DurhamG

Differential Revision: D6792787

fbshipit-source-id: 342e29d3a99e82ccd24effb8df02ac6309e80909
2018-04-13 21:50:55 -07:00
Jun Wu
8a4cb4f683 test-lz4revlog: skip if lz4 module is unavailable
Summary: The test shouldn't run if the dependency (lz4) cannot be imported.

Test Plan:
Run the test on a machine that does not have Python lz4 module installed and
make sure the test gets skipped.

Reviewers: durham, #mercurial

Reviewed By: durham

Subscribers: durham

Differential Revision: https://phabricator.intern.facebook.com/D6678494

Signature: 6678494:1515454247:245401173d9e1ef16ab865c210b1f5412039c1e1
2018-01-08 15:28:27 -08:00
Matt Harbison
f4937719cf hghave: add a check for lfs-test-server
This is consistent with how the other tests require a feature.
2017-11-14 22:35:42 -05:00
Anton Shestakov
034b4b25c1 tests: use jshint when available to check .js files 2017-11-11 15:07:21 +08:00
Augie Fackler
3ea4e52f7c hghave: fix clang-format check to use bytes regex
Fixes hghave on Python 3.

Differential Revision: https://phab.mercurial-scm.org/D1182
2017-10-18 18:29:17 -04:00
Gregory Szorc
35538a441a fsmonitor: warn when fsmonitor could be used
fsmonitor can significantly speed up operations on large working
directories. But fsmonitor isn't enabled by default, so naive users
may not realize there is a potential to make Mercurial faster.

This commit introduces a warning to working directory updates when
fsmonitor could be used.

The following conditions must be met:

* Working directory is previously empty
* New working directory adds >= N files (currently 50,000)
* Running on Linux or MacOS
* fsmonitor not enabled
* Warning not disabled via config override

Because of the empty working directory restriction, most users will
only see this warning during `hg clone` (assuming very few users
actually do an `hg up null`).

The addition of a warning may be considered a BC change. However, clone
has printed warnings before. Until recently, Mercurial printed a warning
with the server's certificate fingerprint when it wasn't explicitly
trusted for example. The warning goes to stderr. So it shouldn't
interfere with scripts parsing meaningful output.

The OS restriction was on the advice of Facebook engineers, who only
feel confident with watchman's stability on the supported platforms.

.. feature::

   Print warning when fsmonitor isn't being used on a large repository

Differential Revision: https://phab.mercurial-scm.org/D894
2017-10-18 22:57:15 +02:00
Saurabh Singh
f7a6f008b9 hghave: disable demandimport when chg is running
Chg disables demandimport on purpose for performance wins and
therefore, it probably makes sense to indicate that demandimport is disabled
when chg is running.

Test Plan:
Ran all the tests.

Differential Revision: https://phab.mercurial-scm.org/D1161
2017-10-17 11:35:58 -07:00
Augie Fackler
e06dbb26d0 hghave: add a check for clang-format
Differential Revision: https://phab.mercurial-scm.org/D1068
2015-09-14 14:16:31 -04:00
Kyle Lippincott
a9ec21ac02 tests: add "have" check for dpkg builddeps
Differential Revision: https://phab.mercurial-scm.org/D879
2017-10-01 05:21:32 -07:00
Kyle Lippincott
8b9d729ef1 hghave: check for debuild being installed as well
Differential Revision: https://phab.mercurial-scm.org/D874
2017-10-01 04:10:01 -07:00
Kyle Lippincott
cfad578e95 hghave: fix dpkg --version check to work on recent dpkg versions
Differential Revision: https://phab.mercurial-scm.org/D869
2017-10-01 03:48:28 -07:00
Rishabh Madan
3c2249874a releasenotes: add import check for fuzzywuzzy
This patch adds the has_fuzzywuzzy for import check of external dependency
fuzzywuzzy.
2017-07-29 14:06:26 +05:30
Yuya Nishihara
f04eb5a8cd tests: drop "absimport" conditionals which should be always true 2017-07-16 17:47:10 +09:00
Sean Farley
cbdeeeebdc hghave: add has_git_range for testing if git understands ext::sh
Even on CentOS 7, git is at version 1.8. It seems git 1.9 is when
ext::sh was introduced so we a check for that. The way these functions
are written follows the same style and format for the way we check svn
and bzr versions.
2017-06-15 17:13:23 -07:00
Matt Harbison
72dbdbc1d5 hghave: enable 'serve' on Windows
I've been using a local hghaveaddon.py to enable this for a couple of months
with reasonable success, and 'killdaemons' is already enabled on Windows.
There's one failure[1] in test-http-proxy.t that this adds, which I can't figure
out.

[1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-April/096987.html
2017-05-07 14:58:40 -04:00
Siddharth Agarwal
1d32a76696 hghave: add test for whether fsmonitor is enabled
This uses the HGFSMONITOR_TESTS environment variable that
fsmonitor-run-tests.py adds.
2017-06-10 14:07:31 -07:00
Augie Fackler
3ec7cf5f67 hghave: add check for virtualenv 2017-06-06 11:16:10 -04:00
Kyle Lippincott
9d816ecc30 tests: hint how to run slow tests when rejecting 2017-05-26 14:52:54 -07:00
Martin von Zweigbergk
09d53c160b merge with stable 2017-05-12 11:20:25 -07:00
Pierre-Yves David
970619c327 hghave: prefill more version of Mercurial
The previous code was unable to go above version 4.0.
2017-05-09 15:08:47 +02:00
Gregory Szorc
23a56c76db hghave: remove py27+ capability
It is now unused. And we require Python 2.7+ now so this check is not
necessary.
2017-05-08 15:31:34 -07:00
Bryan O'Sullivan
cb19aa07d3 stdio: add Linux-specific tests for error checking 2017-04-11 14:54:12 -07:00
Yuya Nishihara
91f8f53f3a statfs: make getfstype() raise OSError
It's better for getfstype() function to not suppress an error. Callers can
handle it as necessary. Now "hg debugfsinfo" will report OSError.
2017-03-25 17:25:23 +09:00
Yuya Nishihara
aa4a627b31 hghave: use util.getfstype 2017-03-25 18:22:27 +09:00
Jun Wu
06da2c689d hghave: add a check about whitelisted filesystem that supports hardlink
This is needed for the test added by the next patch.
2017-03-12 00:26:20 -08:00
Pierre-Yves David
a51384137e test: add a basic 'test-check-pylint.t'
We add a minimal check using pylint for one case we knows we care about:
"mutable default" argument.

We'll likely extend this over time to cover other useful checks but this is a
good starting point.
2016-12-28 23:42:50 +01:00
Yuya Nishihara
1d44bd2bbb ui: factor out ui.load() to create a ui without loading configs (API)
This allows us to write doctests depending on a ui object, but not on global
configs.

ui.load() is a class method so we can do wsgiui.load(). All ui() calls but
for doctests are replaced with ui.load(). Some of them could be changed to
not load configs later.
2016-10-22 14:35:10 +09:00
Gregory Szorc
a002443acc hghave: add check for zstd support
Not all configurations will support zstd. Add a check so we can
conditionalize tests.
2016-11-10 23:38:41 -08:00
Yuya Nishihara
6aeed209ab hghave: check darcs version more strictly
test-convert-darcs.t suddenly started failing on my Debian sid machine. The
reason was Darcs was upgraded from 2.12.0 to 2.12.4 so the original pattern
got to match the last two digits. Fix the pattern to match 2.2+.
2016-11-05 13:20:53 +09:00
Yuya Nishihara
26d053eede hghave: fix 'rmcwd' to ensure temporary directory is removed
On platforms where cwd can't be removed, it should try rmdir() after chdir
to the original cwd.
2016-11-01 21:14:33 +09:00