2019-11-02 03:38:10 +03:00
|
|
|
# Portions Copyright (c) Facebook, Inc. and its affiliates.
|
|
|
|
#
|
|
|
|
# This software may be used and distributed according to the terms of the
|
|
|
|
# GNU General Public License version 2.
|
|
|
|
|
2017-05-21 22:23:04 +03:00
|
|
|
# demandimportpy3 - global demand-loading of modules for Mercurial
|
|
|
|
#
|
2019-11-02 03:38:10 +03:00
|
|
|
# Copyright Matt Mackall <mpm@selenic.com> and others
|
2017-05-21 22:23:04 +03:00
|
|
|
#
|
|
|
|
# This software may be used and distributed according to the terms of the
|
|
|
|
# GNU General Public License version 2 or any later version.
|
|
|
|
|
|
|
|
"""Lazy loading for Python 3.6 and above.
|
|
|
|
|
|
|
|
This uses the new importlib finder/loader functionality available in Python 3.5
|
|
|
|
and up. The code reuses most of the mechanics implemented inside importlib.util,
|
|
|
|
but with a few additions:
|
|
|
|
|
|
|
|
* Allow excluding certain modules from lazy imports.
|
|
|
|
* Expose an interface that's substantially the same as demandimport for
|
|
|
|
Python 2.
|
|
|
|
|
|
|
|
This also has some limitations compared to the Python 2 implementation:
|
|
|
|
|
|
|
|
* Much of the logic is per-package, not per-module, so any packages loaded
|
|
|
|
before demandimport is enabled will not be lazily imported in the future. In
|
|
|
|
practice, we only expect builtins to be loaded before demandimport is
|
|
|
|
enabled.
|
|
|
|
"""
|
|
|
|
|
|
|
|
# This line is unnecessary, but it satisfies test-check-py3-compat.t.
|
|
|
|
from __future__ import absolute_import
|
|
|
|
|
2020-06-22 18:57:50 +03:00
|
|
|
import concurrent.futures
|
2017-05-21 22:23:04 +03:00
|
|
|
import contextlib
|
|
|
|
import importlib.abc
|
|
|
|
import importlib.machinery
|
|
|
|
import importlib.util
|
2017-08-22 21:14:34 +03:00
|
|
|
import sys
|
2017-05-21 22:23:04 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2017-05-21 22:23:04 +03:00
|
|
|
_deactivated = False
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2017-05-21 22:23:04 +03:00
|
|
|
class _lazyloaderex(importlib.util.LazyLoader):
|
|
|
|
"""This is a LazyLoader except it also follows the _deactivated global and
|
|
|
|
the ignore list.
|
|
|
|
"""
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2017-05-21 22:23:04 +03:00
|
|
|
def exec_module(self, module):
|
|
|
|
"""Make the module load lazily."""
|
|
|
|
if _deactivated or module.__name__ in ignore:
|
|
|
|
self.loader.exec_module(module)
|
|
|
|
else:
|
|
|
|
super().exec_module(module)
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2020-02-05 22:21:41 +03:00
|
|
|
class LazyFinder(object):
|
|
|
|
"""A wrapper around a ``MetaPathFinder`` that makes loaders lazy.
|
|
|
|
|
|
|
|
``sys.meta_path`` finders have their ``find_spec()`` called to locate a
|
|
|
|
module. This returns a ``ModuleSpec`` if found or ``None``. The
|
|
|
|
``ModuleSpec`` has a ``loader`` attribute, which is called to actually
|
|
|
|
load a module.
|
|
|
|
|
|
|
|
Our class wraps an existing finder and overloads its ``find_spec()`` to
|
|
|
|
replace the ``loader`` with our lazy loader proxy.
|
|
|
|
|
|
|
|
We have to use __getattribute__ to proxy the instance because some meta
|
|
|
|
path finders don't support monkeypatching.
|
|
|
|
"""
|
|
|
|
|
|
|
|
__slots__ = ("_finder",)
|
|
|
|
|
|
|
|
def __init__(self, finder):
|
|
|
|
object.__setattr__(self, "_finder", finder)
|
|
|
|
|
|
|
|
def __repr__(self):
|
|
|
|
return "<LazyFinder for %r>" % object.__getattribute__(self, "_finder")
|
|
|
|
|
|
|
|
# __bool__ is canonical Python 3. But check-code insists on __nonzero__ being
|
|
|
|
# defined via `def`.
|
|
|
|
def __nonzero__(self):
|
|
|
|
return bool(object.__getattribute__(self, "_finder"))
|
|
|
|
|
|
|
|
__bool__ = __nonzero__
|
|
|
|
|
|
|
|
def __getattribute__(self, name):
|
|
|
|
if name in ("_finder", "find_spec"):
|
|
|
|
return object.__getattribute__(self, name)
|
|
|
|
|
|
|
|
return getattr(object.__getattribute__(self, "_finder"), name)
|
|
|
|
|
|
|
|
def __delattr__(self, name):
|
|
|
|
return delattr(object.__getattribute__(self, "_finder"))
|
|
|
|
|
|
|
|
def __setattr__(self, name, value):
|
|
|
|
return setattr(object.__getattribute__(self, "_finder"), name, value)
|
|
|
|
|
|
|
|
def find_spec(self, *args, **kwargs):
|
|
|
|
finder = object.__getattribute__(self, "_finder")
|
2020-04-10 04:20:36 +03:00
|
|
|
find_spec = getattr(finder, "find_spec", None)
|
|
|
|
if find_spec:
|
|
|
|
spec = find_spec(*args, **kwargs)
|
|
|
|
else:
|
|
|
|
spec = None
|
2020-02-05 22:21:41 +03:00
|
|
|
|
|
|
|
# Lazy loader requires exec_module().
|
|
|
|
if (
|
|
|
|
spec is not None
|
|
|
|
and spec.loader is not None
|
|
|
|
and getattr(spec.loader, "exec_module", None) is not None
|
|
|
|
):
|
|
|
|
spec.loader = _lazyloaderex(spec.loader)
|
|
|
|
|
|
|
|
return spec
|
2017-05-21 22:23:04 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2020-02-01 04:29:30 +03:00
|
|
|
ignore = set()
|
2017-05-21 22:23:04 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2017-05-21 22:23:04 +03:00
|
|
|
def init(ignorelist):
|
|
|
|
global ignore
|
2020-02-01 04:29:30 +03:00
|
|
|
ignore = set(ignorelist)
|
2017-05-21 22:23:04 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2017-05-21 22:23:04 +03:00
|
|
|
def isenabled():
|
2020-02-05 22:21:41 +03:00
|
|
|
return not _deactivated and any(
|
|
|
|
isinstance(finder, LazyFinder) for finder in sys.meta_path
|
|
|
|
)
|
2017-05-21 22:23:04 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2017-05-21 22:23:04 +03:00
|
|
|
def enable():
|
2020-02-05 22:21:41 +03:00
|
|
|
new_finders = []
|
|
|
|
for finder in sys.meta_path:
|
|
|
|
new_finders.append(
|
|
|
|
LazyFinder(finder) if not isinstance(finder, LazyFinder) else finder
|
|
|
|
)
|
|
|
|
|
|
|
|
sys.meta_path[:] = new_finders
|
2017-05-21 22:23:04 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2017-05-21 22:23:04 +03:00
|
|
|
@contextlib.contextmanager
|
|
|
|
def deactivated():
|
|
|
|
# This implementation is a bit different from Python 2's. Python 3
|
|
|
|
# maintains a per-package finder cache in sys.path_importer_cache (see
|
|
|
|
# PEP 302). This means that we can't just call disable + enable.
|
|
|
|
# If we do that, in situations like:
|
|
|
|
#
|
|
|
|
# demandimport.enable()
|
|
|
|
# ...
|
|
|
|
# from foo.bar import mod1
|
|
|
|
# with demandimport.deactivated():
|
|
|
|
# from foo.bar import mod2
|
|
|
|
#
|
|
|
|
# mod2 will be imported lazily. (The converse also holds -- whatever finder
|
|
|
|
# first gets cached will be used.)
|
|
|
|
#
|
|
|
|
# Instead, have a global flag the LazyLoader can use.
|
|
|
|
global _deactivated
|
|
|
|
demandenabled = isenabled()
|
|
|
|
if demandenabled:
|
|
|
|
_deactivated = True
|
|
|
|
try:
|
|
|
|
yield
|
|
|
|
finally:
|
|
|
|
if demandenabled:
|
|
|
|
_deactivated = False
|