If the matching command lives in an in-tree extension (which is all we
scan for), and the user has disabled that extension with
"extensions.<name>=!", we were not finding it, because the path in
_disabledextensions was the empty string. If the user had set
"extensions.<name>=!<valid path>" it would work, so it seems like just
a mistake that it didn't work.
Extensions can have a 'configtable' mapping and use
'registrar.configitem(table)' to retrieve the registration function.
This behave in the same way as the other way for extensions to register new
items (commands, colors, etc).
Before this patch, functions defined in extensions are registered via
extra loaders only in _dispatch(). Therefore, loading extensions in
other code paths like below omits registration of functions.
- WSGI service
- operation across repositories (e.g. subrepo)
- test-duplicateoptions.py, using extensions.loadall() directly
To register functions always at loading new extension, this patch
moves implementation for extra loading from dispatch._dispatch() to
extensions.loadall().
AFAIK, only commands module causes cyclic dependency between
extensions module, but this patch imports all related modules just
before extra loading in loadall(), in order to centralize them.
This patch makes extensions.py depend on many other modules, even
though extensions.py itself doesn't. It should be avoided if possible,
but I don't have any better idea. Some other places like below aren't
reasonable for extra loading, IMHO.
- specific function in newly added module:
existing callers of extensions.loadall() should invoke it, too
- hg.repository() or so:
no-repo commands aren't covered by this.
BTW, this patch removes _loaded.add(name) on relocation, because
dispatch._loaded is used only for extraloaders (for similar reason,
"exts" variable is removed, too).
If an extension was loaded but disabled due to a minimumhgversion check it
will be present in the _extensions map, but set to None. The rest of the
extensions code treats the extension as if it were not present in this case,
but the afterloaded() function called the callback with loaded=True.
459366b580cf makes profiler start early without loading extensions. That
makes it impossible for an extension to add customized profilers.
This patch adds a special case: if a profiler is not found but an extension
with the same name could be loaded, load that extension first, and expect it
to have a "profile" contextmanager method. This allows customized profilers
and extension setup time is still profiled.
When we try to pass a bytes argument to a function from imp library, it
returns TypeError as it deals with unicodes internally. So we can't use bytes
with imp.* functions. Hunting through this, I found we were returning bytes
path variable to loadpath() on Python 3.5 (yes most of our codebase is
dealing with bytes on Python 3 especially the path variables). Passing unicode
does not fails the purpose of loding the extensions and a module object is
returned.
This patch does not exactly solve issue5228 but it results in a better
condition on this issue. For disabled extensions, we used to parse the
module and get the first occurrences of docstring and then return the first
line of that as an introductory heading of extension. This is what we get
today.
This patch returns the whole docstring of the module as a help for extension,
which is more informative. There are some modules which don't have much
docstring at top level except the heading so those are unaffected by this
change. To follow the existing trend of showing commands either we have to
load the extension or have a very ugly parsing method which don't even assure
correctness.
The "load" method does too many things: on-demand import and check version.
This patch moves the import logic out from "load" so it could be wrapped to
change the import behavior, for example, chg will use it to pre-import
extensions.
I've caught multiple extensions in the wild lying about being
'internal', so it's time to move the goalposts on people. Goalpost
moving will continue until third party extensions stop trying to
defeat the system.
Before this patch, we don't have a safe way to undo a wrapfunction because
other extensions may wrap the same function and calling setattr will undo
them accidentally.
This patch adds an "unwrapfunction" to address the issue. It removes the
wrapper from the wrapper chain, and re-wraps everything, which is not the
most efficient but short and easy to understand. We can revisit the code
if we have perf issues with long chains.
The "undo" feature is useful in cases like wrapping a function just in a
scope. Like, having a "select" command to interactively (using arrow keys)
select content from some output (ex. smartlog). It could wrap "ui.label" to
extract interesting texts just in the "select" command.
The getwrapperchain returns a list of wrappers + the original function, making
it easier to understand what has been wrapped by whom. For example:
In : mercurial.extensions.getwrapperchain(mercurial.dispatch, '_runcommand')
Out:
[<function hgext.pager.pagecmd>,
<function hgext.color.colorcmd>,
<function hgext.zeroconf.cleanupafterdispatch>,
<function mercurial.dispatch._runcommand>]
It will also be useful to safely unwrap a function. See the next patch.
This is to make them wrap-able. chgserver wants to know if an extension
accesses config or environment variables during uisetup and extsetup and
include them in confighash accordingly.
Mercurial extensions are not meant to be normal python package/module. Yet the
lack of an official location to install them means that a lot of them actually
install as root level python package, polluting the global Python package
namespace and risking collision with more legit packages. As we recently
discovered, core python actually support namespace package. A way for multiples
distinct "distribution" to share a common top level package without fear of
installation headache. (Namespace package allow submodule installed in different
location (of the 'sys.path') to be imported properly. So we are fine as long as
extension includes a proper 'hgext3rd.__init__.py' to declare the namespace
package.)
Therefore we introduce a 'hgext3rd' namespace packages and search for extension
in it. We'll then recommend third extensions to install themselves in it.
Strictly speaking we could just get third party extensions to install in 'hgext'
as it is also a namespace package. However, this would make the integration of
formerly third party extensions in the main distribution more complicated as the third
party install would overwrite the file from the main install. Moreover, having an
explicit split between third party and core extensions seems like a good idea.
The name 'hgext3rd' have been picked because it is short and seems explicit enough.
Other alternative I could think of where:
- hgextcontrib
- hgextother
- hgextunofficial
To clarify third party extensions lookup, we are about to add a third place
where extensions are searched for. So we factor the error reporting logic out to
be able to easily reuse it in the next patch.
Future patches will make @command decorator set properties such as "norepo" to
a function object. This patch makes sure these properties never be lost by
wrapcommand() or wrapfunction().
This change won't be crazy as the standard functools.wraps() copies __dict__.
Before this patch, there is no easy way to detect if there are extensions
failed to load. While this is okay for most situations, chgserver is designed
to preload all extensions specified in config and does need the information.
This patch adds extensions.notloaded() to return names of extensions failed
to load.
As the author of several 3rd party extensions, I frequently see bug
reports from users attempting to run my extension with an old version
of Mercurial that I no longer support in my extension. Oftentimes, the
extension will import just fine. But as soon as we run extsetup(),
reposetup(), or get into the guts of a wrapped function, we encounter
an exception and abort. Today, Mercurial will print a message about
extensions that don't have a "testedwith" declaring explicit
compatibility with the current version.
The existing mechanism is a good start. But it isn't as robust as I
would like. Specifically, Mercurial assumes compatibility by default.
This means extension authors must perform compatibility checking in
their extsetup() or we wait and see if we encounter an abort at
runtime. And, compatibility checking can involve a lot of code and
lots of error checking. It's a lot of effort for extension authors.
Oftentimes, extension authors know which versions of Mercurial there
extension works on and more importantly where it is broken.
This patch introduces a magic "minimumhgversion" attribute in
extensions. When found, the extension loading mechanism will compare
the declared version against the current Mercurial version. If the
extension explicitly states we require a newer Mercurial version, a
warning is printed and the extension isn't loaded beyond importing
the Python module. This causes a graceful failure while alerting
the user of the compatibility issue.
I would be receptive to the idea of making the failure more fatal.
However, care would need to be taken to not criple every hg command.
e.g. the user may use `hg config` to fix the hgrc and if we aborted
trying to run that, the user would effectively be locked out of `hg`!
A potential future improvement to this functionality would be to catch
ImportError for the extension/module and parse the source code for
"minimumhgversion = 'XXX'" and do similar checking. This way we could
give more information about why the extension failed to load.
This should have been done as part of or as an immediate follow-up to
01b01d59e33f, but presumably this feature of extensions.py was
forgotten at that time.
It was previously not at all obvious what this was for.
We also change it to a set to avoid iterating over an admittedly
small list repeatedly at startup time.
Python 2.6 introduced the "except type as instance" syntax, replacing
the "except type, instance" syntax that came before. Python 3 dropped
support for the latter syntax. Since we no longer support Python 2.4 or
2.5, we have no need to continue supporting the "except type, instance".
This patch mass rewrites the exception syntax to be Python 2.6+ and
Python 3 compatible.
This patch was produced by running `2to3 -f except -w -n .`.
Before this patch, there was no handy way to investigate the reason why
extension couldn't be loaded.
If ui.debug is set, tracebacks of both "hgext.foo" and "foo" are displayed
because the first ImportError could occur at very deep dependency module.