459366b580cf makes profiler start early without loading extensions. That
makes it impossible for an extension to add customized profilers.
This patch adds a special case: if a profiler is not found but an extension
with the same name could be loaded, load that extension first, and expect it
to have a "profile" contextmanager method. This allows customized profilers
and extension setup time is still profiled.
When we try to pass a bytes argument to a function from imp library, it
returns TypeError as it deals with unicodes internally. So we can't use bytes
with imp.* functions. Hunting through this, I found we were returning bytes
path variable to loadpath() on Python 3.5 (yes most of our codebase is
dealing with bytes on Python 3 especially the path variables). Passing unicode
does not fails the purpose of loding the extensions and a module object is
returned.
This patch does not exactly solve issue5228 but it results in a better
condition on this issue. For disabled extensions, we used to parse the
module and get the first occurrences of docstring and then return the first
line of that as an introductory heading of extension. This is what we get
today.
This patch returns the whole docstring of the module as a help for extension,
which is more informative. There are some modules which don't have much
docstring at top level except the heading so those are unaffected by this
change. To follow the existing trend of showing commands either we have to
load the extension or have a very ugly parsing method which don't even assure
correctness.
The "load" method does too many things: on-demand import and check version.
This patch moves the import logic out from "load" so it could be wrapped to
change the import behavior, for example, chg will use it to pre-import
extensions.
I've caught multiple extensions in the wild lying about being
'internal', so it's time to move the goalposts on people. Goalpost
moving will continue until third party extensions stop trying to
defeat the system.
Before this patch, we don't have a safe way to undo a wrapfunction because
other extensions may wrap the same function and calling setattr will undo
them accidentally.
This patch adds an "unwrapfunction" to address the issue. It removes the
wrapper from the wrapper chain, and re-wraps everything, which is not the
most efficient but short and easy to understand. We can revisit the code
if we have perf issues with long chains.
The "undo" feature is useful in cases like wrapping a function just in a
scope. Like, having a "select" command to interactively (using arrow keys)
select content from some output (ex. smartlog). It could wrap "ui.label" to
extract interesting texts just in the "select" command.
The getwrapperchain returns a list of wrappers + the original function, making
it easier to understand what has been wrapped by whom. For example:
In : mercurial.extensions.getwrapperchain(mercurial.dispatch, '_runcommand')
Out:
[<function hgext.pager.pagecmd>,
<function hgext.color.colorcmd>,
<function hgext.zeroconf.cleanupafterdispatch>,
<function mercurial.dispatch._runcommand>]
It will also be useful to safely unwrap a function. See the next patch.
This is to make them wrap-able. chgserver wants to know if an extension
accesses config or environment variables during uisetup and extsetup and
include them in confighash accordingly.
Mercurial extensions are not meant to be normal python package/module. Yet the
lack of an official location to install them means that a lot of them actually
install as root level python package, polluting the global Python package
namespace and risking collision with more legit packages. As we recently
discovered, core python actually support namespace package. A way for multiples
distinct "distribution" to share a common top level package without fear of
installation headache. (Namespace package allow submodule installed in different
location (of the 'sys.path') to be imported properly. So we are fine as long as
extension includes a proper 'hgext3rd.__init__.py' to declare the namespace
package.)
Therefore we introduce a 'hgext3rd' namespace packages and search for extension
in it. We'll then recommend third extensions to install themselves in it.
Strictly speaking we could just get third party extensions to install in 'hgext'
as it is also a namespace package. However, this would make the integration of
formerly third party extensions in the main distribution more complicated as the third
party install would overwrite the file from the main install. Moreover, having an
explicit split between third party and core extensions seems like a good idea.
The name 'hgext3rd' have been picked because it is short and seems explicit enough.
Other alternative I could think of where:
- hgextcontrib
- hgextother
- hgextunofficial
To clarify third party extensions lookup, we are about to add a third place
where extensions are searched for. So we factor the error reporting logic out to
be able to easily reuse it in the next patch.
Future patches will make @command decorator set properties such as "norepo" to
a function object. This patch makes sure these properties never be lost by
wrapcommand() or wrapfunction().
This change won't be crazy as the standard functools.wraps() copies __dict__.
Before this patch, there is no easy way to detect if there are extensions
failed to load. While this is okay for most situations, chgserver is designed
to preload all extensions specified in config and does need the information.
This patch adds extensions.notloaded() to return names of extensions failed
to load.
As the author of several 3rd party extensions, I frequently see bug
reports from users attempting to run my extension with an old version
of Mercurial that I no longer support in my extension. Oftentimes, the
extension will import just fine. But as soon as we run extsetup(),
reposetup(), or get into the guts of a wrapped function, we encounter
an exception and abort. Today, Mercurial will print a message about
extensions that don't have a "testedwith" declaring explicit
compatibility with the current version.
The existing mechanism is a good start. But it isn't as robust as I
would like. Specifically, Mercurial assumes compatibility by default.
This means extension authors must perform compatibility checking in
their extsetup() or we wait and see if we encounter an abort at
runtime. And, compatibility checking can involve a lot of code and
lots of error checking. It's a lot of effort for extension authors.
Oftentimes, extension authors know which versions of Mercurial there
extension works on and more importantly where it is broken.
This patch introduces a magic "minimumhgversion" attribute in
extensions. When found, the extension loading mechanism will compare
the declared version against the current Mercurial version. If the
extension explicitly states we require a newer Mercurial version, a
warning is printed and the extension isn't loaded beyond importing
the Python module. This causes a graceful failure while alerting
the user of the compatibility issue.
I would be receptive to the idea of making the failure more fatal.
However, care would need to be taken to not criple every hg command.
e.g. the user may use `hg config` to fix the hgrc and if we aborted
trying to run that, the user would effectively be locked out of `hg`!
A potential future improvement to this functionality would be to catch
ImportError for the extension/module and parse the source code for
"minimumhgversion = 'XXX'" and do similar checking. This way we could
give more information about why the extension failed to load.
This should have been done as part of or as an immediate follow-up to
01b01d59e33f, but presumably this feature of extensions.py was
forgotten at that time.
It was previously not at all obvious what this was for.
We also change it to a set to avoid iterating over an admittedly
small list repeatedly at startup time.
Python 2.6 introduced the "except type as instance" syntax, replacing
the "except type, instance" syntax that came before. Python 3 dropped
support for the latter syntax. Since we no longer support Python 2.4 or
2.5, we have no need to continue supporting the "except type, instance".
This patch mass rewrites the exception syntax to be Python 2.6+ and
Python 3 compatible.
This patch was produced by running `2to3 -f except -w -n .`.
Before this patch, there was no handy way to investigate the reason why
extension couldn't be loaded.
If ui.debug is set, tracebacks of both "hgext.foo" and "foo" are displayed
because the first ImportError could occur at very deep dependency module.
It was reported that enabling pager without color could cause a hang.
Inserting print statements revealed that the callbacks in
extensions._aftercallbacks were being invoked twice.
extensions.loadall can be called multiple times. If entries in
extensions._aftercallbacks linger between calls, this could result
in double execution of the callbacks. This can lead to unwanted
behavior.
The reproduce steps in the bug seem to only occur when the output of
a command is less than the size of the current screen. This is not
something that can easily be tested. I verified the test case works
with this patch and that pager and color interaction continues to
work. Since we have no existing automated tests for pager, this sadly
appears to be the best testing I can do.
We use partial function application for wrapping existing Mercurial functions,
and it's implemented separately each time. This patch extracts the partial
application into a new bind() function that can be used on its own by extensions
when appropriate.
In particular, the evolve extension needs to wrap functions in the various
bundle2 processing dictionaries, which the pre-existing methods don't support.
Mercurial uses a synopsis string and the docstring of a command for the
command's help output. Today there is no way for an extension that adds
functionality to a command to extend either of these help strings.
This patch enables appending to both the doctring and the synopsis, and adds
a test for this functionality. Example usage is shown in the test and is also
described in the docstring of extensions.wrapcommand().
An upcoming patch will introduce a dependency between the color and
pager extensions. To prepare for this, we teach extensions how to
register callbacks that can execute when another extension loads.
This patch is based on code provided by Matt Mackall. But significant
parts have changed (such as the ability to register multiple callbacks
and the change in behavior to always call a callback).
I believe that always firing the callback is a good practice. I think
the common use for this feature will be for extensions to say "run this
one-time setup code, after this other extension if possible." Always
running the callback will facilitate this.
This was introduced way back in 2006 (rev e8e3582d6f80) as sys.exit(0) if
loading an extension failed when --traceback was on, then at some point morphed
into a 'return 1' in a function that otherwise returns nothing.
At this point, if ui.traceback is enabled and if loading an extension fails for
whatever reason, including one as innocent as it not being present, we leave
any extensions loaded so far in a bogus half-initialized state. That doesn't
really make any sense.
