The statprof sampling profiler runs with significantly less overhead.
Its data is therefore more useful. Furthermore, its default output
shows the hotpath by default, which I've found to be way more useful
than the default profiler's function time table.
There is one behavioral regression with this change worth noting:
the statprof profiler currently doesn't profile individual hgweb
requests like lsprof does. This is because the current implementation
of statprof only profiles the thread that started profiling.
The ability for lsprof to profile individual hgweb requests is
relatively new and likely not widely used. Furthermore, I have plans
to modify statprof to support profiling multiple threads. I expect
that change to go through several iterations. I'm submitting this
patch first so there is more time to test statprof. Perfect is the
enemy of good.
Now that the statprof module is vendored and suitable for use, we
switch our statprof profiler to use it. This required some minor
changes because of drift between the official statprof profiler
and the vendored copy.
We also incorporate Facebook's improvements from the "statprofext"
extension at
https://bitbucket.org/facebook/hg-experimental, notably support for
different display formats.
Because statprof output is different, this is marked as BC. Although
most users likely won't notice since most users don't profile.
Currently, running `hg serve --profile` doesn't yield anything useful:
when the process is terminated the profiling output displays results
from the main thread, which typically spends most of its time in
select.select(). Furthermore, it has no meaningful results from
mercurial.* modules because the threads serving HTTP requests don't
actually get profiled.
This patch teaches the hgweb wsgi applications to profile individual
requests. If profiling is enabled, the profiler kicks in after
HTTP/WSGI environment processing but before Mercurial's main request
processing.
The profile results are printed to the configured profiling output.
If running `hg serve` from a shell, they will be printed to stderr,
just before the HTTP request line is logged. If profiling to a file,
we only write a single profile to the file because the file is not
opened in append mode. We could add support for appending to files
in a future patch if someone wants it.
Per request profiling doesn't work with the statprof profiler because
internally that profiler collects samples from the thread that
*initially* requested profiling be enabled. I have plans to address
this by vendoring Facebook's customized statprof and then improving
it.
This allows specifying '--config profiling.output=blackbox' which will log the
profile output to the blackbox (if enabled). This is useful for doing profiling
on the server since it allows us to record the command, it's result, any
exceptions, and it's profile, all in one spot. And we get log rotation for
free.
The test-profile test would fail if the user had HGPROF set to another
profiler in their environment. This fix makes the test independent of
that environment variable.
Reverts the previous attempt to fix this, which was not cross platoform.
The test-profile test would fail if the user had HGPROF set to another
profiler in their environment. This fix makes the test independent of
that environment variable.
Many tests didn't change back from subdirectories at the end of the tests ...
and they don't have to. The missing 'cd ..' could always be added when another
test case is added to the test file.
This change do that tests (99.5%) consistently end up in $TESTDIR where they
started, thus making it simpler to extend them or move them around.
