Summary:
Added the following flags to statprof.py:
- `-f --file` Location of the .data file from the profiler.
- `-s --script-path` Location of the flamegraph.pl script.
- `-o --output-file` Path to write the flamegraph SVG output.
Test Plan: Ran the statprof.py command with various permutations of the aforementioned flags. Running without these flags produces the same behavior as before.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4034242
Tasks: 13961909
Signature: t1:4034242:1476755279:23d2f3ea2850626ea9afbf60e712bc2b93c2fec0
Summary:
One place we are losing time in the profiling extension is when
our profiling thread is interrupted between two invocations of clock()
that are supposed to fetch the same time. Fix to use now = clock() and
use the same time in both places.
Test Plan:
Running with upstream and custom extensions, I no longer observe
suspended execution times large than accumulated execution times.
Reviewers: ttung, durham, mitrandir, #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3772529
Tasks: 12879683
Signature: t1:3772529:1472151936:192ada10ac4738755c8fd27278dc7ac90faa17be
Summary:
Since statprof is not an extension, it needs to be treated like a normal python
module.
Test Plan:
```
~/local/fb-hgext> python setup.py build
~/local/fb-hgext> ls build/lib.linux-x86_64-2.6/
cfastmanifest.so hgext3rd/ sqldirstate/
fastmanifest/ phabricator/ statprof.py
```
Reviewers: ttung, #sourcecontrol, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3607202
Signature: t1:3607202:1469216683:9aa08aee567425c239ddc5d1bb68fd892a8cf6a9
Summary:
Be a better citizen under system python path.
Fix all tests issues and change setup.py to use glob pattern to include
all extensions.
Test Plan:
Run tests and `make local`.
Also build and install the package and run `hg sl` in major repos.
Reviewers: #mercurial, ttung, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3534311
Signature: t1:3534311:1468275426:fe122646c8bd6c541e1889e73e9df28f86747ff2
Summary:
The existing statprof uses signals to perform stack sampling. Those signals can
interfere with system calls though. This patch introduces a way of using python
threads to perform sampling from a background thread.
This also changes the default to be the threaded profiler instead of the
sampling one, since the sampling can crash the proces sometimes.
In some situations this produces a more accurate stack trace.
Test Plan:
```
sudo cp statprof.py /usr/lib64/python2.6/site-packages/statprof.py
hg pull --profile
# Verified I got output and that it was different from the old statprof (it was
# more accurate, as compared with time.time() measurements in the code)
```
Reviewers: #mercurial, ttung, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3402864
Signature: t1:3402864:1465978363:c8455d119cc03c2c475e190aee28587a9d434c4d
Summary:
This adds a json display format that can dump the raw samples to json. This
allows other processes to read the data and log/analyze them.
Test Plan: hg.real log -r tip --config extensions.statprofext=../fb-hgext/statprofext.py --profile --config statprof.format='json' --config profiling.output=woooo.log
Reviewers: #mercurial, ttung, lcharignon
Reviewed By: lcharignon
Subscribers: lcharignon, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3408369
Signature: t1:3408369:1465436261:5fc344effdea46d2cf07c81cb707791ceea98f11
Summary:
Mercurial stack traces often have a bunch of boiler plate function wrappers in
the stack. This change gives statprof a list of frames it can skip. As a side
benefit, this also means we can get rid of the logic that cuts off the top 2% of
the stack trace. It was meant to prevent the dispatch.py functions from showing
up in the stack, but that's now covered by the skip list.
Test Plan:
```
sudo cp statprof.py /usr/lib64/python2.6/site-packages/statprof.py
hg pull --profile
| 100.0% __init__.py: runcommand line 45: mercurial.dispatch.run()
| 100.0% cachemanager.py: runcommandtrigger line 325: return orig(lui, repo, *arg...
| 100.0% reflog.py: runcommand line 242: result = orig(*args, **kwargs)
| 100.0% dispatch.py: statprofile line 31: return orig(lui, repo, cmd,...
| 99.7% __init__.py: wrappull line 959: return func()
| 99.7% remotenames.py: expullcmd line 119: return orig(ui, repo, *args...
...
```
Reviewers: ttung, #mercurial, lcharignon
Reviewed By: lcharignon
Subscribers: lcharignon, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3402841
Signature: t1:3402841:1465407465:b0060496cd148eb6180cdf732bb7b8e6d3cd4db6
Summary: Final diff of the series
Test Plan:
All tests pass with --extra-config=extensions.lz4revlog= needed
to run check-code
Reviewers: #sourcecontrol, ttung
Differential Revision: https://phabricator.fb.com/D2812684
Summary: This diff is part of a series to cleanup fb-hgext and make it pass check-code.
Test Plan: all tests pass
Reviewers: #sourcecontrol, ttung
Differential Revision: https://phabricator.fb.com/D2811954
Summary: This diff is part of a series to cleanup fb-hgext and make it pass check-code.
Test Plan: all tests pass
Reviewers: #sourcecontrol, ttung
Differential Revision: https://phabricator.fb.com/D2811926
Summary: This diff is part of a series to cleanup fb-hgext and make it pass check-code.
Test Plan: all tests pass
Reviewers: #sourcecontrol, ttung
Differential Revision: https://phabricator.fb.com/D2811871
Summary:
This allows us to target the statprof data at a file of the
wrapper's choosing.
Test Plan: HGPROF=stat STATPROF_DEST=/tmp/statprof.data PYTHONPATH=/Users/tonytung/work/mercurial/facebook-hg-rpms/fb-hgext hg.real --profile status
Reviewers: durham, ericsumner, rmcelroy
Reviewed By: rmcelroy
Subscribers: mitrandir
Differential Revision: https://phabricator.fb.com/D2715997
Signature: t1:2715997:1449105707:c07c83c0889c33cc693836917100d3007d0b631b
statprof attempts to save data to the user's home dir, for future investigation.
If this fails (because of permission issues, or because the home dir doesn't
exist), it was failing the command entirely. Let's catch and eat those
exceptions.
