Commit Graph

2898 Commits

Author SHA1 Message Date
Stanislau Hlebik
40a144610a inhibit: fix test
After upstream commit bab25235cfb5daf578a598bd7ae08f7a70ef7acd strip now also
deletes obsmarkers that are related to the stripped commits and saves them in
the bundle.
2017-06-05 02:37:21 -07:00
Stanislau Hlebik
0aa84957c7 copytrace: remove useless and add useful logging
Summary:
Previously we logged every time there is a changed/deleted file conflict.
This generates lots of useless log entries. Most of these entries are valid
conflicts where one file was deleted and another was added. Besides analyzing
these log entries is manual and very time consuming. Let's instead log cases
where copytracing actually helped.

Test Plan:
arc unit
Run with hg-crew test suite with --extra-config-opt

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5175656

Tasks: 18508761

Signature: t1:5175656:1496651487:fe710731213b21aefd1b957d259dc93470c9545c
2017-06-05 02:17:54 -07:00
Stanislau Hlebik
318a8cd988 conflictinfo: fix tests
There was a bug in require-ext.sh and because of it test-merge-conflictinfo.t
was skipped. This diff fixes the test failure.
2017-06-05 02:16:28 -07:00
Jun Wu
2e143f9478 require-ext: fix the script
It should use `$modname` instead of `$1`.
2017-06-02 12:59:00 -07:00
Stanislau Hlebik
ce077253b5 copytracing: trace copies in draft commits
Summary:
Simple copy tracking for draft commits. It doesn't use db to save move info
because getting move info is quite fast already.

Depends on D5137372

Test Plan: Run tests

Reviewers: #mercurial, durham, quark, rmcelroy

Reviewed By: quark

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5137886

Tasks: 18508761

Signature: t1:5137886:1496348869:8f4a9761fdeb26ddee5e18a318cc85582ef4adbb
2017-06-02 01:27:13 -07:00
Jun Wu
767de0233b annotate: use keyword arguments for skiprevs and diffopts
Summary: This will be safer if upstream API ever changed again.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5147881

Signature: t1:5147881:1496326141:c70559ffe2bb43953a1a0584163ca24c3235c5f3
2017-06-01 08:58:00 -07:00
Jun Wu
3c782f672b README: remove suggested smartlog template
Summary:
This unbreaks our bitbucket homepage.

See https://bitbucket.org/site/master/issues/14338/layout-broken-when-readme-contains-a-block

README is generally outdated. We may want to build better documentation later.

Test Plan: eyes

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5156371

Signature: t1:5156371:1496325598:2e94dd711693a2ce2475f20875424b36ded1484b
2017-06-01 08:55:06 -07:00
Stanislau Hlebik
9e8ca3ae43 copytrace: use filename heuristics to quickly find moves
Summary:
Copytracing that is based on a simple idea: most moves are either directory
moves or moves of the file inside the same directory. That means that either
basename of the moved file or the dirname of the moved file is the same.
More details in the comments.

Test Plan: Run unit-tests

Reviewers: #mercurial, durham, quark, rmcelroy

Reviewed By: quark, rmcelroy

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5137372

Tasks: 18508761

Signature: t1:5137372:1496243148:8d229c1593da196b674318ee8b37af15a60831c8
2017-06-01 04:39:27 -07:00
Adam Simpkins
1b9a5fdc5c tweakdefaults: fix hg pull --rebase with a --tool argument
Summary:
Fix the tweakdefaults code to extract the --tool argument when extracting
--rebase, and to pass it in to the rebasemodule.rebase() function correctly.

Test Plan: Included a new test.

Reviewers: #mercurial, rmcelroy, quark

Reviewed By: quark

Subscribers: quark, net-systems-diffs@fb.com, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5134466

Tasks: 18770938

Signature: t1:5134466:1495848690:308a8d3fbad39e6ecfeaee04b1b3267c1fa1118c
2017-05-31 10:58:49 -07:00
Jun Wu
ff3d5e9c4a codemod: use policy.importmod
Summary:
Do not `from mercurial.cext import ...`. That breaks pure.
Besides, add `from __future__ import absolute_import` for files I touched.

Test Plan: arc unit

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5156067

Signature: t1:5156067:1496252444:eb620a56c5bd852017ce2b2d41a12631d9d0866f
2017-05-31 10:46:16 -07:00
Stanislau Hlebik
72641e2afc copytrace: remove useless sorting
Summary: It was added by mistake, let's remove it.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Differential Revision: https://phabricator.intern.facebook.com/D5155301

Tasks: 18508761

Signature: t1:5155301:1496246623:a650cbffa7edf4311f96071e57c61697dae8014e
2017-05-31 09:04:59 -07:00
Stanislau Hlebik
699e45aef6 copytrace: better copytracing logging
Summary:
1) Log commit hashes. Since we have infinitepush we may see the actual commits that caused the problems.
2) Log reponame
3) Wrap logging code in try/catch so that any errors in this code won't affect clients

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5146976

Tasks: 18508761

Signature: t1:5146976:1496241350:1d8a921527a0e2d984f6a46452697428df1757be
2017-05-31 07:51:29 -07:00
Kostia Balytskyi
44cd7a5717 smartlog: use dbm implementation, available on the system
Summary:
`gdbm` is unavailable on Windows. Python provides module called `anydbm` which
abstracts the implementation of `dbm` and is even capable of using a dump
Python-only implementation if nothing is installed.

Test Plan: - rt

Reviewers: #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5154620
2017-05-31 04:11:53 -07:00
Stanislau Hlebik
355573a423 remotefilectx: do not fail _adjustlinknode if prefetch fails
Summary:
It has bitten us a few times already. Failing `_adjustlinknode()` only because
we couldn't download file from the server seems incorrect. There may be a
transient network issue for example.
Let's wrap this code in try/catch and warn in the case of failure.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5147669

Signature: t1:5147669:1496225752:3cd6d9758e447b7f91b246a9ed1a20e6f73ed176
2017-05-31 03:24:46 -07:00
Phil Cohen
45b775ae72 pushrebase: exchange -> bundle2
Summary: Broken by an upstream refactor

Test Plan: Fixes test

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5149611
2017-05-30 13:13:36 -07:00
Jun Wu
01a471b159 codemod: use vendored directaccess and inhibit extension in tests
Summary:
This diff makes the tests use the vendored directaccess and inhibit instead the
foreign ones.

Test Plan: arc unit

Reviewers: #mercurial, sid0

Reviewed By: sid0

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5142981

Signature: t1:5142981:1496173851:3a8f3a1c689711dbef517c893bf72c90ab144761
2017-05-30 12:56:40 -07:00
Jun Wu
cc95aa58ff test-inhibit: fix the test
Summary: Fix lint errors and make the test pass.

Test Plan: arc unit

Reviewers: #sourcecontrol, sid0

Reviewed By: sid0

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5142477

Signature: t1:5142477:1496173824:bf941d895d3259fc788c09d4fefca7dddd11a33b
2017-05-30 12:56:20 -07:00
Jun Wu
e3e36e37ec inhibit: vendor inhibit and directaccess from mutable-history
Summary:
Vendored using GPL2 license from https://bitbucket.org/marmoute/mutable-history
changeset c9f1118b33d60f8faa4b89988a8155c544f5bb0d without modification.

Test Plan: N/A. The code was imported as is.

Reviewers: #sourcecontrol, sid0

Reviewed By: sid0

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5142063

Signature: t1:5142063:1496173803:d0a9fa50a8423a531432ac3cff378b50952684b1
2017-05-30 12:55:33 -07:00
Jun Wu
78c76613b9 annotate: add skiprevs parameter
Summary:
This adds the `skiprevs` parameter to all `filectx.annotate` methods
to match the upstream change.

For fastannotate, things is a bit more complex since it's a mix of two
algorithms. For now we just fallback to the slow path for correctness.

I'll think about adding back a fast path later.

Test Plan: arc unit

Reviewers: sid0, #mercurial, quark

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5128392
2017-05-30 09:29:33 -07:00
Stanislau Hlebik
23e41d98e5 copytrace: move to hgext3rd
Summary:
copytracing extension is going to be simpler than I expected in the beginning.
There is no need to separate folder, let's move it to hgext3rd

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5146585

Tasks: 18508761

Signature: t1:5146585:1496147733:ad6ad14f663a8b9bf4b687f0767395321f9840b7
2017-05-30 06:39:31 -07:00
Stanislau Hlebik
bbe014cb96 perftweaks: remove newhiddencache
Summary: Upstream dropped hidden cache in 2f38ac4af04e70af3ffce4700c5a15252391e391, let's drop newhiddencache too

Test Plan: arc unit

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5146661

Signature: t1:5146661:1496140799:a5a4ddeac577f58a7e1998a5f9fb2eac305cc9d2
2017-05-30 03:42:56 -07:00
Stanislau Hlebik
4136d61d32 infinitepush: reduce pushbackup memory usage
Summary:
Previously `_getrevstobackup()` would load manifest for each commit in memory
only to check if a file was deleted in this commit or not. Manifest can be
quite big and since every loaded flatmanifest is cached in fastmanifest then
memory usage can be huge.
Since we are doing this only to check if there are any commits that were
downloaded to the client without filelogs and then were stripped server-side.
This cases are rare and it would be easier to track these commits in the
config file.

Test Plan: arc unit

Reviewers: #mercurial, mjpieters, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5111372

Tasks: 15389402

Signature: t1:5111372:1495848867:4028bd48313ac0e2022c3695195bedc6eda80abf
2017-05-29 12:54:26 -07:00
Jun Wu
3ed8d7b512 smartlog: add a simple ancestor cache
Summary:
`revlog.ancestor` is expensive on long changelog, but is only called a few
(about 30) times for smartlog usage. Therefore we could simply cache the
result in a key-value database.

This speeds up smartlog by about 200ms.

Test Plan: Added a new test

Reviewers: #mercurial, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5135746

Signature: t1:5135746:1495783503:411260551fd29fda42c9fc809b56f8f77fb2eaf4
2017-05-26 09:03:58 -07:00
Stanislau Hlebik
78dc07eef8 fix tests after upstream change
b2eb5bb9709893ebd015b97c06f4242b3b4c7b48 in upstream broke our tests.
This commit fixes it
2017-05-25 09:55:51 -07:00
Durham Goode
b8b200a148 tests: add globs to sqldirstate times
Glob out the times so tests aren't flakey.
2017-05-24 12:53:33 -07:00
Durham Goode
5caecec1ee sparse: switch to using repr for hash computation
Upstream is refactoring the matcher api (952017471f9) and recommends using repr
for hash computation now. This is how fsmonitor in core now does it.
2017-05-24 09:24:19 -07:00
Durham Goode
0d916abbed perftweaks: rename _getdynamicblocks to revealedrevs
Upstream has renamed this function.
2017-05-24 09:18:45 -07:00
Durham Goode
1d5a3cb03f tests: fix fastannotate tests 2017-05-23 15:38:09 -07:00
Durham Goode
56be761d2e chistedit: fix lint error 2017-05-23 15:38:09 -07:00
Durham Goode
6e542135ee imports: update c extension imports
Upstream has moved various c extension into mercurial.cext
2017-05-23 15:26:08 -07:00
Jun Wu
a72478f016 codemod: better #includes
Summary:
This patch removes all `#include "../` lines and use the shortest possible
include path.

Test Plan: `make clean build`

Reviewers: durham, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5113672

Signature: t1:5113672:1495565454:961fb6f2f57a81a95013e0b8f67b2917c2e4523e
2017-05-23 11:57:32 -07:00
Jun Wu
6f5f9b9c0d convert: unify two sources and clean it up
Summary:
There are two `convert.h`. This patch unifies them and does cleanups so it's
a valid header file which could be included by multiple .c and .cpp files
and linker won't complain re-definition (by adding `static` to everything).

Besides, reformat the code so it could pass check-code. Also fix a compiler
warning about comparing an unsigned integer with a signed integer.

Test Plan: `make clean local`. It still builds.

Reviewers: durham, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5113550

Signature: t1:5113550:1495565413:e399f898ac513e64af37dab5daf55cedbabfc703
2017-05-23 11:57:12 -07:00
Jun Wu
3b9d8cf876 traceprof: store raw Python code object
Summary:
Previously, the tracer extracts file name, function name from Python code
objects and stores them as std::string. And frame the de-duplication logic
hashes those strings, which is actually quite expensive.

`hg id` takes 1.0 seconds without frame de-duplication, and 2.5 seconds with
de-duplication.

This diff assumes Python code object is unique (i.e. Python won't generate two
code objects for a single code segment). That seems reasonable and `lsprof`
seems to make a same assumption.

With that assumption, just store `PyCodeObject` (contains file, method name and
line number) instead of `std::string` (for file, method names) and line numbers
for code identity and use the raw address of `PyCodeObject` to do frame
de-duplication.

That is helpful during profiling and de-duplication:

  - During profiling, no need to copy strings (convert to std::string),
    therefore less overhead
  - During frame de-duplication, no need to read long strings and hash them,
    therefore much faster

Debug flag during compilation is added to make debugging easier as the code
starts to mix of C++ and Python objects, which means mistakes could be made
more easily.

Test Plan:
With this patch, `hg id` with frame de-duplication takes 1.0 seconds.
There is no visible overhead doing de-duplication. Cheers!

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5112636

Signature: t1:5112636:1495562232:dd0724198ebb7f2eb4e9fd3a83517fc9160fa01e
2017-05-23 11:51:10 -07:00
Jun Wu
85be37794d traceprof: implement frame deduplication
Summary:
Previously, traceprof hashes frames using their raw memory addresses, which
causes a same frame (same traceback) being recorded multiple times because
of memory address change. That's for low overhead because hashing all the
way back to the top frame is slow.

This patch implements slow frame hashing and de-duplication so the output
could look tidier in some cases. The slow de-duplication runs after
profiling ends so it won't affect profiling accuracy.

Test Plan:
Trace IPython, before:

```
  6211          \ run                           posix.py:50
  6152            \ call_inputhook (10 times)   inputhook.py:60
                    ....
  6147              \ select_fds (10 times)     select.py:194
     6                \ select                  select.py:85
     6                 | select                 select.py:179
   501                \ select                  select.py:85
*  501                 | select                 select.py:179
  3775                \ select                  select.py:85
* 3775                 | select                 select.py:179
*  627                \ select (2 times)        select.py:85
   736                \ select (3 times)        select.py:85
*  736                 | select (6 times)       select.py:179
   501                \ select                  select.py:85
*  501                 | select (2 times)       select.py:179
```

After:

```
  4397          \ run                           posix.py:50
  4350            \ call_inputhook (4 times)    inputhook.py:60
  4349             | select_fds (4 times)       select.py:194
  4348             | select (4 times)           select.py:85
* 4348             | select (8 times)           select.py:179
```

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5112561

Signature: t1:5112561:1495562165:bf86c3283156d3b67918706dc875e6d3e28eac18
2017-05-23 11:50:33 -07:00
Jun Wu
c64d830714 traceprof: add a standalone script to trace other scripts
Summary:
Like other profilers in Mercurial, provide a standalone script to measure
performance of other scripts directly.

Test Plan: Run the new script to profile ipython and it works.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5112547

Signature: t1:5112547:1495562125:0d15b48b3aa9e6ead43fbb8166dabf13b6273ef2
2017-05-23 11:49:20 -07:00
Jun Wu
a63cbaaed1 portibility: move to clib
Summary:
It's a small C utility. It should belong to `clib` directory.

Also, `#include "../foo"` does not seem to be a good pattern. It makes
include files harder to follow and make code movement more difficult. Since
`clib` is already included in `-I` during compilation, remove `../`.

Test Plan: `make clean local`. It still builds.

Reviewers: ikostia, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters, vsutaria

Differential Revision: https://phabricator.intern.facebook.com/D5113236

Signature: t1:5113236:1495562436:e8d64083ab0417c67b63223a092470739f4c1176
2017-05-23 11:47:40 -07:00
Jun Wu
c41d432bb1 fastannotate: move to hgext3rd
Summary: Extensions should belong there.

Test Plan: Updated existing tests

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5113146

Signature: t1:5113146:1495562344:47bae69e6b22c14b32a8a9511878e88e89759d8f
2017-05-23 11:47:22 -07:00
Jun Wu
a97e026674 remoteid: move to hgext3rd
Summary: This is what extensions should belong to.

Test Plan: Changed existing test.

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5113042

Signature: t1:5113042:1495561987:d1af0f7cebfdb8b0c1a6bb71cdc826d817a4e441
2017-05-23 11:47:04 -07:00
Jun Wu
7c24b5d437 cmake: remove CMakeLists.txt
Summary:
We now use setup.py to do the build so this is outdated. It does not seem to be
useful so just remove it.

Test Plan: eyes

Reviewers: durham, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, ttung, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5112853

Signature: t1:5112853:1495561149:a4ee408e1fe06dea32f96ae93abb38d7d21d3619
2017-05-23 11:46:44 -07:00
Jun Wu
8235136ede statprof: remove statprof.py
Summary:
statprof is now part of core hg. And core hg will import its own version
instead of the external one. So it's no longer to have a separate version.

I did a comparison. The version in fb-hgext is falling behind - it lacks of
Chrome support. The upstream version is also cleaner in terms of Python3 code
standard and error handling.

Test Plan: eyes

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5112666

Signature: t1:5112666:1495562274:7e1ca7d964bf617695dbefd59c3843a3ebf47f9e
2017-05-23 11:46:25 -07:00
Stanislau Hlebik
2a0abb9dd0 copytrace: add monitoring
Summary:
Let's log whenever one merge parent was deleted and another was modified.
It will help us find out how much disabled copytracing affects users.

Test Plan: test-copytrace-errormsg.t

Reviewers: #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D5103311

Tasks: 18508761
2017-05-23 06:54:52 -07:00
Stanislau Hlebik
2adea9a0aa copytrace: change error msg
Summary:
First step to get better copytracing. For now let's just change merge error
message to suggest re-running command with enabled copytracing.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5102174

Tasks: 18508761
2017-05-23 06:54:26 -07:00
Martin von Zweigbergk
050cb5d1db chistedit: fail only when attempting to run if curses is not found
If curses is not available, we can still be nice and not fail until
the user actually tries to run the chistedit command.
2017-05-22 23:00:54 -07:00
Ben Godfrey
668afbd5d2 Add remoteid extension (prints FQDN of remote host during remote session)
Summary:
Wrap the capabilities wire protocol command to print the hostname to stderr.

This commit contains the implementation of the extension but not code to register it on a particular repo.

Test Plan: `rt`

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: durham, wez, stash, simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5087634

Tasks: 16758829

Signature: t1:5087634:1495498748:213a32c5c474145371bbbc43b0463dee31868a9a
2017-05-22 17:20:16 -07:00
Jun Wu
f1be725a71 traceprof: new extension to perform accurate profiling
Summary:
Instead of sampling periodically like the default profiler `statprof`, trace
every function call so the report is accurate. The output is designed to be
similar to `statprof`.

Written in C++ and inline assembly to minimize overhead. Cython is used as a
glue layer.

Comparing to `lsprof`, the overhead is similarly small. The major difference is
this profiler cares about "frame object" and records `frame->f_back` (parent
frame) so it could draw a statprof-like tree of the callgraph. `lsprof` records
"code object" instead and could have difficulty to deal with recursive calls.

The following code could demostrate the difference:

```
def fib(x):
    if x < 2:
        return x
    return fib(x - 1) + fib(x - 2)

# lsprof - recorded 1 code object for fib and no way to figure out callgraph
import _lsprof
p = _lsprof.Profiler()
p.enable(subcalls=True)
fib(5)
p.disable()
for s in p.getstats():
    print(s)

'''output:
_lsprof.profiler_entry(code="<method 'disable' of '_lsprof.Profiler' objects>", callcount=1, reccallcount=0, totaltime=0.0, inlinetime=0.0, calls=None)
_lsprof.profiler_entry(code=<code object fib at 0x7f27ca587530, file "a.py", line 1>, callcount=15, reccallcount=14, totaltime=8e-06, inlinetime=8e-06, calls=[_lsprof.profiler_subentry(code=<code object fib at 0x7f27ca587530, file "a.py", line 1>, callcount=14, reccallcount=12, totaltime=6e-06, inlinetime=6e-06)])
'''

# traceprof (with timethreshold set to 0) - callgraph drawn correctly
import sys
from hgext3rd import traceprof
with traceprof.profile(None, sys.stderr):
    fib(5)

'''output:
    | <module>                                  a.py:1
  0  \ fib                                      a.py:1
  0    \ fib                                    a.py:1
  0      \ fib                                  a.py:1
  0        \ fib                                a.py:1
  0         | fib                               a.py:1
  0        \ fib                                a.py:1
  0      \ fib                                  a.py:1
  0       | fib                                 a.py:1
  0    \ fib                                    a.py:1
  0     | fib                                   a.py:1
     \ __exit__                                 contextlib.py:21
'''
```

There are some performance / correctness tradeoffs. The current implementation
takes `frame` address and several (but not theoretically complete) fields to
hash a frame.  That could in theory lead to inaccurate frame information being
recorded. But hashing all fields recursively incurs significant overhead. The
related logic is the `hashframe` function, which could be tweaked later.

This is an initial version. It probably does not cover all corner cases like
exception handling well. And it may duplicate a same frame multiple times
because of the way it hashes a frame. The duplicated frame issue could be
solved by adding post-profiling filtering though.

Here is a real-world example of `hg log -r .` on `hg-committed`:

```
      | <module>                                hg:10
      | run                                     dispatch.py:81
      | dispatch                                dispatch.py:132
      | _runcatch                               dispatch.py:202
      | _callcatch                              dispatch.py:309
      | callcatch                               scmutil.py:133
      | _runcatchfunc                           dispatch.py:215
      | _dispatch                               dispatch.py:744
    2  \ repository                             hg.py:166
    2   | _peerorrepo                           hg.py:151
    2   | instance                              localrepo.py:2025
    2   | __init__                              localrepo.py:275
  863  \ runcommand                             dispatch.py:656
  863   | _runcommand                           dispatch.py:926
  863   | <lambda>                              dispatch.py:918
  863   | check                                 util.py:1059
  863   | log                                   commands.py:3262
  480    \ getlogrevs                           cmdutil.py:2236
  479     | _logrevs                            cmdutil.py:2190
  479     | revrange                            scmutil.py:429
  479     | anyrevs                             localrepo.py:622
  478     | mfunc                               revset.py:2275
  478     | __init__ (4 times)                  smartset.py:1033
  478     | __init__ (4 times)                  smartset.py:919
  478     | __len__ (4 times)                   localrepo.py:587
  478     | changelog                           repoview.py:309
  477     | filterrevs (11 times)               repoview.py:260
  477     | computehidden                       repoview.py:171
  455      \ hideablerevs                       repoview.py:24
  455       | getrevs                           obsolete.py:1114
  455       | _computeobsoleteset               obsolete.py:1143
    3        \ __get__ (2 times)                localrepo.py:76
    7        \ node (5760 times)                changelog.py:359
    4         | node (8938 times)               revlog.py:482
    9        \ __get__ (5760 times)             localrepo.py:76
  428        \ __get__                          util.py:798
  428         | successors                      obsolete.py:672
  225          \ __get__                        util.py:798
  224           | _all                          obsolete.py:662
    6            \ tryread                      vfs.py:32
    6             | read                        vfs.py:60
  200            \ _readmarkers                 obsolete.py:442
* 199             | _fm1readmarkers             obsolete.py:430
   15            \ _checkinvalidmarkers         obsolete.py:523
* 203          \ _addsuccessors                 obsolete.py:504
    3      \ tryreadcache                       repoview.py:166
    3       | cachehash                         repoview.py:98
    3       | heads                             localrepo.py:1884
   17      \ <genexpr> (5225 times)             repoview.py:191
   15       | __contains__ (5226 times)         ancestor.py:334
    5       | parentrevs (4010 times)           changelog.py:371
    2       | parentrevs (4332 times)           revlog.py:479
    9    \ pager                                ui.py:843
    9     | _runpager                           ui.py:906
    8     | __init__                            subprocess.py:330
    8     | _execute_child                      subprocess.py:880
    5     | _eintr_retry_call                   subprocess.py:118
  374    \ show                                 cmdutil.py:1308
  374     | _show                               cmdutil.py:1316
  365      \ _changesetlabels                   cmdutil.py:1266
  365       | troubled                          context.py:225
   12        \ unstable (2 times)               context.py:207
   12         | getrevs                         obsolete.py:1114
   12         | _computeunstableset             obsolete.py:1154
    3          \ set (322 times)                localrepo.py:610
    3            \ revs (3 times)               localrepo.py:593
    8            \ __getitem__ (963 times)      localrepo.py:566
    4          \ parents (321 times)            context.py:246
    4           | __get__ (321 times)           util.py:798
    4           | _parents (321 times)          context.py:562
    3          \ <genexpr> (642 times)          obsolete.py:1164
    3           | obsolete (321 times)          context.py:199
    2           | getrevs (321 times)           obsolete.py:1114
  319        \ bumped (2 times)                 context.py:211
  319         | getrevs                         obsolete.py:1114
  319         | _computebumpedset               obsolete.py:1181
    4          \ set (322 times)                localrepo.py:610
  312          \ allprecursors (2021 times)     obsolete.py:850
  308           | __get__                       util.py:798
  308           | precursors                    obsolete.py:678
* 308           | _addprecursors                obsolete.py:509
   34        \ divergent (2 times)              context.py:218
   34         | getrevs                         obsolete.py:1114
   33         | _computedivergentset            obsolete.py:1204
    4          \ set (322 times)                localrepo.py:610
   24          \ successorssets (1368 times)    obsolete.py:899
    3            \ __get__ (1368 times)         localrepo.py:76
    4            \ __contains__ (200 times)     localrepo.py:575
    9      \ names (2 times)                    namespaces.py:184
    9       | <lambda>                          namespaces.py:43
    9       | nodetags                          localrepo.py:741
    8       | __get__                           util.py:798
    8       | _tagscache                        localrepo.py:646
    8       | _findtags                         localrepo.py:685
    7       | findglobaltags                    tags.py:170
    7       | _readtagcache                     tags.py:370
    6       | filteredhash                      scmutil.py:307
       \ __exit__                               contextlib.py:21
        | maybeprofile                          profiling.py:199
        | __exit__                              contextlib.py:21
        | profile                               profiling.py:148
        | __exit__                              contextlib.py:21
Total time: 867 ms
```

Some example conclusions from reading the above output include:

  1. `_fm1readmarkers`, `_addsuccessors`, `_addprecursors` are slow
  2. `changelog.node` and `changelog.parentrevs` are called thousands of times,
     but they are just a few milliseconds.
  3. Reading the `obsstore` file (`vfs.read`) takes 5ms exactly

While `statprof` may get 1 right, `lsprof` may get 2 right, neither of them is
helpful for 3. `statprof` is not accurate enough to be confident about 5ms,
`lsprof` will include other `vfs.read` calls so it's impossible to see just the
read for obsstore.


Test Plan:
With the upstream [patch](https://patchwork.mercurial-scm.org/patch/20822/), run:

```
 HGPROF=traceprof hg id --config extensions.traceprof= --profile
```

Reviewers: #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters, jiangyuan

Differential Revision: https://phabricator.intern.facebook.com/D5104851

Signature: t1:5104851:1495480386:666847ea08bb6a94658bc10d7f0e91c225d56836
2017-05-22 15:49:45 -07:00
Durham Goode
dcaee39e39 obsmarkers: update obsmarker operation wrapping to work with upstream
Summary:
Upstream has added an optional operation argument to createmarkers, which we
need to also accept. This argument is currently only processed if a config
option is set, so for now we will still put the operation in the metadata dict
manually.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5106511

Signature: t1:5106511:1495485044:23d45d9ba389cf0d49b31705ae36902310c5056c
2017-05-22 13:38:37 -07:00
Durham Goode
0766a3469c dirstate: switch to new parentchange context
Summary: Upstream has deprecated begin/endparentchange in favor of a context object.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5106500

Signature: t1:5106500:1495485058:8377335a0f678de7bb50cb93fff29bcfeb53fe1d
2017-05-22 13:38:37 -07:00
Durham Goode
e34660b057 commands: update to use registrar instead of cmdutil
Summary: Upstream has deprecated cmdutil.commands() in favor of registrar.commands()

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5106486

Signature: t1:5106486:1495485074:0e20f00622cc651e8c9dda837f84dd84cc51099e
2017-05-22 13:38:37 -07:00
Jun Wu
f532dddaa4 lfs: implement byte-level progress bar
Summary:
Previously, the progress bar is file-level - it moves when a file is transferred.

When uploading or downloading a single giant file, progress bar matters. And
this diff adds it to make people more patient.

Test Plan:
Manually upload and download several big files.
Make sure the progress bar appears in both cases.

Also make sure the output is sane with `-v`.

Reviewers: #mercurial, davidsp, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5100014

Signature: t1:5100014:1495316719:c500ccd63d3a495e41575cadd72419a41f5fb259
2017-05-22 11:03:14 -07:00
Jun Wu
693b0fa76a remotefilelog: remove filectx.cmp override
Summary:
The `cmp` logic does not look special from the original `basefilectx.cmp` so it
could be removed. This means we will benefit from LFS's override of
`basefilectx.cmp` and have a fast path for LFS binary diff.

Test Plan: Added a test. Make sure the old code fails that test.

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5100529

Signature: t1:5100529:1495468055:ff7fa30f3277b942bf49ba1e9b8764effacd972a
2017-05-22 10:59:53 -07:00