sapling/hgext3rd
Zach Amsden f1ecf43a83 Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior.  To do this, we backtrackfrom the current head to find
 all draft revisions along the path, then find the common public ancestor of
those.  Once we find that, we can begin paging in results from the scmquery
service.

Pretty much a working diff at this point.  Limits and boundary conditions
have not been fully tested.  Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that.  Could also be an issue with conduit.  Other than that,
things seem to work.

Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):

  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a

  real	0m1.895s
  user	0m0.003s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b

  real	0m1.308s
  user	0m0.005s
  sys	0m0.001s
  [zamsden@devbig192.prn1 ~/local/fbcode] diff a b
  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c

  real	0m7.320s
  user	0m0.004s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbcode] diff a c




Testing --user option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a

  real	0m2.765s
  user	0m0.002s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b

  real	0m23.247s
  user	0m0.005s
  sys	0m0.001s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing same output enabled / disabled for -X option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a

  real	0m1.292s
  user	0m0.002s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b

  real	0m2.697s
  user	0m0.002s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing -k option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a

  real	0m1.174s
  user	0m0.003s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b

  real	0m1.259s
  user	0m0.004s
  sys	0m0.002s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing -I option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a

  real	0m1.473s
  user	0m0.005s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b

  real	0m2.911s
  user	0m0.003s
  sys	0m0.002s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]




Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a

  real	0m2.892s
  user	0m0.003s
  sys	0m0.006s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b

  real	0m2.575s
  user	0m0.697s
  sys	0m0.077s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c

  real	0m7.339s
  user	0m0.003s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a c
  [zamsden@devbig192.prn1 ~/local/fbsource]

Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash

Reviewed By: stash

Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv

Differential Revision: https://phabricator.intern.facebook.com/D3634075

Tasks: 12341014

Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-12 15:52:50 -07:00
..
__init__.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
arcdiff.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
backups.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
bundle2hooks.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
catnotate.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
chistedit.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
commitextras.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
dirsync.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
errorredirect.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
extorder.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
extutil.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
fastlog.py Parallel callout to scmquery through conduit 2016-08-12 15:52:50 -07:00
fbamend.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
fbconduit.py Parallel callout to scmquery through conduit 2016-08-12 15:52:50 -07:00
fbhistedit.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
githelp.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
gitlookup.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
gitrevset.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
grepdiff.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
inhibitwarn.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
mergedriver.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
morestatus.py morestatus: rebase takes precedence over update 2016-08-10 22:19:30 +01:00
nointerrupt.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
ownercheck.py ownercheck: new extension to prevent operations on repos not owned 2016-07-26 13:10:15 +01:00
patchpython.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
perftweaks.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
phabdiff.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
phabstatus.py Add sync status and change the conduit call to differential.querydiffhashes 2016-08-05 11:33:36 -07:00
phrevset.py phrevset: do not depend on hgsubversion 2016-07-20 20:09:29 +01:00
pullcreatemarkers.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
pushrebase.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
pushvars.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
rage.py rage: filter "FM:" from blackbox.log 2016-08-05 19:17:45 +01:00
reporootlog.py [reporootlog] report the repo root to the scm wrappers 2016-08-02 11:59:52 -07:00
reset.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
sampling.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
show.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
simplecache.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
smartlog.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
sparse.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00
sshaskpass.py sshaskpass: make the prompt similar to systemd 2016-08-11 21:56:31 +01:00
statprofext.py statprofext: import statprof from standard python path 2016-08-01 12:00:18 -07:00
tweakdefaults.py tweakdefaults: make "bookmarks" use the unfiltered repo 2016-08-11 18:43:06 +01:00
upgradegeneraldelta.py hgext: move single file extensions to hgext3rd 2016-07-08 13:15:42 +01:00