Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
# fastlog.py
|
|
|
|
#
|
|
|
|
# An extension to query remote servers for logs using scmquery / fastlog
|
|
|
|
#
|
|
|
|
# Copyright 2016 Facebook, Inc.
|
|
|
|
"""
|
|
|
|
connect to scmquery servers for fast fetching of logs on files and directories.
|
|
|
|
|
|
|
|
Configure it by adding the following config options to your .hg/hgrc.
|
2016-08-17 01:55:48 +03:00
|
|
|
This relies on fbconduit being setup for the repo; this should already
|
|
|
|
be configured if supported by your repo.
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
[fastlog]
|
2016-08-17 01:55:48 +03:00
|
|
|
enabled=true
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
"""
|
|
|
|
|
|
|
|
from mercurial import (
|
2016-08-23 21:46:34 +03:00
|
|
|
changelog,
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
cmdutil,
|
|
|
|
error,
|
|
|
|
extensions,
|
|
|
|
node,
|
|
|
|
phases,
|
|
|
|
revset,
|
|
|
|
scmutil,
|
2017-02-17 20:01:50 +03:00
|
|
|
smartset,
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
util
|
|
|
|
)
|
|
|
|
from mercurial.i18n import _
|
|
|
|
from mercurial.node import nullrev
|
|
|
|
|
|
|
|
import heapq
|
|
|
|
from threading import Thread, Event
|
|
|
|
from collections import deque
|
|
|
|
|
|
|
|
conduit = None
|
|
|
|
|
|
|
|
FASTLOG_MAX = 500
|
|
|
|
FASTLOG_QUEUE_SIZE = 1000
|
|
|
|
FASTLOG_TIMEOUT = 20
|
|
|
|
|
|
|
|
USE_FASTLOG = False
|
|
|
|
|
|
|
|
def extsetup(ui):
|
|
|
|
global conduit
|
|
|
|
try:
|
|
|
|
conduit = extensions.find("fbconduit")
|
|
|
|
except KeyError:
|
|
|
|
from hgext3rd import fbconduit as conduit
|
|
|
|
except ImportError:
|
|
|
|
ui.warn(_('Unable to find fbconduit extension\n'))
|
|
|
|
return
|
|
|
|
if not util.safehasattr(conduit, 'conduit_config'):
|
|
|
|
ui.warn(_('Incompatible conduit module; disabling fastlog\n'))
|
|
|
|
return
|
|
|
|
if not conduit.conduit_config(ui):
|
|
|
|
ui.warn(_('No conduit host specified in config; disabling fastlog\n'))
|
|
|
|
return
|
|
|
|
|
|
|
|
extensions.wrapfunction(cmdutil, 'getlogrevs', getfastlogrevs)
|
|
|
|
|
|
|
|
def lazyparents(rev, public, parentfunc):
|
|
|
|
"""lazyparents(rev, public)
|
|
|
|
Lazily yield parents of rev in reverse order until all nodes
|
|
|
|
in public have been reached or all revs have been exhausted
|
|
|
|
|
|
|
|
10
|
|
|
|
| \
|
|
|
|
9 8
|
|
|
|
| | \
|
|
|
|
7 6 5
|
|
|
|
| | /
|
|
|
|
4 *3 First move, 4 -3
|
|
|
|
| /
|
|
|
|
2 *2 Second move, 4 -1
|
|
|
|
| *
|
|
|
|
1
|
|
|
|
|
|
|
|
For example:
|
|
|
|
>>> parents = { 10:[9, 8], 9:[7], 8:[6,5], 7:[4], 6:[3], 5:[3], 4:[2] }
|
|
|
|
>>> parents.update({ 3:[2], 2:[1], 1:[] })
|
|
|
|
>>> parentfunc = lambda k: parents[k]
|
|
|
|
>>> public = set([1])
|
|
|
|
>>> for p in lazyparents(10, public, parentfunc): print p,
|
|
|
|
10 9 8 7 6 5 4 3 2 1
|
|
|
|
>>> public = set([2,3])
|
|
|
|
>>> for p in lazyparents(10, public, parentfunc): print p,
|
|
|
|
10 9 8 7 6 5 4 3 2
|
|
|
|
>>> parents[4] = [3]
|
|
|
|
>>> public = set([3,4,5])
|
|
|
|
>>> for p in lazyparents(10, public, parentfunc): print p,
|
|
|
|
10 9 8 7 6 5 4 3
|
|
|
|
>>> parents[4] = [1]
|
|
|
|
>>> public = set([3,5,7])
|
|
|
|
>>> for p in lazyparents(10, public, parentfunc): print p,
|
|
|
|
10 9 8 7 6 5 4 3 2 1
|
|
|
|
"""
|
|
|
|
seen = set()
|
|
|
|
heap = [-rev]
|
|
|
|
|
|
|
|
while heap:
|
|
|
|
cur = -heapq.heappop(heap)
|
|
|
|
if cur not in seen:
|
|
|
|
seen.add(cur)
|
|
|
|
yield cur
|
|
|
|
|
|
|
|
published = cur in public
|
|
|
|
if published:
|
|
|
|
# Down to one public ancestor; end generation
|
|
|
|
if len(public) == 1:
|
|
|
|
return
|
|
|
|
public.remove(cur)
|
|
|
|
|
|
|
|
for p in parentfunc(cur):
|
|
|
|
if p != nullrev:
|
|
|
|
heapq.heappush(heap, -p)
|
|
|
|
if published:
|
|
|
|
public.add(p)
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def dirmatches(files, paths):
|
|
|
|
"""dirmatches(files, paths)
|
|
|
|
Return true if any files match directories in paths
|
|
|
|
Expects paths to be appended by '/'
|
|
|
|
|
|
|
|
>>> dirmatches(['holy/grail'], ['holy/'])
|
|
|
|
True
|
|
|
|
>>> dirmatches(['holy/grail'], ['holly/'])
|
|
|
|
False
|
|
|
|
>>> try: dirmatches(['holy/grail'], ['holy'])
|
|
|
|
... except AssertionError, e: print('caught')
|
|
|
|
caught
|
|
|
|
"""
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
assert paths
|
2016-08-23 21:46:34 +03:00
|
|
|
for path in paths:
|
|
|
|
assert path[-1] == '/'
|
|
|
|
for f in files:
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
if f.startswith(path):
|
|
|
|
return True
|
|
|
|
return False
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def originator(parentfunc, rev):
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
"""originator(repo, rev)
|
|
|
|
Yield parents of rev from repo in reverse order
|
|
|
|
"""
|
|
|
|
# Use set(nullrev, rev) to iterate until termination
|
|
|
|
for p in lazyparents(rev, set([nullrev, rev]), parentfunc):
|
|
|
|
if rev != p:
|
|
|
|
yield p
|
|
|
|
|
|
|
|
def getfastlogrevs(orig, repo, pats, opts):
|
|
|
|
blacklist = ['all', 'branch', 'rev', 'sparse']
|
|
|
|
if any(opts.get(opt) for opt in blacklist) or not opts.get('follow'):
|
|
|
|
return orig(repo, pats, opts)
|
|
|
|
|
|
|
|
reponame = repo.ui.config('fbconduit', 'reponame')
|
|
|
|
if reponame and repo.ui.configbool('fastlog', 'enabled'):
|
|
|
|
wctx = repo[None]
|
|
|
|
match, pats = scmutil.matchandpats(wctx, pats, opts)
|
|
|
|
files = match.files()
|
[fastlog-hg] Get patterns working again
Summary:
When the restriction of only working on a single directory
was dropped, pattern support was neglected. That has to be done through
a revset, not the looser matches() filter.
Test Plan: P56588753
Reviewers: rmcelroy, quark, stash, durham
Reviewed By: durham
Subscribers: mhammell, meisner, calcsaransh, gqchen, dzhulgakov, kerl, bnitka, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3734232
Tasks: 12341014
Signature: t1:3734232:1471906628:d3995b358d683383fe590fa5848d8450a0ea05dd
2016-08-23 02:21:43 +03:00
|
|
|
if not files or '.' in files:
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
# Walking the whole repo - bail on fastlog
|
|
|
|
return orig(repo, pats, opts)
|
|
|
|
|
|
|
|
dirs = set()
|
|
|
|
wvfs = repo.wvfs
|
|
|
|
for path in files:
|
|
|
|
if wvfs.isdir(path) and not wvfs.islink(path):
|
|
|
|
dirs.update([path + '/'])
|
|
|
|
else:
|
|
|
|
# bail on symlinks, and also bail on files for now
|
|
|
|
# with follow behavior, for files, we are supposed
|
|
|
|
# to track copies / renames, but it isn't convenient
|
|
|
|
# to do this through scmquery
|
|
|
|
return orig(repo, pats, opts)
|
|
|
|
|
|
|
|
rev = repo['.'].rev()
|
|
|
|
|
|
|
|
parents = repo.changelog.parentrevs
|
|
|
|
public = set()
|
|
|
|
|
|
|
|
# Our criterion for invoking fastlog is finding a single
|
|
|
|
# common public ancestor from the current head. First we
|
|
|
|
# have to walk back through drafts to find all interesting
|
|
|
|
# public parents. Typically this will just be one, but if
|
|
|
|
# there are merged drafts, we may have multiple parents.
|
|
|
|
if repo[rev].phase() == phases.public:
|
|
|
|
public.add(rev)
|
|
|
|
else:
|
|
|
|
queue = deque()
|
|
|
|
queue.append(rev)
|
|
|
|
seen = set()
|
|
|
|
while queue:
|
|
|
|
cur = queue.popleft()
|
|
|
|
if cur not in seen:
|
|
|
|
seen.add(cur)
|
|
|
|
if repo[cur].mutable():
|
|
|
|
for p in parents(cur):
|
|
|
|
if p != nullrev:
|
|
|
|
queue.append(p)
|
|
|
|
else:
|
|
|
|
public.add(cur)
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def fastlog(repo, startrev, dirs, localmatch):
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
filefunc = repo.changelog.readfiles
|
2016-08-23 21:46:34 +03:00
|
|
|
for parent in lazyparents(startrev, public, parents):
|
|
|
|
files = filefunc(parent)
|
|
|
|
if dirmatches(files, dirs):
|
|
|
|
yield parent
|
|
|
|
repo.ui.debug('found common parent at %s\n' % repo[parent].hex())
|
|
|
|
for rev in combinator(repo, parent, dirs, localmatch):
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
yield rev
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def combinator(repo, rev, dirs, localmatch):
|
|
|
|
"""combinator(repo, rev, dirs, localmatch)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
Make parallel local and remote queries along ancestors of
|
|
|
|
rev along path and combine results, eliminating duplicates,
|
|
|
|
restricting results to those which match dirs
|
|
|
|
"""
|
|
|
|
LOCAL = 'L'
|
|
|
|
REMOTE = 'R'
|
|
|
|
queue = util.queue(FASTLOG_QUEUE_SIZE + 100)
|
|
|
|
hash = repo[rev].hex()
|
|
|
|
|
2016-08-25 19:04:43 +03:00
|
|
|
local = LocalIteratorThread(queue, LOCAL, rev,
|
2016-08-23 21:46:34 +03:00
|
|
|
dirs, localmatch, repo)
|
2016-08-23 21:46:34 +03:00
|
|
|
remote = FastLogThread(queue, REMOTE, reponame, 'hg', hash, dirs,
|
|
|
|
repo)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
# Allow debugging either remote or local path
|
|
|
|
debug = repo.ui.config('fastlog', 'debug')
|
|
|
|
if debug != 'local':
|
|
|
|
repo.ui.debug('starting fastlog at %s\n' % hash)
|
|
|
|
remote.start()
|
|
|
|
if debug != 'remote':
|
|
|
|
local.start()
|
|
|
|
seen = set([rev])
|
|
|
|
|
|
|
|
try:
|
|
|
|
while True:
|
|
|
|
try:
|
|
|
|
producer, success, msg = queue.get(True, 3600)
|
|
|
|
except util.empty:
|
|
|
|
raise error.Abort("Timeout reading log data")
|
|
|
|
if not success:
|
|
|
|
if producer == LOCAL:
|
|
|
|
raise error.Abort(msg)
|
|
|
|
elif msg:
|
|
|
|
repo.ui.warn(msg + '\n')
|
|
|
|
continue
|
|
|
|
|
|
|
|
if msg is None:
|
|
|
|
# Empty message means no more results
|
|
|
|
return
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
rev = msg
|
|
|
|
if debug:
|
|
|
|
if producer == LOCAL:
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
repo.ui.debug('LOCAL:: %s\n' % msg)
|
2016-08-23 21:46:34 +03:00
|
|
|
elif producer == REMOTE:
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
repo.ui.debug('REMOTE:: %s\n' % msg)
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
if rev not in seen:
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
seen.add(rev)
|
|
|
|
yield rev
|
|
|
|
finally:
|
|
|
|
local.stop()
|
|
|
|
remote.stop()
|
|
|
|
|
|
|
|
# Complex match - use a revset.
|
[fastlog-hg] Get patterns working again
Summary:
When the restriction of only working on a single directory
was dropped, pattern support was neglected. That has to be done through
a revset, not the looser matches() filter.
Test Plan: P56588753
Reviewers: rmcelroy, quark, stash, durham
Reviewed By: durham
Subscribers: mhammell, meisner, calcsaransh, gqchen, dzhulgakov, kerl, bnitka, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3734232
Tasks: 12341014
Signature: t1:3734232:1471906628:d3995b358d683383fe590fa5848d8450a0ea05dd
2016-08-23 02:21:43 +03:00
|
|
|
complex = ['date', 'exclude', 'include', 'keyword', 'no_merges',
|
|
|
|
'only_merges', 'prune', 'user']
|
2016-08-23 21:46:34 +03:00
|
|
|
if match.anypats() or any(opts.get(opt) for opt in complex):
|
2016-08-23 21:46:34 +03:00
|
|
|
f = fastlog(repo, rev, dirs, None)
|
2017-02-24 00:09:59 +03:00
|
|
|
revs = smartset.generatorset(f, iterasc=False)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
revs.reverse()
|
|
|
|
if not revs:
|
2017-02-24 00:09:59 +03:00
|
|
|
return smartset.baseset([]), None, None
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
expr, filematcher = cmdutil._makelogrevset(repo, pats, opts, revs)
|
|
|
|
matcher = revset.match(repo.ui, expr)
|
|
|
|
matched = matcher(repo, revs)
|
|
|
|
return matched, expr, filematcher
|
|
|
|
else:
|
|
|
|
# Simple match without revset shaves ~0.5 seconds off
|
|
|
|
# hg log -l 100 -T ' ' on common directories.
|
|
|
|
expr = 'fastlog(%s)' % ','.join(dirs)
|
2016-08-23 21:46:34 +03:00
|
|
|
return fastlog(repo, rev, dirs, dirmatches), expr, None
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
return orig(repo, pats, opts)
|
|
|
|
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
class readonlychangelog(object):
|
|
|
|
def __init__(self, opener):
|
|
|
|
self._changelog = changelog.changelog(opener)
|
|
|
|
|
|
|
|
def parentrevs(self, rev):
|
|
|
|
return self._changelog.parentrevs(rev)
|
|
|
|
|
|
|
|
def readfiles(self, node):
|
|
|
|
return self._changelog.readfiles(node)
|
|
|
|
|
|
|
|
def rev(self, node):
|
|
|
|
return self._changelog.rev(node)
|
|
|
|
|
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
class LocalIteratorThread(Thread):
|
|
|
|
"""Class which reads from an iterator and sends results to a queue.
|
|
|
|
|
|
|
|
Results are sent in a tuple (tag, success, result), where tag is the
|
|
|
|
id passed to this class' initializer, success is a bool, True for
|
|
|
|
success, False on error, and result is the output of the iterator.
|
|
|
|
|
|
|
|
When the iterator is finished, a poison pill is sent to the queue
|
|
|
|
with result set to None to signal completion.
|
|
|
|
|
|
|
|
Used to allow parallel fetching of results from both a local and
|
|
|
|
remote source.
|
|
|
|
|
|
|
|
* queue - self explanatory
|
|
|
|
* id - tag to use when sending messages
|
2016-08-23 21:46:34 +03:00
|
|
|
* rev - rev to start iterating at
|
2016-08-23 21:46:34 +03:00
|
|
|
* dirs - directories against which to match
|
|
|
|
* localmatch - a function to match candidate results
|
2016-08-23 21:46:34 +03:00
|
|
|
* repo - mercurial repository object
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
If an exception is thrown, error result with the message from the
|
|
|
|
exception will be passed along the queue. Since local results are
|
|
|
|
not expected to generate exceptions, this terminates iteration.
|
|
|
|
"""
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def __init__(self, queue, id, rev, dirs, localmatch, repo):
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
Thread.__init__(self)
|
|
|
|
self.daemon = True
|
|
|
|
self.queue = queue
|
|
|
|
self.id = id
|
2016-08-23 21:46:34 +03:00
|
|
|
self.rev = rev
|
2016-08-23 21:46:34 +03:00
|
|
|
self.dirs = dirs
|
|
|
|
self.localmatch = localmatch
|
2016-08-23 21:46:34 +03:00
|
|
|
self.ui = repo.ui
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
self._stop = Event()
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
# Create a private instance of changelog to avoid trampling
|
|
|
|
# internal caches of other threads
|
|
|
|
c = readonlychangelog(repo.svfs)
|
|
|
|
self.generator = originator(c.parentrevs, rev)
|
|
|
|
self.filefunc = c.readfiles
|
|
|
|
self.ui = repo.ui
|
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
def stop(self):
|
|
|
|
self._stop.set()
|
|
|
|
|
|
|
|
def stopped(self):
|
|
|
|
return self._stop.isSet()
|
|
|
|
|
|
|
|
def run(self):
|
2016-08-23 21:46:34 +03:00
|
|
|
generator = self.generator
|
|
|
|
match = self.localmatch
|
|
|
|
dirs = self.dirs
|
2016-08-23 21:46:34 +03:00
|
|
|
filefunc = self.filefunc
|
2016-08-23 21:46:34 +03:00
|
|
|
queue = self.queue
|
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
try:
|
2016-08-23 21:46:34 +03:00
|
|
|
for result in generator:
|
2016-08-23 02:23:10 +03:00
|
|
|
if self.stopped():
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
break
|
2016-08-23 21:46:34 +03:00
|
|
|
if not match or match(filefunc(result), dirs):
|
2016-11-25 03:23:21 +03:00
|
|
|
queue.put((self.id, True, result))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
except Exception as e:
|
2016-08-23 21:46:34 +03:00
|
|
|
self.ui.traceback()
|
2016-11-25 03:23:21 +03:00
|
|
|
queue.put((self.id, False, str(e)))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
finally:
|
2016-11-25 03:23:21 +03:00
|
|
|
queue.put((self.id, True, None))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
|
|
|
|
class FastLogThread(Thread):
|
|
|
|
"""Class which talks to a remote SCMQuery or fastlog server
|
|
|
|
|
|
|
|
Like the above, results are sent to a queue, and tagged with the
|
|
|
|
id passed to this class' initializer. Same rules for termination.
|
|
|
|
|
|
|
|
We page results in windows of up to FASTLOG_MAX to avoid generating
|
|
|
|
too many results; this has been optimized on the server to cache
|
|
|
|
fast continuations but this assumes service stickiness.
|
|
|
|
|
|
|
|
* queue - self explanatory
|
|
|
|
* id - tag to use when sending messages
|
2016-08-23 21:46:34 +03:00
|
|
|
* reponame - repository name (str)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
* scm - scm type (str)
|
|
|
|
* rev - revision to start logging from
|
2016-08-23 21:46:34 +03:00
|
|
|
* paths - paths to request logs
|
2016-08-23 21:46:34 +03:00
|
|
|
* repo - mercurial repository object
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
"""
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def __init__(self, queue, id, reponame, scm, rev, paths, repo):
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
Thread.__init__(self)
|
|
|
|
self.daemon = True
|
|
|
|
self.queue = queue
|
|
|
|
self.id = id
|
2016-08-23 21:46:34 +03:00
|
|
|
self.reponame = reponame
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
self.scm = scm
|
|
|
|
self.rev = rev
|
2016-08-23 21:46:34 +03:00
|
|
|
self.paths = list(paths)
|
2016-08-23 21:46:34 +03:00
|
|
|
self.ui = repo.ui
|
2016-08-23 21:46:34 +03:00
|
|
|
self.changelog = readonlychangelog(repo.svfs)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
self._stop = Event()
|
2016-08-23 21:46:34 +03:00
|
|
|
self._paths_to_fetch = 0
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
def stop(self):
|
|
|
|
self._stop.set()
|
|
|
|
|
|
|
|
def stopped(self):
|
|
|
|
return self._stop.isSet()
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def finishpath(self, path):
|
|
|
|
self._paths_to_fetch -= 1
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def gettodo(self):
|
|
|
|
return max(FASTLOG_MAX / self._paths_to_fetch, 100)
|
|
|
|
|
|
|
|
def generate(self, path):
|
2016-08-23 21:46:34 +03:00
|
|
|
start = str(self.rev)
|
2016-08-23 21:46:34 +03:00
|
|
|
reponame = self.reponame
|
2016-08-23 21:46:34 +03:00
|
|
|
revfn = self.changelog.rev
|
2016-08-23 21:46:34 +03:00
|
|
|
skip = 0
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
while True:
|
2016-08-23 21:46:34 +03:00
|
|
|
if self.stopped():
|
|
|
|
break
|
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
results = None
|
2016-08-23 21:46:34 +03:00
|
|
|
todo = self.gettodo()
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
try:
|
|
|
|
if USE_FASTLOG:
|
|
|
|
results = conduit.call_conduit('fastlog.log',
|
2016-08-23 21:46:34 +03:00
|
|
|
rev = start,
|
2016-08-23 21:46:34 +03:00
|
|
|
file_paths = [path],
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
skip = skip,
|
|
|
|
number = todo,
|
|
|
|
)
|
|
|
|
else:
|
|
|
|
results = conduit.call_conduit('scmquery.log_v2',
|
2016-08-23 21:46:34 +03:00
|
|
|
repo = reponame,
|
2017-04-12 23:51:18 +03:00
|
|
|
scm_type = self.scm,
|
2016-08-23 21:46:34 +03:00
|
|
|
rev = start,
|
2016-08-23 21:46:34 +03:00
|
|
|
file_paths = [path],
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
skip = skip,
|
|
|
|
number = todo,
|
|
|
|
)
|
|
|
|
except Exception as e:
|
2016-08-23 21:46:34 +03:00
|
|
|
if self.ui.config('fastlog', 'debug'):
|
|
|
|
self.ui.traceback(force=True)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
self.queue.put((self.id, False, str(e)))
|
2016-08-23 21:46:34 +03:00
|
|
|
self.stop()
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
return
|
|
|
|
|
|
|
|
if results is None:
|
|
|
|
self.queue.put((self.id, False, 'Unknown error'))
|
2016-08-23 21:46:34 +03:00
|
|
|
self.stop()
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
return
|
|
|
|
|
|
|
|
for result in results:
|
|
|
|
hash = result['hash']
|
2016-08-23 21:46:34 +03:00
|
|
|
try:
|
|
|
|
if len(hash) != 40:
|
|
|
|
raise ValueError('Received invalid hash %s' % hash)
|
2016-08-23 21:46:34 +03:00
|
|
|
rev = revfn(node.bin(hash))
|
|
|
|
if rev is None:
|
|
|
|
raise KeyError('Hash %s not in local repo' % hash)
|
2016-08-23 21:46:34 +03:00
|
|
|
except Exception as e:
|
|
|
|
if self.ui.config('fastlog', 'debug'):
|
|
|
|
self.ui.traceback(force=True)
|
2016-08-23 21:46:34 +03:00
|
|
|
self.queue.put((self.id, False, str(e)))
|
|
|
|
else:
|
|
|
|
yield rev
|
|
|
|
|
|
|
|
skip += todo
|
|
|
|
if len(results) < todo:
|
|
|
|
self.finishpath(path)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
return
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def run(self):
|
|
|
|
revs = None
|
|
|
|
paths = self.paths
|
|
|
|
|
|
|
|
self._paths_to_fetch = len(paths)
|
|
|
|
for path in paths:
|
|
|
|
g = self.generate(path)
|
2017-02-24 00:09:59 +03:00
|
|
|
gen = smartset.generatorset(g, iterasc=False)
|
2016-08-23 21:46:34 +03:00
|
|
|
gen.reverse()
|
|
|
|
if revs:
|
2017-02-17 20:01:50 +03:00
|
|
|
revs = smartset.addset(revs, gen, ascending=False)
|
2016-08-23 21:46:34 +03:00
|
|
|
else:
|
|
|
|
revs = gen
|
|
|
|
|
|
|
|
for rev in revs:
|
|
|
|
if self.stopped():
|
|
|
|
break
|
|
|
|
self.queue.put((self.id, True, rev))
|
|
|
|
|
|
|
|
self.queue.put((self.id, True, None))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
if __name__ == '__main__':
|
|
|
|
import doctest
|
|
|
|
doctest.testmod()
|