Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
# fastlog.py
|
|
|
|
#
|
|
|
|
# An extension to query remote servers for logs using scmquery / fastlog
|
|
|
|
#
|
|
|
|
# Copyright 2016 Facebook, Inc.
|
|
|
|
"""
|
|
|
|
connect to scmquery servers for fast fetching of logs on files and directories.
|
|
|
|
|
|
|
|
Configure it by adding the following config options to your .hg/hgrc.
|
2016-08-17 01:55:48 +03:00
|
|
|
This relies on fbconduit being setup for the repo; this should already
|
|
|
|
be configured if supported by your repo.
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
2019-04-17 08:15:44 +03:00
|
|
|
Config::
|
|
|
|
|
|
|
|
[fastlog]
|
|
|
|
enabled=true
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
"""
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
import heapq
|
|
|
|
from collections import deque
|
|
|
|
from threading import Event, Thread
|
|
|
|
|
2019-01-30 03:25:33 +03:00
|
|
|
from edenscm.mercurial import (
|
2016-08-23 21:46:34 +03:00
|
|
|
changelog,
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
error,
|
|
|
|
extensions,
|
2019-04-17 08:15:44 +03:00
|
|
|
match as matchmod,
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
node,
|
|
|
|
phases,
|
|
|
|
revset,
|
2017-02-17 20:01:50 +03:00
|
|
|
smartset,
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
util,
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
)
|
2019-01-30 03:25:33 +03:00
|
|
|
from edenscm.mercurial.i18n import _
|
|
|
|
from edenscm.mercurial.node import nullrev
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
|
|
|
|
conduit = None
|
|
|
|
|
|
|
|
FASTLOG_MAX = 500
|
|
|
|
FASTLOG_QUEUE_SIZE = 1000
|
|
|
|
FASTLOG_TIMEOUT = 20
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
def extsetup(ui):
|
|
|
|
global conduit
|
|
|
|
try:
|
|
|
|
conduit = extensions.find("fbconduit")
|
|
|
|
except KeyError:
|
2018-01-24 23:30:25 +03:00
|
|
|
from . import fbconduit as conduit
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
except ImportError:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
ui.warn(_("Unable to find fbconduit extension\n"))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
return
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
if not util.safehasattr(conduit, "conduit_config"):
|
|
|
|
ui.warn(_("Incompatible conduit module; disabling fastlog\n"))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
return
|
|
|
|
if not conduit.conduit_config(ui):
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
ui.warn(_("No conduit host specified in config; disabling fastlog\n"))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
return
|
|
|
|
|
2019-04-17 08:15:44 +03:00
|
|
|
extensions.wrapfunction(revset, "_follow", fastlogfollow)
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
def lazyparents(rev, public, parentfunc):
|
|
|
|
"""lazyparents(rev, public)
|
|
|
|
Lazily yield parents of rev in reverse order until all nodes
|
|
|
|
in public have been reached or all revs have been exhausted
|
|
|
|
|
|
|
|
10
|
|
|
|
| \
|
|
|
|
9 8
|
|
|
|
| | \
|
|
|
|
7 6 5
|
|
|
|
| | /
|
|
|
|
4 *3 First move, 4 -3
|
|
|
|
| /
|
|
|
|
2 *2 Second move, 4 -1
|
|
|
|
| *
|
|
|
|
1
|
|
|
|
|
|
|
|
For example:
|
|
|
|
>>> parents = { 10:[9, 8], 9:[7], 8:[6,5], 7:[4], 6:[3], 5:[3], 4:[2] }
|
|
|
|
>>> parents.update({ 3:[2], 2:[1], 1:[] })
|
|
|
|
>>> parentfunc = lambda k: parents[k]
|
|
|
|
>>> public = set([1])
|
|
|
|
>>> for p in lazyparents(10, public, parentfunc): print p,
|
|
|
|
10 9 8 7 6 5 4 3 2 1
|
|
|
|
>>> public = set([2,3])
|
|
|
|
>>> for p in lazyparents(10, public, parentfunc): print p,
|
|
|
|
10 9 8 7 6 5 4 3 2
|
|
|
|
>>> parents[4] = [3]
|
|
|
|
>>> public = set([3,4,5])
|
|
|
|
>>> for p in lazyparents(10, public, parentfunc): print p,
|
|
|
|
10 9 8 7 6 5 4 3
|
|
|
|
>>> parents[4] = [1]
|
|
|
|
>>> public = set([3,5,7])
|
|
|
|
>>> for p in lazyparents(10, public, parentfunc): print p,
|
|
|
|
10 9 8 7 6 5 4 3 2 1
|
|
|
|
"""
|
|
|
|
seen = set()
|
|
|
|
heap = [-rev]
|
|
|
|
|
|
|
|
while heap:
|
|
|
|
cur = -heapq.heappop(heap)
|
|
|
|
if cur not in seen:
|
|
|
|
seen.add(cur)
|
|
|
|
yield cur
|
|
|
|
|
|
|
|
published = cur in public
|
|
|
|
if published:
|
|
|
|
# Down to one public ancestor; end generation
|
|
|
|
if len(public) == 1:
|
|
|
|
return
|
|
|
|
public.remove(cur)
|
|
|
|
|
|
|
|
for p in parentfunc(cur):
|
|
|
|
if p != nullrev:
|
|
|
|
heapq.heappush(heap, -p)
|
|
|
|
if published:
|
|
|
|
public.add(p)
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def dirmatches(files, paths):
|
|
|
|
"""dirmatches(files, paths)
|
|
|
|
Return true if any files match directories in paths
|
|
|
|
Expects paths to be appended by '/'
|
|
|
|
|
|
|
|
>>> dirmatches(['holy/grail'], ['holy/'])
|
|
|
|
True
|
|
|
|
>>> dirmatches(['holy/grail'], ['holly/'])
|
|
|
|
False
|
|
|
|
>>> try: dirmatches(['holy/grail'], ['holy'])
|
|
|
|
... except AssertionError, e: print('caught')
|
|
|
|
caught
|
|
|
|
"""
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
assert paths
|
2016-08-23 21:46:34 +03:00
|
|
|
for path in paths:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
assert path[-1] == "/"
|
2016-08-23 21:46:34 +03:00
|
|
|
for f in files:
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
if f.startswith(path):
|
|
|
|
return True
|
|
|
|
return False
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def originator(parentfunc, rev):
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
"""originator(repo, rev)
|
|
|
|
Yield parents of rev from repo in reverse order
|
|
|
|
"""
|
|
|
|
# Use set(nullrev, rev) to iterate until termination
|
|
|
|
for p in lazyparents(rev, set([nullrev, rev]), parentfunc):
|
|
|
|
if rev != p:
|
|
|
|
yield p
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2019-04-17 08:15:44 +03:00
|
|
|
def fastlogfollow(orig, repo, subset, x, name, followfirst=False):
|
|
|
|
if followfirst:
|
|
|
|
# fastlog does not support followfirst=True
|
|
|
|
repo.ui.debug("fastlog: not used because 'followfirst' is set\n")
|
|
|
|
return orig(repo, subset, x, name, followfirst)
|
|
|
|
|
|
|
|
args = revset.getargsdict(x, name, "file startrev")
|
|
|
|
if "file" not in args:
|
|
|
|
# Not interesting for fastlog case.
|
|
|
|
repo.ui.debug("fastlog: not used because 'file' is not provided\n")
|
|
|
|
return orig(repo, subset, x, name, followfirst)
|
|
|
|
|
|
|
|
if "startrev" in args:
|
|
|
|
revs = revset.getset(repo, smartset.fullreposet(repo), args["startrev"])
|
|
|
|
it = iter(revs)
|
|
|
|
try:
|
|
|
|
startrev = next(it)
|
|
|
|
except StopIteration:
|
|
|
|
startrev = repo["."].rev()
|
|
|
|
try:
|
|
|
|
next(it)
|
|
|
|
# fastlog does not support multiple startrevs
|
|
|
|
repo.ui.debug("fastlog: not used because multiple revs are provided\n")
|
|
|
|
return orig(repo, subset, x, name, followfirst)
|
|
|
|
except StopIteration:
|
|
|
|
# supported by fastlog: startrev contains a single rev
|
|
|
|
pass
|
|
|
|
else:
|
|
|
|
startrev = repo["."].rev()
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
reponame = repo.ui.config("fbconduit", "reponame")
|
2019-04-17 08:15:44 +03:00
|
|
|
if not reponame or not repo.ui.configbool("fastlog", "enabled"):
|
|
|
|
repo.ui.debug("fastlog: not used because fastlog is disabled\n")
|
|
|
|
return orig(repo, subset, x, name, followfirst)
|
|
|
|
|
|
|
|
path = revset.getstring(args["file"], _("%s expected a pattern") % name)
|
|
|
|
if path.startswith("path:"):
|
|
|
|
# strip "path:" prefix
|
|
|
|
path = path[5:]
|
|
|
|
|
|
|
|
if any(path.startswith("%s:" % prefix) for prefix in matchmod.allpatternkinds):
|
|
|
|
# Patterns other than "path:" are not supported
|
|
|
|
repo.ui.debug(
|
|
|
|
"fastlog: not used because '%s:' patterns are not supported\n"
|
|
|
|
% path.split(":", 1)[0]
|
|
|
|
)
|
|
|
|
return orig(repo, subset, x, name, followfirst)
|
|
|
|
|
|
|
|
files = [path]
|
|
|
|
if not files or "." in files:
|
|
|
|
# Walking the whole repo - bail on fastlog
|
|
|
|
repo.ui.debug("fastlog: not used because walking through the entire repo\n")
|
|
|
|
return orig(repo, subset, x, name, followfirst)
|
|
|
|
|
|
|
|
dirs = set()
|
|
|
|
wvfs = repo.wvfs
|
|
|
|
for path in files:
|
|
|
|
if wvfs.isdir(path) and not wvfs.islink(path):
|
|
|
|
dirs.update([path + "/"])
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
else:
|
2019-04-17 08:15:44 +03:00
|
|
|
# bail on symlinks, and also bail on files for now
|
|
|
|
# with follow behavior, for files, we are supposed
|
|
|
|
# to track copies / renames, but it isn't convenient
|
|
|
|
# to do this through scmquery
|
|
|
|
repo.ui.debug("fastlog: not used because %s is not a directory\n" % path)
|
|
|
|
return orig(repo, subset, x, name, followfirst)
|
|
|
|
|
|
|
|
rev = startrev
|
|
|
|
|
|
|
|
parents = repo.changelog.parentrevs
|
|
|
|
public = set()
|
|
|
|
|
|
|
|
# Our criterion for invoking fastlog is finding a single
|
|
|
|
# common public ancestor from the current head. First we
|
|
|
|
# have to walk back through drafts to find all interesting
|
|
|
|
# public parents. Typically this will just be one, but if
|
|
|
|
# there are merged drafts, we may have multiple parents.
|
|
|
|
if repo[rev].phase() == phases.public:
|
|
|
|
public.add(rev)
|
|
|
|
else:
|
|
|
|
queue = deque()
|
|
|
|
queue.append(rev)
|
|
|
|
seen = set()
|
|
|
|
while queue:
|
|
|
|
cur = queue.popleft()
|
|
|
|
if cur not in seen:
|
|
|
|
seen.add(cur)
|
|
|
|
if repo[cur].mutable():
|
|
|
|
for p in parents(cur):
|
|
|
|
if p != nullrev:
|
|
|
|
queue.append(p)
|
|
|
|
else:
|
|
|
|
public.add(cur)
|
|
|
|
|
|
|
|
def fastlog(repo, startrev, dirs, localmatch):
|
|
|
|
filefunc = repo.changelog.readfiles
|
|
|
|
for parent in lazyparents(startrev, public, parents):
|
|
|
|
files = filefunc(parent)
|
|
|
|
if dirmatches(files, dirs):
|
|
|
|
yield parent
|
|
|
|
repo.ui.debug("found common parent at %s\n" % repo[parent].hex())
|
|
|
|
for rev in combinator(repo, parent, dirs, localmatch):
|
|
|
|
yield rev
|
|
|
|
|
|
|
|
def combinator(repo, rev, dirs, localmatch):
|
|
|
|
"""combinator(repo, rev, dirs, localmatch)
|
|
|
|
Make parallel local and remote queries along ancestors of
|
|
|
|
rev along path and combine results, eliminating duplicates,
|
|
|
|
restricting results to those which match dirs
|
|
|
|
"""
|
|
|
|
LOCAL = "L"
|
|
|
|
REMOTE = "R"
|
|
|
|
queue = util.queue(FASTLOG_QUEUE_SIZE + 100)
|
|
|
|
hash = repo[rev].hex()
|
|
|
|
|
|
|
|
local = LocalIteratorThread(queue, LOCAL, rev, dirs, localmatch, repo)
|
|
|
|
remote = FastLogThread(queue, REMOTE, reponame, "hg", hash, dirs, repo)
|
|
|
|
|
|
|
|
# Allow debugging either remote or local path
|
|
|
|
debug = repo.ui.config("fastlog", "debug")
|
|
|
|
if debug != "local":
|
|
|
|
repo.ui.debug("starting fastlog at %s\n" % hash)
|
|
|
|
remote.start()
|
|
|
|
if debug != "remote":
|
|
|
|
local.start()
|
|
|
|
seen = set([rev])
|
|
|
|
|
|
|
|
try:
|
|
|
|
while True:
|
|
|
|
try:
|
|
|
|
producer, success, msg = queue.get(True, 3600)
|
|
|
|
except util.empty:
|
|
|
|
raise error.Abort("Timeout reading log data")
|
|
|
|
if not success:
|
|
|
|
if producer == LOCAL:
|
|
|
|
raise error.Abort(msg)
|
|
|
|
elif msg:
|
|
|
|
repo.ui.log("hgfastlog", msg)
|
|
|
|
continue
|
|
|
|
|
|
|
|
if msg is None:
|
|
|
|
# Empty message means no more results
|
|
|
|
return
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
2019-04-17 08:15:44 +03:00
|
|
|
rev = msg
|
|
|
|
if debug:
|
|
|
|
if producer == LOCAL:
|
|
|
|
repo.ui.debug("LOCAL:: %s\n" % msg)
|
|
|
|
elif producer == REMOTE:
|
|
|
|
repo.ui.debug("REMOTE:: %s\n" % msg)
|
|
|
|
|
|
|
|
if rev not in seen:
|
|
|
|
seen.add(rev)
|
|
|
|
yield rev
|
|
|
|
finally:
|
|
|
|
local.stop()
|
|
|
|
remote.stop()
|
|
|
|
|
|
|
|
revgen = fastlog(repo, rev, dirs, dirmatches)
|
|
|
|
fastlogset = smartset.generatorset(revgen, iterasc=False)
|
|
|
|
# Optimization: typically for "reverse(:.) & follow(path)" used by
|
|
|
|
# "hg log". The left side is more expensive, although it has smaller
|
|
|
|
# "weight". Make sure fastlogset is on the left side to avoid slow
|
|
|
|
# walking through ":.".
|
|
|
|
if subset.isdescending():
|
|
|
|
fastlogset.reverse()
|
|
|
|
return fastlogset & subset
|
|
|
|
return subset & fastlogset
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
class readonlychangelog(object):
|
|
|
|
def __init__(self, opener):
|
|
|
|
self._changelog = changelog.changelog(opener)
|
|
|
|
|
|
|
|
def parentrevs(self, rev):
|
|
|
|
return self._changelog.parentrevs(rev)
|
|
|
|
|
|
|
|
def readfiles(self, node):
|
|
|
|
return self._changelog.readfiles(node)
|
|
|
|
|
|
|
|
def rev(self, node):
|
|
|
|
return self._changelog.rev(node)
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
class LocalIteratorThread(Thread):
|
|
|
|
"""Class which reads from an iterator and sends results to a queue.
|
|
|
|
|
|
|
|
Results are sent in a tuple (tag, success, result), where tag is the
|
|
|
|
id passed to this class' initializer, success is a bool, True for
|
|
|
|
success, False on error, and result is the output of the iterator.
|
|
|
|
|
|
|
|
When the iterator is finished, a poison pill is sent to the queue
|
|
|
|
with result set to None to signal completion.
|
|
|
|
|
|
|
|
Used to allow parallel fetching of results from both a local and
|
|
|
|
remote source.
|
|
|
|
|
|
|
|
* queue - self explanatory
|
|
|
|
* id - tag to use when sending messages
|
2016-08-23 21:46:34 +03:00
|
|
|
* rev - rev to start iterating at
|
2016-08-23 21:46:34 +03:00
|
|
|
* dirs - directories against which to match
|
|
|
|
* localmatch - a function to match candidate results
|
2016-08-23 21:46:34 +03:00
|
|
|
* repo - mercurial repository object
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
If an exception is thrown, error result with the message from the
|
|
|
|
exception will be passed along the queue. Since local results are
|
|
|
|
not expected to generate exceptions, this terminates iteration.
|
|
|
|
"""
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def __init__(self, queue, id, rev, dirs, localmatch, repo):
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
Thread.__init__(self)
|
|
|
|
self.daemon = True
|
|
|
|
self.queue = queue
|
|
|
|
self.id = id
|
2016-08-23 21:46:34 +03:00
|
|
|
self.rev = rev
|
2016-08-23 21:46:34 +03:00
|
|
|
self.dirs = dirs
|
|
|
|
self.localmatch = localmatch
|
2016-08-23 21:46:34 +03:00
|
|
|
self.ui = repo.ui
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
self._stop = Event()
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
# Create a private instance of changelog to avoid trampling
|
|
|
|
# internal caches of other threads
|
|
|
|
c = readonlychangelog(repo.svfs)
|
|
|
|
self.generator = originator(c.parentrevs, rev)
|
|
|
|
self.filefunc = c.readfiles
|
|
|
|
self.ui = repo.ui
|
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
def stop(self):
|
|
|
|
self._stop.set()
|
|
|
|
|
|
|
|
def stopped(self):
|
|
|
|
return self._stop.isSet()
|
|
|
|
|
|
|
|
def run(self):
|
2016-08-23 21:46:34 +03:00
|
|
|
generator = self.generator
|
|
|
|
match = self.localmatch
|
|
|
|
dirs = self.dirs
|
2016-08-23 21:46:34 +03:00
|
|
|
filefunc = self.filefunc
|
2016-08-23 21:46:34 +03:00
|
|
|
queue = self.queue
|
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
try:
|
2016-08-23 21:46:34 +03:00
|
|
|
for result in generator:
|
2016-08-23 02:23:10 +03:00
|
|
|
if self.stopped():
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
break
|
2016-08-23 21:46:34 +03:00
|
|
|
if not match or match(filefunc(result), dirs):
|
2016-11-25 03:23:21 +03:00
|
|
|
queue.put((self.id, True, result))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
except Exception as e:
|
2016-08-23 21:46:34 +03:00
|
|
|
self.ui.traceback()
|
2016-11-25 03:23:21 +03:00
|
|
|
queue.put((self.id, False, str(e)))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
finally:
|
2016-11-25 03:23:21 +03:00
|
|
|
queue.put((self.id, True, None))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
class FastLogThread(Thread):
|
2017-07-28 12:33:00 +03:00
|
|
|
"""Class which talks to a remote SCMQuery
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
Like the above, results are sent to a queue, and tagged with the
|
|
|
|
id passed to this class' initializer. Same rules for termination.
|
|
|
|
|
|
|
|
We page results in windows of up to FASTLOG_MAX to avoid generating
|
|
|
|
too many results; this has been optimized on the server to cache
|
|
|
|
fast continuations but this assumes service stickiness.
|
|
|
|
|
|
|
|
* queue - self explanatory
|
|
|
|
* id - tag to use when sending messages
|
2016-08-23 21:46:34 +03:00
|
|
|
* reponame - repository name (str)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
* scm - scm type (str)
|
|
|
|
* rev - revision to start logging from
|
2016-08-23 21:46:34 +03:00
|
|
|
* paths - paths to request logs
|
2016-08-23 21:46:34 +03:00
|
|
|
* repo - mercurial repository object
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
"""
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def __init__(self, queue, id, reponame, scm, rev, paths, repo):
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
Thread.__init__(self)
|
|
|
|
self.daemon = True
|
|
|
|
self.queue = queue
|
|
|
|
self.id = id
|
2016-08-23 21:46:34 +03:00
|
|
|
self.reponame = reponame
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
self.scm = scm
|
|
|
|
self.rev = rev
|
2016-08-23 21:46:34 +03:00
|
|
|
self.paths = list(paths)
|
2016-08-23 21:46:34 +03:00
|
|
|
self.ui = repo.ui
|
2016-08-23 21:46:34 +03:00
|
|
|
self.changelog = readonlychangelog(repo.svfs)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
self._stop = Event()
|
2016-08-23 21:46:34 +03:00
|
|
|
self._paths_to_fetch = 0
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
def stop(self):
|
|
|
|
self._stop.set()
|
|
|
|
|
|
|
|
def stopped(self):
|
|
|
|
return self._stop.isSet()
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def finishpath(self, path):
|
|
|
|
self._paths_to_fetch -= 1
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def gettodo(self):
|
|
|
|
return max(FASTLOG_MAX / self._paths_to_fetch, 100)
|
|
|
|
|
|
|
|
def generate(self, path):
|
2016-08-23 21:46:34 +03:00
|
|
|
start = str(self.rev)
|
2016-08-23 21:46:34 +03:00
|
|
|
reponame = self.reponame
|
2016-08-23 21:46:34 +03:00
|
|
|
revfn = self.changelog.rev
|
2016-08-23 21:46:34 +03:00
|
|
|
skip = 0
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
|
|
|
while True:
|
2016-08-23 21:46:34 +03:00
|
|
|
if self.stopped():
|
|
|
|
break
|
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
results = None
|
2016-08-23 21:46:34 +03:00
|
|
|
todo = self.gettodo()
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
try:
|
2017-07-28 12:33:00 +03:00
|
|
|
results = conduit.call_conduit(
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
"scmquery.log_v2",
|
2017-07-28 12:33:00 +03:00
|
|
|
repo=reponame,
|
|
|
|
scm_type=self.scm,
|
|
|
|
rev=start,
|
|
|
|
file_paths=[path],
|
|
|
|
skip=skip,
|
|
|
|
number=todo,
|
|
|
|
)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
except Exception as e:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
if self.ui.config("fastlog", "debug"):
|
2016-08-23 21:46:34 +03:00
|
|
|
self.ui.traceback(force=True)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
self.queue.put((self.id, False, str(e)))
|
2016-08-23 21:46:34 +03:00
|
|
|
self.stop()
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
return
|
|
|
|
|
|
|
|
if results is None:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
self.queue.put((self.id, False, "Unknown error"))
|
2016-08-23 21:46:34 +03:00
|
|
|
self.stop()
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
return
|
|
|
|
|
|
|
|
for result in results:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
hash = result["hash"]
|
2016-08-23 21:46:34 +03:00
|
|
|
try:
|
|
|
|
if len(hash) != 40:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
raise ValueError("Received invalid hash %s" % hash)
|
2016-08-23 21:46:34 +03:00
|
|
|
rev = revfn(node.bin(hash))
|
|
|
|
if rev is None:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
raise KeyError("Hash %s not in local repo" % hash)
|
2016-08-23 21:46:34 +03:00
|
|
|
except Exception as e:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
if self.ui.config("fastlog", "debug"):
|
2016-08-23 21:46:34 +03:00
|
|
|
self.ui.traceback(force=True)
|
2016-08-23 21:46:34 +03:00
|
|
|
self.queue.put((self.id, False, str(e)))
|
|
|
|
else:
|
|
|
|
yield rev
|
|
|
|
|
|
|
|
skip += todo
|
|
|
|
if len(results) < todo:
|
|
|
|
self.finishpath(path)
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
return
|
|
|
|
|
2016-08-23 21:46:34 +03:00
|
|
|
def run(self):
|
|
|
|
revs = None
|
|
|
|
paths = self.paths
|
|
|
|
|
|
|
|
self._paths_to_fetch = len(paths)
|
|
|
|
for path in paths:
|
|
|
|
g = self.generate(path)
|
2017-02-24 00:09:59 +03:00
|
|
|
gen = smartset.generatorset(g, iterasc=False)
|
2016-08-23 21:46:34 +03:00
|
|
|
gen.reverse()
|
|
|
|
if revs:
|
2017-02-17 20:01:50 +03:00
|
|
|
revs = smartset.addset(revs, gen, ascending=False)
|
2016-08-23 21:46:34 +03:00
|
|
|
else:
|
|
|
|
revs = gen
|
|
|
|
|
|
|
|
for rev in revs:
|
|
|
|
if self.stopped():
|
|
|
|
break
|
|
|
|
self.queue.put((self.id, True, rev))
|
2017-07-31 10:58:55 +03:00
|
|
|
# The end marker (self.id, True, None) indicates that the thread
|
|
|
|
# completed successfully. Don't send it if the thread is stopped.
|
|
|
|
# The thread can be stopped for one of two reasons:
|
|
|
|
# 1. The fastlog service failed - in this case, flagging a successful
|
|
|
|
# finish is harmful, because it will stop us continuing with local
|
|
|
|
# results, truncating output.
|
|
|
|
# 2. The caller is going to ignore all future results from us. In this
|
|
|
|
# case, it'll ignore the end marker anyway - it's discarding the
|
|
|
|
# entire queue.
|
|
|
|
if not self.stopped():
|
|
|
|
self.queue.put((self.id, True, None))
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
|
|
|
if __name__ == "__main__":
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
import doctest
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior. To do this, we backtrackfrom the current head to find
all draft revisions along the path, then find the common public ancestor of
those. Once we find that, we can begin paging in results from the scmquery
service.
Pretty much a working diff at this point. Limits and boundary conditions
have not been fully tested. Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that. Could also be an issue with conduit. Other than that,
things seem to work.
Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m1.895s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m1.308s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a b
[zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.320s
user 0m0.004s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbcode] diff a c
Testing --user option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a
real 0m2.765s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b
real 0m23.247s
user 0m0.005s
sys 0m0.001s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing same output enabled / disabled for -X option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a
real 0m1.292s
user 0m0.002s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b
real 0m2.697s
user 0m0.002s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -k option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a
real 0m1.174s
user 0m0.003s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b
real 0m1.259s
user 0m0.004s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing -I option:
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a
real 0m1.473s
user 0m0.005s
sys 0m0.003s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b
real 0m2.911s
user 0m0.003s
sys 0m0.002s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource]
Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a
real 0m2.892s
user 0m0.003s
sys 0m0.006s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b
real 0m2.575s
user 0m0.697s
sys 0m0.077s
[zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c
real 0m7.339s
user 0m0.003s
sys 0m0.004s
[zamsden@devbig192.prn1 ~/local/fbsource] diff a b
[zamsden@devbig192.prn1 ~/local/fbsource] diff a c
[zamsden@devbig192.prn1 ~/local/fbsource]
Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash
Reviewed By: stash
Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv
Differential Revision: https://phabricator.intern.facebook.com/D3634075
Tasks: 12341014
Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-13 01:52:50 +03:00
|
|
|
doctest.testmod()
|