Commit Graph

36 Commits

Author SHA1 Message Date
Zach Amsden
f99540d161 [fastlog-hg] Thread safety
Summary:
Sharing repo.changelog across threads is problematic in mercurial
since it may internally cache things, and clearing cache while another
thread is executing is not nice.  In particular, code such as the following
is inherently raceful when sharing changelogs (or anything derived from
revlog) across threads:

    if self._cache:
        if self._cache[0] == node:
            return self._cache[2]
        cachedrev = self._cache[1]

Reworked things a bit to make sure the spawned threads have their own
newly created changelogs and member functions internally referencing them.

Test Plan:
No longer seeing random exceptions being thrown from local log
threads (which terminates iteration)

Reviewers: rmcelroy, #mercurial, ttung, quark, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3733592

Tasks: 12341014

Signature: t1:3733592:1471908572:877b56241e56b6ea92c76023460d4fbe80a4263d
2016-08-23 11:46:34 -07:00
Zach Amsden
205a3b8a81 [fastlog-hg] Better multi-path and file support for fastlog
Summary:
The multi-path support is pretty shameful; we can
do much better by issuing multiple queries and combining
results.  File support was also missing.

Test Plan:
Ran fastlog test.  Need to update test to test files.

P56589012

Reviewers: quark, rmcelroy, durham

Reviewed By: durham

Subscribers: stash, mjpieters, jeroenv

Differential Revision: https://phabricator.intern.facebook.com/D3726318

Tasks: 12341014

Signature: t1:3726318:1471919354:e5e8f40a5b918b493e927c2d620da1fdefa1768c
2016-08-23 11:46:34 -07:00
Zach Amsden
fe94406e9a [fastlog-hg] Add localmatch function
Summary:
Bugfix - we were failing to check local patterns in the fastpath.
Pass a filtering function to the local log thread to limit its results.

Test Plan: Run local log

Reviewers: stash, quark, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3727573

Tasks: 12341014

Signature: t1:3727573:1471907418:37b83d6c02e61ce2705070c7a86dd815a1c1cb29
2016-08-23 11:46:34 -07:00
Zach Amsden
76529f6745 [fastlog-hg] Pass revfn
Summary: Pass revfn to the thread to unify response messages.

Test Plan: Run unit test, try fastlog

Reviewers: quark, durham, rmcelroy, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3727506

Tasks: 12341014

Signature: t1:3727506:1471855144:24e622c6add0584d37539ea5ee319e3454f5313e
2016-08-23 11:46:34 -07:00
Zach Amsden
31382f23a3 [fastlog-hg] Better handling of exceptions
Summary: Without this, exceptions are super hard to debug

Test Plan: Manually force exceptions with bogus array indices on non-arrays

Reviewers: quark, #mercurial, ttung, durham

Reviewed By: durham

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3744317

Tasks: 12341014

Signature: t1:3744317:1471919239:8c8d0fce6c212fd677df8b7f94a48d17dbe64b93
2016-08-23 11:46:34 -07:00
Jun Wu
36544a8c4d smartfixup: add an interactive mode
Summary:
Like `commit -i`, this diff adds an `interactive` mode. I'd like
to reuse some patching code but they are tightly coupled about
writing files to disk - while I want to handle everything in memory.

In the future I may want to add `memorystore` and `memorybackend`
to upstream's `patch.py`. But they are not available now therefore
the "patching" logic is a re-invent. Fortunately the logic is just
about 20 lines.

Test Plan:
Manually run `hg sf -i` with some manually crafted random cases.
Also make sure it works with both `text` and `curses` interfaces.

Reviewers: #mercurial, durham, ttung

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3723179

Signature: t1:3723179:1471582313:b10d278ca5b1a2aee6b41936f315a960edee92ed
2016-08-23 15:20:20 +01:00
Jun Wu
dea474ec1f smartfixup: add command line options to preview changes
Summary:
The `--print-changes` option will show which commit, a changed lines will be
applied to. Note it is a bit expensive so we only calculate the content when
being asked.

The `--dry-run` option will prevent the final commit operation.

Users can now use `-pn` to preview changes to understand what will be changed.

Test Plan: Run `test-smartfixup.t`

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3719673

Signature: t1:3719673:1471582117:7f3be3c3bf3829148c3d096d2aae6e7382449b98
2016-08-23 15:19:54 +01:00
Jun Wu
a51d83bec3 smartfixup: implement the command
Summary:
This is the last piece of the initial smartfixup implementation: the command.
A new test was added to test the command.

Test Plan: Run the newly added test

Reviewers: #mercurial, ttung, jbower, durham

Reviewed By: durham

Subscribers: akushner, simonfar, durham, jbower, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3717354

Signature: t1:3717354:1471629588:202e4a5820de472eb131eb968ec73f194d2c077d
2016-08-23 15:19:29 +01:00
Jun Wu
0087ee33bf smartfixup: implement fixupstate
Summary:
If `filefixupstate` is to `filecontext`, `fixupstate` is to `context`.
This does the actual committing, creating obsmarkers, moving bookmarks work.

Test Plan: A new test will be included in the next diff

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716954

Signature: t1:3716954:1471639171:9891a0c64365b0592f5a4d86e3c83d521b4c5bb2
2016-08-23 15:18:36 +01:00
Jun Wu
7683c76566 smartfixup: implement filefixupstate
Summary:
The `filefixupstate` object maintains states and contains logic related to
do "smartfixup" to a single file.

Test Plan: Added a new test

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716761

Signature: t1:3716761:1471919159:033e79e9d12307785863959dd5d82a7c3c647ff3
2016-08-23 15:17:42 +01:00
Jun Wu
98861bc86e smartfixup: add utilities
Summary:
This series is a rewrite of D3264203, using the `linelog` instead of
`histedit`. The new version has the following advantages:

  - handle more cases where it could be a merge conflict with a traditional
    merge algorithm. the old code tries to "predict" when a merge won't
    cause conflicts. but that's just kinda stupid - comparing with the new
    code using linelog - it is just impossible to produce merge conflicts.
  - do not touch working copy - all intermediate calculations and states are
    in memory. say goodbye to the clean working copy requirement of histedit.
  - handle line numbers cleanly and confidently - the old code struggles (but
    failed) to make line numbers correct after each edits, while it is
    actually hard as the "merge" operation could change line numbers in an
    unpredictable way. the new code using linelog can just edit chunks from
    an old revision, from the latest view without checking out that old
    revision first. therefore no need to mess up with line numbers.

This diff adds utility functions and classes used later. It introduces
unused `import`s but they will be used later.

Test Plan: `arc lint`

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716758

Signature: t1:3716758:1471556512:0fbff8f6fe4bc02d2120c723d2cc28676895c679
2016-08-23 15:16:15 +01:00
Adam Simpkins
b1540c8b8a Fix hgext3rd/__init__.py to make sure it is a namespace package
Summary:
According to the comments hg-crew, hgext3rd is supposed to be a namespace
package, so that extensions can be found in multiple different hgext3rd
directories.

However, the hgext3rd directory in fb-hgext has an empty __init__.py file,
which turns hgext3rd into a plain package instead of a namespace package,
preventing other hgext3rd directories from being found.

This fixes the __init__.py file to perform proper namespace package
initialization, as is done in the hg-crew/hgext3rd/__init__.py file

Test Plan:
Deployed this file on my server and confirmed that extensions in other hgext3rd
directories in $PYTHONPATH are still found correctly.

Reviewers: rmcelroy, durham, quark

Reviewed By: quark

Subscribers: net-systems-diffs@, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3746098

Tasks: 12792845

Signature: t1:3746098:1471679643:44a274807d3598a670e1742dee4cc2654f6bad97
2016-08-22 17:39:02 -07:00
Zach Amsden
bdc73e2393 [fastlog-hg] Remove limit on threads
Summary:
Limits make no sense anymore because we may be filtering
the results further.

Test Plan: fastlog unit test, run extension against fbsource

Reviewers: quark, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3743974

Tasks: 12341014

Signature: t1:3743974:1471906692:25f48349bd70781219acdc3cf6b98134035d6236
2016-08-22 16:23:10 -07:00
Zach Amsden
64d75b4864 [fastlog-hg] Get patterns working again
Summary:
When the restriction of only working on a single directory
was dropped, pattern support was neglected.  That has to be done through
a revset, not the looser matches() filter.

Test Plan: P56588753

Reviewers: rmcelroy, quark, stash, durham

Reviewed By: durham

Subscribers: mhammell, meisner, calcsaransh, gqchen, dzhulgakov, kerl, bnitka, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3734232

Tasks: 12341014

Signature: t1:3734232:1471906628:d3995b358d683383fe590fa5848d8450a0ea05dd
2016-08-22 16:21:43 -07:00
Zach Amsden
cde79943b1 [fastlog-hg] For now, import os is still required
Summary: Accidentally removed this dependency too soon

Test Plan: Run fastlog on fbcode; broken before, now fixed

Reviewers: durham, quark, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3743884

Tasks: 12341014

Signature: t1:3743884:1471853021:ce470e4e42aa1c07aed624285a81824e04e70668
2016-08-22 14:46:39 -07:00
Zach Amsden
f115cd74e9 [fastlog-hg] Remove unused imports
Summary: No need for cruft

Test Plan: Load in Atom, no more complaints about unused imports

Reviewers: durham, rmcelroy, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3733509

Tasks: 12341014

Signature: t1:3733509:1471519620:08e896aaff65b545d7d16e5d46ba6a8e2a2bc186
2016-08-19 11:59:10 -07:00
Jeremy Fitzhardinge
1fd4455c73 Implement 'hg uncommit'
Summary:
TODO:
 - fix handling of empty commit (leaves stray heads)
 - add tests

Test Plan: Add mercurial unit tests

Reviewers: #mercurial, ttung, rmcelroy, durham

Reviewed By: durham

Subscribers: quark, rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3610409

Tasks: 12340638

Signature: t1:3610409:1471398045:355f20502d917374e99feee4681a7ef6f853ad6c
2016-08-17 13:43:25 -07:00
Zach Amsden
092d645baa Fix comment for fastlog
Summary: Document how to enable the extension

Test Plan: Set this in my actual .hgrc and confirm it works.

Reviewers: durham, rmcelroy, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3721128

Signature: t1:3721128:1471342825:e274764e036f1c03b9f8795221af6fe587990340
2016-08-16 15:55:48 -07:00
Jun Wu
fab3975b47 sshaskpass: use mkdtemp to create temporary directory
Summary:
Before this patch, sshaskpass has a hardcoded tmp directory, which, if being
abused, can break sshaskpass from working. This diff fixes the issue by
using tempfile.mkdtemp.

Test Plan:
Run `chg push -r . ssh://root@localhost//tmp/foo -f --allow-ano` and make
sure ssh password prompt works. Also run the sshaskpass test.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3705674

Signature: t1:3705674:1471373405:aec54ac685b060efb616307552cebcccee5cebe8
2016-08-16 19:51:47 +01:00
Zach Amsden
f1ecf43a83 Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior.  To do this, we backtrackfrom the current head to find
 all draft revisions along the path, then find the common public ancestor of
those.  Once we find that, we can begin paging in results from the scmquery
service.

Pretty much a working diff at this point.  Limits and boundary conditions
have not been fully tested.  Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that.  Could also be an issue with conduit.  Other than that,
things seem to work.

Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):

  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a

  real	0m1.895s
  user	0m0.003s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b

  real	0m1.308s
  user	0m0.005s
  sys	0m0.001s
  [zamsden@devbig192.prn1 ~/local/fbcode] diff a b
  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c

  real	0m7.320s
  user	0m0.004s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbcode] diff a c




Testing --user option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a

  real	0m2.765s
  user	0m0.002s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b

  real	0m23.247s
  user	0m0.005s
  sys	0m0.001s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing same output enabled / disabled for -X option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a

  real	0m1.292s
  user	0m0.002s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b

  real	0m2.697s
  user	0m0.002s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing -k option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a

  real	0m1.174s
  user	0m0.003s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b

  real	0m1.259s
  user	0m0.004s
  sys	0m0.002s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing -I option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a

  real	0m1.473s
  user	0m0.005s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b

  real	0m2.911s
  user	0m0.003s
  sys	0m0.002s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]




Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a

  real	0m2.892s
  user	0m0.003s
  sys	0m0.006s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b

  real	0m2.575s
  user	0m0.697s
  sys	0m0.077s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c

  real	0m7.339s
  user	0m0.003s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a c
  [zamsden@devbig192.prn1 ~/local/fbsource]

Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash

Reviewed By: stash

Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv

Differential Revision: https://phabricator.intern.facebook.com/D3634075

Tasks: 12341014

Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-12 15:52:50 -07:00
Jun Wu
5b0b861096 sshaskpass: make the prompt similar to systemd
Summary:
This diff makes the prompt similar to what systemd does. See the screenshot:

{F62774663}

It solves an issue that with echo disabled, the user won't know whether they
have pressed ENTER or not - the cursor won't move. Now it explicitly prints
"AUTHENTICATION COMPLETE".


Test Plan: As the screenshot, and also run the sshaskpass test.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3705703

Signature: t1:3705703:1471008095:863ff9820b87ea3f4631295b10472802fb5e459f
2016-08-11 21:56:31 +01:00
Jun Wu
d4366acbd5 sshaskpass: use a standalone script
Summary:
Before this patch, sshaskpass will set SSH_ASKPASS to itself, aka. `__file__`.
This won't work if sshaskpass.py gets installed by setup.py because setup.py
will remove its `+x` bit.

Test Plan:
Run `chg push -r . ssh://root@localhost//tmp/foo -f --allow-ano` and make
sure ssh password prompt works.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3705657

Signature: t1:3705657:1471008122:9cfa3adf078e4bbe8f6b6ba05d1fb6be513d3e71
2016-08-11 21:42:54 +01:00
Jun Wu
6015a95c00 tweakdefaults: make "bookmarks" use the unfiltered repo
Summary:
We are seeing perf issues with hidden/obsolete handling.
`hg bookmark` is a frequently used command and by making it use the unfiltered
repo, it could be 200ms-300ms faster.

Test Plan: Added a new test

Reviewers: #mercurial, mitrandir, ikostia, ttung

Reviewed By: mitrandir, ikostia

Subscribers: mitrandir, rmcelroy, akushner, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3692968

Signature: t1:3692968:1470777864:72ad5d0ffb52ecfcaaa607082693b88319d778fd
2016-08-11 18:43:06 +01:00
Jun Wu
cf9686ff3a morestatus: rebase takes precedence over update
Summary:
This happens when pressing Ctrl+C during "hg rebase". We should show
"interrupted rebase" instead of "interrupted update".

Test Plan: Added a new test

Reviewers: ttung, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3697984

Signature: t1:3697984:1470863868:acd67eeb5099c73ea7df174b3e1dbcbf9ac54944
2016-08-10 22:19:30 +01:00
Oguz Ulgen
585ede8a25 Add sync status and change the conduit call to differential.querydiffhashes
Summary:
1) Add sync status
2) Combine sync status and phab status to use one unified conduit call,
i.e. differential.querydiffhashes

Test Plan:
cd hg-crew
make local
cd ../fb-hgext
make local
cd tests
../../hg-crew/tests/run-tests.py test-phabstatus.t
../../hg-crew/tests/run-tests.py test-syncstatus.t

Reviewers: wqfish, lcharignon, #sourcecontrol, wez, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3651915

Tasks: 10100400

Signature: t1:3651915:1470340328:bf003006f6afe9b86a40f204e150e0d12350c21d
2016-08-05 11:33:36 -07:00
Jun Wu
4f1d1c3291 rage: filter "FM:" from blackbox.log
Summary:
Fastmanifest debug logs in blackbox.log are less interesting in a rage report,
but they take a lot of lines. This diff uses `grep` to filter them out from rage report.

Test Plan: `hg rage --preview`

Reviewers: #mercurial, ttung

Reviewed By: ttung

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3676751

Signature: t1:3676751:1470421349:3b5e0cb4118473f5550e83bf9c5e7ac27a59bcb0
2016-08-05 19:17:45 +01:00
Jun Wu
bdb47a171c ownercheck: new extension to prevent operations on repos not owned
Summary:
This is mainly to address the same issue as D3609747: prevent issues caused
by running hg as root. The difference is this extension reads owner and uid
dynamically without config when initializing of a localrepo object.

As a side effect it covers more situations like running hg in others' repos, while
being less restrictive for some other commands like `sudo hg version`.

Test Plan: Added a new test

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: ikostia, rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3619997

Tasks: 11723374

Signature: t1:3619997:1470176448:beaf53c09fca498206767641ffa4315a744ee07e
2016-07-26 13:10:15 +01:00
Tony Tung
c0e7082b92 [reporootlog] report the repo root to the scm wrappers
Summary:
The SCM wrappers do not have a reliable and easy way of detecting the repo root.  For instance, if someone does hg log /full/path/to/repo/content, we cannot actually determine that /full/path/to/repo is the repo root.  We have to somehow integrate an overly large portion of mercurial's commmand parsing infrastrastructure to accomplish this.

However, since mercurial knows the repo root, just extract that knowledge and send it down to the client.

Test Plan:
```
[dev8692]:~> USE_DIST_HG= FB_HG_DIAGS= hg log --config 'sampling.key.reporootlog=perfpipe_dev_command_timers' --config 'extensions.reporootlog=work/facebook-hg-rpms/fb-hgext/hgext3rd/reporootlog.py'  work/facebook-hg-rpms/smoketest.sh
chg: enabled by /etc/mercurial/usechg

<snipped for brevity>

internal stats file: /tmp/scm-internal-stats5BoW29
hg profiling mode: LSPROFILER
stats: {
    "int": {
        "builddate": 1469712764,
        "cachehitratio": 50,
        "consumed": 302,
        "diffcachehitratio": -1,
        "elapsed": 130,
        "errorcode": 0,
        "filesnotincachehitratio": -1,
        "time": 1469819716
    },
    "normal": {
        "chg": "true",
        "command": "log",
        "evolution": "createmarkers",
        "evolutionext": "-*- empty -*-",
        "fastmanifest": "-*- empty -*-",
        "filesystem": "",
        "fsmonitor_ext": "-*- not present -*-",
        "fsmonitor_mode": "on",
        "fullcommand": "\/usr\/bin\/hg log --config sampling.key.reporootlog=perfpipe_dev_command_timers --config extensions.reporootlog=work\/facebook-hg-rpms\/fb-hgext\/hgext3rd\/reporootlog.py work\/facebook-hg-rpms\/smoketest.sh",
        "generaldelta": "false",
        "host": "dev8692.prn1",
        "iftype": "disconnected",
        "interactive": "true",
        "kernel": "4.0.9-30_fbk4_2311_gd3e5a5c",
        "metrics_type": "fastmanifest-filesnotincachehitratio",
        "msg": "",
        "parent": "\/bin\/bash",
        "remotefilelog": "false",
        "repo": "work\/facebook-hg-rpms",
        "scm": "hg",
        "sid": "",
        "source": "hg",
        "sqldirstate": "false",
        "sqldirstate_enabled": "-*- not present -*-",
        "sqldirstate_loaded": "-*- empty -*-",
        "sqldirstate_upgrade": "True",
        "sshclient": "localhost",
        "tw_job_cluster": "",
        "tw_job_name": "",
        "tw_job_user": "",
        "tw_oncall_team": "",
        "tw_task_id": "",
        "user": "ttung",
        "version": "3.8.4+443-d0746b"
    }
}
[dev8692]:~>
```

Without this change, the repo would be "unknown-repo":

```
[dev8692]:~> USE_DIST_HG= FB_HG_DIAGS= hg log --config 'sampling.key.reporootlog=perfpipe_dev_command_timers' work/facebook-hg-rpms/smoketest.sh 2>&1 | grep repo
        "fullcommand": "\/usr\/bin\/hg log --config sampling.key.reporootlog=perfpipe_dev_command_timers work\/facebook-hg-rpms\/smoketest.sh",
        "repo": "unknown-repo",
[dev8692]:~>
```

Reviewers: #mercurial, rmcelroy, quark

Reviewed By: quark

Subscribers: quark, mitrandir, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3643342

Tasks: 11194713

Signature: t1:3643342:1470163958:8d56291dad575c7ea7186cf5d00420798c5303ab
2016-08-02 11:59:52 -07:00
Durham Goode
c8b5839178 statprofext: import statprof from standard python path
We temporarily changed the statprof import path when it was accidentally moved
into hgext3rd. Since we've moved it out, let's fix the import here too.
2016-08-01 12:00:18 -07:00
Jun Wu
915ce74059 sshaskpass: handle the case that __file__ ends with .pyo
Summary:
We need to get the `.py` file name from `__file__`, which could be `.pyc`
and `.pyo`. Previously only `.pyc` is handled. This diff handles `.pyo`
as well.

Test Plan: Run existing tests

Reviewers: ttung, mjpieters, #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3627929

Signature: t1:3627929:1469625182:1c0f5f720fe7eb4cbe3e6abd8e9407ced617d2a0
2016-07-27 14:05:16 +01:00
Jun Wu
34c0e30789 sshaskpass: enable ssh to ask password if running with chg
Summary:
During chg pull or push over ssh, ssh is started by chgserver which does not
have a controlling tty. Therefore the ssh process won't be able to ask for
passwords interactively.

This is actually a hard issue because an unprivileged process without a ctty
cannot attach to a ctty of another process.

The discussion at upstream tends to make it clear it's part of limitations
of chg. Therefore if we decide to workaround it, it has to live outside core,
thus fb-hgext.

GUI ssh-askpass is actually a good and clean choice. See D3510178 and D3515604.
However, they are for OS X but not Linux.

This diff is a very hacky solution to make ssh-askpass works in terminal.
It starts a "tty server" providing tty I/O fds and set `SSH_ASKPASS` to use a
custom script talking to the "tty server".

Test Plan:
Run the new test. Start a sshd locally and try:
```
$ hg push ssh://root@localhost/tmp
pushing to ssh://root@localhost/tmp
root@localhost's password:
remote: Permission denied (publickey,password).
abort: no suitable response from remote hg!

$ chg push ssh://root@localhost/tmp
pushing to ssh://root@localhost/tmp
==== SSH Authenticating ====
root@localhost's password:
remote: Permission denied (publickey,password).
abort: no suitable response from remote hg!
```

Reviewers: #mercurial, ttung, mpm

Reviewed By: mpm

Subscribers: durham, mpm, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3577509

Tasks: 12029680

Signature: t1:3577509:1469467700:cd93565bd47e535bb4cb41fcdaa39e45dddfae28
2016-07-17 20:35:34 +01:00
Martijn Pieters
481b9898e0 Remove the journal extension, fully upstreamed
Summary:
All of hg journal is now fully upstreamed to mercurial core and remotenames.
Remove the outdated copy here.

Test Plan: --

Reviewers: #mercurial, ttung

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3611123

Tasks: 10804988
2016-07-24 00:01:33 +01:00
Durham Goode
b3a9a1ebef statprof: move statprof out of hgext3rd
Summary:
Since statprof is not an extension, it needs to be treated like a normal python
module.

Test Plan:
```
~/local/fb-hgext> python setup.py build
~/local/fb-hgext> ls build/lib.linux-x86_64-2.6/
cfastmanifest.so  hgext3rd/         sqldirstate/
fastmanifest/     phabricator/      statprof.py
```

Reviewers: ttung, #sourcecontrol, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3607202

Signature: t1:3607202:1469216683:9aa08aee567425c239ddc5d1bb68fd892a8cf6a9
2016-07-22 12:44:59 -07:00
Mateusz Kwapich
1d2abfa980 statprof: change the hotpath to be less biased towards IO
Summary:
Let's use deltas between the subsequent samples instead of sample count
to count the time spent.

Rationale:
When the process is IO blocked the other thread doing the sampling can be waken
up much more often (GIL is not held) causing the profiler to collect much more
samples in that state.

Test Plan:
works in my sandbox
do we have any tests for statprof.py?

Reviewers: durham

Reviewed By: durham

Subscribers: quark, mjpieters, #mercurial

Differential Revision: https://phabricator.intern.facebook.com/D3551431

Signature: t1:3551431:1468543377:f6b86245c957dd59d0334dbb1898f6ad0bb8c617
2016-07-20 13:41:35 -07:00
Jun Wu
b0380393f2 phrevset: do not depend on hgsubversion
Summary:
`import hgsubversion` can error out if demandimport is disabled and svn
bindings are not found. In that case, we should be able to continue and
just skip handling svn revisions.

Test Plan: Code review

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3594810

Signature: t1:3594810:1469042572:3f9ab69f5503f6925f455769a78dac42a47087de
2016-07-20 20:09:29 +01:00
Jun Wu
8a3a99ba21 hgext: move single file extensions to hgext3rd
Summary:
Be a better citizen under system python path.

Fix all tests issues and change setup.py to use glob pattern to include
all extensions.

Test Plan:
Run tests and `make local`.
Also build and install the package and run `hg sl` in major repos.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3534311

Signature: t1:3534311:1468275426:fe122646c8bd6c541e1889e73e9df28f86747ff2
2016-07-08 13:15:42 +01:00