Commit Graph

1990 Commits

Author SHA1 Message Date
Tony Tung
f22b6105d5 [ctree] remove hack for clang
Summary: D3730823 removes the need for it.

Test Plan: compiles

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: mathieubaudet, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3730880

Signature: t1:3730880:1471467917:2b202c5cab1a10fcfe5899670b11bc44740983f7
2016-08-22 13:58:10 -07:00
Tony Tung
efd9e50796 [ctree] remove dead code
Summary: This is not used.

Test Plan: compiles

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3726149

Signature: t1:3726149:1471401105:3c27f091b180dd14493a3e50d4043b10b077142b
2016-08-22 13:57:58 -07:00
Tony Tung
fd38308160 [ctree] add a Manifest pointer to ManifestEntry
Summary:
If a manifest has already been loaded for a ManifestEntry, we should cache that entry and reuse it.  Two reasons:

1) if someone makes a modification to a tree, we need to persist that.
2) better performance.

Missing in this diff: memory cleanup

Test Plan: compiles

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: durham, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3725842

Signature: t1:3725842:1471401136:cbf4a987c35ea19ca432059cc15e299f0aa5568b
2016-08-22 13:57:41 -07:00
Tony Tung
f1ce770ce5 [ctree] convert ManifestFetcher to standard declaration + definitions layout
Summary: This allows us to use ManifestFetcher inside the ManifestEntry class.

Test Plan: compiles

Reviewers: #fastmanifest, durham

Subscribers: durham, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3725838
2016-08-22 13:55:43 -07:00
Tony Tung
71837a8c16 [ctree] transact in manifests rather than manifestkeys
Summary: This will allow us to replace portions of manifests with in-memory representations.

Test Plan: existing script doesn't crash.

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: durham, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3725831

Signature: t1:3725831:1471400740:4d9891c01f8567f4ceab76d6bd36e7dc595de4a6
2016-08-22 13:55:19 -07:00
Tony Tung
0727c6d030 [ctree] remove the need for a nextentrystart by returning the end of the entry
Summary: `parseptr` refers to where the parsing is going next.  This simplifies the code and makes ManifestEntry less tied to the original memory allocation.

Test Plan: the existing test i've been running doesn't crash.

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3717940

Signature: t1:3717940:1471383988:9b718e137b8ffaf8aad09f78f15791eec57fce6e
2016-08-22 13:42:55 -07:00
Tony Tung
dc8b5e0229 [ctree] parse manifests at load time
Summary: Parse manifests when we construct the Manifest object.  This allows us to do manipulations to the Manifest object and not have to deal with a parallel set of data structures.

Test Plan:
`PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotefilelog:~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/ valgrind ~/work/mercurial/facebook-hg-rpms/hg-crew/hg  --config extensions.perftest=~/work/mercurial/facebook-hg-rpms/remotefilelog/tests/perftest.py testtree --kind flat,ctree,fast --test fulliter,diff,find --build "@~2::@"`

note that I'm running this with Valgrind.

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: durham, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3717936

Signature: t1:3717936:1471383856:012634c1e59f1da9fc1e5a918e7f7d99d30d6992
2016-08-22 13:42:40 -07:00
Tony Tung
4245bfb5d8 [ctree] get rid of unused index field
Summary: It's not necessary for the direction we're going in.

Test Plan: compiles

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3717930

Signature: t1:3717930:1471383901:d8bfddb7ea7fb08d26315cbfce7af65d14662bc8
2016-08-22 13:42:27 -07:00
Tony Tung
56a6784b4d [ctree] Manifest objects are now allocated on the heap to permit them to be persisted
Summary:
We need manifest objects to be able to stick around in memory, because now they have overrides and all that other good stuff.

This probably introduces a metric ton of memory leaks, but we'll slowly whittle them down.

Test Plan: same script.

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: durham, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3717304

Signature: t1:3717304:1471385439:9269ab248d233a970c6725dbb4ca3eb661f6a96e
2016-08-22 13:41:27 -07:00
Tony Tung
a89dcaf23f [ctree] remove InMemoryManifest
Summary: All manifests will be in-memory to simplify the design.

Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotefilelog:~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/ ~/work/mercurial/facebook-hg-rpms/hg-crew/hg  --config extensions.perftest=~/work/mercurial/facebook-hg-rpms/remotefilelog/tests/perftest.py testtree --kind flat,ctree,fast --test fulliter,diff,find --build "@~2::@"`

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: durham, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3717296

Signature: t1:3717296:1471383272:0fca293beaf811ef39de87300437b0bf880e1ca7
2016-08-22 13:40:57 -07:00
Tony Tung
25b86ae774 [ctree] use a common idiom for naming structures
Summary: xxx is the C++ class, py_xxx is the python wrapper for it.

Test Plan: make local

Reviewers: #fastmanifest, akushner

Reviewed By: akushner

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3701536

Signature: t1:3701536:1471150787:5f3dfc3a65360233f9ee2ea1757ed4f368f70d62
2016-08-22 13:40:15 -07:00
Tony Tung
ff44d5f816 [ctree] handle construction and destruction in treemanifest object
Summary: py_treemanifest is initialized to 0s.  We initialize the `tm` field by explicitly calling the constructor in `treemanifest_init` and we destroy everything by explicitly calling the destructor in `treemanifest_dealloc`

Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotefilelog:~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/:~/work/mercurial/facebook-hg-rpms/lz4revlog/ /opt/local/bin/python2.7 ~/work/mercurial/facebook-hg-rpms/hg-crew/hg --config extensions.perftest=~/work/mercurial/facebook-hg-rpms/remotefilelog/tests/perftest.py testtree --kind flat,ctree,fast --test fulliter,diff,find --build "master~5::master"` in fbsource

Reviewers: #fastmanifest

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3730823
2016-08-22 13:39:35 -07:00
Tony Tung
536f6aaaee [ctree] create a py_treemanifest struct that handles the exclusively-python stuff
Summary: treemanifest will become the C++-only treemanifest object.

Test Plan: `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotefilelog:~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/ ~/work/mercurial/facebook-hg-rpms/hg-crew/hg  --config extensions.perftest=~/work/mercurial/facebook-hg-rpms/remotefilelog/tests/perftest.py testtree --kind flat,ctree,fast --test fulliter,diff,find --build "@~2::@"`

Reviewers: #fastmanifest, durham

Subscribers: durham, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3701468
2016-08-22 13:37:53 -07:00
Zach Amsden
f115cd74e9 [fastlog-hg] Remove unused imports
Summary: No need for cruft

Test Plan: Load in Atom, no more complaints about unused imports

Reviewers: durham, rmcelroy, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3733509

Tasks: 12341014

Signature: t1:3733509:1471519620:08e896aaff65b545d7d16e5d46ba6a8e2a2bc186
2016-08-19 11:59:10 -07:00
Tony Tung
11823e1845 [ctree] move conversion methods to convert.h
Summary:
yo refactoring.

Depends on D3699264

Test Plan: compiles

Reviewers: #fastmanifest, akushner

Reviewed By: akushner

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3699498

Signature: t1:3699498:1471151476:b08a69fb474413c9fc780eaffba4058aca00dacc
2016-08-18 14:44:26 -07:00
Tony Tung
3dbcbb215f check-code: fix errors that have creeped in
Test Plan: run check-code

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3733781

Signature: t1:3733781:1471542457:bccf5d849931040f5168b64185e0eb61d94dbece
2016-08-18 14:38:54 -07:00
Jeremy Fitzhardinge
1fd4455c73 Implement 'hg uncommit'
Summary:
TODO:
 - fix handling of empty commit (leaves stray heads)
 - add tests

Test Plan: Add mercurial unit tests

Reviewers: #mercurial, ttung, rmcelroy, durham

Reviewed By: durham

Subscribers: quark, rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3610409

Tasks: 12340638

Signature: t1:3610409:1471398045:355f20502d917374e99feee4681a7ef6f853ad6c
2016-08-17 13:43:25 -07:00
Jun Wu
08ac5e7826 setup: check Cython version
Summary:
The minimal version of Cython to build linelog is 0.22. Add a check so people
know it's not a source code issue, but Cython needs upgrade.

Test Plan:
Run with different Cython releases and make sure it complains when Cython
version < 0.22. Also make sure it works with strange versions like "0.25a1".

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3706559

Signature: t1:3706559:1471463690:92b981e2df63ad63ab55edb5e0b937455254351d
2016-08-17 20:55:54 +01:00
Tony Tung
8894da1d35 [ctree] fix comments for clarity/grammar
Test Plan: meh.

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3718053

Signature: t1:3718053:1471384016:be6ad07218e90e03708eae3c9f8a9bc4b1c67520
2016-08-17 12:42:42 -07:00
Tony Tung
1f25a00b79 [ctree] fix comment style
Summary: Matches javadoc/doxygen style

Test Plan: compiles

Reviewers: #fastmanifest, akushner

Reviewed By: akushner

Subscribers: akushner, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3699264

Signature: t1:3699264:1471152249:d917e1f53426ace7b2e26df2cc72379fa97f93c1
2016-08-17 12:41:54 -07:00
Tony Tung
1083a7147d [ctree] if the flag is unset, return the null character as the flag
Summary: If the flag is not present, then `flag` field is set to NULL.  In that case, the current code will segfault.  Now we will assign `\0` to `*resultflag`.

Test Plan:
run `PYTHONPATH=~/work/mercurial/facebook-hg-rpms/remotefilelog:~/work/mercurial/facebook-hg-rpms/fb-hgext/:~/work/mercurial/facebook-hg-rpms/remotenames/ ~/work/mercurial/facebook-hg-rpms/hg-crew/hg  --config extens
.perftest=~/work/mercurial/facebook-hg-rpms/remotefilelog/tests/perftest.py testtree --kind flat,ctree,fast --test fulliter,diff,find --build "@~2::@"`
 without crashing.

Reviewers: #fastmanifest

Differential Revision: https://phabricator.intern.facebook.com/D3700352
2016-08-17 12:37:19 -07:00
Tony Tung
d1a61c4d0f [ctree] move the treemf reference up to fileiter
Summary: This is the first step in disentangling the C++ code from the python interface.

Test Plan: `PYTHONPATH=~/remotefilelog/build/lib.linux-x86_64-2.6/ python ~/hg/hg --config extensions.remotefilelog=~/remotefilelog/remotefilelog --config extensions.perftest=~/remotefilelog/tests/perftest.py testtree --kind flat,ctree --test fulliter,diff,find --build "master~5000::master"`

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: durham, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3700627

Signature: t1:3700627:1471382403:d8dc6dc5c295ec55878ca020b91fc0b30d930ce8
2016-08-17 11:48:01 -07:00
Tony Tung
c882541a19 CMakeLists.txt to build both ctreemanifest and cdatapack
Summary: Useful if you run CLion.

Test Plan: built everything.

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3700542

Signature: t1:3700542:1471382263:260eb0a1588480f1d4c798df60d559c63ed19c8a
2016-08-17 11:42:31 -07:00
Zach Amsden
092d645baa Fix comment for fastlog
Summary: Document how to enable the extension

Test Plan: Set this in my actual .hgrc and confirm it works.

Reviewers: durham, rmcelroy, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3721128

Signature: t1:3721128:1471342825:e274764e036f1c03b9f8795221af6fe587990340
2016-08-16 15:55:48 -07:00
Jun Wu
fab3975b47 sshaskpass: use mkdtemp to create temporary directory
Summary:
Before this patch, sshaskpass has a hardcoded tmp directory, which, if being
abused, can break sshaskpass from working. This diff fixes the issue by
using tempfile.mkdtemp.

Test Plan:
Run `chg push -r . ssh://root@localhost//tmp/foo -f --allow-ano` and make
sure ssh password prompt works. Also run the sshaskpass test.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3705674

Signature: t1:3705674:1471373405:aec54ac685b060efb616307552cebcccee5cebe8
2016-08-16 19:51:47 +01:00
Tony Tung
5d5892a422 update .hgignore
Summary: .idea is the CLion project directory.  .testtimes is dropped by the linter.

Test Plan: minimal

Reviewers: #fastmanifest, akushner

Reviewed By: akushner

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3700563

Signature: t1:3700563:1471150982:79e96a7f8930c6417fa528375884d288ab661a6a
2016-08-15 11:43:02 -07:00
Tony Tung
155c9bcabf basepack: handle race condition between incremental packing and pack loading
Summary: If someone removes the pack file, getpack will throw an IOError.  Catch it and don't bother trying to add the pack to the list of available packs.

Test Plan: pdb.set_trace() in this method, then remove a file while the debugger is halted.  continue without a traceback.

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3712310

Tasks: 12660285

Signature: t1:3712310:1471080740:85f2e3f49b09f348ec654362eadf5f1e689b8a19
2016-08-15 11:41:49 -07:00
Tony Tung
30ba9cdd24 [cdatapack] madvise the memory away
Summary: Once we're done reading the delta data, we madvise it away.

Test Plan:
dump all the hashes from a datapack into a separate file.  then run a script to fetch all the delta chains.  observed that the memory footprint did not increase significantly.

```
#!/usr/bin/env python

import binascii

import cdatapack

dp = cdatapack.datapack('/dev/shm/hgcache/fbsource/packs/8b5d28f7a5bd7391a0b060c88af8cca3af357c24')

for ix, line in enumerate(open('/tmp/hashes', 'r')):
    line = line.strip()
    dp.getdeltachain(binascii.unhexlify(line))
```

Reviewers: #fastmanifest, durham

Reviewed By: durham

Subscribers: durham, mitrandir

Differential Revision: https://phabricator.intern.facebook.com/D3686830

Signature: t1:3686830:1470716333:e8fc8e3e3fa29c1931f69222c17e10f519a4a8c2
2016-08-15 11:39:14 -07:00
Jun Wu
5aca2ebc2f test-tweakdefaults-bookmarks.t: fix typo 2016-08-15 18:45:19 +01:00
Jun Wu
e2e852370f linelogcli: add new command getalllines
Summary:
Since linelog has the new `getalllines` API, add a corresponding command
to support it.

Test Plan:
```
$ make && ./linelogcli /tmp/lll init annotate 0 replacelines 1 0:0 0:3 \
  replacelines 2 1:2 1:3 getalllines 0:0
init: okay
annotate: run annotate for rev 0
annotate: 0 lines, endoffset 1
replacelines: rev 1, lines 0:0 -> 0:3
replacelines: rev 2, lines 1:2 -> 1:3
getalllines: 5 lines
  0: rev 1, line 0, offset 3
  1: rev 2, line 1, offset 8
  2: rev 2, line 2, offset 9
  3: rev 1, line 1, offset 11
  4: rev 1, line 2, offset 5
```

Reviewers: #mercurial, simonfar, ttung

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716453

Signature: t1:3716453:1471265279:a8f625660ef47ca341ca8f5c4c7370b56277cef8
2016-08-15 12:47:14 +01:00
Jun Wu
d8d7af1727 linelog.pyx: add wrappers for the new API getallines
Summary:
Add `getoffset`, `getallines` to make the new C API `linelog_getalllines`
usable.

Test Plan: This helps the future smartfixup extension to pass its tests

Reviewers: #mercurial, ttung, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716447

Signature: t1:3716447:1471262262:2ad8a547bf956c01e39be3383dc1a50751729c85
2016-08-15 12:45:58 +01:00
Jun Wu
b30c3ad0e5 linelog.pyx: fix copyfrom
Summary:
`copyfrom` does not work because `rhs.buf` is invisible to `self`. Adding
a `readonly` keyword fixes it. Also add some sanity checks and remove an
unnecessary `;`.

Test Plan: `make local` and confirm `copyfrom` works.

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716444

Signature: t1:3716444:1471264420:1e58798222e1515ae6d5328ad6f2727cfe41cdb5
2016-08-15 12:43:44 +01:00
Jun Wu
3258b8586d linelog.pyx: run clear and annotate an empty linelog automatically
Summary:
A newly created empty `linelog` object is not usable (i.e.  calling
`annotate` or `replacelines` will raise an error) until `clear` is called.
And `replacelines` won't work until `annotate` is called. Therefore if we
know it's an empty linelog, just call `clear` and `annotate(0)` in
`__init__` to make it more friendly to use.

Test Plan:
`make local`, and confirm
`__import__('linelog').linelog().replacelines(1, 0, 0, 0, 1)` works.

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716442

Signature: t1:3716442:1471262694:96370f5d3074bc0941e8ee780181c6b688cff79c
2016-08-15 12:41:13 +01:00
Jun Wu
2c9fffd80e linelog: add a new API getalllines
Summary:
The current linelog APIs are designed purely for the "annotate" operation.
It only supports getting lines from one single revisions, which is what
`annotate` does.

However the data structure is also useful to provide lines from different
revisions. Say if you want to know all lines from all revisions for a given
function. Or if you want to confirm if a chunk is "continuous" - nobody
inserts new lines among them. This diff adds the missing API, `getalllines`.

It returns information about all lines the linelog tracks. The caller could
provide an optional interval to limit the lines returned.

Its parameters use low-level raw offsets instead of line numbers, because it
gives the caller more control. The API don't need to handle open or close
interval issues that line numbers may have (see the previous diff about
deletion blocks).

This makes `lineinfo.offset` no longer internal-only - we have exposed the
offset concept to the API parameters. Therefore the comments are updated.

This also requires a clear way to distinguish unconditional jumps from
conditional one. We use `JGE 0` as unconditional jumps. To ensure this,
make `replacelines` reject 0 revision.

Test Plan: `cd linelog && make`

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716439

Signature: t1:3716439:1471265245:5a4b2f585284cba71ca3907c8e8826b2b00a4b49
2016-08-15 12:31:29 +01:00
Jun Wu
c5819e914f linelog: extract appendline logic from annotate
Summary:
The `appendline` logic is used twice in `annotate`: inside and outside the
`for` loop. It will be used again in the following changes. Therefore make
it an inline function.

Test Plan: `cd linelog && make`

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716438

Signature: t1:3716438:1471263197:6add9cfc0a9d712258dd2da332aaf306cb1af71a
2016-08-15 12:14:07 +01:00
Jun Wu
b48ea46da6 linelog: be conservative when deleting a chunk from an old revision
Summary:
This is a subtle change. Consider the following case, where revision 1 has 3
lines and revision 2 has 1 line inserted between the second and the third
lines of revision 1:

  0: +---- INSERTED BY REV 1
     |                          --+             --+
  1: |     LINE REV=1 LINENUM=0   | DEL BLOCK A   | DEL BLOCK B
  2: |     LINE REV=1 LINENUM=1   |               |
     |                          --+               |
  3: | +-- INSERTED BY REV 2                      |
  4: | |   LINE REV=2 LINENUM=2                   |
     | +--                                      --+
  5: |     LINE REV=1 LINENUM=2
     +----

Now what if the user has run `annotate(rev=1)` and wants to delete the first 2
lines, by calling `replacelines(a1=0, a2=2, ...)`?

The current code would use "DEL BLOCK B", which is more efficient because
the block contains more instructions, and more simple because it is just a
JGE to the a2 address.

While that works fine for the "appending" case (always edit new revisions),
it deletes future lines between the second and third line (in this case,
"LINE REV=2 LINENUM=2"). This is undesired by some users, namely smartfixup.

Surely we can add a boolean flag to the `replacelines` API to decide whether
A or B will be used. But that is too low-level and `replacelines` already
has too many parameters. Today we don't see other requirements to choose
between them precisely. Therefore, just change the default behavior, when
deleting chunks from old revisions to "DEL BLOCK A".

Test Plan: This change is needed to help pass the future smartfixup tests

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716435

Signature: t1:3716435:1471262664:0ae40d95d6389db733a88d9fafe19962d9166e35
2016-08-15 12:08:06 +01:00
Ryan McElroy
c132ca0439 tests: check for sparse in hgext3rd 2016-08-11 07:19:06 -07:00
Jun Wu
372dc6028a fileserverclient: remotefilepeer could have not implemented "cleanup"
Summary:
We actually have no idea what "hg.peer" will return and should check
if it has a "cleanup" method or not before wrapping it.

Test Plan: Code Review

Reviewers: ttung, #mercurial, rmcelroy

Reviewed By: rmcelroy

Differential Revision: https://phabricator.intern.facebook.com/D3703201

Signature: t1:3703201:1470924859:852eaf275c89ceced285a4b74d09938e489d9ee0

Blame Revision: D3685587
2016-08-11 15:14:54 +01:00
Jun Wu
0d3a392454 hgignore: ignore linelog non-source files
Summary: This diff adds an `.hgignore` file for the `linelog` directory.

Test Plan: `make local`, and run `hg status`, make sure no unwanted files are printed.

Reviewers: #mercurial, ttung, akushner

Reviewed By: akushner

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3699575

Signature: t1:3699575:1471151389:7e4de7b75d671d3f0db18337e57c65e9972de059
2016-08-10 22:32:21 +01:00
Mateusz Kwapich
675561c76f sqldirstate: call the registered callbacks on wd parent change
Summary:
This diff together with patches introducing the mentioned callback in upstream
will fix the journal under sqldirstate.

Test Plan: journal tests passed, the rest is currently running - will update the test plan soon

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3690878

Signature: t1:3690878:1470800077:3fe48e03bb65eaeed3a7326b6e7d392abe7efd8d
2016-08-15 03:46:52 -07:00
Zach Amsden
f1ecf43a83 Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior.  To do this, we backtrackfrom the current head to find
 all draft revisions along the path, then find the common public ancestor of
those.  Once we find that, we can begin paging in results from the scmquery
service.

Pretty much a working diff at this point.  Limits and boundary conditions
have not been fully tested.  Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that.  Could also be an issue with conduit.  Other than that,
things seem to work.

Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):

  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a

  real	0m1.895s
  user	0m0.003s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b

  real	0m1.308s
  user	0m0.005s
  sys	0m0.001s
  [zamsden@devbig192.prn1 ~/local/fbcode] diff a b
  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c

  real	0m7.320s
  user	0m0.004s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbcode] diff a c




Testing --user option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a

  real	0m2.765s
  user	0m0.002s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b

  real	0m23.247s
  user	0m0.005s
  sys	0m0.001s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing same output enabled / disabled for -X option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a

  real	0m1.292s
  user	0m0.002s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b

  real	0m2.697s
  user	0m0.002s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing -k option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a

  real	0m1.174s
  user	0m0.003s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b

  real	0m1.259s
  user	0m0.004s
  sys	0m0.002s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing -I option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a

  real	0m1.473s
  user	0m0.005s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b

  real	0m2.911s
  user	0m0.003s
  sys	0m0.002s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]




Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a

  real	0m2.892s
  user	0m0.003s
  sys	0m0.006s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b

  real	0m2.575s
  user	0m0.697s
  sys	0m0.077s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c

  real	0m7.339s
  user	0m0.003s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a c
  [zamsden@devbig192.prn1 ~/local/fbsource]

Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash

Reviewed By: stash

Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv

Differential Revision: https://phabricator.intern.facebook.com/D3634075

Tasks: 12341014

Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-12 15:52:50 -07:00
Jun Wu
8813967d50 linelog: add a test editing lines randomly
Summary:
The test will call "replacelines" with random arguments, keeps track of the
content of "lines" on its own. Then it compares its own "lines" with linelog's
annotateresult - should be the same. After that, it verifies the content of
old revisions can be retrieved by using "annotate" correctly.

Test Plan: Run this test

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3709431

Signature: t1:3709431:1471012434:68ca06c0b3b2705740375c32acac8012ded404a5
2016-08-12 15:22:16 +01:00
Jun Wu
5b0b861096 sshaskpass: make the prompt similar to systemd
Summary:
This diff makes the prompt similar to what systemd does. See the screenshot:

{F62774663}

It solves an issue that with echo disabled, the user won't know whether they
have pressed ENTER or not - the cursor won't move. Now it explicitly prints
"AUTHENTICATION COMPLETE".


Test Plan: As the screenshot, and also run the sshaskpass test.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3705703

Signature: t1:3705703:1471008095:863ff9820b87ea3f4631295b10472802fb5e459f
2016-08-11 21:56:31 +01:00
Jun Wu
d4366acbd5 sshaskpass: use a standalone script
Summary:
Before this patch, sshaskpass will set SSH_ASKPASS to itself, aka. `__file__`.
This won't work if sshaskpass.py gets installed by setup.py because setup.py
will remove its `+x` bit.

Test Plan:
Run `chg push -r . ssh://root@localhost//tmp/foo -f --allow-ano` and make
sure ssh password prompt works.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3705657

Signature: t1:3705657:1471008122:9cfa3adf078e4bbe8f6b6ba05d1fb6be513d3e71
2016-08-11 21:42:54 +01:00
Jun Wu
6015a95c00 tweakdefaults: make "bookmarks" use the unfiltered repo
Summary:
We are seeing perf issues with hidden/obsolete handling.
`hg bookmark` is a frequently used command and by making it use the unfiltered
repo, it could be 200ms-300ms faster.

Test Plan: Added a new test

Reviewers: #mercurial, mitrandir, ikostia, ttung

Reviewed By: mitrandir, ikostia

Subscribers: mitrandir, rmcelroy, akushner, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3692968

Signature: t1:3692968:1470777864:72ad5d0ffb52ecfcaaa607082693b88319d778fd
2016-08-11 18:43:06 +01:00
Ryan McElroy
2a07de293c tests: import simplecache from correct place 2016-08-11 06:33:47 -07:00
Ryan McElroy
3682450721 tests: update pushrebase test with new string 2016-08-11 06:33:47 -07:00
Jun Wu
5cd2c64da0 setup: build linelog
Summary:
This diff builds linelog. It brings Cython as a build dependency as linelog
CPython wrapper is written in Cython. Alternatively we can check in the
generated, non-human-readable `.c` code if we get feedback that the Cython
dependency is making things hard.

Note that `setuptools` (Python 2.6 version) has issues with Cython therefore
removed.

Test Plan: `make local`, then check `linelog.so` is built and `import linelog` works.

Reviewers: #mercurial, rmcelroy, ttung

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3669753

Signature: t1:3669753:1470403459:5bf81861b31deed5076c2322c4015d5dd4b4673e
2016-08-10 22:26:05 +01:00
Jun Wu
cf9686ff3a morestatus: rebase takes precedence over update
Summary:
This happens when pressing Ctrl+C during "hg rebase". We should show
"interrupted rebase" instead of "interrupted update".

Test Plan: Added a new test

Reviewers: ttung, #mercurial, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3697984

Signature: t1:3697984:1470863868:acd67eeb5099c73ea7df174b3e1dbcbf9ac54944
2016-08-10 22:19:30 +01:00
Jun Wu
e4089b5388 linelog: resolve compiler warnings
Summary:
This diff resolves some warnings when building on CentOS 6:

```
linelog/pyext/../linelog.c: In function 'replacelines':
linelog/pyext/../linelog.c:290: warning: unused variable 'result'
....
linelog/pyext/linelog.c: In function '__pyx_f_7linelog_11_filebuffer_resize':
linelog/pyext/linelog.c:3253: warning: ignoring return value of 'ftruncate', declared with attribute warn_unused_result
```

Test Plan:
Run `make local` with the linelog `setup.py` patch applied and make sure
there are no warnings.

Reviewers: #mercurial, rmcelroy, ttung

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3699378

Signature: t1:3699378:1470863807:10dc21033ab7301cb3d56e04de1c96f4d0287b6e
2016-08-10 22:18:59 +01:00