Commit Graph

1242 Commits

Author SHA1 Message Date
Stanislau Hlebik
019c55f351 infinitepush: indexapi and store
Summary:
Code was written by Laurent, Durham and Mateusz on hackathon.
Index stores metadata, store stores bundles.
This diff adds interfaces and simple filesystem implementations

Test Plan:
cd tests
py.test *.py

Reviewers: durham, rmcelroy, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3741963

Tasks: 12479677

Signature: t1:3741963:1472207457:c9e7fb6163ef3cd6fdc1c4bd5113c623169c3a7c
2016-08-26 04:08:28 -07:00
Jun Wu
f1c3dc9f91 fastannotate: implement a simple revision map
Summary:
To use linelog, we need to be able to translate between hg commit hashes and
linelog revision numbers. This diff implements such a revmap using the most
direct way.

The revmap also contains an extra "flag" for each revision, which will be used
to mark if the revision is in the main branch or not, to handle merge commits.

Test Plan:
`import revmap` from IPython and test its interface manually. Also have a
simple script to get some idea about its perf with 10000 revisions:

```
import contextlib, time, os, random, revmap, sys

@contextlib.contextmanager
def benchmark(msg):
    sys.stderr.write('%s: ' % msg)
    t1 = time.time()
    yield
    t2 = time.time()
    sys.stderr.write('%f seconds\n' % (t2 - t1))

def randomid():
    return ''.join([chr(random.randint(0,255)) for _ in xrange(0, 20)])

rm = revmap.revmap('revmap1')

with benchmark('insert 10000 random revisions'): # ~0.3 seconds
    for i in xrange(0, 10000):
        rm.append(randomid(), flag=1, flush=False)

with benchmark('writing to disk'): # 0.02 seconds
    rm.flush()

os.rename('revmap1', 'revmap2')
with benchmark('loading'): # ~0.015 seconds
    rm = revmap.revmap('revmap2')
```

Reviewers: ttung, #sourcecontrol, ikostia

Reviewed By: ikostia

Subscribers: ikostia, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3709706

Signature: t1:3709706:1471936489:0bbe35ed39a2af3f06e1000c4f9674149ad43995
2016-08-23 16:24:13 +01:00
Siddharth Agarwal
14a3b298c3 mergedriver: add a test for bug in merge action list
Summary: Test for bug in https://www.mercurial-scm.org/pipermail/mercurial-devel/2016-August/087511.html.

Test Plan: Ran the test.

Reviewers: #sourcecontrol, ttung, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3762235

Signature: t1:3762235:1472028110:ad76c9e39f9226f83bd6cb294459936df0b8a93e
2016-08-25 12:20:07 -07:00
Zach Amsden
ddc1ae4919 Fix clock now
Summary:
One place we are losing time in the profiling extension is when
our profiling thread is interrupted between two invocations of clock()
that are supposed to fetch the same time.  Fix to use now = clock() and
use the same time in both places.

Test Plan:
Running with upstream and custom extensions, I no longer observe
suspended execution times large than accumulated execution times.

Reviewers: ttung, durham, mitrandir, #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3772529

Tasks: 12879683

Signature: t1:3772529:1472151936:192ada10ac4738755c8fd27278dc7ac90faa17be
2016-08-25 12:07:11 -07:00
Mateusz Kwapich
83606d1a5e pushrebase: fix tests
Summary:
We need to fix tests due to de-facto HG_URL variable semantics change in
https://selenic.com/hg/rev/a5fd89db5549

Test Plan: run-tests.py test-pushrebase.t

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3771208

Signature: t1:3771208:1472141889:562c84ca92b91d2dbe60decea53349f53a9676c4
2016-08-25 10:54:18 -07:00
Jun Wu
80f57e9dc3 [tests] add missing PYTHONPATH changes
Summary:
The built linelog.so cannot be discovered automatically. Change `PYTHONPATH`
to make sure it can be found.

Also fix the issue that the rename change misses some places in the test.

Test Plan: `unset PYTHONPATH` and run the touched files.

Reviewers: #sourcecontrol, ttung, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3771583

Signature: t1:3771583:1472145156:d05a1bf0d5920dfd0131acd1998698f2dddae1f8

Blame Revision: D3763992
2016-08-25 17:42:23 +01:00
Stanislau Hlebik
437b663de7 infinitepush: initial commit
Summary: I'll later add most of your hackhaton hackbranch code to this project with a few small fixes

Test Plan: No need

Reviewers: durham, rmcelroy, mitrandir

Reviewed By: rmcelroy, mitrandir

Subscribers: quark, rmcelroy, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D3736089

Signature: t1:3736089:1471876504:98c4efc3bdf66d60f98eed11a3f13bae56fdc716
2016-08-25 09:09:47 -07:00
Mateusz Kwapich
4913062607 check-code fixes
Test Plan: check-code passing

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3771131

Signature: t1:3771131:1472139248:f8ea463552fc0d2261456050e9bf513f26eb727c
2016-08-25 09:04:43 -07:00
Durham Goode
a2f69f2e93 fastmanifest: support tree manifests in hybrid manifest
Summary:
This adds support for the optional ctreemanifest type. If the module is
available and if `fastmanifest.usetree` is enabled (default: False), we'll use
tree manifests before falling back to flat manifests.

Test Plan: Ran the ctreemanifest perf suite and the fastmanifest test suite

Reviewers: #fastmanifest, ttung

Reviewed By: ttung

Subscribers: ttung, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3754309

Signature: t1:3754309:1471929802:3ccba8ce6160b7a6d20b599626efd5876bd969f6
2016-08-24 11:31:06 -07:00
Durham Goode
41e13076e1 fastmanifest: fix broken assert
Summary:
This assert is meant to check that at least one source was provided to the
hybridmanifest. The old version was broken though.

The test requires an update because it attempts to construct a hybridmanifest
with no source.

Test Plan: Ran the tests

Reviewers: #fastmanifest, ttung

Reviewed By: ttung

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3760229

Signature: t1:3760229:1472062113:695085444915cb15a93db7f317580dcffc2f115d
2016-08-24 11:31:06 -07:00
Durham Goode
cede3a3be7 fastmanifest: add config option for disabling cache usage
Summary:
This adds the `fastmaniest.usecache` config option (default: True) which lets
users disable the use of the fastmanifest cache without disabling the entire
extension. This will be useful in a future patch where we'll use this same
hybrid manifest infrastructure to allow swapping in tree manifests (even on
repos that don't want fast manifest caches).

Also contains a minior fix to an assert that didn't appear to make sense (I
think it should be asserting that at least one data source is provided).

Test Plan: Ran the ctreemanifest perf suite and the fastmanifest test suite

Reviewers: #fastmanifest, ttung

Reviewed By: ttung

Subscribers: ttung, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3754301

Signature: t1:3754301:1471990417:0cde06f271d10c7f0c839e4f644310a4f525f3f6
2016-08-24 11:31:06 -07:00
Jun Wu
0184b72cf7 Rename "smartfixup" to "absorb"
Summary:
This seems to be an elegant choice. It does not conflict with other names,
and it does explain the concept to some extent vividly.

Credit goes to [trousers at lobste.rs](https://lobste.rs/s/nws1uj/help_us_name_new_mercurial_feature/comments/u4tsw7#c_u4tsw7).

Test Plan: Run the existing tests.

Reviewers: #sourcecontrol, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3763992

Signature: t1:3763992:1472049269:5bb9de3b7b3a0f0dd9587b952151fc38351bc822
2016-08-24 16:28:47 +01:00
Jun Wu
350dae5587 smartfixup: add a config option to make it a subcommand of "amend"
Summary:
Some people think this command should be a part of "amend". If you check the
meaning of the "amend" English word, that makes sense. But if you look at
the actual "amend" command, there are significant differences:

1. smartfixup rewrites a stack of changesets, and can even delete
   changesets, where amend only rewrites the working directory parent.
2. smartfixup is best-effort, i.e. does not guarantee that 100% of
   user-requested modifications will be included, where amend will just take
   100% (with "-i", it's 100% chunks selected by the user).
3. a lot of "amend" flags do not make much sense to smartfixup, like message
   editing (designed to edit a single changeset), "--addremove", "--secret",
   etc.
4. literally, smartfixup shares little code with the existing "amend" logic.
   "amend" is part of "fbamend" or "evolve". this extension should not
   depend on any of them.

So it's cleaner to be a separate command, not a part of `amend`.

However, I think it makes sense to have an option to satisfy those who want
to use "amend". So they can use "amend --related", "amend --fixups",
"amend --stack", "amend --auto" or whatever they choose. This diff adds such
a config option. We may also ship such a config option to make the command
easier for discovery.

Note the "amend" version is slightly different from the original smartfixup
command. The former targets basic users who expect amend to take all of
their changes, while the latter targets power users understanding what's
going on.

Therefore, the "amend" version will print extra information about what
changes are ignored, for example:

```
# changes not applied and left in working directory:
# M a : 1 modified chunks were ignored
# M c : unsupported file type (ex. binary or link)
# R b : removed files were ignored
```

To support the above change, `fixupstate.status` was added to avoid a second
"status" run and handles the "status" with interactive mode correctly. An
issue about symbolic links being added to `fixupstate.paths` was fixed by
the way.

Test Plan: Run the newly changed test.

Reviewers: #sourcecontrol, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3760498

Signature: t1:3760498:1472052376:7ddbfe763c7327d044b0d471c31a58fcb1e21dac
2016-08-24 16:28:15 +01:00
Zach Amsden
f99540d161 [fastlog-hg] Thread safety
Summary:
Sharing repo.changelog across threads is problematic in mercurial
since it may internally cache things, and clearing cache while another
thread is executing is not nice.  In particular, code such as the following
is inherently raceful when sharing changelogs (or anything derived from
revlog) across threads:

    if self._cache:
        if self._cache[0] == node:
            return self._cache[2]
        cachedrev = self._cache[1]

Reworked things a bit to make sure the spawned threads have their own
newly created changelogs and member functions internally referencing them.

Test Plan:
No longer seeing random exceptions being thrown from local log
threads (which terminates iteration)

Reviewers: rmcelroy, #mercurial, ttung, quark, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3733592

Tasks: 12341014

Signature: t1:3733592:1471908572:877b56241e56b6ea92c76023460d4fbe80a4263d
2016-08-23 11:46:34 -07:00
Zach Amsden
205a3b8a81 [fastlog-hg] Better multi-path and file support for fastlog
Summary:
The multi-path support is pretty shameful; we can
do much better by issuing multiple queries and combining
results.  File support was also missing.

Test Plan:
Ran fastlog test.  Need to update test to test files.

P56589012

Reviewers: quark, rmcelroy, durham

Reviewed By: durham

Subscribers: stash, mjpieters, jeroenv

Differential Revision: https://phabricator.intern.facebook.com/D3726318

Tasks: 12341014

Signature: t1:3726318:1471919354:e5e8f40a5b918b493e927c2d620da1fdefa1768c
2016-08-23 11:46:34 -07:00
Zach Amsden
fe94406e9a [fastlog-hg] Add localmatch function
Summary:
Bugfix - we were failing to check local patterns in the fastpath.
Pass a filtering function to the local log thread to limit its results.

Test Plan: Run local log

Reviewers: stash, quark, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3727573

Tasks: 12341014

Signature: t1:3727573:1471907418:37b83d6c02e61ce2705070c7a86dd815a1c1cb29
2016-08-23 11:46:34 -07:00
Zach Amsden
76529f6745 [fastlog-hg] Pass revfn
Summary: Pass revfn to the thread to unify response messages.

Test Plan: Run unit test, try fastlog

Reviewers: quark, durham, rmcelroy, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3727506

Tasks: 12341014

Signature: t1:3727506:1471855144:24e622c6add0584d37539ea5ee319e3454f5313e
2016-08-23 11:46:34 -07:00
Zach Amsden
31382f23a3 [fastlog-hg] Better handling of exceptions
Summary: Without this, exceptions are super hard to debug

Test Plan: Manually force exceptions with bogus array indices on non-arrays

Reviewers: quark, #mercurial, ttung, durham

Reviewed By: durham

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3744317

Tasks: 12341014

Signature: t1:3744317:1471919239:8c8d0fce6c212fd677df8b7f94a48d17dbe64b93
2016-08-23 11:46:34 -07:00
Jun Wu
36544a8c4d smartfixup: add an interactive mode
Summary:
Like `commit -i`, this diff adds an `interactive` mode. I'd like
to reuse some patching code but they are tightly coupled about
writing files to disk - while I want to handle everything in memory.

In the future I may want to add `memorystore` and `memorybackend`
to upstream's `patch.py`. But they are not available now therefore
the "patching" logic is a re-invent. Fortunately the logic is just
about 20 lines.

Test Plan:
Manually run `hg sf -i` with some manually crafted random cases.
Also make sure it works with both `text` and `curses` interfaces.

Reviewers: #mercurial, durham, ttung

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3723179

Signature: t1:3723179:1471582313:b10d278ca5b1a2aee6b41936f315a960edee92ed
2016-08-23 15:20:20 +01:00
Jun Wu
dea474ec1f smartfixup: add command line options to preview changes
Summary:
The `--print-changes` option will show which commit, a changed lines will be
applied to. Note it is a bit expensive so we only calculate the content when
being asked.

The `--dry-run` option will prevent the final commit operation.

Users can now use `-pn` to preview changes to understand what will be changed.

Test Plan: Run `test-smartfixup.t`

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3719673

Signature: t1:3719673:1471582117:7f3be3c3bf3829148c3d096d2aae6e7382449b98
2016-08-23 15:19:54 +01:00
Jun Wu
a51d83bec3 smartfixup: implement the command
Summary:
This is the last piece of the initial smartfixup implementation: the command.
A new test was added to test the command.

Test Plan: Run the newly added test

Reviewers: #mercurial, ttung, jbower, durham

Reviewed By: durham

Subscribers: akushner, simonfar, durham, jbower, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3717354

Signature: t1:3717354:1471629588:202e4a5820de472eb131eb968ec73f194d2c077d
2016-08-23 15:19:29 +01:00
Jun Wu
0087ee33bf smartfixup: implement fixupstate
Summary:
If `filefixupstate` is to `filecontext`, `fixupstate` is to `context`.
This does the actual committing, creating obsmarkers, moving bookmarks work.

Test Plan: A new test will be included in the next diff

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716954

Signature: t1:3716954:1471639171:9891a0c64365b0592f5a4d86e3c83d521b4c5bb2
2016-08-23 15:18:36 +01:00
Jun Wu
7683c76566 smartfixup: implement filefixupstate
Summary:
The `filefixupstate` object maintains states and contains logic related to
do "smartfixup" to a single file.

Test Plan: Added a new test

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716761

Signature: t1:3716761:1471919159:033e79e9d12307785863959dd5d82a7c3c647ff3
2016-08-23 15:17:42 +01:00
Jun Wu
98861bc86e smartfixup: add utilities
Summary:
This series is a rewrite of D3264203, using the `linelog` instead of
`histedit`. The new version has the following advantages:

  - handle more cases where it could be a merge conflict with a traditional
    merge algorithm. the old code tries to "predict" when a merge won't
    cause conflicts. but that's just kinda stupid - comparing with the new
    code using linelog - it is just impossible to produce merge conflicts.
  - do not touch working copy - all intermediate calculations and states are
    in memory. say goodbye to the clean working copy requirement of histedit.
  - handle line numbers cleanly and confidently - the old code struggles (but
    failed) to make line numbers correct after each edits, while it is
    actually hard as the "merge" operation could change line numbers in an
    unpredictable way. the new code using linelog can just edit chunks from
    an old revision, from the latest view without checking out that old
    revision first. therefore no need to mess up with line numbers.

This diff adds utility functions and classes used later. It introduces
unused `import`s but they will be used later.

Test Plan: `arc lint`

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716758

Signature: t1:3716758:1471556512:0fbff8f6fe4bc02d2120c723d2cc28676895c679
2016-08-23 15:16:15 +01:00
Adam Simpkins
b1540c8b8a Fix hgext3rd/__init__.py to make sure it is a namespace package
Summary:
According to the comments hg-crew, hgext3rd is supposed to be a namespace
package, so that extensions can be found in multiple different hgext3rd
directories.

However, the hgext3rd directory in fb-hgext has an empty __init__.py file,
which turns hgext3rd into a plain package instead of a namespace package,
preventing other hgext3rd directories from being found.

This fixes the __init__.py file to perform proper namespace package
initialization, as is done in the hg-crew/hgext3rd/__init__.py file

Test Plan:
Deployed this file on my server and confirmed that extensions in other hgext3rd
directories in $PYTHONPATH are still found correctly.

Reviewers: rmcelroy, durham, quark

Reviewed By: quark

Subscribers: net-systems-diffs@, yogeshwer, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3746098

Tasks: 12792845

Signature: t1:3746098:1471679643:44a274807d3598a670e1742dee4cc2654f6bad97
2016-08-22 17:39:02 -07:00
Zach Amsden
bdc73e2393 [fastlog-hg] Remove limit on threads
Summary:
Limits make no sense anymore because we may be filtering
the results further.

Test Plan: fastlog unit test, run extension against fbsource

Reviewers: quark, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3743974

Tasks: 12341014

Signature: t1:3743974:1471906692:25f48349bd70781219acdc3cf6b98134035d6236
2016-08-22 16:23:10 -07:00
Zach Amsden
64d75b4864 [fastlog-hg] Get patterns working again
Summary:
When the restriction of only working on a single directory
was dropped, pattern support was neglected.  That has to be done through
a revset, not the looser matches() filter.

Test Plan: P56588753

Reviewers: rmcelroy, quark, stash, durham

Reviewed By: durham

Subscribers: mhammell, meisner, calcsaransh, gqchen, dzhulgakov, kerl, bnitka, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3734232

Tasks: 12341014

Signature: t1:3734232:1471906628:d3995b358d683383fe590fa5848d8450a0ea05dd
2016-08-22 16:21:43 -07:00
Zach Amsden
cde79943b1 [fastlog-hg] For now, import os is still required
Summary: Accidentally removed this dependency too soon

Test Plan: Run fastlog on fbcode; broken before, now fixed

Reviewers: durham, quark, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3743884

Tasks: 12341014

Signature: t1:3743884:1471853021:ce470e4e42aa1c07aed624285a81824e04e70668
2016-08-22 14:46:39 -07:00
Zach Amsden
a011ca516b [fastlog-hg] More extensive fastlog test
Summary:
More extensive test to make sure existing behavior isn't broken.
What is broken currently - globs.

Test Plan: ../../hg/run-tests.py test-fastlog.t

Reviewers: durham, quark, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3743677

Tasks: 12341014

Signature: t1:3743677:1471852994:7684524f1879e6ac16d473cabd53e95adf73d888
2016-08-22 14:42:39 -07:00
Zach Amsden
f115cd74e9 [fastlog-hg] Remove unused imports
Summary: No need for cruft

Test Plan: Load in Atom, no more complaints about unused imports

Reviewers: durham, rmcelroy, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3733509

Tasks: 12341014

Signature: t1:3733509:1471519620:08e896aaff65b545d7d16e5d46ba6a8e2a2bc186
2016-08-19 11:59:10 -07:00
Jeremy Fitzhardinge
1fd4455c73 Implement 'hg uncommit'
Summary:
TODO:
 - fix handling of empty commit (leaves stray heads)
 - add tests

Test Plan: Add mercurial unit tests

Reviewers: #mercurial, ttung, rmcelroy, durham

Reviewed By: durham

Subscribers: quark, rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3610409

Tasks: 12340638

Signature: t1:3610409:1471398045:355f20502d917374e99feee4681a7ef6f853ad6c
2016-08-17 13:43:25 -07:00
Jun Wu
08ac5e7826 setup: check Cython version
Summary:
The minimal version of Cython to build linelog is 0.22. Add a check so people
know it's not a source code issue, but Cython needs upgrade.

Test Plan:
Run with different Cython releases and make sure it complains when Cython
version < 0.22. Also make sure it works with strange versions like "0.25a1".

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3706559

Signature: t1:3706559:1471463690:92b981e2df63ad63ab55edb5e0b937455254351d
2016-08-17 20:55:54 +01:00
Zach Amsden
092d645baa Fix comment for fastlog
Summary: Document how to enable the extension

Test Plan: Set this in my actual .hgrc and confirm it works.

Reviewers: durham, rmcelroy, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3721128

Signature: t1:3721128:1471342825:e274764e036f1c03b9f8795221af6fe587990340
2016-08-16 15:55:48 -07:00
Jun Wu
fab3975b47 sshaskpass: use mkdtemp to create temporary directory
Summary:
Before this patch, sshaskpass has a hardcoded tmp directory, which, if being
abused, can break sshaskpass from working. This diff fixes the issue by
using tempfile.mkdtemp.

Test Plan:
Run `chg push -r . ssh://root@localhost//tmp/foo -f --allow-ano` and make
sure ssh password prompt works. Also run the sshaskpass test.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3705674

Signature: t1:3705674:1471373405:aec54ac685b060efb616307552cebcccee5cebe8
2016-08-16 19:51:47 +01:00
Jun Wu
5aca2ebc2f test-tweakdefaults-bookmarks.t: fix typo 2016-08-15 18:45:19 +01:00
Jun Wu
e2e852370f linelogcli: add new command getalllines
Summary:
Since linelog has the new `getalllines` API, add a corresponding command
to support it.

Test Plan:
```
$ make && ./linelogcli /tmp/lll init annotate 0 replacelines 1 0:0 0:3 \
  replacelines 2 1:2 1:3 getalllines 0:0
init: okay
annotate: run annotate for rev 0
annotate: 0 lines, endoffset 1
replacelines: rev 1, lines 0:0 -> 0:3
replacelines: rev 2, lines 1:2 -> 1:3
getalllines: 5 lines
  0: rev 1, line 0, offset 3
  1: rev 2, line 1, offset 8
  2: rev 2, line 2, offset 9
  3: rev 1, line 1, offset 11
  4: rev 1, line 2, offset 5
```

Reviewers: #mercurial, simonfar, ttung

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716453

Signature: t1:3716453:1471265279:a8f625660ef47ca341ca8f5c4c7370b56277cef8
2016-08-15 12:47:14 +01:00
Jun Wu
d8d7af1727 linelog.pyx: add wrappers for the new API getallines
Summary:
Add `getoffset`, `getallines` to make the new C API `linelog_getalllines`
usable.

Test Plan: This helps the future smartfixup extension to pass its tests

Reviewers: #mercurial, ttung, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716447

Signature: t1:3716447:1471262262:2ad8a547bf956c01e39be3383dc1a50751729c85
2016-08-15 12:45:58 +01:00
Jun Wu
b30c3ad0e5 linelog.pyx: fix copyfrom
Summary:
`copyfrom` does not work because `rhs.buf` is invisible to `self`. Adding
a `readonly` keyword fixes it. Also add some sanity checks and remove an
unnecessary `;`.

Test Plan: `make local` and confirm `copyfrom` works.

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716444

Signature: t1:3716444:1471264420:1e58798222e1515ae6d5328ad6f2727cfe41cdb5
2016-08-15 12:43:44 +01:00
Jun Wu
3258b8586d linelog.pyx: run clear and annotate an empty linelog automatically
Summary:
A newly created empty `linelog` object is not usable (i.e.  calling
`annotate` or `replacelines` will raise an error) until `clear` is called.
And `replacelines` won't work until `annotate` is called. Therefore if we
know it's an empty linelog, just call `clear` and `annotate(0)` in
`__init__` to make it more friendly to use.

Test Plan:
`make local`, and confirm
`__import__('linelog').linelog().replacelines(1, 0, 0, 0, 1)` works.

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716442

Signature: t1:3716442:1471262694:96370f5d3074bc0941e8ee780181c6b688cff79c
2016-08-15 12:41:13 +01:00
Jun Wu
2c9fffd80e linelog: add a new API getalllines
Summary:
The current linelog APIs are designed purely for the "annotate" operation.
It only supports getting lines from one single revisions, which is what
`annotate` does.

However the data structure is also useful to provide lines from different
revisions. Say if you want to know all lines from all revisions for a given
function. Or if you want to confirm if a chunk is "continuous" - nobody
inserts new lines among them. This diff adds the missing API, `getalllines`.

It returns information about all lines the linelog tracks. The caller could
provide an optional interval to limit the lines returned.

Its parameters use low-level raw offsets instead of line numbers, because it
gives the caller more control. The API don't need to handle open or close
interval issues that line numbers may have (see the previous diff about
deletion blocks).

This makes `lineinfo.offset` no longer internal-only - we have exposed the
offset concept to the API parameters. Therefore the comments are updated.

This also requires a clear way to distinguish unconditional jumps from
conditional one. We use `JGE 0` as unconditional jumps. To ensure this,
make `replacelines` reject 0 revision.

Test Plan: `cd linelog && make`

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716439

Signature: t1:3716439:1471265245:5a4b2f585284cba71ca3907c8e8826b2b00a4b49
2016-08-15 12:31:29 +01:00
Jun Wu
c5819e914f linelog: extract appendline logic from annotate
Summary:
The `appendline` logic is used twice in `annotate`: inside and outside the
`for` loop. It will be used again in the following changes. Therefore make
it an inline function.

Test Plan: `cd linelog && make`

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716438

Signature: t1:3716438:1471263197:6add9cfc0a9d712258dd2da332aaf306cb1af71a
2016-08-15 12:14:07 +01:00
Jun Wu
b48ea46da6 linelog: be conservative when deleting a chunk from an old revision
Summary:
This is a subtle change. Consider the following case, where revision 1 has 3
lines and revision 2 has 1 line inserted between the second and the third
lines of revision 1:

  0: +---- INSERTED BY REV 1
     |                          --+             --+
  1: |     LINE REV=1 LINENUM=0   | DEL BLOCK A   | DEL BLOCK B
  2: |     LINE REV=1 LINENUM=1   |               |
     |                          --+               |
  3: | +-- INSERTED BY REV 2                      |
  4: | |   LINE REV=2 LINENUM=2                   |
     | +--                                      --+
  5: |     LINE REV=1 LINENUM=2
     +----

Now what if the user has run `annotate(rev=1)` and wants to delete the first 2
lines, by calling `replacelines(a1=0, a2=2, ...)`?

The current code would use "DEL BLOCK B", which is more efficient because
the block contains more instructions, and more simple because it is just a
JGE to the a2 address.

While that works fine for the "appending" case (always edit new revisions),
it deletes future lines between the second and third line (in this case,
"LINE REV=2 LINENUM=2"). This is undesired by some users, namely smartfixup.

Surely we can add a boolean flag to the `replacelines` API to decide whether
A or B will be used. But that is too low-level and `replacelines` already
has too many parameters. Today we don't see other requirements to choose
between them precisely. Therefore, just change the default behavior, when
deleting chunks from old revisions to "DEL BLOCK A".

Test Plan: This change is needed to help pass the future smartfixup tests

Reviewers: #mercurial, ttung, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3716435

Signature: t1:3716435:1471262664:0ae40d95d6389db733a88d9fafe19962d9166e35
2016-08-15 12:08:06 +01:00
Jun Wu
0d3a392454 hgignore: ignore linelog non-source files
Summary: This diff adds an `.hgignore` file for the `linelog` directory.

Test Plan: `make local`, and run `hg status`, make sure no unwanted files are printed.

Reviewers: #mercurial, ttung, akushner

Reviewed By: akushner

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3699575

Signature: t1:3699575:1471151389:7e4de7b75d671d3f0db18337e57c65e9972de059
2016-08-10 22:32:21 +01:00
Mateusz Kwapich
675561c76f sqldirstate: call the registered callbacks on wd parent change
Summary:
This diff together with patches introducing the mentioned callback in upstream
will fix the journal under sqldirstate.

Test Plan: journal tests passed, the rest is currently running - will update the test plan soon

Reviewers: #mercurial, ttung, durham

Reviewed By: durham

Subscribers: durham, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3690878

Signature: t1:3690878:1470800077:3fe48e03bb65eaeed3a7326b6e7d392abe7efd8d
2016-08-15 03:46:52 -07:00
Zach Amsden
f1ecf43a83 Parallel callout to scmquery through conduit
Summary:
Parallel callout to scmquery through conduit
This implements the special case of log on a single directory, preserving
follow behavior.  To do this, we backtrackfrom the current head to find
 all draft revisions along the path, then find the common public ancestor of
those.  Once we find that, we can begin paging in results from the scmquery
service.

Pretty much a working diff at this point.  Limits and boundary conditions
have not been fully tested.  Every once in a while I run into a bum query,
which I suspect to be either a bad proxy or a service router failure; still
debugging that.  Could also be an issue with conduit.  Other than that,
things seem to work.

Test Plan:
Testing log with fastest setting (no revsets), revsets (using -M), and with extension disabled (--sparse):

  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a

  real	0m1.895s
  user	0m0.003s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b

  real	0m1.308s
  user	0m0.005s
  sys	0m0.001s
  [zamsden@devbig192.prn1 ~/local/fbcode] diff a b
  [zamsden@devbig192.prn1 ~/local/fbcode] time hg log tao --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c

  real	0m7.320s
  user	0m0.004s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbcode] diff a c




Testing --user option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri > a

  real	0m2.765s
  user	0m0.002s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/dragon --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 5 -u dmitri --sparse > b

  real	0m23.247s
  user	0m0.005s
  sys	0m0.001s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing same output enabled / disabled for -X option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' > a

  real	0m1.292s
  user	0m0.002s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -X '**TARGETS' --sparse > b

  real	0m2.697s
  user	0m0.002s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing -k option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 > a

  real	0m1.174s
  user	0m0.003s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/lithium/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -k 'e' -X '**TARGETS' -l 10 --sparse > b

  real	0m1.259s
  user	0m0.004s
  sys	0m0.002s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]



Testing -I option:



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' > a

  real	0m1.473s
  user	0m0.005s
  sys	0m0.003s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/scm/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 20 -I '**.py' --sparse > b

  real	0m2.911s
  user	0m0.003s
  sys	0m0.002s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource]




Testing multiple directory output in all three modes - revset, fast filtered, and forcing original fallback with --sparse



  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 -M > a

  real	0m2.892s
  user	0m0.003s
  sys	0m0.006s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 > b

  real	0m2.575s
  user	0m0.697s
  sys	0m0.077s
  [zamsden@devbig192.prn1 ~/local/fbsource] time hg log fbcode/hphp/hhvm fbcode/hphp/runtime/ --config extensions.fbconduit=~/local/fb-hgext/hgext3rd/fbconduit.py --config fbconduit.host=our.zamsden.devbig192.prn1.facebook.com --config extensions.fastlog=~/local/fb-hgext/hgext3rd/fastlog.py --pager=off -l 100 --sparse > c

  real	0m7.339s
  user	0m0.003s
  sys	0m0.004s
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a b
  [zamsden@devbig192.prn1 ~/local/fbsource] diff a c
  [zamsden@devbig192.prn1 ~/local/fbsource]

Reviewers: rmcelroy, #scmquery, #mercurial, ttung, lcharignon, durham, stash

Reviewed By: stash

Subscribers: cdykes, lcharignon, quark, stash, mjpieters, jeroenv

Differential Revision: https://phabricator.intern.facebook.com/D3634075

Tasks: 12341014

Signature: t1:3634075:1471039212:0989839636847a8e5da6a0ef9150035fcf5bb797
2016-08-12 15:52:50 -07:00
Jun Wu
8813967d50 linelog: add a test editing lines randomly
Summary:
The test will call "replacelines" with random arguments, keeps track of the
content of "lines" on its own. Then it compares its own "lines" with linelog's
annotateresult - should be the same. After that, it verifies the content of
old revisions can be retrieved by using "annotate" correctly.

Test Plan: Run this test

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3709431

Signature: t1:3709431:1471012434:68ca06c0b3b2705740375c32acac8012ded404a5
2016-08-12 15:22:16 +01:00
Jun Wu
5b0b861096 sshaskpass: make the prompt similar to systemd
Summary:
This diff makes the prompt similar to what systemd does. See the screenshot:

{F62774663}

It solves an issue that with echo disabled, the user won't know whether they
have pressed ENTER or not - the cursor won't move. Now it explicitly prints
"AUTHENTICATION COMPLETE".


Test Plan: As the screenshot, and also run the sshaskpass test.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3705703

Signature: t1:3705703:1471008095:863ff9820b87ea3f4631295b10472802fb5e459f
2016-08-11 21:56:31 +01:00
Jun Wu
d4366acbd5 sshaskpass: use a standalone script
Summary:
Before this patch, sshaskpass will set SSH_ASKPASS to itself, aka. `__file__`.
This won't work if sshaskpass.py gets installed by setup.py because setup.py
will remove its `+x` bit.

Test Plan:
Run `chg push -r . ssh://root@localhost//tmp/foo -f --allow-ano` and make
sure ssh password prompt works.

Reviewers: #mercurial, ttung, rmcelroy

Reviewed By: rmcelroy

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3705657

Signature: t1:3705657:1471008122:9cfa3adf078e4bbe8f6b6ba05d1fb6be513d3e71
2016-08-11 21:42:54 +01:00
Jun Wu
6015a95c00 tweakdefaults: make "bookmarks" use the unfiltered repo
Summary:
We are seeing perf issues with hidden/obsolete handling.
`hg bookmark` is a frequently used command and by making it use the unfiltered
repo, it could be 200ms-300ms faster.

Test Plan: Added a new test

Reviewers: #mercurial, mitrandir, ikostia, ttung

Reviewed By: mitrandir, ikostia

Subscribers: mitrandir, rmcelroy, akushner, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3692968

Signature: t1:3692968:1470777864:72ad5d0ffb52ecfcaaa607082693b88319d778fd
2016-08-11 18:43:06 +01:00
Ryan McElroy
2a07de293c tests: import simplecache from correct place 2016-08-11 06:33:47 -07:00