Commit Graph

10 Commits

Author SHA1 Message Date
Jun Wu
4d1e57d182 fastannotate: fix getalllines
Summary:
`getalllines` needs to call annotate before continuing to build the "forked"
linelog in memory.

Test Plan:
Set mainbranch to @, run `hg fa --deleted` on a file that has been changed
in the draft branch and it does not crash.

Pinged: rmcelroy.
2016-10-08 01:39:17 +01:00
Jun Wu
38100f3be4 fastannotate: handle empty file correctly
Summary:
`_decorate` backported from upstream does not handle empty file correctly.
It would raise an assertion error when annotating an empty file:

  File "fastannotate/commands.py", line 138, in fastannotate
    not showdeleted))
  File "fastannotate/context.py", line 331, in annotate
    return self._refineannotateresult(result, revfctx, showpath, showlines)
  File "fastannotate/context.py", line 503, in _refineannotateresult
    if len(lines) != len(result):
  AssertionError

This patch fixes it.

Test Plan: Run the modified test.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3944132

Signature: t1:3944132:1475166894:e2610c6364806b77c8533315a1a0a08b6c158fe5
2016-09-29 16:48:40 +01:00
Jun Wu
56075d02ef fastannotate: encode directory names
Summary:
We need to reserve ".l", ".m" and ".lock". So encode directory names to
avoid collision.

Test Plan: Run the modified test

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3944069

Signature: t1:3944069:1475166648:055811239514cb699a0ebe1cfab809b661c6cfd2
2016-09-29 16:33:46 +01:00
Jun Wu
76bc8110f2 fastannotate: add a perfhack to use linkrev as introrev
Summary:
`introrev` can be slow because `_adjustlinkrev` can be very expensive.
Usually `introrev` is just `linkrev`. So let's use the latter as an
approximate, if `perfhack` is enabled.

Note that we only use `linkrev` when *checking* if the hg hash is in linelog
revmap or not - for the read-only operation. If we need to update linelog,
we fallback to the slow `introrev` implementation.

Test Plan: perfhack related tests will be added later

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3937044

Signature: t1:3937044:1475139823:450c6bb68b33db671f6b95cde4abfd24b7f2af4c
2016-09-28 15:56:47 +01:00
Jun Wu
bd3b5783c6 fastannotate: use a more efficient way to get filectx
Summary:
Usually we simply use `repo[node][path]` to get a `filectx` object.
But that can be painfully slow - it's reading the whole manifest by default.

The flat manifest and tree manifest both provide a method called `find` to
get a `filenode` without reading the entire manifest - which is our use-case
here since we don't need to resolve all files but just need to peek several
individual files.

As we are touching this area, also deal with revset string resolution so
we can simplify the logic in more places - now master can be a revset string
and is no lonnger needed to be lazily resolved. Besides, the `_getbase` logic
is also moved here and to be more efficient - the old `_getbase` still need
to read manifest.

This patch replaces all code using the `repo[node][path]` pattern to use the
more efficient implementation one.

Test Plan: Run existing tests

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3936992

Signature: t1:3936992:1475138449:2ec6c71a592a256fa9487a5f7b41241bf7440f89
2016-09-28 15:56:31 +01:00
Jun Wu
89c9da2b06 fastannotate: add a --deleted option
Summary:
This feature uses the linelog to show all lines ever existed (even deleted) in
a file. Helpful to see the history all the way back to the beginning.

Sadly it has to be inefficient currently as we have chosen to not store line
content (but only numbers) in linelog. Calculating the revisions and line
numbers is very fast because of linelog but resolving the line contents is
painfully slow. We may want a key-value database in the future, answering the
query:

  (path, node, linenum) -> content

How slow is it? With the linelog pre-built, generating the output for
`mercurial/commands.py` needs resolving 400+ revisions and is about 10+
seconds.

Test Plan: Run the changed `test-fastannotate.t`

Reviewers: #sourcecontrol, rmcelroy

Reviewed By: rmcelroy

Subscribers: rmcelroy, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3849503

Signature: t1:3849503:1475086235:83077c571746a7515b5ba75c4df37a1a400d9232
2016-09-11 22:52:29 +01:00
Jun Wu
60eb8a2c22 fastannotate: implement the annotate algorithm
Summary:
This diff implements the `annotate` algorithm. Unlike the vanilla one, the
annotate method takes 2 revisions: the revision specified for annotating,
and the head of the main branch. The algorithm will do a "hybrid" annotate:
incrementally update the linelog (the cache) so it can answer queries of
any revision in the main branch. And use the traditional algorithm to deal
with revisions not in the main branch: like a side branch of a merge commit,
or the revision the user specified not in the main branch.

The main branch is supposed to be something like `master` or `@`, and their
p1s.

Building up linelog with merge handled reasonably for the main branch, and
the non-linelog part that produces final result share a lot internal states
and logic so they are deeply coupled. Splitting them will probably reduce
performance, or have difficulty (no clean way) to share internal states.
If the caller only wants to build linelog without annotate things, just pass
`rev = master`.

While some attempts are made to support "merge" changeset, the result can
still be different from the vanilla one sometimes. In those cases, both
results make sense. It's really hard, if not impossible, to make the new
implementation 100% same with the vanilla one because of the linear history
restriction of linelog so I guess currently it's good enough. The differences
will be covered by a `.t` test later.


Test Plan: Code Review. A `.t` file will be added.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3836438

Signature: t1:3836438:1473778829:27978479a01920833fa146f427178292ea1f5306
2016-09-08 16:54:21 +01:00
Jun Wu
943e37217d fastannotate: add an empty annotatecontext
Summary:
The annotatecontext object contains all the states to perform a fast
annotate operation. This diff adds a skeleton class and a helper method
that handles locking properly.

Test Plan: Code Review

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3836380

Signature: t1:3836380:1473433244:7759527278200dabdfc4c4869c1df28fc156d0dd
2016-09-08 16:46:27 +01:00
Jun Wu
9d919136d2 fastannotate: add annotateopts object
Summary:
The `annotateopts` object is like `diffopts`, maintaining a set of related
annotate options. Currently different options will result in different
(incompatible) linelogs being built. Therefore the `annotateopts` object
also have a method to generate a unique string describing the options, so
we can use the string as part of the path where we write the linelog file.

Test Plan: Code Review

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3836366

Signature: t1:3836366:1473413541:0e88ca2159dab38a5f4067a67d8a86c74d072cae
2016-09-08 16:09:48 +01:00
Jun Wu
84384559c3 fastannotate: cherry pick some functions from the vanilla annotate
Summary:
These are some small functions defined in the core hg `context.annotate`
method and we want to use them later. The ported code was licensed under
GPLv2.

Test Plan: A `.t` test will be added when the fastannotate command is functional.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3836335

Signature: t1:3836335:1473413278:de2510a6e5041455c430199717a1ef5217de5348
2016-09-08 16:08:12 +01:00