Summary:
fastannotate will tell remotefilelog what nodes of a file is already known in
linelog + revmap, so remotefilelog will not prefetch those files. Previously,
fastannotate either prevents all prefetching or allows all prefetching, which
is sub-optimal.
fastannotate could now donate its sshpeer to remotefilelog, so remotefilelog
won't need to start another one (assuming they can share a same sshpeer,
could be turned off via config options). This should reduce run time if SSH
handshake is expensive.
fastannotate could now also steal sshpeer from remotefilelog, so fastannotate
won't need to start another one. Combined with the above change, there would
always be only one sshpeer shared by fastannotate and remotefilelog.
Test Plan: Modified existing tests
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4325382
Signature: t1:4325382:1481933531:39d6344b2c076fbbff1f07997cd268e7cee25e80
Summary:
Previously, the revision number was converted to binary, and parsed using the
revset language. It worked fine in some cases but could be broken when it
matches other bits of the revset language occasionally.
This patch converts the revision number to hex to address the issue.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, ikostia
Reviewed By: ikostia
Subscribers: ikostia, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4327054
Signature: t1:4327054:1481904759:52170ba0ec7baf284b489f0d5a164153a7e8f25f
Summary:
When the server was handling a "getannotate" request, it may read lastnode,
update the revmap, then read lastnode again. So lastnode should not be cached.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4327046
Signature: t1:4327046:1481904335:2a236091b5d36872df5703e4bb9f3651012a2f02
Summary:
The default annotate command will do a "isbinary" check which reads fctx.data()
and contributes to a remotefilelog fetch round-trip. Since the "fastannotate"
command does not do binary check, and we want to minimize the round-trips,
disable the binary check by default.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, ikostia
Reviewed By: ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4327028
Signature: t1:4327028:1481904237:72a59dfb622459cd961b3da615053f80b55f1a7a
Summary:
Previously, when perfhack is enabled, `_matchpaths` will use slow code for
absolute paths. This diff makes it use the fast code for absolute paths.
Test Plan: `arc diff`
Reviewers: #sourcecontrol, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4318343
Signature: t1:4318343:1481724095:9d21a5135486c9d503bee3d8f93a23fa15fa647a
Summary:
The mtime check seems to cause more trouble than benefit - often the
annotate cache was treated "not up-to-date", while it is up-to-date
actually.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4318077
Signature: t1:4318077:1481723495:d4e0ab5e7ef5a520520c07fb7246230639b9b9b4
Summary:
fastannotate has a fastpath, when the node (commit hash) exists in revmap,
it would generate the output instantly from the linelog without going
through the history.
In the "fctx" mode, the vanilla annotate command would pass the revision
number instead of changeset node to fastannotate, which prevents
fastannotate from using the fast path.
This diff converts the revision number to node before testing the fast path
to solve the problem.
Test Plan: Modified existing tests.
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4288927
Signature: t1:4288927:1481073363:3330d60878161139ee195bc968c161d6c0d95ace
Summary:
The `isuptodate` check could still be expensive because of reading the manifest
takes time. Let's add a config option to skip the up-to-date checking to avoid
expensive manifest reading server-side.
Test Plan: Added a new test
Reviewers: #sourcecontrol, durham
Reviewed By: durham
Subscribers: durham, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4287289
Signature: t1:4287289:1481071706:5a3100f1513cd01ae7f78c1dfc9da9f5a3ba79cd
Summary:
If the annotate cache is up-to-date on the main branch, there is likely no need
to prefetch file contents, unless the user is annotating a side branch, which
requires a deeper integration between fastannotate and remotefilelog to just
prefetch the missing part correctly - I'll think about it later while this diff seems
good enough for common cases.
Test Plan: Added a new test
Reviewers: durham, #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4256370
Signature: t1:4256370:1480612767:c2e2a9225fb63ff36ffb579f9641851eb8cd8b39
Summary:
The upcoming remotefilelog change would require some cleanups to avoid code
duplication. Namely:
- Add a `fctxannotatecontext` so we can easily get a `annotatecontext` from
within `filectx.annotate` or `remotefilectx.annotate`.
- Print something instead of ignoring errors silently in `_fctxannotate`.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4256469
Signature: t1:4256469:1480612106:63756d64014a5b9e798bce52eda0d3e7368d6cba
Summary:
If the history of a file is short, it would be easier to just build the
linelog locally, instead of downloading it from the server.
This diff adds a "threshold" config option to make it possible for the client
to skip downloading linelogs.
A special case is, the file only has a single revision, where we can just avoid
all SSH connections - no fileservice peer, and no annotate peer.
Test Plan:
Add a test case for testing filelog repo.
Note: remotefilelog repo is not tested for this feature yet because there is
other planned changes related to remotefilelog.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4255483
Signature: t1:4255483:1480611801:4eb79c66eaf1fb5436d550f13c7a78d09b9a1ee2
Summary:
Previously fastannotate client will always fetch linelog from the server,
regardless of how long the history of a file is.
It actually makes sense for fastannotate client to build up the annotate
cache on its own, if, for example, the file only has one single revision,
where we don't even need any kind of SSH connection.
Test Plan: Run existing tests
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4254602
Signature: t1:4254602:1480611745:a44e70495771cb4f5c508ebb05fcfe8b0bb4d53f
Summary:
Previously, all prefetching work is done by the "fastannotate" command and
we may replace the "annotate" command with "fastannotate" and we run into
issues dealing with command line option differences.
Now that we have the "fctx" mode, do not redirect "annotate" to
"fastannotate" but wrap "annotate" in a lightweight way so it's responsible
for downloading the annotate cache.
Test Plan: Added a new test case
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4239607
Signature: t1:4239607:1480447754:a84268a9e64c678c1f275bc04c44a6f4138eafe6
Summary:
Previously, I have performance concern about the signature of
`fctx.annotate`: it returns fctxs, not ideal for performance since my
initial goal is to get rid of reading revlogs in the best case. And letting
the low-level `fctx.annotate` have some side effects writing to the disk is
not that pretty.
Because of that, fastannotate had re-invent part of the formatter (also
optimized for performance somehow), and still cannot support all the
features the original annotate supports, namely, templates.
Now it makes sense to just replace `fctx.annotate` with a sub-optimal
implementation, just for the flexibility of the template support of the
original annotate command. We can use a "fake" or "lazy" fctx object to
minimal the performance impact when people only need to print changeset
nodes and line numbers - in which case we don't read revlog.
Actually, the hgweb support already did a similar thing - converting
fastannotate output to annotate output. So we can reuse some code.
The next planned steps are:
- Make the original "annotate" command aware of fastannotate protocol, do
the pre-download stuff (downloading cache files at `fctx.annotate` is
possible but inefficient because of not being batched).
- "fastannotate" command remains a separated command optimized for perf
and provides extra features like "--deleted".
Because of the plan, the "commands" option does not make much sense -
"fastannotate" command will not replace "annotate" directly thus dropped
from the config interface. A new "modes" config option is added to control
how fastannotate works.
Test Plan: Modified existing tests
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4238998
Signature: t1:4238998:1480447679:c48e0e565663c086293265e104d9cf414d913aa7
Summary:
Using `testedwith = 'internal'` is not a good habit [1]. Having it
auto-updated in batch would also introduce a lot of churn. This diff makes
them "ships-with-fb-hgext". If we do want to fill the ideal "testedwith"
information, we could put it in a centric place, like a "fbtestedwith"
extension rewriting those "ships-with-fb-hgext" on the fly.
Maybe having in-repo tags for tested Mercurial releases is also a good idea.
[1]: www.mercurial-scm.org/repo/hg/rev/2af1014c2534
Test Plan: `arc lint`
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4244689
Signature: t1:4244689:1480440027:3dc18d017b48beba1176fbfd120351889259eb4b
Summary:
D4233433 changed commands to rebuild the broken cache but missed
wireprotocol. Let's do it for wireprotocol as well.
Test Plan: Added a new test
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4237855
Signature: t1:4237855:1480421375:d5715d61a3c66ec8be5a6a80041ee6204e5e4378
Summary:
It's common that the client's "master" ("main branch") falls behind the
server's, in which case the "is-up-to-date" check will fail and trigger a
re-download while it's unnecessary. Add some checks to avoid downloading
the cache for the client in this case.
Test Plan: Added a test
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4233534
Signature: t1:4233534:1480330716:db995d0fa5c6a2371894fc2a7871277da23608a9
Summary:
Previously, fastannotate's annotatecontext would delete individual cache
files (linelog and revmap) if it meets Exception. While the "fastannotate"
command has some retry logic, "debugbuildannotatecache" doesn't, which means
users have to run "debugbuildannotatecache" O(number-of-corrupted-files)
times to unblock. And "fastannotate" does not handle AssertionError which
could be seen as "linelog is corrupted" nicely - it crashes without retry.
Addressing all the above issues by replacing the "assert" to
"CorruptedFileError", catching them and retry properly.
Test Plan: Added a new test.
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4233433
Signature: t1:4233433:1480330548:618e80762af2209784ad72ad0658f649b050a080
This is to fix the following error discovered by pyflakes:
fastannotate/context.py:407: list comprehension redefines 'f' from line 352
The change is necessary after upstream changeset 4173d21a33de, which enables
the check.
Summary:
In order to support remotefilelog, do not construct
`mercurial.context.context` object directly. This reverts a previous attempt
to use the fastest manifest look up - `ctx[path]` actually seems good
enough.
A minor change is made in `protocol.py` to ignore failed requests.
Test Plan: Run existing tests
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4127025
Signature: t1:4127025:1478541481:c15477332582e2c9b22e2732e9e76bf37b5fba74
Summary:
When building the annotate cache server-side in batch, the current lock
mechanism creates and deletes small files very frequently and is not friendly
to the filesystem.
This diff introduces a `flock` approach which does not write to the disk and is
more friendly to the filesystem. The downside is, some network filesystem won't
support the lock correctly.
Since it seems to be a big improvement, especially at server-side, it's turned
on by default when it is supported by the operating system.
The documentation of config options have been updated so important options
appear first.
Test Plan:
Run the existing tests as the flock is turned on by default on my system.
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4103941
Signature: t1:4103941:1478541436:b4877df397ae2f0d58a611f13917704850fd62c1
Summary:
The `debugbuildannotatecache` is to pre-populate the annotate cache
server-side, or pre-fetch the annotate cache client-side.
To avoid import cycle, `cmdtable` is moved from `__init__.py` to
`commands.py`.
Test Plan: Modified existing tests.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4103932
Signature: t1:4103932:1478094621:e82192a12fe4b0b3ac732dbe4c127a2e93fc6b4c
Summary:
The command docstring was written with a full repo in mind. With a shallow
repo, things become different. This diff updates the documentation so it
also makes sense for a shallow repo.
Test Plan: Vim spellchecker.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4098257
Signature: t1:4098257:1477920765:8a48b332e786c4c538e939536564262214b3cc04
Summary:
This diff makes the client `fastannotate.client=1` download
linelogs and revmaps from the server on demand.
Test Plan: Added a new test
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4103929
Signature: t1:4103929:1477997846:608cf58c090806dafa21ae01feab90a6552100fc
Summary:
This diff adds a simple wireprotocol that the client can download fastannotate
cache files from the server.
Design decisions:
- Transfer full files. The linelog is not append-only and rsync-like
incremental updates is not trivial. Since the files look small (usually <
1MB), send full copies for now. If file size becomes an issue, we can
improve it later.
- The server can send arbitrary number of files with paths defined. This
provides the flexibility to send extra files without changing the protocol.
For example, if we want to speed up `--deleted`, we may want to download
`linemap` files.
- No compression. The revmap is basically not compress-able because it
contains the hashes. The linelog could be compressed though, but they are
small enough, and we may rely on compression on other layers, like SSH.
- `lastnode` check - if the last node of revmap the client reports matches
what the server has, the server will assume it's up-to-date and do nothing.
Note that this diff is only about the protocol and methods on peer, localrepo
objects. The client-side command is not aware of the feature yet.
Test Plan:
- Clone from a local repo A to B, via the ssh protocol.
- Config both repos so B is the client, A is the server.
- Run `hg debugshell --debug` from B, make sure the client can get contents
correctly (check the file contents from both the client and the server),
also make sure the non-existed files will be ignored:
In [1]: repo.prefetchfastannotate(['README', 'Makefile', 'NOT-EXIStED'])
running ssh localhost ...
sending hello command
sending between command
remote: 383
remote: capabilities: .... getannotate
remote: 1
fastannotate: requesting 2 files
sending batch command
fastannotate: server returned
fastannotate: writing 4976 bytes to fastannotate/default/README.l
fastannotate: writing 813 bytes to fastannotate/default/README.m
fastannotate: writing 9024 bytes to fastannotate/default/Makefile.l
fastannotate: writing 2453 bytes to fastannotate/default/Makefile.m
- Make sure the second time fetching these files, they will be skipped because
of the uptodate check:
In [2]: repo.prefetchfastannotate(['README', 'Makefile', 'NOT-EXIStED'])
- Manually create a diverge case where the client and the server disagrees
about what "master" (or "mainbranch") is. Make sure that although the
request will be send by the client, the server can return nothing because of
the lastnode check.
Reviewers: #mercurial, durham, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4063249
Signature: t1:4063249:1477920081:9bfab3031aa3118060c433f6e27c5b32aa25040c
Summary:
As we are going to add a protocol to transfer fastannotate cache over the
network, we need some basic methods to test if the cache is up-to-date.
Namely, this diff adds two methods:
- lastnode: used by isuptodate
- isuptodate: useful for both the server and the client to test if the cache
needs update or re-download
Test Plan:
These methods will be used later and a `.t` will be added when a fastannotate
client can download annotate cache from the server.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4103906
Signature: t1:4103906:1477997881:ad6843fdb683a550561758cd5babc56622f2ead3
Summary:
Previously, when using `with annotatecontext(...) as actx:`, the code will
construct the revmap and linelog objects, unconditionally.
Sometimes the caller just want to know some cheaper information, like the
revmap size, or the last hash of the revmap. It's unnecessary to construct the
revmap or linelog objects, while annotatecontext's locking could be useful.
This patch makes revmap and linelog reading lazily loaded so callers can use
`with annotatecontext(...) as actx` in more cases without worrying about
performance.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4103902
Signature: t1:4103902:1477997160:8638d8c772d326a5ad2a61124c96727ed6c3a87b
Summary:
Previously, to get changeset hashes in the revmap, we have to construct
the `revmap` object, which reads all content of a revmap.
In the following patches, we want to quickly get the last hash of the
revmap to test if the linelog / revmap is up-to-date or not. This patch
adds a method to do so.
Test Plan: Modified existing tests.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4103898
Signature: t1:4103898:1477988583:15869dfe9028068563b03d1f06d2337562049751
Summary:
Previously, the linelog and revmap paths are private information inside
`annotatecontext`. But we still want to get those paths in other places,
like the perfhack path check, or the quick up-to-date check (linelog vs
filelog) server-side. Let's expose the information by adding the helper
class.
Test Plan: Run existing tests
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4062835
Signature: t1:4062835:1477552929:7ece4f492e6b1e6f22cc626044819d5e3e2a66aa
Summary:
Before ba91edc29ce5, `diffopts` does not have `__dict__`. Let's fix that by
querying keys from `diffopts.default`, instead of using `__dict__`.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4086921
Signature: t1:4086921:1477554146:e48647c59769c2951518b091350289aa966aab7b
Summary:
There are some minor differences between the two annotate algorithm.
Both are reasonable in theory. The differences are explained in the test.
The `badfn` change is needed to make exit code correct.
Test Plan: Run the modified test
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4060263
Signature: t1:4060263:1477401942:64a08d2fa28ea0fa9b54fa0f0ea48ca489c308df
Summary:
Although `wdir()` is an undocumented feature, it exists in upstream annotate
tests. Let's support it so it is more similar to the vanilla annotate.
Test Plan: Added a new test
Reviewers: #mercurial, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4060169
Signature: t1:4060169:1477402083:021cae092e695a72fff65cd07640c9f7984b2a60
Summary: The vanilla annotate command supports `-Tjson`. Let's implement it as well.
Test Plan: Added a new test
Reviewers: #mercurial, simonfar
Reviewed By: simonfar
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4059577
Signature: t1:4059577:1477074802:09b0b3ea0769d480eb3a2e42308636ff2b8d40d2
Summary:
This diff tries to fix `test-check-*` issues for fastannotate.
One issue cannot be fixed:
fastannotate/context.py:31: imports not lexically sorted: linelog < os
If these two imports are swapped, the checker will report:
stdlib import "os" follows local import: linelog
instead. So leave it as-is for now until we get the checker fixed upstream.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4044157
Signature: t1:4044157:1476949486:13052173bc694edd99d21affa336d252bc2aae88
Summary:
Hopefully this will prevent new undocumented config options.
Issues about fastannotate are fixed. We still have many other issues, which
are ignored by this diff for now.
Test Plan: `arc unit`
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4037173
Signature: t1:4037173:1476876491:280e839a8e3d4f4dfbdf6b6953ab860bb7cd3ed4
Summary:
Previously we can only answer the "path" information when the revision is in
the linelog revmap, and the code would crash if a revision is in a side
branch, and the user requests path information. This diff fixes it.
Besides, this diff improves rename handling. For example, given the following
chart:
```
o---o -o file name: a
/
o---o- file name: b
^ ^ ^
1 2 3 revisions
```
Depending on the position of the `main branch` reference, fastannotate may
or may not use linelog:
- main branch is at rev 2, annotate a -r 3 will not take advantage of linelog
(fallback to slow annotate)
- main branch is at rev 3, annotate a -r 2 will not take advantage of linelog
This is not ideal, but seems to be the best we can do for now.
Test Plan:
Added a new test, updated existing relevant tests. Some debug messages are
changed to reflect internals more precisely.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4010964
Signature: t1:4010964:1476458201:79875d96399d023d0000d0c4bb8b8d40ea43eef0
Summary: This config option exists but is not documented.
Test Plan: Run existing tests
Reviewers: #sourcecontrol, eql, rmcelroy
Reviewed By: rmcelroy
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4011108
Signature: t1:4011108:1476307414:96191408f56b42a7949c2804114c7449ec8b5b27
Summary:
This adds fastannotate support for hgweb. There are some issues with "path"
handling, which will be addressed in follow up patches.
A minor change has been made in this patch to support revision numbers
(previously only global changeset hashes are supported).
Test Plan:
Run `hg serve --config fastannotate.hgweb=1` on the `hg-committed` repo, open
the following URL and confirm it's using fastannotate:
http://localhost:8000/annotate/9af6f8434430/mercurial/commands.py
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3993636
Signature: t1:3993636:1476026586:cb0628aa7107bdbfde852a6a1471f70dcb21a5ef
Summary:
The vanilla annotate command takes diffopts, let's add it to fastannotate.
This is also useful to support hgweb correctly.
Test Plan: Added a new test
Reviewers: #sourcecontrol, simonfar
Reviewed By: simonfar
Subscribers: simonfar, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3992672
Signature: t1:3992672:1475944204:2fbdf6a90d8c965775e3b7d5bc55fdb37c8909e7
Summary:
`getalllines` needs to call annotate before continuing to build the "forked"
linelog in memory.
Test Plan:
Set mainbranch to @, run `hg fa --deleted` on a file that has been changed
in the draft branch and it does not crash.
Pinged: rmcelroy.
Summary:
`_decorate` backported from upstream does not handle empty file correctly.
It would raise an assertion error when annotating an empty file:
File "fastannotate/commands.py", line 138, in fastannotate
not showdeleted))
File "fastannotate/context.py", line 331, in annotate
return self._refineannotateresult(result, revfctx, showpath, showlines)
File "fastannotate/context.py", line 503, in _refineannotateresult
if len(lines) != len(result):
AssertionError
This patch fixes it.
Test Plan: Run the modified test.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3944132
Signature: t1:3944132:1475166894:e2610c6364806b77c8533315a1a0a08b6c158fe5
Summary:
We need to reserve ".l", ".m" and ".lock". So encode directory names to
avoid collision.
Test Plan: Run the modified test
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3944069
Signature: t1:3944069:1475166648:055811239514cb699a0ebe1cfab809b661c6cfd2
Summary:
The mercurial matcher will read the manifest - unacceptably slow for our
usecase. Instead, if perfhack is enabled and matched linelog files exist,
treat patterns as valid file names directly.
Test Plan: perfhack tests will be added in the next patch
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3937103
Signature: t1:3937103:1475167055:dc27b65fe10f7a3a561afdb31292531f38e90dfb
Summary:
`introrev` can be slow because `_adjustlinkrev` can be very expensive.
Usually `introrev` is just `linkrev`. So let's use the latter as an
approximate, if `perfhack` is enabled.
Note that we only use `linkrev` when *checking* if the hg hash is in linelog
revmap or not - for the read-only operation. If we need to update linelog,
we fallback to the slow `introrev` implementation.
Test Plan: perfhack related tests will be added later
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3937044
Signature: t1:3937044:1475139823:450c6bb68b33db671f6b95cde4abfd24b7f2af4c
Summary:
Usually we simply use `repo[node][path]` to get a `filectx` object.
But that can be painfully slow - it's reading the whole manifest by default.
The flat manifest and tree manifest both provide a method called `find` to
get a `filenode` without reading the entire manifest - which is our use-case
here since we don't need to resolve all files but just need to peek several
individual files.
As we are touching this area, also deal with revset string resolution so
we can simplify the logic in more places - now master can be a revset string
and is no lonnger needed to be lazily resolved. Besides, the `_getbase` logic
is also moved here and to be more efficient - the old `_getbase` still need
to read manifest.
This patch replaces all code using the `repo[node][path]` pattern to use the
more efficient implementation one.
Test Plan: Run existing tests
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3936992
Signature: t1:3936992:1475138449:2ec6c71a592a256fa9487a5f7b41241bf7440f89
Summary:
This feature uses the linelog to show all lines ever existed (even deleted) in
a file. Helpful to see the history all the way back to the beginning.
Sadly it has to be inefficient currently as we have chosen to not store line
content (but only numbers) in linelog. Calculating the revisions and line
numbers is very fast because of linelog but resolving the line contents is
painfully slow. We may want a key-value database in the future, answering the
query:
(path, node, linenum) -> content
How slow is it? With the linelog pre-built, generating the output for
`mercurial/commands.py` needs resolving 400+ revisions and is about 10+
seconds.
Test Plan: Run the changed `test-fastannotate.t`
Reviewers: #sourcecontrol, rmcelroy
Reviewed By: rmcelroy
Subscribers: rmcelroy, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3849503
Signature: t1:3849503:1475086235:83077c571746a7515b5ba75c4df37a1a400d9232
Summary:
This diff adds a config option to replace the default annotate command using
fastannotate.
Test Plan: Run the modified test
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3837499
Signature: t1:3837499:1475076620:65dc656c74e9c8a969f68cc4a2480f5dcbeb6361
Summary:
This diff adds the `fastannotate` command, which puts all components
(formatter, annotatecontext - linelog, revmap) together.
Note that `fastannotate.mainbranch` needs to be carefully chosen to avoid
rebuilds. If `mainbranch` moves backwards, the `fastannotate` command
can fail because it could not move linelog backwards and thus cannot
update linelog incrementally.
Test Plan: Code Review. The `.t` file is coming in the next diff.
Reviewers: #sourcecontrol, stash
Reviewed By: stash
Subscribers: stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D3836670
Signature: t1:3836670:1474016576:b323b62db3bfd69bff88f6911abbe26075734737
