Summary:
Upstream hg recently introduced new developer warning if files inside .hg are
modified without taking a lock. The usual fix is to add file in
localrepository._wlockfreeprefix list, but it doesn't work for "needrebuild"
fastpartialmatch file because it's stored inside .hg/store.
So to get rid of developer warning let's move needrebuild file to .hg/.
And since we are detecting that partial index needs to be rebuilt in revlog class,
I also had to wrap `runcommand()` and create needrebuild file there, because
`runcommand()` has vfs object, while revlog class doesn't.
Test Plan: Run tests
Reviewers: #fbhgext, durham
Reviewed By: #fbhgext, durham
Subscribers: durham
Differential Revision: https://phab.mercurial-scm.org/D99
Summary: Upstream has deprecated cmdutil.commands() in favor of registrar.commands()
Test Plan: Ran the tests
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D5106486
Signature: t1:5106486:1495485074:0e20f00622cc651e8c9dda837f84dd84cc51099e
Summary:
We saw a couple of weird exceptions during clone. Since read path is wrapped
in try/catch, let's wrap write path too. If exception happened then let's
remove index completely since it will be rebuilt on the next pull.
Test Plan:
arc unit
Unit-tests are actually almost useless in this case.
So I added fake exception raises and checked that it behaves as expected.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4843497
Tasks: 17153898
Signature: t1:4843497:1491536062:8e72a4ad4fffe5625866a90b3b89acda75211ce8
Summary:
Upstream mercurial doesn't run hooks for temporary amended commits
(see https://bz.mercurial-scm.org/show_bug.cgi?id=3501 and
c2ca20984e3c9f30d73f0f35c35904b64edbd692). No other hook contains info about
temporary amended commits.
Let's wrap localrepository.commit() function and add missing commits inside
this wrapper.
Test Plan: arc unit
Reviewers: #mercurial, durham
Reviewed By: durham
Subscribers: durham, mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4634950
Tasks: 16387175
Signature: t1:4634950:1488399457:1d299c6ad3cfd418dda32a7ec20c60290f8d7778
Summary:
bundlerepo adds new commits that are not present in fastpartialmatch index.
Without this diff any access to the commit from bundle fails if
fastpartialmatch.raiseifinconsistent config option is set.
Let's not use fastpartialmatch in bundlerepos
Test Plan: arc unit
Reviewers: #mercurial, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4635368
Tasks: 16394593
Signature: t1:4635368:1488393506:15b8f6a9fddabe681de7fd1296bbfafb9512f558
Summary:
Turned out that wrapping `changectx.__init__()` doesn't give any performance
benefits but instead it breaks initializing of changectx. For example, if
there is a commit which revision number N is a prefix of another commit A then
`hg log -r N` outputs commit A instead of commit N. Let's get rid of this
wrapping.
Test Plan: arc unit
Reviewers: #mercurial, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters, #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4635311
Tasks: 16394593
Signature: t1:4635311:1488393427:eae091a0d155df336adbb122f2881a28e95ba6b6
Summary:
fastpartialmatch index may be incorrect if fastpartialmatch extension was
disabled, then new commits were added and then fastpartialmatch was
enabled back. We need to detect this situation and rebuild the index in
this case.
One approach would be to use _handleinconsistentindex() function to mark index
as invalid. There are a couple of drawbacks to this approach:
1) It requires more significant code changes
2) The output may still be incorrect if hash with the same prefix was added while fastpartialmatch was disabled.
Another approach would be to use header version from index files
(see class _header). That's even worse because we'll need to bump index
version in the code whenever we disable fastpartialmatch.
Third approach would be to use changelog mtime + size as a validation key but
then we need to ensure partialmatch index is written after changelog.
Instead I suggest to use generation numbers. It's just a number in the config
and in the special file inside partialmatch index directory. If the
two doesn't match then index is incorrect and needs to be deleted. The biggest
drawback to this approach is that we need to bump generation number each time
the extension is enabled. I think it should be fine because the one who enables
extensions usually understands it and he or she will bump the generation number.
Test Plan: arc unit
Reviewers: #sourcecontrol, durham, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4571926
Signature: t1:4571926:1487263035:53419c9ca4e344127f391ac19c1071a826fea3f1
Summary:
User reported failures because revlog.opener doesn't have `exists` function.
The reason is that sometimes opener is wrapped inside the function.
Let's use _realopener instead
Test Plan: arc unit
Reviewers: #sourcecontrol, mjpieters
Reviewed By: mjpieters
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4571659
Signature: t1:4571659:1487267797:a950ab7c88d75573df64ec1964755c054307c7ea
Summary: Sort the output to fix tests on different platforms
Test Plan: arc unit on mac
Reviewers: #sourcecontrol, simonfar, ikostia
Reviewed By: simonfar, ikostia
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4501449
Signature: t1:4501449:1486050577:3041dc3c3691f48e39a0e9e05b1f97954a6d88e2
Summary:
If we have too many unsorted entries then finding matching revisions is
slower. To fix it we will rebuild partial index from time to time.
_findcandidates function marks index as need-to-be-rebuilt if it notices
index file with too many unsorted entries. Then _changegrouphook rebuilds
the index.
Test Plan: arc unit
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4494617
Summary:
Since we have some of the nodes sorted we can now use bisect to quickly find
matching nodes.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4494419
Summary:
Append header to each fast partial match index file. For now it contains
version number and number of sorted entries in the file.
File is ignored if version number from file is different.
Test Plan: arc unit
Reviewers: #sourcecontrol
Differential Revision: https://phabricator.intern.facebook.com/D4488417
Summary:
We'll add bisect algorithm soon.
Let's sort the nodes first during index rebuilding
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4488263
Summary:
strip may change revision numbers of many nodes at once.
It's better to rebuild index in case of strip.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474710
Summary:
This hook builds index if it's not present and it appends new entries
from the changegroup.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474704
Summary: These two functions actually use fastpartial index to resolve revision for a node.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474658
Summary:
Add commit hook to append new entries to the index.
It uses the same transactions as revlogs.
Do not append entries if index is not built (i.e.
if _partialindex does not exist). This is to ensure that we either
have complete index or none at all. Besides next diff will add
hooks to build index after pull if it doesn't exist.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474654
Summary:
Index to make node partial match fast i. e. speed up matching of partial hashes like f9bbd5.
This is an initial diff and it doesn't add any useful functionality, just debug commands.
Storage format is simple. There are a few files (at most 256).
Each file contains entries:
<20-byte node hash><4 byte encoded rev>
Each entry represents a commit (node).
Name of the file is the first two letters of the hex node hash. Nodes with the
same first two letters go to the same file. Nodes are NOT sorted inside the file
to make appends of new nodes easier.
Partial index should always be correct i.e. it should contain only nodes that
are present in the repo (regardless of whether they are visible or not) and
rev numbers for nodes should be correct too.
Test Plan: arc unit
Reviewers: #sourcecontrol
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4474642
