Summary:
The gettreepack protocol already had the arguments for the client to specify
what nodes they want and what nodes they have, but it wasn't implemented (we
just always returned the entire tree). This patch adds support for only sending
parts of the tree that are new. In addition, if the client requests many nodes,
we will only send the portion of each node that is not contained in the other
nodes. For instance if you ask for X and Y, where X is the parent of Y, then we
will send all of X and only the part of Y that is new.
The order changed a bit, probably because we stopped over delivering data.
Test Plan:
Ran and updated the tests. The next patch will actually add a test
for this code.
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4911531
Signature: t1:4911531:1492643897:58aabe5a32f156dd7424cc07c990301dbd5dc1c7
Summary:
A future patch will make tree prefetching more efficient. Let's change the
prefetch test to have changes in two separate directories so we can observe that
one directory is not downloaded when doing an efficient prefetch.
Test Plan: Ran the test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4911523
Signature: t1:4911523:1492641009:ac108ebac9573444c87c403f0be0b246d734dda7
Summary:
The name being passed to the store was wrong, because it had a trailing slash.
This wasn't caught before because the code was reading and writing the paths
with a slash at the end, so it matched as long as we only interacted with packs
produced by the code. The issue became more obvious when I tried to have packs
generated from revlogs interact with this code.
All the tests are affected since the entry keys changed.
Also use 'const ManifestFetcher &x' to pass the ref to avoid the copy.
Test Plan: Tests updated
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4901274
Signature: t1:4901274:1492638476:ff28f8976657baec99effbd82ecd436f6282ea5b
Summary:
The server treemanifest reading code was super slow because it used the pure
python treemanifest implementation. Let's switch it to use the ctreemanifest
implementation on top of a revlog store class.
The tests change because the output is now in reverse topological order (deepest
trees first).
Test Plan: Tests updated
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4901268
Signature: t1:4901268:1492637601:371a2303ac4f6e2b81f69a85ca657849150abcf7
Summary:
This adds a remote tree datastore to the store, so if we don't have a tree on
local disk, we can reach out to the server for it.
This also makes a small change to fastmanifest to use get() instead of
getmissing() to test if the tree is available. getmissing() would indicate the
tree is missing if it wasn't local, while get() will attempt to fetch it from
the server.
Test Plan: Added a test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: quark, stash, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4864645
Signature: t1:4864645:1492627823:7afff5db54c895507507e57ab7263f4dbf10b5f7
Summary:
This adds a new prefetchtrees command and gettrees wireprotocol command for
receiving trees from the server. The actual wire protocol is a custom request
format, with a normal pack file response format (same as used in remotefilelog).
The request args can specify which manifest nodes are wanted, for which
directory and which subdirectories.
I attempted to use the changegroup3 response format instead of the pack format,
but changegroup is extremely intertwined with revlogs and with producing
changegroups for linear swaths of the repo history, instead of random parts of
just the manifest.
Test Plan: Added a test
Reviewers: #mercurial, quark
Reviewed By: quark
Subscribers: stash, quark, mjpieters
Differential Revision: https://phabricator.intern.facebook.com/D4864435
Signature: t1:4864435:1492626392:c3603eeb261a1f9b9d836f0d44614025b58d45ea
