Summary:
When requesting trees from the server we try to provide a base tree
that we already have. In the hybrid manifest days, when we were fetching
manifest X, we would look up the linknode for X in the flat manifest revlog,
then scan the changelog up and down from that linknode. In the treeonly world we
don't have that revlog anymore though, so the current algorithm just searches from
the tip. This ended up being a really bad algorithm, since if you have tip and you
request tip~1000, then tip~2000, then tip~3000, you end up basing them all
against tip and redownloading a lot of data. This makes hg blame and hg log -p
super slow.
Let's change the algorithm to base off of a hint linknode, and let's change
changectx to set that hint every time it attempts to read a manifest. In manual
testing this significantly sped up hg log -p
Reviewed By: phillco
Differential Revision: D8399613
fbshipit-source-id: 771a94ee8b82be682ea0091b8d6c0fcd5f4e6646
Summary:
Previously we weren't including p2 when creating trees, this resulted
in incorrect conversions.
Reviewed By: phillco
Differential Revision: D8152253
fbshipit-source-id: c4f8c79b40532c5162b15032962fbc6a78d44b5f
Summary:
Make the (user-facing) log for prefetching trees say how many commits (or which
commit) it is prefetching for.
When remotefilelog.debug is enabled, print out the manifest identity when
fetching a tree during normal operation.
Reviewed By: quark-zju
Differential Revision: D8056555
fbshipit-source-id: dc0c7bc5c949a0674d5f553661e736bc545134b4
Summary:
We had a bug where the cache store had a value that didn't correspond
with it's key. Let's add the key to the value so we can ensure they are always
paired correctly.
Reviewed By: phillco
Differential Revision: D8065151
fbshipit-source-id: 01be3a3432d1b4f5e9a234b35b8315c37b1869c3
Summary:
Pushrebase previously defaulted to bundlev1. Now that the server can
support v2, let's default to v2.
Reviewed By: quark-zju
Differential Revision: D7591174
fbshipit-source-id: bc5448490ff319543baedf3f5a1aab160a73ed27
Summary:
We want to deprecate the bundlev1 format, so let's start by adding a
develwarn. Later diffs will update the tests to not use v1, then remove v1 as a
supported outgoing bundle entirely.
Reviewed By: quark-zju
Differential Revision: D7591166
fbshipit-source-id: 143ad029bfe4d141f91d6d5077342dfa44ad2944
Summary:
Previously we relied on the list of files created by strip to decide
what directories to strip. It turns out this list wasn't adequate though, since
it's possible for directories to change even when files do not. For example,
during a merge if different files were added on different sides of the merge,
the files don't change during the merge but their containing directory does.
This diff special cases merge commits to make sure all the directories affected
by the merge are included in the strip.
Reviewed By: quark-zju
Differential Revision: D7409156
fbshipit-source-id: 9bf67eefb70189300c29db60d9945a7f608dfdda
Summary:
A prior diff optimized push/pull to not send public trees between
peers, since those trees can be downloaded from the main server. Let's be
careful when sending data to the main server and always send everything.
In the future we should add validation on the server that the received data is
complete, but Mercurial doesn't currently do that today.
Differential Revision: D7296253
fbshipit-source-id: 49513685d19991a70d66da1d734ddae23491ed0c
Summary:
This allows pushing a treeonly pack to a server without using
pushrebase.
Differential Revision: D7295686
fbshipit-source-id: b0bfe4fbb04bc765e57f1db82909fa1ae7b3063b
Summary:
Previously pushrebase pushes would send a tree pack part as well as a
rebase pack part. The tree pack part was ignored. Let's make receiving tree pack
parts on the server an error, and instead make the client not send them if it's
doing a pushrebase push.
A future diff will come back and add support for non-pushrebase push of tree
packs. This diff just makes use sure that the new logic won't run during
pushrebase.
Reviewed By: ryanmce
Differential Revision: D7267802
fbshipit-source-id: cabff989c178afb3706b77a03a299bcf3195f289
Summary:
Previously the server would reject pushes that didn't use pushrebase,
since we relied on pushrebase to recreate the commits with the treemanifests.
Now that the flat-to-tree conversion logic is cleaner, we can use it to do the
conversion during a normal push.
Reviewed By: quark-zju
Differential Revision: D7143453
fbshipit-source-id: c13713e2ff59f1dff3ee1f44b7f8db7a92cfe1de
Summary:
Previously we weren't able to commit right after creating a treeonly
repository. This was caused by the code attempting to read the null tree from
the store, which doesn't exist. The fix is to handle the null case and return an
empty tree instead of trying to look it up in the store. We already do this in a
number of other cases, so this was just a missing one.
Reviewed By: singhsrb
Differential Revision: D6930919
fbshipit-source-id: e227612be2640282eb997f4d563102d86f0be43a
Summary:
A future diff will add tree prefetching during changegroup creation. This
requires access to the tree prefetch function from the shallowbundle class.
let's move the prefetch functions onto the repo object in preparation for that.
Reviewed By: quark-zju
Differential Revision: D6873055
fbshipit-source-id: 18de0ee0f6ab566587509f9e23ebb2e5779ed1c8
Summary:
Now they are unnecessary since `run-tests.py` will set up `PYTONPATH`
correctly.
Differential Revision: D6865042
fbshipit-source-id: ca95314f725968e14349a9d916434aa832c596f9
Summary:
When pushing a treeonly commit to a tree+flat hybrid server, the server
needs to execute the hooks in treeonly mode so we never try to access the flat
manifests. Previously we did this for shell hooks by setting some environment
variables, but we didn't do it for python hooks that ran in process.
This diff makes the python hooks run against a bundle repo that is instantiated
in treeonly mode. No changes to any hooks are required, as the repo object they
are given is already in the correct mode.
Reviewed By: quark-zju
Differential Revision: D6840971
fbshipit-source-id: 9fcb97d972076911b35bccf3f79b60972bcafc33
Summary:
We're seeing high load on the server when many clients are fetching treemanifest
entries. A lot of this stems from the cost of opening revlogs. Let's introduce a
simple on-disk cache that let's us avoid opening revlogs as much as possible.
Reviewed By: quark-zju
Differential Revision: D6797698
fbshipit-source-id: c131aa57be333dcb2e4453c6a1f9f4e24532a7ee
Summary:
Now that fastmanifest and treemanifest are moved, we can moved the
tests.
Test Plan: make local && ./run-tests.py
Reviewers: singhsrb, #mercurial
Reviewed By: singhsrb
Subscribers: singhsrb
Differential Revision: https://phabricator.intern.facebook.com/D6685334
Signature: 6685334:1515526455:57791d7f9d2a3f2a6119c0e79a81b31e63a11c60
