Commit Graph

8 Commits

Author SHA1 Message Date
Durham Goode
3707e186b6 treemanifest: add local pack store to unionstore
Summary:
In a future patch we will start writing user local tree data into a local
directory. This patches add the local store to the union store so the contents
will be accessible once we start writing it.

It also renames manifest to manifests for consistency with the other store names
(like 'packs').

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4055827

Signature: t1:4055827:1477059457:0d5b0d999b47d88c73f5ab2721d8d27deacc01bc
2016-10-21 11:02:22 -07:00
Durham Goode
4235752853 remotefilelog: rename getpackpath to getcachepackpath
Summary:
In a future diff we will be introducing packs into .hg/store, so we need to
differentiate between cache packs and local packs. This patch renames
getpackpath to getcachepackpath. A future diff will add getlocalpackpath.

This exposed a pyflakes error, so we fix that too.

Test Plan: Ran the tests

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4055815

Signature: t1:4055815:1477059415:e0221557bbeec6701820c826f00390d3a71cd2d3
2016-10-21 11:02:20 -07:00
Durham Goode
4f326d2c38 treemanifest: don't call set for files that didn't actually change
We compute which files have changed by looking at the manifest deltas. It's
possible that a manifest delta may contain an entry that deletes a file, then
replaces that file with the exact same content. This results in an unnecessary
set operation. Let's catch that earlier and avoid the set.
2016-10-14 16:01:12 -07:00
Durham Goode
e2c52435b0 treemanifest: optimize which trees we keep in memory during auto create
Previously, autocreate would keep every tree we created in memory. This result
in a memory explosion such that the process was eating 10's of GB of memory.
Let's optimize this to keep track of how many times each tree will be needed,
then throw the tree away once it is no longer needed.

Testing this via 'hg pull' showed that memory stayed constant even when pulling
and converting thousands of commits.
2016-09-21 13:51:39 -07:00
Durham Goode
599de92eab treemanifest: move verification behind an option
This moves our auto create tree verification to be behind an option. It defaults
to True for now.
2016-09-21 13:57:06 -07:00
Durham Goode
df86b3486d treemanifest: auto create manifest pack directory
The pack/manifests directory wasn't being automatically created, so let's make
it so.
2016-09-21 13:51:39 -07:00
Durham Goode
4da14809c2 tree: add option to automatically create trees during hg pull
Summary:
This adds a treemanifest.autocreatetrees config option. When it is set, hg pull
will automatically create a pack file that contains tree contents during an hg
pull.

We'll need to wait until the setitem and dirtybit logic is landed before we land
this, since that's required for us to test the full iteration logic here.

Test Plan:
Ran hg pull and verified a datapack was produced with the correct
manifest contents. The contents currently only contained the root manifest,
since we don't have the setitem + dirtybit logic necessary to actually modify
tree yet.

Reviewers: #fastmanifest

Differential Revision: https://phabricator.intern.facebook.com/D3838836
2016-09-19 16:30:17 -07:00
Durham Goode
3b7beae747 ctree: add ctreemanifest hg extension
Summary:
Adds the initial extension that sets up the ctreemanifest. It currently relies
on the fastmanifest extension to hook into all the manifest APIs to construct
ctreemanifests.

Test Plan:
In a future patch, I was able to run 'hg manifests' on a commit and
have it return the manifest contents by reading the treemanifest.

Reviewers: #fastmanifest, ttung

Reviewed By: ttung

Subscribers: ttung

Differential Revision: https://phabricator.intern.facebook.com/D3755327

Signature: t1:3755327:1472114482:0c5862cba68ed4db643d28c2fae01f33f5352970
2016-08-29 16:19:52 -07:00