Commit Graph

13 Commits

Author SHA1 Message Date
Durham Goode
15c644baef cstore: implement markForRefresh on cdatapackstore
Summary:
The markForRefresh function allows callers to tell the datapackstore that new
pack files have been written. This is important for being able to support
generation of treemanifests during pull and is a blocker for using the native
datapackstore in production.

Test Plan: Added a test

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4609076

Signature: t1:4609076:1488365022:d5480897f2b7674b4d07243e20a495551f985321
2017-03-01 16:55:18 -08:00
Durham Goode
3b5fb8770e cstore: implement UnionDatapackStore.getdeltachain()
Summary:
Implements the getdeltachain function on the new UnionDatapackStore class.

This required some modifications to the DeltaChainIterator. Since the results of
the iterator may cross multiple different chains, we need to keep each chain
alive until the iterator is destructed, so we need to keep a reference to each
chain. We also had to remove the size() property from the iterator since the
fact that the chain spans chains means we don't know the size up front.

Test Plan: Adds a test

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4556458

Signature: t1:4556458:1487199872:07dffa3121acfbeb6d6993b518e6f4887122d4d5
2017-02-23 14:03:03 -08:00
Durham Goode
767c10a592 cstore: implement UnionDatapackStore
Summary:
This adds a new C++ UnionDatapackStore implementation that only has the
getmissing() function at the moment.

Test Plan: Adds a test

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4556052

Signature: t1:4556052:1487161607:664752df19c63c06819ee2af5b4c436f1b76609d
2017-02-23 14:03:03 -08:00
Durham Goode
de60b62da4 cstore: implement datapackstore.getmissing()
Summary:
This implements a C++ version of the datapackstore getmissing() function and
adds a test.

Test Plan: Added a test

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, ikostia, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4552853

Signature: t1:4552853:1487191581:c108be4a85f8eacafddbea8086d5d22fdcabdacf
2017-02-23 14:03:03 -08:00
Durham Goode
4fd00d751a cstore: C++ implementation of datapackstore
Summary:
The remaining python parts of the store are a perf bottleneck when accessing
hundreds of thousands of pack file entries (like in treemanifest). Let's
implement them in C++.

This first patch just add the basic boiler plate, and implements a single
function getdeltachain(), with a test. Future patches will add more
functionality and other parts of the store.

Since cstore depends on cdatapack and ctreemanifest (the pythonutils.h part for
now), we need to tweak our setup.py to enforce a certain build order too.

Test Plan: Added a test, yo

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4547929

Signature: t1:4547929:1487181318:21c146cf370d26cb97efe6a883868b85b4e32f49
2017-02-23 14:03:03 -08:00
Durham Goode
a808c980f0 Backed out changeset c84de4b54530
The cstore changes are breaking the build in some unusual ways and I will need
some time to fix them. Let's back it out for now.
2017-02-16 14:37:23 -08:00
Durham Goode
885c9a00a3 Backed out changeset c00bd7acec6e
The cstore changes are breaking the build in some unusual ways and I will need
some time to fix them. Let's back it out for now.
2017-02-16 14:37:23 -08:00
Durham Goode
f797211a24 Backed out changeset 7f46fb9d639b
The cstore changes are breaking the build in some unusual ways and I will need
some time to fix them. Let's back it out for now.
2017-02-16 14:37:23 -08:00
Durham Goode
ac609a8fcd Backed out changeset f2366f7aceae
The cstore changes are breaking the build in some unusual ways and I will need
some time to fix them. Let's back it out for now.
2017-02-16 14:37:23 -08:00
Durham Goode
455a287639 cstore: implement UnionDatapackStore.getdeltachain()
Summary:
Implements the getdeltachain function on the new UnionDatapackStore class.

This required some modifications to the DeltaChainIterator. Since the results of
the iterator may cross multiple different chains, we need to keep each chain
alive until the iterator is destructed, so we need to keep a reference to each
chain. We also had to remove the size() property from the iterator since the
fact that the chain spans chains means we don't know the size up front.

Test Plan: Adds a test

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4556458

Signature: t1:4556458:1487199872:07dffa3121acfbeb6d6993b518e6f4887122d4d5
2017-02-15 15:19:36 -08:00
Durham Goode
9c206a743f cstore: implement UnionDatapackStore
Summary:
This adds a new C++ UnionDatapackStore implementation that only has the
getmissing() function at the moment.

Test Plan: Adds a test

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4556052

Signature: t1:4556052:1487161607:664752df19c63c06819ee2af5b4c436f1b76609d
2017-02-15 15:19:36 -08:00
Durham Goode
99a0c9b488 cstore: implement datapackstore.getmissing()
Summary:
This implements a C++ version of the datapackstore getmissing() function and
adds a test.

Test Plan: Added a test

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, ikostia, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4552853

Signature: t1:4552853:1487191581:c108be4a85f8eacafddbea8086d5d22fdcabdacf
2017-02-15 15:19:36 -08:00
Durham Goode
41486c3f47 cstore: C++ implementation of datapackstore
Summary:
The remaining python parts of the store are a perf bottleneck when accessing
hundreds of thousands of pack file entries (like in treemanifest). Let's
implement them in C++.

This first patch just add the basic boiler plate, and implements a single
function getdeltachain(), with a test. Future patches will add more
functionality and other parts of the store.

Since cstore depends on cdatapack and ctreemanifest (the pythonutils.h part for
now), we need to tweak our setup.py to enforce a certain build order too.

Test Plan: Added a test, yo

Reviewers: #mercurial, simonfar

Reviewed By: simonfar

Subscribers: simonfar, stash, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4547929

Signature: t1:4547929:1487181318:21c146cf370d26cb97efe6a883868b85b4e32f49
2017-02-15 15:19:36 -08:00