Commit Graph

15 Commits

Author SHA1 Message Date
Stanislau Hlebik
0d9f759213 infinitepush: add debugfillinfinitepushmetadata method
Summary:
Special command that fills optional metadata about infinitepush nodes. For now
it's info about changed files + info about how many lines were
added/removed. Since information is saved in json format it's easily
extensible.

It's a separate command so that we will be able to run it asynchronously:
during infinitepush push separate `hg debugfillinfinitepushmetadata` process
will be started. Push process won't wait for `debugfillinfinitepushmetadata`
process.

sqlindexapi will be implemented in the next diffs.

Test Plan: arc unit

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: quark, mjpieters, medson, azich, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D5237547

Tasks: 19103180

Signature: t1:5237547:1497397211:569d1ffe3be37c8d14fb7e07e42cd085ad5686d4
2017-06-15 05:19:37 -07:00
Stanislau Hlebik
8c7c7319dd infinitepush: record nodesmetadata
Summary: Let's add a table that will store metadata about infinitepush nodes.

Test Plan: arc unit

Reviewers: #mercurial

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4778867

Tasks: 16899722
2017-03-30 02:07:30 -07:00
Stanislau Hlebik
f6d771aeda infinitepush: addmanybookmarks indexapi method
Summary:
Right now `hg debugbackup` backups only changesets.  We also want to backup
bookmarks.  Since user may have many bookmarks let's add a separate method
that will save many bookmarks at once.

Test Plan: Will be tested in next diffs

Reviewers: durham, rmcelroy, mitrandir, quark

Reviewed By: quark

Subscribers: quark, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4244823

Tasks: 12479677

Signature: t1:4244823:1480596116:7114128dafe2b8a10f599d05d68c1a6dce522f4a
2016-12-01 08:41:23 -08:00
Stanislau Hlebik
c2e54d52a7 infinitepush: make indexapi a context manager
Summary:
As @quark suggested let's make `indexapi` a contextmanager.
It greatly simplifies code

Test Plan: Run `test-infinitepush-*`

Reviewers: durham, rmcelroy, mitrandir, quark

Reviewed By: quark

Subscribers: quark, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4258082

Tasks: 12479677

Signature: t1:4258082:1480596008:32c39150dbb1966e76064ad8d93a808c89d78b71
2016-12-01 08:40:09 -08:00
Stanislau Hlebik
d93c9f9d97 infinitepush: add deletebookmarks method to indexapi
Summary:
This methods accepts list of bookmark patterns to delete (patterns are not
supported yet, but they will be supported).

Test Plan: Will be tested in subsquent diffs

Reviewers: andrasbelo, rmcelroy, durham

Reviewed By: rmcelroy, durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4176016

Tasks: 12479677

Signature: t1:4176016:1479144865:8c9121970878d4d462cc2c679a414999dfdce1f4
2016-11-30 06:49:21 -08:00
Stanislau Hlebik
563a57128e infinitepush: add mysql waittimeout server-side
Summary:
Saw a few timeout errors during infinitepush test.
Previous timeout was 10 secs, it makes sense to increase it.

There was a unit test failures because of the config options.
So this diff also adds missing config docs and copyrights headers.

Test Plan: Build the package, install it locally and run integrationtests

Reviewers: durham, rmcelroy, mitrandir, andrasbelo

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4103689

Tasks: 12479677
2016-11-01 01:44:38 -07:00
stash
b421695c98 infinitepush: do not require mysql for tests
Summary:
Tests doesn't require mysql since it uses fileindexapi.
Let's move sqlindexapi and fileindexapi to different files and include
only necessary file.

Test Plan: Run test-infinitepush-*

Reviewers: durham, simonfar, quark

Reviewed By: quark

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4029460

Tasks: 12479677

Signature: t1:4029460:1476709161:556f80605db9bd3c18dce1e16e996b13bbe96229
2016-10-17 14:23:56 +01:00
Stanislau Hlebik
edf67f905d infinitepush: use addbookmarkandbundle
Summary:
Let's use a function that saves bookmarks, nodes and bundle
in the same time. For fileindexapi this call is equivalent
to addbundle() and addbookmarks() call. For sqlindexapi this
call makes two calls in one transaction.

Test Plan:
1) Run `test-infinitepush-bundlestore.t`
2) Test sql index locally

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4001243

Tasks: 12479677

Signature: t1:4001243:1476227921:9d449b743686762edce4c07d95266ef7ae69c2b5
2016-10-12 02:59:23 -07:00
Stanislau Hlebik
87084f5d59 infinitepush: add check for db connectness
Summary: This check was missing before

Test Plan: It's difficult to test and the change should be safe

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4001229

Tasks: 12479677

Signature: t1:4001229:1476227703:9e40d6efa9892e1e0d7fbcb24fdce3d607a3a7fe
2016-10-12 02:59:07 -07:00
Stanislau Hlebik
7f75c82f54 infinitepush: fix mysql indexapi
Summary:
Bookmark can be moved to another node and node can be moved to another bundle.
In this case INSERT will fail because of the PRIMARY KEY violation.
Let's add ON DUPLICATE KEY UPDATE which will update existing row instead of
creating a new one.

Test Plan: Run `test-infinitepush-bundlestore.t`

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3936281

Tasks: 12479677

Signature: t1:3936281:1475078339:d2ff0a01e1d753570c60172c5ef34ceeb7af1b96
2016-10-02 02:55:36 -07:00
Stanislau Hlebik
21b475a2f0 infinitepush: lint fixes
Summary:
Remove generic exception.
Remove trailing whitespace.
Remove unused import

#accept2ship

Test Plan: Run `test-check-code-hg.t` and `test-infinitepush-bundlestore.t`

Reviewers: #sourcecontrol, ttung, jeroenv

Reviewed By: jeroenv

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3876225

Signature: t1:3876225:1474015261:adff2496c4e36f19e3315856f1a0102eb7e9dc7e
2016-09-16 01:45:33 -07:00
Stanislau Hlebik
39fa21c146 infinitepush: sqlindex
Summary:
Infinitepush index implementation that uses mysql db to store data.

Three tables:
1) Node to bundle
2) Bookmark to node
3) All bundles (this is necessary to
 find unused bundles.

Test Plan: Code was tested but tests are not included in the diff - see comments below

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3811247

Tasks: 12479677

Signature: t1:3811247:1473378520:bc23b3436aa2e618a7bfefd2d571257dfe8ace94
2016-09-09 09:16:43 -07:00
Stanislau Hlebik
812bc05f85 Add get_bookmarks() method
Differential Revision: https://phabricator.intern.facebook.com/D3791007
2016-08-31 01:44:59 -07:00
Stanislau Hlebik
573c1723b1 Use vfs wherever possible
Differential Revision: https://phabricator.intern.facebook.com/D3791002
2016-08-31 01:44:59 -07:00
Stanislau Hlebik
019c55f351 infinitepush: indexapi and store
Summary:
Code was written by Laurent, Durham and Mateusz on hackathon.
Index stores metadata, store stores bundles.
This diff adds interfaces and simple filesystem implementations

Test Plan:
cd tests
py.test *.py

Reviewers: durham, rmcelroy, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D3741963

Tasks: 12479677

Signature: t1:3741963:1472207457:c9e7fb6163ef3cd6fdc1c4bd5113c623169c3a7c
2016-08-26 04:08:28 -07:00