Commit Graph

15 Commits

Author SHA1 Message Date
Maxime Montinet
ae3b7f2364 infinitepush: prevent hg pullbackup from doing an automatic backup
Summary:
Instead of making an autobackup, make hg pullbackup simply
write the backup state from the recently restored backup.

Test Plan:
cd ~/facebook-hg-rpms/fb-hgext/tests
source ../../hg-dev
rt test-infinitepush-*.t

Reviewers: #mercurial, cdown, stash

Reviewed By: stash

Subscribers: mjpieters, medson

Differential Revision: https://phabricator.intern.facebook.com/D6186148

Tasks: T22856338

Tags: bootcamp

Signature: 6186148:1509457565:e66cefc6e95d9b1cfb91e9bf8fa5c40b05e4fb82
2017-10-31 07:00:31 -07:00
Jun Wu
5b2261bcad test-infinitepush: allow database password to be empty
When mysql password is empty, we should not pass `-p` flag.
2017-10-17 13:05:18 -07:00
Stanislau Hlebik
8bc5444998 infinitepush: run at most one backup per hg command
Summary:
Previously we ran auto backups in txnclose hook. But commands like histedit and
rebase create many transactions, and that means that lots of backup processes
are started at the same time. That means that sometimes backup processes use
too much memory.

Instead let's start it once per command if any transaction was opened
during the command.

Test Plan:
Run unit-test.
Run histedit with autobackup enabled, and make sure just one backup process
was started.

Reviewers: #fbhgext, durham

Reviewed By: #fbhgext, durham

Subscribers: durham

Differential Revision: https://phab.mercurial-scm.org/D678
2017-09-20 07:25:52 -07:00
Durham Goode
3102e456ab infinitepush: backout lfs fix
Summary: This broke hg bundle on non-generaldelta repositories.

Test Plan: Ran the tests

Reviewers: #fbhgext, stash, quark

Reviewed By: #fbhgext, quark

Differential Revision: https://phab.mercurial-scm.org/D74
2017-07-13 13:53:40 -07:00
stash@fb.com
35d988093a infinitepush: handle lfs correctly
Summary:
Make sure that infinitepush work fine with lfs. To achieve it we try to use
changegroup3 if it's available and also run prepush hook before infinitepush.

Test Plan: Run `arc unit`

Reviewers: #fbhgext, quark, dsp

Reviewed By: #fbhgext, dsp

Differential Revision: https://phab.mercurial-scm.org/D17
2017-07-12 09:52:03 -07:00
Jun Wu
4a936cee21 codemod: remove extpath in tests
Summary:
Previously, the following pattern is common in our tests:

```
   $ extpath=`dirname $TESTDIR`
   $ cp $extpath/hgext3rd/name.py $TESTTMP # use $TESTTMP substitution in message
   $ cat >> $HGRCPATH<<EOF
   > [extensions]
   > name=$TESTTMP/name.py
   > EOF
```

Now, it gets simplified to:

```
   $ cat >> $HGRCPATH<<EOF
   > [extensions]
   > name=$TESTDIR/../hgext3rd/name.py
   > EOF
```

This removes unnecessary `dirname` and `cp`.

Also fixed a regex that does not match `bytes`:

```
-  transferred 268 bytes in [\d.]+ seconds \([\d.]+ KB/sec\) (re)
+  transferred 268 bytes in 0.3 seconds (939 bytes/sec)
```

Test Plan: arc unit

Reviewers: #mercurial, phillco

Reviewed By: phillco

Subscribers: medson, mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D5270897

Signature: t1:5270897:1497663052:bf860a0b480c751b1e4b53cebf6526193f0f6652
2017-06-16 18:31:25 -07:00
Stanislau Hlebik
8c7c7319dd infinitepush: record nodesmetadata
Summary: Let's add a table that will store metadata about infinitepush nodes.

Test Plan: arc unit

Reviewers: #mercurial

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4778867

Tasks: 16899722
2017-03-30 02:07:30 -07:00
Stanislau Hlebik
526184c5c4 infinitepush: pass reponame to 'DELETE from bookmarkstonode...' query
Summary:
I forgot to insert it, fixing it now. It didn't cause problems before because
we delete bookmarks only during backups and backups usually have different
bookmark names

Test Plan: arc unit

Reviewers: #mercurial, quark

Reviewed By: quark

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4795310

Tasks: 15389402

Signature: t1:4795310:1490810441:0a044ebe776c65b03c633017c33771d32bfa3e77
2017-03-30 01:16:06 -07:00
Stanislau Hlebik
1c556e7a8c infinitepush: move sql schemas to schema.sql file
Summary: Let's avoid copy-paste.

Test Plan: arc unit

Reviewers: #mercurial, rmcelroy, quark

Reviewed By: rmcelroy, quark

Subscribers: rmcelroy, quark, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4778949

Signature: t1:4778949:1490700154:51f6144cdb7682d03c0f6af88691fd907178319c
2017-03-29 03:21:30 -07:00
Stanislau Hlebik
314a434baf infinitepush: better client-side logging
Summary: New logging will have rotation and will separate different users and repos

Test Plan: arc unit

Reviewers: #mercurial, durham

Reviewed By: durham

Subscribers: durham, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4689191

Tasks: 16474976

Signature: t1:4689191:1489172091:d304d4b19ae6cf52f86c11f442fa760b50d1fdf9
2017-03-14 02:13:16 -07:00
Stanislau Hlebik
8c14ed67f3 infinitepush: add new fields to the log
Summary: Add `errormsg` and `reponame`

Test Plan: Run `test-infinitepush-*`

Reviewers: durham, rmcelroy, mitrandir, andrasbelo

Reviewed By: andrasbelo

Subscribers: andrasbelo, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4339606

Tasks: 12479677

Signature: t1:4339606:1482312794:15c04789ba03059c9a178d1adcf2bd09d9cf7b57
2016-12-21 01:41:44 -08:00
Jun Wu
e4e12d3f95 test-infinishpush: some cleanups on setupdb
Summary:
The "parsing getdb.sh's output and cut -d" approach is not the best practise of
passing multiple values between shell scripts. Use "source" and setting shell
variables instead.

Besides, skip the test (exit 80) if getdb.sh does not exist

Test Plan:
Provide a good getdb.sh and test on my devserver.
Delete getdb.sh and make sure the test is skipped.

Reviewers: #sourcecontrol, stash

Reviewed By: stash

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4262570

Signature: t1:4262570:1480668031:cd5371c620889856ff892d671d1ee498438b8487
2016-12-13 13:43:58 -08:00
Jun Wu
5d5fedd10c tweakdefaults: fix check config test
Summary:
This fixes test-check-config-hg.t for tweakdefaults. And did some clean-up
for other minor issues.

I was trying to implement another feature (along with the clean-up) in
tweakdefaults and finally realized it's infeasible and drop the feature. But
the clean-up seems useful thus sent here.

Also change `cp -r` to `cp -R` to pass the usptream check-code test.

Test Plan: `arc unit`

Reviewers: #sourcecontrol, mitrandir

Reviewed By: mitrandir

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4253852

Signature: t1:4253852:1480613350:398e9b234fcc2360dcb8a3e3ad4e5bc5c4377857
2016-11-30 20:58:08 +00:00
Stanislau Hlebik
7dc24c7c82 infinitepush: backup local heads and bookmarks
Summary:
Backuping changesets is not enougn for restore. We also want to backup heads
and bookmarks to completely capture the state of the repo.

This diff is the first step in implementing this functionality. It adds new
bundle2 part which contains encoded dict (simple json encoding is used).
If value in the dict is empty then key is the bookmark pattern to delete.
If value is not empty then key is the bookmark name to save and value is a
node hash. The reason to put them in the same part is to make it possible to
delete and insert into indexapi in one transaction. It's also possible to pass
patterns to delete in part parameters but there is a bug in upstream hg that
limits parameters' size to 256 and we can potentially have longer bookmarks.

Local bookmarks are saved in infinitepush in the following form:
  infinitepush/backups/USERNAME/HOSTNAME/REPOROOT/bookmarks/LOCAL_BOOKMARK_NAME

Local heads are saved in infintiepush in the following form:
  infinitepush/backups/USERNAME/HOSTNAME/REPOROOTheads/HEAD_HASH

Hostname, username and repo root is necessary to distinguish different backups.

Test Plan: Run `test-infinitepush-*`

Reviewers: durham, rmcelroy, mitrandir, quark

Reviewed By: quark

Subscribers: quark, mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4245080

Tasks: 12479677

Signature: t1:4245080:1480518959:aa199d67fac4e2cd2f543651ff56fdd649dac729
2016-12-01 08:44:57 -08:00
Stanislau Hlebik
fd8f0625ef infinitepush: add hg backup command
Summary:
Backups all new non-extinct [1] commits to bundlestore.

When it is called for the first time `hg backup` will backup all draft
visible commits. Next backups will save only new commits since the last
backup (it is recorded in `.hg/store/infinitepushbackuptip`[2]).

It's an initial implementation. Later the following features will be added:
1) It will be called automatically whenever user creates or strips commits
or even creates or deletes bookmarks
2) It will also save all local bookmarks and all local heads
(probably only visible).

Note: calling `pushcmd` directly does not set default values for opts. That means that `--to` will be None and `_scratchbranchmatcher` will throw exception. Let's add a check to ensure that `--to` is never None.

[1] I also want to backup extinct commits (i.e. obsolete invisible commits).
But it will require bigger changes in discovery algorithm, so I'd leave it
for later.
[2] The name is a bit verbose. But I want to keep `infinitepush` part to make
it easier to debug problems.

Test Plan: Run `test-infinitepush-*`

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4147651

Tasks: 12479677

Signature: t1:4147651:1479229440:3eb38880c14f18e9a2fb4eaba44bedf079bca506
2016-11-16 02:13:17 -08:00