facebook/sapling
1
1
Fork 0
mirror of https://github.com/facebook/sapling.git synced 2024-10-07 15:27:13 +03:00
Code Issues Projects Releases Wiki Activity

infinitepush: pullbackup command

Browse Source 
Summary:
As the name suggest it will restore backup made by `hg pushbackup`.

If user has only one backup for the `dest` repo then it will be restored.
But user may have backed up many local repos that points to `dest` repo.
These local repos may reside on different hosts or in different
repo roots. It makes restore ambiguous; `--reporoot` and `--hostname`
options are used to disambiguate.

Example situation:
1) User has only one laptop with mercurial repo `repo` that was cloned
from remote server. He or she run `hg pushbackup`. Then laptop breaks
and user gets a new one, clones the `repo` again and runs `hg restore`.
It automatically restores the backup.
2) User has devserver and laptop and backups were made from both.
Then if user decides to switch devserver and run `hg restore` on the new
devserver he or she has to specify `--hostname`.


Future plans:
1) Add `--user` option to make it possible to restore another user's backup

Test Plan: Run `test-infinitepush-*`

Reviewers: rmcelroy, mitrandir, durham

Reviewed By: durham

Subscribers: mjpieters, #sourcecontrol

Differential Revision: https://phabricator.intern.facebook.com/D4280832

Tasks: 12479677

Signature: t1:4280832:1481565335:2a21ceafa2ff80242076a79693046514434afb40
This commit is contained in:
Stanislau Hlebik 2016-12-16 08:29:08 -08:00
parent 73c92dfa70
commit feecb32f3a
2 changed files with 238 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

125
infinitepush/__init__.py
View File

@ -63,12 +63,13 @@ import json
import logging
import os
import random
import re
import socket
import struct
import tempfile
import time
from collections import defaultdict
from collections import defaultdict, namedtuple
from functools import partial
from hgext3rd.extutil import runshellcommand
from mercurial import (
@ -113,6 +114,9 @@ configscratchpush = 'infinitepush-scratchpush'
_scratchbranchmatcher = lambda x: False
backupbookmarktuple = namedtuple('backupbookmarktuple',
['hostname', 'reporoot', 'localbookmark'])
def _buildexternalbundlestore(ui):
put_args = ui.configlist('infinitepush', 'put_args', [])
put_binary = ui.config('infinitepush', 'put_binary')
@ -747,14 +751,14 @@ def _push(orig, ui, repo, dest=None, *args, **opts):
exchange._localphasemove = oldphasemove
return result
def _getcommonprefix(ui, repo):
def _getcommonuserprefix(ui):
username = ui.shortuser(ui.username())
return '/'.join(('infinitepush', 'backups', username))
def _getcommonprefix(ui, repo):
hostname = socket.gethostname()
result = '/'.join(('infinitepush',
'backups',
username,
hostname))
result = '/'.join((_getcommonuserprefix(ui), hostname))
if not repo.origroot.startswith('/'):
result += '/'
result += repo.origroot
@ -768,15 +772,19 @@ def _getbackupbookmarkprefix(ui, repo):
def _escapebookmark(bookmark):
'''
If `bookmark` contains "/bookmarks/" as a substring then replace it with
"/bookmarksbookmarks/". This will make parsing remote bookmark name
If `bookmark` contains "bookmarks" as a substring then replace it with
"bookmarksbookmarks". This will make parsing remote bookmark name
unambigious.
'''
return bookmark.replace('/bookmarks/', '/bookmarksbookmarks/')
bookmark = encoding.fromlocal(bookmark)
return bookmark.replace('bookmarks', 'bookmarksbookmarks')
def _unescapebookmark(bookmark):
bookmark = encoding.tolocal(bookmark)
return bookmark.replace('bookmarksbookmarks', 'bookmarks')
def _getbackupbookmarkname(ui, bookmark, repo):
bookmark = encoding.fromlocal(bookmark)
bookmark = _escapebookmark(bookmark)
return '/'.join((_getbackupbookmarkprefix(ui, repo), bookmark))
@ -795,6 +803,11 @@ def _getremote(repo, ui, dest, **opts):
dest = path.pushloc or path.loc
return hg.peer(repo, opts, dest)
def _getcommandandoptions(command):
pushcmd = commands.table[command][0]
pushopts = dict(opt[1:3] for opt in commands.table[command][1])
return pushcmd, pushopts
@command('pushbackup',
[('', 'background', None, 'run backup in background')])
def backup(ui, repo, dest=None, **opts):
@ -892,6 +905,98 @@ def backup(ui, repo, dest=None, **opts):
f.write(str(currenttiprev) + ' ' + currentbookmarkshash)
return 0
def _parsebackupbookmark(ui, backupbookmark):
'''Parses backup bookmark and returns info about it
Backup bookmark may represent either a local bookmark or a head.
Returns None if backup bookmark has wrong format or tuple.
First entry is a hostname where this bookmark came from.
Second entry is a root of the repo where this bookmark came from.
Third entry in a tuple is local bookmark if backup bookmark
represents a local bookmark and None otherwise.
'''
commonre = '^{}/([-\w.]+)(/.*)'.format(re.escape(_getcommonuserprefix(ui)))
bookmarkre = commonre + '/bookmarks/(.*)$'
headsre = commonre + '/heads/[a-f0-9]{40}$'
match = re.search(bookmarkre, backupbookmark)
if not match:
match = re.search(headsre, backupbookmark)
if not match:
return None
# It's a local head not a local bookmark.
# That's why localbookmark is None
return backupbookmarktuple(hostname=match.group(1),
reporoot=match.group(2),
localbookmark=None)
return backupbookmarktuple(hostname=match.group(1),
reporoot=match.group(2),
localbookmark=_unescapebookmark(match.group(3)))
@command('pullbackup', [
('', 'reporoot', '', 'root of the repo to restore'),
('', 'hostname', '', 'hostname of the repo to restore')])
def restore(ui, repo, dest=None, **opts):
"""
Pulls commits from infinitepush that were previously saved with
`hg pushbackup`.
If user has only one backup for the `dest` repo then it will be restored.
But user may have backed up many local repos that points to `dest` repo.
These local repos may reside on different hosts or in different
repo roots. It makes restore ambiguous; `--reporoot` and `--hostname`
options are used to disambiguate.
"""
other = _getremote(repo, ui, dest, **opts)
sourcereporoot = opts.get('reporoot')
sourcehostname = opts.get('hostname')
pattern = _getcommonuserprefix(ui) + '/*'
fetchedbookmarks = other.listkeyspatterns('bookmarks', patterns=[pattern])
reporoots = set()
hostnames = set()
nodestopull = set()
localbookmarks = {}
for book, node in fetchedbookmarks.iteritems():
parsed = _parsebackupbookmark(ui, book)
if parsed:
if sourcereporoot and sourcereporoot != parsed.reporoot:
continue
if sourcehostname and sourcehostname != parsed.hostname:
continue
nodestopull.add(node)
if parsed.localbookmark:
localbookmarks[parsed.localbookmark] = node
reporoots.add(parsed.reporoot)
hostnames.add(parsed.hostname)
else:
ui.warn(_('wrong format of backup bookmark: %s') % book)
if len(reporoots) > 1:
raise error.Abort(
_('ambiguous repo root to restore: %s') % sorted(reporoots),
hint=_('set --reporoot to disambiguate'))
if len(hostnames) > 1:
raise error.Abort(
_('ambiguous hostname to restore: %s') % sorted(hostnames),
hint=_('set --hostname to disambiguate'))
pullcmd, pullopts = _getcommandandoptions('^pull')
pullopts['rev'] = list(nodestopull)
result = pullcmd(ui, repo, **pullopts)
with repo.wlock():
with repo.lock():
with repo.transaction('bookmark') as tr:
for scratchbook, hexnode in localbookmarks.iteritems():
repo._bookmarks[scratchbook] = bin(hexnode)
repo._bookmarks.recordchange(tr)
return result
def _phasemove(orig, pushop, nodes, phase=phases.public):
"""prevent commits from being marked public

123
tests/test-infinitepush-pullbackup.t Normal file
View File

@ -0,0 +1,123 @@
$ . "$TESTDIR/library.sh"
$ . "$TESTDIR/library-infinitepush.sh"
$ setupcommon
Setup server
$ hg init repo
$ cd repo
$ setupserver
$ cd ..
Create backup source
$ hg clone ssh://user@dummy/repo backupsource -q
Create restore target
$ hg clone ssh://user@dummy/repo restored -q
Backup
$ cd backupsource
$ mkcommit firstcommit
$ hg book abook
$ hg pushbackup
searching for changes
remote: pushing 1 commit:
remote: 89ecc969c0ac firstcommit
$ cd ..
Restore
$ cd restored
$ hg pullbackup
pulling from ssh://user@dummy/repo
no changes found
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
$ hg log --graph -T '{desc}'
o firstcommit
$ hg book
abook 0:89ecc969c0ac
$ cd ..
Create second backup source
$ hg clone ssh://user@dummy/repo backupsource2 -q
$ cd backupsource2
$ mkcommit secondcommit
$ hg book secondbook
$ hg pushbackup
searching for changes
remote: pushing 1 commit:
remote: c1bfda8efb6e secondcommit
$ cd ..
Restore with ambiguous repo root
$ rm -rf restored
$ hg clone ssh://user@dummy/repo restored -q
$ cd restored
$ hg pullbackup
abort: ambiguous repo root to restore: ['$TESTTMP/backupsource', '$TESTTMP/backupsource2']
(set --reporoot to disambiguate)
[255]
$ hg pullbackup --reporoot $TESTTMP/backupsource2
pulling from ssh://user@dummy/repo
no changes found
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files
$ hg log --graph -T '{desc}'
o secondcommit
$ cd ..
Check bookmarks escaping
$ cd backupsource
$ hg book book/bookmarks/somebook
$ hg book book/bookmarksbookmarks/somebook
$ hg pushbackup
$ cd ../restored
$ hg pullbackup --reporoot $TESTTMP/backupsource
pulling from ssh://user@dummy/repo
searching for changes
no changes found
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
$ hg book
abook 1:89ecc969c0ac
book/bookmarks/somebook 1:89ecc969c0ac
book/bookmarksbookmarks/somebook 1:89ecc969c0ac
secondbook 0:c1bfda8efb6e
$ cd ..
Create a repo with `/bookmarks/` in path
$ mkdir bookmarks
$ cd bookmarks
$ hg clone ssh://user@dummy/repo backupsource3 -q
$ cd backupsource3
$ mkcommit commitinweirdrepo
$ hg book bookbackupsource3
$ hg pushbackup
searching for changes
remote: pushing 1 commit:
remote: a2a9ae518b62 commitinweirdrepo
$ cd ../../restored
$ hg pullbackup --reporoot $TESTTMP/bookmarks/backupsource3
pulling from ssh://user@dummy/repo
searching for changes
no changes found
adding changesets
adding manifests
adding file changes
added 1 changesets with 1 changes to 1 files (+1 heads)
$ hg book
abook 1:89ecc969c0ac
book/bookmarks/somebook 1:89ecc969c0ac
book/bookmarksbookmarks/somebook 1:89ecc969c0ac
bookbackupsource3 2:a2a9ae518b62
secondbook 0:c1bfda8efb6e
$ cd ..