p4fastimport : introducing fast Perforce to Mercurial convert extension

Summary:
`p4fastimport` is a fast convert extensions for Perforce to Mercurial. It
is designed to generate filelogs in parallel from Perforce. It tries to
minimize the use of Perforce commands and reads from the the Perforce
store on a Perforce server directly.

The core of p4fastimport is the idea to generate a Mercurial filelog
directly from the underlying Perforce data, as a Perforce file in most
cases matches a filelog directly (per-file branches is an exception). To
generate a filelog we are reading each file for an imported revision. A
file in Perforce is locally either stored in RCS, as a compressed GZIP
or as an flat file (binaries). If we do not find a version locally on
disk we fallback to downloading it from Perforce.

We are generating manifests after all filelogs are imported. A manifest
is constructed by adding and removing files from an initial state. We
are generating the correct offset from a manifest into the filelog by
keeping track of how often a file was touched.

We then generate the changelog.

Linkrev generation is a bit tricky. For every file in Perforce know
to which changelist it belongs, as it's stored revisions contains the
changelist. E.g.  1.1422 is the file changed in the changelist 1422 (this
refers to the "original" changelist, before a potential renumbering,
which is why we use the -O switch).  We use the CL number obtained
from the revision to reverse lookup the offset in the sorted list of
changelists, which corresponds to it's place in the changelog later,
and therefore it's correct linkrev.

Parallel imports: In order to run parallel imports we MUST keep one lock
at a time, even if we import multiple file logs at the same time. However
filelogs use a singular `fncache`, which will be corrupted if we generate
filelogs in parallel. To avoid this, repositories must be generated with
*fncache* disabled! This restricts `p4fastimport` with workers to run
only on case sensitive file systems.

Test Plan:
The included tests as well as multiple imports from a small testing
Perforce client. Afterwards successfully run `hg verify`

  make tests

Reviewers: #idi, quark, durham

Reviewed By: durham

Subscribers: mjpieters

Differential Revision: https://phabricator.intern.facebook.com/D4776651

Signature: t1:4776651:1492015012:0161c4f45eab4d3b64597d012188c5f2007e8f7d
This commit is contained in:
David Soria Parra 2017-04-13 11:11:09 -07:00
parent 57dc185f42
commit ef08c10f5b
12 changed files with 1628 additions and 0 deletions

173
p4fastimport/__init__.py Normal file
View File

@ -0,0 +1,173 @@
# (c) 2017-present Facebook Inc.
"""p4fastimport - A fast importer from Perforce to Mercurial
Config example:
[p4fastimport]
# whether use worker or not
useworker = false
# trace copies?
copytrace = false
"""
from __future__ import absolute_import
import collections
import json
from . import (
p4,
importer,
util,
)
from mercurial.i18n import _
from mercurial import (
cmdutil,
error,
worker,
)
def create(tr, ui, repo, importset, filelogs):
for filelog in filelogs:
# If the Perforce is case insensitive a filelog can map to
# multiple filenames. For exmaple A.txt and a.txt would show up in the
# same filelog. It would be more appropriate to update the filelist
# after receiving the initial filelist but this would not be parallel.
fi = importer.FileImporter(ui, repo, importset, filelog)
mdict = fi.create(tr)
yield 1, json.dumps(mdict)
# -> Dict[Int, List[str]]
#def create_runlist(ui, repo, filelist, path):
# def follow(fi, depmap):
# # XXX: Careful about stackoverflow
# if fi.dependency[0] is not None:
# # XXX: Don't visit the same files twice
# flog = importer.FileImporter(ui, repo, path, fi.dependency[1])
# add, depmap = follow(flog, depmap)
# depmap[fi._depotfname] += add
# return depmap[fi._depotfname] + 1, depmap
#
# depmap = collections.defaultdict(lambda: 0)
# for filename in filelist:
# fi = importer.FileImporter(ui, repo, path, filename)
# __, depmap = follow(fi, depmap)
# runlist = collections.defaultdict(list)
# for k, v in depmap.iteritems():
# runlist[v].append(k)
# return runlist
cmdtable = {}
command = cmdutil.command(cmdtable)
@command(
'p4fastimport',
[('s', 'start', None, _('start of the CL range to import'), _('REV')),
('e', 'end', None, _('end of the CL range to import'), _('REV')),
('P', 'path', '.', _('path to the local depot store'), _('PATH'))],
_('hg p4fastimport [-s start] [-e end] [-P PATH] [CLIENT]'),
inferrepo=True)
def p4fastimport(ui, repo, client, **opts):
if 'fncache' in repo.requirements:
raise error.Abort(_('fncache must be disabled'))
basepath = opts.get('path')
# A client defines checkout behavior for a user. It contains a list of
# views.A view defines a set of files and directories to check out from a
# Perforce server and their mappins to local disk, e.g.:
# //depot/foo/... //client/x/...
# would map the files that are stored on the
# server under foo/* locally under x/*.
# 1. Return all the changelists touching files in a given client view.
ui.note(_('loading changelist numbers.\n'))
changelists = list(p4.parse_changes(client))
ui.note(_('%d changelists to import.\n') % len(changelists))
# 2. Get a list of files that we will have to import from the depot with
# it's full path in the depot.
ui.note(_('loading list of files.\n'))
filelist = set()
for fileinfo in p4.parse_filelist(client):
if fileinfo['action'] in p4.SUPPORTED_ACTIONS:
filelist.add(fileinfo['depotFile'])
else:
ui.warn(_('unknown action %s: %s\n') % (fileinfo['action'],
fileinfo['depotFile']))
ui.note(_('%d files to import.\n') % len(filelist))
importset = importer.ImportSet(changelists, filelist, storagepath=basepath)
p4filelogs = []
for i, f in enumerate(importset.filelogs()):
ui.progress(_('loading filelog'), i, item=f, unit="filelog",
total=len(filelist))
p4filelogs.append(f)
ui.progress(_('loading filelog'), None)
# runlist is used to topologically order files which were branched (Perforce
# uses per-file branching, not per-repo branching). If we do copytracing a
# file A' which was branched off A will be considered a copy of A. Therefore
# we need to import A' before A. In this case A' will have a dependency
# counter +1 of A's, and therefore being imported after A. If copy tracing
# is disabled this is not needed and we can import files in arbitrary order.
runlist = collections.OrderedDict()
if ui.configbool('p4fastimport', 'copytrace', False):
raise error.Abort(_('copytracing is broken'))
# ui.note(_('Tracing file copies.\n'))
# runlist = create_runlist(ui, repo, changelists, linkrevmap,
# filelist, basepath)
# copy_tracer = importer.CopyTracer(filelist)
else:
runlist[0] = p4filelogs
ui.note(_('importing repository.\n'))
wlock = repo.wlock()
lock = repo.lock()
tr = None
try:
tr = repo.transaction('import')
for a, b in importset.caseconflicts:
ui.warn(_('case conflict: %s and %s\n') % (a, b))
# 3. Import files.
count = 0
fileflags = {}
for filelogs in map(sorted, runlist.values()):
wargs = (tr, ui, repo, importset)
# 0.4 is the cost per argument. So if we have at least 100 files
# on a 4 core machine than our linear cost outweights the
# drawback of spwaning. We are overwritign this if we force a
# worker to run with a ridiculous high number.
weight = 0.0 # disable worker
if ui.config('p4fastimport', 'useworker', None) == 'force':
weight = 100000.0 # force worker
elif ui.configbool('p4fastimport', 'useworker', False):
weight = 0.04 # normal weight
# Fix duplicated messages before
# https://www.mercurial-scm.org/repo/hg-committed/rev/9d3d56aa1a9f
ui.flush()
prog = worker.worker(ui, weight, create, wargs, filelogs)
for i, serialized in prog:
ui.progress(_('importing'), count, item='file', unit='file',
total=len(p4filelogs))
# Json converts to UTF8 and int keys to strings, so we have to
# convert back. TODO: Find a better way to handle this.
fileflags.update(util.decodefileflags(json.loads(serialized)))
count += i
ui.progress(_('importing'), None)
# 4. Generate manifest and changelog based on the filelogs we imported
clog = importer.ChangeManifestImporter(ui, repo, importset)
clog.create(tr, fileflags)
tr.close()
ui.note(_('%d revision(s), %d file(s) imported.\n') % (
len(changelists), count))
finally:
if tr:
tr.release()
lock.release()
wlock.release()

348
p4fastimport/importer.py Normal file
View File

@ -0,0 +1,348 @@
# (c) 2017-present Facebook Inc.
from __future__ import absolute_import
import collections
import gzip
import os
import re
from mercurial.i18n import _
from mercurial import (
error,
manifest,
node,
util,
)
from . import p4
from .util import localpath, caseconflict
class ImportSet(object):
def __init__(self, changelists, filelist, storagepath):
self.changelists = sorted(changelists)
self.filelist = filelist
self.storagepath = storagepath
def linkrev(self, cl):
return self._linkrevmap[cl]
@util.propertycache
def _linkrevmap(self):
return {c.cl: idx for idx, c in enumerate(self.changelists)}
@util.propertycache
def caseconflicts(self):
return caseconflict(self.filelist)
def filelogs(self):
return list(p4.parse_filelogs(self.changelists, self.filelist))
class ChangeManifestImporter(object):
def __init__(self, ui, repo, importset):
self._ui = ui
self._repo = repo
self._importset = importset
@util.propertycache
def usermap(self):
m = {}
for user in p4.parse_usermap():
m[user['User']] = '%s <%s>' % (user['FullName'], user['Email'])
return m
def create(self, tr, fileflags):
mp1, mp2 = node.nullid, node.nullid
cp1, cp2 = node.nullid, node.nullid
mrevlog = self._repo._constructmanifest()
clog = self._repo.changelog
mf = manifest.manifestdict()
# revnumdict keeps track of the rev per file, when we see a file
# modified or added we increment it.
revnumdict = collections.defaultdict(lambda: 0)
for i, change in enumerate(self._importset.changelists):
self._ui.progress(_('importing change'), pos=i, item=change,
unit='changes', total=len(self._importset.changelists))
self._ui.debug(
_('changelist %d: Writing manifest.\n') % change.cl)
added, modified, removed = change.files
# generate manifest mappings of filenames to filenodes
rmod = filter(lambda f: f in self._importset.filelist, removed)
rf = map(localpath, rmod)
for path in rf:
if path in mf:
del mf[path]
addmod = filter(lambda f: f in self._importset.filelist,
added + modified)
amf = map(localpath, addmod)
for path in amf:
filelog = self._repo.file(path)
try:
fnode = filelog.node(revnumdict[path])
except (error.LookupError, IndexError):
raise error.Abort("can't find rev %d for %s cl %d" %
(revnumdict[path], path, change.cl))
revnumdict[path] += 1
mf[path] = fnode
if path in fileflags and change.cl in fileflags[path]:
mf.setflag(path, fileflags[path][change.cl])
linkrev = self._importset.linkrev(change.cl)
mp1 = mrevlog.addrevision(
mf.text(mrevlog._usemanifestv2), tr, linkrev, mp1, mp2)
desc = change.parsed['desc']
if desc == '':
desc = '** empty changelist description **'
desc = desc.decode('ascii', 'ignore')
shortdesc = desc.splitlines()[0]
username = change.parsed['user']
self._ui.debug(
_('changelist %d: Writing changelog: %s\n') % (change.cl,
shortdesc))
cp1 = clog.add(
mp1,
amf + rf,
desc,
tr,
cp1,
cp2,
user=username,
date=(change.parsed['time'], 0),
extra={'p4changelist': change.cl})
mf = mf.copy()
self._ui.progress(_('importing change'), pos=None)
class RCSImporter(collections.Mapping):
def __init__(self, path):
self._path = path
@property
def rcspath(self):
return '%s,v' % self._path
def __getitem__(self, rev):
if rev in self.revisions:
return self.content(rev)
return IndexError
def __len__(self):
return len(self.revisions)
def __iter__(self):
for r in self.revisions:
yield r, self[r]
def content(self, rev):
text = None
if os.path.isfile(self.rcspath):
cmd = 'co -q -p1.%d %s' % (rev, util.shellquote(self.rcspath))
with util.popen(cmd, mode='rb') as fp:
text = fp.read()
return text
@util.propertycache
def revisions(self):
revs = set()
if os.path.isfile(self.rcspath):
stdout = util.popen('rlog %s' % util.shellquote(self.rcspath),
mode='rb')
for l in stdout.readlines():
m = re.match('revision 1.(\d+)', l)
if m:
revs.add(int(m.group(1)))
return revs
T_FLAT, T_GZIP = 1, 2
class FlatfileImporter(collections.Mapping):
def __init__(self, path):
self._path = path
@property
def dirpath(self):
return '%s,d' % self._path
def __len__(self):
return len(self.revisions)
def __iter__(self):
for r in self.revisions:
yield r, self[r]
def filepath(self, rev):
flat = '%s/1.%d' % (self.dirpath, rev)
gzip = '%s/1.%d.gz' % (self.dirpath, rev)
if os.path.exists(flat):
return flat, T_FLAT
if os.path.exists(gzip):
return gzip, T_GZIP
return None, None
def __getitem__(self, rev):
text = self.content(rev)
if text is None:
raise IndexError
return text
@util.propertycache
def revisions(self):
revs = set()
if os.path.isdir(self.dirpath):
for name in os.listdir(self.dirpath):
revs.add(int(name.split('.')[1]))
return revs
def content(self, rev):
path, type = self.filepath(rev)
text = None
if type == T_GZIP:
with gzip.open(path, 'rb') as fp:
text = fp.read()
if type == T_FLAT:
with open(path, 'rb') as fp:
text = fp.read()
return text
class P4FileImporter(collections.Mapping):
"""Read a file from Perforce in case we cannot find it locally, in
particular when there was branch or a rename"""
def __init__(self, filelog):
self._filelog = filelog # type: p4.P4Filelog
def __len__(self):
return len(self.revisions)
def __iter__(self):
for r in self.revisions:
yield r, self[r]
def __getitem__(self, rev):
text = self.content(rev)
if text is None:
raise IndexError
return text
@util.propertycache
def revisions(self):
return self._filelog.revisions
def content(self, clnum):
return p4.get_file(self._filelog.depotfile, clnum=clnum)
class CopyTracer(object):
def __init__(self, repo, filelist, depotname):
self._repo = repo
self._filelist = filelist
self._depotpath = depotname
def iscopy(self, cl):
bcl, bsrc = self.dependency
return bcl is not None and bcl == cl
def copydata(self, cl):
meta = {}
bcl, bsrc = self.dependency
if bcl is not None and bcl == cl:
assert False
p4fi = P4FileImporter(self._depotpath)
copylog = self._repo.file(localpath(bsrc))
# XXX: This is most likely broken, as we don't take add->delete->add into
# account
copynode = copylog.node(p4fi.filelog.branchrev - 1)
meta["copy"] = localpath(bsrc)
meta["copyrev"] = node.hex(copynode)
return meta
@util.propertycache
def dependency(self):
"""Returns a tuple. First value is the cl number when the file was
branched, the second parameter is the file it was branchedfrom. Other
otherwise it returns (None, None)
"""
filelog = p4.parse_filelog(self._depotpath)
bcl = filelog.branchcl
bsrc = filelog.branchsource
if bcl is not None and bsrc in self._filelist:
return filelog.branchcl, filelog.branchsource
return None, None
class FileImporter(object):
def __init__(self, ui, repo, importset, filelog):
self._ui = ui
self._repo = repo
self._i = importset
self._filelog = filelog # type: p4.P4Filelog
@property
def relpath(self):
# XXX: Do the correct mapping to the clientspec
return localpath(self._filelog.depotfile)
@util.propertycache
def storepath(self):
path = os.path.join(self._i.storagepath, self.relpath)
if p4.config('caseHandling') == 'insensitive':
return path.lower()
return path
def create(self, tr, copy_tracer=None):
assert tr is not None
p4fi = P4FileImporter(self._filelog)
rcs = RCSImporter(self.storepath)
flat = FlatfileImporter(self.storepath)
local_revs = rcs.revisions | flat.revisions
revs = []
for c in self._i.changelists:
if c.cl in p4fi.revisions and not self._filelog.isdeleted(c.cl):
revs.append(c)
fileflags = collections.defaultdict(dict)
lastlinkrev = 0
for c in sorted(revs):
linkrev = self._i.linkrev(c.cl)
fparent1, fparent2 = node.nullid, node.nullid
# invariant: our linkrevs do not criss-cross.
assert linkrev >= lastlinkrev
lastlinkrev = linkrev
filelog = self._repo.file(self.relpath)
if len(filelog) > 0:
fparent1 = filelog.tip()
# select the content
text = None
if c.origcl in local_revs:
if c.origcl in rcs.revisions:
text, src = rcs.content(c.origcl), 'rcs'
elif c.origcl in flat.revisions:
text, src = flat.content(c.origcl), 'gzip'
elif c.cl in p4fi.revisions:
text, src = p4fi.content(c.cl), 'p4'
if text is None:
raise error.Abort('error generating file content %d %s' % (
c.cl, self.relpath))
meta = {}
# iscopy = copy_tracer and copy_tracer.iscopy(c.cl)
#if iscopy:
# meta = copy_tracer.copydata(c.cl)
if self._filelog.isexec(c.cl):
fileflags[self.relpath][c.cl] = 'x'
if self._filelog.issymlink(c.cl):
fileflags[self.relpath][c.cl] = 'l'
if self._filelog.iskeyworded(c.cl):
# Replace keyword expansion
pass
h = filelog.add(text, meta, tr, linkrev, fparent1, fparent2)
self._ui.debug(
'writing filelog: %s, p1 %s, linkrev %d, %d bytes, src: %s, '
'path: %s\n' % (node.short(h), node.short(fparent1), linkrev,
len(text), src, self.relpath))
return fileflags

286
p4fastimport/p4.py Normal file
View File

@ -0,0 +1,286 @@
# (c) 2017-present Facebook Inc.
import collections
import marshal
from mercurial import (
util,
)
class P4Exception(Exception):
pass
def loaditer(f):
"Yield the dictionary objects generated by p4"
try:
while True:
d = marshal.load(f)
if not d:
break
yield d
except EOFError:
pass
def revrange(start=None, end=None):
"""Returns a revrange to filter a Perforce path. If start and end are None
we return an empty string as lookups without a revrange filter are much
faster in Perforce"""
revrange = ""
if end is not None or start is not None:
start = '0' if start is None else str(start)
end = '#head' if end is None else str(end)
revrange = "@%s,%s" % (start, end)
return revrange
def parse_info():
cmd = 'p4 -ztag -G info'
stdout = util.popen(cmd, mode='rb')
return marshal.load(stdout)
_config = None
def config(key):
global _config
if _config is None:
_config = parse_info()
return _config[key]
def parse_changes(client, startrev=None, endrev=None):
"Read changes affecting the path"
cmd = 'p4 --client %s -ztag -G changes -s submitted //%s/...%s' % (
util.shellquote(client),
util.shellquote(client),
revrange(startrev, endrev))
stdout = util.popen(cmd, mode='rb')
for d in loaditer(stdout):
c = d.get("change", None)
oc = d.get("oldChange", None)
if oc:
yield P4Changelist(int(oc), int(c))
elif c:
yield P4Changelist(int(c), int(c))
def parse_filelist(client, startrev=None, endrev=None):
if startrev is None:
startrev = 0
cmd = 'p4 --client %s -G files -a //%s/...%s' % (
util.shellquote(client),
util.shellquote(client),
revrange(startrev, endrev))
stdout = util.popen(cmd, mode='rb')
for d in loaditer(stdout):
c = d.get('depotFile', None)
if c:
yield d
def get_file(path, rev=None, clnum=None):
"""Returns a file from Perforce"""
r = '#head'
if rev:
r = '#%d' % rev
if clnum:
r = '@%d' % clnum
cmd = 'p4 print -q %s%s' % (util.shellquote(path), r)
stdout = util.popen(cmd, mode='rb')
content = stdout.read()
return content
def parse_cl(clnum):
"""Returns a description of a change given by the clnum. CLnum can be an
original CL before renaming"""
cmd = 'p4 -ztag -G describe -O %d' % clnum
stdout = util.popen(cmd, mode='rb')
try:
return marshal.load(stdout)
except Exception:
raise P4Exception(stdout)
def parse_usermap():
cmd = 'p4 -G users'
stdout = util.popen(cmd, mode='rb')
try:
for d in loaditer(stdout):
if d.get('User'):
yield d
except Exception:
raise P4Exception(stdout)
def parse_client(client):
cmd = 'p4 -G client -o %s' % util.shellquote(client)
stdout = util.popen(cmd, mode='rb')
try:
clientspec = marshal.load(stdout)
except Exception:
raise P4Exception(stdout)
views = {}
for client in clientspec:
if client.startswith("View"):
sview, cview = clientspec[client].split()
# XXX: use a regex for this
cview = cview.lstrip('/') # remove leading // from the local path
cview = cview[cview.find("/") + 1:] # remove the clientname part
views[sview] = cview
return views
def parse_fstat(clnum, filter=None):
cmd = 'p4 -G fstat -e %d -T ' \
'"depotFile,headAction,headType,headRev" "//..."' % clnum
stdout = util.popen(cmd, mode='rb')
try:
for d in loaditer(stdout):
if d.get('depotFile') and (filter is None or filter(d)):
yield {
'depotFile': d['depotFile'],
'action': d['headAction'],
'type': d['headType'],
'rev': d['headRev'],
}
except Exception:
raise P4Exception(stdout)
_filelogs = collections.defaultdict(dict)
def parse_filelogs(changelists, filelist):
# we can probably optimize this by using fstat only in the case-inensitive
# case and only for conflicts.
global _filelogs
for cl in changelists:
fstats = parse_fstat(cl.cl, lambda f: f['depotFile'] in filelist)
for fstat in fstats:
_filelogs[fstat['depotFile']][cl.cl] = fstat
for p4filename, filelog in _filelogs.iteritems():
yield P4Filelog(p4filename, filelog)
class P4Filelog(object):
def __init__(self, depotfile, data):
self._data = data
self._depotfile = depotfile
# @property
# def branchcl(self):
# return self._parsed[1]
#
# @property
# def branchsource(self):
# if self.branchcl:
# return self.parsed[self.branchcl]['from']
# return None
#
# @property
# def branchrev(self):
# if self.branchcl:
# return self.parsed[self.branchcl]['rev']
# return None
def __cmp__(self, other):
return (self.depotfile > other.depotfile) - (self.depotfile <
other.depotfile)
@property
def depotfile(self):
return self._depotfile
@property
def revisions(self):
return sorted(self._data.keys())
def isdeleted(self, clnum):
return self._data[clnum]['action'] in ['move/delete', 'delete']
def isexec(self, clnum):
t = self._data[clnum]['type']
return 'xtext' == t or '+x' in t
def issymlink(self, clnum):
t = self._data[clnum]['type']
return 'symlink' in t
def iskeyworded(self, clnum):
t = self._data[clnum]['type']
return '+k' in t
ACTION_EDIT = ['edit', 'integrate']
ACTION_ADD = ['add', 'branch', 'move/add']
ACTION_DELETE = ['delete', 'move/delete']
SUPPORTED_ACTIONS = ACTION_EDIT + ACTION_ADD + ACTION_DELETE
class P4Changelist(object):
def __init__(self, origclnum, clnum):
self._clnum = clnum
self._origclnum = origclnum
def __repr__(self):
return '<P4Changelist %d>' % self._clnum
@property
def cl(self):
return self._clnum
@property
def origcl(self):
return self._origclnum
def __cmp__(self, other):
return (self.cl > other.cl) - (self.cl < other.cl)
def __hash__(self):
"""Ensure we are matching changelist numbers in sets and hashtables,
which the importer uses to ensure uniqueness of an imported changeset"""
return hash((self.origcl, self.cl))
@util.propertycache
def parsed(self):
return self.load()
def load(self):
"""Parse perforces awkward format"""
files = {}
info = parse_cl(self._clnum)
i = 0
while True:
fidx = 'depotFile%d' % i
aidx = 'action%d' % i
ridx = 'rev%d' % i
#XXX: Handle oldChange vs change
if fidx not in info:
break
files[info[fidx]] = {
'rev': int(info[ridx]),
'action': info[aidx],
}
i += 1
return {
'files': files,
'desc': info['desc'],
'user': info['user'],
'time': int(info['time']),
}
def rev(self, fname):
return self.parsed['files'][fname]['rev']
@property
def files(self):
"""Returns added, modified and removed files for a changelist.
The current mapping is:
Mercurial | Perforce
---------------------
add | add, branch, move/add
modified | edit, integrate
removed | delete, move/delte
"""
a, m, r = [], [], []
for fname, info in self.parsed['files'].iteritems():
if info['action'] in ACTION_EDIT:
m.append(fname)
elif info['action'] in ACTION_ADD:
a.append(fname)
elif info['action'] in ACTION_DELETE:
r.append(fname)
else:
assert False
return a, m, r

32
p4fastimport/util.py Normal file
View File

@ -0,0 +1,32 @@
# (c) 2017-present Facebook Inc.
from __future__ import absolute_import
import collections
import os
def localpath(p):
return p.lstrip('/')
def storepath(b, p, ci=False):
p = os.path.join(b, p)
if ci:
p = p.lower()
return p
def caseconflict(filelist):
temp = {}
conflicts = []
for this in filelist:
if this.lower() in temp:
other = temp[this.lower()]
if this != other:
conflicts.append(sorted([this, other]))
temp[this.lower()] = this
return sorted(conflicts)
def decodefileflags(json):
r = collections.defaultdict(dict)
for k, d in json.items():
for n, v in d.items():
r[k][int(n)] = v.encode('ascii')
return r

View File

@ -17,6 +17,9 @@ New errors are not allowed. Warnings are strongly discouraged.
> hg files --cwd $RUNTESTDIR/.. "set:(**.py or **.txt) - tests/**" | sed "s#^#${RUNTESTDIR}/../#"
> ) | sed 's|\\|/|g' |
> $PYTHON $RUNTESTDIR/../contrib/check-config.py
elif ui.configbool('p4fastimport', 'useworker', False):
conflict on p4fastimport.useworker: ('bool', '') != ('str', '')
repo.ui.config("paths", "default")))
conflict on paths.default: ('str', '') != ('str', '<variable>')

View File

@ -0,0 +1,115 @@
$ echo "[extensions]" >> $HGRCPATH
$ echo "p4fastimport= " >> $HGRCPATH
create p4 depot
$ p4wd=`pwd`/p4
$ hgwd=`pwd`/hg
$ P4ROOT=`pwd`/depot; export P4ROOT
$ P4AUDIT=$P4ROOT/audit; export P4AUDIT
$ P4JOURNAL=$P4ROOT/journal; export P4JOURNAL
$ P4LOG=$P4ROOT/log; export P4LOG
$ P4PORT=localhost:$HGPORT; export P4PORT
$ P4DEBUG=1; export P4DEBUG
$ mkdir $hgwd
$ mkdir $p4wd
$ cd $p4wd
start the p4 server
$ [ ! -d $P4ROOT ] && mkdir $P4ROOT
$ p4d -C1 -f -J off >$P4ROOT/stdout 2>$P4ROOT/stderr &
$ echo $! >> $DAEMON_PIDS
$ trap "echo stopping the p4 server ; p4 admin stop" EXIT
$ # wait for the server to initialize
$ while ! p4 ; do
> sleep 1
> done >/dev/null 2>/dev/null
create a client spec
$ cd $p4wd
$ P4CLIENT=hg-p4-import; export P4CLIENT
$ DEPOTPATH=//depot/...
$ p4 client -o | sed '/^View:/,$ d' >p4client
$ echo View: >>p4client
$ echo " $DEPOTPATH //$P4CLIENT/..." >>p4client
$ p4 client -i <p4client
Client hg-p4-import saved.
populate the depot
$ mkdir Main
$ echo a > Main/a
$ p4 add Main/a
//depot/Main/a#1 - opened for add
$ p4 submit -d initial
Submitting change 1.
Locking 1 files ...
add //depot/Main/a#1
Change 1 submitted.
$ p4 edit Main/a
//depot/Main/a#1 - opened for edit
$ p4 move Main/a Main/b
//depot/Main/b#1 - moved from //depot/Main/a#1
$ p4 submit -d moveway
Submitting change 2.
Locking 2 files ...
move/delete //depot/Main/a#2
move/add //depot/Main/b#1
Change 2 submitted.
$ p4 edit Main/b
//depot/Main/b#1 - opened for edit
$ p4 move Main/b Main/A
//depot/Main/A#2 - moved from //depot/Main/b#1
$ p4 submit -d moveback
Submitting change 3.
Locking 2 files ...
move/add //depot/Main/A#3
move/delete //depot/Main/b#2
Change 3 submitted.
import
$ cd $hgwd
$ hg init --config 'format.usefncache=False'
$ hg p4fastimport --debug -P $P4ROOT hg-p4-import
loading changelist numbers.
3 changelists to import.
loading list of files.
3 files to import.
importing repository.
case conflict: //depot/Main/A and //depot/Main/a
writing filelog: b789fdd96dc2, p1 000000000000, linkrev 2, 2 bytes, src: *, path: depot/Main/A (glob)
writing filelog: b789fdd96dc2, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/a (glob)
writing filelog: b789fdd96dc2, p1 000000000000, linkrev 1, 2 bytes, src: *, path: depot/Main/b (glob)
changelist 1: Writing manifest.
changelist 1: Writing changelog: initial
changelist 2: Writing manifest.
changelist 2: Writing changelog: moveway
changelist 3: Writing manifest.
changelist 3: Writing changelog: moveback
3 revision(s), 3 file(s) imported.
Verify
$ hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
3 files, 3 changesets, 3 total revisions
Update
$ hg update -r 0
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat depot/Main/a
a
$ hg update -r 1
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ cat depot/Main/b
a
$ hg update -r 2
1 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ cat depot/Main/A
a
stopping the p4 server

View File

@ -0,0 +1,107 @@
$ echo "[extensions]" >> $HGRCPATH
$ echo "p4fastimport= " >> $HGRCPATH
create p4 depot
$ p4wd=`pwd`/p4
$ hgwd=`pwd`/hg
$ P4ROOT=`pwd`/depot; export P4ROOT
$ P4AUDIT=$P4ROOT/audit; export P4AUDIT
$ P4JOURNAL=$P4ROOT/journal; export P4JOURNAL
$ P4LOG=$P4ROOT/log; export P4LOG
$ P4PORT=localhost:$HGPORT; export P4PORT
$ P4DEBUG=1; export P4DEBUG
$ mkdir $hgwd
$ mkdir $p4wd
$ cd $p4wd
start the p4 server
$ [ ! -d $P4ROOT ] && mkdir $P4ROOT
$ p4d -C1 -f -J off >$P4ROOT/stdout 2>$P4ROOT/stderr &
$ echo $! >> $DAEMON_PIDS
$ trap "echo stopping the p4 server ; p4 admin stop" EXIT
$ # wait for the server to initialize
$ while ! p4 ; do
> sleep 1
> done >/dev/null 2>/dev/null
create a client spec
$ cd $p4wd
$ P4CLIENT=hg-p4-import; export P4CLIENT
$ DEPOTPATH=//depot/...
$ p4 client -o | sed '/^View:/,$ d' >p4client
$ echo View: >>p4client
$ echo " $DEPOTPATH //$P4CLIENT/..." >>p4client
$ p4 client -i <p4client
Client hg-p4-import saved.
populate the depot
$ mkdir Main
$ echo a > Main/a
$ p4 add Main/a
//depot/Main/a#1 - opened for add
$ p4 submit -d initial
Submitting change 1.
Locking 1 files ...
add //depot/Main/a#1
Change 1 submitted.
$ p4 delete Main/a
//depot/Main/a#1 - opened for delete
$ p4 submit -ddelete
Submitting change 2.
Locking 1 files ...
delete //depot/Main/a#2
Change 2 submitted.
$ echo a > Main/A
$ p4 add Main/A
//depot/Main/A#2 - opened for add
$ p4 submit -d 'add with case-inensitivity match'
Submitting change 3.
Locking 1 files ...
add //depot/Main/A#3
Change 3 submitted.
import
$ cd $hgwd
$ hg init --config 'format.usefncache=False'
$ hg p4fastimport --debug -P $P4ROOT hg-p4-import
loading changelist numbers.
3 changelists to import.
loading list of files.
2 files to import.
importing repository.
case conflict: //depot/Main/A and //depot/Main/a
writing filelog: b789fdd96dc2, p1 000000000000, linkrev 2, 2 bytes, src: *, path: depot/Main/A (glob)
writing filelog: b789fdd96dc2, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/a (glob)
changelist 1: Writing manifest.
changelist 1: Writing changelog: initial
changelist 2: Writing manifest.
changelist 2: Writing changelog: delete
changelist 3: Writing manifest.
changelist 3: Writing changelog: add with case-inensitivity match
3 revision(s), 2 file(s) imported.
Verify
$ hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
2 files, 3 changesets, 2 total revisions
Update
$ hg update -r 0
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat depot/Main/a
a
$ hg update -r 1
0 files updated, 0 files merged, 1 files removed, 0 files unresolved
$ hg update -r 2
1 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ cat depot/Main/A
a
stopping the p4 server

View File

@ -0,0 +1,124 @@
$ echo "[extensions]" >> $HGRCPATH
$ echo "p4fastimport= " >> $HGRCPATH
create p4 depot
$ p4wd=`pwd`/p4
$ hgwd=`pwd`/hg
$ P4ROOT=`pwd`/depot; export P4ROOT
$ P4AUDIT=$P4ROOT/audit; export P4AUDIT
$ P4JOURNAL=$P4ROOT/journal; export P4JOURNAL
$ P4LOG=$P4ROOT/log; export P4LOG
$ P4PORT=localhost:$HGPORT; export P4PORT
$ P4DEBUG=1; export P4DEBUG
$ mkdir $hgwd
$ mkdir $p4wd
$ cd $p4wd
start the p4 server
$ [ ! -d $P4ROOT ] && mkdir $P4ROOT
$ p4d -f -J off >$P4ROOT/stdout 2>$P4ROOT/stderr &
$ echo $! >> $DAEMON_PIDS
$ trap "echo stopping the p4 server ; p4 admin stop" EXIT
$ # wait for the server to initialize
$ while ! p4 ; do
> sleep 1
> done >/dev/null 2>/dev/null
create a client spec
$ cd $p4wd
$ P4CLIENT=hg-p4-import; export P4CLIENT
$ DEPOTPATH=//depot/...
$ p4 client -o | sed '/^View:/,$ d' >p4client
$ echo View: >>p4client
$ echo " $DEPOTPATH //$P4CLIENT/..." >>p4client
$ p4 client -i <p4client
Client hg-p4-import saved.
populate the depot
$ mkdir Main
$ mkdir Main/b
$ echo '1' > Main/a
$ p4 add Main/a
//depot/Main/a#1 - opened for add
$ p4 submit -d'CL1(1)'
Submitting change 1.
Locking 1 files ...
add //depot/Main/a#1
Change 1 submitted.
$ p4 edit Main/a
//depot/Main/a#1 - opened for edit
$ echo '4' > Main/a
$ cat >desc <<EOF
> Change: new
> Client: $P4CLIENT
> User: $USER
> Status: new
> Description: CL4(2)
> Files:
> //depot/Main/a # edit
> EOF
$ p4 shelve -i < desc
Change 2 created with 1 open file(s).
Shelving files for change 2.
edit //depot/Main/a#1
Change 2 files shelved.
$ p4 reopen -c default Main/a
//depot/Main/a#1 - reopened; default change
$ echo '3' > Main/a
$ p4 submit -d'CL3(3)'
Submitting change 3.
Locking 1 files ...
edit //depot/Main/a#2
Change 3 submitted.
$ cat Main/a
3
$ p4 unshelve -s 2 -c 2
//depot/Main/a#1 - unshelved, opened for edit
$ p4 shelve -c 2 -d
Shelved change 2 deleted.
$ p4 sync
//depot/Main/a#2 - is opened and not being changed
* //depot/Main/a - must resolve #2 before submitting (glob)
$ p4 resolve -ay
$TESTTMP/p4/Main/a - vs //depot/Main/a#2
//hg-p4-import/Main/a - ignored //depot/Main/a
$ p4 submit -c 2
Submitting change 2.
Locking 1 files ...
edit //depot/Main/a#3
Change 2 renamed change 4 and submitted.
$ cat Main/a
4
Import
$ cd $hgwd
$ hg init --config 'format.usefncache=False'
$ hg p4fastimport --debug -P $P4ROOT hg-p4-import
loading changelist numbers.
3 changelists to import.
loading list of files.
1 files to import.
importing repository.
writing filelog: b8e02f643373, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/a (glob)
writing filelog: 059c099e8c05, p1 b8e02f643373, linkrev 1, 2 bytes, src: *, path: depot/Main/a (glob)
writing filelog: de9e19b2b7a1, p1 059c099e8c05, linkrev 2, 2 bytes, src: *, path: depot/Main/a (glob)
changelist 1: Writing manifest.
changelist 1: Writing changelog: CL1(1)
changelist 3: Writing manifest.
changelist 3: Writing changelog: CL3(3)
changelist 4: Writing manifest.
changelist 4: Writing changelog: CL4(2)
3 revision(s), 1 file(s) imported.
$ hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
1 files, 3 changesets, 3 total revisions
$ hg cat -r tip depot/Main/a
4
stopping the p4 server

View File

@ -0,0 +1,110 @@
$ echo "[extensions]" >> $HGRCPATH
$ echo "p4fastimport= " >> $HGRCPATH
create p4 depot
$ p4wd=`pwd`/p4
$ hgwd=`pwd`/hg
$ P4ROOT=`pwd`/depot; export P4ROOT
$ P4AUDIT=$P4ROOT/audit; export P4AUDIT
$ P4JOURNAL=$P4ROOT/journal; export P4JOURNAL
$ P4LOG=$P4ROOT/log; export P4LOG
$ P4PORT=localhost:$HGPORT; export P4PORT
$ P4DEBUG=1; export P4DEBUG
$ mkdir $hgwd
$ mkdir $p4wd
$ cd $p4wd
start the p4 server
$ [ ! -d $P4ROOT ] && mkdir $P4ROOT
$ p4d -f -J off >$P4ROOT/stdout 2>$P4ROOT/stderr &
$ echo $! >> $DAEMON_PIDS
$ trap "echo stopping the p4 server ; p4 admin stop" EXIT
$ # wait for the server to initialize
$ while ! p4 ; do
> sleep 1
> done >/dev/null 2>/dev/null
create a client spec
$ cd $p4wd
$ P4CLIENT=hg-p4-import; export P4CLIENT
$ DEPOTPATH=//depot/...
$ p4 client -o | sed '/^View:/,$ d' >p4client
$ echo View: >>p4client
$ echo " $DEPOTPATH //$P4CLIENT/..." >>p4client
$ p4 client -i <p4client
Client hg-p4-import saved.
populate the depot
$ mkdir Main
$ mkdir Main/b
$ echo a > Main/a
$ echo c > Main/b/c
$ echo d > Main/d
$ p4 add Main/a Main/b/c Main/d
//depot/Main/a#1 - opened for add
//depot/Main/b/c#1 - opened for add
//depot/Main/d#1 - opened for add
$ p4 submit -d initial
Submitting change 1.
Locking 3 files ...
add //depot/Main/a#1
add //depot/Main/b/c#1
add //depot/Main/d#1
Change 1 submitted.
$ p4 delete Main/a
//depot/Main/a#1 - opened for delete
$ p4 submit -d second
Submitting change 2.
Locking 1 files ...
delete //depot/Main/a#2
Change 2 submitted.
$ echo a > Main/a
$ p4 add Main/a
//depot/Main/a#2 - opened for add
$ p4 submit -d third
Submitting change 3.
Locking 1 files ...
add //depot/Main/a#3
Change 3 submitted.
Simple import
$ cd $hgwd
$ hg init --config 'format.usefncache=False'
$ hg p4fastimport --debug -P $P4ROOT hg-p4-import
loading changelist numbers.
3 changelists to import.
loading list of files.
3 files to import.
importing repository.
writing filelog: b789fdd96dc2, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/a (glob)
writing filelog: f9597ff22e3f, p1 b789fdd96dc2, linkrev 2, 2 bytes, src: *, path: depot/Main/a (glob)
writing filelog: 149da44f2a4e, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/b/c (glob)
writing filelog: a9092a3d84a3, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/d (glob)
changelist 1: Writing manifest.
changelist 1: Writing changelog: initial
changelist 2: Writing manifest.
changelist 2: Writing changelog: second
changelist 3: Writing manifest.
changelist 3: Writing changelog: third
3 revision(s), 3 file(s) imported.
Verify
$ hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
3 files, 3 changesets, 4 total revisions
$ hg update tip
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
End Test
stopping the p4 server

View File

@ -0,0 +1,112 @@
$ echo "[extensions]" >> $HGRCPATH
$ echo "p4fastimport= " >> $HGRCPATH
create p4 depot
$ p4wd=`pwd`/p4
$ hgwd=`pwd`/hg
$ P4ROOT=`pwd`/depot; export P4ROOT
$ P4AUDIT=$P4ROOT/audit; export P4AUDIT
$ P4JOURNAL=$P4ROOT/journal; export P4JOURNAL
$ P4LOG=$P4ROOT/log; export P4LOG
$ P4PORT=localhost:$HGPORT; export P4PORT
$ P4DEBUG=1; export P4DEBUG
$ mkdir $hgwd
$ mkdir $p4wd
$ cd $p4wd
start the p4 server
$ [ ! -d $P4ROOT ] && mkdir $P4ROOT
$ p4d -f -J off >$P4ROOT/stdout 2>$P4ROOT/stderr &
$ echo $! >> $DAEMON_PIDS
$ trap "echo stopping the p4 server ; p4 admin stop" EXIT
$ # wait for the server to initialize
$ while ! p4 ; do
> sleep 1
> done >/dev/null 2>/dev/null
create a client spec
$ cd $p4wd
$ P4CLIENT=hg-p4-import; export P4CLIENT
$ DEPOTPATH=//depot/...
$ p4 client -o | sed '/^View:/,$ d' >p4client
$ echo View: >>p4client
$ echo " $DEPOTPATH //$P4CLIENT/..." >>p4client
$ p4 client -i <p4client
Client hg-p4-import saved.
populate the depot
$ mkdir Main
$ mkdir Main/b
$ echo a > Main/a
$ ln -s Main/a Main/b/c
$ echo d > Main/d
$ chmod +x Main/d
$ p4 add Main/a Main/b/c Main/d
//depot/Main/a#1 - opened for add
//depot/Main/b/c#1 - opened for add
//depot/Main/d#1 - opened for add
$ p4 submit -d initial
Submitting change 1.
Locking 3 files ...
add //depot/Main/a#1
add //depot/Main/b/c#1
add //depot/Main/d#1
Change 1 submitted.
$ p4 edit Main/a Main/b/c Main/d
//depot/Main/a#1 - opened for edit
//depot/Main/b/c#1 - opened for edit
//depot/Main/d#1 - opened for edit
$ echo a >> Main/a
$ echo d >> Main/d
$ p4 submit -d second
Submitting change 2.
Locking 3 files ...
edit //depot/Main/a#2
edit //depot/Main/b/c#2
edit //depot/Main/d#2
Change 2 submitted.
Simple import
$ cd $hgwd
$ hg init --config 'format.usefncache=False'
$ hg p4fastimport --debug -P $P4ROOT hg-p4-import
loading changelist numbers.
2 changelists to import.
loading list of files.
3 files to import.
importing repository.
writing filelog: b789fdd96dc2, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/a (glob)
writing filelog: a80d06849b33, p1 b789fdd96dc2, linkrev 1, 4 bytes, src: *, path: depot/Main/a (glob)
writing filelog: 8aa36f7e9a8d, p1 000000000000, linkrev 0, 7 bytes, src: *, path: depot/Main/b/c (glob)
writing filelog: ee47780ebabc, p1 8aa36f7e9a8d, linkrev 1, 7 bytes, src: *, path: depot/Main/b/c (glob)
writing filelog: a9092a3d84a3, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/d (glob)
writing filelog: f83f0637e55e, p1 a9092a3d84a3, linkrev 1, 4 bytes, src: *, path: depot/Main/d (glob)
changelist 1: Writing manifest.
changelist 1: Writing changelog: initial
changelist 2: Writing manifest.
changelist 2: Writing changelog: second
2 revision(s), 3 file(s) imported.
Verify
$ hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
3 files, 2 changesets, 6 total revisions
$ hg update tip
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg --debug manifest
a80d06849b333b8a3d5c445f8ba3142010dcdc9e 644 depot/Main/a
ee47780ebabc4dd227d21ef3b71ca3ab381eb4cf 644 @ depot/Main/b/c
f83f0637e55e3c48e9922f14a016761626d79d3d 755 * depot/Main/d
End Test
stopping the p4 server

View File

@ -0,0 +1,110 @@
$ echo "[extensions]" >> $HGRCPATH
$ echo "p4fastimport=" >> $HGRCPATH
$ echo "[p4fastimport]" >> $HGRCPATH
$ echo "useworker=force" >> $HGRCPATH
create p4 depot
$ p4wd=`pwd`/p4
$ hgwd=`pwd`/hg
$ P4ROOT=`pwd`/depot; export P4ROOT
$ P4AUDIT=$P4ROOT/audit; export P4AUDIT
$ P4JOURNAL=$P4ROOT/journal; export P4JOURNAL
$ P4LOG=$P4ROOT/log; export P4LOG
$ P4PORT=localhost:$HGPORT; export P4PORT
$ P4DEBUG=1; export P4DEBUG
$ mkdir $hgwd
$ mkdir $p4wd
$ cd $p4wd
start the p4 server
$ [ ! -d $P4ROOT ] && mkdir $P4ROOT
$ p4d -f -J off >$P4ROOT/stdout 2>$P4ROOT/stderr &
$ echo $! >> $DAEMON_PIDS
$ trap "echo stopping the p4 server ; p4 admin stop" EXIT
$ # wait for the server to initialize
$ while ! p4 ; do
> sleep 1
> done >/dev/null 2>/dev/null
create a client spec
$ cd $p4wd
$ P4CLIENT=hg-p4-import; export P4CLIENT
$ DEPOTPATH=//depot/...
$ p4 client -o | sed '/^View:/,$ d' >p4client
$ echo View: >>p4client
$ echo " $DEPOTPATH //$P4CLIENT/..." >>p4client
$ p4 client -i <p4client
Client hg-p4-import saved.
populate the depot
$ mkdir Main
$ mkdir Main/b
$ echo a > Main/a
$ echo c > Main/b/c
$ echo d > Main/d
$ p4 add Main/a Main/b/c Main/d
//depot/Main/a#1 - opened for add
//depot/Main/b/c#1 - opened for add
//depot/Main/d#1 - opened for add
$ p4 submit -d initial
Submitting change 1.
Locking 3 files ...
add //depot/Main/a#1
add //depot/Main/b/c#1
add //depot/Main/d#1
Change 1 submitted.
$ p4 edit Main/a Main/b/c Main/d
//depot/Main/a#1 - opened for edit
//depot/Main/b/c#1 - opened for edit
//depot/Main/d#1 - opened for edit
$ echo a >> Main/a
$ echo c >> Main/b/c
$ echo d >> Main/d
$ p4 submit -d second
Submitting change 2.
Locking 3 files ...
edit //depot/Main/a#2
edit //depot/Main/b/c#2
edit //depot/Main/d#2
Change 2 submitted.
Simple import
$ cd $hgwd
$ hg init --config 'format.usefncache=False'
$ FORCE_J=1 hg p4fastimport --debug -P $P4ROOT hg-p4-import
loading changelist numbers.
2 changelists to import.
loading list of files.
3 files to import.
importing repository.
writing filelog: * (glob)
writing filelog: * (glob)
writing filelog: * (glob)
writing filelog: * (glob)
writing filelog: * (glob)
writing filelog: * (glob)
changelist 1: Writing manifest.
changelist 1: Writing changelog: initial
changelist 2: Writing manifest.
changelist 2: Writing changelog: second
2 revision(s), 3 file(s) imported.
Verify
$ hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
3 files, 2 changesets, 6 total revisions
$ hg update tip
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
End Test
stopping the p4 server

View File

@ -0,0 +1,108 @@
$ echo "[extensions]" >> $HGRCPATH
$ echo "p4fastimport= " >> $HGRCPATH
create p4 depot
$ p4wd=`pwd`/p4
$ hgwd=`pwd`/hg
$ P4ROOT=`pwd`/depot; export P4ROOT
$ P4AUDIT=$P4ROOT/audit; export P4AUDIT
$ P4JOURNAL=$P4ROOT/journal; export P4JOURNAL
$ P4LOG=$P4ROOT/log; export P4LOG
$ P4PORT=localhost:$HGPORT; export P4PORT
$ P4DEBUG=1; export P4DEBUG
$ mkdir $hgwd
$ mkdir $p4wd
$ cd $p4wd
start the p4 server
$ [ ! -d $P4ROOT ] && mkdir $P4ROOT
$ p4d -f -J off >$P4ROOT/stdout 2>$P4ROOT/stderr &
$ echo $! >> $DAEMON_PIDS
$ trap "echo stopping the p4 server ; p4 admin stop" EXIT
$ # wait for the server to initialize
$ while ! p4 ; do
> sleep 1
> done >/dev/null 2>/dev/null
create a client spec
$ cd $p4wd
$ P4CLIENT=hg-p4-import; export P4CLIENT
$ DEPOTPATH=//depot/...
$ p4 client -o | sed '/^View:/,$ d' >p4client
$ echo View: >>p4client
$ echo " $DEPOTPATH //$P4CLIENT/..." >>p4client
$ p4 client -i <p4client
Client hg-p4-import saved.
populate the depot
$ mkdir Main
$ mkdir Main/b
$ echo a > Main/a
$ echo c > Main/b/c
$ echo d > Main/d
$ p4 add Main/a Main/b/c Main/d
//depot/Main/a#1 - opened for add
//depot/Main/b/c#1 - opened for add
//depot/Main/d#1 - opened for add
$ p4 submit -d initial
Submitting change 1.
Locking 3 files ...
add //depot/Main/a#1
add //depot/Main/b/c#1
add //depot/Main/d#1
Change 1 submitted.
$ p4 edit Main/a Main/b/c Main/d
//depot/Main/a#1 - opened for edit
//depot/Main/b/c#1 - opened for edit
//depot/Main/d#1 - opened for edit
$ echo a >> Main/a
$ echo c >> Main/b/c
$ echo d >> Main/d
$ p4 submit -d second
Submitting change 2.
Locking 3 files ...
edit //depot/Main/a#2
edit //depot/Main/b/c#2
edit //depot/Main/d#2
Change 2 submitted.
Simple import
$ cd $hgwd
$ hg init --config 'format.usefncache=False'
$ hg p4fastimport --debug -P $P4ROOT hg-p4-import
loading changelist numbers.
2 changelists to import.
loading list of files.
3 files to import.
importing repository.
writing filelog: b789fdd96dc2, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/a (glob)
writing filelog: a80d06849b33, p1 b789fdd96dc2, linkrev 1, 4 bytes, src: *, path: depot/Main/a (glob)
writing filelog: 149da44f2a4e, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/b/c (glob)
writing filelog: b11e10a88bfa, p1 149da44f2a4e, linkrev 1, 4 bytes, src: *, path: depot/Main/b/c (glob)
writing filelog: a9092a3d84a3, p1 000000000000, linkrev 0, 2 bytes, src: *, path: depot/Main/d (glob)
writing filelog: f83f0637e55e, p1 a9092a3d84a3, linkrev 1, 4 bytes, src: *, path: depot/Main/d (glob)
changelist 1: Writing manifest.
changelist 1: Writing changelog: initial
changelist 2: Writing manifest.
changelist 2: Writing changelog: second
2 revision(s), 3 file(s) imported.
Verify
$ hg verify
checking changesets
checking manifests
crosschecking files in changesets and manifests
checking files
3 files, 2 changesets, 6 total revisions
$ hg update tip
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
End Test
stopping the p4 server