Now that the 'vfs' classes moved in their own module, lets use the new module
directly. We update code iteratively to help with possible bisect needs in the
future.
This is the only place where strutil is used. I don't think it's worth to
keep the strutil module, so inline it.
Also, strutil.rfindall() appears to have off-by-one error. 'end = c - 1' is
wrong because 'end' is exclusive.
The cPickle is renamed to _pickle in python3 and this C extension is available
in pickle which was not included in earlier versions. So imports are conditionalized
to import cPickle in py2 and pickle in py3. Moreover the use of pickle in py2 is
switched to cPickle as the C extension is faster. The hack is added in util.py and
the modules import util.pickle
Since (b) is banned, we should do the same for (a) for consistency.
a) from mercurial import hg
from mercurial.i18n import _
b) from . import hg
from .i18n import _
Before this patch, import-checker reports "relative import of stdlib
module" error for importing Pool and SubversionException from svn.core
in subversion.py.
To fix this relative import of stdlib module, this patch adds prefix
'svn.core.' to Pool and SubversionException in source.
These 'svn.core.' relative accessing shouldn't cause performance
impact, because there are much more code paths accessing to
'svn.core.' relative properties.
BTW, in transport.py, this error is avoided by assignment below.
SubversionException = svn.core.SubversionException
But this can't be used in subversion.py case, because:
- such assignment in indented code block causes "don't use camelcase
in identifiers" error of check-code.py
- but it should be placed in indented block, because svn is None at
failure of importing subversion python binding libraries (=
examination of 'svn' is needed)
The home of 'Abort' is 'error' not 'util' however, a lot of code seems to be
confused about that and gives all the credit to 'util' instead of the
hardworking 'error'. In a spirit of equity, we break the cycle of injustice and
give back to 'error' the respect it deserves. And screw that 'util' poser.
For great justice.
Previously convert could only take one '--rev'. This change allows the user to
specify multiple --rev entries. For instance, this could allow converting
multiple branches (but not all branches) at once from git.
In this first patch, we disable support for this for all sources. Future
patches will enable it for select sources (like git).
Python 2.6 introduced the "except type as instance" syntax, replacing
the "except type, instance" syntax that came before. Python 3 dropped
support for the latter syntax. Since we no longer support Python 2.4 or
2.5, we have no need to continue supporting the "except type, instance".
This patch mass rewrites the exception syntax to be Python 2.6+ and
Python 3 compatible.
This patch was produced by running `2to3 -f except -w -n .`.
We've had a couple reports of subversion tracebacks that trigger when
the parents list is empty, and thus block showing what the commit
failure was on the next two lines.
Conversion of a merge starts with p1 and re-adds the files that were changed in
the merge or came unmodified from p2. Files that are unmodified from p1 will
thus not be touched and take no time. Files that are unmodified from p2 would be
retrieved and rehashed. They would end up getting the same hash as in p2 and end
up reusing the filelog entry and look like the p1 case ... but it was slow.
Instead, make getchanges also return 'files that are unmodified from p2' so the
sink can reuse the existing p2 entry instead of calling getfile.
Reuse of filelog entries can make a big difference when files are big and with
long revlong chains so they take time to retrieve and hash, or when using an
expensive custom getfile function (think
http://mercurial.selenic.com/wiki/ConvertExtension#Customization with a code
reformatter).
This in combination with changes to reuse filectx entries in
localrepo._filecommit make 'unchanged from p2' almost as fast as 'unchanged
from p1'.
This is so far only implemented for the combination of hg source and hg sink.
This is a refactoring/optimization. It is covered by existing tests and show no
changes - which is a good thing.
Although Python supports `X = Y if COND else Z`, this was only
introduced in Python 2.5. Since we have to support Python 2.4, it was
a very common thing to write instead `X = COND and Y or Z`, which is a
bit obscure at a glance. It requires some intricate knowledge of
Python to understand how to parse these one-liners.
We change instead all of these one-liners to 4-liners. This was
executed with the following perlism:
find -name "*.py" -exec perl -pi -e 's,(\s*)([\.\w]+) = \(?(\S+)\s+and\s+(\S*)\)?\s+or\s+(\S*)$,$1if $3:\n$1 $2 = $4\n$1else:\n$1 $2 = $5,' {} \;
I tweaked the following cases from the automatic Perl output:
prev = (parents and parents[0]) or nullid
port = (use_ssl and 443 or 80)
cwd = (pats and repo.getcwd()) or ''
rename = fctx and webutil.renamelink(fctx) or []
ctx = fctx and fctx or ctx
self.base = (mapfile and os.path.dirname(mapfile)) or ''
I also added some newlines wherever they seemd appropriate for readability
There are probably a few ersatz ternary operators still in the code
somewhere, lurking away from the power of a simple regex.
We have our own fast-path logic to see if something passes a sniff
test for being a Subversion repository, but it's possible for a user
to svnsync a repo using svn 1.8 and then use svn 1.7 bindings (as in
the bug) to try and convert the repo. If we at least tell the user the
version of libsvn that we used, they might get enough of a hint to
check on their own for format incompatibilities between their
svn{admin,sync} and the libsvn used by hg.
This produces slightly bad results when branches are in play, but
overall I think it's probably worthwhile. We might be able to do
better with branches somehow, but I haven't given it any thought.
Convert will normally only process files that were changed in a source
revision, apply the filemap, and record it has a change in the target
repository. (If it ends up not really changing anything, nothing changes.)
That means that _if_ the filemap is changed before continuing an incremental
convert, the change will only kick in when the files it affects are modified in
a source revision and thus processed.
With --full, convert will make a full conversion every time and process
all files in the source repo and remove target repo files that shouldn't be
there. Filemap changes will thus kick in on the first converted revision, no
matter what is changed.
This flag should in most cases not make any difference but will make convert
significantly slower.
Other names has been considered for this feature, such as "resync", "sync",
"checkunmodified", "all" or "allfiles", but I found that they were less obvious
and required more explanation than "full" and were harder to describe
consistently.
Since it was introduced in 670e8681d92a, tidy_dirs has been comparing
the result of os.listdir with a string - which never can be true.
Convert apparently works anyway and there is no test coverage of it.
It also seems like it could make a bigger difference on older svn versions but
is less relevant with more recent versions.
Instead of trying to fix the code, we take the low risk option and remove it.
The internal API used IOError to indicate that a file should be marked as
removed.
There is some correlation between IOError (especially with ENOENT) and files
that should be removed, but using IOErrors to represent file removal internally
required some hacks.
Instead, use the value None to indicate that the file not is present.
Before, spurious IO errors could cause commits that silently removed files.
They will now be reported like all other IO errors so the root cause can be
fixed.
In all the remaining cases the comprehension variable is used for the same
thing as a previous loop variable.
This will mute some pyflakes "list comprehension redefines" warnings.
Mercurial has stable revision identifiers and rollback and strip. Revisions
referenced in the shamap are thus not necessarily still present but we can
easily check for it.
Subversion do not have stable identifiers and no rollback or strip(?). We must
thus assume that all revisions referenced from a shamap still must be present.
This method is similar to hascommitforsplicemap but different ...
The name 'hascommit' sounds like something generic ... but it might
also throw exceptions in specific cases and it is thus (apparently)
only useful for splicemap.
Tagmap solves a very specific use case. It would be better to have a more
generic solution than to have to maintain this forever.
Tagmap has not been released yet and removing it now will not break any
backward compatibility contract.
There is no test coverage for tagmap but it seems like the same can be achieved
with a (relatively) simple and much more powerful custom extension:
import hgext.convert.hg
def f(tag):
return tag.replace('some', 'other')
class source(hgext.convert.hg.mercurial_source):
def gettags(self):
return dict((f(tag), node)
for tag, node in in super(source, self).gettags().items())
def getfile(self, name, rev):
data, flags = super(source, self).getfile(name, rev)
if name == '.hgtags':
data = ''.join(l[:41] + f(l[41:]) + '\n' for l in data.splitlines())
return data, flags
hgext.convert.hg.mercurial_source = source
Subversion issues involving svn log such as 1e493b49245f can be tricky to
debug when it is run in an 'hg debugsvnlog' sub process. Debugging is simpler
when convert only uses one process.
With this change convert will invoke the svn log directly when setting
[convert]
svn.debugsvnlog = False
This is intentionally not documented.
Previously, there was no way to rewrite tags on the fly while converting. Now,
we add similar logic to branchmap to provide a way to map old tags to new tags.
Currently, this is not enabled since there is not yet a command-line option.
Upcoming patches will add new map files so we change the calling sequence of
checkrevformat so that error messages will let the user know which file has the
wrong rev format.
get_log started calling back with orig_paths=None on Fedora 20 with
subversion-1.8.3. That broke test-convert-svn-source.t .
There used to be some handling of that situation until d17c619e40d5 apparently
broke it. This patch restores what seems to be the most obvious handling of the
situation.
Implemented error handling on splicemap file when source is
subversion (This checks are similar to when source is hg or git).
The revision string is expected to be of svn:<uuid><path>@<number>
format.
the test case has been enhanced to check this format.
The default for the time zone offset in a converted changeset has
always been 0 (UTC). With this patch, the converted changeset is
modified so that the local offset from UTC is specified as the time
zone offset.
The option is specified as the boolean convert.localtimezone (default
False). Example usage:
hg convert -s cvs --config convert.localtimezone=True example-cvs example-hg
IMPORTANT: the patch only applies to conversions from cvs or svn.
The documentation for the option only appears in those two sections
in the convert help text.
'hg debugsvnlog' failed with a crash when using the uninitialized transport in
get_log_child if the import of the svn libraries had failed.
'convert' should never get as far as launching 'hg debugsvnlog' if the svn
libraries are missing, but by launching a subprocess there is risk that the
environment is mangled so the second import fails.
It is in principle also possible to launch the command manually.
Subversion can handle ':' quoted as '%3A' but urllib.url2pathname can't and
Mercurial thus rejected some valid subversions URLs.
This particular case will now be handled by some preprocessing before handing
it over to urllib.url2pathname.
This is tested by a0c992a723f9 when test-convert-svn-source.t and
test-convert-svn-move.t can be run on Windows.
Calling propset/propdel with subversion 1.6 on FAT gave
abort: svn exited with status 256
and made test-convert-hg-svn.t and test-convert-svn-sink.t fail. 1.7 worked.
This is a rework of 5ba59c098f03 but ignores the executable bit when it isn't
supported instead of using an approximation.
