Mercurial 4.3 has completelu dropped the join and wjoin functions. Let's use the
appropriate repo.vfs.join and repo.wvfs.join functions instead.
I ran the tests against each version of Mercurial from 2.8 to 4.2. Things
before 2.8 seem to already be broken for unrelated reasons.
Upstream has deprecated repo.join in favor of repo.vfs.join, so let's update to
match them. Old hg should have repo.vfs.join so I don't think this breaks
backwards compatibility.
Mercurial users are using bookmarks to represent git refs, so when
we are unable to push a git ref to remote, we need to tell
the hg user to add a bookmark (something they understand),
instead of mentioning a ref (which is a foreign concept to them).
When importing changesets, hggit uses the config knob hggit.mapsavefrequency to
determine how often to save the mapfile. This allows a user to interrupt the
import without losing all progress.
This patch adds this same functionality to the export mechanism.
A commit's extras field should be considered user-supplied input that can take
any form. Trusting it to be properly formatted is dangerous and can prevent
forward progress. Instead, swallow errors due to malformed extras and carry on.
The ReviewBoard repository contains a Mercurial repository within its
Git repository; if you just convert that into Mercurial, you can't
check it out. We handle this similar to invalid Git paths: by default,
refuse the conversion, but with a configuration knob to force it
through with a warning.
See also:
https://github.com/reviewboard/reviewboard/https://reviewboard.org/bugs/3190
Dulwich treats ref names internally as unicode strings (probably
because of Python 3?), which means that at some points it tries to do
os.path.join between the repo path and the unicode of the ref name,
which fails miserably if we construct the repo with a str and not a
unicode. Kludge around this problem.
Fixes issue 172.
This is a roll-forward of 3df4d529a2f2, which should be valid now that
the previous change defends against accidentally writing unicode tags
inside the templater.
bookmarks.write is deprecated and it was showing warning messages in
test-hg-branch.t with the latest test runner from core mercurial. Tested with
both hg 2.8 and hg tip.
Dulwich treats ref names internally as unicode strings (probably
because of Python 3?), which means that at some points it tries to do
os.path.join between the repo path and the unicode of the ref name,
which fails miserably if we construct the repo with a str and not a
unicode. Kludge around this problem.
Fixes issue 172.
We avoid using dulwich's refs method because it is incredibly slow. On a repo
with a few hundred branches and a few thousand tags, dulwich took about 200ms
to load everything.
This patch only traveses the remote ref directory and cuts that time down to
about 50ms.
It is unclear to me why we keep a file (which can become out of sync) of remote
refs instead of just using dulwich. This caught a missing remote ref in the
test suite.
By testing the uri early, we can reuse logic later in the method to parse the
git uri. We rely on the isgitsshuri heuristic to return True or False, and if
True, prepend 'git+ssh://' to the uri.
Arguably, this is fragile, and am open to better ideas, but can't think of
anything else currently.
Previously, there was an edge case for Git repositories that started as
Mercurial repositories and had used subrepos where a deleted .hgsubstate would
be ignored and therefore reintroduced.
This patch fixes that behavior by checking for the deleted .hgsubstate file
first.
A test has been added to verify behavior.
If the importer encountered an error half way through a large import, all the
commits are saved, but the mapfile is not written, so the process starts over
from the beginning when run again.
This adds the option for a config value that will save the map file every X
commits. I thought about just hard coding this to 100 or something, but doing it
this way seems a little less invasive.
The default dulwich graph walker only walks from refs/heads. During the
discovery phase of fetching this causes it to redownload commits that are only
referenced by refs/remotes. In a normal hggit case, this seems to mean it
redownloads the entire git repo on every hg pull.
Added a --debug to a test to check the object count (it decreased from 21 to 10
as part of this patch).
filectx.renamed() returns a 2-tuple or None. memfilectx.__init__ expects
the copied argument to be either None or a string. Before, we were
passing a 2-tuple, leading to the memfilectx storing the wrong type.
This eventually resulted in doing a key lookup against a manifest
with a 2-tuple, which made manifest.c throw an error.
There really is no point to this -- the sorting is expensive to compute and
the structure is never actually used.
For a mapfile with 1.5 million entries, this speeds up save_map from 3.6
seconds to 0.87.
This is probably the limit of the speedups we can get with pure-Python code.
Any further speedups will have to be made by rewriting these bits in C.
Sorting a list of tuples is much more expensive than sorting a list of strings.
For a mapfile with 1.5 million entries, this speeds up save_map from 6 seconds
to 3.5.
While this has been done since the beginning of time, there's no apparent
justification for it. If an imported commit works out to the same hash as an
existing one, it simply won't be added to the revlog.
The tests all continue to pass. There's already test coverage for reimporting
commits in test-pull-after-strip.t. Also, gimport has worked this way all this
while.
Consider a Mercurial commit with hash 'h1'. Originally, if the only Mercurial
field stored is the branch info (which is stored in the commit message rather
than as an extra field), we'd store the rename source explicitly as a Git extra
field -- let's call the original exported hash 'g1'.
In Git, some operations throw all extra fields away. (One such example is a
rebase.) If such an operation happens, we'll be left with a frankencommit with
the branch info but without the rename source. Let's call this hash 'g2'. For a
setup where Git is the source of truth, let's say that this 'g2' frankencommit
is what gets pushed to the server.
When 'g2' is subsequently imported into Mercurial, we'd look at the fact that
it contains a Mercurial field in the commit message and believe that it was a
legacy commit from the olden days when all info was stored in the commit
message. In that case, in an attempt to preserve the hash, we wouldn't store
any extra rename source info, resulting in 'h1'. Then, when the commit is
re-exported to Git, we'd add the rename source again and produce 'g1' -- and
thus break bidirectionality.
Prevent this situation by not storing the rename source if we're adding branch
info to the commit message. Then for 'h1' we export as 'g2' directly and never
produce 'g1'.
What happens if we not only need to store branch info but also other extra
info, like renames? For 'h1' we'd produce 'g1', then it'd be rewritten on the
Git side to 'g2' throwing away all that extra information. 'g2' being
subsequently imported into Mercurial would produce a new hash, say 'h2'. That's
fine because the commit did get rewritten in Git. We unfortunately wouldn't
perform rename detection thinking that the commit is from Mercurial and had no
renames recorded there, but when the commit is re-exported to Git we'd export
it to 'g2' again. This at least preserves bidirectionality.