Mercurial commit a6d634896102 added support for hg cat across subrepositories.
That caused these tests to break. Fix this by using hg manifest instead.
Previously, whenever a tree that wasn't the root ('') was stored, we'd prepend
a '/' to it. Then, when we'd try retrieving the entry, we'd do so without the
leading '/'. This caused data loss because existing tree entries were dropped
on the floor. Fix that by only adding '/' if we're adding to a non-empty
initial path.
This wasn't detected in tests because most of them deal only with files in the
root and not ones in subdirectories.
The usage of getattr was unsafe. Use hgutil.safehasattr instead.
util.safehasattr has been around since Mercurial 2.0.
This also fixes the formerly disabled test in test-pull.t.
Previously we'd attempt to import every single reachable commit in the Git
object store.
The test adds another branch to the Git repo and doesn't import it until much
later. Previously we'd import it when we ran `hg -R hgrepo pull -r beta`. Now
we won't.
The return value as implemented in git_handler.fetch was pretty bogus. It used
to return the number of values that changed in the 'refs/heads/' namespace,
regardless of whether multiple values in there point to the same Mercurial
commit, or whether particular heads were even imported. Fix all of that by
using the actual heads in the changelog, just like vanilla Mercurial.
The test output changes demonstrate examples where the code was buggy.
Since Mercurial is commit-oriented, the 'no changes found' message really
should rely on what new commits are in the repo, not on new heads. This also
makes an upcoming patch much simpler.
Since everything around this code is completely broken anyway, writing a test
for this that doesn't trigger other bugs is close to impossible. An upcoming
patch will include tests.
The test output change is for an empty clone -- the output is precisely how
vanilla Mercurial treats an empty clone.
Since the Git to Mercurial conversion process is incremental, it's at risk of
missing files, or recording files the wrong way, or recording the wrong commit
metadata. Add a command called 'gverify' that can verify the contents of a
particular Mercurial rev against the corresponding Git commit.
Currently, this is limited to checking file names, flags and contents, but this
can be made as robust as desired. Further additions will probably require
refactoring git_handler.py a bit though.
This function is pretty fast: on a Linux machine with a warm cache, verifying a
repository with around 50,000 files takes just 20 seconds. There is scope for
further improvement through parallelization, but conducting tree walks in
parallel is non-trivial with the current worker infrastructure in Mercurial.
Previously, we'd try to access commit.parents[0] and fail. Now, check for
commit.parents being empty and return what Mercurial thinks is a repository
root in that case.
This is the version in Mercurial rev 8f2c4360fa44, plus a patch to make
--with-hg work for system hg (sent upstream). Importantly, this gets us the
hash seed randomization we need for bugs like the one fixed by the parent
commit to be detected.
Before this patch, in the git to hg conversion, .hgsubstate once created is
never deleted, even if no submodules are any longer present. This is broken
state, as shown by the test for which the SHA changes. Fix that by looking at
the diff instead of just what submodules are present.
Since 'gitlinks' now contains *changed* gitlinks, not *all* gitlinks, it no
longer makes sense to gate gitmodules checks on that.
This patch simply demonstrates that the test was broken; an upcoming patch will
introduce more tests.
Bonus: this also makes the import process faster because we no longer need to
walk the entire tree to collect gitlinks.
This will cause the SHAs of repos that have submodules added and then removed
to change.
Consider two octopus merges, one of which is a child of the other. Without this
patch, get_git_parents() called on the second octopus merge checks that each p1
is neither in the middle of an octopus merge nor the end of it. Since the end
of the first octopus merge is a p1 of the second one, this asserts.
Change the sanity check to only make sure that p1 is not in the middle of an
octopus merge.
A recent real world occurrence - user hand edited the timezone field in
an hg export to provide a unique value (from prior export). Hg imported
the export okay, but dulwich threw an exception.
This test shows the fault.
As pointed out by l33t, Hg-Git's output for push doesn't currently do a very
good job of telling the user what happened. My previous changes in this area
had moved some of the output from status to note, making it only show if
--verbose was specified. However, I hadn't realized at the time that the
reference information (though overly verbose) was providing a valueable purpose
that otherwise wasn't met; telling the user that a remote reference had changed.
This changeset makes it so that:
* default output will include simple messages like "adding reference
refs/heads/feature" and "updating reference refs/heads/master" (omitting any
mention of unchanged references)
* verbose output will include more detailed messages like "adding reference
default::refs/heads/feature => GIT:aba43c" and "updating reference
default::refs/heads/master => GIT:aba43c" (omitting any mention of unchanged
references)
* debug output will include the detailed output like in verbose, but
addtionally will include messages like "unchanged reference
default::refs/heads/other => GIT:aba43c"
https://bitbucket.org/durin42/hg-git/issue/64/push-confirmation
l33t pointed out that currently, Hg-Git doesn't provide any confirmation that a
push was successful other than the exit code. Normal Mercurial provides a
couple other messages followed by "added X changesets with Y changes to
Z files". After this change, Hg-Git will provide much more similar output.
It's not identical, as the underlying model is substantially different, but the
concept is the same. The main message is "added X commits with Y trees and
Z blobs".
This change doesn't affect the output of what references/branches were touched.
That will be addressed in a subsequent commit.
Dulwich doesn't provide an easy hook to get the information needed for this
output. Instead of passing generate_pack_contents as the pack generator
function to send_pack, I pass a custom function that determines the "missing"
objects, stores the counts, and then calls generate_pack_contents (which then
will determine the "missing" objects again.
The new expected output:
searching for changes # unless quiet true
<N> commits found # if verbose true
list of commits: # if debugflag true and at least one commit found
<each hash> # if debugflag true and at least one commit found
adding objects # if at least one commit found unless quiet true
added <N> commits with <N> trees and <N> blobs # if at least one object unless
# quiet true
https://bitbucket.org/durin42/hg-git/issue/64/push-confirmation
In f32e473ff520, the "commit" function was extracted into a testutil for re-use.
However, test-encoding.t was skipped over in that changeset, as I was seeing
unexplained test failures. Since those test failures have now been explained
(and fixed), this changeset performs the same extraction on test-encoding.t as
was done on all the other tests.
The version of fn_git_commit that was used in testutil redirected all output
(including errors) to /dev/null, which didn't match the expectations of this
test. The test utility functions for commit/tag now no longer throw away error
output, instead leaving it to individual tests to decide if error output should
be ignored.
It looks like Git 1.8.0 started silently converting latin1 commit messages to
utf-8. That changed the result of this test. This changeset alters the test
to make it accept both the pre-1.8.0 and post-1.8.0 behaviors.
https://raw.github.com/git/git/master/Documentation/RelNotes/1.8.0.txt
This test had some form of legacy hash filtering, marked with a TODO to remove
it when we're only supporting Mercurial 1.5 or later. Well, that time has
come, so I removed it.
This test was only running on Mercurial 1.7 or later. Since now we only
support versions that are 1.7 or later, there isn't a need to perform this
check any more.
Now that we're in the unified test format, there isn't a need to use echo
to provide context to command output. This technique actually ends up resulting
in redundant output. To preserve the original context, but eliminate the
redundancy, such echo statements have been converted into comment lines.
Mercurial allows specifying which repository to use via the -R/--repository
option. Git allows a similar function using the --git-dir option. By using
these options, in many cases we can avoid checking the current directory.
This makes tests easier to understand, as you don't need to remember which
directory you're in to understand what's going on. It also makes tests easier
to write, as you don't need to remember to cd out of a directory when you're
done doing things there.
Thanks to Felipe Contreras for the patch which this was based on.
One or both of these requirements were in almost every test in exactly the same
way. Now, these checks are performed in every test that uses the testutil.
This makes it easier for test authors to add these checks into new tests (just
add a reference to the testutil, which you'd probably want anyway).
We considered having each test declare their requirements (currently, either
"git" or "dulwich"), but in this case, preferred the simplicity of having the
check always performed (even if a particular test doesn't need one or the
other). You can't perform any meaningful testing of Hg-Git without both of
these dependencies properly configured. The main value to checking for them
in the tests (rather than just letting the tests fail) is that it gives a
meaningful error message to help people figure out how to fix their environment.
In the case that either git or dulwich is missing, the information will be
just as clearly conveyed regardless of whether its all the tests that are
skipped, or just most of them.
I didn't add dulwich to hghave (even though this is clearly the sort of thing
that hghave is intended for) because hghave is currently pulled from Mercurial
completely unchanged, and it's probably best to keep it that way.
Tested by running the tests in three configurations:
* No dulwich installed (ran 0, skipped 28, failed 0, output:
Skipped *: missing feature: dulwich)
* Bad git on path (ran 1, skipped 27, failed 0, output:
Skipped *: missing feature: git command line client)
* Working git and correct version of dulwich installed
(ran 28, skipped 0, failed 0)
Thanks to Felipe Contreras for the idea to extract this logic into a library.
It's functionally equivalent to create a directory, cd into it, git init, and
cd out of the directory, or simply git init with the directory specified.
In several cases, we were doing the former without performing any other
operations in the git repo, which just made the test unneccesarily complex.
Even in the case where we still want to cd into the directory, calling git
init with the directory name eliminates the need for a separate mkdir command.
This changeset converts the former approach to the latter with the goal of
increasing the readability of the tests.
Thanks to Felipe Contreras for the patch which this was based on.
Previously, if dulwich wasn't available, this test would fail with a traceback
(example included below). This changeset makes it so that the test will be
skipped with an informative message if dulwich isn't available.
Traceback (most recent call last):
File "/Users/carrd/hg-repos/hg-git-queue/tests/test-url-parsing.py", line 6, in <module>
from hggit.git_handler import GitHandler
File "/Users/carrd/hg-repos/hg-git-queue/tests/../hggit/__init__.py", line 42, in <module>
import gitrepo, hgrepo
File "/Users/carrd/hg-repos/hg-git-queue/tests/../hggit/gitrepo.py", line 13, in <module>
from git_handler import GitHandler
File "/Users/carrd/hg-repos/hg-git-queue/tests/../hggit/git_handler.py", line 4, in <module>
from dulwich.errors import HangupException, GitProtocolError, UpdateRefsError
ImportError: No module named dulwich.errors
Thanks to Felipe Contreras for the patch which this was based on.
The functions were renamed to make it clearer that these are shell functions
rather than normal git/hg commands, and to make it clearer which tool is being
invoked.
Old name | New name
------------------------
commit | fn_git_commit
tag | fn_git_tag
hgcommit | fn_hg_commit
hgtag | fn_hg_tag
Extraction from test-encoding.t was left for a subsequent patch, as I was seeing
unexpected output changes when I attempted the extraction.
The gitcommit and hgcommit functions in test-bookmark-workflow.t were left
as-is for now, as they have a different behavior than the standard version
(separate counters for each).
Thanks to Felipe Contreras for the patch which this was based on.
Even though the MQ extension was only used in a single test
(test-pull-after-strip.t), I included it in the testutil. It shouldn't hurt
anything to have it enabled and not used, and saves us from having to deal
with enabling extensions in individual tests at all.
Similarly, this changeset results in the graphlog extension being enabled
for all tests, even though there were some that didn't use it before. This is
even less significant in Mercurial 2.3+, since in those versions, graphlog is
part of core, and is available even when the extension is disabled.
In converting this test to the unified format, it looks like we missed this
line. It was accidentally being treated as a comment rather than executable.
Previously, the hghave checks that were commented out in the tests were broken
if uncommented. One cause was that it was expecting hghave in the testdir,
while our testdir didn't contain hghave. Now it does.
The hghave was pulled unmodified from Mercurial 2.3, to match the version of
run-tests.py in use.
This should fix a bug introduced by 4f4ab2d which caused all tags to be
duplicated as bookmarks on pull.
Test coverage has been added for pull to allow verifying the fix.
