See inline comments for why the additional metadata needs to be stored.
This literally breaks all the hashes because of the additional metadata. The
changing of hashes is unfortunate but necessary to preserve bidirectionality.
While this could be broken up into multiple commits, there was no way to do
that while preserving bidirectionality. Following the principle that every
intermediate commit must result in a correct state, I decided to combine the
commits.
Since the Git to Mercurial conversion process is incremental, it's at risk of
missing files, or recording files the wrong way, or recording the wrong commit
metadata. Add a command called 'gverify' that can verify the contents of a
particular Mercurial rev against the corresponding Git commit.
Currently, this is limited to checking file names, flags and contents, but this
can be made as robust as desired. Further additions will probably require
refactoring git_handler.py a bit though.
This function is pretty fast: on a Linux machine with a warm cache, verifying a
repository with around 50,000 files takes just 20 seconds. There is scope for
further improvement through parallelization, but conducting tree walks in
parallel is non-trivial with the current worker infrastructure in Mercurial.
l33t pointed out that currently, Hg-Git doesn't provide any confirmation that a
push was successful other than the exit code. Normal Mercurial provides a
couple other messages followed by "added X changesets with Y changes to
Z files". After this change, Hg-Git will provide much more similar output.
It's not identical, as the underlying model is substantially different, but the
concept is the same. The main message is "added X commits with Y trees and
Z blobs".
This change doesn't affect the output of what references/branches were touched.
That will be addressed in a subsequent commit.
Dulwich doesn't provide an easy hook to get the information needed for this
output. Instead of passing generate_pack_contents as the pack generator
function to send_pack, I pass a custom function that determines the "missing"
objects, stores the counts, and then calls generate_pack_contents (which then
will determine the "missing" objects again.
The new expected output:
searching for changes # unless quiet true
<N> commits found # if verbose true
list of commits: # if debugflag true and at least one commit found
<each hash> # if debugflag true and at least one commit found
adding objects # if at least one commit found unless quiet true
added <N> commits with <N> trees and <N> blobs # if at least one object unless
# quiet true
https://bitbucket.org/durin42/hg-git/issue/64/push-confirmation
Mercurial allows specifying which repository to use via the -R/--repository
option. Git allows a similar function using the --git-dir option. By using
these options, in many cases we can avoid checking the current directory.
This makes tests easier to understand, as you don't need to remember which
directory you're in to understand what's going on. It also makes tests easier
to write, as you don't need to remember to cd out of a directory when you're
done doing things there.
Thanks to Felipe Contreras for the patch which this was based on.
One or both of these requirements were in almost every test in exactly the same
way. Now, these checks are performed in every test that uses the testutil.
This makes it easier for test authors to add these checks into new tests (just
add a reference to the testutil, which you'd probably want anyway).
We considered having each test declare their requirements (currently, either
"git" or "dulwich"), but in this case, preferred the simplicity of having the
check always performed (even if a particular test doesn't need one or the
other). You can't perform any meaningful testing of Hg-Git without both of
these dependencies properly configured. The main value to checking for them
in the tests (rather than just letting the tests fail) is that it gives a
meaningful error message to help people figure out how to fix their environment.
In the case that either git or dulwich is missing, the information will be
just as clearly conveyed regardless of whether its all the tests that are
skipped, or just most of them.
I didn't add dulwich to hghave (even though this is clearly the sort of thing
that hghave is intended for) because hghave is currently pulled from Mercurial
completely unchanged, and it's probably best to keep it that way.
Tested by running the tests in three configurations:
* No dulwich installed (ran 0, skipped 28, failed 0, output:
Skipped *: missing feature: dulwich)
* Bad git on path (ran 1, skipped 27, failed 0, output:
Skipped *: missing feature: git command line client)
* Working git and correct version of dulwich installed
(ran 28, skipped 0, failed 0)
Thanks to Felipe Contreras for the idea to extract this logic into a library.
It's functionally equivalent to create a directory, cd into it, git init, and
cd out of the directory, or simply git init with the directory specified.
In several cases, we were doing the former without performing any other
operations in the git repo, which just made the test unneccesarily complex.
Even in the case where we still want to cd into the directory, calling git
init with the directory name eliminates the need for a separate mkdir command.
This changeset converts the former approach to the latter with the goal of
increasing the readability of the tests.
Thanks to Felipe Contreras for the patch which this was based on.
Thanks to Felipe Contreras for the patch which this was based on.
The functions were renamed to make it clearer that these are shell functions
rather than normal git/hg commands, and to make it clearer which tool is being
invoked.
Old name | New name
------------------------
commit | fn_git_commit
tag | fn_git_tag
hgcommit | fn_hg_commit
hgtag | fn_hg_tag
Extraction from test-encoding.t was left for a subsequent patch, as I was seeing
unexpected output changes when I attempted the extraction.
The gitcommit and hgcommit functions in test-bookmark-workflow.t were left
as-is for now, as they have a different behavior than the standard version
(separate counters for each).
Thanks to Felipe Contreras for the patch which this was based on.
Even though the MQ extension was only used in a single test
(test-pull-after-strip.t), I included it in the testutil. It shouldn't hurt
anything to have it enabled and not used, and saves us from having to deal
with enabling extensions in individual tests at all.
Similarly, this changeset results in the graphlog extension being enabled
for all tests, even though there were some that didn't use it before. This is
even less significant in Mercurial 2.3+, since in those versions, graphlog is
part of core, and is available even when the extension is disabled.
When communicating with the user on push/outgoing, Mercurial doesn't show a
"exporting hg objects to git" message, so we shouldn't. The message has been
changed to be shown if --verbose is specified.
The output for "hg push" when there were no changes didn't quite match between
Mercurial with and without Hg-Git, so I changed the behavior to bring it into
synch. The existing "creating and sending data" message was changed to be
included if --verbose is specified.
Dulwich now supports local repositories just fine. Not using the daemon makes
the tests easier to read and more reliable (less likely to be skipped because
a stray daemon is holding onto the port).
In many cases we were piping to a python script to strip out the varying leading
path to the test repos. This is no longer needed, as the modern run-test.py
automatically substitutes the leading path as $TESTTMP. Eliminating the piping
makes the tests easier to read and write, as well as allowing the exit codes
to be verified by the test.
