We do not want anything computed with the bundle overlay to be written back in
the repo. Such write will likely contains invalid data.
The short terms goal of this change is to drop use of `_cacheabletip` in bundle
repo.
Instead of preventing any cache write we allow writing cache for all content of
the original repo.
The motivation for this change is to drop the custom _writebranchcache of
bundlerepo to help extraction of the branchmap logic out of localrepo.
Problem:
getremotechanges would return the 'other' repo if nothing was incoming and
there thus wasn't any bundle to base the repo on. The 'other' could be a http
peer which only implement the functionality available over the http protocol.
Transplant could thus fail with
TypeError: argument of type 'httppeer' is not iterable
Solution:
Return the local repo instead of the remote peer if there is no reason to place
a bundlerepo on top of the local repo.
All filecache usage on repo is for logic that should be unfiltered. The
caches should be common to all filtered instances, and computation must
be done unfiltered. A dedicated storecache subclass is created for
this purpose.
bundle() revset expression returns all changes that are present
in the bundle file (no matter whether they are in the repo or not).
Bundle file should be specified via -R option.
This introduces a peer method into all repository classes, which currently
simply returns self. It also changes hg.repository so it now raises an
exception if the supplied paths does not resolve to a localrepo or descendant.
Finally, all call sites are changed to use the peer and local methods as
appropriate, where peer is used whenever the code is dealing with a remote
repository (even if it's on local disk).
Bundle repo contains both the bundle content and the content of the repository
used as a base. This create bugs with phases exchange because the "remote"
repository claim to contains changeset it does not. The easiest way to fix this
bug is to ensure a bundle repo as non publishing. This way changeset will be
seen in the same phase than locally.
This patch does not alter in which phase bundle revision are seen. For now they
are seen as if an old client had add them on the remote: They inherit their
phase from parent whatever the parent is. This is to be fixed in a later patch
Calling branchmap() or similar on a bundlerepo would write the bundle-augmented
branch cache to disk, requiring a subsequent expensive rebuild when the repo
is used without the bundle.
Extensions can hook discovery.findcommonincoming to filter out unwanted remote
changesets. This patch makes getremotechanges respect the changed remote heads
returned by such extensions.
The introduction of the new URL parsing code has created a startup
time regression. This is mainly due to the use of url.hasscheme() in
the ui class. It ends up importing many libraries that the url module
requires.
This fix helps marginally, but if we can get rid of the urllib import
in the URL parser all together, startup time will go back to normal.
perfstartup time before the URL refactoring (707e4b1e8064):
! wall 0.050692 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
current startup time (9ad1dce9e7f4):
! wall 0.070685 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
after this change:
! wall 0.064667 comb 0.000000 user 0.000000 sys 0.000000 (best of 100)
This is a long desired cleanup and paves the way for new discovery.
To specify subsets for bundling changes, all code should use the heads
of the desired subset ("heads") and the heads of the common subset
("common") to be excluded from the bundled set. These can be used
revlog.findmissing instead of revlog.nodesbetween.
This fixes an actual bug exposed by the change in test-bundle-r.t
where we try to bundle a changeset while specifying that said changeset
is to be assumed already present in the target. This used to still
bundle the changeset. It no longer does. This is similar to the bugs
fixed by the recent switch to heads/common for incoming/pull.
This replaces util.drop_scheme() with url.localpath(), using url.url for
parsing instead of doing it on its own. The function is moved from
util to url to avoid an import cycle.
hg.localpath() is removed in favor of using url.localpath(). This
provides more consistent behavior between "hg clone" and other
commands.
To preserve backwards compatibility, URLs like bundle://../foo still
refer to ../foo, not /foo.
If a URL contains a scheme, percent-encoded entities are decoded. When
there's no scheme, all characters are left untouched.
Comparison of old and new behaviors:
URL drop_scheme() hg.localpath() url.localpath()
=== ============= ============== ===============
file://foo/foo /foo foo/foo /foo
file://localhost:80/foo /foo localhost:80/foo /foo
file://localhost:/foo /foo localhost:/foo /foo
file://localhost/foo /foo /foo /foo
file:///foo /foo /foo /foo
file://foo (empty string) foo /
file:/foo /foo /foo /foo
file:foo foo foo foo
file:foo%23bar foo%23bar foo%23bar foo#bar
foo%23bar foo%23bar foo%23bar foo%23bar
/foo /foo /foo /foo
Windows-related paths on Windows:
URL drop_scheme() hg.localpath() url.localpath()
=== ============= ============== ===============
file:///C:/foo C:/C:/foo /C:/foo C:/foo
file:///D:/foo C:/D:/foo /D:/foo D:/foo
file://C:/foo C:/foo C:/foo C:/foo
file://D:/foo C:/foo D:/foo D:/foo
file:////foo/bar //foo/bar //foo/bar //foo/bar
//foo/bar //foo/bar //foo/bar //foo/bar
\\foo\bar //foo/bar //foo/bar \\foo\bar
Windows-related paths on other platforms:
file:///C:/foo C:/C:/foo /C:/foo C:/foo
file:///D:/foo C:/D:/foo /D:/foo D:/foo
file://C:/foo C:/foo C:/foo C:/foo
file://D:/foo C:/foo D:/foo D:/foo
file:////foo/bar //foo/bar //foo/bar //foo/bar
//foo/bar //foo/bar //foo/bar //foo/bar
\\foo\bar //foo/bar //foo/bar \\foo\bar
For more information about file:// URL handling, see:
http://www-archive.mozilla.org/quality/networking/testing/filetests.html
Related issues:
- issue1153: File URIs aren't handled correctly in windows
This patch should preserve the fix implemented in
5c92d05b064e. However, it goes a step further and "promotes"
Windows-style drive letters from being interpreted as host names to
being part of the path.
- issue2154: Cannot escape '#' in Mercurial URLs (#1172 in THG)
The fragment is still interpreted as a revision or a branch, even in
paths to bundles. However, when file: is used, percent-encoded
entities are decoded, so file:test%23bundle.hg can refer to
test#bundle.hg ond isk.
Immediately sends local's heads to the server to check whether the server knows them all.
If it does, we can call getbundle immediately.
Interesting test output changes are:
- added 1 changesets with 0 changes to 1 files (+1 heads)
+ added 1 changesets with 0 changes to 0 files (+1 heads)
-> The new getbundle() actually fixes a bug vs. changegroupsubset() in that it no longer
returns unnecessary files when file revs are reused.
warning: repository is unrelated
+ requesting all changes
-> The new use of common instead of bases correctly indicates that an unrelated pull
gets all changes from the server.
Add empty repository.close() and call it in dispatch.
Remove bundlerepository.__del__(), merging it into bundlerepository.close(),
which overrides repository.close().
http://docs.python.org/reference/datamodel.html says:
"It is not guaranteed that __del__() methods are called for objects that
still exist when the interpreter exits."
This avoids problem with unexpanded paths when it's not possible to
expand it at higher level (for example, if file:~/path/ is supplied as
path in schemes).
- create error.py for exception classes to reduce demandloading
- move revlog exceptions to it
- change users to import error and drop revlog import if possible
Uses ui.setconfig() to tell bundlerepo where the main repo is. This is
needed for when the --repository option is used.
Adds tests to test-bundle and a new test script
test-mq-pull-from-bundle, which plays out the situation that initially
made me detect this bug
(hg -R .hg/patches pull ../bundle.hg).
We expand our index by one entry so that index[nullrev] points to a
unique entry, the null revision. This naturally eliminates numerous
extra tests in the performance-sensitive index access functions, most
of which are now trivial again.
Adding new entries is now done with insert(-1, e) rather than
append(e).
$ hg -R bundle://foo.hg
abort: No such file or directory: foo.hg
Exception exceptions.AttributeError: "'bundlerepository' object has no attribute 'bundlefile'" in <bound method bundlerepository.__del__ of <mercurial.bundlerepo.bundlerepository object at 0xa7ab9fac>> ignored
The bundle repo code was adding a field to the index tuple, which
confused the revlogng changes. This creates a new dict instead
to maintain the extra bundle info.
Incoming ssh needs to detect the end of the changegroup, otherwise it would
block trying to read from the ssh pipe. This is done by parsing the
changegroup chunks.
bundlerepo.getchunk() already is identical to
localrepo.addchangegroup.getchunk(), which is followed by getgroup which
looks much like what you can re-use in bundlerepository.__init__() and in
write_bundle(). bundlerevlog.__init__.genchunk() looks very similar, too,
as do some while loops in localrepo.py.
Applied patch from Benoit Boissinot to move duplicate/related code
to mercurial/changegroup.py and use this to fix incoming ssh.
- use HG10UN instead of HG11 for uncompressed bundles header
- check HG10BZ for compressed bundle
- better error handling for invalid header
some notes:
- people who created uncompressed bundle will no longer be
able to use them (it could be fixed with hand-editing)
- older hg cannot detect an uncompressed bundle
(bzip2 decompression will fail).
The idea is to create a repo consisting of a normal local repository
plus all the changesets contained in a bundle.
The bundle needs to be uncompressed. A futur version could implement
the seeking through a compressed bundle.
