The changed test lines were added in 6a934a7b8d54, which
landed on stable. When this changeset merged to the default
branch, the test failed because 31ed0254094d (on default
but not stable) changed the order of working directory update
when performing a share-based clone from pooled storage.
The changes in this patch are similar to test changes in
31ed0254094d.
There are race conditions between clients performing a shared clone
to pooled storage:
1) Clients race to create the new shared repo in the pool directory
2) 1 client is seeding the repo in the pool directory and another goes
to share it before it is fully cloned
We prevent these race conditions by obtaining a lock in the pool
directory that is derived from the name of the repo we will be
accessing.
To test this, a simple generic "lockdelay" extension has been added.
The extension inserts an optional, configurable delay before or after
lock acquisition. In the test, we delay 2 seconds after lock acquisition
in the first process and 1 second before lock acquisition in the 2nd
process. This means the first process has 1s to obtain the lock. There
is a race condition here. If we encounter it in the wild, we could
change the dummy extension to wait on the lock file to appear instead
of relying on timing. But that's more complicated. Let's see what
happens first.
When pooled storage is enabled, `hg clone` will initialize a repo
from a local repo using the store sharing mechanism then pull from
the originally requested repo.
Before this patch, the working directory update occurred between
these steps. This meant that we would only update to revisions that
were already present in the local pooled storage.
This patch moves the update to after we pull from the originally
requested repository so we may check out a revision that didn't yet
exist locally. In other words, it makes the behavior like normal
`hg clone`.
Before this patch, repo could be set to None for wrong -R. It's okay for
commands that can reject repo=None, but the command server have a problem
because:
- it accepts repo=None for "unbound" mode
- and it reenters dispatch() where repo object is created for cwd by default
Test outputs are changed because the error is detected earlier. I think new
message is better than ".hg not found".
Before this patch, when auto sharing is enabled, 'hg.clone()' tries to
create local clone regardless of locality of the clone destination on
the host, and causes failure.
To avoid auto sharing when the clone destination is remote, this patch
adds examination of 'islocal(dest)' before auto sharing in
'hg.clone()'.
'islocal(dest)' is examined after 'sharepool', because:
- the former is more expensive than the latter
- without enabling share extension, the later is always negative
Many 3rd party consumers of Mercurial have created wrappers to
essentially perform clone+share as a single operation. This is
especially popular in automated processes like continuous integration
systems. The Jenkins CI software and Mozilla's Firefox release
automation infrastructure have both implemented custom code that
effectively perform clone+share. The common use case here is that
clients want to obtain N>1 checkouts while minimizing disk space and
network requirements. Furthermore, they often don't care that a clone
is an exact mirror of a remote: they are simply looking to obtain
checkouts of specific revisions.
When multiple third parties implement a similar feature, it's a good
sign that the feature is worth adding to the core product. This patch
adds support for an easy-to-use clone+share feature.
The internal "clone" function now accepts options to control auto
sharing during clone. When the auto share mode is active, a store will
be created/updated under the base directory specified and a new
repository pointing to the shared store will be created at the path
specified by the user.
The share extension has grown the ability to pass these options into
the clone command/function.
No command line options for this feature are added because we don't
feel the feature will be popular enough to warrant their existence.
There are two modes for auto share mode. In the default mode, the shared
repo is derived from the first changeset (rev 0) in the remote
repository. This enables related repositories existing at different URLs
to automatically use the same storage. In environments that operate
several repositories (separate repo for branch/head/bookmark or separate
repo per user), this has the potential to drastically reduce storage
and network requirements. In the other mode, the name is derived from the
remote's path/URL.
This warning exists to prevent git users from prematurely polluting
their namespace when trying out Mercurial. But for repos that already
have multiple branches, understanding what branches are is not
optional so we should just shut up.
When the progress extension is not enabled, each call to 'ui.progress' used to
issue a debug message. This results is a very verbose output and often redundant
in tests. Dropping it makes tests less volatile to factor they do not meant to
test.
We had to alter the sed trick in 'test-rename-merge2.t'. Sed is used to drop all
output from a certain point and hidding the progress output remove its anchor.
So we anchor on something else.
The revbranchcache is read on demand before it will be used for updating the
branch map. It is written back when the branchmap is written and it will thus
use the same locking as branchmap. The revbranchcache instance is short-lived;
it is only stored in the branchmap from .update() is invoked and until .write()
is invoked. Branchmap already assume that the repo is locked in that case.
The use of revbranchcache for branch map updates will make sure that the
revbranchcache "always" is kept up-to-date.
The perfbranchmap benchmark is somewhat bogus, especially when we can see that
the caching makes a significant difference between the realistic case of a
first run and the rare case of rerunning it with a full cache. Here are some
'base' numbers on mozilla-central:
Before:
! wall 6.912745 comb 6.910000 user 6.840000 sys 0.070000 (best of 3)
After - initial, cache is empty:
! wall 7.792569 comb 7.790000 user 7.720000 sys 0.070000 (best of 3)
After - cache is full:
! wall 0.879688 comb 0.880000 user 0.870000 sys 0.010000 (best of 4)
The overhead when running with empty cache comes from checking, missing and
updating it every time.
Most of the performance improvement comes from not having to extract the branch
info from the changelog. The last doubling of performance comes from no longer
having to convert all branch names to local encoding but reuse the few already
converted branch names.
On the hg repo:
Before:
! wall 0.715703 comb 0.710000 user 0.710000 sys 0.000000 (best of 14)
After:
! wall 0.105489 comb 0.110000 user 0.110000 sys 0.000000 (best of 87)
Local clones copy/hardlink over files directly, so the branchcaches should all
be valid. There's a slight chance that a read operation would update one of the
branchcaches, but we hold a lock over the source repo so they shouldn't cause
an invalid branchcache to be copied over, just a different, valid one.
Changset 39a4d61c40d6 rewriting "hg.copystore()" with vfs uses
'dstbase + "/lock"' instead of "os.path.join()", because target files
given from "store.copyfiles()" already uses "/" as path separator
But in the repository using revlog format 0, "dstbase" becomes empty
("data" directory is located under ".hg" directly), and 'dstbase +
"/lock"' is treated as "/lock": in almost all cases, write access to
"/lock" causes "permission denied".
This patch uses "os.path.join()" to join path components which may be
empty in "hg.copystore()".
This adds a new root hghave to test against. Almost all of these are a
subset of unix-permissions, but that is also used for checking exec
bit handling.
Mercurial would sometimes exit with:
abort: No such file or directory
where str of the actual OSError exception was the more helpful:
[Errno 2] No such file or directory: ''
The exception will now always show the filename and quote it:
abort: No such file or directory: ''
As a part of migration to vfs, this patch uses "self.root", which can
be recognized as the path relative to "self.vfs", instead of "path"
argument.
This fix allows to make invocations of "util.makedirs()" and
"os.path.exists()" while ensuring repository directory in
"localrepository.__init__()" ones indirectly via vfs.
But this fix also raises issue 2528: "hg clone" with empty destination.
"path" argument is empty in many cases, so this issue can't be fixed
in the view of "localrepository.__init__()".
Before this patch, it is fixed by empty-ness check ("not name") of
exception handler in "util.makedirs()".
try:
os.mkdir(name)
except OSError, err:
if err.errno == errno.EEXIST:
return
if err.errno != errno.ENOENT or not name:
raise
This requires "localrepository.__init__()" to invoke "util.makedirs()"
with "path" instead of "self.root", because empty "path" is treated as
"current directory" and "self.root" becomes valid path.
But "hg clone" with empty destination can be detected also in
"hg.clone()" before "localrepository.__init__()" invocation, so this
patch re-fixes issue2528 by checking it in "hg.clone()".
The case
$ hg clone a ''
already aborted before c1e68cd8cfa8, whereas
$ hg clone --pull a ''
tripped the fixed issue2528.
Both basic cases are expected to fail with an abort.
When issuing `hg pull -r REV` in a repo with no common ancestor with the
remote repo, the message 'requesting all changes' is printed, even though only
the changese that are ancestors of REV are actually requested. This can be
confusing for users (see
http://www.selenic.com/pipermail/mercurial/2010-October/035508.html).
This silences the message if (and only if) the '-r' option was passed.
This adds a " (glob)" marker that works like a simpler version of
(re): "*" is converted to ".*", and "?" is converted to ".".
Both special characters can be escaped using "\", and the backslash
itself can be escaped as well.
Other glob-style syntax, like "**", "[chars]", or "[!chars]", isn't
supported.
Consider this test:
$ hg glog --template '{rev}:{node|short} "{desc}"\n'
@ 2:20c4f79fd7ac "3"
|
| o 1:38f24201dcab "2"
|/
o 0:2a18120dc1c9 "1"
Because each line beginning with "|" can be compiled as a regular
expression (equivalent to ".*|"), they will match any output.
Similarly:
$ echo foo
The blank output line can be compiled as a regular expression and will
also match any output.
With this patch, none of the above output lines will be matched as
regular expressions. A line must end in " (re)" in order to be matched
as one.
Lines are still matched literally first, so the following will pass:
$ echo 'foo (re)'
foo (re)
