Clients expect a bundle2 reply to their bundle2 submission. So we
catch the Abort error and turn it into a bundle2 containing a part
transporting the exception data. The unbundling of this reply will
raise the error again.
Bid merge is a new rarely used feature that the user explicitly enabled - we
should tell/warn when the user actually is using it, just like we tell when we
not are using it.
Give a message like
note: merging 3b08d01b0ab5+ and adfe50279922 using bids from ancestors 0f6b37dbe527 and 40663881a6dd
We call a dedicated hook right after closing the transaction. This will let
people react to the transaction with all the information in hand. This hook is
experimental and will not survive in future versions.
We call a dedicated hook right before closing the transaction. This will let
people abort unbundling with all the information in hand. This hook is
experimental and will not survive in future versions.
We are registering data related to the process into the transaction hook data.
This lets other parties using the same transaction get informed of the
addchangegroup result.
It is now possible to register parameters to be used when invoking hooks in this
transaction. This will cope with the fact that bundle2 adds multiple data types
in a single transaction.
Do not expect any wide and consistent usages of this in the next release. This
will be used by bundle2 experiments first. It will be made better for the release
after.
We highlight the fact that this is experimental by moving it to an "experimental"
section, and we match the config name with the server capability name
`bundle2-exp`.
All currently core parts are moved to a `bx2` namespace (for "bundle 2
experimental"). This should avoid conflicts between the final stable
format and the one about to be released.
For the same reason, we advertise this bundle2 implementation and format as
experimental. This will leave room for field testing in 3.0 but won't conflict
with a stable implementation in 3.1.
The current implementation of bundle2 is still very experimental and the 3.0
freeze is yesterday. The current bundle2 format has never been field-tested, so
we rename the header to HG2X. This leaves the HG20 header available for real
usage as a stable format in Mercurial 3.1.
We won't guarantee that future mercurial versions will keep supporting this
`HG2X` format.
We can now retrieve them from the server during push. The capabilities are
encoded the same way as in `replycaps` part (with an extra layer of urlquoting
to escape separators).
This attribute conveys the capabilities supported by the destination of the
bundle. It is used to decide which parts to include in the bundle.
This is currently a set but will probably be turned into a dictionary to allow
capabilities with values.
We need an easy way to capture both stderr and stdout during bundle2 processing
of a remote bundle. This changeset adds simple changes to the `ui` class to make
this possible.
I expect the interface to change in future releases as bundle2 will probably want to
distinguish stdout and stderr. The current change will, however, do for now.
When a reply is built, the bundle processing will capture the output of each
handler and sends it to the client in a dedicated part.
As a side effect, this add a "remote: " prefix to destination output on local
push. This is considered okay for now as:
1. bundle2 is still experimental,
2. Matt said he could be okay to change output for bundle2,
3. This keeps the implementation simple.
This changeset does it for stdout only. stderr will be done in a future changeset.
The bundle2 processing does not create a bundle2 reply by default anymore. It
is only done if the client requests it with a `replycaps` part. This part is
called `replycaps` as it will eventually contain data about which bundle2
capabilities are supported by the client.
We have to add a flag to the test command to control whether a reply is
generated or not.
The basic idea is to do the merge planning with all the available ancestors,
consider the resulting actions as "bids", make an "auction" and
automatically pick the most favourable action for each file.
This implements the basic functionality and will only consider "keep" and
"get" actions. The heuristics for picking the best action can be tweaked later
on.
By default it will only pass ctx.ancestor as the single ancestor to
calculateupdates. The code path for merging with a single ancestor is not
changed.
Now that revsets work in a lazy way, log code can be changed to parse every
option into a revset and then evaluate it lazily.
Now expressions like
"hg log -b default -b ."
are converted into a revset using the same code as graphlog.
Multiple revisions can be specified in merge.preferancestor, separated by
whitespace. First match wins.
This makes it possible to overrule the default of picking the common ancestor
with the lowest hash value among the "best" (introduced in f19507e1bcf2).
This can for instance help with some merges where the 'wrong' ancestor is used.
There will thus be some overlap between this and the problems that can be
solved with a future 'consensus merge'.
Mercurial will show a note like
note: using 40663881a6dd as ancestor of 3b08d01b0ab5 and adfe50279922
alternatively, use --config merge.preferancestor=0f6b37dbe527
when the option is available, listing all the alternative ancestors.
Show a message like
note: using 0f6b37dbe527 as ancestor of adfe50279922 and cf89f02107e5
So far this is just a warning - there is nothing the user can do to select
another ancestor.
Before this patch, each log entries in "changelog" and "revisions"
pages of "spartan" style are not aligned by column, because:
- each log entries are separated "<table>" entries, and
- there are no fixed "width" information for each "<th>"/"<td>" entries
This patch aligns entries in "changelog" and "revisions" pages of
"spartan" style by:
- adding 'label' class to '<th>' for 'age' information, and
- setting 'width' of '<th class="label">' with fixed size
'class="age"' is not used for this purpose, because it is also used to
set "bold" font-weight
"16em" seems to be wide enough to show date information fully, when
web browser disables (or doesn't support) javascript.
Before this patch, revision numbers and hash values in "comparison"
page are gotten from not changelog but filelog.
Such filelog information is useful only for hgweb debugging, and may
confuse users.
This patch shows revision numbers and hash values gotten from
changelog in "comparison" page.
Before this patch, "parents" in pages for file doesn't show as same
parents as "hg parents -r REV FILE", when the specified file is not
modified in the specified revision.
For example, it is assumed that revision A, B and D change file "f".
changelog (A) ---> (B) ---> (C) ---> (D)
filelog "f" (x) ---> (y) ------------> (z)
"/file/D/f" invokes "webutil.parents()" with filectx(z) gotten from
changectx(D), and it returns changectx(B). This is as same result as
"hg parents -r D f".
In the other hand, "/file/C/f" invokes "webutil.parents()" with
filectx(y') gotten from changectx(C), and it returns changectx(A),
because filectx(y') is linked to changectx(B), and works like
filectx(y) in some cases.
In this case, revision B is hidden from users browsing file "f" in
revision C.
This patch shows as same parents as "hg parents -r REV FILE" in pages
for file, by making "webutil.parents()" return:
- "linkrev()"-ed revision only, if:
- specified context instance is "filectx" (because
"webutil.parents()" is invoked with changectx, too), and
- (1) the revision from which filectx is gotten and (2) the one to
which filectx is linked are different from each other
- revision gotten from "ctx.parents()", otherwise
Before this patch, "comparison" shows unexpected result, when the
specified file is not modified in the specified revision, even though
"diff" shows empty result.
When REV doesn't change specified FILE, "diff" shows:
"hg diff -c REV FILE"
but "comparison" shows:
"hg diff -c `hg parents -r REV FILE` FILE"
In other words, the former gets parent from changelog, but the latter
gets one from filelog.
This may confuse users browsing (and switching "diff" and
"comparison" of) files in the specified revision.
This patch makes "comparison" get parent from not filelog but
changelog, to show "hg diff -c REV FILE" in both "diff" and
"comparison" pages.
This patch also fixes same problem of "coal" style, because it re-uses
"filerevision.tmpl" of "paper" style.
"gitweb" and "monoblue" styles don't have such problems.
"spartan" style doesn't have "bookmarks" page definition itself.
This function allows returning only the nth "word" from a string. By default
a string is split as by Python's split() function default, but an optional
third parameter can also override what string the string is split by.
This was mistakenly moved from workingctx to committablectx in
edbbc56a5e4f. Since the method is querying the dirstate, the only logical place
is for it to reside is in workingctx.
In 4c8873aad79a, memctx was changed to inherit from committablectx, this in
turn added the 'substate' property to memctx. It turns out that the
newcommitphase method tested for this property:
def newcommitphase(ui, ctx):
commitphase = phases.newcommitphase(ui)
substate = getattr(ctx, "substate", None)
if not substate:
return commitphase
Currently, memctx isn't ready to handle substates, nor removed files, so we
explicitly must set substate=None to get the old behavior back. In the future,
we can decide how memctx should play with substate. For now, this fixes
third-party extensions and some internal code dealing with subrepos.
No real changes.
pattern: 'kind:pat' as specified on the command line
patterns, pats: list of patterns
kind: 'path', 'glob' or 're' or ...
pat: string in the corresponding 'kind' format
kindpats: list of (kind, pat) tuples
If two revisions are linearly related, there will only be one ancestor, and
commonancestors and commonancestorsheads would give the same result.
commonancestorsheads is however slightly simpler, faster and more correct.
If two revisions are linearly related, there will only be one ancestor, and
commonancestors and commonancestorsheads would give the same result.
commonancestorsheads is however slightly simpler, faster and more correct.
We have seen some failures on Windows that could seem like the unlinks of
temporary files were failing. That could perhaps be because the merge tool
somehow still held the files open.
Instead of the bare bone os.unlink, use our util.unlink with special
rename/retry handling on Windows.
Future patches will allow patch.diff to take a basectx so node1 (and node2)
could make hexfunc error out. Instead, we'll call the node function on the
context object directly.
This function returns a string only if it starts with a given string.
It is particularly useful when combined with splitlines and/or used with
conditionals that fail when empty strings are passed in to take action
based on the contents of a line.
This is useful for applying changes to each line, and it's especially powerful
when used in conjunction with conditionals to modify lines based on content.
If stdout is piped, status message won't be flushed until client connects to
the server and access log is written to stdout. It seems bad idea to queue
start-up banner of server process.
This lets us preserve the original tag order (as specified on the .hgtags file
that is being read). This will be useful to preserve the tag order when saving
the result of a successful automated .hgtags merge (which will be introduced on
a future patch).
There shouldn't be much impact on performance because the sortdict that
_readtags returns is then used to update the alltags regular dict (which
remains a regular dict).
With this patch, hg status and hg diff regain their previous speed.
The following tests are run against a working copy with over 270,000 files.
Here, 'before' means without this or the previous patch applied.
Note that in this case `hg perfstatus` isn't representative since it doesn't
take dirstate parsing time into account.
$ time hg status # best of 5
before: 2.03s user 1.25s system 99% cpu 3.290 total
after: 2.01s user 1.25s system 99% cpu 3.261 total
$ time hg diff # best of 5
before: 1.32s user 0.78s system 99% cpu 2.105 total
after: 1.27s user 0.79s system 99% cpu 2.066 total
Previously, while unpacking the dirstate we'd create 3-4 new CPython objects
for most dirstate values:
- the state is a single character string, which is pooled by CPython
- the mode is a new object if it isn't 0 due to being in the lookup set
- the size is a new object if it is greater than 255
- the mtime is a new object if it isn't -1 due to being in the lookup set
- the tuple to contain them all
In some cases such as regular hg status, we actually look at all the objects.
In other cases like hg add, hg status for a subdirectory, or hg status with the
third-party hgwatchman enabled, we look at almost none of the objects.
This patch eliminates most object creation in these cases by defining a custom
C struct that is exposed to Python with an interface similar to a tuple. Only
when tuple elements are actually requested are the respective objects created.
The gains, where they're expected, are significant. The following tests are run
against a working copy with over 270,000 files.
parse_dirstate becomes significantly faster:
$ hg perfdirstate
before: wall 0.186437 comb 0.180000 user 0.160000 sys 0.020000 (best of 35)
after: wall 0.093158 comb 0.100000 user 0.090000 sys 0.010000 (best of 95)
and as a result, several commands benefit:
$ time hg status # with hgwatchman enabled
before: 0.42s user 0.14s system 99% cpu 0.563 total
after: 0.34s user 0.12s system 99% cpu 0.471 total
$ time hg add new-file
before: 0.85s user 0.18s system 99% cpu 1.033 total
after: 0.76s user 0.17s system 99% cpu 0.931 total
There is a slight regression in regular status performance, but this is fixed
in an upcoming patch.
Upcoming patches will switch away from using Python tuples for dirstate values
in compiled builds. Make that easier by introducing a variable called
dirstatetuple, currently set to tuple. In upcoming patches, this will be set to
an object from the parsers module.
For files written out in the last second, Mercurial used to invalidate all the
stat data (state, size, mode, mtime) while persisting to disk. This included
invalidating the data in the dirstate dict as well.
In commit c7a2ac9361a7, this was found to be unnecessary, and Mercurial
switched to invalidating only the mtime. However, in the C version of
pack_dirstate the value set in the dict was still the fully invalidated one.
Switch to invalidating just the mtime in the dict as well.
This patch uses destvfs with base as .hg directory in shared destination
repository to update filesystem function with vfs. Some methods are changed to
use vfs functions with destvfs.
branch 'default' HG: bookmark '@' HG: changed mercurial/hg.py
In this patch, dstwvfs is added to use vfs functions in working directory of
destination shared repository. Existing filesystem operations are updated
to use vfs functions through dstwvfs.
Since decorators are evaluated at module load time and since the
@command decorator imports commands, the norepo variable (along with
its friends) may not be declared yet. These variables are now declared
before @command usage to ensure they are present.
In case of an unknown parameter passed to ``getbundle``, the server prints a
message and ignores the parameter. This message was misleadingly prefixed with
"abort: ". The use of "abort: " here is clearly wrong as nothing is actually
aborted and the ``getbundle`` call proceeds without the parameter.
The message is now prefixed with "warning: " and rephrased to make it clearer
that the parameter was simply ignored.
basectx.status may reorder the list after workingctx._poststatus is called,
so workingctx must copy it. Otherwise, wctx.deleted() would return "unknown"
files, for example.
This method is needed to have memfilectx behave like the other file
contexts. We can't just inherit this method because each file context has
different behavior: filectx reads from the filelog, and workingfilectx reads
from the disk. Therefore, we define memfilectx to return the size of the data
in memory.
According to foozy:
This patch should be backed out, because "bundlename" and "bundle" in
this case are not relative paths to the root of repositories.
The former is specified via "hg incoming --bundle BUNDLENAME"
(relative path to cwd, or absolute one), and the latter is generated
in "changegroup.writebundle" by "tempfile.mkstemp" for internal
temporary usage (absolute path).
To be exact, the latter hunk in this patch can be applied, because
"os.join" for two absolute paths can generate correct result. But the
former hunk can't, because it may unexpected result, if specified path
is relative to cwd and cwd != root.
We already have a ":" after the user name to denote the start of the
description. The current usage of quotes around the description is
problematic as the truncation to 80 chars is likely to drop the
closing quote. This may confuse syntax coloration in some editors.
This patch changes the calling signature of memfilectx's __init__ to fall in
line with the other file contexts.
Calling code and tests have been updated accordingly.
commitablectx has a much more robust implementation of flags() so we will use
that instead of just blindly calling the flags function for the given path.
This is a slight change in definition from memctx returning only modified() but
its parent's definition is more consistent with other contexts' behavior so we
can call this change a slight bugfix and step in the right direction.
This patch marks the start of having memctx inherit from committablectx,
thereby making it a full-fledged context that will eventually grow the ability
to perform diffing and also merging.
We now use a bundle2 container to push all phase updates at the same time. This
is a significant step forward, even if further refactoring is needed to unify
phase push with the changeset push.
This function is just a shorthand for ``encoding.fromlocal``. It will help
hiding the encoding business from other code exchanging pushkey data over the
wire.
After ``listkeys`` we can now include ``pushkey`` request in a bundle2. The part
uses a very simple scheme closest as possible to the current wireproto command
for ``pushkey``. We may eventually decide for a more sophisticated part format
before the protocol becomes final.
This function is just a shorthand for ``decoding.fromlocal``. It will help
hiding the encoding business from other code exchanging pushkey data over the
wire.
We use bundle2 to retrieve the remote phase data at the same time as
changesets. This reduces the amount of requestis and should improve consistency
as the server can ensure nothing changed between the retrieval of those parts.
A new ``listkeys`` is supported by getbundle. It is a list of namespaces whose
content should be included in the bundle.
An appropriate entry has been added to the wireproto map of getbundle arguments
and a new bundle2 capability is advertised.
There are still no codes that request such parts in core mercurial.
In addition to listing the expected options for ``getbundle``, we also list their
types and handle the encoding/decoding automatically. This should make it easier
for extensions to transmit additional information to getbundle.
This function factors the creation of appropriate entries to use in
``bundlecaps`` argument of ``getbundle``. This cleans up code calling
``getbundle`` and helps its usage in more part of the code.
The process of decoding remote bundle2caps blob into a dictionary is cumbersome.
We move it into a small helper function. This will clarify code that reads
bundle2 capabilities of peers and helps using it in new places.
Advisory parts are advisory. If a handler exists but does not support the
proper parameters, we can safely ignore it.
Test has been updated to include this case.
Once we picked a handler, we check that all mandatory parameter keys are
properly supported. If not we raise an exception.
We added a test for this case.
The code now fails for any part with unknown mandatory parameters. We will
ignore such errors for advisory parts in a later changeset.
If we are to enforce the mandatory aspect of parameter, we need a way to
discover what a handler supports. The best option we end up with is this a simple
declaration of known parameters at registration time.
We simply plug the list of parameters on the function object because Python lets
us do that and there is no benefit for a more complicated way.
One of the handlers is updated for example and testing.
We picked a null character to split each parameter during the transfer. This is
fragile if the same character is used in parameter name. However other
codes will already behave in a strange way in that case, so we are not
introducing any regression. A better format may be picked for the final
version of the protocol.
This is a backward compatibility breakage per se. But bundle2 was explicitly
flagged as experimental, and this is one an error path anyway. So the worse
possible outcome from this change is to still have a crash but with a different
message.
We are going to raise exceptions for a wider range of cases: unsupported
mandatory stream and part parameters. We rename the exception with a wider
name.
In the refactoring of removing localrepo.status, e283486177bd, we accidentally
changed the return type from a tuple to a list. Philosophically, this is
incorrect so we explicitly return a tuple again.
