If a matcher doesn't implement visitdir, we should be returning True so that
tree traversals are not prematurely pruned. The old value of False would prevent
tree traversals when using any matcher that didn't implement visitdir.
Differential Revision: https://phab.mercurial-scm.org/D83
Thinking a bit further about list/dict subscript operation (proposed by
issue 5534), I noticed the current data structure, a dict of dicts, might
not be ideal.
For example, if there were "'[' index ']'" and "'.' key" operators,
"{parents[0]}" would return "{p1rev}:{p1node}", and we would probably want to
write "{parents[0].desc}" to get the first element of "{parents % "{desc}"}".
This will basically execute parents[0].makemap()['desc'] in Python.
Given the rule above, "{peerpaths.default.pushurl}" will be translated to
peerpaths['default'].makemap()['pushurl'], which means {peerpaths} should
be a single-level dict and sub-options should be makemap()-ed.
"{peerpaths % "{name} = {url}, {pushurl}, ..."}"
(Well, it could be peerpaths['default']['pushurl'], but in which case,
peerpaths['default'] should be a plain dict, not a hybrid object.)
So, let's mark the current implementation experimental and revisit it later.
find_deepest is used to find the "best" ancestors given a list. In the main
loop it keeps an invariant called 'ninteresting' which is supposed to contain
the number of non-zero entries in the 'interesting' array. This invariant is
incorrectly maintained, however, which leads the the algorithm returning an
empty result for certain graphs. This has been fixed.
Also, the 'interesting' array is supposed to fit 2^ancestors values, but is
incorrectly allocated to twice that size. This has been fixed as well.
The tests in test-ancestor.py compare the Python and C versions of the code,
and report the error correctly, since the Python version works correct. Even
so, I have added an additional test against the expected result, in the event
that both algorithms have an identical error in the future.
This fixes issue5623.
In some case, the default of one value is derived from other value. We add a
way to register them anyway and an associated devel-warning.
The registration is very naive for the moment. We might be able to have a
better way for registering each of these cases but it could be done later.
cg.apply used to returns the added nodes. Callers doesn't have a use for it
anymore, remove the added node and stops recording it in the current
operation.
This information was added in the current release cycle so no extensions
breakage should happens.
updatephases have no use of the 'addednodes' parameter since 44be3dc1fec8.
However caller are still passing it for nothing, remove the parameter and
remove computing of the added nodes in caller.
We adds new computation to find and record the revision affected by the
boundary retraction. This add more complication to the function but this seems
fine since it is only used in a couple of rare and explicit cases (`hg phase
--force` and `hg qimport`).
Having strong tracking of phase changes is worth the effort.
It is useful to detect noop and avoid expensive operations in this case.
We return the information to inform the caller of a possible update. Top level
function might need to react to the phase update (eg: invalidating some
caches, tracking phase change).
We rework the code to call 'registernew' before any other phase advancement.
This make 'changegroup.apply' register correct phase movement for the added
and bundled nodes.
This new function will be used by code that adds new changesets. It ajusts the
phase boundary to make sure added changesets are at least in their target
phase (they end up in an higher phase if their parents are in a higher phase).
Having a dedicated function also simplify the phases tracking. All the new
nodes are passed as argument, so we know that all of them needs to have their
new phase registered. We also know that no other nodes will be affected, so no
extra computation are needed.
This function differ from 'retractboundary' where some nodes might change
phase while some other might not. It can also affect nodes not passed as
parameters.
These simplification also apply to the computation itself. For now we use
'_retractboundary' there by convenience, but we may introduces simpler code
later.
While registering new revisions, we still need to check the actual phases of
the added node because it might be higher than the target phase (eg: target is
draft but parent is secret).
We will migrate users over the next changesets.
At the moment the 'retractboundary' function is called for multiple reasons:
First, actually retracting boundaries. There are only two cases for theses:
'hg phase --force' and 'hg qimport'. This will need extra graph computation to
retrieve the phase changes.
Second, setting the phases of newly added changesets. In this case we already
know all the affected nodes and we just needs to register different
information (old phase is None).
Third, when reducing the set of roots when advancing phase. The phase are
already properly tracked so we do not needs anything else in this case.
To deal with this difference in phase tracking, we extract the core logic into
a private method that all three cases can use.
Makes advanceboundary record the phase movement of affected revisions in
tr.changes['phases'].
The tracking is not usable yet because the 'retractboundary' function can also
affect phases.
We'll improve that in the coming changesets.
When advancing phases, we compute the new roots for the phases above. During
this process, we need to compute all the revisions that change phases (to the
new target phases). Extract these revisions into a separate variable. This
will be useful to record the phase changes in the transaction.
It seems that we were calling retractboundary for each phases to process.
Putting the retractboundary out of the loop reduce the number of calls,
helping tracking the phases changes.
unionmatcher is currently used where only a limited subset of its
functions will be called. Specifically, visitdir() is never
called. The next patch will pass it to dirstate.walk() where it will
matter that visitdir() is correctly implemented, so let's fix
that. Also add the explicitdir etc that will also be assumed by
dirstate.walk() to exist on a matcher.
Differential Revision: https://phab.mercurial-scm.org/D58
The forceincludematcher is simply a unionmatcher of a includematcher
(matching paths recursively) with the given matcher. Since the
forceincludematcher is only used by sparse, move it there.
I don't have a good sparse repo setup to test performance impact on.
Differential Revision: https://phab.mercurial-scm.org/D57
rebase will have similar logic, so let's extract it. Besides, it makes
the histedit code more readable.
We may want to parametrize acceptintervention() by the exception(s)
that should result in transaction close.
Differential Revision: https://phab.mercurial-scm.org/D66
Update the dirstate functions so that the caller supplies the full backup
filename rather than just a prefix and suffix.
The localrepo code was already hard-coding the fact that the backup name must
be (exactly prefix + "dirstate" + suffix): it relied on this in _journalfiles()
and undofiles(). Making the caller responsible for specifying the full backup
name removes the need for the localrepo code to assume that dirstate._filename
is always "dirstate".
Differential Revision: https://phab.mercurial-scm.org/D68
This was meant as a substitute for Python's "with" with multiple
context managers before we moved to Python 2.7. We're now on 2.7, so
we should have no reason to keep ctxmanager. "hg grep --all
ctxmanager" says that it was never used anyway.
Differential Revision: https://phab.mercurial-scm.org/D73
This is the result of running:
python codemod_nestedwith.py **/*.py
where codemod_nestedwith.py looks like this:
#!/usr/bin/env python
# codemod_nestedwith.py - codemod tool to rewrite nested with
#
# Copyright 2017 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import, print_function
import sys
import redbaron
def readpath(path):
with open(path) as f:
return f.read()
def writepath(path, content):
with open(path, 'w') as f:
f.write(content)
def main(argv):
if not argv:
print('Usage: codemod_nestedwith.py FILES')
for i, path in enumerate(argv):
print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
changed = False
red = redbaron.RedBaron(readpath(path))
processed = set()
for node in red.find_all('with'):
if node in processed or node.type != 'with':
continue
top = node
child = top[0]
while True:
if len(top) > 1 or child.type != 'with':
break
# estimate line length after merging two "with"s
new = '%swith %s:' % (top.indentation, top.contexts.dumps())
new += ', %s' % child.contexts.dumps()
# only do the rewrite if the end result is within 80 chars
if len(new) > 80:
break
processed.add(child)
top.contexts.extend(child.contexts)
top.value = child.value
top.value.decrease_indentation(4)
child = child[0]
changed = True
if changed:
print('updating %s' % path)
writepath(path, red.dumps())
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
Differential Revision: https://phab.mercurial-scm.org/D77
we wrap 'repo.svfs.audit' to check for the store lock when accessing file in
'.hg/store' for writing. This caught a couple of instance where the transaction
was released after the lock, we should probably have a dedicated checker for
that case.
When the appropriate developer warnings are enabled, We wrap 'repo.vfs.audit' to
check for locks when accessing file in '.hg' for writing. Another changeset will
add a 'ward' for the store vfs (svfs).
This check system has caught a handful of locking issues that have been fixed
in previous series (mostly in 4.0). I expect another batch to be caught in third
party extensions.
We introduce two real exceptions from extensions 'blackbox.log' (because a lot of
read-only operations add entry to it), and 'last-email.txt' (because 'hg email'
is currently a read only operation and there is value to keep it this way).
In addition we are currently allowing bisect to operate outside of the lock
because the current code is a bit hard to get properly locked for now. Multiple
clean up have been made but there is still a couple of them to do and the freeze
is coming.
We want to be able to do more precise check when auditing a path depending of
the intend of the file access (eg read versus write). So we now pass the 'mode'
value to 'audit' and update the audit function to accept them.
This will be put to use in the next changeset.
This function already does an excellent job of reading from context objects;
we simply need to change the single write call to eliminate all uses of the
wvfs.
As with past changes, the effect should be a no-op but opens the door to
in-memory merge later by using different context objects.
I ran into this ctypes bug while working with the Crypto API. While this could
be an issue with any Win32 API in theory, the handful of things that we call are
older functions that are unlikely to return COM errors, so I didn't retrofit
this everywhere.
The proposed syntax [1] was originally 'set{n rel}', but it seemed slightly
confusing if template is involved. On the other hand, we want to keep 'set[n]'
for future extension. So this patch introduces 'set#rel[n]' ternary operator.
I chose '#' just because it looks like applying an attribute.
This also adds stubs for 'set[n]' and 'set#rel' operators since these syntax
elements are fundamental for constructing 'set#rel[n]'.
[1]: https://www.mercurial-scm.org/wiki/RevsetOperatorPlan#ideas_from_mpm
It's sometimes useful to show hyperlinks in log output.
"{get(peerpaths, "default")}/rev/{node}"
Since each path may have sub options, "{peerpaths}" is structured as a dict
of dicts, but the inner dict is rendered as if it were a string URL. The
implementation is ad-hoc, so there are some weird behaviors described in
the test. We might need to introduce a proper way of handling a hybrid
scalar object.
This patch adds _hybrid.__getitem__() so d['path']['url'] works.
The keyword is named as "peerpaths" since "paths" seemed too generic in
log context.
The new 'phase-heads' forced all added node to secret before advancing the
boundary to work around the fact changesets were added as draft by default.
This is no longer necessary since the changegroup part can now use the
'targetphase' parameter.
Not doing this retract boundary call has a couple of advantages:
* This makes implementing phases change tracking in the transaction much
simpler since retract boundary can become a rare case.
* Bundling secret changesets is not the norm. Exchange never does that and
even for strip, the use-case is not common.Skipping the retract boundary
will avoid useless work here.
* Sending phase update on push can be simplified since we can rely on the
behavior of 'cg.apply' for most of it.
This means less phases update send for example.
* We no longer needs to track and use the addednodes during unbundling. This
make it possible to have multiple 'changegroup' and 'phase-heads' parts in the
same bundle without them interfering with each others.
The new part has not been part of any release yet so we do not offer backward
compatibility yet. It is important to update this semantic before the 4.3
freeze happens.
If we are bundling secret changeset and the bundle will contain phase, we
request the changegroup to be applied as secret.
It will be useful for next patch as we are now sure that secrets changesets
are applied as secret and not applied as draft then forced to secret.
By default unbundled changesets are drafts. We want to reduce the number of
phases changes during unbundling by giving the possibility to the bundle to
indicate the phase of unbundled changesets.
The longer terms goal is to add phase movement tracking in tr.changes and the
'retractboundary' call is making it more complicated than we want.
Since 26d535788092, the strip bundle includes the phases of the stripping
node. Hence we don't need this special case anymore.
Dropping it will helps make the phase behavior more consistent across all
exchanges medium.
This changeset attempts to solve two issues with the "followlines" UI in
hgweb. First the "followlines" action is currently not easily discoverable
(one has to hover on a line for some time, wait for the invite message to
appear and then perform some action). Second, it gets in the way of natural
line selection, especially in filerevision view.
This changeset introduces an additional markup element (a <button
class="btn-followlines">) alongside each content line of the view. This button
now holds events for line selection that were previously plugged onto content
lines directly. Consequently, there's no more action on content lines, hence
restoring the "natural line selection" behavior (solving the second problem).
These buttons are hidden by default and get displayed upon hover of content
lines; then upon hover of a button itself, a text inviting followlines section
shows up. This solves the first problem (discoverability) as we now have a
clear visual element indicating that "some action could be perform" (i.e. a
button) and that is self-documented.
In followlines.js, all event listeners are now attached to these <button>
elements. The custom "floating tooltip" element is dropped as <button>
elements are now self-documented through a "title" attribute that changes
depending on preceding actions (selection started or not, in particular).
The new <button> element is inserted in followlines.js script (thus only
visible if JavaScript is activated); it contains a "+" and "-" with a
"diff-semantics" style; upon hover, it scales up.
To find the parent element under which to insert the <button> we either rely
on the "data-selectabletag" attribute (which defines the HTML tag of children
of class="sourcelines" element e.g. <span> for filerevision view and <tr> for
annotate view) or use a child of the latter elements if we find an element
with class="followlines-btn-parent" (useful for annotate view, for which we
have to find the <td> in which to insert the <button>).
On noticeable change in CSS concerns the "margin-left" of span:before
pseudo-elements in filelog view that has been increased a bit in order to
leave space for the new button to appear between line number column and
line content one.
Also note the "z-index" addition for "annotate-info" box so that the latter
appears on top of new buttons (instead of getting hidden).
In some respect, the UI similar to line commenting feature that is implemented
in popular code hosting site like GitHub, BitBucket or Kallithea.
Python introduces a reference cycle on dynamically created types
via __mro__, making them very easy to leak. See
https://bugs.python.org/issue17950.
Previously, repo.filtered() created a type on every invocation.
Long-running processes (like `hg convert`) could call this
function thousands of times, leading to a steady memory leak.
Since we're Unable to stop the leak because this is a bug in
Python, the next best thing is to contain it.
This patch adds a cache of of the dynamically generated repoview/filter
types on the localrepo object. Since we only generate each type
once, we cap the amount of memory that can leak to something
reasonable.
After this change, `hg convert` no longer leaks memory on every
revision. The process will likely grow memory usage over time due
to e.g. larger manifests. But there are no leaks.
isfilecached() encapsulates internal implementation of filecache-ed
property.
"name in repo.unfiltered().__dict__" or so can't be used for this
purpose, because corresponded entry in __dict__ might be discarded by
repo.invalidate(), repo.invalidatedirstate() or so (fsmonitor does so,
for example).
This patch makes isfilecached() return not only whether filecache-ed
property is already cached, but also already cached value (or None),
in order to avoid subsequent access to cached object via "repo.NAME",
which prevents main Mercurial procedure after reposetup() from
validating cache.
Currently, sslutil._hostsettings() performs validation that web.cacerts
exists. However, client certificates are passed in to the function
and not all callers may validate them. This includes
httpconnection.readauthforuri(), which loads the [auth] section.
If a missing file is specified, the ssl module will raise a generic
IOException. And, it doesn't even give us the courtesy of telling
us which file is missing! Mercurial then prints a generic
"abort: No such file or directory" (or similar) error, leaving users
to scratch their head as to what file is missing.
This commit introduces explicit validation of all paths passed as
arguments to wrapsocket() and wrapserversocket(). Any missing file
is alerted about explicitly.
We should probably catch missing files earlier - as part of loading
the [auth] section. However, I think the sslutil functions should
check for file presence regardless of what callers do because that's
the only way to be sure that missing files are always detected.
The matchers that were recently moved into core from the sparse
extension override __call__, while the previously existing matchers
override matchfn. Let's switch to the latter for consistency.
When I added prefix() in 559ee9ecae07 (match: introduce boolean
prefix() method, 2014-10-28), we already had always(), isexact(), and
anypats(), so it made sense to write it in terms of them (a prefix
matcher is one that isn't any of the other types). It's only now that
I realize that it's much more natural to define prefix() explicitly
(it's one that uses path: patterns, roughly speaking) and let
anypats() be defined in terms of the others. Remember that these
methods are all used for determining which fast paths are
possible. anypats() simply means that no fast paths are possible (it
could be called complex() instead). Further evidence is that
rootfilesin:some/dir does not have any patterns, but it's still
considered to be an anypats() matcher. That's because anypats() really
just means that it's not a prefix() matcher (and not always() and not
isexact()).
This patch thus changes prefix() to return False by default and
anypats() to return True only if the other three are False. Having
anypats() be True by default also seems like a good thing, because it
means forgetting to override it will lead only to performance bugs,
not correctness bugs.
Since the base class's implementation changes, we're also forced to
update the subclasses. That change exposed and fixed a bug in the
differencematcher: for example when both its two input matchers were
prefix matchers, we would say that the result was also a prefix
matcher, which is incorrect, because e.g "path:dir - path:dir/foo" no
longer matches everything under "dir" (which is what prefix() means).
The m.isexact() and m.prefix() methods are used by callers to
determine whether m.files() can be used for fast paths. It seems safe
to let callers to any fast paths it can that rely on the empty
m.files().
This revset returns all successors, including transit nodes and the source
nodes (to be consistent with existing revsets like "ancestors").
To filter out transit nodes, use `successors(X)-obsolete()`.
To filter out divergent case, use `successors(X)-divergent()-obsolete()`.
The revset could be useful to define rebase destination, like:
`max(successors(BASE)-divergent()-obsolete())`. The `max` is to deal with
splits.
There are other implementations where `successors` returns just one level of
successors, and `allsuccessors` returns everything. I think `successors`
returning all successors by default is more user friendly. We have seen
cases in production where people use 1-level `successors` while they really
want `allsuccessors`. So it seems better to just have one single revset
returning all successors by default to avoid user errors.
In the future we might want to add `depth` keyword argument to it and for
other revsets like `ancestors` etc. Or even build some flexible indexing
syntax [1] to satisfy people having the depth limit requirement.
[1]: https://www.mercurial-scm.org/pipermail/mercurial-devel/2017-July/101140.html
* Use context manager for wlock
* Rename oldsparsematch to oldmatcher
* Always call parseconfig() because parsing an empty string yields
the same result as the old code
The sparse extension performs a lot of monkeypatching of dirstate
to make it sparse aware. Essentially, various operations need to
take the active sparse config into account. They do this by obtaining
a matcher representing the sparse config and filtering paths through
it.
The monkeypatching is done by stuffing a reference to a repo on
dirstate and calling sparse.matcher() (which takes a repo instance)
during each function call. The reason this function takes a repo
instance is because resolving the sparse config may require resolving
file contents from filelogs, and that requires a repo. (If the
current sparse config references "profile" files, the contents of
those files from the dirstate's parent revisions is resolved.)
I seem to recall people having strong opinions that the dirstate
object not have a reference to a repo. So copying what the sparse
extension does probably won't fly in core. Plus, the dirstate
modifications shouldn't require a full repo: they only need a matcher.
So there's no good reason to stuff a reference to the repo in
dirstate.
This commit exposes a sparse matcher to dirstate via a property that
when looked up will call a function that eventually calls
sparse.matcher(). The repo instance is bound in a closure, so it
isn't exposed to dirstate.
This approach is functionally similar to what the sparse extension does
today, except it hides the repo instance from dirstate. The approach
is not optimal because we have to call a proxy function and
sparse.matcher() on every property lookup. There is room to cache
the matcher instance in dirstate. After all, the matcher only changes
if the dirstate's parents change or if the sparse config changes. It
feels like we should be able to detect both events and update the
matcher when this occurs. But for now we preserve the existing
semantics so we can move the dirstate sparseness bits into core. Once
in core, refactoring becomes a bit easier since it will be clearer how
all these components interact.
The sparse extension has been updated to use the new property.
Because all references to the repo on dirstate have been removed,
the code for setting it has been removed.
This is a pretty straightforward port. Some code cleanup was
performed. But no major changes to the logic were made.
I'm not a huge fan of this function because it does multiple
things. I'd like to get things into core first to facilitate
refactoring later.
Please also note the added inline comment about the oddities
of writeconfig() and the try..except to undo it. This is because
of the hackiness in which the sparse matcher is obtained by
various consumers, notably dirstate. We'll need a massive
refactor to address this. That refactor is effectively blocked
on having the sparse dirstate hacks live in core.
activeprofiles() is a special case of a more generic function.
Furthermore, that generic function is essentially already
implemented inline in the sparse extension.
So, refactor activeprofiles() to a generic activeconfig(). Change
the only consumer of activeprofiles() to use it. And have the
inline implementation in the sparse extension use it.
Well, mostly. The annotation on subrepo functions tacks on a parenthetical to
the abort message, which seems reasonable for a generic mechanism. But now all
messages consistently spell out 'subrepository', and double quote the name of
the repo. I noticed the inconsistency in the change for the last commit.
This simply passes the 'missing' argument down from the context of the parent
repo, so the same rules apply. subrepo.bailifchanged() is hardcoded to care
about missing files, because cmdutil.bailifchanged() is too.
In the end, it looks like this addresses inconsistencies with 'archive',
'identify', blackbox logs, 'merge', and 'update --check'. I wasn't sure how to
implement this in git, so that's left for someone more familiar with it.
Since the identify command adds a '+' for missing files, it's reasonable that
this does too. Perhaps the node field's hex value should be p1+p2 for merges?
This is equivalent to the previous code. But it seems to me that if the user is
going to be prompted that a commit is needed, missing files should be ignored,
but branch and merge changes shouldn't be.
This is equivalent to the previous code, but it seems better to be explicit
about what aspects of dirty are being ignored. Perhaps they shouldn't be, since
the help text says 'followed by a "+" if the working directory has uncommitted
changes'. Both merges and branch changes are committable, even if the files are
unchanged.
Additionally, this will make the `identify` command notice missing subrepo
files, once subrepos are taught to look for missing files.
The regexes for path: and relpath: patterns are the same (since the
paths have already been normalized at the point we create the
regexes).
I don't think the "if pat == '.'" will have any effect relpath:
because relpath: patterns will have the root directory already
normalized to '' by pathutil.canonpath() (unlike path:, for which the
root gets normalized to '.' by util.normpath()).
The regexes are passed to re.match(), which matches against the
beginning of the input, so the '^' doesn't do anything.
Note that unrooted patterns, such as globs and regexes from .hgignore
are instead achieved by adding '.*' to the expression given by the
user. (That's unless the user's expression started with '^', in which
case the '.*' is not added, perhaps to keep the regex cleaner?)
Instead of wrapping committablectx.markcommitted(), we inline
the call into workingctx.markcommitted().
Per smf's review, workingctx is the proper location for this
code, as committablectx is the shared base class for it and
memctx. Since this code touches the working directory, it belongs
in workingctx.
The 'successors' part of the mappings used of be a tuple. This avoid issue from
code consuming the generator "by mistake". For example, an extension inspecting the
mapping content used to be able to iterate over the successors mapping without
consequence.
Since the mapping are small we do not expect any performance impact we use tuple
again for this.
Previously repo.anyrevs only expand aliases in [revsetalias] config. This
patch makes it more flexible to accept a customized dict defining aliases
without having to couple with ui.
revsetlang.expandaliases now has the signature (tree, aliases, warn=None)
which is more consistent with templater.expandaliases. revsetlang.py is now
free from "ui", which seems to be a good thing.
cleanupnodes takes care of bookmark movement, and bookmark movement could
cause bookmark divergent resolution as a side effect. This patch adds such
bookmark divergent resolution logic so future rebase migration will be
easier.
The revset is carefully written to be equivalent to what rebase does today.
Although I think it might make sense to remove divergent bookmarks more
aggressively, for example:
F book@1
|
E book@2
|
| D book
| |
| C
|/
B book@3
|
A
When rebase -s C -d E, "book@1" will be removed, "book@3" will be kept,
and the end result is:
D book
|
C
|
F
|
E book@2 (?)
|
B book@3
|
A
The question is should we keep book@2? The current logic keeps it. If we
choose not to (makes some sense to me), the "deleterevs" revset could be
simplified to "newnode % oldnode".
For now, I just make it compatible with the existing behavior. If we want to
make the "deleterevs" revset simpler, we can always do it in the future.
In some valid usecases, the "mapping" received by scmutil.cleanupnodes have
filtered nodes. Use unfiltered repo to access them correctly.
The added test case will fail with the old cleanupnodes code.
This is important to migrate histedit to use the cleanupnodes API.
Aliases define optional alternatives to existing options. For example the old
option ui.user was deprecated and replaced by ui.username. With this mechanism,
it's even possible to create an alias to an option in a different section.
Add ui.user as alias to ui.username as an example of this concept.
The old alternates principle in ui.config is removed as it was used only for
this option.
If the matching command lives in an in-tree extension (which is all we
scan for), and the user has disabled that extension with
"extensions.<name>=!", we were not finding it, because the path in
_disabledextensions was the empty string. If the user had set
"extensions.<name>=!<valid path>" it would work, so it seems like just
a mistake that it didn't work.
This is a pretty straightforward move of the code.
I converted the "force" argument to a keyword argument.
Like other recent changes, this code is tightly coupled with
working directory update code in merge.py. I suspect the code
will become more tightly coupled over time, possibly even moved
to merge.py. For now, let's get the code in core.
merge.calculateupdates() now filters the update actions through sparse
by default.
The filtering no-ops if sparse isn't enabled or no sparse config
is defined.
The function has been refactored to behave more like a filter
instead of a wrapper of merge.calculateupdates().
We should arguably take sparse into account earlier in
merge.calculateupdates(). This patch preserves the old behavior
of applying sparse at the end of update calculation, which is the
simplest and safest approach.
This was our last method on the custom repo type, meaning we could
remove that custom type and inline the 2 lines of code into
reposetup().
As part of the move, instead of wrapping merge.update() from
the sparse extension, we inline the function call. The ported
function now no-ops if sparse isn't enabled, making it safe to
always call.
The call site in update() may not be the most appropriate. But
it matches the previous behavior, which is the safest thing
to do. It can be improved later.
As part of the move, the function arguments changed so revs are
passed as a list instead of *args. This allows us to use keyword
arguments properly.
Since the plan is to integrate sparse into core and have it
enabled by default, we need to prepare for a sparse matcher
to always be obtained and operated on. As part of the move,
we inserted code that returns an always matcher if sparse
isn't enabled. Some callers in the sparse extension take this
into account and conditionally perform matching depending on
whether the special always matcher is seen. I /think/ this
may have sped up some operations where the extension is
installed but no sparse config is activated.
One thing I'm ensure of in this code is whether os.path.dirname()
is semantically correct. os.posixpath.dirname() (which is
exported as pathutil.dirname) might be a better choise because
all patterns should be using posix directory separators (/)
instead of Windows (\). There's an inline comment that implies
Windows was tested. So hopefully it won't be a problem. We
can improve this in a follow-up. I've added a TODO to track it.
The sparse extension contains some matcher types that are
generic and can exist in core.
As part of the move, the classes now inherit from basematcher.
always(), files(), and isexact() have been dropped because
they match the default implementations in basematcher.
Before, 0 was being used as the default signature value and we cast
the int to a string. We also handled I/O exceptions manually.
The new code uses cfs.tryread() so we always feed data into the
hasher. The empty string does hash and and should be suitable
for input into a cache key.
The changes made the code simple enough that the separate checksum
function could be inlined.
This should make optimize() more readable and less error-prone, but it doubles
the parsing cost.
(original)
$ python -m timeit -n10000 -s 'from mercurial import revsetlang as L' \
'L.optimize(L.analyze(L.parse("ancestors(x) and not ancestors(y)")))'
10000 loops, best of 3: 79.3 usec per loop
(this patch)
$ python -m timeit -n10000 -s 'from mercurial import revsetlang as L' \
'L._treecache.clear(); \
L.optimize(L.analyze(L.parse("ancestors(x) and not ancestors(y)")))'
10000 loops, best of 3: 201 usec per loop
This should make optimize() more readable, but it doubles the parsing cost.
(original)
$ python -m timeit -n10000 -s 'from mercurial import revsetlang as L' \
'L.optimize(L.analyze(L.parse("::tip")))'
10000 loops, best of 3: 18.1 usec per loop
(this patch)
$ python -m timeit -n10000 -s 'from mercurial import revsetlang as L' \
'L._treecache.clear(); L.optimize(L.analyze(L.parse("::tip")))'
10000 loops, best of 3: 48.4 usec per loop
30usec isn't dominant compared to the revset evaluation, but that is a cost.
That's why a parsed tree is cached, which can benefit in hgweb or chg server.
Extensions will want to have interactive thing for more operations or
particulary want to show more verbs. So this patch takes out the message thing
from the function so that extensions can add verbs to this. The curses one is
also not in any function so extensions can add more actions and verbs there.
Differential Revision: https://phab.mercurial-scm.org/D567
This restricts the creation of temporary files to just `_xmerge`, when we call
an external tool.
Differential Revision: https://phab.mercurial-scm.org/D403
These functions will be modified by in-memory merge, so let's extract them first and add some comments.
This also shortens `_filemerge` a bit.
Differential Revision: https://phab.mercurial-scm.org/D388
The minimum input size to exploit is ~682MB (= INT_MAX / len('\\u0000') * 2)
on 32bit system, which isn't easy to achieve using Python str in 2GB process
address space, but probably doable.
Changes the API of `ui.edit()` to take an optional `action` argument,
which is used when constructing the suffix of the temp file.
Previously, it was possible to set the suffix by specifying a `suffix` to the
optional `extra` dict that was passed to `ui.edit()`, but the goal is to
drop support for `extra.suffix` and make `action` a required argument.
To this end, `ui.edit()` now yields a `develwarn()` if `action` is not set
or if `extra.suffix` is set.
I updated all calls to `ui.edit()` I could find in `hg-crew` to specify the
appropriate `action`. This means that when creating a commit, instead
of the path to the editor file being something like:
`/tmp/hg-editor-XXXXXX.txt`
it is now something like:
`/tmp/hg-editor-XXXXXX.commit.hg.txt`
Some editors (such as Atom) make it possible to statically define a [TextMate]
grammar for files with a particular suffix. For example, because Git reliably
uses `.git/COMMIT_EDITMSG` and `.git/MERGE_MSG` as the paths for commit-type
messages, it is trivial to define a grammar that is applied when files of
either name are opened in Atom:
https://github.com/atom/language-git/blob/v0.19.1/grammars/git%20commit%20message.cson#L4-L5
Because Hg historically used a generic `.txt` suffix, it was much harder to
disambiguate whether a file was an arbitrary text file as opposed to one
created for the specific purpose of authoring an Hg commit message.
This also makes it easier to add special support for `histedit`, as it has its own
suffix that is distinct from a commit:
`/tmp/hg-histedit-XXXXXX.histedit.hg.txt`
Test Plan:
Added an integration test: `test-editor-filename.t`.
Manually tested: ran `hg ci --amend` for this change and saw that it
used `/tmp/hg-editor-ZZjcz0.commit.hg.txt` as the path instead of
`/tmp/hg-editor-ZZjcz0.txt` as the path.
Verified `make tests` passes.
Differential Revision: https://phab.mercurial-scm.org/D464
The `pushbuffer`, `popbuffer` APIs are intended to capture internal output.
They will prevent `ui.write` from writing to the actual `ui.fout`. So a
pager won't receive the output and do the right thing. In general, it does
not make sense to start a pager if ui is in the "pushbuffer" mode.
Differential Revision: https://phab.mercurial-scm.org/D574
Rewrite `flipand(y, x)` to `andsmally(x, y)` so the AST order is unchanged,
which could be more friendly to developers.
Differential Revision: https://phab.mercurial-scm.org/D579
This should be sensible default since mfunc(subset) is roughly equivalent
to 'subset & mfunc'. The order argument is still there so we can specify
'anyorder' if the order doesn't really matter.
When pruning a changeset that added a tag and then adding another tag, the
"pruned" tag gets restored. This is because the tag creation step (tags._tag()
call in tags.tag()) is currently done on the unfiltered repo. This behavior
has been there from ba5c1b80e99a which backs out 39c37a1a9e2d with no clear
reason but caution on unthought situations at that time. In this changeset, we
pass the filtered repo to tags._tag(), preventing "pruned" tags to reappear.
This somehow restores 39c37a1a9e2d, though now we arguably have a valid use
case for.
Several extensions exist that temporarily want to wrap a function (at
least narrowhg, any many of the extensions in hg-experimental). That's
why we have the unwrapfunction() that was introduced in c8cda8f6f043
(extensions: add unwrapfunction to undo wrapfunction, 2016-08-10).
This patch adds a simple wrappedfunction() that returns a context
manager.
Differential Revision: https://phab.mercurial-scm.org/D472
Add an option that only ignores whitespaces at EOL. The name of the option is
the same as Git.
.. feature::
Added `--ignore-space-at-eol` diff option to ignore whitespace differences
at line endings.
Differential Revision: https://phab.mercurial-scm.org/D422
The old documentation is a bit confusing. Namely, it's unclear whether
`define` means "I should ALWAYS define a new order", or "I should SOMETIMES
define a new order", and if it's the latter, what's the difference between
`define` and `any`?
This patch clarifies that and adds more examples.
Differential Revision: https://phab.mercurial-scm.org/D523
Keeping `order` in tree makes AST operation harder. And there could be
invalid cases if trees could be generated and compounded freely, like:
SetA(order=define) & SetB(order=define)
^^^^^^ couldn't be satisfied
This patch changes the code to calculate order on the fly, during tree
traversal. Optimization of reordering `and` arguments is preserved by
introducing a new internal operation `flipand`.
.. api::
revset.stringset() now takes 'order' as the last argument.
Differential Revision: https://phab.mercurial-scm.org/D451
The reordering optimization is more important for "and" than "or", given the
implementation details about "addset" and "filteredset" - reordering "or"
may help "__contains__" test but not iteration, reordering "and" could help
both. We are going to simplify the tree to remove ordering information.
Removing "or" reordering optimization would make things simpler.
This effectively reverts 6820a8a645ef. It tracks back to the "orset"
function added by the initial commit of revset (c9ce8ecd6).
In the future, we might consider optimization at runtime (ex. do reordering
and rewrites inside "orset").
Differential Revision: https://phab.mercurial-scm.org/D561
This patch defines `SRC` (a single source revision) and `ALLSRC` (all source
revisions) to be valid names in `--dest` revset if `--src` or `--rev` is
used. So destination could be defined differently according to source
revisions. The names are capitalized to make it clear they are "dynamically
defined", distinguishable from normal revsets (Thanks Augie for the
suggestion).
This is useful, for example, `-r 'orphan()' -d 'calc-dest(SRC)'` to solve
instability, which seems to be a highly wanted feature.
The feature is not completed, namely if `-d` overlaps with `-r`, things
could go wrong. A later patch will handle that case.
The feature is also gated by `experimental.rebase.multidest` config option
which is default off.
Differential Revision: https://phab.mercurial-scm.org/D469
Tools like Buck have patterns to ignore the creation of files (in the working
copy) that match certain patterns:
39278a4f07/src/com/facebook/buck/cli/Main.java (L259-L299)
When Buck sees a new source file (as reported by Watchman), it has to invalidate
a number of caches associated with the directory that contains the file.
Using a standard suffix, such as `~`, would make it easier for Buck and others
to filter out these types of file creation events.
The other uses of `tempfile.mkstemp()` in Hg do not appear to be problematic
because they (generally speaking) do not specify the `dir` parameter, so the
new file is created in the system-appropriate temp directory, which is outside
the working copy.
Test Plan:
`make tests`
Differential Revision: https://phab.mercurial-scm.org/D468
This was originally fixed by Mateusz Kwapich for the `metaedit` command in
fb-hgext with a test for the `metaedit` command. It didn't get upstreamed
because `metaedit` was not in core.
This patch fixes the crash and adds a test about `metadataonlyctx` to
avoid future regressions.
Differential Revision: https://phab.mercurial-scm.org/D550
The metadataonlyctx is to copy an existing context with some minor metadata
changes. If the caller only wants to change "extra", or "user", ideally it
does not have to read and pass "parents" and "text" information.
This patch makes "parents" and "text" optionally to convenient callers.
Differential Revision: https://phab.mercurial-scm.org/D548
successorssets don't returns good results for pruned commit, add a workaround
for simple cases.
A proper fix would require a large rework of successorssets algorithm, I will
send a separate series for this refactoring.
Extract, deduplicate users informations from obs markers in order to display
them.
Print all users for the moment, we might want to display users only in verbose
mode later.
Add a template function obsfateverb which use the markers information to
compute a better obsfate verb.
The current logic behind the obsfate verb is simple for the moment:
- If the successorsets is empty, the changeset has been pruned, for example:
Obsfate: pruned
- If the successorsets length is 1, the changeset has been rewritten without
divergence, for example:
Obsfate: rewritten as 2:337fec4d2edc, 3:f257fde29c7a
- If the successorsets length is more than 1, the changeset has diverged, for
example:
Obsfate: split as 2:337fec4d2edc, 3:f257fde29c7a
As the divergence might occurs on a subset of successors, we might see some
successors twice:
Obsfate: split as 9:0b997eb7ceee, 5:dd800401bd8c, 10:eceed8f98ffc; split
as 8:b18bc8331526, 5:dd800401bd8c, 10:eceed8f98ffc
This replaces patterns such as this:
```
if f in self._copymap:
del self._copymap[f]
```
with this:
```
self._copymap.pop(f, None)
```
Although eliminating the extra lookup/call may be a negligible performance win
in the standard dirstate, alternative implementations, such as
[sqldirstate](https://bitbucket.org/facebook/hg-experimental/src/default/sqldirstate/)
may see a bigger win where each of these calls results in an RPC,
so the savings is greater.
Test Plan:
`make tests`
Differential Revision: https://phab.mercurial-scm.org/D493
Previously, after pull, we show:
(run 'hg update' to get a working copy)
unconditionally. People might run `hg update` and get an exception if
`update.requiredest` is set, and get a bit frustrated. This patch changes
the code to not prompt `hg update` in that case.
Differential Revision: https://phab.mercurial-scm.org/D516
Similar to the previous patch which prevented writing the null
revision to any revlog, but this is for the wdir revision.
Thanks to Jun for pointing this out.
Differential Revision: https://phab.mercurial-scm.org/D524
With the former crecord extension, the user could edit the commit
message while he was de-/selecting hunks. By pressing 'm', an editor
showed up to edit the commit message.
With record being part of mercurial, this feature is not available
anymore. However, the help text still mentions it.
As the infrastructure needed is still present, this feature is quite
easily ported from the crecord extension to mercurial.
It seems there is no test coverage for record ui, so I tested this patch
manually on my local machine.
My repo got corrupted yesterday by something that ended up writing the
null revision to the revlog (nullid hash, not nullrev index, of
course). We use many extensions internally (narrowhg, remotefilelog,
evolve, internal extensions) and treemanifests are on. The null
revision was written to the changelog, the root manifest log, and one
subdirectory manifest log. I have no idea exactly why the null
revision was written, but it seems cheap enough to check that we
should fail instead of corrupting the repo.
Differential Revision: https://phab.mercurial-scm.org/D522
ctx._dirstatestatus() is called only from workingctx._buildstatus()
and that function, in turn, is called only from
basectx.status(). basectx.status() will always pass a matcher to
_buildstatus(), so there's no need to handle a None matcher there.
Differential Revision: https://phab.mercurial-scm.org/D492
This just makes it a little easier to follow and removes the need to
call the superclass's method in workingctx.
Differential Revision: https://phab.mercurial-scm.org/D491
This is micro optimization, but seems not bad since to/fromlocal() is called
lots of times and isasciistr() is cheap and simple.
We boldly assume that any non-ASCII characters have at least one 8-bit byte.
This isn't true for some email character sets (e.g. ISO-2022-JP and UTF-7),
but I believe no such encodings are used as a platform default. Shift_JIS,
a major crap, is okay as it should have a leading byte in 0x80-0xff range.
(with mercurial repo)
$ export HGRCPATH=/dev/null HGPLAIN=
$ hg log --time --config experimental.stabilization=all > /dev/null
(original)
time: real 7.460 secs (user 7.420+0.000 sys 0.030+0.000)
time: real 7.670 secs (user 7.590+0.000 sys 0.080+0.000)
time: real 7.560 secs (user 7.510+0.000 sys 0.040+0.000)
(this patch)
time: real 7.340 secs (user 7.260+0.000 sys 0.060+0.000)
time: real 7.260 secs (user 7.210+0.000 sys 0.030+0.000)
time: real 7.310 secs (user 7.260+0.000 sys 0.060+0.000)
The goal of this series is to have templates capable of displaying the
evolution of a changeset in a clean and human-readable way.
Add the succsandmarkers template return successors and markers so it can be
used separately like this:
> {succsandmarkers % "{get(succsandmarkers, "markers")|json};"}
The following patches will add template functions that takes successors and
markers as inputs and compute various obsfate fields from them.
We now also store markers in _succs. It will be useful for the obsfate template that
will use them to display more meaningful information like the list of users
that have evolved a changeset into its successors.
Mimic the standard API for copying in the _succs class, it makes the code
slightly cleaner and will be needed later for copying markers at the same time
than copying the list content.
It will be useful later when we will be adding markers to _succs in order to
represent a successorset with the list of markers from the root to each
successors sets. This information will be needed for the obsfate template I will
introduce.
Makes it a subclass of list so all callers will continue to work.
This is the last place we used the filepath arguments without first using the
context version.
Differential Revision: https://phab.mercurial-scm.org/D379
This will be used as an abstraction by simplemerge to get the data it used to
read off the filesystem.
Differential Revision: https://phab.mercurial-scm.org/D434
This moves the bundle2 part handling for bundlerepo out to a separate function
so extensions can participate in bundlerepo setup when using bundle2 bundles.
Differential Revision: https://phab.mercurial-scm.org/D290
Previously, iterparts would yield the part to users, then consume the part. This
changed the part after the user was given it and left it at the end, both of
which seem unexpected. Let's seek back to the beginning after we've consumed
it. I tried not seeking to the end at all, but that seems important for the
overall bundle2 consumption.
This is used in a future patch to let us move the bundlerepo
bundle2-changegroup-part to be handled entirely within the for loop, instead of
having to do a seek back to 0 after the entire loop finishes.
Differential Revision: https://phab.mercurial-scm.org/D289
A future patch will refactor certain parts of bundlerepo initiatlization such
that we need to create temp bundles from another function. Let's move this to
another function to support that.
Differential Revision: https://phab.mercurial-scm.org/D288
The Mercurial core server doesn't yet include phase-heads parts in the
bundle, but our Google-internal server wants to do
that. Unfortunately, the usual exchange still happens even if
phase-heads part is included (including the short-circuited one for
old/publishing servers). That means that even if our server (again,
the Google-internal one, but also future Mercurial core servers)
includes a phase-heads part to indicate that some heads should be
drafts, that would still get overwritten by the phase updating that
happens after. So let's fix that by marking the phase step done if we
receive at least one phase-heads part in the bundle.
Differential Revision: https://phab.mercurial-scm.org/D440
Setting `repo._shellvars` works but is not a clean way to pass the pushvars
information from the push command to the exchange operation. Therefore
change it to actually pass `pushvars` as a push operation argument instead.
This makes third party extension like remotenames easier to support pushvars
cleanly. The key value parsing and verification code has been moved to a
lower level so it's harder to be bypassed and easier to be used in
remotenames which could replace `push` command entirely.
Differential Revision: https://phab.mercurial-scm.org/D423
There are cases like bisect when the conflict message can be None. So we make
sure that we don't print None in that case.
Thanks to Martin for catching this.
Differential Revision: https://phab.mercurial-scm.org/D461
This enables the optimization introduced by b8d938230143 for non-rebase cases.
Before, the match couldn't be narrowed if it was e.g. alwaysmatcher.
The logic is copied from fca0d99edf8e.
In chg's case, making modules lazily loaded could actually slow down things
since chg pre-imports them. Therefore disable demandimport if chg is being
used.
This is not done by setting `HGDEMANDIMPORT` chg client-side because that
has side-effects on child processes (hooks, etc).
Differential Revision: https://phab.mercurial-scm.org/D351
Wrapping text in templates for 'hg log --graph' can't be done very well,
because the template doesn't know how wide the graph drawing is. The edge
drawing function needs to know the number of lines in the template output, so
we need to also determine how wide that drawing would be before we call the
edgefn or evaluate the template.
This patch makes edgefn compute the graph width and pass it into the template
so that we can do something like this:
COLUMNS=10 hg log --graph --template "{fill(desc, termwidth - graphwidth)}"
@ a a a a
| a a a a
| a a a a
o a a a
|\ a a a
| | a a a
| | a a a
Using extensions to do this would be relatively complicated due to a lack of
hooks in this area of the code.
In the future it may make sense to have a more generic "textwidth" that tells
you how many columns you can expect to fill without causing the terminal to
wrap your output. I'm not sure there are other situations to motivate this yet,
or if it is entirely feasible.
Differential Revision: https://phab.mercurial-scm.org/D360
Update all calls to formatter.write first arguments to remove references to
precnode and use prednode consistently everywhere.
Differential Revision: https://phab.mercurial-scm.org/D414
Seems the code block misses `::`. This patch makes sure `[push]` and
`pushvars.server = true` are in two lines.
Differential Revision: https://phab.mercurial-scm.org/D411
In order to make this work, we have to wrap the io streams in a
TextIOWrapper so that __builtins__.input() can do unicode IO on Python
3. We can't just restore the original (unicode) sys.std* because we
might be running a cmdserver, and if we blindly restore sys.* to the
original values then we end up breaking the cmdserver. Sadly,
TextIOWrapper tries to close the underlying stream during its __del__,
so we have to make a sublcass to prevent that.
If you see errors like:
TypeError: a bytes-like object is required, not 'str'
On an input() or print() call on Python 3, the substitution of
sys.std* is probably the root cause.
A previous version of this change tried to put the bytesinput() method
in pycompat - it turns out we need to do some encoding handling, so we
have to be in a higher layer that's allowed to use
mercurial.encoding.encoding. As a result, this is in util for now,
with the TextIOWrapper subclass hiding in encoding.py. I'm not sure of
a better place for the time being.
Differential Revision: https://phab.mercurial-scm.org/D299
Rename the existing parameters for clarity.
These will, in subsequent patches, allow callers to redirect reads (of the
three sides of the merge) and writes (of the result) to the given contexts,
instead of using the filesystem.
While in most cases, the writes will go to a workingfilectx, this opens the
door for it to be a memfilectx in the case of an in-memory merge.
Repo will be necessary in a subsequent comit.
Differential Revision: https://phab.mercurial-scm.org/D372
When I wrote f33f2b5b4874 (commit: don't let failed commit with
--addremove update dirstate (issue5645), 2017-07-31), Durham's
422ca3501516 (rebase: use one dirstateguard for when using
rebase.singletransaction, 2017-07-20) had not yet landed, so I had to
write it in the old-fashioned way. Now that Durham's patch is in, we
can simplify by using a context manager.
Differential Revision: https://phab.mercurial-scm.org/D406
This is just a simple refactoring to make the next patch simpler. If
the dirstateguard constructor raises an exception, the finally-block
won't do anything anyway, so this is functionally equivalent (and
there is no except-block).
Differential Revision: https://phab.mercurial-scm.org/D405
This documentation is very helpful for any developer to understand what
copytracing is and what the function does. Since this is the main function of
doing copytracing, I have also included bits about copytracing in it.
This additions are picked from a doc by Stash@Fb. So thanks to him.
Differential Revision: https://phab.mercurial-scm.org/D409
This is probably only used in the 'unbundle' command, but the code
ended up being cleaner to make it generic and treat *all* httppostargs
with a non-args request body as though they were file-like in
nature. It also means we get test coverage more or less for free. A
previous version of this change didn't use io.BytesIO, and it was a
lot more complicated.
This also fixes a server-side bug, so anyone using httppostargs should
update all of their servers to this revision or later *before* this
gets to their clients, otherwise servers will hang trying to over-read
the POST body.
Differential Revision: https://phab.mercurial-scm.org/D231
To list up available compression types instead of
".. bundlecompressionmarker" in "hg help bundlespec" output, proxy
object "docobject" is used, because:
- current online help system requires that __doc__ of registered
object (maybe, function) is already well formatted in reST syntax
- bundletype() method of compressionengine classes is used to list up
available compression types, but
- __doc__ of bundletype() object (= "instancemethod") is read-only
On the other hand, hggettext requires original function object, in
order to get document location in source code.
Therefore, description of each compression types isn't yet
translatable. Even if translatable, translators should make much
effort to determine location of original texts in source code.
To get actual function information, this patch makes hggettext use
function object saved as "_origfunc", if it is available. This patch
also changes bundlecompressiontopics() side, in order to explain how
these changes work easily.
This patch is a part of preparations for making description of each
compression types translatable.
The status line in the crecord has the "space" status field which has variable
length depending on the length of the status label in the language of choice.
In English, the status labels are "space: deselect" and "space:select". The
"deselect" label is 2 glyphs longer. This makes the terminal output jump
around if the terminal width is just right so that the shorter label makes
the status line 1 line long, and the longer label makes it 2 lines long.
This patch formats the selected status into a fixed-width field. The field
width is the maximum of the lengths of the two possible labels, to account for
differing translations and label lengths. This should make the label behavior
uniform across localizations.
There does not seem to be a test for crecord, so I verified the change manually
with a local build of 'hg'.
The wirepeer class provides concrete implementations of peer interface
methods for calling wire protocol commands. It makes sense for this
class to inherit from the peer abstract base class. So we change
that.
Since httppeer and sshpeer have already been converted to the new
interface, peerrepository is no longer adding any value. So it has
been removed. httppeer and sshpeer have been updated to reflect the
loss of peerrepository and the inheritance of the abstract base
class in wirepeer.
The code changes in wirepeer are reordering of methods to group
by interface.
Some Python code in tests was updated to reflect changed APIs.
.. api::
peer.peerrepository has been removed. Use repository.peer abstract
base class to represent a peer repository.
Differential Revision: https://phab.mercurial-scm.org/D338
We need the same @property conversion of ui like we did for localpeer.
We renamed _capabilities() to capabilities() to satisfy the new
naming requirement.
However, since we're inheriting from wireproto.wirepeer which inherits
from peer.peerrepository and provides its own code accessing
_capabilities(), we need to keep the old alias around. This wonkiness
will disappear once wirepeer is cleaned up in subsequent commits.
We also implement methods for basepeer that are identical to the
defaults in peer.peerrepository in preparation for the removal of
peerrepository.
Differential Revision: https://phab.mercurial-scm.org/D336
We now have a formal abstract base class for peers. Let's
transition the peer classes in localrepo to it.
As part of the transition, we reorder methods so they are grouped
by interface and match the order they are defined in the interface.
We also had to change self.ui from an instance attribute to a
property to satisfy the @abstractproperty requirement.
As part of this change, we uncover the first "bug" as part of
enforcing interfaces: stream_out() wasn't implemented on localpeer!
This isn't technically a bug since the repo isn't advertising the
stream capability, so clients shouldn't be attempting to call it.
But I don't think there's a good reason why this is the case.
We implement a dummy method to satisfy the interface requriements.
We can make localpeer instances streamable as a future enhancement.
# no-check-commit
Differential Revision: https://phab.mercurial-scm.org/D335
These methods are part of the peer interface, are generic, and can
be implemented in terms of other members of the peer interface. So we
implement them on the peer base class as a convenience.
The implementation is essentially copied from peer.py. The code
in peer.py will eventually be deleted.
Differential Revision: https://phab.mercurial-scm.org/D334
There are a well-defined set of commands constituting the wire
protocol. Interaction with these and methods for calling them in
batches are exposed via methods on peer instances.
Let's formalize support for these features in abstract classes.
The command parts come from the existing wireproto.wirepeer class.
The batch methods come from peer.peerrepository.
Ample documentation has been added as part of defining the interfaces.
# no-check-commit
Differential Revision: https://phab.mercurial-scm.org/D333
There are various interfaces for interacting with repositories
and peers. They form a contract for how one should interact with
a repo or peer object.
The contracts today aren't very well-defined or enforced. There
have been several bugs over the years where peers or repo types
have forgotten to implement certain methods. In addition, the
inheritance of some classes is wonky. For example, localrepository
doesn't inherit from an interface and the god-object nature of
that class means the repository interface isn't well-defined. Other
repository types inherit from localrepository then stub out
methods that don't make sense (e.g. statichttprepository
re-defining locking methods to fail fast).
Not having well-defined interfaces makes implementing alternate
storage backends, wire protocol transports, and repository types
difficult because it isn't clear what exactly needs to be
implemented.
This patch starts the process of attempting to establish more
order to the type system around repositories and peers.
Our first patch starts with a problem space that already has a
partial solution: peers. The peer.peerrepository class already
somewhat defines a peer interface. But it is missing a few things
and the total interface isn't well-defined because it is combined
with wireproto.wirepeer.
Our newly-established basepeer class uses the abc module to
declare an abstract base class with the properties and methods that
a generic peer must implement.
We create a new class that inherits from it. This class will hold
our other future abstract base classes / interfaces so we can expose
a unified base class/interface.
We don't yet use the new interface because subsequent additions
will break existing code without some refactoring first.
A new module (repository.py) was created to hold the interfaces.
I could have put things in peer.py. However, I have plans to
eventually add interfaces to define repository and storage types.
These almost certainly require a new module. And I figured having
all the interfaces live in one module makes sense. So I created
repository.py to be that future home.
Differential Revision: https://phab.mercurial-scm.org/D332
a3022f57803b made `util.nogc` a no-op. It was to optimize PyPy. But it slows
down CPython if many objects (like 300k+) are created.
For example, running `hg log -r .` without extensions in `hg-committed` with
14k+ obsmarkers have the following times:
before | after
hg | chg | hg | chg
-----------------------------
1.262 | 0.860 | 1.077 | 0.619 (seconds, best of 20 runs)
Therefore let's re-enable nogc for CPython.
Differential Revision: https://phab.mercurial-scm.org/D402
troubled has been renamed into isunstable but troubled was calling unstable
instead. Fix the mistake.
Differential Revision: https://phab.mercurial-scm.org/D384
We have at least three types with a close() and a release() method
where the close() method is supposed to be called on success and the
release() method is supposed to be called last, whether successful or
not. Two of them (transaction and dirstateguard) already have
identical implementations of __enter__ and __exit__. Let's extract a
base class for this, so we reuse the code and so the third type
(transactionmanager) can also be used as a context manager.
Differential Revision: https://phab.mercurial-scm.org/D392
The transactionmanager() constructor just assigned a few variables and
cannot fail, so it's safe to move it inside the earlier try/except.
Differential Revision: https://phab.mercurial-scm.org/D391
Since 6f17bd68a306 (exchange: drop support for lock-based unbundling
(BC), 2017-08-06), there is no more remote locking.
Differential Revision: https://phab.mercurial-scm.org/D390
The variable has been used only within a single function since
5d683cc9670f (push: elevate phase transaction to cover entire
operation, 2014-11-21), so there's no need to keep it on the pushop
object.
Differential Revision: https://phab.mercurial-scm.org/D389
morestatus extension in fbext use to show more context about the state of the
repo like the repository is in a unfinished merge state, or a rebase is going
on, or histedit is going on, listing the files which need to be resolved and
also suggesting ways to handle the situation.
This patch moves the extension directly to core by plugging it into the
--verbose flag of the status command. So now if you are in any unfinished state
and you do hg status -v, it will show you details and help related to the state.
The extension in fbext also shows context about unfinished update state
which is not ported to core as that plug in hooks to update command which need
to be tackled somewhat differently.
The following configuration will turn the behaviour on by default
[commands]
status.verbose = 1
You can also skip considering some states like bisect as follows:
[commands]
status.skipstates=bisect
This patch also adds test for the feature.
.. feature::
``hg status -v`` can now show unfinished state. For example, when in
an unfinished rebase state, ``hg status -v`` might show::
# The repository is in an unfinished *rebase* state.
# No unresolved merge conflicts.
# To continue: hg rebase --continue
# To abort: hg rebase --abort
Differential Revision: https://phab.mercurial-scm.org/D219
Changeset aa97e972460f introduce more complex logic around
'bundleoperation.gettransaction'. In that process it turns the old "attribute"
into a proper method which breaks the code that detects the "transaction
availability".
The change was visible in 'test-acl.t', fixing this reverts the test changes.
Differential Revision: https://phab.mercurial-scm.org/D303
Peer types are supposed to conform to a formal interface defined by
peer.peerrepository and wireproto.wirepeer. Every "public" attribute on
*peer types makes it harder to understand what attributes are part
of the interface and what are instance specific.
This commit converts a number of "public" instance attributes and
methods on sshpeer to internal so they can't be confused to be part of
the peer API.
The URL-related instance attributes were introduced in 904c418bea16
in 2005. AFAICT most of them aren't used and could potentially be
removed. But I kept them around anyway.
I also reorded some code to make things slightly easier to read.
.. api::
Rename attributes on sshpeer to reflect peer API
Differential Revision: https://phab.mercurial-scm.org/D331
The last use of this API was removed in 3bcb9f9a4a63 in 2016. While
not formally deprecated, as of the last commit the code is no longer
explicitly tested. I think the new API has existed long enough for
people to transition to it.
I also have plans to more formalize the peer API and removing batch()
makes that work easier.
I'm not convinced the current client-side API around batching is
great. But it's the best we have at the moment.
.. api:: remove peer.batch()
Replace with peer.iterbatch().
Differential Revision: https://phab.mercurial-scm.org/D320
The remote batching code is difficult to read. Let's improve it.
As part of the refactor, the future returned by method calls on
batchiter() instances is now populated. However, you still need to
consume the results() generator for the future to be set. But at
least now we can stuff the future somewhere and not have to worry
about aligning method call order with result order since you can
use a future to hold the result.
Also as part of the change, we now verify that @batchable generators
yield exactly 2 values. In other words, we enforce their API.
The non-iter batcher has been unused since 3bcb9f9a4a63. And to my
surprise we had no explicit unit test coverage of it! test-batching.py
has been overhauled to use the iterating batcher.
Since the iterating batcher doesn't allow non-batchable method
calls nor local calls, tests have been updated to reflect reality.
The iterating batcher has been used for multiple releases apparently
without major issue. So this shouldn't cause alarm.
.. api::
@peer.batchable functions must now yield exactly 2 values
Differential Revision: https://phab.mercurial-scm.org/D319
@peer.batchable decorated generator functions have two forms:
yield value, None
and
yield args, future
yield value
These forms have been present since the decorator was introduced.
There are currently no in-repo consumers of the first form. So this
commit removes support for it.
Note that remoteiterbatcher.submit() asserts the 2nd form. And
3bcb9f9a4a63 removed the last user of remotebatcher, forcing everyone
to remoteiterbatcher. So anything relying on this in the wild would
have been broken since 3bcb9f9a4a63.
.. api::
@peer.batchable can no longer emit local values
Differential Revision: https://phab.mercurial-scm.org/D318
remoteiterbatcher (unlike remotebatcher) only supports batchable
commands. This claim can be validated by comparing their
implementations of submit() and noting how remoteiterbatcher assumes
the invoked method has a "batchable" attribute, which is set by
@peer.batchable.
remoteiterbatcher has a custom __getitem__ that was trying to
validate that only batchable methods are called. However, it was only
validating that the called method exists, not that it is batchable.
This wasn't a big deal since remoteiterbatcher.submit() would raise
an AttributeError attempting to `mtd.batchable(...)`.
Let's fix the check and convert it to ProgrammingError, which may
not have been around when this was originally implemented.
Differential Revision: https://phab.mercurial-scm.org/D317
This extracts charencode.c from parsers.c, which seems big enough for me
to hesitate to add new JSON functions. Still charencode.o is linked to
parsers.so to avoid duplication of binary codes.
As we changed the meaning of unstable between the old vocabulary and the new
one, we can't reuse the unstable method name at the risk of breaking
extensions calling unstable and getting a wrong result.
Instead rename troubled into isunstable so extensions will continue to work.
The renaming is done according to
https://www.mercurial-scm.org/wiki/CEDVocabulary.
Differential Revision: https://phab.mercurial-scm.org/D242
Rename troubles context method into instabilities.
Copy the old troubles method and add a deprecation warning. This way
extensions calling troubles will see the deprecation warning but will not
break due to new return values.
The renaming is done according to
https://www.mercurial-scm.org/wiki/CEDVocabulary.
Differential Revision: https://phab.mercurial-scm.org/D238
This moves manifest stripping to a separate function so implementations of the
manifest that don't support stripping can replace this function with a no-op.
I considered adding a strip api to the manifestlog, so other implementations
could make it a no-op there, but it seems like strip might be unique to the
revlog implementation, and therefore shouldn't be present on the generic api.
Differential Revision: https://phab.mercurial-scm.org/D292
This refactors broken linkrev collection such that manifest collection is in a
separate function. This allows extensions to replace the manifest collection
with a non-revlog oriented version.
I considered moving the collect changes function onto the manifestlog itself, so
it would be behind the abstraction, but since the store we're building doesn't
even have the concept of strip, embeding that concept in the manifestlog api
seemed odd.
Differential Revision: https://phab.mercurial-scm.org/D291
I'm not sure this is right, since this should either be bytes or str
to match what's going on in the revlog layer.
Differential Revision: https://phab.mercurial-scm.org/D271
I'm not sure if this is a realistic problem, but doing this avoids
some pretty awful test failures on Python 3, and it looks like it
should be harmless.
Differential Revision: https://phab.mercurial-scm.org/D274
Fixes some Python 3 regressions.
We don't use %d here because the part id is actually an
Optional[int]. It should always be initialized to a non-None value by
the time this code executes, but we shouldn't blindly depend on that
being the case.
Differential Revision: https://phab.mercurial-scm.org/D272
The changelog object can store recently added revisions in memory
until the transaction is committed. We don't want to lose those
changes even if repo.invalidate(clearfilecache=True), so let's skip
the changelog when it has such 'delayed' changes.
Differential Revision: https://phab.mercurial-scm.org/D152
Peer instances are supposed to conform to a well-defined API so
consumers can be agnostic about the underlying peer type.
To reinforce this, this commit renames a handful of instance
attributes on httpeer so they no longer have a "public" name.
Differential Revision: https://phab.mercurial-scm.org/D268
The previous changeset removed the last consumer of requirements. I'm
not sure when supportedformats became unused. But I couldn't find
any obvious instances where it is being used. It likely stems from
peers being derived from repository instances several years ago and
is a holdover from that day.
Differential Revision: https://phab.mercurial-scm.org/D266
As part of formalizing the peer interface, I audited for attribute
accesses for non-internal names to find API violations. This
uncovered the code changed in this commit.
localpeer.requirements is just an alias to the repo's requirements
attribute. So, change the code to get the data from the source
instead of relying on a one-off attribute in the localpeer type.
Differential Revision: https://phab.mercurial-scm.org/D265
Locking over the wire protocol and the "addchangegroup" wire
protocol command has been deprecated since f8e443eb02c9, which was
first part of Mercurial 0.9.1.
Support for handling these commands from sshserver was dropped in
93297d5f4df2 in 2015, effectively locking out pre 0.9.1 clients
from new servers.
However, client-side code for calling lock and addchangegroup is
still present in exchange.py and the various peer classes to
facilitate pushing to pre 0.9.1 servers.
The lock-based pushing mechanism is extremely brittle. 0.9.1 was
released in July 2006 and I highly doubt anyone is still running
such an ancient version of Mercurial on a server. I'm about to
refactor the peer API and I don't think it is worth keeping
support for this ancient protocol feature. So, this commit removes
client support for the lock-based pushing mechanism. This means
modern clients will no longer be able to push to pre 0.9.1 servers.
Differential Revision: https://phab.mercurial-scm.org/D264
This patch also adds extra loading entry for internal merge tools to
extensions.py, for similarity to other decorators defined in
registrar.py.
This patch uses "internalmerge" for decorator class name, instead of
original "internaltool", because the latter is too generic.
BTW, after this patch, 4-spaces indentation is added to the 1st line
of internal merge tool description docstring, and this may make
already translated entries in *.po fuzzy.
Even though this indentation is required for "definition list" in reST
syntax, absence of it has been overlooked, because help.makeitemsdoc()
forcibly inserts it at generation of online help.
But this forcible insertion causes formatting issue (I'll send another
patch series for this). Therefore, this additional indentation should
be reasonable.
Let "isignoreddir" function first check that supplied directory is itself
ignored before walking recursively into its content. Otherwise, the command is
awfully slow when one has an ignored directory with a lot of content.
Update and rephrase function docstring accordingly.
pushvars extension in fbext adds a --pushvars flag to push command using which
one send strings to server which becomes environment variables there prepended
with HG_USERVAR_. These variables can then be used to run hooks on the server.
The extension is moved directly to core and unbundling of the strings and
converting them to environment variables at server is disabled by default for
security reasons. One can turn that on by following config:
[push]
pushvars.server = true
This patch also adds the test for the extension.
Differential Revision: https://phab.mercurial-scm.org/D210
Rename bumped to phase-divergent in all external user-facing output. Only
update user-facing output for the moment, variables names, templates keyword
and potentially configuration would be done in later series.
The renaming is done according to
https://www.mercurial-scm.org/wiki/CEDVocabulary.
Differential Revision: https://phab.mercurial-scm.org/D216
Rename divergent to content-divergent in all external user-facing output. Only
update user-facing output for the moment, variables names, templates keyword
and potentially configuration would be done in later series.
The renaming is done according to
https://www.mercurial-scm.org/wiki/CEDVocabulary.
Differential Revision: https://phab.mercurial-scm.org/D215
This commit makes it so sparse treats passed paths as CWD-relative,
not repo-root-realive. This is a more intuitive behavior in my (and some
other FB people's) opinion.
This is breaking change however. My hope here is that since sparse is
experimental, it's ok to introduce BCs.
The reason (glob)s are needed in the test is this: in these two cases we
do not supply path together with slashes, but `os.path.join` adds them, which
means that under Windows they can be backslashes. To demonstrate this behavior,
one could remove the (glob)s and run `./run-tests.py test-sparse.t` from
MinGW's terminal on Windows.
Current logic is misleading (it says it drops only absolute paths, but
it actually drops all of them), not cross-platform (does not support Windows)
and IMO just wrong (as it should just error out if absolute paths are given).
This commit fixes it.
If the user sets color.mode=terminfo, and then runs in the shell inside of emacs
(so TERM=dumb), the previous behavior was that it would warn about no terminfo
entry for setab/setaf, and then warn about 'failed to set color mode to
terminfo'. The first warning is silenced by carrying 'formatted' through to
_terminfosetup, the second is silenced by using 'formatted' instead of
ui.formatted().
If --color=on (or ui.color=always) is specified, this will still warn, since the
formatted boolean is set to true in these cases.
Differential Revision: https://phab.mercurial-scm.org/D223
Rename unstable to orphan in all external user-facing output. Only update
user-facing output for the moment, variables names, templates keyword and
potentially configuration would be done in later series.
The renaming is done according to
https://www.mercurial-scm.org/wiki/CEDVocabulary.
Differential Revision: https://phab.mercurial-scm.org/D214
Rename trouble(s) to instability in all external user-facing output. Only
update user-facing output for the moment, variables names, templates keyword
and potentially configuration would be done in later series.
The renaming is done according to
https://www.mercurial-scm.org/wiki/CEDVocabulary.
Differential Revision: https://phab.mercurial-scm.org/D213
When a transaction is started, we must load the hookargs from the
bundleoperation object to the transaction so that they can be used in the
transaction. Also this patch makes sure no more hookargs are added to the
bundleoperation object once the transaction starts.
This is a part of porting fb extension bundle2hooks to core.
Differential Revision: https://phab.mercurial-scm.org/D209
There are extensions like pushrebase, pushvars which run hooks on a server
before taking the lock. Since the lock is not taken, transaction is not there,
so the hookargs can't be stored on the transaction. Adding hooksargs to bundle
operation object will help in running hooks before taking the lock.
This is a part of moving fb's extension bundle2hooks to core.
Differential Revision: https://phab.mercurial-scm.org/D208
This was previously landed as 4bc0c14fb501 but backed out in b63351f6a2 because
it broke hooks mid-rebase and caused conflict resolution data loss in the event
of unexpected exceptions. This new version adds the behavior back but behind a
config flag, since the performance improvement is notable in large repositories.
The old commit message was:
Recently we switched rebases to run the entire rebase inside a single
transaction, which dramatically improved the speed of rebases in repos with
large working copies. Let's also move the dirstate into a single dirstateguard
to get the same benefits. This let's us avoid serializing the dirstate after
each commit.
In a large repo, rebasing 27 commits is sped up by about 20%.
I believe the test changes are because us touching the dirstate gave the
transaction something to actually rollback.
(grafted from 9e3dc3a1638b9754b58a0cb26aaa75d868058109)
(grafted from 7d38b41d2266d9a02a15c64229fae0da5738dcec)
Differential Revision: https://phab.mercurial-scm.org/D135
This simplifies the method slightly. It does create a full list of
paths while doing so, but it's not a lot of data anyway (besides, I
would think references to strings are no larger than (references to?)
True).
mergestate.unresolved() is a generator, so it seems better for it to
rely on iteritems() than items(), although it also seems unlikely for
it to make a noticeable difference.
I don't know why applying an empty changegroup should be an error. It
seems harmless. I suspect the check was there to find code that
creates empty changegroups just because that would be wasteful. Let's
use develwarn() for that instead, so we catch any such cases that run
with our test runner, but we still allow others to generate empty
changegroups if they want to.
We have run into this check at Google once or twice and had to work
around it, but I'm changing this not so much because of that, but
because it seems like it shouldn't be an error.
I also changed the message slightly to be more modern ("changelog
group" -> "changegroup") and more generic ("received" -> "applied").
Applying an empty changegroup has been an error since the
beginning. The only exception was strip, which would allow to apply an
empty changegroup from the temporary bundle. However, the emptyok=True
option was only set for bundle1 bundles. In other words, temporary
bundle2 bundles would fail if they were empty.
Bundle2 has now been used enough that it seems safe to say that we
simply don't create bundle2 bundles with empty changegroups. That also
suggests that we never create bundle1 bundles with empty changegroups
(i.e. empty bundle1 bundles, since bundle1 is just a changegroup),
because, AFAICT, the code leading up to the application of the bundle
is the same for bundle1 and bundle2.
Therefore, let's stop passing emptyok=True, so we more clearly get the
same behavior for bundle1 and bundle2.
The "patterns"/"include" in "patternspat"/"includepat" is redundant,
so drop it. Also a "_" prefix since it's "private".
Inline the "pm"/"im" variables.
The code was refactored during the move to be more procedural
instead of using string formatting. This has the benefit of not
writing empty sections, which changed tests.
