Commit Graph

33207 Commits

Author SHA1 Message Date
Boris Feld
87cd8ac872 phases: detect when boundaries has been actually retracted
It is useful to detect noop and avoid expensive operations in this case.

We return the information to inform the caller of a possible update. Top level
function might need to react to the phase update (eg: invalidating some
caches, tracking phase change).
2017-07-12 23:15:09 +02:00
Boris Feld
6ee0c66cd0 phases: rework phase movement code in 'cg.apply' to use 'registernew'
We rework the code to call 'registernew' before any other phase advancement.
This make 'changegroup.apply' register correct phase movement for the added
and bundled nodes.
2017-07-11 01:17:36 +02:00
Boris Feld
1d64a7418f convert: use the new 'phase.registernew' function 2017-07-11 00:59:23 +02:00
Boris Feld
9d590cdd8f localrepo: use the 'registernew' function to set the phase of new commit 2017-07-11 01:05:27 +02:00
Boris Feld
a6e00b1e49 phases: add a 'registernew' method to set new phases
This new function will be used by code that adds new changesets. It ajusts the
phase boundary to make sure added changesets are at least in their target
phase (they end up in an higher phase if their parents are in a higher phase).

Having a dedicated function also simplify the phases tracking. All the new
nodes are passed as argument, so we know that all of them needs to have their
new phase registered. We also know that no other nodes will be affected, so no
extra computation are needed.

This function differ from 'retractboundary' where some nodes might change
phase while some other might not. It can also affect nodes not passed as
parameters.

These simplification also apply to the computation itself. For now we use
'_retractboundary' there by convenience, but we may introduces simpler code
later.

While registering new revisions, we still need to check the actual phases of
the added node because it might be higher than the target phase (eg: target is
draft but parent is secret).

We will migrate users over the next changesets.
2017-07-11 03:47:25 +02:00
Boris Feld
3de612f757 phases: extract the core of boundary retraction in '_retractboundary'
At the moment the 'retractboundary' function is called for multiple reasons:

First, actually retracting boundaries. There are only two cases for theses:
'hg phase --force' and 'hg qimport'. This will need extra graph computation to
retrieve the phase changes.

Second, setting the phases of newly added changesets. In this case we already
know all the affected nodes and we just needs to register different
information (old phase is None).

Third, when reducing the set of roots when advancing phase. The phase are
already properly tracked so we do not needs anything else in this case.

To deal with this difference in phase tracking, we extract the core logic into
a private method that all three cases can use.
2017-07-10 23:50:16 +02:00
Boris Feld
794ac05dda phases: track phase movements in 'advanceboundary'
Makes advanceboundary record the phase movement of affected revisions in
tr.changes['phases'].

The tracking is not usable yet because the 'retractboundary' function can also
affect phases.

We'll improve that in the coming changesets.
2017-07-11 02:39:52 +02:00
Boris Feld
e0ae9be376 phases: extract the intermediate set of affected revs
When advancing phases, we compute the new roots for the phases above. During
this process, we need to compute all the revisions that change phases (to the
new target phases). Extract these revisions into a separate variable. This
will be useful to record the phase changes in the transaction.
2017-07-10 22:18:41 +02:00
Boris Feld
7dd5b39982 phase: put retractboundary out of the loop in advanceboundary
It seems that we were calling retractboundary for each phases to process.
Putting the retractboundary out of the loop reduce the number of calls,
helping tracking the phases changes.
2017-07-10 22:22:42 +02:00
Martin von Zweigbergk
2ff08ff937 match: make unionmatcher a proper matcher
unionmatcher is currently used where only a limited subset of its
functions will be called. Specifically, visitdir() is never
called. The next patch will pass it to dirstate.walk() where it will
matter that visitdir() is correctly implemented, so let's fix
that. Also add the explicitdir etc that will also be assumed by
dirstate.walk() to exist on a matcher.

Differential Revision: https://phab.mercurial-scm.org/D58
2017-07-11 10:46:10 -07:00
Martin von Zweigbergk
3c2e121c63 match: write forceincludematcher using unionmatcher
The forceincludematcher is simply a unionmatcher of a includematcher
(matching paths recursively) with the given matcher. Since the
forceincludematcher is only used by sparse, move it there.

I don't have a good sparse repo setup to test performance impact on.

Differential Revision: https://phab.mercurial-scm.org/D57
2017-07-07 14:39:59 -07:00
Martin von Zweigbergk
cce08d0996 histedit: extract InterventionRequired transaction handling to utils
rebase will have similar logic, so let's extract it. Besides, it makes
the histedit code more readable.

We may want to parametrize acceptintervention() by the exception(s)
that should result in transaction close.

Differential Revision: https://phab.mercurial-scm.org/D66
2017-07-12 13:57:03 -07:00
Martin von Zweigbergk
7259b500cc histedit: create transaction outside of try
Just a little refactoring to simplify the next patch.

Differential Revision: https://phab.mercurial-scm.org/D65
2017-07-12 13:17:49 -07:00
Martin von Zweigbergk
018f2d955f histedit: remove transaction from state object
The transaction is only used within a single function, so no need to
store it on the state object.

Differential Revision: https://phab.mercurial-scm.org/D64
2017-07-12 11:18:02 -07:00
Jun Wu
1823368531 phabricator: verify local tags before trusting them
Previously we trust local tags blindly and that could cause wrong
Differential Revision to be updated, when people switch between Phabricator
instances.

This patch adds verification logic to detect such issue and remove
problematic tags. For example, a tag "D19" was on node "X", the code will
fetch all diffs attached to D19, and check if nodes server-side overlaps
with nodes in precursors. If they do not overlap, create a new Differential
Revision.

Test Plan:
Use a test Phabricator instance, send patches using `hg phabsend`, then
change the local tag manually to a wrong Differential Revision number.
Amend the patch and send again. Make sure the tag gets ignored and deleted.

Differential Revision: https://phab.mercurial-scm.org/D36
2017-07-11 08:17:29 -07:00
Jun Wu
44e64fc8ff phabricator: finding old nodes in batch
This allows us to do extra sanity checks using batch APIs to prevent
updating a wrong revision, which could happen when people switch Phabricator
instances and having stale tags living in the repo.

Differential Revision: https://phab.mercurial-scm.org/D34
2017-07-10 13:50:50 -07:00
Jun Wu
09b591871f phabricator: respect metadata sent by arc
Previously we only respect hg:meta sent by phabsend. This patch makes it
respect local:commits sent by arc as well. This avoids issues where phabread
could lose the author information.

Test Plan:
Commit using a customized user, send the patch using arc to a test
Phabricator instance, and then read the patch using phabread. Make sure it
preserves the user information.

Differential Revision: https://phab.mercurial-scm.org/D33
2017-07-10 22:37:33 -07:00
Adam Simpkins
2a467faccd dirstate: update backup functions to take full backup filename
Update the dirstate functions so that the caller supplies the full backup
filename rather than just a prefix and suffix.

The localrepo code was already hard-coding the fact that the backup name must
be (exactly prefix + "dirstate" + suffix): it relied on this in _journalfiles()
and undofiles().  Making the caller responsible for specifying the full backup
name removes the need for the localrepo code to assume that dirstate._filename
is always "dirstate".

Differential Revision: https://phab.mercurial-scm.org/D68
2017-07-12 15:24:07 -07:00
Martin von Zweigbergk
dde7459563 util: remove unused ctxmanager
This was meant as a substitute for Python's "with" with multiple
context managers before we moved to Python 2.7. We're now on 2.7, so
we should have no reason to keep ctxmanager. "hg grep --all
ctxmanager" says that it was never used anyway.

Differential Revision: https://phab.mercurial-scm.org/D73
2017-07-13 09:51:50 -07:00
Jun Wu
c41f5ca68e codemod: simplify nested withs
This is the result of running:

  python codemod_nestedwith.py **/*.py

where codemod_nestedwith.py looks like this:

#!/usr/bin/env python
# codemod_nestedwith.py - codemod tool to rewrite nested with
#
# Copyright 2017 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
from __future__ import absolute_import, print_function

import sys

import redbaron

def readpath(path):
    with open(path) as f:
        return f.read()

def writepath(path, content):
    with open(path, 'w') as f:
        f.write(content)

def main(argv):
    if not argv:
        print('Usage: codemod_nestedwith.py FILES')

    for i, path in enumerate(argv):
        print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
        changed = False
        red = redbaron.RedBaron(readpath(path))
        processed = set()
        for node in red.find_all('with'):
            if node in processed or node.type != 'with':
                continue
            top = node
            child = top[0]
            while True:
                if len(top) > 1 or child.type != 'with':
                    break
                # estimate line length after merging two "with"s
                new = '%swith %s:' % (top.indentation, top.contexts.dumps())
                new += ', %s' % child.contexts.dumps()
                # only do the rewrite if the end result is within 80 chars
                if len(new) > 80:
                    break
                processed.add(child)
                top.contexts.extend(child.contexts)
                top.value = child.value
                top.value.decrease_indentation(4)
                child = child[0]
                changed = True
        if changed:
            print('updating %s' % path)
            writepath(path, red.dumps())

if __name__ == "__main__":
    sys.exit(main(sys.argv[1:]))


Differential Revision: https://phab.mercurial-scm.org/D77
2017-07-13 18:31:35 -07:00
Boris Feld
ca27090453 reposvfs: add a ward to check if locks are properly taken
we wrap 'repo.svfs.audit' to check for the store lock when accessing file in
'.hg/store' for writing. This caught a couple of instance where the transaction
was released after the lock, we should probably have a dedicated checker for
that case.
2016-08-08 18:14:42 +02:00
Boris Feld
c9fe43d98b repovfs: add a ward to check if locks are properly taken
When the appropriate developer warnings are enabled, We wrap 'repo.vfs.audit' to
check for locks when accessing file in '.hg' for writing. Another changeset will
add a 'ward' for the store vfs (svfs).

This check system has caught a handful of locking issues that have been fixed
in previous series (mostly in 4.0). I expect another batch to be caught in third
party extensions.

We introduce two real exceptions from extensions 'blackbox.log' (because a lot of
read-only operations add entry to it), and 'last-email.txt' (because 'hg email'
is currently a read only operation and there is value to keep it this way).

In addition we are currently allowing bisect to operate outside of the lock
because the current code is a bit hard to get properly locked for now. Multiple
clean up have been made but there is still a couple of them to do and the freeze
is coming.
2017-07-11 12:38:17 +02:00
Boris Feld
71ca1ed413 vfs: allow to pass more argument to audit
We want to be able to do more precise check when auditing a path depending of
the intend of the file access (eg read versus write). So we now pass the 'mode'
value to 'audit' and update the audit function to accept them.

This will be put to use in the next changeset.
2017-07-11 12:27:58 +02:00
Martin Tietze
36d2e1297d zsh_completion: add '--partial' flag to completions for 'import' (issue5618)
When using the 'import' command running ZSH together with its
Mercurial-specific completions the flag '--partial' (introduced in Mercurial
3.1, 2014-08-01) is not offered as completion option.

This patch adds it to the list of completions for the 'import' command.
2017-07-12 10:11:02 +02:00
Boris Feld
7b61f9660d configitems: register the 'bugzilla.password' config 2017-07-07 10:04:05 +02:00
Boris Feld
c06cce9036 configitems: register the 'bugzilla.mocklog' config 2017-07-07 10:04:03 +02:00
Matt Harbison
fd4fd16173 test-amend: fix HGEDITOR shell script to run on Windows
Windows doesn't know how to launch *.sh directly.  This workaround is used in
several other tests.
2017-07-13 22:57:11 -04:00
Matt Harbison
8b2afcc831 test-paths: mask out JSON specific path differences for Windows
The backslashes in the local paths were being escaped with another backslash,
and $TESTTMP doesn't match against the double backslashed path.  This doesn't
happen without the 'json' filter.
2017-07-13 22:47:35 -04:00
FUJIWARA Katsunori
aa294c443a tests: categorize entries in blacklist for fsmonitor for future reference 2017-07-13 03:17:59 +09:00
FUJIWARA Katsunori
256361464a tests: avoid tests related to directory permission if running with fsmonitor 2017-07-13 03:17:59 +09:00
FUJIWARA Katsunori
c0a5085319 tests: add extra output for fsmonitor at checking under .hg 2017-07-13 03:17:58 +09:00
FUJIWARA Katsunori
2f38cb5170 tests: add extra output for fsmonitor at showconfig 2017-07-13 03:17:58 +09:00
FUJIWARA Katsunori
8f3aa7a7e3 tests: take extra care for fsmonitor at enabling incompatible extension
This is worthwhile, because these tests don't focus mainly on using
incompatible extensions itself.
2017-07-13 03:17:58 +09:00
FUJIWARA Katsunori
8d7e2a7179 tests: remove meaningless entries from blacklist for fsmonitor
IMHO, these tests were listed up in blacklist at f82d98daa0de, because
fsmonitor extension wasn't correctly installed before actual testing
at that revision (this issue was fixed by 5b36c6365794).
2017-07-13 03:17:57 +09:00
FUJIWARA Katsunori
d8d2132e21 tests: remove meaningless comment from blacklist for fsmonitor
These tests can run successfully since 5af769016577.
2017-07-13 03:17:57 +09:00
Matt Harbison
3fbb2994cb test-https: stabilize for Windows
A leading '/' without a following drive letter is resolved to the location of
the MSYS installation.  These could be collapsed into a single line of output,
but this seems more self documenting.
2017-07-11 22:06:57 -04:00
Phil Cohen
92de40e6df tagmerge: use workingfilectx to write merged tags
This function already does an excellent job of reading from context objects;
we simply need to change the single write call to eliminate all uses of the
wvfs.

As with past changes, the effect should be a no-op but opens the door to
in-memory merge later by using different context objects.
2017-07-11 16:48:15 -07:00
Matthieu Laneuville
91e78ef185 run-tests: add color to output if pygments is available
The output of run-tests has no formatting by default, which hampers readability.
This patch colors the diff output when pygments is available. To avoid coloring
even when pygments is available, use --color never.
2017-06-29 20:45:12 +09:00
Matt Harbison
dbfcbcfa2b win32: work around a WinError problem handling HRESULT types
I ran into this ctypes bug while working with the Crypto API.  While this could
be an issue with any Win32 API in theory, the handful of things that we call are
older functions that are unlikely to return COM errors, so I didn't retrofit
this everywhere.
2017-03-30 00:33:00 -04:00
Jun Wu
6fb604b0fb test-check-pyflakes: do not leave test.py in the repo
Differential Revision: https://phab.mercurial-scm.org/D67
2017-07-12 15:27:56 -07:00
Yuya Nishihara
9955f11c5c revset: add experimental ancestors/descendants relation subscript
The relation name is 'generations' now, which may be changed in future.
2017-07-08 13:15:17 +09:00
Yuya Nishihara
ad66ada8bf revset: add experimental relation and subscript operators
The proposed syntax [1] was originally 'set{n rel}', but it seemed slightly
confusing if template is involved. On the other hand, we want to keep 'set[n]'
for future extension. So this patch introduces 'set#rel[n]' ternary operator.
I chose '#' just because it looks like applying an attribute.

This also adds stubs for 'set[n]' and 'set#rel' operators since these syntax
elements are fundamental for constructing 'set#rel[n]'.

 [1]: https://www.mercurial-scm.org/wiki/RevsetOperatorPlan#ideas_from_mpm
2017-07-08 13:07:59 +09:00
Yuya Nishihara
f742cb36a6 revset: do not compute weight for integer literal argument
In x^n and x~n, n isn't a set expression. There's no need to optimize the
right-hand side.
2017-07-08 12:49:46 +09:00
Yuya Nishihara
963c78d353 templatekw: export ui.paths as {peerpaths}
It's sometimes useful to show hyperlinks in log output.

  "{get(peerpaths, "default")}/rev/{node}"

Since each path may have sub options, "{peerpaths}" is structured as a dict
of dicts, but the inner dict is rendered as if it were a string URL. The
implementation is ad-hoc, so there are some weird behaviors described in
the test. We might need to introduce a proper way of handling a hybrid
scalar object.

This patch adds _hybrid.__getitem__() so d['path']['url'] works.

The keyword is named as "peerpaths" since "paths" seemed too generic in
log context.
2017-07-13 00:35:54 +09:00
Yuya Nishihara
17207a3d76 summary: fix type of empty unresolved list
It was okay because tested as a boolean prior to calling len(), but looked
incorrect.
2017-07-07 23:13:04 +09:00
Yuya Nishihara
12efb6fc0b vfs: rename auditvfs to proxyvfs
Since we've removed mustaudit property, auditvfs has no auditing business.
It's just a utility class for vfs wrappers.
2017-07-07 23:40:00 +09:00
Yuya Nishihara
027352b7a8 streamclone: comment why path auditing is disabled in generatev1()
Copied from 8809f5acb29a. I wasn't sure whether it's for optimization or
suppressing unwanted error.
2017-07-07 23:19:31 +09:00
Yuya Nishihara
954fcd3b6c streamclone: close large revlog files explicitly in generatev1() 2017-07-07 23:25:16 +09:00
Boris Feld
015cfc156e bundle2: no longer use 'retractboundary' in updatephases
The new 'phase-heads' forced all added node to secret before advancing the
boundary to work around the fact changesets were added as draft by default.
This is no longer necessary since the changegroup part can now use the
'targetphase' parameter.

Not doing this retract boundary call has a couple of advantages:

* This makes implementing phases change tracking in the transaction much
  simpler since retract boundary can become a rare case.

* Bundling secret changesets is not the norm. Exchange never does that and
  even for strip, the use-case is not common.Skipping the retract boundary
  will avoid useless work here.

* Sending phase update on push can be simplified since we can rely on the
  behavior of 'cg.apply' for most of it.
  This means less phases update send for example.

* We no longer needs to track and use the addednodes during unbundling. This
  make it possible to have multiple 'changegroup' and 'phase-heads' parts in the
  same bundle without them interfering with each others.

The new part has not been part of any release yet so we do not offer backward
compatibility yet. It is important to update this semantic before the 4.3
freeze happens.
2017-07-11 05:06:01 +02:00
Boris Feld
20ea8cb94b bundle2: automatically add 'targetphase' parameter in writenewbundle
If we are bundling secret changeset and the bundle will contain phase, we
request the changegroup to be applied as secret.

It will be useful for next patch as we are now sure that secrets changesets
are applied as secret and not applied as draft then forced to secret.
2017-07-11 05:12:03 +02:00