2019-11-01 22:33:28 +03:00
|
|
|
# Portions Copyright (c) Facebook, Inc. and its affiliates.
|
|
|
|
#
|
|
|
|
# This software may be used and distributed according to the terms of the
|
|
|
|
# GNU General Public License version 2.
|
|
|
|
|
2011-05-02 21:21:30 +04:00
|
|
|
# setdiscovery.py - improved discovery of common nodeset for mercurial
|
|
|
|
#
|
|
|
|
# Copyright 2010 Benoit Boissinot <bboissin@gmail.com>
|
|
|
|
# and Peter Arrenbrecht <peter@arrenbrecht.ch>
|
|
|
|
#
|
|
|
|
# This software may be used and distributed according to the terms of the
|
|
|
|
# GNU General Public License version 2 or any later version.
|
2014-03-06 15:37:28 +04:00
|
|
|
"""
|
|
|
|
Algorithm works in the following way. You have two repository: local and
|
|
|
|
remote. They both contains a DAG of changelists.
|
|
|
|
|
|
|
|
The goal of the discovery protocol is to find one set of node *common*,
|
|
|
|
the set of nodes shared by local and remote.
|
|
|
|
|
|
|
|
One of the issue with the original protocol was latency, it could
|
|
|
|
potentially require lots of roundtrips to discover that the local repo was a
|
|
|
|
subset of remote (which is a very common case, you usually have few changes
|
|
|
|
compared to upstream, while upstream probably had lots of development).
|
|
|
|
|
|
|
|
The new protocol only requires one interface for the remote repo: `known()`,
|
|
|
|
which given a set of changelists tells you if they are present in the DAG.
|
|
|
|
|
|
|
|
The algorithm then works as follow:
|
|
|
|
|
|
|
|
- We will be using three sets, `common`, `missing`, `unknown`. Originally
|
|
|
|
all nodes are in `unknown`.
|
|
|
|
- Take a sample from `unknown`, call `remote.known(sample)`
|
|
|
|
- For each node that remote knows, move it and all its ancestors to `common`
|
|
|
|
- For each node that remote doesn't know, move it and all its descendants
|
|
|
|
to `missing`
|
|
|
|
- Iterate until `unknown` is empty
|
|
|
|
|
|
|
|
There are a couple optimizations, first is instead of starting with a random
|
|
|
|
sample of missing, start by sending all heads, in the case where the local
|
|
|
|
repo is a subset, you computed the answer in one round trip.
|
|
|
|
|
|
|
|
Then you can do something similar to the bisecting strategy used when
|
|
|
|
finding faulty changesets. Instead of random samples, you can try picking
|
|
|
|
nodes that will maximize the number of nodes that will be
|
|
|
|
classified with it (since all ancestors or descendants will be marked as well).
|
|
|
|
"""
|
2011-05-02 21:21:30 +04:00
|
|
|
|
2015-08-09 05:53:25 +03:00
|
|
|
from __future__ import absolute_import
|
|
|
|
|
2015-05-16 21:28:04 +03:00
|
|
|
import collections
|
2013-11-07 01:48:06 +04:00
|
|
|
import random
|
2015-08-09 05:53:25 +03:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
from . import dagutil, error, progress, util
|
2015-08-09 05:53:25 +03:00
|
|
|
from .i18n import _
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
from .node import nullid, nullrev
|
|
|
|
|
2011-05-02 21:21:30 +04:00
|
|
|
|
2015-01-07 21:32:17 +03:00
|
|
|
def _updatesample(dag, nodes, sample, quicksamplesize=0):
|
2015-01-07 04:02:32 +03:00
|
|
|
"""update an existing sample to match the expected size
|
|
|
|
|
|
|
|
The sample is updated with nodes exponentially distant from each head of the
|
|
|
|
<nodes> set. (H~1, H~2, H~4, H~8, etc).
|
|
|
|
|
|
|
|
If a target size is specified, the sampling will stop once this size is
|
|
|
|
reached. Otherwise sampling will happen until roots of the <nodes> set are
|
|
|
|
reached.
|
|
|
|
|
|
|
|
:dag: a dag object from dagutil
|
|
|
|
:nodes: set of nodes we want to discover (if None, assume the whole dag)
|
|
|
|
:sample: a sample to update
|
|
|
|
:quicksamplesize: optional target size of the sample"""
|
2011-05-02 21:21:30 +04:00
|
|
|
# if nodes is empty we scan the entire graph
|
|
|
|
if nodes:
|
|
|
|
heads = dag.headsetofconnecteds(nodes)
|
|
|
|
else:
|
|
|
|
heads = dag.heads()
|
|
|
|
dist = {}
|
2015-05-16 21:28:04 +03:00
|
|
|
visit = collections.deque(heads)
|
2011-05-02 21:21:30 +04:00
|
|
|
seen = set()
|
|
|
|
factor = 1
|
|
|
|
while visit:
|
|
|
|
curr = visit.popleft()
|
|
|
|
if curr in seen:
|
|
|
|
continue
|
|
|
|
d = dist.setdefault(curr, 1)
|
|
|
|
if d > factor:
|
|
|
|
factor *= 2
|
|
|
|
if d == factor:
|
2015-01-07 21:32:17 +03:00
|
|
|
sample.add(curr)
|
|
|
|
if quicksamplesize and (len(sample) >= quicksamplesize):
|
|
|
|
return
|
2011-05-02 21:21:30 +04:00
|
|
|
seen.add(curr)
|
|
|
|
for p in dag.parents(curr):
|
|
|
|
if not nodes or p in nodes:
|
|
|
|
dist.setdefault(p, d + 1)
|
|
|
|
visit.append(p)
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2015-01-07 03:32:23 +03:00
|
|
|
def _takequicksample(dag, nodes, size):
|
2015-01-08 07:44:20 +03:00
|
|
|
"""takes a quick sample of size <size>
|
|
|
|
|
|
|
|
It is meant for initial sampling and focuses on querying heads and close
|
|
|
|
ancestors of heads.
|
|
|
|
|
|
|
|
:dag: a dag object
|
|
|
|
:nodes: set of nodes to discover
|
|
|
|
:size: the maximum size of the sample"""
|
2015-01-07 04:07:44 +03:00
|
|
|
sample = dag.headsetofconnecteds(nodes)
|
|
|
|
if size <= len(sample):
|
|
|
|
return _limitsample(sample, size)
|
2015-01-07 21:32:17 +03:00
|
|
|
_updatesample(dag, None, sample, quicksamplesize=size)
|
2011-05-02 21:21:30 +04:00
|
|
|
return sample
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2011-05-02 21:21:30 +04:00
|
|
|
def _takefullsample(dag, nodes, size):
|
2015-01-07 21:32:17 +03:00
|
|
|
sample = dag.headsetofconnecteds(nodes)
|
2011-05-02 21:21:30 +04:00
|
|
|
# update from heads
|
2015-01-07 21:32:17 +03:00
|
|
|
_updatesample(dag, nodes, sample)
|
2011-05-02 21:21:30 +04:00
|
|
|
# update from roots
|
2015-01-07 21:32:17 +03:00
|
|
|
_updatesample(dag.inverse(), nodes, sample)
|
2011-05-02 21:21:30 +04:00
|
|
|
assert sample
|
2015-01-07 23:09:51 +03:00
|
|
|
sample = _limitsample(sample, size)
|
|
|
|
if len(sample) < size:
|
|
|
|
more = size - len(sample)
|
|
|
|
sample.update(random.sample(list(nodes - sample), more))
|
2011-05-02 21:21:30 +04:00
|
|
|
return sample
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2014-10-27 19:40:32 +03:00
|
|
|
def _limitsample(sample, desiredlen):
|
|
|
|
"""return a random subset of sample of at most desiredlen item"""
|
|
|
|
if len(sample) > desiredlen:
|
|
|
|
sample = set(random.sample(sample, desiredlen))
|
|
|
|
return sample
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
|
2019-06-04 01:09:21 +03:00
|
|
|
def fastdiscovery(ui, local, remote):
|
|
|
|
# The normal findcommonheads implementation tries to find the exact boundary
|
|
|
|
# between what the client has and what the server has. But normally we
|
|
|
|
# have pretty good knowledge about what local commits already exist on the
|
|
|
|
# server, so we can short circuit all the discovery logic by just assuming
|
|
|
|
# the current public heads are representative of what's on the server. In the
|
|
|
|
# worst case the data might be slightly out of sync and the server sends us
|
|
|
|
# more data than necessary, but this should be rare.
|
|
|
|
cl = local.changelog
|
|
|
|
|
|
|
|
publicheads = []
|
|
|
|
# That should be equivalent to "heads(public())" but much faster
|
|
|
|
revs = list(local.revs("head() & public() + parents(roots(draft()))"))
|
|
|
|
|
|
|
|
for r in revs:
|
|
|
|
publicheads.append(local[r].node())
|
|
|
|
|
|
|
|
bookmarks = ui.configlist("discovery", "knownserverbookmarks")
|
|
|
|
knownbookmarksvalues = []
|
|
|
|
for book in bookmarks:
|
|
|
|
if book in local:
|
|
|
|
knownbookmarksvalues.append(local[book].node())
|
|
|
|
|
|
|
|
# If we have no remotenames, fallback to normal discovery.
|
|
|
|
if not publicheads:
|
|
|
|
return None
|
|
|
|
|
|
|
|
publicheads = set(publicheads)
|
|
|
|
|
|
|
|
# Check which remote nodes still exist on the server
|
|
|
|
ui.status(_("searching for changes\n"))
|
|
|
|
batch = remote.iterbatch()
|
|
|
|
batch.heads()
|
|
|
|
batch.known(knownbookmarksvalues)
|
|
|
|
batch.known(publicheads)
|
|
|
|
batch.submit()
|
|
|
|
srvheadhashes, yesnoknownbookmarks, yesnopublicheads = batch.results()
|
|
|
|
|
|
|
|
if knownbookmarksvalues and not any(yesnoknownbookmarks):
|
2019-10-07 13:29:03 +03:00
|
|
|
ui.status(_("No known server bookmarks\n"))
|
2019-06-04 01:09:21 +03:00
|
|
|
# Server doesn't known any remote bookmark. That's odd and it's better
|
|
|
|
# to fallback to normal discovery process. Otherwise we might request
|
|
|
|
# too many commits from the server
|
|
|
|
return None
|
|
|
|
|
|
|
|
common = list(n for i, n in enumerate(publicheads) if yesnopublicheads[i])
|
|
|
|
common.extend(
|
|
|
|
(n for i, n in enumerate(knownbookmarksvalues) if yesnoknownbookmarks[i])
|
|
|
|
)
|
|
|
|
|
|
|
|
# If we don't know of any server commits, fall back to legacy discovery
|
|
|
|
if not common:
|
|
|
|
# If this path is hit, it will print "searching for changes" twice,
|
|
|
|
# which is weird. This should be very rare though, since it only happens
|
|
|
|
# if the client has remote names, but none of those names exist on the
|
|
|
|
# server (i.e. the server has been completely replaced, or stripped).
|
|
|
|
ui.status(
|
|
|
|
_(
|
|
|
|
"server has changed since last pull - falling back to the "
|
|
|
|
"default search strategy\n"
|
|
|
|
)
|
|
|
|
)
|
|
|
|
return None
|
|
|
|
|
|
|
|
ui.debug("using fastdiscovery\n")
|
|
|
|
if cl.tip() == nullid:
|
|
|
|
if srvheadhashes != [nullid]:
|
|
|
|
return [nullid], True, srvheadhashes
|
|
|
|
return ([nullid], False, [])
|
|
|
|
|
|
|
|
# early exit if we know all the specified remote heads already
|
|
|
|
clcontains = cl.__contains__
|
|
|
|
srvheads = list(n for n in srvheadhashes if clcontains(n))
|
|
|
|
if len(srvheads) == len(srvheadhashes):
|
|
|
|
ui.debug("all remote heads known locally\n")
|
|
|
|
return (srvheadhashes, False, srvheadhashes)
|
|
|
|
|
|
|
|
return (common, True, srvheadhashes)
|
|
|
|
|
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
def findcommonheads(
|
|
|
|
ui,
|
|
|
|
local,
|
|
|
|
remote,
|
|
|
|
initialsamplesize=100,
|
|
|
|
fullsamplesize=200,
|
|
|
|
abortwhenunrelated=True,
|
|
|
|
ancestorsof=None,
|
2019-06-04 01:09:21 +03:00
|
|
|
needlargestcommonset=True,
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
):
|
|
|
|
"""Return a tuple (common, anyincoming, remoteheads) used to identify
|
2011-05-05 19:21:37 +04:00
|
|
|
missing nodes from or in remote.
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
"""
|
2019-06-04 01:09:21 +03:00
|
|
|
|
|
|
|
# fastdiscovery might returns *some* common set, but it might not be
|
|
|
|
# necessary the largest common set. In some cases (e.g. during `hg push`)
|
|
|
|
# we actually want largest common set
|
|
|
|
if ui.configbool("discovery", "fastdiscovery") and not needlargestcommonset:
|
|
|
|
res = fastdiscovery(ui, local, remote)
|
|
|
|
if res is not None:
|
|
|
|
return res
|
|
|
|
|
2017-06-07 12:29:39 +03:00
|
|
|
start = util.timer()
|
|
|
|
|
2011-05-02 21:21:30 +04:00
|
|
|
roundtrips = 0
|
|
|
|
cl = local.changelog
|
2017-12-07 00:44:51 +03:00
|
|
|
localsubset = None
|
|
|
|
if ancestorsof is not None:
|
|
|
|
rev = local.changelog.rev
|
|
|
|
localsubset = [rev(n) for n in ancestorsof]
|
|
|
|
dag = dagutil.revlogdag(cl, localsubset=localsubset)
|
2011-05-02 21:21:30 +04:00
|
|
|
|
2011-06-15 00:58:00 +04:00
|
|
|
# early exit if we know all the specified remote heads already
|
2011-05-02 21:21:30 +04:00
|
|
|
ui.debug("query 1; heads\n")
|
|
|
|
roundtrips += 1
|
2011-06-15 00:58:00 +04:00
|
|
|
ownheads = dag.heads()
|
2014-10-27 19:52:33 +03:00
|
|
|
sample = _limitsample(ownheads, initialsamplesize)
|
2014-11-05 15:05:32 +03:00
|
|
|
# indices between sample and externalized version must match
|
|
|
|
sample = list(sample)
|
2016-03-02 01:44:41 +03:00
|
|
|
batch = remote.iterbatch()
|
|
|
|
batch.heads()
|
|
|
|
batch.known(dag.externalizeall(sample))
|
2015-08-05 21:21:46 +03:00
|
|
|
batch.submit()
|
2016-03-02 01:44:41 +03:00
|
|
|
srvheadhashes, yesno = batch.results()
|
2011-05-02 21:21:30 +04:00
|
|
|
|
|
|
|
if cl.tip() == nullid:
|
|
|
|
if srvheadhashes != [nullid]:
|
|
|
|
return [nullid], True, srvheadhashes
|
|
|
|
return [nullid], False, []
|
|
|
|
|
2011-05-05 19:21:37 +04:00
|
|
|
# start actual discovery (we note this before the next "if" for
|
|
|
|
# compatibility reasons)
|
2011-05-02 21:21:30 +04:00
|
|
|
ui.status(_("searching for changes\n"))
|
|
|
|
|
|
|
|
srvheads = dag.internalizeall(srvheadhashes, filterunknown=True)
|
|
|
|
if len(srvheads) == len(srvheadhashes):
|
2011-07-05 23:30:42 +04:00
|
|
|
ui.debug("all remote heads known locally\n")
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
return (srvheadhashes, False, srvheadhashes)
|
2011-05-02 21:21:30 +04:00
|
|
|
|
2015-05-16 21:34:19 +03:00
|
|
|
if sample and len(ownheads) <= initialsamplesize and all(yesno):
|
2011-11-11 04:07:10 +04:00
|
|
|
ui.note(_("all local heads known remotely\n"))
|
2011-06-15 00:58:00 +04:00
|
|
|
ownheadhashes = dag.externalizeall(ownheads)
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
return (ownheadhashes, True, srvheadhashes)
|
2011-06-15 00:58:00 +04:00
|
|
|
|
2011-05-02 21:21:30 +04:00
|
|
|
# full blown discovery
|
|
|
|
|
2012-05-12 17:54:54 +04:00
|
|
|
# own nodes I know we both know
|
|
|
|
# treat remote heads (and maybe own heads) as a first implicit sample
|
|
|
|
# response
|
2014-11-16 11:40:29 +03:00
|
|
|
common = cl.incrementalmissingrevs(srvheads)
|
|
|
|
commoninsample = set(n for i, n in enumerate(sample) if yesno[i])
|
|
|
|
common.addbases(commoninsample)
|
2015-01-07 03:30:37 +03:00
|
|
|
# own nodes where I don't know if remote knows them
|
2014-11-16 11:40:29 +03:00
|
|
|
undecided = set(common.missingancestors(ownheads))
|
|
|
|
# own nodes I know remote lacks
|
|
|
|
missing = set()
|
2011-05-02 21:21:30 +04:00
|
|
|
|
2011-06-15 00:58:00 +04:00
|
|
|
full = False
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
with progress.bar(ui, _("searching"), _("queries")) as prog:
|
2018-03-21 23:36:37 +03:00
|
|
|
while undecided:
|
|
|
|
|
|
|
|
if sample:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
missinginsample = [n for i, n in enumerate(sample) if not yesno[i]]
|
2018-03-21 23:36:37 +03:00
|
|
|
missing.update(dag.descendantset(missinginsample, missing))
|
|
|
|
|
|
|
|
undecided.difference_update(missing)
|
|
|
|
|
|
|
|
if not undecided:
|
|
|
|
break
|
|
|
|
|
|
|
|
if full or common.hasbases():
|
|
|
|
if full:
|
|
|
|
ui.note(_("sampling from both directions\n"))
|
|
|
|
else:
|
|
|
|
ui.debug("taking initial sample\n")
|
|
|
|
samplefunc = _takefullsample
|
|
|
|
targetsize = fullsamplesize
|
2015-01-07 03:30:52 +03:00
|
|
|
else:
|
2018-03-21 23:36:37 +03:00
|
|
|
# use even cheaper initial sample
|
|
|
|
ui.debug("taking quick initial sample\n")
|
|
|
|
samplefunc = _takequicksample
|
|
|
|
targetsize = initialsamplesize
|
|
|
|
if len(undecided) < targetsize:
|
|
|
|
sample = list(undecided)
|
|
|
|
else:
|
|
|
|
sample = samplefunc(dag, undecided, targetsize)
|
|
|
|
sample = _limitsample(sample, targetsize)
|
|
|
|
|
|
|
|
roundtrips += 1
|
|
|
|
prog.value = roundtrips
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
ui.debug(
|
|
|
|
"query %i; still undecided: %i, sample size is: %i\n"
|
|
|
|
% (roundtrips, len(undecided), len(sample))
|
|
|
|
)
|
2018-03-21 23:36:37 +03:00
|
|
|
# indices between sample and externalized version must match
|
|
|
|
sample = list(sample)
|
|
|
|
yesno = remote.known(dag.externalizeall(sample))
|
|
|
|
full = True
|
|
|
|
|
|
|
|
if sample:
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
commoninsample = set(n for i, n in enumerate(sample) if yesno[i])
|
2018-03-21 23:36:37 +03:00
|
|
|
common.addbases(commoninsample)
|
|
|
|
common.removeancestorsfrom(undecided)
|
2014-11-16 11:40:29 +03:00
|
|
|
|
|
|
|
# heads(common) == heads(common.bases) since common represents common.bases
|
|
|
|
# and all its ancestors
|
|
|
|
result = dag.headsetofconnecteds(common.bases)
|
|
|
|
# common.bases can include nullrev, but our contract requires us to not
|
|
|
|
# return any heads in that case, so discard that
|
|
|
|
result.discard(nullrev)
|
2017-06-07 12:29:39 +03:00
|
|
|
elapsed = util.timer() - start
|
|
|
|
ui.debug("%d total queries in %.4fs\n" % (roundtrips, elapsed))
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
msg = "found %d common and %d unknown server heads," " %d roundtrips in %.4fs\n"
|
2017-06-10 20:47:09 +03:00
|
|
|
missing = set(result) - set(srvheads)
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
ui.log("discovery", msg, len(result), len(missing), roundtrips, elapsed)
|
2011-05-02 21:21:30 +04:00
|
|
|
|
|
|
|
if not result and srvheadhashes != [nullid]:
|
|
|
|
if abortwhenunrelated:
|
2015-10-08 22:55:45 +03:00
|
|
|
raise error.Abort(_("repository is unrelated"))
|
2011-05-02 21:21:30 +04:00
|
|
|
else:
|
|
|
|
ui.warn(_("warning: repository is unrelated\n"))
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
return ({nullid}, True, srvheadhashes)
|
2011-05-02 21:21:30 +04:00
|
|
|
|
codemod: join the auto-formatter party
Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
2018-05-26 07:34:37 +03:00
|
|
|
anyincoming = srvheadhashes != [nullid]
|
2011-07-28 02:32:54 +04:00
|
|
|
return dag.externalizeall(result), anyincoming, srvheadhashes
|