Summary:
Turned on the auto formatter. Ran `arc lint --apply-patches --take BLACK **/*.py`.
Then run `arc lint` again so some other autofixers like spellchecker etc. looked
at the code base. Manually accept the changes whenever they make sense, or use
a workaround (ex. changing "dict()" to "dict constructor") where autofix is false
positive. Disabled linters on files that are hard (i18n/polib.py) to fix, or less
interesting to fix (hgsubversion tests), or cannot be fixed without breaking
OSS build (FBPYTHON4).
Conflicted linters (test-check-module-imports.t, part of test-check-code.t,
test-check-pyflakes.t) are removed or disabled.
Duplicated linters (test-check-pyflakes.t, test-check-pylint.t) are removed.
An issue of the auto-formatter is lines are no longer guarnateed to be <= 80
chars. But that seems less important comparing with the benefit auto-formatter
provides.
As we're here, also remove test-check-py3-compat.t, as it is currently broken
if `PYTHON3=/bin/python3` is set.
Reviewed By: wez, phillco, simpkins, pkaush, singhsrb
Differential Revision: D8173629
fbshipit-source-id: 90e248ae0c5e6eaadbe25520a6ee42d32005621b
Summary:
This check is useful and detects real errors (ex. fbconduit). Unfortunately
`arc lint` will run it with both py2 and py3 so a lot of py2 builtins will
still be warned.
I didn't find a clean way to disable py3 check. So this diff tries to fix them.
For `xrange`, the change was done by a script:
```
import sys
import redbaron
headertypes = {'comment', 'endl', 'from_import', 'import', 'string',
'assignment', 'atomtrailers'}
xrangefix = '''try:
xrange(0)
except NameError:
xrange = range
'''
def isxrange(x):
try:
return x[0].value == 'xrange'
except Exception:
return False
def main(argv):
for i, path in enumerate(argv):
print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
content = open(path).read()
try:
red = redbaron.RedBaron(content)
except Exception:
print(' warning: failed to parse')
continue
hasxrange = red.find('atomtrailersnode', value=isxrange)
hasxrangefix = 'xrange = range' in content
if hasxrangefix or not hasxrange:
print(' no need to change')
continue
# find a place to insert the compatibility statement
changed = False
for node in red:
if node.type in headertypes:
continue
# node.insert_before is an easier API, but it has bugs changing
# other "finally" and "except" positions. So do the insert
# manually.
# # node.insert_before(xrangefix)
line = node.absolute_bounding_box.top_left.line - 1
lines = content.splitlines(1)
content = ''.join(lines[:line]) + xrangefix + ''.join(lines[line:])
changed = True
break
if changed:
# "content" is faster than "red.dumps()"
open(path, 'w').write(content)
print(' updated')
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
```
For other py2 builtins that do not have a py3 equivalent, some `# noqa`
were added as a workaround for now.
Reviewed By: DurhamG
Differential Revision: D6934535
fbshipit-source-id: 546b62830af144bc8b46788d2e0fd00496838939
Wrapping text in templates for 'hg log --graph' can't be done very well,
because the template doesn't know how wide the graph drawing is. The edge
drawing function needs to know the number of lines in the template output, so
we need to also determine how wide that drawing would be before we call the
edgefn or evaluate the template.
This patch makes edgefn compute the graph width and pass it into the template
so that we can do something like this:
COLUMNS=10 hg log --graph --template "{fill(desc, termwidth - graphwidth)}"
@ a a a a
| a a a a
| a a a a
o a a a
|\ a a a
| | a a a
| | a a a
Using extensions to do this would be relatively complicated due to a lack of
hooks in this area of the code.
In the future it may make sense to have a more generic "textwidth" that tells
you how many columns you can expect to fill without causing the terminal to
wrap your output. I'm not sure there are other situations to motivate this yet,
or if it is entirely feasible.
Differential Revision: https://phab.mercurial-scm.org/D360
This module hosts the following functions. They are somewhat similar (e.g.
scanning revisions using heap queue or stack) and seem non-trivial in
algorithmic point of view.
- _revancestors()
- _revdescendants()
- reachableroots()
- _toposort()
I was thinking of adding revset._fileancestors() generator for better follow()
implementation, but it would be called from context.py as well. So I decided
to create new module.
Naming is hard. I couldn't come up with any better module name, so it's called
"dag operation" now. I rejected the following candidates:
- ancestor.py - existing, revlog-level DAG algorithm
- ancestorset.py - doesn't always return a set
- dagalgorithm.py - hard to type
- dagutil.py - existing
- revancestor.py - I want to add fileancestors()
% wc -l mercurial/dagop.py mercurial/revset.py
339 mercurial/dagop.py
2020 mercurial/revset.py
2359 total
Python 2:
>>> ls = []
>>> ls.extend(b'abc')
>>> ls
['a', 'b', 'c']
Python 3:
>>> ls = []
>>> ls.extend(b'abc')
>>> ls
[97, 98, 99]
So to make sure things work fine in Py3, this patch does:
>>> ls = []
>>> ls.extend([b'a', b'b', b'c'])
>>> ls
[b'a', b'b', b'c']
After this `hg log -G` works!
Sort revisions in reverse revision order but grouped by topographical branches.
Visualised as a graph, instead of:
o 4
|
| o 3
| |
| o 2
| |
o | 1
|/
o 0
revisions on a 'main' branch are emitted before 'side' branches:
o 4
|
o 1
|
| o 3
| |
| o 2
|/
o 0
where what constitutes a 'main' branch is configurable, so the sort could also
result in:
o 3
|
o 2
|
| o 4
| |
| o 1
|/
o 0
This sort was already available as an experimental option in the graphmod
module, from which it is now removed.
This sort is best used with hg log -G:
$ hg log -G "sort(all(), topo)"
Allow for a style to only apply to the last N lines (for positive N) or
everything but the first N lines (for negative N) of the section along the
current node. This allows for more subtle grandparent styling.
So from the default:
$ hg log -G ...
o Lorem ipsum dolor sit
:\ amet, consectetur
: : adipiscing elit, sed
: : do eiusmod tempor
: :
o : incididunt ut labore
| : et dolore magna
| : aliqua. Ut enim ad
| : minim veniam, quis
|/
o nostrud exercitation
: ullamco laboris nisi
: ut aliquip ex ea
: commodo consequat.
:
o Duis aute irure dolor
| in reprehenderit in
~ voluptate velit esse
cillum dolore eu
to
$ hg log -G --config "experimental.graphstyle.grandparent=2." ...
o Lorem ipsum dolor sit
|\ amet, consectetur
| | adipiscing elit, sed
. . do eiusmod tempor
. .
o | incididunt ut labore
| | et dolore magna
| | aliqua. Ut enim ad
| | minim veniam, quis
|/
o nostrud exercitation
| ullamco laboris nisi
| ut aliquip ex ea
. commodo consequat.
.
o Duis aute irure dolor
| in reprehenderit in
~ voluptate velit esse
cillum dolore eu
or
$ hg log -G --config "experimental.graphstyle.grandparent=1:" ...
o Lorem ipsum dolor sit
|\ amet, consectetur
| | adipiscing elit, sed
| | do eiusmod tempor
: :
o | incididunt ut labore
| | et dolore magna
| | aliqua. Ut enim ad
| | minim veniam, quis
|/
o nostrud exercitation
| ullamco laboris nisi
| ut aliquip ex ea
| commodo consequat.
:
o Duis aute irure dolor
| in reprehenderit in
~ voluptate velit esse
cillum dolore eu
or
$ hg log -G --config "experimental.graphstyle.grandparent=-2!" ...
o Lorem ipsum dolor sit
|\ amet, consectetur
! ! adipiscing elit, sed
! ! do eiusmod tempor
! !
o | incididunt ut labore
| | et dolore magna
| | aliqua. Ut enim ad
| | minim veniam, quis
|/
o nostrud exercitation
| ullamco laboris nisi
! ut aliquip ex ea
! commodo consequat.
!
o Duis aute irure dolor
| in reprehenderit in
~ voluptate velit esse
cillum dolore eu
When there are more than 2 parents for a given node (in a sparse graph), extra
dummy nodes are inserted to transition the lines more gradually. However, since
the seen state was not updated when yielding the extra nodes, the wrong graph
styles were being applied to the nodes.
Shorten the graph, cutting the all vertical (not oblique) edges rows.
Activate with 'graphshorten = true' in [experimental] section.
Example graph with deactivated option:
$ hg log --graph --template '{rev} {desc|firstline}' --rev 1035:1015
o 1035 Merge with BOS
|\
| o 1034 Fix help output, and a few broken tests.
| |
| o 1033 Merge with MPM.
| |\
| | o 1032 Get patchbomb working with tip again.
| | |
| | o 1031 Rewrite log command. New version is faster and more featureful.
| | |
| | o 1030 Merge with MPM.
| | |\
| | | o 1029 Emacs: implement hg-incoming, hg-outgoing and hg-push.
| | | |
| | | o 1028 Add commands.debugconfig.
| | | |
| | | o 1027 Emacs: fix up hg-log and hg-diff to operate more uniformly.
| | | |
| | | o 1026 Merge with MPM.
| | | |\
| | | | o 1025 Merge with MPM.
| | | | |
| | | | ~
| | | o 1024 Sync buffers prior to doing a diff.
| | | |
| | | ~
o | | 1023 Minor tweak to the revgen algorithm
|/ /
o | 1022 Minor hgwebdir tweaks
| |
o | 1021 Add Makefile to the manifest
| |
o | 1020 Add default make rule
| |
o | 1019 Create helper functions for I/O to files in the working directory
| |
o | 1018 Add some aliases
| |
o | 1017 Fix up help for binary options
|/
o 1016 Teach annotate about binary files
|
o 1015 Add automatic binary file detection to diff and export
|
~
Example graph with activated option:
$ hg log --graph --template '{rev} {desc|firstline}' --rev 1035:1015
o 1035 Merge with BOS
|\
| o 1034 Fix help output, and a few broken tests.
| o 1033 Merge with MPM.
| |\
| | o 1032 Get patchbomb working with tip again.
| | o 1031 Rewrite log command. New version is faster and more featureful.
| | o 1030 Merge with MPM.
| | |\
| | | o 1029 Emacs: implement hg-incoming, hg-outgoing and hg-push.
| | | o 1028 Add commands.debugconfig.
| | | o 1027 Emacs: fix up hg-log and hg-diff to operate more uniformly.
| | | o 1026 Merge with MPM.
| | | |\
| | | | o 1025 Merge with MPM.
| | | | |
| | | | ~
| | | o 1024 Sync buffers prior to doing a diff.
| | | |
| | | ~
o | | 1023 Minor tweak to the revgen algorithm
|/ /
o | 1022 Minor hgwebdir tweaks
o | 1021 Add Makefile to the manifest
o | 1020 Add default make rule
o | 1019 Create helper functions for I/O to files in the working directory
o | 1018 Add some aliases
o | 1017 Fix up help for binary options
|/
o 1016 Teach annotate about binary files
o 1015 Add automatic binary file detection to diff and export
|
~
Rather than draw an edge all the way to the bottom of the graph, make it
possible to end an edge to parents that are not part of the graph early on.
This results in a far cleaner graph.
Any edge type can be set to end early; set the ui.graphstyle.<edgetype>
parameter to the empty string to enable this.
For example, setting the following configuration:
[ui]
graphstyle.grandparent = :
graphstyle.missing =
would result in a graph like this:
o changeset: 32:d06dffa21a31
|\ parent: 27:886ed638191b
| : parent: 31:621d83e11f67
| :
o : changeset: 31:621d83e11f67
|\: parent: 21:d42a756af44d
| : parent: 30:6e11cd4b648f
| :
o : changeset: 30:6e11cd4b648f
|\ \ parent: 28:44ecd0b9ae99
| ~ : parent: 29:cd9bb2be7593
| /
o : changeset: 28:44ecd0b9ae99
|\ \ parent: 1:6db2ef61d156
| ~ : parent: 26:7f25b6c2f0b9
| /
o : changeset: 26:7f25b6c2f0b9
|\ \ parent: 18:1aa84d96232a
| | : parent: 25:91da8ed57247
| | :
| o : changeset: 25:91da8ed57247
| |\: parent: 21:d42a756af44d
| | : parent: 24:a9c19a3d96b7
| | :
| o : changeset: 24:a9c19a3d96b7
| |\ \ parent: 0:e6eb3150255d
| | ~ : parent: 23:a01cddf0766d
| | /
| o : changeset: 23:a01cddf0766d
| |\ \ parent: 1:6db2ef61d156
| | ~ : parent: 22:e0d9cccacb5d
| | /
| o : changeset: 22:e0d9cccacb5d
|/:/ parent: 18:1aa84d96232a
| : parent: 21:d42a756af44d
| :
| o changeset: 21:d42a756af44d
| |\ parent: 19:31ddc2c1573b
| | | parent: 20:d30ed6450e32
| | |
+---o changeset: 20:d30ed6450e32
| | | parent: 0:e6eb3150255d
| | ~ parent: 18:1aa84d96232a
| |
| o changeset: 19:31ddc2c1573b
| |\ parent: 15:1dda3f72782d
| ~ ~ parent: 17:44765d7c06e0
|
o changeset: 18:1aa84d96232a
parent: 1:6db2ef61d156
parent: 15:1dda3f72782d
The default configuration leaves all 3 types set to |.
This is part of the work towards moving smartlog upstream; currently smartlog
injects extra nodes into the graph to indicate grandparent relationships (nodes
elided).
The walker knows when an edge leads to a direct parent, a grandparent (skipping
revisions not part of the revset) and parents that are missing altogether
(neither it nor a grandparent is in the revset). Add this information to the
parents sequence yielded.
Because revsets query is evaluated lazily, "list(revs)" may take long for
complicated query. So we shouldn't iterate revs many times. This patch is the
easiest workaround for the issue4782. We could introduce more aggressive
caching, but it wouldn't be as fast as the simple baseset operation.
Gregory Szorc said "this makes `hg wip` on my Firefox clone ~4x faster than
3.5.1 (~6.5s to ~1.5s). This is after a regression in @ to ~45s."
Not only does this defeat the purpose of having `branchgroupiter` an iterable, but
this also makes revs something else than a smartset preventing to use revs.min()
in the call to reachablerevs.
This patch is part of a series of patches to speed up the computation of
revset.reachableroots by introducing a C implementation. The main motivation
is to speed up smartlog on big repositories. At the end of the series, on our
big repositories the computation of reachableroots is 10-50x faster and
smartlog on is 2x-5x faster.
Before this patch, we had a custom computation for grandparent that was very
close to the idea of reacheablerooots. This patch expresses grandparent with
reachableroots to reduce the amount of code.
Instead of just bootstrapping the algorithm with the first revision we
see, allow callers to pass revs that should be displayed first. All
branches are retained until we can display such revision.
Expected usage is to display the current working copy parent first.
The algorithm now works when some revisions are skipped. We now use "first
included ancestors" instead of just "parent" to link changesets with each other.
We are going to add an additional layer of indentation to support non-contiguous
revset. We do it in a pure code movement changeset to help the readability of
the next changeset.
We add an experimental config option to use the topological sorting. I first
tried to hook the 'groupbranchiter' function in the 'sort' revset but this was useless
because graphlog enforces revision number sorting :(
As the goal is to advance on the topological iteration logic, I see this
experimental option as a good way to move forward.
We have to use turn the iterator into a list because the graphlog is apparently
not ready for pure iterator input yet.
This changeset introduces a function to perform topological (one branch after
the other) iteration over a set of changesets. This first version has a lot of
limitations, but the approach should be flexible enough to allow many
improvements in the future. This changeset aims to set the first stone more
than providing a complete solution.
The algorithm does not need to know the whole set of nodes involved
before emitting revision. This makes it a good candidate for usage in place
like `hg log` or graphical tools that need a fast first result time.
- Avoid flipping lineWidth state around the edge() call, pass it to the
function instead.
- Pass the line width and color appended to the other parameters instead of in
a dictionary. The javascript code is simpler, no need to check for all
containers existence, and the JSON output is smaller.
- Reindent setColor() comments and fix code spacing.
Before, we'd lookup the branch for every edge segment in the entire
graph: extremely expensive. This happened even when no per-branch
settings existed.
Now we define a revision -> config cache function that's LRU-cached
and is a no-op when no configuration exists. Still not terribly fast,
but hopefully only one real branch lookup per revision. This might
degenerate for wide graphs as the LRU is hard-coded to 20 elements.
Our goal is not to strictly disallow _invalid_ input, simply disallow _hostile_ input.
Avoid using re
Avoid creating empty dicts when no branch parameters are recognized
You can specify color to visually distinguish main branch (trunk)
on hgweb's graph page. If color specified, all branch heads will share
same color. Settings format is branch_name.color = value, where color
is six hexadecimal digits e.g.:
[graph]
default.color = FF0000
You can specify width to visually distinguish main branch (trunk)
on hgweb's graph page. Settings format is branch_name.width = value,
where width in px e.g.:
[graph]
default.width = 3
The grandparent() function was returning only the closest predecessor of a
missing parent while it must return all of them to display a correct ancestry
graph.
a32a0f72065a introduced the ability to walk the DAG
given arbitrary revisions, but changed the behaviour of
it to return a list of all nodes (and create a changectx
for each one) rather than doing it lazily.
This has a pretty significant impact on performance for large
repositories (tested on CPython repo, with output disabled):
$ time hg glog
real 0m2.642s
user 0m2.560s
sys 0m0.080s
Before a32a0f72065a:
$ time hg glog
real 0m0.143s
user 0m0.112s
sys 0m0.032s
And after this fix:
$ time hg glog
real 0m0.213s
user 0m0.184s
sys 0m0.028s
Thanks for the idea and most of the implementation to Klaus Koch
Backs revisions() and filerevs() with DAG walker which can iterate through
arbitrary list of revisions instead of strict one by one iteration from start to
stop. When a gap occurs in a revisions (i.e. in file log), the next topological
parent within the revset is searched and the connection to it is printed in the
ascii graph.
File graph can draw sometimes more connections than previous version, because
graph is produced according to the revset, not according to a file's filelog.
In case the graph contains several branches where the left parent is null, the
graphs for each are printed sequentially, not in parallel as it was a case
earlier (see for example the graph for README in hg-dev).
While this situation should never under normal use, some real
life repos sometimes contain such changesets (older hg versions,
broken rebases, etc...)
hgweb was displaying an "Internal error" in this case, and graphlog
displayed a redundant branch all the way to null: it does not cost us
much to just ignore this extra parent when constructing the DAG.