Summary:
This check is useful and detects real errors (ex. fbconduit). Unfortunately
`arc lint` will run it with both py2 and py3 so a lot of py2 builtins will
still be warned.
I didn't find a clean way to disable py3 check. So this diff tries to fix them.
For `xrange`, the change was done by a script:
```
import sys
import redbaron
headertypes = {'comment', 'endl', 'from_import', 'import', 'string',
'assignment', 'atomtrailers'}
xrangefix = '''try:
xrange(0)
except NameError:
xrange = range
'''
def isxrange(x):
try:
return x[0].value == 'xrange'
except Exception:
return False
def main(argv):
for i, path in enumerate(argv):
print('(%d/%d) scanning %s' % (i + 1, len(argv), path))
content = open(path).read()
try:
red = redbaron.RedBaron(content)
except Exception:
print(' warning: failed to parse')
continue
hasxrange = red.find('atomtrailersnode', value=isxrange)
hasxrangefix = 'xrange = range' in content
if hasxrangefix or not hasxrange:
print(' no need to change')
continue
# find a place to insert the compatibility statement
changed = False
for node in red:
if node.type in headertypes:
continue
# node.insert_before is an easier API, but it has bugs changing
# other "finally" and "except" positions. So do the insert
# manually.
# # node.insert_before(xrangefix)
line = node.absolute_bounding_box.top_left.line - 1
lines = content.splitlines(1)
content = ''.join(lines[:line]) + xrangefix + ''.join(lines[line:])
changed = True
break
if changed:
# "content" is faster than "red.dumps()"
open(path, 'w').write(content)
print(' updated')
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
```
For other py2 builtins that do not have a py3 equivalent, some `# noqa`
were added as a workaround for now.
Reviewed By: DurhamG
Differential Revision: D6934535
fbshipit-source-id: 546b62830af144bc8b46788d2e0fd00496838939
We only want to do I/O in terms of bytes, so lets explode early
instead of recursing forever.
Differential Revision: https://phab.mercurial-scm.org/D1136
Before, it crashed because mapping['templ'] was missing. As it didn't support
the legacy list template from the beginning, we can simply use hybridlist().
See the previous commit for why.
splitlines() does not pass a mapping dict, which would probably mean the
legacy template didn't work from the beginning.
The goal is to fix "{hybrid_obj|json}" output.
A _hybrid object must act as a list or a dict as well as a generator of
legacy template strings. Before, _hybrid.__iter__() was assigned for legacy
template, which conflicted with list.__iter__() API.
This patch drops _hybrid.__iter__() and makes stringify/flatten functions
unwrap a generator instead.
When disabling the '#requires serve' check in test-hgwebdir.t and running it on
Windows, several 500 errors popped up when querying '?style=json', with the
following in the error log:
File "...\\mercurial\\templater.py", line 393, in runfilter
"keyword '%s'") % (filt.func_name, dt))
Abort: template filter 'json' is not compatible with keyword 'lastchange'
The swallowed exception at that point was:
File "...\\mercurial\\templatefilters.py", line 242, in json
raise TypeError('cannot encode type %s' % obj.__class__.__name__)
TypeError: cannot encode type long
This corresponds to 'lastchange' being populated by hgweb.common.get_stat(),
which uses os.stat().st_mtime. os.stat_float_times() is being disabled in util,
so the type for the times is 'long' on Windows, and 'int' on Linux.
Using decorator can localize changes for adding (or removing) a
template filter function in source code.
This patch also removes leading ":FILTER:" part in help document of
each filters, because using templatefilter makes it useless.
This patch uses not 'filter' but 'templatefilter' as a decorator name,
because the former name hides Python built-in one, even though the
latter is a little redundant in 'templatefilters.py'.
This patch also adds loadfilter() to templatefilters, because this
combination helps to figure out how they cooperate with each other.
Listing up loadfilter() in dispatch.extraloaders causes implicit
loading template filter functions at loading (3rd party) extension.
This change requires that "templatefilter" attribute of (3rd party)
extension is registrar.templatefilter or so.
This is necessary to preserve filename encoding over JSON. Instead, this
patch inserts "|utf8" where non-ascii local-encoding texts can be passed
to "|json".
See also the commit that introduced "utf8" filter.
This will be applied prior to "|json" filter. This sounds like odd, but it
is necessary to handle local-encoding text as well as raw filename bytes.
Because filenames are bytes in Mercurial and Unix world, {filename|json} should
preserve the original byte sequence, which implies
{x|json} -> '"' toutf8b(x) '"'
On the other hand, most template strings are in local encoding. Because
"|json" filter have to be byte-transparent to filenames, we need something to
annotate an input as a local string, that's what "|utf8" will do.
{x|utf8|json} -> '"' toutf8b(fromlocal(x)) '"'
"|utf8" is an explicit call, so aborts if input bytes can't be converted to
UTF-8.
It's been unused, undocumented and flawed in that it expects a unicode input,
never works correctly if an input has non-ascii character. We should use "json"
filter instead.
As JSON string is known to be a unicode, we should try round-trip conversion
for localstr type. This patch tests localstr type explicitly because
encoding.fromlocal() may raise Abort for undecodable str, which is probably
not what we want. Maybe we can refactor json filter to use encoding module
more later.
Still "{desc|json}" can't round-trip because showdescription() modifies a
localstr object.
We were assuming everything under 128 was printable ascii, but there are a lot
of control characters in that range that can't simply be included in json and
other targets. We forcibly encode everything under 32, because they are either
control char or oddly printable (like tab or line ending).
We also add the hypothesis-powered test that caught this.
There needs to be a way to escape symbolic revisions containing forward
slashes, but urlescape filter doesn't escape slashes at all (in fact, it is
used in places where forward slashes must be preserved).
The filter considers @ to be safe just for bookmarks like @ and @default to
look good in urls.
A few template keywords can in fact return None, such as {bisect}. In
some contexts, these get stringified into None instead of "". This is
leaking Python details into the UI.
The json filter was previously iterating over keys in an object in an
undefined order. Let's throw a sorted() in there so output is
consistent.
It's somewhat frightening that there are no tests for the json filter.
Subsequent commits will add them, so we pass on the opportunity to add
them here.
The "changeset" template from hgweb is using a lambda in the
"diffsummary" key. In preparation for enabling JSON output from hgweb,
teach the json filter how to call functions.
Previously there was no way of telling how much children or bookmarks or tags a
certain changeset has in a template. It was possible to tell if a changeset has
either 0 or not 0 bookmarks, but not to tell if it has 1 or 2 of them, for
example.
This filter, simply named count, makes it possible to count the number of items
in a list or the length of a string (or, anything that python's len can count).
E.g.: {children|count}, {bookmarks|count}, {file_adds|count}.
Testing the filter on node hash and shortened node hash is chosen because they
both have defined length.
As for lists of strings - children, tags and file_adds are used, because they
provide some variety and also prove that what's counted is the number of string
items in the list, and not the list stringified (they are lists of non-empty,
multi-character strings).
Additionally, revset template function is used for testing the filter, since
the combination is very flexible and will possibly be used together a lot.
(The previous version of this patch had an incorrect email subject and was
apparently lost - patchwork says the patch has been accepted, but it's not so.
The changes between that and this patch are minimal: now the filter does not
disturb the alphabetical order of function definitions and dict keys.)
This is useful for applying changes to each line, and it's especially powerful
when used in conjunction with conditionals to modify lines based on content.