I've tried to unify gettemplate() with buildtemplate(), but it didn't go well
because gettemplate() have to bypass mapping dict.
For example, web templates have '{tags%changelogtag}' and 'changelogtag' is
defined in both mapping, the default, and context.cache, sourced from map file.
In general, mapping shadows context variables, but gettemplate() have to pick
it from context.cache.
The previous patch made 'string' is always interpreted as a template. So
this patch removes the special handling of r'rawstring' instead. Now r''
disables template processing at all.
This patch series is intended to unify the interpretation of string literals.
It is breaking change that boldly assumes
a. string literal "..." never contains template-like fragment or it is
intended to be a template
b. we tend to use raw string literal r"..." for regexp pattern in which "{"
should have different meaning
Currently, we don't have a comprehensible rule how string literals are
evaluated in template functions. For example, fill() takes "initialindent"
and "hangindent" as templates, but not for "text", whereas "text" is a
template in pad() function.
date(date, fmt)
diff(includepattern, excludepattern)
fill(text, width, initialident: T, hangindent: T)
get(dict, key)
if(expr, then: T, else: T)
ifcontains(search, thing, then: T, else: T)
ifeq(expr1, expr2, then: T, else: T)
indent(text, indentchars, firstline)
join(list, sep)
label(label: T, expr: T)
pad(text: T, width, fillchar, right)
revset(query, formatargs...])
rstdoc(text, style)
shortest(node, minlength)
startswith(pattern, text)
strip(text, chars)
sub(pattern, replacement, expression: T)
word(number, text, separator)
expr % template: T
T: interpret "string" or r"rawstring" as template
This patch series adjusts the rule as follows:
a. string literal, '' or "", starts template processing (BC)
b. raw string literal, r'' or r"", disables both \-escape and template
processing (BC, done by subsequent patches)
c. fragment not surrounded by {} is non-templated string
"ccc{'aaa'}{r'bbb'}"
------------------ *: template
--- c: string
--- a: template
--- b: rawstring
Because this can eliminate the compilation of template arguments from the
evaluation phase, "hg log -Tdefault" gets faster.
% cd mozilla-central
% LANG=C HGRCPATH=/dev/null hg log -Tdefault -r0:10000 --time > /dev/null
before: real 4.870 secs (user 4.860+0.000 sys 0.010+0.000)
after: real 3.480 secs (user 3.440+0.000 sys 0.030+0.000)
Also, this will allow us to parse nested templates at once for better error
indication.
The next patch will introduce buildtemplate function that should be defined
near runtemplate. But I don't want to insert it between buildmap and runmap.
The problem was spotted at cd1b50e99ed8, that says "this patch invokes it
with "strtoken='rawstring'" in "_evalifliteral()", because "t" is the result
of "arg" evaluation and it should be "string-escape"-ed if "arg" is "string"
expression." This workaround is no longer valid since 72be08a15d8d introduced
strict parsing of '\{'.
Instead, we should interpret bare token as "string" or "rawstring" template.
This is what buildmap() does at parsing phase.
Because double backslashes are processed as a string escape sequence, '\\{'
should start the template syntax. On the other hand, r'' disables any sort
of \-escapes, so r'\{' can go either way, never start the template syntax
or always start it. I simply chose the latter, which means r'\{' is the same
as '\\{'.
This patch brings back pre-2.8.1 behavior.
The result of parsestring() is stored in templater's cache, t.cache, and then
it is parsed as a template string by compiletemplate(). So t.cache should keep
an unparsed string no matter if it is sourced from config value. Otherwise
backslashes would be processed twice.
The test vector is borrowed from 83ff877959a6.
The content of "hg help templating" is largely derived from docstrings
on functions providing functionality. Template functions are the long
holdout.
Prepare for generating them dynamically by defining docstrings for all
template functions.
There are numerous ways these docs could be improved. Right now, the
help output simply shows function names and arguments. So literally
any accurate data is better than what is there now.
Although Python supports `X = Y if COND else Z`, this was only
introduced in Python 2.5. Since we have to support Python 2.4, it was
a very common thing to write instead `X = COND and Y or Z`, which is a
bit obscure at a glance. It requires some intricate knowledge of
Python to understand how to parse these one-liners.
We change instead all of these one-liners to 4-liners. This was
executed with the following perlism:
find -name "*.py" -exec perl -pi -e 's,(\s*)([\.\w]+) = \(?(\S+)\s+and\s+(\S*)\)?\s+or\s+(\S*)$,$1if $3:\n$1 $2 = $4\n$1else:\n$1 $2 = $5,' {} \;
I tweaked the following cases from the automatic Perl output:
prev = (parents and parents[0]) or nullid
port = (use_ssl and 443 or 80)
cwd = (pats and repo.getcwd()) or ''
rename = fctx and webutil.renamelink(fctx) or []
ctx = fctx and fctx or ctx
self.base = (mapfile and os.path.dirname(mapfile)) or ''
I also added some newlines wherever they seemd appropriate for readability
There are probably a few ersatz ternary operators still in the code
somewhere, lurking away from the power of a simple regex.
A style name should not contain "/", "\", "." and "..". Otherwise, templates
could be loaded from outside of the specified templates directory by invalid
?style= parameter. hgweb should not allow such requests.
This change means subdir/name is also rejected.
Template functions use "yield"s assuming that the result will be combined
into a string, which means both "f -> str" and "f -> generator" should behave
in the same way.
Before this patch, piping generator function resulted in a cryptic error.
We had to insert "|stringify" in this case.
$ hg log --template '{if(author, author)|user}\n'
abort: template filter 'userfilter' is not compatible with keyword
'[(<function runsymbol at 0x7f5af2e8d8c0>, 'author'),
(<function runsymbol at 0x7f5af2e8d8c0>, 'author')]'
7678263f920c is fine for "{revset()}", but "i.values()[0]" does not work if
each item has more than one values such as "{bookmarks}".
This fixes the problem by using list.__contains__ or dict.__contains__
appropriately.
This change is intended to avoid exposing the implementation detail to
callers. I'm going to extend fullreposet to support "null" revision, so
these mfunc calls will have to use fullreposet() instead of spanset().
"pad" function and "rawstring" type were introduced in parallel, 89145c35f76e
in default and cd1b50e99ed8 in stable respectively. Therefore, "pad" function
lacked handling of "rawstring" unintentionally.
Before this patch, we had to quote integer literals to pass to template
functions. It was error-prone, so we should allow "word(0, x)" syntax.
Currently only decimal integers are allowed. It's easy to support 0x, 0b and 0
prefixes, but I don't think they are useful.
This patch assumes that template keywords and names defined in map files do
not start with digits, except for positional variables seen in the schemes
extension.
The next patch will introduce integer literals, but the schemes extension
expects that '{1}', '{2}', ... are interpreted as keywords. This patch allows
us to process '{foo(1)}' as 'func(integer)', whereas '{1}' as 'symbol'.
e926f2ef639a fixed the issue of double escapes, but it made the following
template fail with syntax error because of <\">. Strictly speaking, <\">
appears to be invalid in non-string part, but we are likely to escape <">
if surrounded by quotes, and we are used to write such templates by trial
and error.
[templates]
sl = "{tags % \"{ifeq(tag,'tip','',label('log.tag', ' {tag}'))}\"}"
So, for backward compatibility between 2.8.1 and 3.4, a single backslash
before quotation mark is stripped only in quoted template. We don't care
for <\"> in string literal in quoted template, which never worked as expected
before.
template result
--------- ------------------------
{\"\"} parse error
"{""}" {""} -> <>
"{\"\"}" {""} -> <>
{"\""} {"\""} -> <">
'{"\""}' {"\""} -> <">
"{"\""}" parse error (don't care)
This keyword remapping was introduced in 236440938a03 as part of converting
generator based iterators into list based iterators, mentioning "undesired
behavior in template" when a generator is exhausted, but doesn't say what and
introduces no tests.
The problem with the remapping was that it corrupted the output for keywords
like 'extras', 'file_copies' and 'file_copies_switch' in templates such as:
$ hg log -r 82a4f5557c6b --template "{file_copies % ' File: {file_copy}\n'}"
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
What was happening was that in the first call to runtemplate() inside runmap(),
'lm' mapped the keyword (e.g. file_copies) to the appropriate showxxx() method.
On each subsequent call to runtemplate() in that loop however, the keyword was
mapped to a list of the first item's pieces, e.g.:
'file_copy': ['mercurial/changelog.py', ' (', 'mercurial/hg.py', ')']
Therefore, the dict for the second and any subsequent items were not processed
through the corresponding showxxx() method, and the first item's data was
reused.
The 'extras' keyword regressed in 56b014c52204, and 'file_copies' regressed in
4e182fb53989 for other reasons. The common thread of things fixed by this seems
to be when a list of dicts are passed to the templatekw._hybrid class.
The search was introduced in 501e9ec9a85d. It might have been necessary back
then when using __file__ directly and frozen-ness wasn't considered. Now we
should know exactly where the templates can be found.
templates, help and locale data is normally stored as sub folders in the
directory containing the source of the mercurial module. In a frozen build they
live as sub folders next to 'hg.exe' and 'library.zip'.
These different kind of data were handled in different ways. Unify that by
introducing util.datapath. The value is computed from the environment and is
always used, so we just calculate the value on module load.
"diff" allows to embed changes in the target revision into template
output, even if the command itself doesn't take "--patch" option
Combination of "[committemplate]" configuration and "diff" template
function can achieve the feature like issue231 ("option to have diff
displayed in commit editor buffer")
http://bz.selenic.com/show_bug.cgi?id=231
For example, templating below can be used to add each "diff" output
lines "HG: " prefix::
{splitlines(diff) % 'HG: {line}\n'}
This patch implements "diff" not as "a template keyword" but as "a
template function" to take include/exclude patterns at runtime.
It allows to specify target files of command (by -I/-X command line
options) and "diff" separately.
Before this patch, predicates defined in "[revsetalias]" can't be used
in the query specified to template function "revset()", because:
- "revset()" uses "localrepository.revs()" to get query result, but
- "localrepository.revs()" passes "None" as "ui" to "revset.match()", then
- "revset.match()" can't recognize any alias predicates
To enable alias predicates to be used in "revset()" function, this
patch invokes "revset.match()" directly with "repo.ui".
This patch doesn't make "localrepository.revs()" pass "self.ui" to
"revset.match()", because this may be intentional implementation to
prevent alias predicates from shadowing built-in ones and breaking
functions internally using "localrepository.revs()".
Even if it isn't intentional one, the check for shadowing should be
implemented (maybe on default branch) before fixing it for safety.
This function allows returning only the nth "word" from a string. By default
a string is split as by Python's split() function default, but an optional
third parameter can also override what string the string is split by.
This function returns a string only if it starts with a given string.
It is particularly useful when combined with splitlines and/or used with
conditionals that fail when empty strings are passed in to take action
based on the contents of a line.
Previously the ifcontains revset was checking against the set using a pure
__contains__ check. It turns out the set was actually a list of
formatted strings meant for ui output, which meant the contains check failed if
the formatted string wasn't significantly different from the raw value.
This change makes it check against the raw data, prior to it being formatted.
This should correct an earlier couple of bad merges (5433856b2558 and
596960a4ad0d, now pruned) that accidentally brought in a change that had
been marked obsolete (244ac996a821).
Previously, if a template '{foo()}' was given, the buildfunc would not be able
to match it and hit a code path that would not return so it would error out
later in the templater stating that NoneType was not iterable. This patch makes
sure that a proper error is raised so that the user can be informed.
Tests have been updated.
Changeset 83ff877959a6 (released with 2.8.1) fixed "recursively
evaluate string literals as templates" problem (issue4102) by moving
the location of "string-escape"-ing from "tokenizer()" to
"compiletemplate()".
But some parts in template expressions below are not processed by
"compiletemplate()", and it may cause unexpected result.
- 'expr' of 'if(expr, then, else)'
- 'expr's of 'ifeq(expr, expr, then, else)'
- 'sep' of 'join(list, sep)'
- 'text' and 'style' of 'rstdoc(text, style)'
- 'text' and 'chars' of 'strip(text, chars)'
- 'pat' and 'repl' of 'sub(pat, repl, expr)'
For example, '\n' of "{join(extras, '\n')}" is not "string-escape"-ed
and treated as a literal '\n'. This breaks "Display the contents of
the 'extra' field, one per line" example in "hg help templates".
Just "string-escape"-ing on each parts above may not work correctly,
because inside expression of nested ones already applies
"string-escape" on string literals. For example:
- "{join(files, '\n')}" doesn't return "string-escape"-ed string, but
- "{join(files, if(branch, '\n', '\n'))}" does
To fix this problem, this patch does:
- introduce "rawstring" token and "runrawstring" method to handle
strings not to be "string-escape"-ed correctly, and
- make "runstring" method return "string-escape"-ed string, and
delay "string-escape"-ing until evaluation
This patch invokes "compiletemplate()" with "strtoken=exp[0]" in
"gettemplate()", because "exp[1]" is not yet evaluated. This code path
is tested via mapping ("expr % '{template}'").
In the other hand, this patch invokes it with "strtoken='rawstring'"
in "_evalifliteral()", because "t" is the result of "arg" evaluation
and it should be "string-escape"-ed if "arg" is "string" expression.
This patch doesn't test "string-escape"-ing on 'expr' of 'if(expr,
then, else)', because it doesn't affect the result.
