This can simplify the interface of parse() function. Our tokenizer tends to
have optional arguments other than the message to be parsed.
Before this patch, the "lookup" argument existed only for the revset, and the
templater had to pack [program, start, end] to be passed to its tokenizer.
The problem was spotted at cd1b50e99ed8, that says "this patch invokes it
with "strtoken='rawstring'" in "_evalifliteral()", because "t" is the result
of "arg" evaluation and it should be "string-escape"-ed if "arg" is "string"
expression." This workaround is no longer valid since 72be08a15d8d introduced
strict parsing of '\{'.
Instead, we should interpret bare token as "string" or "rawstring" template.
This is what buildmap() does at parsing phase.
Because double backslashes are processed as a string escape sequence, '\\{'
should start the template syntax. On the other hand, r'' disables any sort
of \-escapes, so r'\{' can go either way, never start the template syntax
or always start it. I simply chose the latter, which means r'\{' is the same
as '\\{'.
This patch brings back pre-2.8.1 behavior.
The result of parsestring() is stored in templater's cache, t.cache, and then
it is parsed as a template string by compiletemplate(). So t.cache should keep
an unparsed string no matter if it is sourced from config value. Otherwise
backslashes would be processed twice.
The test vector is borrowed from 83ff877959a6.
The content of "hg help templating" is largely derived from docstrings
on functions providing functionality. Template functions are the long
holdout.
Prepare for generating them dynamically by defining docstrings for all
template functions.
There are numerous ways these docs could be improved. Right now, the
help output simply shows function names and arguments. So literally
any accurate data is better than what is there now.
I've tried to unify gettemplate() with buildtemplate(), but it didn't go well
because gettemplate() have to bypass mapping dict.
For example, web templates have '{tags%changelogtag}' and 'changelogtag' is
defined in both mapping, the default, and context.cache, sourced from map file.
In general, mapping shadows context variables, but gettemplate() have to pick
it from context.cache.
The previous patch made 'string' is always interpreted as a template. So
this patch removes the special handling of r'rawstring' instead. Now r''
disables template processing at all.
This patch series is intended to unify the interpretation of string literals.
It is breaking change that boldly assumes
a. string literal "..." never contains template-like fragment or it is
intended to be a template
b. we tend to use raw string literal r"..." for regexp pattern in which "{"
should have different meaning
Currently, we don't have a comprehensible rule how string literals are
evaluated in template functions. For example, fill() takes "initialindent"
and "hangindent" as templates, but not for "text", whereas "text" is a
template in pad() function.
date(date, fmt)
diff(includepattern, excludepattern)
fill(text, width, initialident: T, hangindent: T)
get(dict, key)
if(expr, then: T, else: T)
ifcontains(search, thing, then: T, else: T)
ifeq(expr1, expr2, then: T, else: T)
indent(text, indentchars, firstline)
join(list, sep)
label(label: T, expr: T)
pad(text: T, width, fillchar, right)
revset(query, formatargs...])
rstdoc(text, style)
shortest(node, minlength)
startswith(pattern, text)
strip(text, chars)
sub(pattern, replacement, expression: T)
word(number, text, separator)
expr % template: T
T: interpret "string" or r"rawstring" as template
This patch series adjusts the rule as follows:
a. string literal, '' or "", starts template processing (BC)
b. raw string literal, r'' or r"", disables both \-escape and template
processing (BC, done by subsequent patches)
c. fragment not surrounded by {} is non-templated string
"ccc{'aaa'}{r'bbb'}"
------------------ *: template
--- c: string
--- a: template
--- b: rawstring
Because this can eliminate the compilation of template arguments from the
evaluation phase, "hg log -Tdefault" gets faster.
% cd mozilla-central
% LANG=C HGRCPATH=/dev/null hg log -Tdefault -r0:10000 --time > /dev/null
before: real 4.870 secs (user 4.860+0.000 sys 0.010+0.000)
after: real 3.480 secs (user 3.440+0.000 sys 0.030+0.000)
Also, this will allow us to parse nested templates at once for better error
indication.
The next patch will introduce buildtemplate function that should be defined
near runtemplate. But I don't want to insert it between buildmap and runmap.
Although Python supports `X = Y if COND else Z`, this was only
introduced in Python 2.5. Since we have to support Python 2.4, it was
a very common thing to write instead `X = COND and Y or Z`, which is a
bit obscure at a glance. It requires some intricate knowledge of
Python to understand how to parse these one-liners.
We change instead all of these one-liners to 4-liners. This was
executed with the following perlism:
find -name "*.py" -exec perl -pi -e 's,(\s*)([\.\w]+) = \(?(\S+)\s+and\s+(\S*)\)?\s+or\s+(\S*)$,$1if $3:\n$1 $2 = $4\n$1else:\n$1 $2 = $5,' {} \;
I tweaked the following cases from the automatic Perl output:
prev = (parents and parents[0]) or nullid
port = (use_ssl and 443 or 80)
cwd = (pats and repo.getcwd()) or ''
rename = fctx and webutil.renamelink(fctx) or []
ctx = fctx and fctx or ctx
self.base = (mapfile and os.path.dirname(mapfile)) or ''
I also added some newlines wherever they seemd appropriate for readability
There are probably a few ersatz ternary operators still in the code
somewhere, lurking away from the power of a simple regex.
A style name should not contain "/", "\", "." and "..". Otherwise, templates
could be loaded from outside of the specified templates directory by invalid
?style= parameter. hgweb should not allow such requests.
This change means subdir/name is also rejected.
Template functions use "yield"s assuming that the result will be combined
into a string, which means both "f -> str" and "f -> generator" should behave
in the same way.
Before this patch, piping generator function resulted in a cryptic error.
We had to insert "|stringify" in this case.
$ hg log --template '{if(author, author)|user}\n'
abort: template filter 'userfilter' is not compatible with keyword
'[(<function runsymbol at 0x7f5af2e8d8c0>, 'author'),
(<function runsymbol at 0x7f5af2e8d8c0>, 'author')]'
7678263f920c is fine for "{revset()}", but "i.values()[0]" does not work if
each item has more than one values such as "{bookmarks}".
This fixes the problem by using list.__contains__ or dict.__contains__
appropriately.
This change is intended to avoid exposing the implementation detail to
callers. I'm going to extend fullreposet to support "null" revision, so
these mfunc calls will have to use fullreposet() instead of spanset().
"pad" function and "rawstring" type were introduced in parallel, 89145c35f76e
in default and cd1b50e99ed8 in stable respectively. Therefore, "pad" function
lacked handling of "rawstring" unintentionally.
Before this patch, we had to quote integer literals to pass to template
functions. It was error-prone, so we should allow "word(0, x)" syntax.
Currently only decimal integers are allowed. It's easy to support 0x, 0b and 0
prefixes, but I don't think they are useful.
This patch assumes that template keywords and names defined in map files do
not start with digits, except for positional variables seen in the schemes
extension.
The next patch will introduce integer literals, but the schemes extension
expects that '{1}', '{2}', ... are interpreted as keywords. This patch allows
us to process '{foo(1)}' as 'func(integer)', whereas '{1}' as 'symbol'.
e926f2ef639a fixed the issue of double escapes, but it made the following
template fail with syntax error because of <\">. Strictly speaking, <\">
appears to be invalid in non-string part, but we are likely to escape <">
if surrounded by quotes, and we are used to write such templates by trial
and error.
[templates]
sl = "{tags % \"{ifeq(tag,'tip','',label('log.tag', ' {tag}'))}\"}"
So, for backward compatibility between 2.8.1 and 3.4, a single backslash
before quotation mark is stripped only in quoted template. We don't care
for <\"> in string literal in quoted template, which never worked as expected
before.
template result
--------- ------------------------
{\"\"} parse error
"{""}" {""} -> <>
"{\"\"}" {""} -> <>
{"\""} {"\""} -> <">
'{"\""}' {"\""} -> <">
"{"\""}" parse error (don't care)
This keyword remapping was introduced in 236440938a03 as part of converting
generator based iterators into list based iterators, mentioning "undesired
behavior in template" when a generator is exhausted, but doesn't say what and
introduces no tests.
The problem with the remapping was that it corrupted the output for keywords
like 'extras', 'file_copies' and 'file_copies_switch' in templates such as:
$ hg log -r 82a4f5557c6b --template "{file_copies % ' File: {file_copy}\n'}"
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
File: mercurial/changelog.py (mercurial/hg.py)
What was happening was that in the first call to runtemplate() inside runmap(),
'lm' mapped the keyword (e.g. file_copies) to the appropriate showxxx() method.
On each subsequent call to runtemplate() in that loop however, the keyword was
mapped to a list of the first item's pieces, e.g.:
'file_copy': ['mercurial/changelog.py', ' (', 'mercurial/hg.py', ')']
Therefore, the dict for the second and any subsequent items were not processed
through the corresponding showxxx() method, and the first item's data was
reused.
The 'extras' keyword regressed in 56b014c52204, and 'file_copies' regressed in
4e182fb53989 for other reasons. The common thread of things fixed by this seems
to be when a list of dicts are passed to the templatekw._hybrid class.
The search was introduced in 501e9ec9a85d. It might have been necessary back
then when using __file__ directly and frozen-ness wasn't considered. Now we
should know exactly where the templates can be found.
templates, help and locale data is normally stored as sub folders in the
directory containing the source of the mercurial module. In a frozen build they
live as sub folders next to 'hg.exe' and 'library.zip'.
These different kind of data were handled in different ways. Unify that by
introducing util.datapath. The value is computed from the environment and is
always used, so we just calculate the value on module load.
"diff" allows to embed changes in the target revision into template
output, even if the command itself doesn't take "--patch" option
Combination of "[committemplate]" configuration and "diff" template
function can achieve the feature like issue231 ("option to have diff
displayed in commit editor buffer")
http://bz.selenic.com/show_bug.cgi?id=231
For example, templating below can be used to add each "diff" output
lines "HG: " prefix::
{splitlines(diff) % 'HG: {line}\n'}
This patch implements "diff" not as "a template keyword" but as "a
template function" to take include/exclude patterns at runtime.
It allows to specify target files of command (by -I/-X command line
options) and "diff" separately.
Before this patch, predicates defined in "[revsetalias]" can't be used
in the query specified to template function "revset()", because:
- "revset()" uses "localrepository.revs()" to get query result, but
- "localrepository.revs()" passes "None" as "ui" to "revset.match()", then
- "revset.match()" can't recognize any alias predicates
To enable alias predicates to be used in "revset()" function, this
patch invokes "revset.match()" directly with "repo.ui".
This patch doesn't make "localrepository.revs()" pass "self.ui" to
"revset.match()", because this may be intentional implementation to
prevent alias predicates from shadowing built-in ones and breaking
functions internally using "localrepository.revs()".
Even if it isn't intentional one, the check for shadowing should be
implemented (maybe on default branch) before fixing it for safety.
This function allows returning only the nth "word" from a string. By default
a string is split as by Python's split() function default, but an optional
third parameter can also override what string the string is split by.
This function returns a string only if it starts with a given string.
It is particularly useful when combined with splitlines and/or used with
conditionals that fail when empty strings are passed in to take action
based on the contents of a line.
Previously the ifcontains revset was checking against the set using a pure
__contains__ check. It turns out the set was actually a list of
formatted strings meant for ui output, which meant the contains check failed if
the formatted string wasn't significantly different from the raw value.
This change makes it check against the raw data, prior to it being formatted.
This should correct an earlier couple of bad merges (5433856b2558 and
596960a4ad0d, now pruned) that accidentally brought in a change that had
been marked obsolete (244ac996a821).
Previously, if a template '{foo()}' was given, the buildfunc would not be able
to match it and hit a code path that would not return so it would error out
later in the templater stating that NoneType was not iterable. This patch makes
sure that a proper error is raised so that the user can be informed.
Tests have been updated.
Changeset 83ff877959a6 (released with 2.8.1) fixed "recursively
evaluate string literals as templates" problem (issue4102) by moving
the location of "string-escape"-ing from "tokenizer()" to
"compiletemplate()".
But some parts in template expressions below are not processed by
"compiletemplate()", and it may cause unexpected result.
- 'expr' of 'if(expr, then, else)'
- 'expr's of 'ifeq(expr, expr, then, else)'
- 'sep' of 'join(list, sep)'
- 'text' and 'style' of 'rstdoc(text, style)'
- 'text' and 'chars' of 'strip(text, chars)'
- 'pat' and 'repl' of 'sub(pat, repl, expr)'
For example, '\n' of "{join(extras, '\n')}" is not "string-escape"-ed
and treated as a literal '\n'. This breaks "Display the contents of
the 'extra' field, one per line" example in "hg help templates".
Just "string-escape"-ing on each parts above may not work correctly,
because inside expression of nested ones already applies
"string-escape" on string literals. For example:
- "{join(files, '\n')}" doesn't return "string-escape"-ed string, but
- "{join(files, if(branch, '\n', '\n'))}" does
To fix this problem, this patch does:
- introduce "rawstring" token and "runrawstring" method to handle
strings not to be "string-escape"-ed correctly, and
- make "runstring" method return "string-escape"-ed string, and
delay "string-escape"-ing until evaluation
This patch invokes "compiletemplate()" with "strtoken=exp[0]" in
"gettemplate()", because "exp[1]" is not yet evaluated. This code path
is tested via mapping ("expr % '{template}'").
In the other hand, this patch invokes it with "strtoken='rawstring'"
in "_evalifliteral()", because "t" is the result of "arg" evaluation
and it should be "string-escape"-ed if "arg" is "string" expression.
This patch doesn't test "string-escape"-ing on 'expr' of 'if(expr,
then, else)', because it doesn't affect the result.
Templating syntax allows nested expression to be specified as parts
below, but they are evaluated as a generator and don't work correctly.
- 'sep' of 'join(list, sep)'
- 'text' and 'chars' of 'strip(text, chars)'
In the former case, 'sep' returns expected string only for the first
separation, and empty one for the second or later, because the
generator has only one element.
In the latter case, templating is aborted by exception, because the
generator doesn't have 'strip()' method (as 'text') and can't be
passed as the argument to 'str.strip()' (as 'chars').
This patch applies "stringify()" on these sub expression to get string
correctly.
Changeset c84f81c3e120 (released with 2.8.1) fixed "recursively
evaluate string literals as templates" problem (issue4103) by
introducing "_evalifliteral()".
But some parts in template expressions below are still processed by
the combination of "compiletemplate()" and "runtemplate()", and may
cause same problem unexpectedly.
- 'init' and 'hang' of 'fill(text, width, init, hang)'
- 'expr' of 'sub(pat, repl, expr)'
- 'label' of 'label(label, expr)'
This patch processes them by "_evalifliteral()" instead of the
combination of "compiletemplate()" and "runtemplate()" to avoid
recursive evaluation of string literals completely.
Originally, the addition of the 'shorten' template function in 83d773060c28
would not consider pure integers for shortening. This patch considers two
simple cases: when the integer starts with zero (which is parsed by Mercurial
as a hash first) and when the integer is larger than the tip (obviously not a
rev).
Adds a template function that executes a revset and returns the list of
revisions as the result. It has the signature 'revset(query [, args...])'. The
args are optional and are applied to the query string using the standard
python string.format(args) pattern. This allows things like:
'{revset("parents({0})", rev)}' to produce the parents of each individual
commit in the log output. If no args are specified, the revset result is
cached for the duration of the templater; so it's better to not use args if
performance is a concern.
By itself, revset() can be used to print commit parents, print the common
ancestor of a commit with the main branch, etc.
It can be used with the ifcontains() function to do things like
'{ifcontains(rev, revset('.'), label(...), ...)}' to color the working copy
parent, to color certain branches, to color draft commits, etc.
Adds a template function with the signature 'ifcontains(item, set, then[,
else])'. It can be used to do things like '{ifcontains('.hgignore',
file_mods, label(...), ...)}' to color commits that edit the .hgignore file.
A future patch will add the revset() function which will combine with
ifcontains to allow us to color commits if they are in the revset.
Pure mercurial (i.e. without c extensions) does not support partialmatch() on
the revlog index, so we must fall back to use revlog._partialmatch() to handle
that case for us. The tests caught this.
We don't use revlog._partialmatch() for the normal case because it performs a
very expensive index iteration when the string being tested fails to find a
unique result via index.partialmatch(). It does this in order to filter
out hidden revs in hopes of the string being unique amongst non-hidden revs.
For the shortest(node) case, we'd prefer performance over worrying about
hidden revs.
Adds a pad template function with the following signature:
pad(text, width, fillchar=' ', right=False)
This uses the standard python ljust and rjust functions to produce a string
that is at least a certain width. This is useful for aligning variable length
strings in log output (like user names or shortest(node) output).
Adds a '{shortest(node)}' template function that results in the shortest hex node
that uniquely identifies the changeset at that time. The minimum length can be
specified as an optional second argument and defaults to 4.
This is useful for producing prettier log output, like so:
@ durham shortestnode
| 77cf template: add pad function for padding output
|
o durham
| b183 template: add shortestnode keyword
|
o pierre-yves @
| 6545 backout: add a message after backout that need manual commit
|
| o durham manifestcache
|/ 93f0 manifest cache
|
| o durham catperf
| | c765 cat: increase perf when catting single files
| |
| o durham
|/ 9c53 changectx: increase perf of walk function
|
Running `hg log --style compact` (or any other style) raised a traceback when
no template directory was there. Now there is a message:
Abort: style 'compact' not found
(available styles: no templates found, try `hg debuginstall` for more info)
There is no test because this would require to rename the template directory.
But this would influence other tests running in parallel. And when the test
would be aborted the wrong named directory would remain, especially a problem
when running with -l.
Before the templater got extended for nested expressions, it made
sense to decode string escapes across the whole string. Now we do it
on a piece by piece basis.
This allows using a template keyword on calls to the sub function and brings
the function inline with most other semantics of the other template functions.
During merge of branches, it is useful to compare merge results against
the two parents. This change adds this support to hgweb. To specify
which parent to compare to, use rev/12300:12345 where 12300 is a
parent changeset number. Two links are added to changeset web page so
that one can choose which parent to compare to.
If a template iterator is implemented with generator, the iterator is exhau=
sted
after we use it. This leads to undesired behavior in template. This chang=
e
converts a generator-based iterator to list-based iterator when template en=
gine
first detects a generator-based iterator. All future usages of iterator wi=
ll
use list instead.
These leaks may occur in environments that don't employ a reference
counting GC, i.e. PyPy.
This implies:
- changing opener(...).read() calls to opener.read(...)
- changing opener(...).write() calls to opener.write(...)
- changing open(...).read(...) to util.readfile(...)
- changing open(...).write(...) to util.writefile(...)
Before this, to display file copies in templates, you had to write something
like {file_copies%filecopy}. For some reason, the {file_copy} subtemplate was
used by default but not defined by default in changeset_templater, while styles
were already using it. Here we define {file_copy} in changeset_templater, and
change the templater to handle formatting strings like {file_copies%filecopy}
with already expanded keys (in this case {file_copies}), for backward
compatibility.
Currently hgweb is not streaming its output -- it accumulates the
entire response before sending it. This patch restores streaming
behaviour. To avoid having to synchronously write many tiny fragments,
this patch also adds buffering to the template generator. Local
testing of a fetch of a 100,000 line file with wget produces a slight
slowdown overall (up from 6.5 seconds to 7.2 seconds), but instead of
waiting 6 seconds for headers to arrive, output begins immediately.
- make renamelink take a context
- pass current node to shortlog page rather than tip
- pass user/desc/date/rename to filediff
- improve template error reporting for foo%bar
