Commit Graph

60 Commits

Author SHA1 Message Date
Yuya Nishihara
dcc07e5503 doctest: use print_function and convert bytes to unicode where needed 2017-09-03 14:56:31 +09:00
Yuya Nishihara
a71f259bd2 doctest: bulk-replace string literals with b'' for Python 3
Our code transformer can't rewrite string literals in docstrings, and I
don't want to make the transformer more complex.
2017-09-03 14:32:11 +09:00
Yuya Nishihara
f0fc1531a4 parser: stabilize output of prettyformat() by using byte-safe repr()
The format of leaf nodes is slightly changed so they look more similar to
internal nodes.
2017-09-03 21:17:25 +09:00
Yuya Nishihara
27c162ca6b parser: add helper function to test if pattern matches parsed tree
This function will be used as follows:

  match('ancestors(_) and not ancestors(_)', x)

See the next patch for details.
2016-02-17 21:31:09 +09:00
Yuya Nishihara
ee31d80d0f parser: add helper function that constructs parsed tree from template
This function will be used as follows:

  build('only(_, _)', x, y)

See the next patch for details.
2016-02-17 21:30:04 +09:00
Yuya Nishihara
e70ac1c73a parser: preserve order of keyword arguments
This helps building dict(key1=value1, ...) in deterministic way.
2017-04-09 11:58:27 +09:00
Yuya Nishihara
33d96b70bc parser: extend buildargsdict() to support arbitrary number of **kwargs
Prepares for adding dict(key1=value1, ...) template function. More tests
will be added later.
2017-04-03 22:07:09 +09:00
Yuya Nishihara
2b723f40bc parser: verify excessive number of args excluding kwargs in buildargsdict()
This makes the next patch slightly simpler. We don't need to check the
excessive number of keyword arguments since unknown and duplicated kwargs
are rejected.
2017-04-08 20:07:37 +09:00
Yuya Nishihara
dc88179a4e util: wrap s.decode('string_escape') calls for future py3 compatibility 2017-03-17 23:42:46 +09:00
Augie Fackler
ef815f4375 parser: use %d instead of %s for interpolating error position
Error position is an int, so we should use %d instead of %s. Fixes
failures on Python 3.
2017-03-12 00:44:59 -05:00
Yuya Nishihara
b1575d5948 parser: extend buildargsdict() to support variable-length positional args
This can simplify the argument parsing of followlines(). Tests are added by
the next patch.
2017-01-09 15:25:52 +09:00
Yuya Nishihara
3d36c04638 parser: make buildargsdict() precompute position where keyword args start
This prepares for adding *varargs support. See the next patch.
2017-01-09 15:15:21 +09:00
Mads Kiilerich
38cb771268 spelling: fixes of non-dictionary words 2016-10-17 23:16:55 +02:00
Yuya Nishihara
e316e79cfe parser: remove unused binding parameter from suffix action
Because a suffix action never takes subsequent tokens, it should have
no binding strength nor closing character. I've tried if this value could
be used to resolve infix/suffix ambiguity of x^:y, but it appears not. So
I decided to resend this patch.
2015-07-05 21:11:19 +09:00
Yuya Nishihara
10fda3d262 parser: shorten prefix of alias parsing errors
These messages seemed to be a bit long. We should try making them fit to
80-col console.
2016-04-17 12:31:06 +09:00
Yuya Nishihara
77a605117c parser: rephrase "'$' not for alias arguments" message
Say which symbol caused the error. The word "alias" is removed since these
messages are prefixed by "failed to parse ... revset alias "...":".
2016-04-17 12:20:57 +09:00
Yuya Nishihara
85d1a67355 parser: factor out _trygetfunc() that extracts function name and arguments
This provides a customization point for templater. In templater, there are
two ways to call a unary function: func(x) and x|func. They are processed
differently in templater due to historical reasons, but they should be
handled in the same way while expanding aliases. In short, x|func should be
processed as syntactic sugar for func(x).

_funcnode and _getlist() are replaced by _trygetfunc().
2016-03-29 17:27:34 +09:00
Yuya Nishihara
f873b2cea1 parser: make _getalias() return (alias, pattern-args) pair
This allows us to factor out a function that extracts a function (name, args)
pair. See the next patch for why.
2016-03-29 17:21:11 +09:00
Yuya Nishihara
22818c3975 parser: drop redundant comparison between alias declaration tree and pattern
Since _getalias() explicitly tests the type and name of the pattern tree, we
don't need to compare "a.tree == tree" for 'symbol', and "a.tree == tree[:2]"
for 'func', where tree is either ('symbol', name) or ('func', ('symbol', name)).

This change helps implementing better handling of template aliases. See the
subsequent patches for details.

The alias.tree field is removed as it is no longer used.
2016-03-29 16:50:16 +09:00
Yuya Nishihara
4d9561138b revset: rename findaliases() to expandaliases()
This function returns a full tree of alias expansion applied, which sounds
different from what "findaliases" would do.
2016-02-29 22:58:15 +09:00
Yuya Nishihara
e178513a20 parser: add short comment how aliases are expanded in phases 2016-03-29 16:30:59 +09:00
Yuya Nishihara
8c48f711f6 parser: reorder alias expansion routine to return early
I think it improves readability to move trivial cases first, and unindent
blocks.
2016-03-29 16:19:31 +09:00
Yuya Nishihara
45e9c21bc0 parser: move functions that process alias expansion to rule-set class
They will be commonly used by revset and templater. It isn't easy to understand
how _expand() works, so I'll add comments by a follow-up patch.

The local variable 'alias' is renamed to 'a' to avoid shadowing the global
'alias' class.
2016-02-29 22:15:44 +09:00
Yuya Nishihara
a98783fa06 parser: extract helper that creates a dict of aliases
This will be common between revset and templater.

The local variable 'alias' is renamed to 'a' to avoid shadowing the global
'alias' class.
2016-02-29 19:24:15 +09:00
Yuya Nishihara
53c5cbd213 parser: construct alias object by rule-set class
It was odd that the revsetalias did the whole parsing stuff in __init__().
Instead, this patch adds a factory function to the aliasrules class, and
makes the alias (= revsetalias) class a plain-old value object.
2016-02-29 18:33:30 +09:00
Yuya Nishihara
2e16bca7bf parser: unify parser function of alias declaration and definition
We no longer have to keep them separately.
2016-03-29 00:08:25 +09:00
Yuya Nishihara
5392dbd7d3 parser: move alias definition parser to common rule-set class
The original _parsealiasdefn() function is split into common _builddefn()
and revset-specific _parsealiasdefn(). revset._relabelaliasargs() is removed
as it is no longer used.

The doctests are ported by using the dummy parse().
2016-02-29 18:10:07 +09:00
Yuya Nishihara
188dee35c2 parser: move _relabelaliasargs() to common rule-set class
This has no doctest because it will be covered by _builddefn() introduced
by the next patch.

revset._relabelaliasargs() will be removed soon.
2016-02-29 18:00:51 +09:00
Yuya Nishihara
6b447db164 parser: move alias declaration parser to common rule-set class
The original _parsealiasdecl() function is split into common _builddecl()
and revset-specific _parsealiasdecl(). And the original _parsealiasdecl()
call is temporarily replaced by rules._builddecl(), which should be eliminated
later.

The doctests are mostly ported by using the dummy parse(), but the test for
'foo bar' is kept in _parsealiasdecl() as it checks if "pos != len(decl)" is
working. Also, 'foo($1)' test is added to make sure the alias tokenizer can
handle '$1' symbol, which is the only reason why we need _parsealiasdecl().
2016-02-29 17:54:03 +09:00
Yuya Nishihara
c51267a286 parser: add stub class that will host alias parsing and expansion
This class will keep syntax rules that are necessary to parse and expand
aliases. The implementations will be extracted from the revset module. In
order to make the porting easier, this class keeps parsedecl and parsedefn
separately, which will be unified later. Also, getlist and funcnode will
be refactored by future patches for better handling of the template aliases.

The following public functions will be added:

  aliasrules.build(decl, defn) -> aliasobj
    parse decl and defn into an object that keeps alias name, arguments
    and replacement tree.
  aliasrules.buildmap(aliasitems) -> aliasdict
    helper to build() a dict of alias objects from a list of (decl, defn)
  aliasrules.expand(aliasdict, tree) -> tree
    expand aliases in tree recursively

Because these functions aren't introduced by this series, there would remain
a few wrapper functions in the revset module. These ugly wrappers should be
eliminated by the next series.

This class is considered an inheritable namespace, which will host only
class/static methods. That's because it won't have no object-scope variables.
I'm not a big fan of using class as a syntax sugar, but I admit it can improve
code readability at some level. So let's give it a try.
2016-04-03 16:55:23 +09:00
Yuya Nishihara
1ad58e18af parser: move parsererrordetail() function from revset module
This will be used by common alias functions introduced by future patches.
2016-02-29 17:02:56 +09:00
Yuya Nishihara
9e9d73bc11 parser: move unescape helper from templater
revset and fileset have a similar problem, so let's make it a common helper
function.
2015-09-10 23:25:10 +09:00
Gregory Szorc
a87fcbe79d parser: use absolute_import 2015-08-08 19:44:05 -07:00
Yuya Nishihara
6cb504f763 parser: take suffix action if no infix action is defined
If no infix action is defined, a suffix action isn't ambiguous, so it should
be taken no matter if the next token can be an operand. This is exactly the
same flow as prefix/primary handling.

This change has no effect now because all suffix tokens have infix actions.
2015-07-06 22:01:41 +09:00
Yuya Nishihara
43ae52143a parser: reorder infix/suffix handling to be similar to prefix/primary flow
It can be exactly the same flow as the prefix/primary handling. A suffix
action is accepted only if the next token never starts new term.
2015-07-06 21:55:55 +09:00
Yuya Nishihara
cdaa060788 parser: resolve ambiguity where both prefix and primary actions are defined
If both actions are defined, a primary-expression action is accepted only if
the next token never starts new term. For example,

  parsed as primary expression:
  ":"   # next token 'end' has no action
  "(:)" # next token ')' has no action
  ":+y" # next token '+' is infix operator

  parsed as prefix operator:
  ":y"  # next token 'y' is primary expression
  ":-y" # next token '-' is prefix operator

This is mostly the same resolution as the infix/suffix rules.
2015-07-05 12:09:27 +09:00
Yuya Nishihara
b4caf94446 parser: separate actions for primary expression and prefix operator
This will allow us to define both a primary expression, ":", and a prefix
operator, ":y". The ambiguity will be resolved by the next patch.

Prefix actions in elements table are adjusted as follows:

  original prefix      primary  prefix
  -----------------    -------- -----------------
  ("group", 1, ")") -> n/a      ("group", 1, ")")
  ("negate", 19)    -> n/a      ("negate", 19)
  ("symbol",)       -> "symbol" n/a
2015-07-05 12:02:13 +09:00
Yuya Nishihara
718c23ab58 parser: extract function that tests if next token may start new term
Future patches will separate primary expression and prefix operator actions.
This function will be used to resolve ambiguity of them.

This is a step to remove the old-style revexpr parser. We need both ":" and
":y" operators for backward compatibility.
2015-07-05 11:54:14 +09:00
Yuya Nishihara
c84ce8d445 parser: factor out function that parses right-hand side of prefix/infix ops
These two had common pattern. The significant difference was just a result
expression:

  prefix: (op-name, rhs)
  infix:  (op-name, lhs, rhs)
2015-07-05 18:09:15 +09:00
Yuya Nishihara
8905d3395c parser: remove unused parameter 'pos' from _match()
This backs out 58030471e557. The issue spotted by that changeset was addressed
earlier by a50a014eb0ba.
2015-07-05 17:50:35 +09:00
Yuya Nishihara
4645c24be5 parser: fill invalid infix and suffix actions by None
This can simplify the expansion of (prefix, infix, suffix) actions.
2015-07-05 11:17:22 +09:00
Yuya Nishihara
d1927459b6 revset: add function to build dict of positional and keyword arguments
Keyword arguments will be convenient for functions that will take more than
one optional or boolean flags. For example,

  file(pattern[, subrepos=false])
  subrepo([[pattern], status])

Because I don't think all functions should accept key=value syntax, getkwargs()
does not support variadic functions such as 'ancestor(*changeset)'.

The core logic is placed in the parser module because keyword arguments will
be more useful in the templater, where functions take more options. Test cases
will be added by the next patch.
2015-06-27 17:25:01 +09:00
Yuya Nishihara
5747b2c8e6 parser: update documentation about tokenizer and elements 2015-06-21 00:56:09 +09:00
Yuya Nishihara
fe462ed8ac parser: accept iterator of tokens instead of tokenizer function and program
This can simplify the interface of parse() function. Our tokenizer tends to
have optional arguments other than the message to be parsed.

Before this patch, the "lookup" argument existed only for the revset, and the
templater had to pack [program, start, end] to be passed to its tokenizer.
2015-06-21 00:49:26 +09:00
Yuya Nishihara
ca51f820ee parser: add helper to reduce nesting of chained infix operations
This will be used to avoid stack overflow caused by chained 'or' operations
in revset.
2015-04-26 18:05:23 +09:00
Yuya Nishihara
1ec56bf564 parser: extract closure of prettyformat() to a top-level function
There was no capture until I added 'leafnodes' argument.
2015-05-06 10:17:41 +09:00
Yuya Nishihara
09759e9679 parser: move prettyformat() function from revset module
I want to use it in doctests that I'll add by future patches. Also, it can
be used in "hg debugfileset" command.
2015-04-26 22:20:03 +09:00
Pierre-Yves David
5ec3f86b27 parsers: use 'next' instead of try/except
This get rid of another StopIteration abomination. The change in self.current
value is supposed to not matter as nobody should be calling '_advance' after
that (as per Matt wisdom).
2015-05-18 12:27:15 -05:00
Matt Mackall
39f29cd5bc parser: allow passing a lookup function to a tokenizer
This will allow us to dynamically handle hyphenated symbols in revsets.
2014-03-18 17:17:23 -05:00
timeless@mozdev.org
023e023a87 en-us: labeled 2012-08-17 13:58:18 -07:00