git.iko.soy/Idris-dev
on master help sign in
Implement Left Recursion Elimination
eb780f7martinbaker
docs/parserLibrary/parser.rst+134−4
compare with master
unifiedsplit
@@ -430,10 +430,10 @@ then it will return 'Right' (to indicate success)
430Infix Notation
430Infix Notation
431--------------
431--------------
432
432
433−So far we have implemented a prefix notation for operators (like this: '+ expr expr')
433+So far we have implemented a prefix notation for operators (like this:
434−but the aim is to implemented an infix notation (like this: 'expr + expr'). To do
434+'+ expr expr') but the aim is to implemented an infix notation (like this:
435−this we must be able to deal with potentially infinite data structures (see Codata Types
435+'expr + expr'). To do this we must be able to deal with potentially
436−here :ref:`sect-typefuns`).
436+infinite data structures (see Codata Types here :ref:`sect-typefuns`).
437
437
438First alter the grammar to have infix operations:
438First alter the grammar to have infix operations:
439
439
@@ -479,4 +479,134 @@ First alter the grammar to have infix operations:
479
479
480However, if this was run, the code would not terminate.
480However, if this was run, the code would not terminate.
481
481
482+Implement Left Recursion Elimination
483+------------------------------------
484+
485+Left factoring, like this, is a general problem.
486+
487+If we have a rule like this:
488+
489+.. code-block:: idris
490+
491+ A -> (x<*>y) <|> (x<*>z)
492+
493+If 'x<*>y' fails but 'x<*>z' would succeed a problem is that, 'x<*>y' has already consumed 'x', so now 'x<*>z' will fail.
494+
495+so we could write code to backtrack. That is 'try' 'x<*>y' without consuming so that, if the first token succeeds but the following tokens fail, then the first tokens would not be consumed.
496+
497+Another approach is left factoring:
498+
499+Left Factoring
500+--------------
501+
502+Replace the rule with two rules (that is we add a non-terminal symbol) so for example, instead of:
503+
504+.. code-block:: idris
505+
506+ A -> (x<*>y) <|> (x<*>z)
507+
508+we add an extra rule to give:
509+
510+.. code-block:: idris
511+
512+ A -> x<*>N
513+ N -> y <|> z
514+
515+That is we convert a general context-free grammar to a LL(1) grammar. Although not every context-free grammar can be converted to a LL(1) grammar.
516+
517+This still does not solve the infinite recursion issue and there is another problem: the precedence of the operators +, - and * is not explicit.
518+
519+To resolve this we can alter the example like this:
520+
521+.. code-block:: idris
522+
523+ paren : Rule Integer -> Rule Integer
524+ paren exp = openParen *> exp <* closeParen
525+
526+ addInt : Integer -> Integer -> Integer
527+ addInt a b = a+b
528+
529+ subInt : Integer -> Integer -> Integer
530+ subInt a b = a-b
531+
532+ multInt : Integer -> Integer -> Integer
533+ multInt a b = a*b
534+
535+ partial
536+ expr : Rule Integer
537+
538+ partial
539+ exprAtom : Rule Integer
540+ exprAtom = intLiteral <|> (paren expr)
541+
542+ partial
543+ expr1 : Rule Integer
544+ expr1 = map multInt exprAtom <*> ((op "*") *> exprAtom)
545+
546+ partial
547+ exprMult : Rule Integer
548+ exprMult = expr1 <|> exprAtom
549+
550+ partial
551+ expr2 : Rule Integer
552+ expr2 = map addInt exprMult <*> ((op "+") *> exprMult)
553+
554+ partial
555+ exprAdd : Rule Integer
556+ exprAdd = expr2 <|> exprMult
557+
558+ partial
559+ expr3 : Rule Integer
560+ expr3 = map subInt exprAdd <*> ((op "-") *> expr)
561+
562+ expr = expr3 <|> exprAdd
563+
564+As the following tests show, this now handles infix operators and precedence.
565+
566+.. list-table::
567+
568+ * - For example '*' is an infix operator:
569+
570+ - .. code-block:: idris
571+
572+ *parserExInfix> test "2*3"
573+ Right (6,
574+ []) : Either (ParseError (TokenData ExpressionToken))
575+ (Integer, List (TokenData ExpressionToken))
576+
577+ * - Check that atomic number literals work:
578+
579+ - .. code-block:: idris
580+
581+ *parserExInfix> test "2"
582+ Right (2,
583+ []) : Either (ParseError (TokenData ExpressionToken))
584+ (Integer, List (TokenData ExpressionToken))
585+
586+ * - Check that '*' has a higher precedence than '+'.
587+
588+ - .. code-block:: idris
589+
590+ *parserExInfix> test "2+3*4"
591+ Right (14,
592+ []) : Either (ParseError (TokenData ExpressionToken))
593+ (Integer, List (TokenData ExpressionToken))
594+
595+ * - Also '*' has a higher precedence than '+' when the order is reversed.
596+
597+ - .. code-block:: idris
598+
599+ *parserExInfix> test "3*4+2"
600+ Right (14,
601+ []) : Either (ParseError (TokenData ExpressionToken))
602+ (Integer, List (TokenData ExpressionToken))
603+
604+ * - Also precedence can be overridden by parenthesis.
605+
606+ - .. code-block:: idris
607+
608+ *parserExInfix> test "(2+3)*4"
609+ Right (20,
610+ []) : Either (ParseError (TokenData ExpressionToken))
611+ (Integer, List (TokenData ExpressionToken))
482
612