@@ -190,14 +190,61 @@ We can see how Idris2 does it here
190190 init : IndentInfo
191191 init = 0
192192
193+EndInput Token
194+--------------
193195
196+So far, the parser will return a successful result even if the full input
197+is not consumed. To ensure that the top level syntax is fully matched we
198+add a ``EndInput`` token to indicate the last token.
194199
200+``EndInput`` is added to the other tokens like this:
195201
202+.. code-block:: idris
203+
204+ data ExpressionToken = Number Integer
205+ | Operator String
206+ | OParen
207+ | CParen
208+ | Comment String
209+ | EndInput
196210
211+A rule to consume this token is added:
197212
213+.. code-block:: idris
198214
215+ eoi : Rule Integer
216+ eoi = terminal (\x => case tok x of
217+ EndInput => Just 0
218+ _ => Nothing)
199219
220+Instead of using ``expr`` at the top level of the syntax we can now define
221+``exprFull`` as shown here. This will make sure that only when ``EndInput``
222+is consumed will the parse be successful:
200223
224+.. code-block:: idris
201225
226+ exprFull : Rule Integer
227+ exprFull = expr <* eoi
228+
229+The following code makes sure ``EndInput`` is added to the end of the token
230+list:
231+
232+.. code-block:: idris
233+
234+ processWhitespace : (List (TokenData ExpressionToken), Int, Int, String)
235+ -> (List (TokenData ExpressionToken), Int, Int, String)
236+ processWhitespace (x,l,c,s) = ((filter notComment x)++
237+ [MkToken l c EndInput],l,c,s) where
238+ notComment : TokenData ExpressionToken -> Bool
239+ notComment t = case tok t of
240+ Comment _ => False
241+ _ => True
242+
243+All we have to do now is use ``exprFull`` instead of ``expr``:
244+
245+.. code-block:: idris
246+
247+ calc : String -> Either (ParseError (TokenData ExpressionToken))
248+ (Integer, List (TokenData ExpressionToken))
249+ calc s = parse exprFull (fst (processWhitespace (lex expressionTokens s)))
202250
203−