Closes#29.
Now testing function can return ‘Either [Message] a’ so it can construct
full list of error messages. This may be useful in some cases when
tokens are more complex than simple characters.
Multi-character operators should use ‘try’ in order to be reported
correctly (as “operator”). I've mentioned it in doc-string of
‘makeExprParser’.
It's tempting to include ‘try’ directly in expression parsing code, but
following general spirit of Parsec toward ‘try’, I think current
solution is the best.
Various languages may vary in how hexadecimal and octal literals should
be prefixed. Following the spirit of the new lexer we leave this to
programmer to decide.
Eliminated ‘Text.Megaparsec.Language’ module because at this point it is
clear that already existing definitions are of little use in
Megaparsec. I started writing “default” language definition in
‘Text.Megaparsec.Lexer’.
At this point it should be possible to parse languages where indentation
matters, although we will need to provide more helpers to make it
easier.
Obviously order does matter here, since ‘Monoid’ instance for ‘Hints’ is
derived from [], so (<>) is the same as (++) and we should be careful
to keep things in the right order.
These parsers are considered deprecated:
* ‘chainl’
* ‘chainl1’
* ‘chainr’
* ‘chainr1’
* ‘sepEndBy’
* ‘sepEndBy1’
Apart from this, the commit includes various cosmetic changes in
module ‘Text.Megaparsec.Combinator’.
The following functions and data types have been renamed:
* ‘permute’ → ‘makePermParser’
* ‘buildExpressionParser’ → ‘makeExprParser’
* ‘GenLanguageDef’ → ‘LanguageDef’
* ‘GenTokenParser’ → ‘Lexer’
* ‘makeTokenParser’ → ‘makeLexer’
The improved error messages in Megaparsec are quite sensitive to how
parsers are written, which parts of parser are labeled, etc. Current
implementation of token parsers in ‘Text.Megaparsec.Token’ is written
without this in mind. We will improve the module later, for now let us
rewrite/simplify some parts to avoid failing tests.
If ‘x’ in ‘x >>= y’ consumes input but produces some hints, we should
accumulate them nonetheless. Why it's important can be demonstrated by
the following test:
many (char 'a') >> many (char 'b') >> eof
This should fail on input "ac" with the following message:
parse error at line 1, column 2:
unexpected 'c'
expecting 'a', 'b' or end of input
As you can see even though parser ‘many (char 'a')’ consumed input, its
hits may be useful later.
* Removed ‘optionMaybe’ parser, because ‘optional’ from
‘Control.Applicative’ does the same thing.
* Renamed ‘tokenPrim’ → ‘token’, removed old ‘token’, because
‘tokenPrim’ is more general and ‘token’ is little used.
* Fixed bug with ‘notFollowedBy’ always succeeded with parsers that
don't consume input, see #6.
* Hint system introduced that greatly improved quality of error messages
and made code of ‘Text.Megaparsec.Prim’ a lot clearer.
The improvements affected other modules too:
* Some parsers from ‘Text.Megaparsec.Combinators’ now live in
‘Text.Megaparsec.Prim’.
* Hint system improved error messages, so I needed to rewrite test for
‘Text.Megaparsec.Char.eol’, since it's error messages are very
intelligent now and cannot be emulated by ‘newline’ and ‘crlf’ parsers
used separately.
* Test for Bug9 from old-tests is passed successfully again.
This parser can be told to parse from ‘m’ to ‘n’ occurrences of some
thing. Old parser ‘count’ is now named ‘count’' and defined in terms of
that more powerful one.
This commit also reorders functions in module
‘Text.Megaparsec.Combinator’ and everywhere where the functions are
listed. The same order is used everywhere.
Added new character parsers in ‘Text.Megaparsec.Char’:
* ‘controlChar’
* ‘printChar’
* ‘markChar’
* ‘numberChar’
* ‘punctuationChar’
* ‘symbolChar’
* ‘separatorChar’
* ‘asciiChar’
* ‘latin1Char’
* ‘charCategory’
Renamed some parsers:
‘spaces’ → ‘space’
‘space’ → ‘spaceChar’
‘lower’ → ‘lowerChar’
‘upper’ → ‘upperChar’
‘letter’ → ‘letterChar’
‘alphaNum’ → ‘alphaNumChar’
‘digit’ → ‘digitChar’
‘octDigit’ → ‘octDigitChar’
‘hexDigit’ → ‘hexDigitChar’
Descriptions of old parsers have been updated to accent some
Unicode-specific moments. For example, old description of ‘letter’
stated that it parses letters from “a” to “z” and from “A” to “Z”. This
is wrong, since it used ‘Data.Char.isAlpha’ predicate internally and
thus parsed many more characters.
New tests shows that I had wrong assumption about workings of this
particular function. This is not a problem, though, complete test-suite
will eliminate this sort of nuisance soon.
