2020-05-18 15:59:07 +03:00
|
|
|
module Language.JSON.String.Lexer
|
|
|
|
|
|
Fix import loading
This was taking too long, and adding too many things, because it was
going too deep in the name of having everything accessible at the REPL
and for the compiler. So, it's done a bit differently now, only chasing
everything on a "full" load (i.e., final load at the REPL)
This has some effects:
+ As systems get bigger, load time gets better (on my machine, checking
Idris.Main now takes 52s from scratch, down from 76s)
+ You might find import errors that you didn't previously get, because
things were being imported that shouldn't have been. The new way is
correct!
An unfortunate effect is that sometimes you end up getting "undefined
name" errors even if you didn't explicitly use the name, because
sometimes a module uses a name from another module in a type, which then
gets exported, and eventually needs to be reduced. This mostly happens
because there is a compile time check that should be done which I
haven't implemented yet. That is, public export definitions should only
be allowed to use names that are also public export. I'll get to this
soon.
2020-05-27 17:49:03 +03:00
|
|
|
import Data.Nat
|
2020-05-18 15:59:07 +03:00
|
|
|
import Language.JSON.String.Tokens
|
|
|
|
|
import Text.Lexer
|
|
|
|
|
|
|
|
|
|
%default total
|
|
|
|
|
|
|
|
|
|
export
|
|
|
|
|
quo : Lexer
|
|
|
|
|
quo = is '"'
|
|
|
|
|
|
|
|
|
|
export
|
|
|
|
|
esc : Lexer -> Lexer
|
|
|
|
|
esc = escape '\\'
|
|
|
|
|
|
|
|
|
|
private
|
|
|
|
|
unicodeEscape : Lexer
|
|
|
|
|
unicodeEscape = esc $ is 'u' <+> count (exactly 4) hexDigit
|
|
|
|
|
|
|
|
|
|
private
|
|
|
|
|
simpleEscape : Lexer
|
|
|
|
|
simpleEscape = esc $ oneOf "\"\\/bfnrt"
|
|
|
|
|
|
|
|
|
|
private
|
|
|
|
|
legalChar : Lexer
|
|
|
|
|
legalChar = non (quo <|> is '\\' <|> control)
|
|
|
|
|
|
|
|
|
|
private
|
|
|
|
|
jsonStringTokenMap : TokenMap JSONStringToken
|
|
|
|
|
jsonStringTokenMap = toTokenMap $
|
|
|
|
|
[ (quo, JSTQuote)
|
|
|
|
|
, (unicodeEscape, JSTUnicodeEscape)
|
|
|
|
|
, (simpleEscape, JSTSimpleEscape)
|
|
|
|
|
, (legalChar, JSTChar)
|
|
|
|
|
]
|
|
|
|
|
|
|
|
|
|
export
|
|
|
|
|
lexString : String -> Maybe (List JSONStringToken)
|
|
|
|
|
lexString x = case lex jsonStringTokenMap x of
|
|
|
|
|
(toks, _, _, "") => Just $ map TokenData.tok toks
|
|
|
|
|
_ => Nothing
|