github/semantic
1
1
Fork 0
mirror of https://github.com/github/semantic.git synced 2025-01-03 04:51:57 +03:00
Code Issues Projects Releases Wiki Activity

Extract word & punctuation & space into atoms.

Browse Source
This commit is contained in:
Rob Rix 2015-12-14 16:32:24 -05:00
parent 2ba26d8a47
commit cf072208fd
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
src/Range.hs
View File

@ -25,6 +25,9 @@ rangesAndWordsFrom startIndex string =
Just parsed -> skipAndContinue parsed Just parsed -> skipAndContinue parsed
Nothing -> [] Nothing -> []
where where
word = parse isWord string
punctuation = parse (not . isWordOrSpace) string
space = parse Char.isSpace string
takeAndContinue (parsed, rest) = (Range startIndex $ startIndex + length parsed, parsed) : rangesAndWordsFrom (startIndex + length parsed) rest takeAndContinue (parsed, rest) = (Range startIndex $ startIndex + length parsed, parsed) : rangesAndWordsFrom (startIndex + length parsed) rest
skipAndContinue (parsed, rest) = rangesAndWordsFrom (startIndex + length parsed) rest skipAndContinue (parsed, rest) = rangesAndWordsFrom (startIndex + length parsed) rest
parse predicate string = case span predicate string of parse predicate string = case span predicate string of