The input string `(prn "abc` caused the nim implementaion (from step1
onwards) to enter an endless loop in tokenize (which eventually caused
out-of-memory).
The fix verfies that indeed the regex matches a non-empty substring
before adding that substring as a token.