enso/lib/rust/parser/doc-parser
Kaz Wesley 0de490be24
Text literals: Accept unpaired-surrogate escape codes. (#9731)
* Text literals: Accept unpaired-surrogate escape codes.

Unpaired surrogates are not allowed by Unicode, but they occur in practice
because many systems accept them; for example, they may be present in filenames
on Windows (which are otherwise constrained to UTF-16).

Programs written in Enso should be able to work with them, if only because they
represent edge cases that should be tested when converting encodings and at
other system boundaries.

- Generalize the representation of interpreted-text-escapes in the lexer, so
  that we are not tied to the strict Unicode of Rust's `str`.
- Move some doc-comment code from the parser to test utilities.
- Simplify token serialization.
2024-04-18 09:21:05 -04:00
..
src Text literals: Accept unpaired-surrogate escape codes. (#9731) 2024-04-18 09:21:05 -04:00
Cargo.toml Reduce parser dependencies (#9671) 2024-04-11 00:27:19 -04:00