enso/lib/rust/parser/debug/tests
Kaz Wesley 0de490be24
Text literals: Accept unpaired-surrogate escape codes. (#9731)
* Text literals: Accept unpaired-surrogate escape codes.

Unpaired surrogates are not allowed by Unicode, but they occur in practice
because many systems accept them; for example, they may be present in filenames
on Windows (which are otherwise constrained to UTF-16).

Programs written in Enso should be able to work with them, if only because they
represent edge cases that should be tested when converting encodings and at
other system boundaries.

- Generalize the representation of interpreted-text-escapes in the lexer, so
  that we are not tied to the strict Unicode of Rust's `str`.
- Move some doc-comment code from the parser to test utilities.
- Simplify token serialization.
2024-04-18 09:21:05 -04:00
..
metadata Ensure all spans have document offsets (#8039) 2023-10-19 12:36:42 +00:00
parse.rs Text literals: Accept unpaired-surrogate escape codes. (#9731) 2024-04-18 09:21:05 -04:00