Commit Graph

25 Commits

Author SHA1 Message Date
Kaz Wesley
9fd1ab9092
Parser TS bindings (#7881)
Generate TS bindings and lazy deserialization for the parser types.

# Important Notes
- The new API is imported into `ffi.ts`, but not yet used.
- I have tested the generated code in isolation, but cannot commit tests as we are not currently able to load WASM modules when running in `vitest`.
2023-10-11 13:04:38 +00:00
Pavel Marek
c22928ecc2
Implement private modules (#7840)
Adds the ability to declare a module as *private*. Modifies the parser to add the `private` keyword as a reserved keyword. All the checks for private modules are implemented as an independent *Compiler pass*. No checks are done at runtime.

# Important Notes
- Introduces new keyword - `private` - a reserved keyword.
- Modules that have `private` keyword as the first statement are declared as *private* (Project private)
- Public module cannot have private submodules and vice versa.
- This would require runtime access checks
- See #7088 for the specification.
2023-10-04 10:33:10 +00:00
Kaz Wesley
e3de68858c
Eager macro resolution. (#7711)
Resolve macros eagerly. Improves performance; allows parser to handle arbitrarily-long lines (fixes #7691).

# Important Notes
- A new utility, `lib/rust/parser/debug/tools/parse_all_enso_files.sh`, supports comparing ASTs parsed with different versions of the parser. This tool has been used to verify that this refactor doesn't change the result of parsing any standard library or test file.
2023-09-19 14:26:42 +00:00
Kaz Wesley
7a934cadcc
Parser: Fix handling of syntax errors such as x =- 2 (#7399)
* Parser: Correctly handle #7335 syntax errors.
2023-07-27 08:25:45 -07:00
Kaz Wesley
75fda33837
Fix precedence of -1.x (#5830)
Fixes #5826.

# Important Notes
- Change frontend representation of negation.
- Fix a precedence issue: The `.` operators in -1.x and -1.2 must have different precedences.
- Remove a no-longer-needed special case from backend translation.
- Add tests for this case after all translations.
2023-03-17 18:53:34 +00:00
Kaz Wesley
e171fba301
New documentation parser (#5917)
Implement new Enso documentation parser; remove old Scala Enso parser.

Performance: Total time parsing documentation is now ~2ms.

# Important Notes
- Doc parsing is now done only in the frontend.
- Some engine tests had never been switched to the new parser. We should investigate tests that don't pass after the switch: #5894.
- The option to run the old searcher has been removed, as it is obsolete and was already broken before this (see #5909).
- Some interfaces used only by the old searcher have been removed.
2023-03-15 15:43:51 +00:00
Kaz Wesley
d1af25793a
Port graph editor to new AST (#4113)
Use the Rust parser rather than the Scala parser to parse Enso code in the IDE.

Implements:
- https://www.pivotaltracker.com/story/show/182975925
- https://www.pivotaltracker.com/story/show/182988419
- https://www.pivotaltracker.com/story/show/182970096
- https://www.pivotaltracker.com/story/show/182973659
- https://www.pivotaltracker.com/story/show/182974161
- https://www.pivotaltracker.com/story/show/182974205

There is additional functionality needed before the transition is fully-completed, however I think it's time for this to see review and testing, so I've opened separate issues. In rough order of urgency (these issues are also linked from the corresponding disabled tests):
- #5573
- #5571
- #5572
- #5574

# Important Notes
The implementation is based partly on translation, and partly on new analysis. Method- and operator-related shapes are translated to the old `Ast` variants, so that all the analysis applied to them doesn't need to be ported at this time. Everything else (mostly "macros" in the old AST) is implemented with new analysis.
2023-02-10 18:05:40 +00:00
Kaz Wesley
4f70bcc5ff
Fix unhandled macro match failure case (#5056) (#5599)
Fix an unhandled case in macro resolution
2023-02-09 23:00:29 +00:00
Michael Mauderer
349cc210e0
Bump rustc to nightly-2023-01-12 (#4053)
Bump rustc nightly-2022-08-30 and fix new errors and lints.
https://www.pivotaltracker.com/story/show/184229094
2023-02-02 23:05:25 +00:00
Kaz Wesley
1097c41297
Fix lexing bug encountered when doc-comments contain space-only lines (#4070)
See: https://www.pivotaltracker.com/story/show/184249780
2023-01-26 12:53:20 +00:00
Kaz Wesley
591cacb79a
Reject @ as binary operator (#4021)
`@` should not be legal to use as a binary operator. I accepted it in the parser because it occurred in the .enso sources, but it was actually used to create a syntax error to test error recovery.

See: https://www.pivotaltracker.com/story/show/184054024
2023-01-19 20:31:14 +00:00
Kaz Wesley
e15583fe65
Parser: Support annotations in type defs (#4036)
Support application of the new type of annotation to method bindings in type definitions.
2023-01-12 16:51:44 +00:00
Kaz Wesley
0a782a2ed1
AST support for new annotation syntax (#4018)
* rename Annotated -> AnnotatedBuiltin

* update Panic.catch test

* new annotations

* Revert "update Panic.catch test"

This reverts commit ccda46e3a7.
2023-01-04 09:50:38 -08:00
Kaz Wesley
d24019aa57
Implement SKIP/FREEZE in parser/TreeToIr (#3942)
See: https://www.pivotaltracker.com/story/show/183919788

# Important Notes
`SKIP` would be simpler if implemented in the parser, but there is some work needed before the Rust AST and Java IR are able to represent the results of macro-expansion: https://www.pivotaltracker.com/story/show/184004555
2022-12-20 17:32:59 +00:00
Kaz Wesley
336bbf505c
Parser: Newline normalization in text literals (#3903)
- Newlines in text literals are now normalized to `\n` when producing IR representation.
- Re-enabled tests that were dependent on the old behavior.
2022-11-27 09:40:44 +00:00
Kaz Wesley
580ed74726
Parser: identify lines earlier (#3900)
Unify line-finding by doing it at the beginning of parsing. See: https://www.pivotaltracker.com/story/show/183797744
2022-11-22 19:59:33 +00:00
Kaz Wesley
8f3bfe8ce2
Parser: Fix @Ignored tests (#3879)
- 3 ignored cases were old-parser bugs (one of which was quite interesting)
- 1 was a new-lexer bug, fixed here

See: https://www.pivotaltracker.com/story/show/183797809
2022-11-18 20:58:41 +00:00
Kaz Wesley
a1db36b57c
Support mixed constructors/bindings in types (#3870)
Libraries: Revert changes that were necessitated by a new rule we have decided not to introduce.

Parser:
- Support mixed constructors/bindings in types.
- Disallow zero-length hex sequences in character escapes: `\x`, `\u`, `\u{}`, `\U`, `\U{}` are no longer legal synonyms for `\0` (matches old parser behavior).
2022-11-14 20:24:07 +00:00
Kaz Wesley
e8f3ad3979
Ensure parses of invalid inputs represent all tokens (#3860)
Ensure all tokens from the input are represented in trees resulting from invalid inputs--tests now cover every reachable code line that creates an `Invalid` node. (Also implemented stricter validation, mainly of `import`/`export` statements.)

See: https://www.pivotaltracker.com/story/show/183405907
2022-11-09 02:57:40 +00:00
Kaz Wesley
330612119a
Parse the standard library (#3830)
Fix bugs in `TreeToIr` (rewrite) and parser. Implement more undocumented features in parser. Emulate some old parser bugs and quirks for compatibility.

Changes in libs:
- Fix some bugs.
- Clean up some odd syntaxes that the old parser translates idiosyncratically.
- Constructors are now required to precede methods.

# Important Notes
Out of 221 files:
- 215 match the old parser
- 6 contain complex types the old parser is known not to handle correctly

So, compared to the old parser, the new parser parses 103% of files correctly.
2022-10-31 16:19:12 +00:00
Kaz Wesley
feb8eb4f83
fix span bug in doc comments (#3808) 2022-10-18 20:37:36 +00:00
Kaz Wesley
28daf14f75
Parse case-by-type, add an old-lambda syntax rule (#3802)
- Special precedence rules for case-of so that `:` operator works without parens or nospace-grouping.
- Support an old-lambda syntax: `x->x-> x`. According to the usual rules, the first nospace group would be parsed as an operator section. The expression now parses as a lambda that contains a lambda.
- Match old parser treatment of # in doc comments.
- Tweak precedence so (a : B = c) works.
- Documented constructors.
2022-10-17 22:46:52 +00:00
Kaz Wesley
2740406f93
Lex doc comments and attach text to AST (#3795)
- New `Documented` node attaches documentation, lexed as a raw text literal, to a statement.
- Handle a case of lambdas with body blocks.
2022-10-15 06:13:32 +00:00
Kaz Wesley
0e412044f6
Macro contexts (#3792)
- Implement macro-contexts-lite (`from` is now only a keyword at the beginning of a line)
- Support special nospace-group handling for old lambdas (so expressions like this work: `x-> y-> x + y`)
- Fix a text-escape incompatibility

# Important Notes
- There is now an `OperatorFunction`, which is like a `Function` but has an operator for a name, and likewise an `OperatorTypeSignature`.
2022-10-13 22:47:02 +00:00
Kaz Wesley
5668cbcc24
Qualified defs (#3785)
Allow qualified names in LHS of type signatures and method definitions.
2022-10-12 17:40:16 +00:00