bc66078251
Implement generation of Java AST types from the Rust AST type definitions, with support for deserializing in Java syntax trees created in Rust. ### New Libraries #### `enso-reflect` Implements a `#[derive(Reflect)]` macro to enable runtime analysis of datatypes. Macro interface includes helper attributes; **the Rust types and the `reflect` attributes applied to them fully determine the Java types** ultimately produced (by `enso-metamodel`). This is the most important API, as it is used in the subject crates (`enso-parser`, and dependencies with types used in the AST). [Module docs](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/reflect/macros/src/lib.rs). #### `enso-metamodel` Provides data models for data models in Rust/Java/Meta (a highly-abstracted language-independent model--I have referred to it before as the "generic representation", but that was an overloaded term). The high-level interface consists of operations on data models, and between them. For example, the only operations needed by [the binary that drives datatype transpilation](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/parser/generate-java/src/main.rs) are: `rust::to_meta`, `java::from_meta`, `java::transform::optional_to_null`, `java::to_syntax`. The low-level interface consists of direct usage of the datatypes; this is used by [the module that implements some serialization overrides](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/parser/generate-java/src/serialization.rs) (so that the Java interface to `Code` references can produce `String`s on demand based on serialized offset/length pairs). The serialization override mechanism is based on customizing, not replacing, the generated deserialization methods, so as to be as robust as possible to changes in the Rust source or in the transpilation process. ### Important Notes - Rust/Java serialization is exhaustively tested for structural compatibility. A function [`metamodel::meta::serialization::testcases`](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/metamodel/src/meta/serialization.rs) uses `reflect`-derived data to generate serialized representations of ASTs to use as test cases. Its should-accept cases cover every type a tree can contain; it also produces a representative set of should-reject cases. A Rust `#[test]` confirms that these cases are accepted/rejected as expected, and generated Java tests (see Binaries below) check the generated Java deserialization code against the same test cases. - Deserializing `Code` is untested. The mechanism is in place (in Rust, we serialize only the offset/length of the `Cow`; in Java, during deserialization we obtain a context object holding a buffer for all string data; the accessor generated in Java uses the buffer and the offset/length to return `String`s), but it will be easier to test once we have implemented actually parsing something and instantiating the `Cow`s with source code. - `#[tagged_enum]` [now supports](https://github.com/enso-org/enso/blob/wip/kw/parser/ast-transpiler/lib/rust/shapely/macros/src/tagged_enum.rs#L36-L51) control over what is done with container-level attributes; they can be applied to the container and variants (default), only to the container, or only to variants. - Generation of `sealed` classes is supported, but currently disabled by `TARGET_VERSION` in `metamodel::java::syntax` so that tests don't require Java 15 to run. (The same logic is run either way; there is a shallow difference in output.) ### Binaries The `enso-parser-generate-java` crate defines several binaries: - `enso-parser-generate-java`: Performs the transpilation; after integration, this will be invoked by the build script. - `java-tests`: Generates the Java code that tests format deserialization; after integration this command will be invoked by the build script, and its Java output compiled and run during testing. - `graph-rust`/`graph-meta`/`graph-java`: Produce GraphViz representations of data models in different typesystems; these are for developing and understanding model transformations. Until integration, a **script regenerates the Java and runs the format tests: `./tools/parser_generate_java.sh`**. The generated code can be browsed in `target/generated_java`. |
||
|---|---|---|
| .cargo | ||
| .github | ||
| actions/setup-build | ||
| app | ||
| build | ||
| distribution | ||
| docs | ||
| engine | ||
| integration-test | ||
| lib | ||
| project | ||
| std-bits | ||
| test | ||
| tools | ||
| .gitignore | ||
| .ignore | ||
| .jvmopts | ||
| .prettierignore | ||
| .prettierrc.yaml | ||
| .scalafmt.conf | ||
| build-config.yaml | ||
| build.sbt | ||
| Cargo.lock | ||
| Cargo.toml | ||
| CHANGELOG.md | ||
| LICENSE | ||
| package.json | ||
| README.md | ||
| RELEASES.md | ||
| run | ||
| run.cmd | ||
| rust-toolchain.toml | ||
| rustfmt.toml | ||
Enso.org. Get insights you can rely on. In real time.
Enso is an award-winning interactive programming language with dual visual and textual representations. It is a tool that spans the entire stack, going from high-level visualisation and communication to the nitty-gritty of backend services, all in a single language. Watch the following introduction video to learn what Enso is, and how it helps companies build data workflows in minutes instead of weeks.
Enso's Features
Turning your data into knowledge is slow and error-prone. You can’t trust tools that don’t embrace best practices and provide quality assurance. Enso redefines the way you can work with your data: it is interactive, provides intelligent assistance, and was designed on a strong mathematical foundation, so you can always trust the results you get.
-
Intelligent suggestions of possible next steps. Build workflows in minutes instead of weeks.
Enso analyses the data, suggests possible next steps, and displays related help and examples. It lets you build dashboards, RPA workflows, and apps, with no coding required. Enso ships with a robust set of libraries, allowing you to work with local files, databases, HTTP services and other applications in a seamless fashion.
Learn more →
-
Reproducible, trustworthy results.
Versioning and visual data quality management allow you to trust the results that you get.
Learn more →
-
A powerful, purely functional language. Both visual and textual.
Enso incorporates many recent innovations in data processing and programming language design to allow you to work interactively and trust the results that you get. It is a purely functional programming language with higher-order functions, user-defined algebraic datatypes, pattern-matching, and two equivalent representations that you can switch between on-demand.
Learn more →
-
Mix languages with close-to-zero interop overhead.
Import any library from Enso, Java, JavaScript, R, or Python, and use functions, callbacks, and data types without any wrappers. Enso uses GraalVM to compile them to the same instruction set with a unified memory model.
Learn more →
-
Fast. Up to 80x faster than Python.
It can even run other languages faster than their official runtimes. Enso-R (using FastR on the GraalVM) is 36x faster than GNU-R.
See benchmarks →
-
A cutting-edge visualization engine.
Enso is equipped with a highly-tailored WebGL visualization engine capable of displaying many millions of data points at 60 frames per second in a web browser. Currently, Enso includes a set of core data visualizations out of the box, and you can easily extend it with libraries such as D3.js, Three.js, Babylon.js, deck.gl, VTK.js, Potree, and many more.
Learn more →
-
Runs everywhere.
Enso is available on MacOS, Windows, and Linux, and the Enso IDE runs on web-native technologies. In time, you'll be able to run it in the web-browser, giving even your tablet and phone access to your data.
Learn more →
Getting Started
-
Download Enso
-
Keep Up With the Latest Updates
Enso Source Code
If you want to start using Enso, please see the download links in the getting started section above. Alternatively you can get the IDE here and the language itself here. This section is intended for people interested in contributing to the development of Enso.
Enso is a community-driven open source project which is, and will always be, open and free to use. Join us, help us to build it, and spread the word!
Project Components
Enso consists of several sub projects:
-
Enso Engine: The Enso Engine is the set of tools that implement the Enso language and its associated services. These include the Enso interpreter, a just-in-time compiler and runtime (both powered by GraalVM), and a language server that lets you inspect Enso code as it runs. These components can be used on their own as command line tools.
-
Enso IDE: The Enso IDE is the desktop application that allows working with the visual form of Enso. It consists of an Electron application, a high performance WebGL UI framework, and the searcher which provides contextual search, hints, and documentation for all of Enso's functionality.
License
The Enso Engine is licensed under the Apache 2.0, as specified in the LICENSE file. The Enso IDE is licensed under the AGPL 3.0, as specified in the LICENSE file.
This license set was choosen to both provide you with a complete freedom to use Enso, create libraries, and release them under any license of your choice, while also allowing us to release commercial products on top of the platform, including Enso Cloud and Enso Enterprise server managers.
Contributing to Enso
Enso is a community-driven open source project which is and will always be open and free to use. We are committed to a fully transparent development process and highly appreciate every contribution. If you love the vision behind Enso and you want to redefine the data processing world, join us and help us track down bugs, implement new features, improve the documentation or spread the word!
If you'd like to help us make this vision a reality, please feel free to join our chat, and take a look at our development and contribution guidelines. The latter describes all the ways in which you can help out with the project, as well as provides detailed instructions for building and hacking on Enso.
If you believe that you have found a security vulnerability in Enso, or that you have a bug report that poses a security risk to Enso's users, please take a look at our security guidelines for a course of action.
Enso's Design
If you would like to gain a better understanding of the principles on which Enso
is based, or just delve into the why's and what's of Enso's design, please take
a look in the docs/ folder. It is split up into subfolders for each
component of Enso. You can view this same documentation in a rendered form at
the developer docs website.
This folder also contains a document on Enso's design philosophy, that details the thought process that we use when contemplating changes or additions to the language.
This documentation will evolve as Enso does, both to help newcomers to the project understand the reasoning behind the code, but also to act as a record of the decisions that have been made through Enso's evolution.