enso/docs/parser/tech-analysis.md

---
layout: developer-doc
title: Technology Analysis
category: syntax
tags: [parser, tech-analysis]
order: 1
---

# Parser Technology Analysis

As the Enso parser has some fairly unique requirements placed upon it, the
choice of implementation technology is of paramount importance. Choosing the
correct technology ensures that we can meet all of the requirements placed upon
the parser.

<!-- MarkdownTOC levels="2,3" autolink="true" -->

- [Technology Requirements for the Parser](#technology-requirements-for-the-parser)
- [Issues With the Previous Implementation](#issues-with-the-previous-implementation)
- [Choosing Rust](#choosing-rust)
  - [Downsides of Rust](#downsides-of-rust)

<!-- /MarkdownTOC -->

## Technology Requirements for the Parser

As the parser has to work both for the Engine and for the IDE, it has a strange
set of requirements:

- The implementation language must be able to run on native platforms, as well
  as in the browser via WASM (not JavaScript due to the marshalling overhead).
- The implementation language should permit _excellent_ native performance on
  both native and web platforms, by giving implementers fine-grained control
  over memory usage.
- The implementation language must be able to target all primary platforms:
  macOS, Linux and Windows.

## Issues With the Previous Implementation

The previous implementation of the parser was implemented in Scala, and had some
serious issues that have necessitated this rewrite:

- **Performance:** The structures used to implement the parser proved inherently
  difficult for a JIT to optimise, making performance far worse than expected on
  the JVM.
- **ScalaJS Sub-Optimal Code Generation:** The JavaScript generated by ScalaJS
  was very suboptimal for these structures, making the parser _even_ slower when
  run in the browser.
- **JS as a Browser Target:** To transfer textual data between WASM and JS
  incurs a significant marshalling overhead. As the IDE primarily works with
  textual operations under the hood, this proved to be a significant slowdown.

## Choosing Rust

Rust, then, is an obvious choice for the following reasons:

- It can be compiled _natively_ into the IDE binary, providing them with
  excellent performance.
- As a native language it can use JNI to directly create JVM objects on the JVM
  heap, for use by the compiler.
- As a native language it can be called directly via JNI.
- There is potential in the future for employing Graal's LLVM bitcode
  interpreter to execute the parser safely in a non-native context.

### Downsides of Rust

This is not to say that choosing rust doesn't come with some compromises:

- It significantly complicates the CI pipeline for the engine, as we will have
  to build native artefacts for use by the runtime itself.
- As a non-JVM language, the complexity of working with it from Scala and Java
  is increased. We will need to maintain a full definition of the AST in Scala
  to permit the compiler to work properly with it.